NGINX: How to Execute Shell Commands on Every Request

Overview

In the world of web servers, NGINX is known for its high performance, stability, rich feature set, simple configuration, and low resource consumption. However, sometimes there’s a need to perform certain actions beyond the scope of static content delivery or reverse proxy. In some scenarios, you might want to run a shell command for every request that your NGINX server handles. This could be for logging, processing data, or integrating with other systems in your infrastructure.

This tutorial will guide you through the process of configuring NGINX to execute a shell command on every request. We’ll cover basic configurations and provide examples to help you understand how to implement this functionality in your own setup.

Prerequisites

• An understanding of basic NGINX configuration.
• A server with NGINX installed.
• Shell access to the server running NGINX.
• Basic knowledge of shell scripting and UNIX commands.

Introduction to NGINX Subrequests

Subrequests in NGINX are internal server requests that are initiated and handled entirely within the server. They are not directly triggered by the external client’s request but are a result of the server’s processing logic. These subrequests can be used to fetch data from different locations, aggregate responses, or perform additional processing.

Key Features and Benefits

1. Flexibility: Subrequests allow NGINX to combine content from different locations, providing flexibility in content generation and manipulation.
2. Efficiency: They can be used to parallelize data fetching, which can improve response times and server efficiency.
3. Modularity: By separating concerns into different locations or servers, subrequests help maintain a modular and maintainable server configuration.

Use Cases

• Content Aggregation: Combining data from multiple endpoints into a single response.
• Authentication and Authorization: Checking credentials or permissions by making a subrequest to an authentication server before proceeding with the main request.
• Data Enrichment: Adding additional information to a response by making a subrequest to an API or database.

Installing Lua for NGINX

One of the ways to achieve execution of shell commands on each request is by using NGINX subrequests mechanism with the ngx_http_subrequest module, combined with an external script interpreter like lua-nginx-module for embedding Lua scripts within NGINX configurations.

sudo apt-get install liblua5.1-dev libnginx-mod-http-lua

Step 3: Download and Compile NGINX with Lua

If Lua is not included in your NGINX build, you’ll need to recompile NGINX from the source with the Lua module:

1. Download NGINX Source Code: Get the latest NGINX source code from the official NGINX website.

2. Download ngx_http_lua_module: Clone or download the module from its GitHub repository.

3. Compile NGINX with Lua Module: Navigate to the NGINX source directory and run the ./configure script with additional parameters to include the Lua module. For example:

./configure --add-module=/path/to/lua-nginx-module
make
sudo make install

Replace /path/to/lua-nginx-module with the actual path to the downloaded Lua module.

Step 4: Test the Configuration

After installation, add a basic Lua script to your NGINX configuration to test if it’s working:

location /test {
    default_type 'text/plain';
    content_by_lua 'ngx.say("Hello from Lua!")';
}

Reload NGINX and navigate to http://your-server-address/test to see if the Lua script runs correctly.

Step 5: Write Custom Lua Scripts

With the Lua module installed, you can now start integrating Lua scripts into your NGINX configuration to add custom functionality or modify HTTP requests and responses.

Configuring Lua Block Within NGINX

http {
    ... 
    server {
        location / {
            content_by_lua_block {
                os.execute("/path/to/your/script.sh")
            }
        }
    }
}

The above configuration snippet shows how you might configure a Lua block within your NGINX configuration to execute a script. Here’s what you’re doing:

• content_by_lua_block: This is where you embed your Lua code directly within the NGINX configuration.
• os.execute("/path/to/your/script.sh"): The Lua function for executing a shell command.

Ensuring Non-Blocking Behavior

While the os.execute function call will run your shell script, it should be noted that the script will run synchronously, blocking NGINX until it completes. To run the command in the background, you can modify your Lua code:

content_by_lua_block {
    os.execute("/path/to/your/script.sh &")
}

By adding an ampersand (&), the command is executed as a background process.

Accessing Request Data

A common requirement is to have access to request data within the shell script that’s executed. This might include headers, query parameters, or the request body. You can pass this information to the shell script using environment variables:

content_by_lua_block {
    local headers = ngx.req.get_headers()
    os.execute("HEADERS='" .. ngx.encode_args(headers) .. "' /path/to/your/script.sh &")
}

This code snippet grabs headers from the request and passes them to the shell script as part of the command.

Logging the Output of Shell Command

It might be useful to log the output of the command for debugging purposes or to have a record of the script’s actions. To do so, you’ll need to direct the command’s output to a file:

content_by_lua_block {
    os.execute("/path/to/your/script.sh > /path/to/your/logfile.log 2>&1 &")
}

This command redirects both the standard output (stdout) and standard error (stderr) to a log file.

Executing the Command from a Named Location

If you have a situation where the shell command should be executed without interfering with the original request flow, consider using a named location:

location / {
    post_action @run_shell;
}

location @run_shell {
    internal;
    content_by_lua_block {
        os.execute("/path/to/your/script.sh &")
    }
}

Here, post_action is used to call a named location after the main request completes, without altering its response.

Security Considerations

Running shell scripts on every HTTP request may have serious security implications, and should only be used when necessary. Ensure that:

• Your script is secure and has no harmful side-effects.
• The user NGINX runs as has limited permissions and cannot compromise your system.
• Error handling is robust to prevent any script failures from affecting the NGINX service.

Performance Implications

Spawning a shell command for every request can significantly impact performance. It’s generally recommended to use this technique sparingly and monitor the server’s performance closely.

Conclusion

Engineering solutions using NGINX to execute shell commands on every request requires careful planning, an understanding of Linux operations and NGINX configurations, and strong considerations for both performance and security. Hopefully, this guide has provided you with a starting point for integrating shell command execution into your NGINX-hosted environment. Always remember to test your configuration on a non-production setup before deploying it into a live environment.

To learn more about this process or to explore advanced NGINX configuration strategies, consider reviewing further documentation or seeking support from the NGINX community.