NGINX Documentation

Prometheus-njs

Expose Prometheus metrics endpoint directly from NGINX Plus.

Module Info

The nginx-plus-module-prometheus module is an njs module written to convert miscellaneous NGINX Plus status metrics exposed by the API module to a Prometheus compliant format. The module uses subrequests to the /api endpoint to access the metrics. In case you have configured dynamic upstream routing with generic names for upstream groups, the module can understand replacements for these names and display the correct statistics.

Exported Metrics

The following NGINX status metrics are exported to Prometheus:

Note: The state metric values in /http/upstreams/ and /stream/upstreams/ are converted using the following rule:
NGINX Prometheus
“up” 1
“draining” 2
“down” 3
“unavail” 4
“checking” 5
“unhealthy” 6

Installation Instructions

Install the nginx-plus-module-prometheus module.

  • For Amazon Linux, CentOS, Oracle Linux, and RHEL:

    $ yum install nginx-plus-module-prometheus
    
  • For Debian and Ubuntu:

    $ apt-get install nginx-plus-module-prometheus
    
  • For SLES:

    $ zypper install nginx-plus-module-prometheus
    

    Note: the nginx-plus-module-njs module will also be installed together with the module.

Configuration

After module installation, perform the following steps in NGINX configuration file (nginx.conf):

  1. Enable the nginx-plus-module-njs module in the top‑level (“main”) context:

    load_module modules/ngx_http_js_module.so;
    
    http {
        # ...
    }
    
  2. Include the main.js file:

    http {
        # ...
        js_include /usr/share/nginx-plus-module-prometheus/main.js;
    }
    
  3. Create a location for Prometheus metrics, for example, /metrics:

    location = /metrics {
        js_content prometheus_metrics;
    }
    
  4. Enable the API to be able to expose the /metrics endpoint from Prometheus:

    location /api {
        api;
        #...
    }
    
  5. (optional, in case of too big subrequest response error in error log) Since the module uses subrequests for API calls, you may need to increase the size of the buffer that stores the response body of a subrequest:

    http {
        # ...
        subrequest_output_buffer_size 32k;
    }
    
  6. Configure Prometheus to obtain metrics from NGINX Plus by specifying the network address of the NGINX Plus instance in a scrape_config section of the Prometheus configuration file.

Module variables

The module supports several embedded variables:

The variables can be set in NGINX configuration file with the set directive.

Using the $prom_keyval Variable

The $prom_keyval variable has been created to correctly show statistics for dynamic configuration of upstream routing: when names of upstreams are generic and these names are dynamically replaced by real names from the key-value storage.

To add the $prom_keyval variable, add the set directive to the location that exposes metrics to Prometheus (for example, = /metrics), in the following format:

    set $prom_keyval "upstream_keyval";

where $prom_keyval will hold all values from upstream_keyval key-value storage specified in the keyval_zone directive:

http {

    #...

    keyval_zone zone=upstream_keyval:32k;
    keyval      $domain $upstream zone=http_upstream_keyval;

    upstream 0 {
        zone   http_backend 64k;
        server backend1.example.com;
        server backend2.example.com;
    }

    server {
        #...

        location / {
            proxy_pass http://$upstream;
            #...
        }

        location /api {
            api;
            #...
        }

        location = /metrics {
            set        $prom_keyval "http_upstream_keyval";
            js_content prometheus_metrics;
        }
    }
}

Using the $prom_keyval_stream Variable

In order to perform dynamic replacements in the stream {} context, you will need to specify a new key-value storage in the stream block and set the name of the key-value storage in the $prom_keyval_stream variable:

http {

    # the contents of the 'http' block is the same as for the $prom_keyval example
    # except the 'location /metrics' block:
    # ...

        location = /metrics {
            set $prom_keyval        "http_upstream_keyval";
            set $prom_keyval_stream "stream_upstream_keyval";
            js_content              prometheus_metrics;
        }
    }
}

stream {

    keyval_zone zone=stream_keyval:32k;

    upstream stream_backend {
        zone   stream_backend 64k;
        server backend1.example.com:12345;
        server backend2.example.com:12345;
    }

    server {
        listen 12345;

        proxy_pass  stream_backend;
        status_zone backend_stream_zone;
    }
}

Using the $prom_metrics_disabled Variable

The variable disables exporting particular NGINX status metrics to Prometheus. If you are concerned about the amount of storage NGINX metrics are taking in Prometheus, you can disable particular endpoints via this variable. The variable value must be a comma separated list of NGINX endpoints:

#...
location /metrics {
    set $prom_keyval           "http_upstream_keyval";
    set $prom_keyval_stream    "stream_upstream_keyval";
    set $prom_metrics_disabled "stream/upstreams, resolvers";
    js_content                 prometheus_metrics;
}
#...

You can disable exporting the following NGINX status metrics to Prometheus:

  • nginx
  • processes
  • resolvers
  • connections
  • ssl
  • slabs
  • http/requests
  • http/caches
  • http/upstreams
  • http/server_zones
  • http/location_zones
  • stream/upstreams
  • stream/server_zones
  • stream/zone_sync

Example

load_module modules/ngx_http_js_module.so;

#...

http {

    js_include /usr/share/nginx-plus-module-prometheus/main.js;

    subrequest_output_buffer_size 32k;

    upstream backend {
        zone   backend 64k;
        server backend1.example.com;
        server backend2.example.com;
    }

    server {
        listen 80;

        location / {
            proxy_pass http://backend;
        }

        location /api {
            api;
        }

        location = /metrics {
            set $prom_keyval           "http_upstream_keyval";
            set $prom_keyval_stream    "stream_upstream_keyval";
            set $prom_metrics_disabled "stream/upstreams, resolvers";
            js_content                 prometheus_metrics;
        }

        status_zone backend_zone;
    }
}

stream {

    keyval_zone zone=stream_keyval:32k;

    upstream stream_backend {
        zone   stream_backend 64k;
        server backend1.example.com:12345;
        server backend2.example.com:12345;
    }

    server {
        listen 12345;

        proxy_pass  stream_backend;
        status_zone backend_stream_zone;
}

More Info