NGINX Documentation

Prometheus-njs

Expose Prometheus metrics endpoint directly from NGINX Plus.

Module Info

The nginx-plus-module-prometheus module is an njs module written to convert miscellaneous NGINX status metrics exposed by the API module to a Prometheus compliant format. The module uses subrequests to the /api endpoint to access the metrics.

Exported Metrics

The following NGINX status metrics are exported to Prometheus:

Note: the state metric values in /http/upstreams/ and /stream/upstreams/ are converted using the following rule:

NGINX Prometheus
“up” 1
“draining” 2
“down” 3
“unavail” 4
“checking” 5
“unhealthy” 6

Installation Instructions

  1. Install the nginx-plus-module-prometheus module.

    For Amazon Linux, CentOS, Oracle Linux, and RHEL:

    $ yum install nginx-plus-module-prometheus
    

    For Debian and Ubuntu:

    $ apt-get install nginx-plus-module-prometheus
    

    For SLES:

    $ zypper install nginx-plus-module-prometheus
    

    Note: the nginx-plus-module-njs module will also be installed together with the module.

  2. After module installation, perform the following steps in NGINX configuration file (nginx.conf):

  • Enable the nginx-plus-module-njs module:

    load_module modules/ngx_http_js_module.so;
    
  • Include the main.js file:

    js_include /usr/share/nginx-plus-module-prometheus/main.js;
    
  • Enable the API to be able to expose the /metrics endpoint from Prometheus:

    location /api {
        api write=on;
        #...
    }
    
  • Create an endpoint /metrics for Prometheus metrics:

    location /metrics {
        js_content prometheus_metrics;
    }
    
  • [Optional, for dynamic configuration of upstream routing] Set the $prom_keyval variable with the keyval holding upstream replacements:

    keyval_zone zone=upstream_keyval:32k;
    
    location /metrics {
        set  $prom_keyval "upstream_keyval";
    }
    

    Note: you will need to fill the upstream_keyval keyval with a map of the form {replacement : upstream-to-replace}. See Dynamic Configuration of Upstream Routing for details.

Usecase: Dynamic Configuration of Upstream Routing

When configuring NGINX upstreams, it is possible to use a combination of NGINX key/value store and generic upstream names to dynamically configure upstream routing without having to reload NGINX. However, this means that the upstream names inside the NGINX configuration file will be generic and not useful for metric reporting:

keyval_zone zone=upstream_keyval:128k;
keyval      $domain $placeholder zone=upstream_keyval;

upstream 0 {
    zone dynamic 64k;
}

upstream 1 {
    zone dynamic 64k;
}
#...

upstream n {
    zone dynamic 64k;
}

server ... {

    location / {
        proxy_pass $scheme://$placeholder;
    }
}

Setting Module Variables

You can set variables in NGINX using the set directive. Setting variables in the same server {} or location {} block as the nginx-plus-module-prometheus module will adjust how it processes requests:

location /metrics {
    set  $prom_keyval "upstream_keyval";
}

Setting the $prom_keyval variable will allow you to specify a keyval containing a map of upstream replacements with the form {replacement : upstream-to-replace}. This will replace any upstreams in the Prometheus output matching the upstream-to-replace and convert them using the corresponding replacement.

To replace the upstream 0 with a more informative upstream name, add to the upstream_keyval map:

curl -X POST -d '{"backend":"0"}' -s 'localhost/api/4/http/keyvals/upstream_keyval'

Complete Example

load_module modules/ngx_http_js_module.so;

worker_processes  auto;

events {
    worker_connections  1024;
}

http {

    js_include /usr/share/nginx-plus-module-prometheus/main.js;

    keyval_zone zone=upstream_keyval:32k;

    error_log  /var/log/nginx/error_log notice;
    access_log /var/log/nginx/access_log;

    upstream 0 {
        zone backend_zone 64k;
        server 127.0.0.1:8080;
        server 127.0.1.2:8080;
    }

    upstream backend2 {
        zone   backend_zone2 64k;
        server 127.0.1.1:80;
    }

    server {
        listen 80;

        location / {
            return 200 OK;
        }

        location /api {
            api write=on;
        }

        location /metrics {
            set        $prom_keyval "upstream_keyval";
            js_content prometheus_metrics;
        }

        location /0 {
            proxy_pass http://0;
        }

        status_zone backend_zone;
    }

    server {
        listen 8080;

        location / {
            return 200 OK_backend;
        }

        status_zone backend_zone2;
    }
}

stream {

    upstream   stream_backend {
        zone   backend_stream_zone 64k;
        server 127.0.0.3:8080;
    }

    server {
        listen      8081 udp;
        proxy_pass  stream_backend;

        status_zone backend_stream_zone;
    }
}