Prometheus-njs

Module Info

The nginx-plus-module-prometheus module is an njs module that converts miscellaneous F5 NGINX Plus status metrics exposed by the API module to a Prometheus compliant format. The module uses subrequests to the /api endpoint to access the metrics. In case you have configured dynamic upstream routing with generic names for upstream groups, the module can understand replacements for these names and display the correct statistics.

Exported Metrics

The following NGINX Plus status metrics are exported to Prometheus:

nginx: /nginx
Processes: /processes
Resolvers: /resolvers/
Connections: /connections
SSL: /ssl
Slabs: /slabs/
Workers: /workers/
HTTP: /http/
HTTP Requests: /http/requests
HTTP Caches: /http/caches/
HTTP Server Zones: /http/server_zones/
HTTP Location Zones: /http/location_zones/
HTTP Upstreams: /http/upstreams/
HTTP Limit Conn: /http/limit_conns/
HTTP Limit Req: /http/limit_reqs/
Stream Server Zones: /stream/server_zones/
Stream Zone Sync: /stream/zone_sync/
Stream Upstreams: /stream/upstreams/
Stream Limit Conn: /stream/limit_conns/

Note:

The state metric values in /http/upstreams/ and /stream/upstreams/ are converted using the following rule:

NGINX Prometheus

“up” 1

“draining” 2

“down” 3

“unavail” 4

“checking” 5

“unhealthy” 6

NGINX	Prometheus
“up”	`1`
“draining”	`2`
“down”	`3`
“unavail”	`4`
“checking”	`5`
“unhealthy”	`6`

Installation

Install the nginx-plus-module-prometheus module.

For Amazon Linux 2, CentOS, Oracle Linux, and RHEL:
```
yum install nginx-plus-module-prometheus
```
For Amazon Linux 2023, AlmaLinux, Rocky Linux:
```
dnf install nginx-plus-module-prometheus
```

For Debian and Ubuntu:

apt-get install nginx-plus-module-prometheus

For SLES:

zypper install nginx-plus-module-prometheus

For Alpine:

apk add nginx-plus-module-prometheus

For FreeBSD:

pkg install nginx-plus-module-prometheus

Note:
The nginx-plus-module-njs module will also be installed together with the module.

Configuration

After module installation, perform the following steps in NGINX Plus configuration file (nginx.conf):

Enable the nginx-plus-module-njs module in the top‑level context:

load_module modules/ngx_http_js_module.so;

http {
    # ...
}

Include the prometheus.js file:

http {
    # ...
    js_import /usr/share/nginx-plus-module-prometheus/prometheus.js;
}

Create a location for Prometheus metrics, for example, /metrics:

location = /metrics {
    js_content prometheus.metrics;
}

Enable the API to be able to expose the /metrics endpoint from Prometheus:
```
location /api {
    api;
    #...
}
```
(optional, in case of “too big subrequest response” error in error log) Since the module uses subrequests for API calls, you may need to increase the size of the buffer that stores the response body of a subrequest:
```
http {
    # ...
    subrequest_output_buffer_size 32k;
}
```
Configure Prometheus to obtain metrics from NGINX Plus by specifying the network address of the NGINX Plus instance in a scrape_config section of the Prometheus configuration file.

Upgrade Instructions

When upgrading Prometheus-njs version to version 1.3.1 and later, it is important to update NGINX Plus configuration file with the changes introduced in NGINX Plus R22 and njs 0.4.0:

the js_include directive is deprecated
the js_import directive is used instead of js_include
the js_content and js_set directives can reference a module function

See Configuration for the example of Prometheus-njs configuration in NGINX Plus configuration file.

Module Variables

The module supports several embedded variables:

$prom_keyval
$prom_keyval_stream
$prom_metrics_disabled

The variables can be set in NGINX Plus configuration file with the set directive.

Using the $prom_keyval Variable

The $prom_keyval variable has been created to correctly show statistics for dynamic configuration of upstream routing: when names of upstreams are generic and these names are dynamically replaced by real names from the key-value storage.

To add the $prom_keyval variable, add the set directive to the location that exposes metrics to Prometheus (for example, = /metrics), in the following format:

set $prom_keyval "upstream_keyval";

where $prom_keyval will hold all values from upstream_keyval key-value storage specified in the keyval_zone directive:

http {

    #...

    keyval_zone zone=upstream_keyval:64k;
    keyval      $domain $upstream zone=http_upstream_keyval;

    upstream 0 {
        zone   http_backend 64k;
        server backend1.example.com;
        server backend2.example.com;
    }

    server {
        #...

        location / {
            proxy_pass http://$upstream;
            #...
        }

        location /api {
            api;
            #...
        }

        location = /metrics {
            set        $prom_keyval "http_upstream_keyval";
            js_content prometheus.metrics;
        }
    }
}

Using the $prom_keyval_stream Variable

In order to perform dynamic replacements in the stream {} context, you will need to specify a new key-value storage in the stream block and set the name of the key-value storage in the $prom_keyval_stream variable:


http {

    # the contents of the 'http' block is the same as for the $prom_keyval example
    # except the 'location /metrics' block:
    # ...

        location = /metrics {
            set $prom_keyval        "http_upstream_keyval";
            set $prom_keyval_stream "stream_upstream_keyval";
            js_content              prometheus.metrics;
        }
    }
}

stream {

    keyval_zone zone=stream_keyval:32k;

    upstream stream_backend {
        zone   stream_backend 64k;
        server backend1.example.com:12345;
        server backend2.example.com:12345;
    }

    server {
        listen 12345;

        proxy_pass  stream_backend;
        status_zone backend_stream_zone;
    }
}

Using the $prom_metrics_disabled Variable

The $prom_metrics_disabled variable disables exporting particular NGINX Plus status metrics to Prometheus. If you are concerned about the amount of storage NGINX Plus metrics are taking in Prometheus, you can disable particular endpoints via this variable. The variable value must be a comma separated list of NGINX Plus endpoints:

#...
location /metrics {
    set $prom_keyval           "http_upstream_keyval";
    set $prom_keyval_stream    "stream_upstream_keyval";
    set $prom_metrics_disabled "stream/upstreams, resolvers";
    js_content                 prometheus.metrics;
}
#...

You can disable exporting the following NGINX Plus status metrics to Prometheus:

nginx
processes
resolvers
connections
ssl
slabs
workers
http/requests
http/caches
http/upstreams
http/server_zones
http/location_zones
http/limit_conns
http/limit_reqs
stream/upstreams
stream/server_zones
stream/zone_sync
stream/limit_conns

Example

load_module modules/ngx_http_js_module.so;

#...

http {

    js_import /usr/share/nginx-plus-module-prometheus/prometheus.js;

    subrequest_output_buffer_size 32k;

    upstream backend {
        zone   backend 64k;
        server backend1.example.com;
        server backend2.example.com;
    }

    server {
        listen 80;

        location / {
            proxy_pass http://backend;
        }

        location /api {
            api;
        }

        location = /metrics {
            set $prom_keyval           "http_upstream_keyval";
            set $prom_keyval_stream    "stream_upstream_keyval";
            set $prom_metrics_disabled "stream/upstreams, resolvers";
            js_content                 prometheus.metrics;
        }

        status_zone backend_zone;
    }
}

stream {

    keyval_zone zone=stream_keyval:32k;

    upstream stream_backend {
        zone   stream_backend 64k;
        server backend1.example.com:12345;
        server backend2.example.com:12345;
    }

    server {
        listen 12345;

        proxy_pass  stream_backend;
        status_zone backend_stream_zone;
}

More Info

Last modified August 22, 2024