# NGINX Documentation > Learn how to deliver, manage, and protect your applications using F5 NGINX products. -------------------------------------------------------------------------------- title: "F5 NGINX Product Documentation" url: https://docs.nginx.com/ description: Learn how to deliver, manage, and protect your applications using F5 NGINX products. -------------------------------------------------------------------------------- # F5 NGINX Product Documentation Learn how to deliver, manage, and protect your applications using F5 NGINX products. ### F5 NGINX One ### [F5 NGINX One Console](/nginx-one-console/) Monitor your infrastructure, address security vulnerabilities, and assess the health of your NGINX fleet, all from a single console. ### [F5 NGINX Plus](/nginx/) The all-in-one load balancer, reverse proxy, web server, content cache, and API gateway. ### [F5 NGINX Instance Manager](/nginx-instance-manager/) Track and control NGINX Open Source and NGINX Plus instances. ### [F5 NGINX Ingress Controller](/nginx-ingress-controller/) Kubernetes traffic management with API gateway, identity, and observability features. ### [F5 NGINX Gateway Fabric](/nginx-gateway-fabric/) Next-generation Kubernetes connectivity using the Gateway API. ### [NGINX Open Source](https://nginx.org/en/docs/) The open source all-in-one load balancer, content cache, and web server. ### [NGINX Agent](/nginx-agent/) A daemon providing observability data and remote configuration for NGINX Open Source and NGINX Plus instances. ### [Subscription licensing & solutions](/solutions/) Stay compliant with your NGINX subscription licenses and see how you can use NGINX One to build secure, scalable, and high-performing applications and APIs. ### F5 WAF for NGINX ### [F5 WAF for NGINX](/waf/) Lightweight, high-performance, advanced protection against Layer 7 attacks on your apps and APIs. ### [F5 DoS for NGINX](/nginx-app-protect-dos/) Defend, adapt, and mitigate against Layer 7 denial-of-service attacks on your apps and APIs. ### F5 NGINXaaS ### [F5 NGINXaaS for AWS](/nginxaas/aws/overview/) Managed NGINX service for your AWS application stack. ### [F5 NGINXaaS for Google Cloud](/nginxaas/google/overview/) Managed NGINX service for your Google Cloud application stack. ### [F5 NGINXaaS for Azure](/nginxaas-azure/) Infrastructure-as-a-Service (IaaS) version of NGINX Plus for your Microsoft Azure application stack. ### Glossary ### [F5 NGINX Glossary](/glossary/) Glossary of common terms and definitions for all F5 NGINX products. -------------------------------------------------------------------------------- title: "F5 NGINX Glossary" url: https://docs.nginx.com/glossary/index.md -------------------------------------------------------------------------------- This glossary defines terms used in F5 NGINX. ## General terms | Term | Definition | | ---- | ---------- | | **Config Sync Group** / **Instance Group** | A group of NGINX systems (or instances) with identical configurations. They may also share the same certificates. However, the instances in a Config Sync Group could belong to different systems and even different clusters. Also known as an Instance Group in NGINX Instance Manager. For more information, see this explanation of [Important considerations](/nginx-one-console/nginx-configs/config-sync-groups/manage-config-sync-groups.md#important-considerations) | | **Control Plane** | The control plane is the part of a network architecture that manages and controls the flow or data or traffic (the Data Plane). It is responsible for system-level tasks such as routing and traffic management. | | **Data Plane** | The data plane is the part of a network architecture that carries user traffic. It handles tasks like forwarding data packets between devices and managing network communication. In the context of NGINX, the data plane is responsible for tasks such as load balancing, caching, and serving web content. | | **Instance** | An instance is an individual system with NGINX installed. You can group the instances of your choice in a Config Sync Group. When you add an instance to NGINX One Console, you need to use a data plane key. | | **Namespace** | In F5 Distributed Cloud, a namespace groups a tenant's configuration objects, similar to administrative domains. Every object in a namespace must have a unique name, and each namespace must be unique to its tenant. This setup ensures isolation, preventing cross-referencing of objects between namespaces. You'll see the namespace in the NGINX One Console URL as `/namespaces//`. To switch an instance between namespaces, you have to deregister an instance from an old namespace, and register it on the new namespace. | | **NGINX Agent** | A lightweight software component installed on NGINX instances to enable communication with the NGINX One console. NGINX Agent also enables communication with NGINX Instance Manager. | | **Staged Configurations** | Also known as **Staged Configs**. Allows you to save "work in progress." You can create it from scratch, an Instance, another Staged Config, or a Config Sync Group. It does _not_ have to be a working configuration until you publish it to an instance or a Config Sync Group. You can even manage your **Staged Configurations** through our [API](/nginx-one-console/api/api-reference-guide/#tag/StagedConfigs). | | **Tenant** | A tenant in F5 Distributed Cloud is an entity that owns a specific set of configuration and infrastructure. It is fundamental for isolation, meaning a tenant cannot access objects or infrastructure of other tenants. Tenants can be either individual or enterprise, with the latter allowing multiple users with role-based access control (RBAC). | ## Authentication and Authorization terms | Term | Definition | | ---- | ---------- | | **Access Token** | Defined in OAuth2, this (optional) short lifetime token provides access to specific user resources as defined in the scope values in the request to the authorization server (can be a JSON token as well). | | **ID Token** | Specific to OIDC, the primary use of the token in JWT format is to provide information about the authentication operation's outcome. | | **Identity Provider (IdP)** | A service that authenticates users and verifies their identity for client applications. | | **JSON Web Token (JWT)** | An open standard (RFC 7519) that defines a compact and self-contained way for securely transmitting information between parties as a JSON object. This information can be verified and trusted because it is digitally signed. | | **Protected Resource** | A resource that is hosted by the resource server and requires an access token to be accessed. | | **Refresh Token** | Coming from OAuth2 specs, the token is usually long-lived and may be used to obtain new access tokens. | | **Relying Party (RP)** | A client service required to verify user identity. | ## Kubernetes and Ingress Controller terms {#k8s-ingress-controller} | Term | Definition | | ---- | ---------- | | **Ingress** | Refers to an *Ingress Resource*, a Kubernetes API object which allows access to [Services](https://kubernetes.io/docs/concepts/services-networking/service/) within a cluster. They are managed by an [Ingress Controller](/glossary/#ingress-controller). *Ingress* resources enable the following functionality:
* **Load balancing**, extended through the use of Services
* **Content-based routing**, using hosts and paths
* **TLS/SSL termination**, based on hostnames

For additional information, please read the official [Kubernetes Ingress Documentation](https://kubernetes.io/docs/concepts/services-networking/ingress/). | | **Ingress Controller** | Ingress Controllers are applications within a Kubernetes cluster that enable [Ingress](/glossary/#ingress) resources to function. They are not automatically deployed with a Kubernetes cluster, and can vary in implementation based on intended use, such as load balancing algorithms for Ingress resources. [The design of NGINX Ingress Controller](/nic/overview/design.md) explains the technical details of NGINX Ingress Controller. | ## F5 WAF for NGINX This section defines terminology used when describing functionality of F5 WAF for NGINX. It assumes you are familiar with various layer 7 (L7) hypertext transfer protocol (HTTP) concepts such as: - Cookies - HTTP methods and status codes - HTTP headings, requests, responses, and parameters - Uniform Resource Identifier (URI) - Uniform Resource Location (URL) | Term | Definition | | ---- | ---------- | | Alarm | If selected, the F5 WAF for NGINX system records requests that trigger the violation in the remote log (depending on the settings of the logging profile). | | Attack signature | Textual patterns which can be applied to HTTP requests and/or responses by F5 WAF for NGINX to determine if traffic is malicious. For example, the string ` NGINX Logo

Server name: server_hostname

Server address: server_address

User Agent: client_browser

URI: server_url

Doc Root: document_root

Date: server_date

NGINX Frontend Load Balancer IP: remote_addr

Client IP: proxied_for_ip

NGINX Version: nginx_version

Auto Refresh
``` ## Configuring NGINX Plus for load balancing The steps in this section configure an NGINX Plus instance to load balance requests across the group of NGINX Open Source web servers you configured in the [previous section](#nginx-oss). If you are using these instructions to satisfy the prerequisites for one of our deployment guides, the Appendix in the guide specifies the names of the NGINX Plus instances used in it. Repeat these instructions on each instance. Alternatively, you can configure one instance and share the configuration with its peers in a cluster. See the [NGINX Plus Admin Guide](nginx/admin-guide/high-availability/configuration-sharing.md). 1. Open a connection to the NGINX Plus instance and change the directory to **/**etc/nginx/conf.d**: ```shell cd /etc/nginx/conf.d ``` 2. Rename **default.conf** to **default.conf.bak** so that NGINX Plus does not use it. ```shell mv default.conf default.conf.bak ``` 3. Create a new file called **lb.conf** with the following contents. **Note:** In the `upstream` blocks, include a [server](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#server) directive for each NGINX Open Source instance that serves the relevant application. ```nginx # in the 'http' context upstream app1 { server ; # 'server' directives for additional App 1 servers, if using zone app1 64k; } upstream app2 { server ; # 'server' directives for additional App 2 servers, if using zone app2 64k; } server { listen 80; status_zone backend; root /usr/share/nginx/html; location / { # directives for serving the site's HTML landing page } location /application1 { proxy_set_header Host $host; proxy_set_header X-Forwarded-For $remote_addr; proxy_pass http://app1/; } location /application2 { proxy_set_header Host $host; proxy_set_header X-Forwarded-For $remote_addr; proxy_pass http://app2/; } location /api { api write=on; # directives to control access, such as 'allow' and 'deny' } location = /dashboard.html { root /usr/share/nginx/html; } location = /status.html { # redirect requests that are made to pre-R14 dashboard return 301 /dashboard.html; } } ``` Directive documentation: [api](https://nginx.org/en/docs/http/ngx_http_api_module.html#api), [listen](https://nginx.org/en/docs/http/ngx_http_core_module.html#listen), [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location), [proxy_pass](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass), [proxy_set_header](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_header), [return](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#return), [root](https://nginx.org/en/docs/http/ngx_http_core_module.html#root), [server](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#server) (upstream),[server](https://nginx.org/en/docs/http/ngx_http_core_module.html#server) (virtual), [server_name](https://nginx.org/en/docs/http/ngx_http_core_module.html#server_name), [status_zone](https://nginx.org/en/docs/http/ngx_http_status_module.html#status_zone), [upstream](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#upstream), [zone](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#zone) 4. Include the following directive in the top‑level ("main") context in **/etc/nginx/nginx.conf**, if it does not already appear there. ```nginx include conf.d/*.conf; ``` Directive documentation: [include](http://nginx.org/en/docs/ngx_core_module.html#include) ### Revision history - Version 2 (April 2019) – Generalized instructions for use with deployment guides - Version 1 (April 2018) – Initial version -------------------------------------------------------------------------------- title: "Single Sign-On with Amazon Cognito and njs" url: https://docs.nginx.com/nginx/deployment-guides/single-sign-on/oidc-njs/cognito/index.md description: Enable OpenID Connect-based single-sign for applications proxied by NGINX Plus, using Amazon Cognito as the identity provider (IdP). -------------------------------------------------------------------------------- **Note:** This guide applies to NGINX Plus [Release 15](nginx/releases.md#r15) and later, based on the [`nginx-openid-connect`](https://github.com/nginxinc/nginx-openid-connect) GitHub repo. Starting with NGINX Plus [Release 34](nginx/releases.md#r34), use the simpler solution with the [native OpenID connect module](https://nginx.org/en/docs/http/ngx_http_oidc_module.html). See [Single Sign-On With Amazon Cognito](nginx/deployment-guides/single-sign-on/cognito.md) for details. This guide explains how to enable single sign‑on (SSO) for applications being proxied by F5 NGINX Plus. The solution uses OpenID Connect as the authentication mechanism, with [Amazon Cognito](https://aws.amazon.com/cognito/) as the identity provider (IdP), and NGINX Plus as the relying party. **Note:** You can find more information about the NGINX Plus OpenID Connect integration in the project's [GitHub repo](https://github.com/nginxinc/nginx-openid-connect#nginx-openid-connect). ## Prerequisites The instructions assume you have the following: - An [AWS account](https://aws.amazon.com/premiumsupport/knowledge-center/create-and-activate-aws-account/). - An NGINX Plus subscription and NGINX Plus R15 or later. For installation instructions, see the [NGINX Plus Admin Guide](https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-plus/). - The [NGINX JavaScript module](https://nginx.org/en/docs/njs/index.html) (njs), required for handling the interaction between NGINX Plus and the IdP. After installing NGINX Plus, install the module with the command for your operating system. For Debian and Ubuntu: ```none sudo apt install nginx-plus-module-njs ``` For CentOS, RHEL, and Oracle Linux: ```shell sudo yum install nginx-plus-module-njs ``` - The following directive included in the top-level ("main") configuration context in **/etc/nginx/nginx.conf**, to load the NGINX JavaScript module: ```nginx load_module modules/ngx_http_js_module.so; ``` ## Configuring Amazon Cognito **Note:** The following procedure reflects the Cognito GUI at the time of publication, but the GUI is subject to change. Use this guide as a reference and adapt to the current Cognito GUI as necessary. Create a new application for NGINX Plus in the Cognito GUI: 1. Log in to your AWS account, open the AWS Management Console ([console.aws.amazon.com](https://console.aws.amazon.com)), and navigate to the Cognito dashboard (you can, for example, click **Cognito** in the **Security, Identity, & Compliance** section of the **Services** drop‑down menu). 2. On the Cognito dashboard, click **Manage User Pools** to open the **Your User Pools** window. Click the  Create a user pool  button or the highlighted phrase. 3. In the **Create a user pool** window that opens, type a value in the **Pool name** field (in this guide, it's **nginx-plus-pool**), then click the Review defaults button. 4. On the **Review** tab which opens, click Add app client... in the **App clients** field near the bottom. 5. On the **App clients** tab which opens, click Add an app client. 6. On the **Which app clients will have access to this user pool?** window which opens, enter a value (in this guide, **nginx-plus-app**) in the **App client name** field. Make sure the **Generate client secret** box is checked, then click the  Create app client  button. 7. On the confirmation page which opens, click **Return to pool details** to return to the **Review** tab. On that tab click the  Create pool  button at the bottom. (The screenshot in [Step 4](#cognito-review-tab) shows the button.) 8. On the details page which opens to confirm the new user pool was successfully created, make note of the value in the **Pool Id** field; you will add it to the NGINX Plus configuration in [Step 3 of _Configuring NGINX Plus_](#nginx-plus-variables). 'General settings' tab in Amazon Cognito GUI 9. Click **Users and groups** in the left navigation column. In the interface that opens, designate the users (or group of users, on the **Groups** tab) who will be able to use SSO for the app being proxied by NGINX Plus. For instructions, see the Cognito documentation about [creating users](https://docs.aws.amazon.com/cognito/latest/developerguide/how-to-create-user-accounts.html), [importing users](https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-pools-using-import-tool.html), or [adding a group](https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-pools-user-groups.html). 'Users and groups' tab in Amazon Cognito GUI 10. Click **App clients** in the left navigation bar. On the tab that opens, click the Show Details button in the box labeled with the app client name (in this guide, **nginx-plus-app**). 'App clients' tab in Amazon Cognito GUI 11. On the details page that opens, make note of the values in the **App client id** and **App client secret** fields. You will add them to the NGINX Plus configuration in [Step 3 of _Configuring NGINX Plus_](#nginx-plus-variables). 12. Click **App client settings** in the left navigation column. In the tab that opens, perform the following steps: 1. In the **Enabled Identity Providers** section, click the **Cognito User Pool** checkbox (the **Select all** box gets checked automatically). 2. In the **Callback URL(s)** field of the **Sign in and sign out URLs** section, type the URI of the NGINX Plus instance including the port number, and ending in **/\_codexch**. Here we’re using **https://my-nginx-plus.example.com:443/_codexch**. **Notes:** - For production, we strongly recommend that you use SSL/TLS (port 443). - The port number is mandatory even when you're using the default port for HTTP (80) or HTTPS (443). 3. In the **OAuth 2.0** section, click the **Authorization code grant** checkbox under **Allowed OAuth Flows** and the **email**, **openid**, and **profile** checkboxes under **Allowed OAuth Scopes**. 4. Click the  Save changes  button. 13. Click **Domain name** in the left navigation column. In the tab that opens, type a domain prefix in the **Domain prefix** field under **Amazon Cognito domain** (in this guide, **my-nginx-plus**). Click the  Save changes  button. ## Configuring NGINX Plus Configure NGINX Plus as the OpenID Connect relying party: 1. Create a clone of the [**nginx-openid-connect**](https://github.com/nginxinc/nginx-openid-connect) GitHub repository. ```shell git clone https://github.com/nginxinc/nginx-openid-connect ``` 2. Copy these files from the clone to **/etc/nginx/conf.d**: - **frontend.conf** - **openid_connect.js** - **openid_connect.server\_conf** 3. In your preferred text editor, open **/etc/nginx/conf.d/frontend.conf**. Change the second parameter of each of the following [set](http://nginx.org/en/docs/http/ngx_http_rewrite_module.html#set) directives to the specified value. The `` variable is the full value in the **Domain prefix** field in [Step 13 of _Configuring Amazon Cognito_](#cognito-domain-name). In this guide it is **https://my-nginx-plus.auth.us-east-2.amazoncognito.com**. - `set $oidc_authz_endpoint` – `/oauth2/authorize` - `set $oidc_token_endpoint` – `/oauth2/token` - `set $oidc_client` – Value in the **App client id** field from [Step 11 of _Configuring Amazon Cognito_](#cognito-app-client-id-secret) (in this guide, `2or4cs8bjo1lkbq6143tqp6ist`) - `set $oidc_client_secret` – Value in the **App client secret** field from [Step 11 of _Configuring Amazon Cognito_](#cognito-app-client-id-secret) (in this guide, `1k63m3nrcnu...`) - `set $oidc_hmac_key` – A unique, long, and secure phrase 4. Configure the JWK file. The file's URL is **https://cognito-idp.**_region_**.amazonaws.com/**_User-Pool-ID_**/.well-known/jwks.json** where - _region_ is the same AWS region name as in the `` variable used in [Step 3](#nginx-plus-variables) - _User-Pool-ID_ is the value in the **Pool Id** field in [Step 8 of _Configuring Amazon Cognito_](#cognito-pool-id) In this guide, the URL is **https://cognito-idp.us-east-2.amazonaws.com/us-east-2_mLoGHJpOs/.well-known/jwks.json**. The method for configuring the JWK file depends on which version of NGINX Plus you are using: - In NGINX Plus R17 and later, NGINX Plus can read the JWK file directly. Change **/etc/nginx/conf.d/frontend.conf** as follows: 1. Comment out (or remove) the [auth_jwt_key_file](http://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_file) directive. 2. Uncomment the [auth_jwt_key_request](http://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_request) directive. (Its parameter, `/_jwks_uri`, refers to the value of the `$oidc_jwt_keyfile` variable, which you set in the next step.) 3. Change the second parameter of the `set $oidc_jwt_keyfile` directive to the URL of the JWK file (`https://cognito-idp.../.well-known/jwks.json`). - In NGINX Plus R16 and earlier, the JWK file must be on the local disk. (You can also use this method with NGINX Plus R17 and later if you wish.) 1. Copy the JSON contents from the JWK file (****) to a local file (for example, **/etc/nginx/my\_cognito\_jwk.json**). 2. In **/etc/nginx/conf.d/frontend.conf**, change the second parameter of the `set $oidc_jwt_keyfile` directive to the local file path. 5. At the time of publication, Cognito does not support the OpenID **offline_access** scope. Open **/etc/nginx/conf.d/openid\_connect.server\_conf** in a text editor and remove `+offline_access` from the list of scopes on line 10, so that it looks like this: ```nginx return 302 "$oidc_authz_endpoint?response_type=code&scope=openid+profile+email&client_id=$oidc_clientaws...; ``` 6. Confirm that the user named by the [user](http://nginx.org/en/docs/ngx_core_module.html#user) directive in the NGINX Plus configuration (in **/etc/nginx/nginx.conf** by convention) has read permission on the JWK file. ## Testing In a browser, enter the address of your NGINX Plus instance and try to log in using the credentials of a user assigned to the application (see [Step 9 in _Configuring Amazon Cognito_](#cognito-users)). The NGINX logo that appears in the screenshot was added on Cognito's **UI customization** tab (not shown in this guide). ## Troubleshooting See the [**Troubleshooting**](https://github.com/nginxinc/nginx-openid-connect#troubleshooting) section at the **nginx-openid-connect** repository on GitHub. ### Revision History - Version 1 (March 2020) – Initial version (NGINX Plus Release 20) -------------------------------------------------------------------------------- title: "Single Sign-On With Auth0" url: https://docs.nginx.com/nginx/deployment-guides/single-sign-on/auth0/index.md description: Enable OpenID Connect-based single sign-on (SSO) for applications proxied by NGINX Plus, using Auth0 as the identity provider (IdP). -------------------------------------------------------------------------------- This guide explains how to enable single sign-on (SSO) for applications being proxied by F5 NGINX Plus. The solution uses OpenID Connect as the authentication mechanism, with [Auth0](https://auth0.com/features/single-sign-on) as the Identity Provider (IdP), and NGINX Plus as the Relying Party, or OIDC client application that verifies user identity. **Note:** This guide applies to [NGINX Plus Release 36](nginx/releases.md#r36) and later. In earlier versions, NGINX Plus relied on an [njs-based solution](#legacy-njs-guide), which required NGINX JavaScript files, key-value stores, and advanced OpenID Connect logic. In the latest NGINX Plus version, the new [OpenID Connect module](https://nginx.org/en/docs/http/ngx_http_oidc_module.html) simplifies this process to just a few directives. ## Prerequisites - An [Auth0](https://auth0.com/) tenant with administrator privileges. - An NGINX Plus [subscription](https://www.f5.com/products/nginx/nginx-plus) and NGINX Plus [Release 36](nginx/releases.md#r36) or later. For installation instructions, see [Installing NGINX Plus](https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-plus/). - A domain name pointing to your NGINX Plus instance, for example, `demo.example.com`. ## Create a new Auth0 Application {#auth0-create} 1. Log in to your Auth0 Dashboard at [manage.auth0.com](https://manage.auth0.com/). 2. Select **Applications > Applications** from the sidebar menu. 3. On the **Applications** screen, select **Create Application**. 4. On the **Create application** screen: - Enter the **Name** for the application, for example, **NGINX Demo App**. - In **Application Type**, select **Regular Web Applications**. - Select **Create**. 5. On the **Settings** screen of your application: - Copy your **Client ID** and **Client Secret** displayed in the **Basic Information** section — you will need them later when configuring NGINX Plus. 6. On the **Application URIs** section of your application: - Add the NGINX Plus callback URI in the **Allowed Callback URLs** field, for example: `https://demo.example.com/oidc_callback`. - Add the post logout URL in the **Allowed Logout URLs** field, for example: `https://demo.example.com/post_logout/`. ### Get the OpenID Connect Discovery URL Check the OpenID Connect Discovery URL. By default, Auth0 publishes the `.well-known/openid-configuration` document at the following address: `https://yourTenantId.us.auth0.com/.well-known/openid-configuration`. 1. Run the following `curl` command in a terminal: ```shell curl https://yourTenantId.us.auth0.com/.well-known/openid-configuration | jq ``` Where: - the `yourTenantId` is your Auth0 [Tenant ID](https://auth0.com/docs/get-started/tenant-settings/find-your-tenant-name-or-tenant-id) - the `yourTenantId.us.auth0.com/` is your Auth0 server address - the `/.well-known/openid-configuration` is the default address for Auth0 for document location - the `jq` command (optional) is used to format the JSON output for easier reading and requires the [jq](https://jqlang.github.io/jq/) JSON processor to be installed. The configuration metadata is returned in the JSON format: ```json { ... "issuer": "https://{yourTenantId}.us.auth0.com/", "authorization_endpoint": "https://{yourTenantId}.us.auth0.com/oauth/token", "token_endpoint": "https://{yourTenantId}.us.auth0.com/oauth/token", "jwks_uri": "https://{yourTenantId}.us.auth0.com/.well-known/jwks.json", "userinfo_endpoint": "https://{yourTenantId}.us.auth0.com/userinfo", "end_session_endpoint": "https://{yourTenantId}.us.auth0.com/oidc/logout", ... } ``` 2. Copy the **issuer** value, you will need it later when configuring NGINX Plus. Typically, the OpenID Connect Issuer for Auth0 is `https://yourTenantId.us.auth0.com/` (including the trailing slash). To verify the accuracy of the endpoints, refer to the [Auth0 official documentation](https://auth0.com/docs/get-started/applications/configure-applications-with-oidc-discovery). **Note:** You will need the values of **Client ID**, **Client Secret**, and **Issuer** in the next steps. ## Set up NGINX Plus {#nginx-plus-setup} With Auth0 configured, you can enable OIDC on NGINX Plus. NGINX Plus serves as the Rely Party (RP) application — a client service that verifies user identity. 1. Ensure that you are using the latest version of NGINX Plus by running the `nginx -v` command in a terminal: ```shell nginx -v ``` The output should match NGINX Plus Release 36 or later: ```none nginx version: nginx/1.29.3 (nginx-plus-r36) ``` 2. Ensure that you have the values of the **Client ID**, **Client Secret**, and **Issuer** obtained during [Auth0 Configuration](#auth0-setup). 3. In your preferred text editor, open the NGINX configuration file (`/etc/nginx/nginx.conf` for Linux or `/usr/local/etc/nginx/nginx.conf` for FreeBSD). 4. In the [`http {}`](https://nginx.org/en/docs/http/ngx_http_core_module.html#http) context, make sure your public DNS resolver is specified with the [`resolver`](https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver) directive: By default, NGINX Plus re‑resolves DNS records at the frequency specified by time‑to‑live (TTL) in the record, but you can override the TTL value with the `valid` parameter: ```nginx http { resolver 10.0.0.1 ipv4=on valid=300s; # ... } ``` 5. In the [`http {}`](https://nginx.org/en/docs/http/ngx_http_core_module.html#http) context, define the Auth0 provider named `auth0` by specifying the [`oidc_provider {}`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#oidc_provider) context: ```nginx http { resolver 10.0.0.1 ipv4=on valid=300s; oidc_provider auth0 { # ... } # ... } ``` 6. In the [`oidc_provider {}`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#oidc_provider) context, specify: - Your actual Auth0 **Client ID** obtained in [Auth0 Configuration](#auth0-create) with the [`client_id`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_id) directive - Your **Client Secret** obtained in [Auth0 Configuration](#auth0-create) with the [`client_secret`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_secret) directive - The **Issuer** URL obtained in [Auth0 Configuration](#auth0-create) with the [`issuer`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_secret) directive The `issuer` is typically your Auth0 OIDC URL. For Auth0, a trailing slash is included, for example: `https://yourTenantId.us.auth0.com/`. - The **logout_uri** is URI that a user visits to start an RP‑initiated logout flow. - The **post_logout_uri** is absolute HTTPS URL where Auth0 should redirect the user after a successful logout. This value **must also be specified** in **Allowed Logout URLs** on the Auth0 side. - If the **logout_token_hint** directive set to `on`, NGINX Plus sends the user’s ID token as a *hint* to Auth0. This directive is **optional**, however, if it is omitted the Auth0 may display an extra confirmation page asking the user to approve the logout request. - If the **userinfo** directive is set to `on`, NGINX Plus will fetch `/userinfo` from the Auth0 and append the claims from userinfo to the `$oidc_claims_` variables. - PKCE (Proof Key for Code Exchange) is automatically enabled when Auth0's OpenID Connect discovery document advertises the `S256` code challenge method in the `code_challenge_methods_supported` field. You can override this behavior with the [`pkce`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#pkce) directive: set `pkce off;` to disable PKCE even when `S256` is advertised, or `pkce on;` to force PKCE even if the IdP metadata does not list `S256`. - The module automatically selects the client authentication method for the token endpoint based on the provider metadata `token_endpoint_auth_methods_supported`. When only `client_secret_post` is advertised, NGINX Plus uses the `client_secret_post` method and sends the client credentials in the POST body. When both `client_secret_basic` and `client_secret_post` are present, the module prefers HTTP Basic (`client_secret_basic`), which remains the default for Auth0. - **Note:** All interaction with the IdP is secured exclusively over SSL/TLS, so NGINX must trust the certificate presented by the IdP. By default, this trust is validated against your system’s CA bundle (the default CA store for your Linux or FreeBSD distribution). If the IdP’s certificate is not included in the system CA bundle, you can explicitly specify a trusted certificate or chain with the [`ssl_trusted_certificate`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#ssl_trusted_certificate) directive so that NGINX can validate and trust the IdP’s certificate. ```nginx http { resolver 10.0.0.1 ipv4=on valid=300s; oidc_provider auth0 { issuer https://yourTenantId.us.auth0.com/; client_id ; client_secret ; logout_uri /logout; post_logout_uri https://demo.example.com/post_logout/; logout_token_hint on; userinfo on; # Optional: PKCE configuration. By default, PKCE is automatically # enabled when the IdP advertises the S256 code challenge method. # pkce on; } # ... } ``` 7. Make sure you have configured a [server](https://nginx.org/en/docs/http/ngx_http_core_module.html#server) that corresponds to `demo.example.com`, and there is a [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) that [points](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass) to your application (see [Step 10](#oidc_app)) at `http://127.0.0.1:8080` that is going to be OIDC-protected: ```nginx http { # ... server { listen 443 ssl; server_name demo.example.com; ssl_certificate /etc/ssl/certs/fullchain.pem; ssl_certificate_key /etc/ssl/private/key.pem; location / { # ... proxy_pass http://127.0.0.1:8080; } } # ... } ``` 8. Protect this [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) with Auth0 OIDC by specifying the [`auth_oidc`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#auth_oidc) directive that will point to the `auth0` configuration specified in the [`oidc_provider {}`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#oidc_provider) context in [Step 5](#auth0-setup-oidc-provider): ```nginx # ... location / { auth_oidc auth0; # ... proxy_pass http://127.0.0.1:8080; } # ... ``` 9. Pass the OIDC claims as headers to the application ([Step 10](#oidc_app)) with the [`proxy_set_header`](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_header) directive. These claims are extracted from the ID token returned by Auth0: - [`$oidc_claim_email`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) the e-mail address of the user - [`$oidc_claim_name`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) - the full name of the user - any other OIDC claim using the [`$oidc_claim_ `](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) variable ```nginx # ... location / { auth_oidc auth0; proxy_set_header sub $oidc_claim_sub; proxy_set_header email $oidc_claim_email; proxy_set_header name $oidc_claim_name; proxy_pass http://127.0.0.1:8080; } # ... ``` 10. Provide endpoint for completing logout: ```nginx # ... location /post_logout/ { return 200 "You have been logged out.\n"; default_type text/plain; } # ... ``` 11. Create a simple test application referenced by the `proxy_pass` directive which returns the authenticated user's full name and email upon successful authentication: ```nginx # ... server { listen 8080; location / { return 200 "Hello, $http_name!\nEmail: $http_email\nAuth0 sub: $http_sub\n"; default_type text/plain; } } ``` 12. Save the NGINX configuration file and reload the configuration: ```nginx nginx -s reload ``` ### Complete Example This configuration example summarizes the steps outlined above. It includes only essential settings such as specifying the DNS resolver, defining the OIDC provider, configuring SSL, and proxying requests to an internal server. ```nginx http { # Use a public DNS resolver for Issuer discovery, etc. resolver 10.0.0.1 ipv4=on valid=300s; oidc_provider auth0 { # Issuer from your Auth0 tenant's .well-known/openid-configuration issuer https://yourTenantId.us.auth0.com/; # Replace with your actual Client ID and Secret from Auth0 client_id ; client_secret ; # RP‑initiated logout logout_uri /logout; post_logout_uri https://demo.example.com/post_logout/; logout_token_hint on; # Fetch userinfo claims userinfo on; # Optional: PKCE configuration # pkce on; } server { listen 443 ssl; server_name demo.example.com; ssl_certificate /etc/ssl/certs/fullchain.pem; ssl_certificate_key /etc/ssl/private/key.pem; location / { # Enforce OIDC for root path with Auth0 auth_oidc auth0; # Forward OIDC claims to the upstream as headers if desired proxy_set_header email $oidc_claim_email; proxy_set_header name $oidc_claim_name; proxy_pass http://127.0.0.1:8080; } location /post_logout/ { return 200 "You have been logged out.\n"; default_type text/plain; } } server { # Simple test upstream server listen 8080; location / { return 200 "Hello, $http_name!\nEmail: $http_email\nAuth0 sub: $http_sub\n"; default_type text/plain; } } } ``` ### Testing 1. Open `https://demo.example.com/` in a browser. You will be redirected to the Auth0 sign-in page. 2. Enter valid Auth0 credentials of a user who has access the application. Upon successful sign-in, Auth0 redirects you back to NGINX Plus, and you will see the proxied application content (for example, “Hello, Jane Doe!”). 3. Navigate to `https://demo.example.com/logout`. NGINX Plus initiates an RP‑initiated logout; Auth0 ends the session and redirects back to `https://demo.example.com/post_logout/`. 4. Refresh `https://demo.example.com/` again. You should be redirected to Auth0 for a fresh sign‑in, proving the session has been terminated. ## Legacy njs-based Auth0 Solution {#legacy-njs-guide} If you are running NGINX Plus R33 and earlier or if you still need the njs-based solution, refer to the [Legacy njs-based Auth0 Guide](nginx/deployment-guides/single-sign-on/oidc-njs/auth0.md) for details. The solution uses the [`nginx-openid-connect`](https://github.com/nginxinc/nginx-openid-connect) GitHub repository and NGINX JavaScript files. ## See Also - [NGINX Plus Native OIDC Module Reference documentation](https://nginx.org/en/docs/http/ngx_http_oidc_module.html) - [Release Notes for NGINX Plus R36](nginx/releases.md#r36) ## Revision History - Version 3 (November 2025) – Updated for NGINX Plus R36; added PKCE configuration (`pkce` directive) and the `client_secret_post` token endpoint authentication method. - Version 2 (August 2025) – Updated for NGINX Plus R35; added RP‑initiated logout (`logout_uri`, `post_logout_uri`, `logout_token_hint`) and `userinfo` support. - Version 1 (March 2025) – Initial version (NGINX Plus Release 34). -------------------------------------------------------------------------------- title: "Single Sign-On With Auth0 and njs" url: https://docs.nginx.com/nginx/deployment-guides/single-sign-on/oidc-njs/auth0/index.md description: Learn how to enable single sign-on (SSO) with [Auth0](https://auth0.com/) for applications proxied by F5 NGINX Plus. -------------------------------------------------------------------------------- **Note:** This guide applies to NGINX Plus [Release 15](nginx/releases.md#r15) and later, based on the [`nginx-openid-connect`](https://github.com/nginxinc/nginx-openid-connect) GitHub repo. Starting with NGINX Plus [Release 34](nginx/releases.md#r34), use the simpler solution with the [native OpenID connect module](https://nginx.org/en/docs/http/ngx_http_oidc_module.html). See [Single Sign-On With Auth0](nginx/deployment-guides/single-sign-on/auth0.md) for details. You can use F5 NGINX Plus with [Auth0](https://auth0.com/) and OpenID Connect to enable single sign-on (SSO) for your proxied applications. By following the steps in this guide, you will learn how to set up SSO using OpenID Connect as the authentication mechanism, with Auth0 as the identity provider (IdP), and NGINX Plus as the relying party. **Note:** You can find more information about the NGINX Plus OpenID Connect integration in the project's [GitHub repo](https://github.com/nginxinc/nginx-openid-connect#nginx-openid-connect). ## Prerequisites To complete the steps in this guide, you need the following: - An Auth0 tenant with administrator privileges. - [NGINX Plus](https://www.f5.com/products/nginx/nginx-plus) with a valid subscription. - The [NGINX JavaScript module](https://nginx.org/en/docs/njs/index.html) (`njs`) -- the `njs` module handles the interaction between NGINX Plus and Auth0. ## Install NGINX Plus and the njs Module {#install-nginx-plus-njs} 1. If you do not already have NGINX Plus installed, follow the steps in the [NGINX Plus Admin Guide](https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-plus/) to do so. 2. Install the NGINX JavaScript module by following the steps in the [`njs` installation guide](https://nginx.org/en/docs/njs/install.html). 3. Add the following directive to the top-level ("main") configuration context in the NGINX Plus configuration (`/etc/nginx/nginx.conf`) to load the `njs` module: ```Nginx configuration file load_module modules/ngx_http_js_module.so; ``` ## Configure Auth0 {#config-auth0} Take the steps in this section to create a new application for NGINX Plus. **Note:** This section contains images that reflect the state of the Auth0 web interface at the time of publication. The actual Auth0 GUI may differ from the examples shown here. Use this guide as a reference and adapt the instructions to suit the current Auth0 GUI as necessary. ### Create a new Auth0 Application {#create-auth0-app} 1. Log in to your Auth0 Dashboard at [manage.auth0.com](https://manage.auth0.com/). 1. Select **Applications > Applications** from the sidebar menu. 1. On the **Applications** page, select the **Create Application** button. 1. In the **Create application** window, provide the information listed below and then select **Create**. - **Name**: A name for the application, for example "nginx-plus-app". - **Application Type**: **Regular Web Applications** ![image showing the Create application window in the Auth0 dashboard](/img/sso/auth0/sso-auth0-create-app.png) ### Set up the Web Application {#web-app-setup} In this section, you'll set up a web application that follows the Auth0 [Authorization Code Flow](https://auth0.com/docs/get-started/authentication-and-authorization-flow/authorization-code-flow). 1. On the **Application** page in the [Auth0 dashboard](https://manage.auth0.com/), select your web application. 1. Select the **Settings** tab for your application. 1. Make note of the Client ID and Client Secret displayed in the **Basic Information** section. ![image showing the basic information section of the web application settings in the Auth0 dashboard](/img/sso/auth0/sso-auth0-app.png) 1. In the **Application URIs** section, provide the URI of the NGINX Plus instance in the **Allowed Callback URLs** field. - The URL must include the port number and end in **/_codexch**. In our example, we used the URL `http://nginx-plus-app:8010/_codexch`. - The port is always required, even if you use the default port for HTTP (`80`) or HTTPS (`443`). - The use of SSL/TLS (`443`) is strongly recommended for production environments. ![image showing the Application URIs settings in the Auth0 dashboard](/img/sso/auth0/sso-auth0-app-settings.png) 1. In the **Advanced Settings** section, select the **Endpoints** tab. 1. Make note of the **OpenID Configuration** URL. ![image showing the Advanced Application Settings section of the Auth0 dashboard](/img/sso/auth0/sso-auth0-app-advanced-settings.png) 1. Select **Save Changes**. ### Set up Authentication {#authn-setup} **Note:** For the purposes of this guide, we will add a new Auth0 user database and user account to use for testing. You can set up authentication using any of the available [Auth0 identity providers](https://auth0.com/docs/authenticate/identity-providers). To set up a new user database and add a user account to it, take the steps below. 1. Log in to the [Auth0 dashboard](https://manage.auth0.com/) and select **Authentication > Database** from the sidebar menu. 1. Select the **Create DB Connection** button. 1. Provide a **Name** for the database connection, then select **Create**. 1. On the **Database** page, select the **Applications** tab. Then, select the toggle button next to the [application you created earlier](#create-a-new-auth0-application). ![image showing the Applications settings for an OIDC Authentication database in the Auth0 dashboard](/img/sso/auth0/sso-auth0-db-app.png) 1. In the sidebar menu, select **User Management > Users**. 1. On the **Users** page, select the **Create User** button. 1. In the **Create user** window, provide the following information, then select **Create**. - **Email**: user's email - **Password**: a password for the user account - **Connection**: select your **database** from the list. ![image showing the Create User settings window in the Auth0 dashboard](/img/sso/auth0/sso-auth0-create-user.png) The user should receive an email to the email address provided. Once the user verifies their account by clicking on the link in the email, the account creation process is complete. ## Set up NGINX Plus {#nginx-plus-setup} Take the steps in this section to set up NGINX Plus as the OpenID Connect relying party. ### Configure NGINX OpenID Connect {#nginx-plus-oidc-config} 1. Clone the [nginx-openid-connect](https://github.com/nginxinc/nginx-openid-connect) GitHub repository, or download the repo files. ```bash git clone https://github.com/nginxinc/nginx-openid-connect.git ``` 1. Run the *configure.sh* script, which will update the NGINX configuration files with the values for your Auth0 application. For example: ```bash ./nginx-openid-connect/configure.sh \ --auth_jwt_key request \ --client_id Nhotzxx...IERmUi \ --client_secret 6ZHd0j_r...UtDZ5bkdu \ https://.us.auth0.com/.well-known/openid-configuration ``` 1. In the `frontend.conf` file, update the **my_backend** upstream with the address of the application that you want to add OIDC authorization to. For example: ```Nginx configuration file upstream my_backend { zone my_backend 64k; server my-backend-app.com:80; } ``` 1. In the *openid_connect.server_conf* file, add the [`proxy_set_header`](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_header) directive to the `/_jwks_uri` and `/_token` locations to `Accept-Encoding "gzip"`, as shown below. ```Nginx configuration file ... location = /_jwks_uri { ... proxy_set_header Accept-Encoding "gzip" } ... location = /_token { ... proxy_set_header Accept-Encoding "gzip" } ... ``` 1. Copy the following files to the */etc/nginx/conf.d* directory on the host machine where NGINX Plus is installed: - `frontend.conf` - `openid_connect.js` - `openid_connect.server_conf` - `openid_connect_configuration.conf` 1. Reload the NGINX configuration: ```bash sudo nginx -s reload ``` ## Test the Setup 1. In a browser, enter the address of your NGINX Plus instance. You should be directed to the Auth0 login page, as shown in the example below. ![image showing an example Auth0 login screen that contains username and password fields](/img/sso/auth0/sso-auth0-login-test.png) 1. You should be able to log in using the credentials of the user account that you created in the Auth0 database. ## Troubleshooting Refer to the [Troubleshooting](https://github.com/nginxinc/nginx-openid-connect#troubleshooting) section in the `nginx-openid-connect` repository on GitHub. ## Revision History - Version 1 (May 2022) - Initial version -------------------------------------------------------------------------------- title: "Single Sign-On with Keycloak and njs" url: https://docs.nginx.com/nginx/deployment-guides/single-sign-on/oidc-njs/keycloak/index.md description: Enable OpenID Connect-based single-sign for applications proxied by NGINX Plus, using Keycloak as the identity provider (IdP). -------------------------------------------------------------------------------- **Note:** This guide applies to NGINX Plus [Release 15](nginx/releases.md#r15) and later, based on the [`nginx-openid-connect`](https://github.com/nginxinc/nginx-openid-connect) GitHub repo. Starting with NGINX Plus [Release 34](nginx/releases.md#r34), use the simpler solution with the [native OpenID connect module](https://nginx.org/en/docs/http/ngx_http_oidc_module.html). See [Single Sign-On With Keycloak](nginx/deployment-guides/single-sign-on/keycloak.md) for details. This guide explains how to enable single sign-on (SSO) for applications being proxied by F5 NGINX Plus. The solution uses OpenID Connect as the authentication mechanism, with [Keycloak](https://www.keycloak.org/) as the identity provider (IdP), and NGINX Plus as the relying party. **Note:** You can find more information about the NGINX Plus OpenID Connect integration in the project's [GitHub repo](https://github.com/nginxinc/nginx-openid-connect#nginx-openid-connect). ## Prerequisites The instructions assume you have the following: - A running Keycloak server. See the Keycloak documentation for [Getting Started](https://www.keycloak.org/guides#getting-started) and [Server](https://www.keycloak.org/guides#server) configuration instructions. - An NGINX Plus subscription and NGINX Plus R15 or later. For installation instructions, see the [NGINX Plus Admin Guide](https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-plus/). - The [NGINX JavaScript module](https://nginx.org/en/docs/njs/index.html) (njs), required for handling the interaction between NGINX Plus and the IdP. After installing NGINX Plus, install the module with the command for your operating system. For Debian and Ubuntu: ```none sudo apt install nginx-plus-module-njs ``` For CentOS, RHEL, and Oracle Linux: ```shell sudo yum install nginx-plus-module-njs ``` - The following directive included in the top-level ("main") configuration context in **/etc/nginx/nginx.conf**, to load the NGINX JavaScript module: ```nginx load_module modules/ngx_http_js_module.so; ``` ## Configuring Keycloak **Note:** The following procedure reflects the Keycloak GUI at the time of publication, but the GUI is subject to change. Use this guide as a reference and adapt to the current Keycloak GUI as necessary. Create a Keycloak client for NGINX Plus in the Keycloak GUI: 1. Access the Keycloak Admin Console at **http://_keycloak-server-address_:8080/auth/admin/** and log in. 2. In the left navigation column, click **Clients**. On the **Clients** page that opens, click the **Create** button in the upper right corner. 3. On the **Add Client** page that opens, enter or select these values, then click the  Save  button. - **Client ID** – The name of the application for which you're enabling SSO (Keycloak refers to it as the “client”). Here we're using **NGINX-Plus**. - **Client Protocol** – **openid-connect**. 4. On the **NGINX Plus** page that opens, enter or select these values on the Settings tab: - **Access Type** – **confidential** - **Valid Redirect URIs** – The URI of the NGINX Plus instance, including the port number, and ending in **/\_codexch** (in this guide it is **https://my-nginx.example.com:443/_codexch**) **Notes:** - For production, we strongly recommend that you use SSL/TLS (port 443). - The port number is mandatory even when you're using the default port for HTTP (80) or HTTPS (443). 5. Click the Credentials tab and make a note of the value in the **Secret** field. You will copy it into the NGINX Plus configuration file in [Step 4 of _Configuring NGINX Plus_](#nginx-plus-variables). 6. Click the Roles tab, then click the **Add Role** button in the upper right corner of the page that opens. 7. On the **Add Role** page that opens, type a value in the **Role Name** field (here it is **nginx-keycloak-role**) and click the  Save  button. 8. In the left navigation column, click **Users**. On the **Users** page that opens, either click the name of an existing user, or click the **Add user** button in the upper right corner to create a new user. For complete instructions, see the [Keycloak documentation](https://www.keycloak.org/docs/latest/server_admin/index.html#user-management). 9. On the management page for the user (here, **user01**), click the Role Mappings tab. On the page that opens, select **NGINX-Plus** on the **Client Roles** drop‑down menu. Click **nginx-keycloak-role** in the **Available Roles** box, then click the **Add selected** button below the box. The role then appears in the **Assigned Roles** and **Effective Roles** boxes, as shown in the screenshot. ## Configuring NGINX Plus Configure NGINX Plus as the OpenID Connect relying party: 1. Create a clone of the [**nginx-openid-connect**](https://github.com/nginxinc/nginx-openid-connect) GitHub repository. ```shell git clone https://github.com/nginxinc/nginx-openid-connect ``` 2. Copy these files from the clone to **/etc/nginx/conf.d**: - **frontend.conf** - **openid\_connect.js** - **openid\_connect.server\_conf** - **openid\_connect\_configuration.conf** 3. Get the URLs for the authorization endpoint, token endpoint, and JSON Web Key (JWK) file from the Keycloak configuration. Run the following `curl` command in a terminal, piping the output to the indicated `python` command to output the entire configuration in an easily readable format. We've abridged the output to show only the relevant fields. ```shell $ curl https:///realms/master/.well-known/openid-configuration | python -m json.tool ... { "authorization_endpoint": "https:///realms/master/protocol/openid-connect/auth", ... "jwks_uri": "https:///realms/master/protocol/openid-connect/certs", ... "token_endpoint": "https:///realms/master/protocol/openid-connect/token", ... } ``` 4. Using your preferred text editor, open **/etc/nginx/conf.d/openid_connect_configuration.conf**. Change the "default" parameter value of each of the following [map](https://nginx.org/en/docs/http/ngx_http_map_module.html#map) directives to the specified value: - `map $host $oidc_authz_endpoint` – Value of `authorization_endpoint` from [Step 3](#nginx-plus-urls) (in this guide, `https:///realms/master/protocol/openid-connect/auth`) - `map $host $oidc_token_endpoint` – Value of `token_endpoint` from [Step 3](#nginx-plus-urls) (in this guide, `https:///realms/master/protocol/openid-connect/token`) - `map $host $oidc_client` – Value in the **Client ID** field from [Step 3 of _Configuring Keycloak_](#keycloak-client-id) (in this guide, `NGINX Plus`) - `map $host $oidc_client_secret` – Value in the **Secret** field from [Step 5 of _Configuring Keycloak_](#keycloak-secret) (in this guide, ``) - `map $host $oidc_hmac_key` – A unique, long, and secure phrase 5. Configure the JWK file. The procedure depends on which version of NGINX Plus you are using. - In NGINX Plus R17 and later, NGINX Plus can read the JWK file directly from the URL reported as `jwks_uri` in [Step 3](#nginx-plus-urls). Change **/etc/nginx/conf.d/frontend.conf** as follows: 1. Comment out (or remove) the [auth_jwt_key_file](http://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_file) directive. 2. Uncomment the [auth_jwt_key_request](http://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_request) directive. (Its parameter, `/_jwks_uri`, refers to the value of the `$oidc_jwt_keyfile` variable, which you set in the next step.) 3. Change the "default" parameter of the `map $host $oidc_jwt_keyfile` directive to the value reported in the `jwks_uri` field in [Step 3](#nginx-plus-urls) (in this guide, `https:///realms/master/protocol/openid-connect/certs`). - In NGINX Plus R16 and earlier, the JWK file must be on the local disk. (You can also use this method with NGINX Plus R17 and later if you wish.) 1. Copy the JSON contents from the JWK file named in the `jwks_uri` field in [Step 3](#nginx-plus-urls) (in this guide, `https:///realms/master/protocol/openid-connect/certs`) to a local file (for example, `/etc/nginx/my_keycloak_jwk.json`). 2. In **/etc/nginx/conf.d/openid_connect_configuration.conf**, change the "default" parameter of the `map $host $oidc_jwt_keyfile` directive to the local file path. 6. Confirm that the user named by the [user](http://nginx.org/en/docs/ngx_core_module.html#user) directive in the NGINX Plus configuration (in **/etc/nginx/nginx.conf** by convention) has read permission on the JWK file. ## Testing In a browser, enter the address of your NGINX Plus instance and try to log in using the credentials of a user mapped to the role for NGINX Plus (see [Step 9 of _Configuring Keycloak_](#keycloak-users)). ## Troubleshooting See the [**Troubleshooting**](https://github.com/nginxinc/nginx-openid-connect#troubleshooting) section at the **nginx-openid-connect** repository on GitHub. ### Revision History - Version 2 (March 2020) – Updates to _Configuring NGINX Plus_ section - Version 1 (November 2019) – Initial version (NGINX Plus Release 19) -------------------------------------------------------------------------------- title: "Single Sign-On with Microsoft AD FS and njs" url: https://docs.nginx.com/nginx/deployment-guides/single-sign-on/oidc-njs/active-directory-federation-services/index.md description: Enable OpenID Connect-based single-sign for applications proxied by NGINX Plus, using Microsoft AD FS as the identity provider (IdP). -------------------------------------------------------------------------------- **Note:** This guide applies to NGINX Plus [Release 15](nginx/releases.md#r15) and later, based on the [`nginx-openid-connect`](https://github.com/nginxinc/nginx-openid-connect) GitHub repo. Starting with NGINX Plus [Release 34](nginx/releases.md#r34), use the simpler solution with the [native OpenID connect module](https://nginx.org/en/docs/http/ngx_http_oidc_module.html). See [Single Sign-On With Microsoft AD FS](nginx/deployment-guides/single-sign-on/active-directory-federation-services.md) for details. This guide explains how to enable single sign-on (SSO) for applications being proxied by F5 NGINX Plus. The solution uses OpenID Connect as the authentication mechanism, with [Microsoft Active Directory Federation Services](https://docs.microsoft.com/en-us/windows-server/identity/active-directory-federation-services) (AD FS) as the identity provider (IdP) and NGINX Plus as the relying party. **Note:** You can find more information about the NGINX Plus OpenID Connect integration in the project's [GitHub repo](https://github.com/nginxinc/nginx-openid-connect#nginx-openid-connect). ## Prerequisites The instructions assume you have the following: - A running deployment of AD FS, either on‑premises or in Azure. - An NGINX Plus subscription and NGINX Plus R15 or later. For installation instructions, see the [NGINX Plus Admin Guide](https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-plus/). - The [NGINX JavaScript module](https://nginx.org/en/docs/njs/index.html) (njs), required for handling the interaction between NGINX Plus and the IdP. After installing NGINX Plus, install the module with the command for your operating system. For Debian and Ubuntu: ```none sudo apt install nginx-plus-module-njs ``` For CentOS, RHEL, and Oracle Linux: ```shell sudo yum install nginx-plus-module-njs ``` - The following directive included in the top-level ("main") configuration context in **/etc/nginx/nginx.conf**, to load the NGINX JavaScript module: ```nginx load_module modules/ngx_http_js_module.so; ``` ## Configuring AD FS Create an AD FS application for NGINX Plus: 1. Open the AD FS Management window. In the navigation column on the left, right‑click on the **Application Groups** folder and select **Add Application Group** from the drop‑down menu. The **Add Application Group Wizard** window opens. The left navigation column shows the steps you will complete to add an application group. 2. In the **Welcome** step, type the application group name in the **Name** field. Here we are using **ADFSSSO**. In the **Template** field, select **Server application** under **Standalone applications**. Click the  Next >  button. 3. In the **Server application** step: 1. Make a note of the value in the **Client Identifier** field. You will add it to the NGINX Plus configuration in [Step 4 of _Configuring NGINX Plus_](#nginx-plus-variables).
2. In the **Redirect URI** field, type the URI of the NGINX Plus instance including the port number, and ending in **/\_codexch**. Here we’re using **https://my-nginx.example.com:443/\_codexch**. Click the  Add  button. **Notes:** - For production, we strongly recommend that you use SSL/TLS (port 443). - The port number is mandatory even when you're using the default port for HTTP (80) or HTTPS (443). 3. Click the  Next >  button. 4. In the **Configure Application Credentials** step, click the **Generate a shared secret** checkbox. Make a note of the secret that AD FS generates (perhaps by clicking the **Copy to clipboard** button and pasting the clipboard content into a file). You will add the secret to the NGINX Plus configuration in [Step 4 of _Configuring NGINX Plus_](#nginx-plus-variables). Click the  Next >  button. 5. In the **Summary** step, verify that the information is correct, make any necessary corrections to previous steps, and click the  Next >  button. ## Configuring NGINX Plus Configure NGINX Plus as the OpenID Connect relying party: 1. Create a clone of the [**nginx-openid-connect**](https://github.com/nginxinc/nginx-openid-connect) GitHub repository. ```shell git clone https://github.com/nginxinc/nginx-openid-connect ``` 2. Copy these files from the clone to **/etc/nginx/conf.d**: - **frontend.conf** - **openid\_connect.js** - **openid\_connect.server\_conf** - **openid\_connect\_configuration.conf** 3. Get the URLs for the authorization endpoint, token endpoint, and JSON Web Key (JWK) file from the AD FS configuration. Run the following `curl` command in a terminal, piping the output to the indicated `python` command to output the entire configuration in an easily readable format. We've abridged the output to show only the relevant fields. ```shell $ curl https:///oidc/adfs/.well-known/openid-configuration | python -m json.tool { ... "authorization_endpoint": "https:///oidc/adfs/auth", ... "jwks_uri": "https:///oidc/adfs/certs", ... "token_endpoint": "https:///oidc/adfs/token", ... } ``` 4. In your preferred text editor, open **/etc/nginx/conf.d/frontend.conf**. Change the "default" parameter value of each of the following [map](https://nginx.org/en/docs/http/ngx_http_map_module.html#map) directives to the specified value: - `map $host $oidc_authz_endpoint` – Value of `authorization_endpoint` from [Step 3](#nginx-plus-urls) (in this guide, `https:///oidc/adfs/auth`) - `map $host $oidc_token_endpoint` – Value of `token_endpoint` from [Step 3](#nginx-plus-urls) (in this guide, `https:///oidc/adfs/token`) - `map $host $oidc_client` – Value in the **Client ID** field from [Step 3 of _Configuring AD FS_](#ad-fs-server-application) (in this guide, `3e23f0eb-9329-46ff-9d37-6ad24afdfaeb`) - `map $host $oidc_client_secret` – Value in the **Client secret** field from [Step 4 of _Configuring AD FS_](#ad-fs-configure-application-credentials) (in this guide, `NUeuULtSCjgXTGSkq3ZwEeCOiig4-rB2XiW_W`) - `map $host $oidc_hmac_key` – A unique, long, and secure phrase 5. Configure the JWK file. The procedure depends on which version of NGINX Plus you are using. - In NGINX Plus R17 and later, NGINX Plus can read the JWK file directly from the URL reported as `jwks_uri` in [Step 3](#nginx-plus-urls). Change **/etc/nginx/conf.d/frontend.conf** as follows: 1. Comment out (or remove) the [auth_jwt_key_file](http://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_file) directive. 2. Uncomment the [auth_jwt_key_request](http://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_request) directive. (Its parameter, `/_jwks_uri`, refers to the value of the `$oidc_jwt_keyfile` variable, which you set in the next step.) 3. Change the second parameter of the `set $oidc_jwt_keyfile` directive to the value reported in the `jwks_uri` field in [Step 3](#nginx-plus-urls) (in this guide, `https:///oidc/adfs/certs`). - In NGINX Plus R16 and earlier, the JWK file must be on the local disk. (You can also use this method with NGINX Plus R17 and later if you wish.) 1. Copy the JSON contents from the JWK file named in the `jwks_uri` field in [Step 3](#nginx-plus-urls) (in this guide, `https:///oidc/adfs/certs`) to a local file (for example, `/etc/nginx/my_adfs_jwk.json`). 2. In **/etc/nginx/conf.d/frontend.conf**, change the second parameter of the `set $oidc_jwt_keyfile` directive to the local file path. 6. Confirm that the user named by the [user](http://nginx.org/en/docs/ngx_core_module.html#user) directive in the NGINX Plus configuration (in **/etc/nginx/nginx.conf** by convention) has read permission on the JWK file. ## Testing In a browser, enter the address of your NGINX Plus instance and try to log in using the credentials of a user who has access to the application. ## Troubleshooting See the [**Troubleshooting**](https://github.com/nginxinc/nginx-openid-connect#troubleshooting) section at the **nginx-openid-connect** repository on GitHub. ### Revision History - Version 2 (March 2020) – Updates to _Configuring NGINX Plus_ section - Version 1 (December 2019) – Initial version (NGINX Plus Release 20) -------------------------------------------------------------------------------- title: "Single Sign-On with Okta and njs" url: https://docs.nginx.com/nginx/deployment-guides/single-sign-on/oidc-njs/okta/index.md description: Learn how to enable single sign-on (SSO) with Okta for applications proxied by F5 NGINX Plus. -------------------------------------------------------------------------------- **Note:** This guide applies to NGINX Plus [Release 15](nginx/releases.md#r15) and later, based on the [`nginx-openid-connect`](https://github.com/nginxinc/nginx-openid-connect) GitHub repo. Starting with NGINX Plus [Release 34](nginx/releases.md#r34), use the simpler solution with the [native OpenID connect module](https://nginx.org/en/docs/http/ngx_http_oidc_module.html). See [Single Sign-On With Okta](nginx/deployment-guides/single-sign-on/okta.md) for details. You can use NGINX Plus with Okta and OpenID Connect to enable single sign-on (SSO) for your proxied applications. By following the steps in this guide, you will learn how to set up SSO using OpenID Connect as the authentication mechanism, with Okta as the identity provider (IdP), and NGINX Plus as the relying party. **Note:** You can find more information about the NGINX Plus OpenID Connect integration in the project's [GitHub repo](https://github.com/nginxinc/nginx-openid-connect#nginx-openid-connect). ## Prerequisites To complete the steps in this guide, you need the following: - An Okta administrator account. - [NGINX Plus](https://www.f5.com/products/nginx/nginx-plus) with a valid subscription. - The [NGINX JavaScript module](https://nginx.org/en/docs/njs/index.html) (`njs`) -- the `njs` module handles the interaction between NGINX Plus and Okta. - Install `jq` on the host machine where you installed NGINX Plus. ## Install NGINX Plus and the njs Module 1. If you do not already have NGINX Plus installed, follow the steps in the [NGINX Plus Admin Guide](https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-plus/) to do so. 2. Install the NGINX JavaScript module by following the steps in the [`njs` installation guide](https://nginx.org/en/docs/njs/install.html). 3. Add the following directive to the top-level ("main") configuration context in the NGINX Plus configuration (`/etc/nginx/nginx.conf`) to load the `njs` module: ```Nginx configuration file load_module modules/ngx_http_js_module.so; ``` ## Configure Okta {#okta} Take the steps in this section to create a new application for NGINX Plus. **Note:** This section contains images that reflect the state of the Okta web interface at the time of publication. The actual Okta GUI may differ from the examples shown here. Use this guide as a reference and adapt the instructions to suit the current Okta GUI as necessary. This section describes the Okta Workforce Identity SSO product. You will need administrator access to your organization in Okta to complete this task. Your experience may differ somewhat if you're using the Okta Customer Identity product. ### Create a New Okta Web Application 1. Log in to Okta at [okta.com](https:///www.okta.com). 1. Select the **Admin** button next to your username to access the Admin console. 1. On your Admin dashboard, select **Applications** in the left-hand navigation menu. 1. On the **Applications** page, select the **Create App Integration** button. 1. In the **Create a new app integration** window, define the following values, then select **Next**: - **Sign-in method**: OIDC - OpenID Connect - **Application type**: Web Application ![image showing the Create a new app integration window in the Okta UI, with OIDC and Web Application options selected](/img/sso/okta/Okta-Create-New-Application-Integration.png) ### Set up the Web App Integration {#okta-integration} On the **New Web App Integration** page in the Okta web interface, fill in the following information, then select **Save**. | Field | Desciption | Example Value | |-------------|---------|----------| | **App integration name** | The name of the OpenID Connect relying party. Okta refers to this as the "application". | **NGINX-Plus** | | **Sign-in redirect URIs** | The URI of the NGINX Plus instance -- including the port number -- ending in **`/_codexch`**.
  • The port is always required, even if you use the default port for HTTP (`80`) or HTTPS (`443`).
  • The use of SSL/TLS (`443`) is strongly recommended for production environments.
| `https://my-nginx.example.com:443/_codexch` | | **Sign-out redirect URIs** | The URI to redirect users to after logging out.
This is an optional field with a default value of `http://localhost:8080`. | We removed the default value in our example. | | **Controlled access** | Controls who can access the application. | "Allow everyone in your organization to access"
**You should select the appropriate value for your use case.**| ![Okta Create OpenID Connect Integration](/img/sso/okta/Okta-Create-OpenID-Connect-Integration.png) ### Get the Okta App Client Credentials {#okta-client-id-secret} After you finish creating your application, the Okta Application page should display. You can find the Client Credentials for your Okta Application here. ![Image showing the application landing page in Okta, which contains the Client Credentials for the application.](/img/sso/okta/Okta-Client-Credentials.png) **Note:** If you need to find this information later, log in to your Okta admin account as [described above](#okta-login), select **Applications** in the left-hand menu, then select your application. Make note of the **Client ID** and **Client secret** values for your application. You will need these when you [configure NGINX Plus](#nginx-plus). ### Manage Access to your Okta Application {#okta-assign-applications} To change the users and groups that have access to your Okta Application: 1. Log in to Okta as an Admin as [described above](#okta-login). 1. Select **Applications** in the left-hand menu, then select your application. 1. Select the **Assignments** tab for the Application. Here, you can manage which users in your organization are granted access to this application. ## Set up NGINX Plus {#nginx-plus} Take the steps in this section to set up NGINX Plus as the OpenID Connect relying party. ### Configure NGINX OpenID Connect {#nginx-plus-oidc-config} 1. Clone the [nginx-openid-connect](https://github.com/nginxinc/nginx-openid-connect) GitHub repository, or download the repo files. ```shell git clone https://github.com/nginxinc/nginx-openid-connect.git ``` 1. Copy the following files to the `/etc/nginx/conf.d` directory on the host machine where NGINX Plus is installed: - `frontend.conf` - `openid_connect.js` - `openid_connect.server_conf` - `openid_connect_configuration.conf` 1. Get the URLs for the authorization endpoint, token endpoint, and JSON Web Key (JWK) file from the Okta configuration. Run the following `curl` command in a terminal. **Note:** We recommend piping the output to `jq` to output the entire configuration in an easily readable format. The output in the example below is abridged to show only the relevant fields. ```shell curl https://-admin.okta.com/.well-known/openid-configuration | jq ... { "authorization_endpoint": "https://.okta.com/oauth2/v1/authorize", ... "jwks_uri": "https://.okta.com/oauth2/v1/keys", ... "token_endpoint": "https://.okta.com/oauth2/v1/token", ... } ``` 1. Add the correct values for your IdP to the OpenID Connect configuration file (`/etc/nginx/conf.d/openid_connect_configuration.conf`). This file contains the primary configuration for one or more IdPs in `map{}` blocks. You should modify the `map…$oidc_` blocks as appropriate to match your IdP configuration. - Define the `$oidc_authz_endpoint`, `$oidc_token_endpoint`, and `$oidc_jwt_keyfile` values using the information returned in the previous step. - Change the URI defined in `map…$oidc_logout_redirect` to the URI of a resource (for example, your home page) that should be displayed after a client requests the `/logout` location. - Set a unique, long, and secure phrase for `$oidc_hmac_key` to ensure nonce values are unpredictable. ### Set up JSON Web Key Authorization {#nginx-plus-jwk-config} NGINX Plus can read the JWK file directly from the URL reported as `jwks_uri` in the output of the `curl` command you ran in the [previous section](#nginx-plus-oidc-config). **Note:** If you are using NGINX Plus R16 or earlier, refer to [Set up JWK Authorization using a local file](#nginx-plus-jwk-auth-local). Take the following steps to set up NGINX Plus to access the JWK file by using a URI. 1. In the `/etc/nginx/conf.d/frontend.conf` file, remove (or comment out) the [auth_jwt_key_file](http://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_file) directive. 1. Uncomment the [auth_jwt_key_request](http://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_request) directive. The parameter `/_jwks_uri` refers to the value of the `$oidc_jwt_keyfile` variable, which you already set in the OpenID Connect configuration file (`/etc/nginx/conf.d/openid_connect_configuration.conf`). #### Set up JWK Authorization using a Local File {#nginx-plus-jwk-auth-local} In NGINX Plus R16 and earlier, NGINX Plus cannot access the JWK file via the URI. Instead, the JWK file must be on the local disk. Take the steps below to set up JWK authorization using a local file: 1. Copy the JSON contents from the JWK file named in the `jwks_uri` field to a local file. For example, `/etc/nginx/my_okta_jwk.json` 1. In `/etc/nginx/conf.d/frontend.conf`, change the second parameter of the `set $oidc_jwt_keyfile` directive to the local file path of the JWK file. 1. Confirm that the user named by the [user](http://nginx.org/en/docs/ngx_core_module.html#user) directive in the NGINX Plus configuration -- usually found in `/etc/nginx/nginx.conf` -- has read permission on the JWK file. ## Test Your Setup 1. In a browser, enter the address of your NGINX Plus instance. You should be directed to the okta login page, as shown in the example below. ![image](img/sso/okta/Okta-login-window.png) 1. Try to log in using the credentials of a user who is part of your organization. **Note:** If you restricted access to a group of users, be sure to select a user who has access to the application. ## Troubleshooting Refer to the [Troubleshooting](https://github.com/nginxinc/nginx-openid-connect#troubleshooting) section in the `nginx-openid-connect` repository on GitHub. ### Revision History - Version 3 (March 2022) – Full edit incorporating updates to _Configuring Okta_ and _Configuring NGINX Plus_ - Version 2 (March 2020) – Updates to _Configuring NGINX Plus_ section - Version 1 (April 2019) – Initial version (NGINX Plus Release 17) -------------------------------------------------------------------------------- title: "Single Sign-On with OneLogin and njs" url: https://docs.nginx.com/nginx/deployment-guides/single-sign-on/oidc-njs/onelogin/index.md description: Learn how to enable single sign-on (SSO) with [OneLogin](https://www.onelogin.com/) for applications proxied by F5 NGINX Plus. -------------------------------------------------------------------------------- **Note:** This guide applies to NGINX Plus [Release 15](nginx/releases.md#r15) and later, based on the [`nginx-openid-connect`](https://github.com/nginxinc/nginx-openid-connect) GitHub repo. Starting with NGINX Plus [Release 34](nginx/releases.md#r34), use the simpler solution with the [native OpenID connect module](https://nginx.org/en/docs/http/ngx_http_oidc_module.html). See [Single Sign-On With OneLogin](nginx/deployment-guides/single-sign-on/onelogin.md) for details. You can use NGINX Plus with [OneLogin](https://www.onelogin.com/) and the OpenID Connect protocol to enable single sign-on (SSO) for your proxied applications. By following the steps in this guide, you will learn how to set up SSO using OpenID Connect as the authentication mechanism, with OneLogin as the identity provider (IdP) and NGINX Plus as the relying party. **Note:** You can find more information about the NGINX Plus OpenID Connect integration in the project's [GitHub repo](https://github.com/nginxinc/nginx-openid-connect#nginx-openid-connect). ## Prerequisites To complete the steps in this guide, you need the following: - A OneLogin tenant with administrator privileges. - [NGINX Plus](https://www.f5.com/products/nginx/nginx-plus) with a valid subscription. - The [NGINX JavaScript module](https://nginx.org/en/docs/njs/index.html) (`njs`) -- the `njs` module handles the interaction between NGINX Plus and OneLogin identity provider (IdP). ## Install NGINX Plus and the njs Module {#install-nginx-plus-njs} 1. If you do not already have NGINX Plus installed, follow the [NGINX Plus Admin Guide](https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-plus/) steps to do so. 2. Install the NGINX JavaScript module by following the steps in the [`njs` installation guide](https://nginx.org/en/docs/njs/install.html). 3. Add the following directive to the top-level ("main") configuration context in the NGINX Plus configuration (`/etc/nginx/nginx.conf`) to load the `njs` module: ```Nginx configuration file load_module modules/ngx_http_js_module.so; ``` ## Configure OneLogin {#config-onelogin} **Note:** The following procedure reflects the OneLogin GUI at the time of publication, but the GUI is subject to change. Use this guide as a reference and adapt to the current OneLogin GUI as necessary. Create a new application for NGINX Plus in the OneLogin GUI: 1. Log in to your OneLogin account at **https://**_domain_**.onelogin.com**, where _domain_ is the domain you chose when you created your account. 2. Click  Applications  in the title bar and then click the  Add App  button in the upper right corner of the window that opens. 3. On the **Find Applications** page that opens, type **OpenID Connect** in the search box. Click on the **OpenID Connect (OIDC)** row that appears. 4. On the **Add OpenId Connect (OIDC)** page that opens, change the value in the **Display Name** field to **NGINX Plus** and click the  Save  button. 5. When the save completes, a new set of choices appears in the left navigation bar. Click **Configuration**. In the **Redirect URI's** field, type the URI of the NGINX Plus instance including the port number, and ending in **/\_codexch** (in this guide it is **https://my-nginx.example.com:443/_codexch**). Then click the  Save  button. **Notes:** - For production, we strongly recommend that you use SSL/TLS (port 443). - The port number is mandatory even when you're using the default port for HTTP (80) or HTTPS (443). 6. When the save completes, click **SSO** in the left navigation bar. Click **Show client secret** below the **Client Secret** field. Record the values in the **Client ID** and **Client Secret** fields. You will add them to the NGINX Plus configuration in [Step 4 of _Configuring NGINX Plus_](#nginx-plus-variables). 7. Assign users to the application (in this guide, **NGINX Plus**) to enable them to access it for SSO. OneLogin recommends using [roles](https://onelogin.service-now.com/kb_view_customer.do?sysparm_article=KB0010606) for this purpose. You can access the **Roles** page under  Users  in the title bar. ## Set up NGINX Plus Take the steps in this section to set up NGINX Plus as the OpenID Connect Client. ### Configure NGINX OpenID Connect {#nginx-plus-oidc-config} 1. Clone the [nginx-openid-connect](https://github.com/nginxinc/nginx-openid-connect) GitHub repository, or download the repository files. ```shell git clone https://github.com/nginxinc/nginx-openid-connect.git ``` 1. Run the _configure.sh_ script to update the NGINX configuration files with the values for your OneLogin application. For example: ```bash ./nginx-openid-connect/configure.sh \ --auth_jwt_key request \ --client_id 168d5600-9224-... \ --client_secret c9210a67d09e85... \ https://.onelogin.com/oidc/2/.well-known/openid-configuration ``` 2. In the `frontend.conf` file, update the **my_backend** upstream with the address of the application that you want to add OIDC authorization to. For example: ```Nginx configuration file upstream my_backend { zone my_backend 64k; server my-backend-app.com:80; } ``` 3. In the _openid_connect.server_conf_ file, add the [`proxy_set_header`](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_header) directive to the `/_jwks_uri` and `/_token` locations to `Accept-Encoding "gzip"`, as shown below. ```Nginx configuration file ... location = /_jwks_uri { ... proxy_set_header Accept-Encoding "gzip" } ... location = /_token { ... proxy_set_header Accept-Encoding "gzip" } ... ``` 4. Copy the following files to the _/etc/nginx/conf.d_ directory on the host machine where NGINX Plus is installed: - `frontend.conf` - `openid_connect.js` - `openid_connect.server_conf` - `openid_connect_configuration.conf` 5. Reload the NGINX configuration: ```bash sudo nginx -s reload ``` ## Test Your Setup In a browser, enter the address of your NGINX Plus instance and try to log in using the credentials of a user assigned to the application (see [Step 7 of _Configuring OneLogin_](#onelogin-roles)). ## Troubleshooting Refer to the [Troubleshooting](https://github.com/nginxinc/nginx-openid-connect#troubleshooting) section in the `nginx-openid-connect` repository on GitHub. ### Revision History - Version 3 (May 2022) - Updates OneLogin's OpenId Connect API endpoints from version 1 to version 2 - Version 2 (March 2020) – Updates to _Configuring NGINX Plus_ section - Version 1 (July 2019) – Initial version (NGINX Plus Release 18) -------------------------------------------------------------------------------- title: "Single Sign-On with Ping Identity and njs" url: https://docs.nginx.com/nginx/deployment-guides/single-sign-on/oidc-njs/ping-identity/index.md description: Enable OpenID Connect-based single-sign for applications proxied by NGINX Plus, using Ping Identity as the identity provider (IdP). -------------------------------------------------------------------------------- **Note:** This guide applies to NGINX Plus [Release 15](nginx/releases.md#r15) and later, based on the [`nginx-openid-connect`](https://github.com/nginxinc/nginx-openid-connect) GitHub repo. Starting with NGINX Plus [Release 34](nginx/releases.md#r34), use the simpler solution with the [native OpenID connect module](https://nginx.org/en/docs/http/ngx_http_oidc_module.html). See [Single Sign-On With Ping Identity](nginx/deployment-guides/single-sign-on/ping-identity.md) for details. This guide explains how to enable single sign-on (SSO) for applications being proxied by F5 NGINX Plus. The solution uses OpenID Connect as the authentication mechanism, with Ping Identity as the identity provider (IdP) and NGINX Plus as the relying party. The instructions in this document apply to both Ping Identity's on‑premises and cloud products, PingFederate and PingOne for Enterprise. **Note:** You can find more information about the NGINX Plus OpenID Connect integration in the project's [GitHub repo](https://github.com/nginxinc/nginx-openid-connect#nginx-openid-connect). ## Prerequisites The instructions assume you have the following: - A running deployment of PingFederate or PingOne for Enterprise, and a Ping Identity account. For installation and configuration instructions, see the documentation for [PingFederate](https://docs.pingidentity.com/bundle/pingfederate-93/page/tau1564002955783.html) or [PingOne for Enterprise](https://docs.pingidentity.com/bundle/pingone/page/fjn1564020491958-1.html). - An NGINX Plus subscription and NGINX Plus R15 or later. For installation instructions, see the [NGINX Plus Admin Guide](https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-plus/). - The [NGINX JavaScript module](https://nginx.org/en/docs/njs/index.html) (njs), required for handling the interaction between NGINX Plus and the IdP. After installing NGINX Plus, install the module with the command for your operating system. For Debian and Ubuntu: ```none sudo apt install nginx-plus-module-njs ``` For CentOS, RHEL, and Oracle Linux: ```shell sudo yum install nginx-plus-module-njs ``` - The following directive included in the top-level ("main") configuration context in **/etc/nginx/nginx.conf**, to load the NGINX JavaScript module: ```nginx load_module modules/ngx_http_js_module.so; ``` ## Configuring PingFederate or PingOne for Enterprise **Note:** This guide uses the GUI provided with PingOne for Enterprise. It reflects the GUI at the time of initial publication, but the GUI is subject to change. The PingFederate user interface might also differ. Use this guide as a reference and adapt as necessary for the UI you are using. Create a new application for NGINX Plus: 1. Log in to your Ping Identity account. The administrative dashboard opens automatically. In this guide, we show the PingOne for Enterprise dashboard, and for brevity refer simply to ”PingOne”. 2. Click  APPLICATIONS  in the title bar, and on the **My Applications** page that opens, click **OIDC** and then the **+ Add Application** button. 3. The **Add OIDC Application** window pops up. Click the ADVANCED CONFIGURATION box, and then the  Next  button. 4. In section 1 (PROVIDE DETAILS ABOUT YOUR APPLICATION), type a name in the **APPLICATION NAME** field and a short description in the **SHORT DESCRIPTION** field. Here, we're using **nginx-plus-application** and **NGINX Plus**. Choose a value from the **CATEGORY** drop‑down menu; here we’re using **Information Technology**. You can also add an icon if you wish. Click the  Next  button. 5. In section 2 (AUTHORIZATION SETTINGS), perform these steps: 1. Under **GRANTS**, click both **Authorization Code** and **Implicit**.
2. Under **CREDENTIALS**, click the **+ Add Secret** button. PingOne creates a client secret and opens the **CLIENT SECRETS** field to display it, as shown in the screenshot. To see the actual value of the secret, click the eye icon.
3. Click the  Next  button. 6. In section 3 (SSO FLOW AND AUTHENTICATION SETTINGS): 1. In the **START SSO URL** field, type the URL where users access your application. Here we’re using **https://example.com**. 2. In the **REDIRECT URIS** field, type the URI of the NGINX Plus instance including the port number, and ending in **/\_codexch**. Here we’re using **https://my-nginx-plus.example.com:443/\_codexch** (the full value is not visible in the screenshot). **Notes:** - For production, we strongly recommend that you use SSL/TLS (port 443). - The port number is mandatory even when you're using the default port for HTTP (80) or HTTPS (443). 7. In section 4 (DEFAULT USER PROFILE ATTRIBUTE CONTRACT), optionally add attributes to the required **sub** and **idpid** attributes, by clicking the **+ Add Attribute** button. We’re not adding any in this example. When finished, click the  Next  button. 8. In section 5 (CONNECT SCOPES), click the circled plus-sign on the **OpenID Profile (profile)** and **OpenID Profile Email (email)** scopes in the **LIST OF SCOPES** column. They are moved to the **CONNECTED SCOPES** column, as shown in the screenshot. Click the  Next  button. 9. In section 6 (ATTRIBUTE MAPPING), map attributes from your identity repository to the claims available to the application. The one attribute you must map is **sub**, and here we have selected the value **Email** from the drop‑down menu (the screenshot is abridged for brevity). 10. In section 7 (GROUP ACCESS), select the groups that will have access to the application, by clicking the circled plus-sign on the corresponding boxes in the **AVAILABLE GROUPS** column. The boxes move to the **ADDED GROUPS** column. As shown in the screenshot we have selected the two default groups, **Domain Administrators@directory** and **Users@directory**. Click the  Done  button. 11. You are returned to the **My Applications** window, which now includes a row for **nginx-plus-application**. Click the toggle switch at the right end of the row to the “on” position, as shown in the screenshot. Then click the “expand” icon at the end of the row, to display the application’s details. 12. On the page that opens, make note of the values in the following fields on the **Details** tab. You will add them to the NGINX Plus configuration in [Step 4 of _Configuring NGINX Plus_](#nginx-plus-variables). - **CLIENT ID** (in the screenshot, **28823604-83c5-4608-88da-c73fff9c607a**) - **CLIENT SECRETS** (in the screenshot, **7GMKILBofxb...**); click on the eye icon to view the actual value ## Configuring NGINX Plus Configure NGINX Plus as the OpenID Connect relying party: 1. Create a clone of the [**nginx-openid-connect**](https://github.com/nginxinc/nginx-openid-connect) GitHub repository. ```shell git clone https://github.com/nginxinc/nginx-openid-connect ``` 2. Copy these files from the clone to **/etc/nginx/conf.d**: - **frontend.conf** - **openid\_connect.js** - **openid\_connect.server\_conf** 3. Get the URLs for the authorization endpoint, token endpoint, and JSON Web Key (JWK) file from the Ping Identity configuration. Run the following `curl` command in a terminal, piping the output to the indicated `python` command to output the entire configuration in an easily readable format. We've abridged the output to show only the relevant fields. The `` variable is the value in the **CLIENT ID** field that you noted in [Step 12 of _Configuring PingFederate or PingOne for Enterprise_](#ping-client-id-secrets). **Note:** This `curl` command is appropriate for Ping One for Enterprise. For PingFederate, you might need to replace `sso.connect.pingidentity.com` with the IP address of your local PingFederate server. ```shell $ curl sso.connect.pingidentity.com//.well-known/openid-configuration | python -m json.tool ... { "authorization_endpoint": "https://sso.connect.pingidentity.com/sso/as/authorization.oauth2", ... "jwks_uri": "https://sso.connect.pingidentity.com/sso/as/jwks", ... "token_endpoint": "https://sso.connect.pingidentity.com/sso/as/token.oauth2", ... } ``` 4. In your preferred text editor, open **/etc/nginx/conf.d/frontend.conf**. Change the second parameter of each of the following [set](http://nginx.org/en/docs/http/ngx_http_rewrite_module.html#set) directives to the specified value: - `set $oidc_authz_endpoint` – Value of `authorization_endpoint` from [Step 3](#nginx-plus-urls) (in this guide, `https://sso.connect.pingidentity.com/sso/as/authorization.oauth2`) - `set $oidc_token_endpoint` – Value of `token_endpoint` from [Step 3](#nginx-plus-urls) (in this guide, `https://sso.connect.pingidentity.com/sso/as/token.oauth2`) - `set $oidc_client` – Value in the **CLIENT ID** field in [Step 12 of _Configuring PingFederate or PingOne for Enterprise_](#ping-client-id-secrets) (in this guide, `28823604-83c5-4608-88da-c73fff9c607a`) - `set $oidc_client_secret` – Value in the **CLIENT SECRETS** field in [Step 12 of _Configuring PingFederate or PingOne for Enterprise_](#ping-client-id-secrets) (in this guide, `7GMKILBofxb...`) - `set $oidc_hmac_key` – A unique, long, and secure phrase 5. Configure the JWK file. The procedure depends on which version of NGINX Plus you are using. - In NGINX Plus R17 and later, NGINX Plus can read the JWK file directly from the URL reported as `jwks_uri` in [Step 3](#nginx-plus-urls). Change **/etc/nginx/conf.d/frontend.conf** as follows: 1. Comment out (or remove) the [auth_jwt_key_file](http://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_file) directive. 2. Uncomment the [auth_jwt_key_request](http://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_request) directive. (Its parameter, `/_jwks_uri`, refers to the value of the `$oidc_jwt_keyfile` variable, which you set in the next step.) 3. Change the second parameter of the `set $oidc_jwt_keyfile` directive to the value reported in the `jwks_uri` field in [Step 3](#nginx-plus-urls) (in this guide, `https://sso.connect.pingidentity.com/sso/as/jwks`). - In NGINX Plus R16 and earlier, the JWK file must be on the local disk. (You can also use this method with NGINX Plus R17 and later if you wish.) 1. Copy the JSON contents from the JWK file named in the `jwks_uri` field in [Step 3](#nginx-plus-urls) (in this guide, `https://sso.connect.pingidentity.com/sso/as/jwks`) to a local file (for example, `/etc/nginx/my_ping_identity_jwk.json`). 2. In **/etc/nginx/conf.d/frontend.conf**, change the second parameter of the `set $oidc_jwt_keyfile` directive to the local file path. 6. Confirm that the user named by the [user](http://nginx.org/en/docs/ngx_core_module.html#user) directive in the NGINX Plus configuration (in **/etc/nginx/nginx.conf** by convention) has read permission on the JWK file. ## Testing In a browser, enter the address of your NGINX Plus instance and try to log in using the credentials of a user assigned to the application (see [Step 10 of _PingFederate or PingOne for Enterprise_](#ping-group-access)). ## Troubleshooting See the [**Troubleshooting**](https://github.com/nginxinc/nginx-openid-connect#troubleshooting) section at the **nginx-openid-connect** repository on GitHub. ### Revision History - Version 2 (March 2020) – Updates to _Configuring NGINX Plus_ section - Version 1 (January 2020) – Initial version (NGINX Plus Release 20) -------------------------------------------------------------------------------- title: "SPNEGO" url: https://docs.nginx.com/nginx/admin-guide/dynamic-modules/spnego/index.md description: This module adds [SPNEGO](https://tools.ietf.org/html/rfc4178) support to F5 NGINX Plus. Currently, only Kerberos authentication via GSSAPI is supported. -------------------------------------------------------------------------------- The SPNEGO Auth module adds [SPNEGO](https://tools.ietf.org/html/rfc4178) support to F5 NGINX Plus. Currently, only Kerberos authentication via GSSAPI is supported. ## Installation 1. Check the [Technical Specifications](/nginx/technical-specs.md#dynamic-modules) page to verify that the module is supported by your operating system. 2. Make sure that your operating system is configured to retrieve binary packages from the official NGINX Plus repository. See installation instructions for your operating system on the [Installing NGINX Plus](/nginx/admin-guide/installing-nginx/installing-nginx-plus.md) page. 3. Install the SPNEGO Auth module package `nginx-plus-module-auth-spnego` from the official NGINX Plus repository. For Amazon Linux 2, CentOS, Oracle Linux, and RHEL: ```shell sudo yum update && \ sudo yum install nginx-plus-module-auth-spnego ``` For Amazon Linux 2023, AlmaLinux, Rocky Linux: ```shell sudo dnf update && \ sudo dnf install nginx-plus-module-auth-spnego ``` For Debian and Ubuntu:: ```shell sudo apt update && \ sudo apt install nginx-plus-module-auth-spnego ``` For SLES: ```shell sudo zypper refresh && \ sudo zypper install nginx-plus-module-auth-spnego ``` For Alpine: ```shell apk add nginx-plus-module-auth-spnego ``` For FreeBSD: ```shell sudo pkg update && \ sudo pkg install nginx-plus-module-auth-spnego ``` ## Configuration After installation you will need to enable and configure the module in F5 NGINX Plus configuration file `nginx.conf`. 1. Enable dynamic loading of the module with the [`load_module`](https://nginx.org/en/docs/ngx_core_module.html#load_module) directive specified in the top-level (“`main`”) context: ```nginx load_module modules/spnego-http-auth-nginx-module.so; http { # ... } ``` 2. Perform additional configuration as required by the [module](https://github.com/stnoonan/spnego-http-auth-nginx-module). 3. Test the NGINX Plus configuration. In a terminal, type-in the command: ```shell nginx -t ``` Expected output of the command: ```shell nginx: the configuration file /etc/nginx/nginx.conf syntax is ok nginx: configuration file /etc/nginx/nginx.conf is successful ``` 4. Reload the NGINX Plus configuration to enable the module: ```shell nginx -s reload ``` ## More info - [NGINX module for HTTP SPNEGO auth GitHub poject](https://github.com/stnoonan/spnego-http-auth-nginx-module) - [NGINX dynamic modules](dynamic-modules.md) - [NGINX Plus technical specifications](/nginx/technical-specs.md) - [Uninstalling a dynamic module](uninstall.md) -------------------------------------------------------------------------------- title: "Uninstalling a dynamic module" url: https://docs.nginx.com/nginx/admin-guide/dynamic-modules/uninstall/index.md description: This section describes how to uninstall an F5 NGINX Plus dynamic module. -------------------------------------------------------------------------------- You may need to uninstall a dynamic module in NGINX Plus in several scenarios: - The module is being replaced with a newer or alternative version. For example, the [OpenTracing](opentracing.md) module has been replaced by the [OpenTelemetry](opentelemetry.md) module. The [Cookie-Flag](cookie-flag.md) dynamic module has been superseded by natively supported [`proxy_cookie_flags`](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cookie_flags) directive. - The module is no longer supported, for example, the [NGINX ModSecurity WAF](nginx-waf.md) module. - The operating system is not supported by the module. See the the [Dynamic Modules](/nginx/technical-specs.md#dynamic-modules) section of the [NGINX Plus Technical Specifications](/nginx/technical-specs.md#dynamic-modules). - The module it is no longer required, uninstalling it helps optimize resource usage and reduce maintenance overhead. - The module is causing runtime errors or configuration conflicts that affect stability or performance. - Security or compliance policies require removal, particularly in regulated environments where only approved or certified modules are allowed. ## Instructions To uninstall a dynamic module, run the appropriate command for your operating system in a terminal, replacing `` with the actual package name, for example, `nginx-plus-module-cookie-flag`. - For Amazon Linux 2, CentOS, Oracle Linux, and RHEL: ```shell sudo yum remove ``` - For Amazon Linux 2023, AlmaLinux, Rocky Linux: ```shell sudo dnf remove ``` - For Debian and Ubuntu: ```shell sudo apt remove ``` - For SLES: ```shell sudo zypper remove ``` - For FreeBSD: ```shell sudo pkg delete ``` ## Configuration After uninstalling the package, you will need to disable the module in the NGINX configuration file. 1. In a text editor, open the NGINX Plus configuration file: - `/etc/nginx/nginx.conf` for Linux - `/usr/local/etc/nginx/nginx.conf` for FreeBSD 2. Disable dynamic loading of the module by removing the corresponding [`load_module`](https://nginx.org/en/docs/ngx_core_module.html#load_module) directive. 3. Remove all directives related to the dynamic module. 4. Save the configuration. 5. Test the NGINX Plus configuration. In a terminal, type-in the command: ```shell nginx -t ``` Expected output of the command: ```shell nginx: the configuration file /etc/nginx/nginx.conf syntax is ok nginx: configuration file /etc/nginx/nginx.conf is successful ``` 5. Reload the NGINX Plus configuration: ```shell nginx -s reload ``` ## More info - [NGINX modules reference](https://nginx.org/en/docs/) - [NGINX dynamic modules](dynamic-modules.md) - [NGINX Plus technical specifications](/nginx/technical-specs.md) -------------------------------------------------------------------------------- title: "XSLT" url: https://docs.nginx.com/nginx/admin-guide/dynamic-modules/xslt/index.md description: Modify XML code in response bodies using one or more XSLT stylesheets, with the XSLT dynamic module, supported by NGINX, Inc. -------------------------------------------------------------------------------- With the XSLT dynamic module, you can modify XML code in response bodies using one or more XSLT stylesheets. ## Installation 1. Check the [Technical Specifications](/nginx/technical-specs.md#dynamic-modules) page to verify that the module is supported by your operating system. 2. Make sure that your operating system is configured to retrieve binary packages from the official NGINX Plus repository. See installation instructions for your operating system on the [Installing NGINX Plus](/nginx/admin-guide/installing-nginx/installing-nginx-plus.md) page. 3. Install the XSLT module package `nginx-plus-module-xslt` from the official NGINX Plus repository. For Amazon Linux 2, CentOS, Oracle Linux, and RHEL: ```shell sudo yum update && \ sudo yum install nginx-plus-module-xslt ``` For Amazon Linux 2023, AlmaLinux, Rocky Linux: ```shell sudo dnf update && \ sudo dnf install nginx-plus-module-xslt ``` For Debian and Ubuntu: ```shell sudo apt update && \ sudo apt install nginx-plus-module-xslt ``` For SLES: ```shell sudo zypper refresh && \ sudo zypper install nginx-plus-module-xslt ``` For Alpine: ```shell apk add nginx-plus-module-xslt ``` For FreeBSD: ```shell sudo pkg update && \ sudo pkg install nginx-plus-module-xslt ``` ## Configuration After installation you will need to enable and configure the module in F5 NGINX Plus configuration file `nginx.conf`. 1. Enable dynamic loading of the module with the [`load_module`](https://nginx.org/en/docs/ngx_core_module.html#load_module) directive specified in the top-level (“`main`”) context: ```nginx load_module modules/ngx_http_xslt_module.so; http { # ... } ``` 2. Perform additional configuration as required by the [module](https://nginx.org/en/docs/http/ngx_http_xslt_module.html). 3. Test the NGINX Plus configuration. In a terminal, type-in the command: ```shell nginx -t ``` Expected output of the command: ```shell nginx: the configuration file /etc/nginx/nginx.conf syntax is ok nginx: configuration file /etc/nginx/nginx.conf is successful ``` 4. Reload the NGINX Plus configuration to enable the module: ```shell nginx -s reload ``` ## More info - [NGINX `ngx_http_xslt_module` module reference](https://nginx.org/en/docs/http/ngx_http_xslt_module.html) - [NGINX dynamic modules](dynamic-modules.md) - [NGINX Plus technical specifications](/nginx/technical-specs.md) - [Uninstalling a dynamic module](uninstall.md) -------------------------------------------------------------------------------- title: "Installing NGINX Plus LTS" url: https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-plus-lts/index.md description: Install F5 NGINX Plus LTS with step-by-step instructions for the base package on all supported distributions. -------------------------------------------------------------------------------- Since May 13, 2026, NGINX Plus is published in two release types: Long-Term Support (LTS) and Continuous Release (CR). The NGINX Plus LTS release model is designed for mission-critical production environments. Each LTS release is supported for three years and receives security fixes and CVE mitigations without introducing new features. New features are delivered in CRs during the same annual LTS cycle. - **Cadence**: one LTS per year - **Patching model**: security/CVE fixes only, no feature changes - **Support window**: up to 3 years for each LTS release - **Concurrency**: up to 3 LTS versions supported at the same time - **Version format**: an LTS release has `0` as the second numeric component, for example: `PLS.37.0.0.1`. LTS updates increment the third component, for example: `PLS.37.0.1.1`. NGINX Plus CRs are published several times within an annual LTS cycle. Each CR contains the latest features and performance improvements. The CR cycle ends when a new LTS is released. ## Repository configuration options {#repo-options} By default, NGINX Plus repositories are configured to receive [Continuous Releases](/nginx/admin-guide/installing-nginx/installing-nginx-plus.md). To use LTS, update your repository configuration to point to the LTS package URL, replacing the default URL. You can choose one of the options during installation: - **Pin to current LTS only**: receive only security updates for this LTS, no CRs, no upgrade to next LTS; supported up to three years. Follow the steps for your operating system in this guide. - **Pin to LTS track**: upgrade to the newest LTS when it is released annually, no CRs. Follow the steps for your operating system in this guide. - **Default**: receive Continuous Releases within the current LTS release, upgrade to each new LTS when it is released annually and then receive its CRs. See [Installing NGINX Plus](/nginx/admin-guide/installing-nginx/installing-nginx-plus.md). ## Prerequisites {#prereq} Before you begin, make sure you have: - [MyF5 Customer Portal](https://account.f5.com/myf5) access, credentials are provided in the email from F5, Inc. - An active NGINX Plus subscription. Details can be verified on the [MyF5 Customer Portal](https://account.f5.com/myf5). - A [supported operating system and architecture](nginx/technical-specs.md). - Administrative privileges: `root` access or `sudo`, or see [Unprivileged installation](/nginx/admin-guide/installing-nginx/installing-nginx-plus.md#unpriv_install). - Internet access, or see [Offline installation](/nginx/admin-guide/installing-nginx/installing-nginx-plus.md#offline_install). ## Preparation steps for all operating systems {#common-steps} 1. Check if your operating system and architecture are supported. For a complete list of supported platforms and architectures, see the [Technical Specifications](nginx/technical-specs.md). 1. Back up your NGINX Plus configuration and log files if you have an older NGINX Plus package installed. For more information, see [Upgrading NGINX Plus](nginx/admin-guide/installing-nginx/upgrading-nginx-plus.md). 1. Download the SSL certificate, private key, and the JWT license file associated with your NGINX Plus subscription from the MyF5 Customer Portal: - Log in to [MyF5](https://my.f5.com/manage/s/). - Go to **My Products & Plans > Subscriptions** to see your active subscriptions. - Find your NGINX products or services subscription, and select the **Subscription ID** for details. - Download the **.crt** and **.key** from the subscription page. - Download the **JSON Web Token** (JWT) from the subscription page. See [About Subscription Licenses](/solutions/about-subscription-licenses.md) for details. 1. Create the **/etc/ssl/nginx** directory: ```shell sudo mkdir -p /etc/ssl/nginx ``` 1. Copy the downloaded **.crt** and **.key** files to the **/etc/ssl/nginx/** directory and make sure they are named **nginx-repo.crt** and **nginx-repo.key**: ```shell sudo cp .crt /etc/ssl/nginx/nginx-repo.crt sudo cp .key /etc/ssl/nginx/nginx-repo.key ``` 1. Follow the installation instructions for your operating system: [Amazon Linux 2023](#install_amazon2023), [Amazon Linux 2](#install_amazon2), [RHEL-based](#install_rhel), [Debian](#install_debian), [Ubuntu](#install_debian_ubuntu), [FreeBSD](#install_freebsd), [SLES](#install_suse), [Alpine](#install_alpine). ## Amazon Linux 2023 LTS packages {#install_amazon2023} 1. Make sure you have met all [prerequisites](#prereq) and completed the [common steps for all operating systems](#common-steps). 1. Install the **ca-certificates** dependency: ```shell sudo dnf update && \ sudo dnf install ca-certificates ``` 1. Add the NGINX Plus repository to your Amazon Linux 2023 instance. Download the [plus-amazonlinux2023.repo](https://cs.nginx.com/static/files/plus-amazonlinux2023.repo) file to **/etc/yum.repos.d**: ```shell sudo wget -P /etc/yum.repos.d https://cs.nginx.com/static/files/plus-amazonlinux2023.repo ``` 1. **Modify your NGINX Plus repository configuration to pin to the desired LTS track**. To change your update channel, edit the `/etc/yum.repos.d/plus-amazonlinux2023.repo` file and update the `baseurl` to the [appropriate value](#repo-options) for your target version. - Pin to current LTS version: ```none baseurl=https://pkgs.nginx.com/plus/R37.0/amzn/2023/$basearch ``` - Pin to LTS track: ```none baseurl=https://pkgs.nginx.com/plus/LTS/amzn/2023/$basearch ``` 1. Install the **nginx-plus** package. Any older NGINX Plus package is automatically replaced. ```shell sudo dnf install nginx-plus ``` 1. Copy the downloaded JWT file to the **/etc/nginx/** directory and make sure it is named **license.jwt**: ```shell sudo cp .jwt /etc/nginx/license.jwt ``` 1. Check the `nginx` version to verify that NGINX Plus LTS is installed correctly: ```shell nginx -v ``` The command output should indicate an LTS release: the second numeric component of the Plus release version should be `0`: ```none nginx version: nginx/1.29.8 (nginx-plus-r37.0.6) ``` 1. Make sure license reporting to F5 licensing endpoint is configured. By default, no configuration is required. However, it becomes necessary when NGINX Plus is installed in a disconnected environment, uses NGINX Instance Manager for usage reporting, or uses a custom path for the license file. Configuration can be done in the [`mgmt {}`](https://nginx.org/en/docs/ngx_mgmt_module.html) block of the NGINX Plus configuration file (`/etc/nginx/nginx.conf`). For more information, see [About Subscription Licenses](/solutions/about-subscription-licenses.md). 1. If you are using [NGINX Instance Manager](/nim/) in your infrastructure, install and enable [NGINX Agent](https://docs.nginx.com/nginx-agent/overview/). See [Install and Configure NGINX Agent](https://github.com/nginx/agent/blob/main/README.md) for details. ## Amazon Linux 2 LTS packages {#install_amazon2} 1. Make sure you have met all [prerequisites](#prereq) and completed the [common steps for all operating systems](#common-steps). 1. Install the **ca-certificates** dependency: ```shell sudo yum update && \ sudo yum install ca-certificates ``` 1. Add the NGINX Plus repository to your Amazon Linux 2 instance. Download the [nginx-plus-amazon2.repo](https://cs.nginx.com/static/files/nginx-plus-amazon2.repo) file to **/etc/yum.repos.d**: ```shell sudo wget -P /etc/yum.repos.d https://cs.nginx.com/static/files/nginx-plus-amazon2.repo ``` 1. **Modify your NGINX Plus repository configuration to pin to the desired LTS track**. To change your update channel, edit the `/etc/yum.repos.d/nginx-plus-amazon2` file and update the `baseurl` to the [appropriate value](#repo-options) for your target version. - Pin to current LTS version: ```none baseurl=https://pkgs.nginx.com/plus/R37.0/amzn2/$releasever/$basearch ``` - Pin to LTS track: ```none baseurl=https://pkgs.nginx.com/plus/LTS/amzn2/$releasever/$basearch ``` 1. Install the **nginx-plus** package. Any older NGINX Plus package is automatically replaced. ```shell sudo yum install nginx-plus ``` 1. Copy the downloaded JWT file to the **/etc/nginx/** directory and make sure it is named **license.jwt**: ```shell sudo cp .jwt /etc/nginx/license.jwt ``` 1. Check the `nginx` version to verify that NGINX Plus LTS is installed correctly: ```shell nginx -v ``` The command output should indicate an LTS release: the second numeric component of the Plus release version should be `0`: ```none nginx version: nginx/1.29.8 (nginx-plus-r37.0.6) ``` 1. Make sure license reporting to F5 licensing endpoint is configured. By default, no configuration is required. However, it becomes necessary when NGINX Plus is installed in a disconnected environment, uses NGINX Instance Manager for usage reporting, or uses a custom path for the license file. Configuration can be done in the [`mgmt {}`](https://nginx.org/en/docs/ngx_mgmt_module.html) block of the NGINX Plus configuration file (`/etc/nginx/nginx.conf`). For more information, see [About Subscription Licenses](/solutions/about-subscription-licenses.md). 1. If you are using [NGINX Instance Manager](/nim/) in your infrastructure, install and enable [NGINX Agent](https://docs.nginx.com/nginx-agent/overview/). See [Install and Configure NGINX Agent](https://github.com/nginx/agent/blob/main/README.md) for details. ## RHEL-based systems LTS packages {#install_rhel} Supported RHEL-based operating systems include Red Hat Enterprise Linux, Oracle Linux, AlmaLinux, and Rocky Linux for versions 8.1+, 9.7+, and 10+. 1. Make sure you meet the [prerequisites](#prereq) and have completed the [common steps for all operating systems](#common-steps). 1. Install the **ca-certificates** dependency: ```shell sudo dnf update && \ sudo dnf install ca-certificates ``` 1. Add the NGINX Plus repository by downloading **nginx-plus-.repo** file that matches your OS major version to **/etc/yum.repos.d**. - For **RHEL-based 8.1+**, download the [plus-8.repo](https://cs.nginx.com/static/files/plus-8.repo) file: ```shell sudo wget -P /etc/yum.repos.d https://cs.nginx.com/static/files/plus-8.repo ``` - For **RHEL-based 9.7+**, download the [plus-9.repo](https://cs.nginx.com/static/files/plus-9.repo) file: ```shell sudo wget -P /etc/yum.repos.d https://cs.nginx.com/static/files/plus-9.repo ``` - For **RHEL-based 10+**, download the [plus-10.repo](https://cs.nginx.com/static/files/plus-10.repo) file: ```shell sudo wget -P /etc/yum.repos.d https://cs.nginx.com/static/files/plus-10.repo ``` 1. **Modify your NGINX Plus repository configuration to pin to the desired LTS track**. To change your update channel, edit the `/etc/yum.repos.d/plus-.repo` file and update the `baseurl` to the [appropriate value](#repo-options) for your target version.

For **RHEL-based 8.1+** - Pin to current LTS version: ```none baseurl=https://pkgs.nginx.com/plus/R37.0/centos/8/$basearch/ ``` - Pin to LTS track: ```none baseurl=https://pkgs.nginx.com/plus/LTS/centos/8/$basearch/ ``` For **RHEL-based 9.7+** - Pin to current LTS version: ```none baseurl=https://pkgs.nginx.com/plus/R37.0/centos/9/$basearch/ ``` - Pin to LTS track: ```none baseurl=https://pkgs.nginx.com/plus/LTS/centos/9/$basearch/ ``` For **RHEL-based 10+** - Pin to current LTS version: ```none baseurl=https://pkgs.nginx.com/plus/R37.0/centos/10/$basearch/ ``` - Pin to LTS track: ```none baseurl=https://pkgs.nginx.com/plus/LTS/centos/10/$basearch/ ``` - Save the changes and exit. - Update the repository information: ```shell sudo dnf update ``` 1. Install the **nginx-plus** package. Any older NGINX Plus package is automatically replaced. ```shell sudo dnf install nginx-plus ``` 1. Copy the downloaded JWT file to the **/etc/nginx/** directory and make sure it is named **license.jwt**: ```shell sudo cp .jwt /etc/nginx/license.jwt ``` 1. To enable the NGINX service to start at boot, run the following command: ```shell sudo systemctl enable nginx.service ``` 1. Check the `nginx` version to verify that NGINX Plus LTS is installed correctly: ```shell nginx -v ``` The command output should indicate an LTS release: the second numeric component of the Plus release version should be `0`: ```none nginx version: nginx/1.29.8 (nginx-plus-r37.0.6) ``` 1. Make sure license reporting to F5 licensing endpoint is configured. By default, no configuration is required. However, it becomes necessary when NGINX Plus is installed in a disconnected environment, uses NGINX Instance Manager for usage reporting, or uses a custom path for the license file. Configuration can be done in the [`mgmt {}`](https://nginx.org/en/docs/ngx_mgmt_module.html) block of the NGINX Plus configuration file (`/etc/nginx/nginx.conf`). For more information, see [About Subscription Licenses](/solutions/about-subscription-licenses.md). 1. If you are using [NGINX Instance Manager](/nim/) in your infrastructure, install and enable [NGINX Agent](https://docs.nginx.com/nginx-agent/overview/). See [Install and Configure NGINX Agent](https://github.com/nginx/agent/blob/main/README.md) for details. ## Debian LTS packages {#install_debian} 1. Make sure you have met all [prerequisites](#prereq) and completed the [common steps for all operating systems](#common-steps). 1. Install the prerequisites packages: ```shell sudo apt update && \ sudo apt install apt-transport-https \ lsb-release \ ca-certificates \ wget \ gnupg2 \ debian-archive-keyring ``` 1. Download and add NGINX signing key: ```shell wget -qO - https://cs.nginx.com/static/keys/nginx_signing.key \ | gpg --dearmor \ | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null ``` 1. Add the NGINX Plus repository: ```shell printf "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] \ https://pkgs.nginx.com/plus/debian `lsb_release -cs` nginx-plus\n" \ | sudo tee /etc/apt/sources.list.d/nginx-plus.list ``` 1. Download the **nginx-plus** apt configuration to **/etc/apt/apt.conf.d**: ```shell sudo wget -P /etc/apt/apt.conf.d https://cs.nginx.com/static/files/90pkgs-nginx ``` 1. **Modify your NGINX Plus repository configuration to pin to the desired LTS track**. To change your update channel, edit the `/etc/apt/sources.list.d/nginx-plus.list` file and update the URL to the [appropriate value](#repo-options) for your target version. - Pin to current LTS version: ```none https://pkgs.nginx.com/plus/R37.0/debian ``` - Pin to LTS track: ```none https://pkgs.nginx.com/plus/LTS/debian ``` 1. Update the repository information: ```shell sudo apt update ``` 1. Install the **nginx-plus** package. Any older NGINX Plus package is automatically replaced. ```shell sudo apt install -y nginx-plus ``` 1. Copy the downloaded JWT file to the **/etc/nginx/** directory and make sure it is named **license.jwt**: ```shell sudo cp .jwt /etc/nginx/license.jwt ``` 1. Check the `nginx` version to verify that NGINX Plus LTS is installed correctly: ```shell nginx -v ``` The command output should indicate an LTS release: the second numeric component of the Plus release version should be `0`: ```none nginx version: nginx/1.29.8 (nginx-plus-r37.0.6) ``` 1. Make sure license reporting to F5 licensing endpoint is configured. By default, no configuration is required. However, it becomes necessary when NGINX Plus is installed in a disconnected environment, uses NGINX Instance Manager for usage reporting, or uses a custom path for the license file. Configuration can be done in the [`mgmt {}`](https://nginx.org/en/docs/ngx_mgmt_module.html) block of the NGINX Plus configuration file (`/etc/nginx/nginx.conf`). For more information, see [About Subscription Licenses](/solutions/about-subscription-licenses.md). 1. If you are using [NGINX Instance Manager](/nim/) in your infrastructure, install and enable [NGINX Agent](https://docs.nginx.com/nginx-agent/overview/). See [Install and Configure NGINX Agent](https://github.com/nginx/agent/blob/main/README.md) for details. ## Ubuntu LTS packages {#install_debian_ubuntu} 1. Make sure you have met all [prerequisites](#prereq) and completed the [common steps for all operating systems](#common-steps). 1. Install the prerequisites packages: ```shell sudo apt update && \ sudo apt install apt-transport-https \ lsb-release \ ca-certificates \ wget \ gnupg2 \ ubuntu-keyring ``` 1. Download and add NGINX signing key: ```shell wget -qO - https://cs.nginx.com/static/keys/nginx_signing.key \ | gpg --dearmor \ | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null ``` 1. Add the NGINX Plus repository: ```shell printf "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] \ https://pkgs.nginx.com/plus/ubuntu `lsb_release -cs` nginx-plus\n" \ | sudo tee /etc/apt/sources.list.d/nginx-plus.list ``` 1. Download the **nginx-plus** apt configuration to **/etc/apt/apt.conf.d**: ```shell sudo wget -P /etc/apt/apt.conf.d https://cs.nginx.com/static/files/90pkgs-nginx ``` 1. **Modify your NGINX Plus repository configuration to pin to the desired LTS track**. To change your update channel, edit the `/etc/apt/sources.list.d/nginx-plus.list` file and update the URL to the [appropriate value](#repo-options) for your target version. - Pin to current LTS version: ```none https://pkgs.nginx.com/plus/R37.0/ubuntu ``` - Pin to LTS track: ```none https://pkgs.nginx.com/plus/LTS/ubuntu ``` 1. Update the repository information: ```shell sudo apt update ``` 1. Install the **nginx-plus** package. Any older NGINX Plus package is automatically replaced. ```shell sudo apt install -y nginx-plus ``` 1. Copy the downloaded JWT file to the **/etc/nginx/** directory and make sure it is named **license.jwt**: ```shell sudo cp .jwt /etc/nginx/license.jwt ``` 1. Check the `nginx` version to verify that NGINX Plus LTS is installed correctly: ```shell nginx -v ``` The command output should indicate an LTS release: the second numeric component of the Plus release version should be `0`: ```none nginx version: nginx/1.29.8 (nginx-plus-r37.0.6) ``` 1. Make sure license reporting to F5 licensing endpoint is configured. By default, no configuration is required. However, it becomes necessary when NGINX Plus is installed in a disconnected environment, uses NGINX Instance Manager for usage reporting, or uses a custom path for the license file. Configuration can be done in the [`mgmt {}`](https://nginx.org/en/docs/ngx_mgmt_module.html) block of the NGINX Plus configuration file (`/etc/nginx/nginx.conf`). For more information, see [About Subscription Licenses](/solutions/about-subscription-licenses.md). 1. If you are using [NGINX Instance Manager](/nim/) in your infrastructure, install and enable [NGINX Agent](https://docs.nginx.com/nginx-agent/overview/). See [Install and Configure NGINX Agent](https://github.com/nginx/agent/blob/main/README.md) for details. ## FreeBSD LTS packages {#install_freebsd} 1. Make sure you have met all [prerequisites](#prereq) and completed the [common steps for all operating systems](#common-steps). 1. Install the prerequisite **ca_root_nss** package: ```shell sudo pkg update && \ sudo pkg install ca_root_nss ``` 1. Copy the [nginx-plus.conf](https://cs.nginx.com/static/files/nginx-plus.conf) file to the **/etc/pkg/** directory: ```shell sudo fetch -o /etc/pkg/nginx-plus.conf http://cs.nginx.com/static/files/nginx-plus.conf ``` 1. Add the following lines to the **/usr/local/etc/pkg.conf** file: ```none PKG_ENV: { SSL_NO_VERIFY_PEER: "1", SSL_CLIENT_CERT_FILE: "/etc/ssl/nginx/nginx-repo.crt", SSL_CLIENT_KEY_FILE: "/etc/ssl/nginx/nginx-repo.key" } ``` 1. **Modify your NGINX Plus repository configuration to pin to the desired LTS track**. To change your update channel, edit the `/etc/pkg/nginx-plus.conf` file and update the `URL` to the [appropriate value](#repo-options) for your target version. - Pin to current LTS version: ```none URL: pkg+https://pkgs.nginx.com/plus/R37.0/freebsd/${ABI}/latest ``` - Pin to LTS track: ```none URL: pkg+https://pkgs.nginx.com/plus/LTS/freebsd/${ABI}/latest ``` 1. Install the **nginx-plus** package. Any older NGINX Plus package is automatically replaced. Back up your NGINX Plus configuration and log files if you have an older NGINX Plus package installed. For more information, see [Upgrading NGINX Plus](#upgrade). ```shell sudo pkg install nginx-plus ``` 1. Copy the downloaded JWT file to the **/usr/local/etc/nginx** directory and make sure it is named **license.jwt**: ```shell sudo cp license.jwt /usr/local/etc/nginx ``` 1. Check the `nginx` version to verify that NGINX Plus LTS is installed correctly: ```shell nginx -v ``` The command output should indicate an LTS release: the second numeric component of the Plus release version should be `0`: ```none nginx version: nginx/1.29.8 (nginx-plus-r37.0.6) ``` 1. Make sure license reporting to F5 licensing endpoint is configured. By default, no configuration is required. However, it becomes necessary when NGINX Plus is installed in a disconnected environment, uses NGINX Instance Manager for usage reporting, or uses a custom path for the license file. Configuration can be done in the [`mgmt {}`](https://nginx.org/en/docs/ngx_mgmt_module.html) block of the NGINX Plus configuration file (`/usr/local/etc/nginx/nginx.conf`). For more information, see [About Subscription Licenses](https://docs.nginx.com/solutions/about-subscription-licenses/). 1. If you are using [NGINX Instance Manager](/nim/) in your infrastructure, install and enable [NGINX Agent](https://docs.nginx.com/nginx-agent/overview/). See [Install and Configure NGINX Agent](https://github.com/nginx/agent/blob/main/README.md) for details. ## SUSE Linux Enterprise LTS packages {#install_suse} 1. Make sure you have met all [prerequisites](#prereq) and completed the [common steps for all operating systems](#common-steps). 1. Create a file bundle of the certificate and key: ```shell cat /etc/ssl/nginx/nginx-repo.crt /etc/ssl/nginx/nginx-repo.key > /etc/ssl/nginx/nginx-repo-bundle.crt ``` 1. Install the required **ca-certificates** dependency: ```shell zypper refresh zypper install ca-certificates ``` 1. Add the **nginx-plus** repo. For **SLES 15**, pinned to current LTS version: ```shell zypper addrepo -G -t yum -c \ "https://pkgs.nginx.com/plus/R37.0/sles/15?ssl_clientcert=/etc/ssl/nginx/nginx-repo-bundle.crt&ssl_verify=peer" \ nginx-plus ``` For **SLES 15**, pinned to LTS track: ```shell zypper addrepo -G -t yum -c \ "https://pkgs.nginx.com/plus/LTS/sles/15?ssl_clientcert=/etc/ssl/nginx/nginx-repo-bundle.crt&ssl_verify=peer" \ nginx-plus ``` For **SLES 16**, pinned to current LTS version: ```shell zypper addrepo -G -t yum -c \ "https://pkgs.nginx.com/plus/R37.0/sles/16?ssl_clientcert=/etc/ssl/nginx/nginx-repo-bundle.crt&ssl_verify=peer" \ nginx-plus ``` For **SLES 16**, pinned to LTS track: ```shell zypper addrepo -G -t yum -c \ "https://pkgs.nginx.com/plus/LTS/sles/16?ssl_clientcert=/etc/ssl/nginx/nginx-repo-bundle.crt&ssl_verify=peer" \ nginx-plus ``` 1. Install the **nginx-plus** package. Any older NGINX Plus package is automatically replaced. ```shell zypper install nginx-plus ``` 1. Copy the downloaded JWT file to the **/etc/nginx/** directory and make sure it is named **license.jwt**: ```shell sudo cp .jwt /etc/nginx/license.jwt ``` 1. Check the `nginx` version to verify that NGINX Plus LTS is installed correctly: ```shell nginx -v ``` The command output should indicate an LTS release: the second numeric component of the Plus release version should be `0`: ```none nginx version: nginx/1.29.8 (nginx-plus-r37.0.6) ``` 1. Make sure license reporting to F5 licensing endpoint is configured. By default, no configuration is required. However, it becomes necessary when NGINX Plus is installed in a disconnected environment, uses NGINX Instance Manager for usage reporting, or uses a custom path for the license file. Configuration can be done in the [`mgmt {}`](https://nginx.org/en/docs/ngx_mgmt_module.html) block of the NGINX Plus configuration file (`/etc/nginx/nginx.conf`). For more information, see [About Subscription Licenses](/solutions/about-subscription-licenses.md). 1. If you are using [NGINX Instance Manager](/nim/) in your infrastructure, install and enable [NGINX Agent](https://docs.nginx.com/nginx-agent/overview/). See [Install and Configure NGINX Agent](https://github.com/nginx/agent/blob/main/README.md) for details. ## Alpine LTS packages {#install_alpine} 1. Make sure you have met all [prerequisites](#prereq) and completed the [common steps for all operating systems](#common-steps). 1. Upload **nginx-repo.key** to **/etc/apk/cert.key** and **nginx-repo.crt** to **/etc/apk/cert.pem**. Ensure these files contain only the specific key and certificate — Alpine Linux doesn't support mixing client certificates for multiple repositories. 1. Put the NGINX signing public key in the **/etc/apk/keys** directory: ```shell sudo wget -O /etc/apk/keys/nginx_signing.rsa.pub https://cs.nginx.com/static/keys/nginx_signing.rsa.pub ``` 1. Add the NGINX repository to the **/etc/apk/repositories** file. - Pin to current LTS version: ```shell printf "https://pkgs.nginx.com/plus/R37.0/alpine/v`egrep -o '^[0-9]+\.[0-9]+' /etc/alpine-release`/main\n" \ | sudo tee -a /etc/apk/repositories ``` - Pin to LTS track: ```shell printf "https://pkgs.nginx.com/plus/LTS/alpine/v`egrep -o '^[0-9]+\.[0-9]+' /etc/alpine-release`/main\n" \ | sudo tee -a /etc/apk/repositories ``` 1. Remove all community-supported NGINX packages. Note that this will also remove all NGINX modules: ```shell sudo apk del -r nginx ``` 1. Install the NGINX Plus package: ```shell sudo apk add nginx-plus ``` 1. Copy the downloaded JWT file to the **/etc/nginx/** directory and make sure it is named **license.jwt**: ```shell sudo cp .jwt /etc/nginx/license.jwt ``` 1. Check the `nginx` version to verify that NGINX Plus LTS is installed correctly: ```shell nginx -v ``` The command output should indicate an LTS release: the second numeric component of the Plus release version should be `0`: ```none nginx version: nginx/1.29.8 (nginx-plus-r37.0.6) ``` 1. Make sure license reporting to F5 licensing endpoint is configured. By default, no configuration is required. However, it becomes necessary when NGINX Plus is installed in a disconnected environment, uses NGINX Instance Manager for usage reporting, or uses a custom path for the license file. Configuration can be done in the [`mgmt {}`](https://nginx.org/en/docs/ngx_mgmt_module.html) block of the NGINX Plus configuration file (`/etc/nginx/nginx.conf`). For more information, see [About Subscription Licenses](/solutions/about-subscription-licenses.md). 1. If you are using [NGINX Instance Manager](/nim/) in your infrastructure, install and enable [NGINX Agent](https://docs.nginx.com/nginx-agent/overview/). See [Install and Configure NGINX Agent](https://github.com/nginx/agent/blob/main/README.md) for details. ## Upgrade NGINX Plus {#upgrade} For general upgrade instructions, see [Upgrading NGNIX Plus](/nginx/admin-guide/installing-nginx/upgrading-nginx-plus.md). -------------------------------------------------------------------------------- title: "Upgrading NGINX Plus" url: https://docs.nginx.com/nginx/admin-guide/installing-nginx/upgrading-nginx-plus/index.md description: Upgrade F5 NGINX Plus with step-by-step instructions for the base package and dynamic modules on all supported Linux distributions. -------------------------------------------------------------------------------- This article explains how to upgrade existing NGINX Plus installation and dynamic modules. ## About {#upgrade} Keeping your NGINX Plus installation updated ensures it includes the latest features, security patches, and fixes. Critical bug patches and security updates are provided for the **two** most recent releases of NGINX Plus. Each NGINX Plus release reaches End of Software Development upon the next version's release, meaning no new features or routine bug fixes will be added to that version. ## Prerequisites Before upgrading, verify the following: 1. Your operating system is configured to retrieve binary packages from the official NGINX Plus repository: ensure that `nginx-plus.crt` and `nginx-plus.key` and repository file are set. See installation instructions for your operating system: [Amazon Linux 2023](/nginx/admin-guide/installing-nginx/installing-nginx-plus.md#install_amazon2023), [Amazon Linux 2](/nginx/admin-guide/installing-nginx/installing-nginx-plus.md#install_amazon2), [RHEL-based 8.1](/nginx/admin-guide/installing-nginx/installing-nginx-plus.md#install_rhel8), [RHEL-based 9](/nginx/admin-guide/installing-nginx/installing-nginx-plus.md#install_rhel), [RHEL-based 10](/nginx/admin-guide/installing-nginx/installing-nginx-plus.md#install_rhel10), [Debian](/nginx/admin-guide/installing-nginx/installing-nginx-plus.md#install_debian), [Ubuntu](/nginx/admin-guide/installing-nginx/installing-nginx-plus.md#install_debian_ubuntu), [FreeBSD](/nginx/admin-guide/installing-nginx/installing-nginx-plus.md#install_freebsd), [SLES](/nginx/admin-guide/installing-nginx/installing-nginx-plus.md#install_suse), [Alpine](/nginx/admin-guide/installing-nginx/installing-nginx-plus.md#install_alpine). 2. Your NGINX Plus subscription is active. You can verify your subscription on the [MyF5 Customer Portal](https://account.f5.com/myf5). 3. For NGINX Plus R33 and later, [license reporting](/solutions/about-subscription-licenses/getting-started.md) is configured. If upgrading from R32 or earlier, add the license file before upgrading and configure usage reporting. See [NGINX Plus R32 upgrade note](#nginx-plus-r32-and-earlier). ## Upgrade steps 1. Back up the configuration and log files. - For **Linux**: ```shell sudo cp -a /etc/nginx /etc/nginx-plus-backup && \ sudo cp -a /var/log/nginx /var/log/nginx-plus-backup ``` - For **FreeBSD**: ```shell sudo cp -a /usr/local/etc/nginx /usr/local/etc/nginx-plus-backup && \ sudo cp -a /var/log/nginx /var/log/nginx-plus-backup ``` 2. Upgrade to the newest NGINX Plus package. - For **RHEL-based**: ```shell sudo yum upgrade nginx-plus ``` - For **Debian** and **Ubuntu**: ```shell sudo apt update && \ sudo apt install nginx-plus ``` - For **FreeBSD**: ```shell sudo pkg upgrade nginx-plus ``` 3. Verify the upgrade: - Check the NGINX Plus version: ```shell nginx -v ``` The output of the command: ```shell nginx version: nginx/1.29.8 (nginx-plus-r37.0.0) ``` - Check the error log: ```shell tail /var/log/nginx/error.log ``` ## Upgrade notes ### NGINX Plus R24 and earlier Starting from [Release 24](nginx/releases.md#r24) (R24), NGINX Plus repositories have been separated into individual repositories based on operating system distribution and license subscription. Before upgrading from NGINX Plus R24 and earlier versions, you must first reconfigure your repositories to point to the correct location. To reconfigure your repository, follow the installation instructions above for your operating system: [Amazon Linux 2023](#install_amazon2023), [Amazon Linux 2](#install_amazon2), [RHEL-based 8.1](#install_rhel8), [RHEL-based 9](#install_rhel), [Debian or Ubuntu](#install_debian_ubuntu), [FreeBSD](#install_freebsd), [SLES](#install_suse). ### NGINX Plus R32 and earlier Starting from [NGINX Plus Release 33](nginx/releases.md#r33), a JWT license file is required for each NGINX Plus instance. For more information, see [About Subscription Licenses](/solutions/about-subscription-licenses.md). 1. Get the JWT file associated with your NGINX Plus subscription from the [MyF5 Customer Portal](https://account.f5.com/myf5): 1. Log in to [MyF5](https://my.f5.com/manage/s/). 1. Go to **My Products & Plans > Subscriptions** to see your active subscriptions. 1. Find your NGINX subscription, and select the **Subscription ID** for details. 1. Download the **JSON Web Token** file from the subscription page. 2. Create the **/etc/nginx/** directory for Linux or the **/usr/local/etc/nginx** directory for FreeBSD: - For **Linux**: ```shell sudo mkdir -p /etc/nginx ``` - For **FreeBSD**: ```shell sudo mkdir -p /usr/local/etc/nginx ``` **Note:** If you’re upgrading from NGINX Plus R32 or earlier to R33 or later and plan to use a custom path for the license file, note that the custom path isn’t recognized until after the upgrade. You must first create a placeholder file at `/etc/nginx/license.jwt` (or `/usr/local/etc/nginx/license.jwt` on FreeBSD). 1. **Before upgrading**: Create the placeholder file: ```bash touch /etc/nginx/license.jwt ``` 1. **After upgrading**: Update the [`license_token`](https://nginx.org/en/docs/ngx_mgmt_module.html#license_token) directive in the [`mgmt`](https://nginx.org/en/docs/ngx_mgmt_module.html) block of the configuration to point to your custom path: ```nginx mgmt { license_token ; } ``` 3. After downloading the JWT file, copy it to the **/etc/nginx/** directory for Linux, or to the **/usr/local/etc/nginx** directory for FreeBSD, and make sure it's named **license.jwt**: - For **Linux**: ```shell sudo cp .jwt /etc/nginx/license.jwt ``` - For **FreeBSD**: ```shell sudo cp .jwt /usr/local/etc/nginx/license.jwt ``` 4. Perform an upgrade. 5. After upgrade, it is possibly necessary to configure NGINX Plus usage reporting. By default, no configuration is required. However, if NGINX Plus is installed in an [offline environment](#offline_install) or if the JWT license file is located in a non-default directory, extra configuration is required. For offline environments, usage reporting should be configured for NGINX Instance Manager 2.18 or later. In the `nginx.conf` configuration file, specify the following directives: - the [`mgmt`](https://nginx.org/en/docs/ngx_mgmt_module.html#mgmt) context handles NGINX Plus licensing and usage reporting configuration, - the [`usage_report`](https://nginx.org/en/docs/ngx_mgmt_module.html#usage_report) directive specifies the domain name or IP address of the NGINX Instance Manager, - the [`enforce_initial_report`](https://nginx.org/en/docs/ngx_mgmt_module.html#usage_report) directive enables a 180-day grace period for sending the initial usage report. The initial usage report must be received by F5 licensing endpoint within this grace period. If the report is not received in time, traffic processing will be stopped: ```nginx mgmt { usage_report endpoint=NIM_FQDN; enforce_initial_report off; } ``` In NGINX Instance Manager, prepare and send the usage report to F5 licensing endpoint. For more information, see [Report usage to F5 in a disconnected environment](nim/licensing-and-reporting/report-usage-disconnected-deployment.md). If the JWT license file is located in a directory other than **/etc/nginx/** for Linux or **usr/local/etc/nginx/** for FreeBSD, you must specify its name and path in the [`license_token`](https://nginx.org/en/docs/ngx_mgmt_module.html#license_token) directive: ```nginx mgmt { license_token custom/file/path/license.jwt; } ``` ## Upgrade NGINX Plus modules {#upgrade_modules} The upgrade procedure depends on how the module was supplied and installed. - [NGINX‑authored](/nginx/admin-guide/installing-nginx/installing-nginx-plus.md#nginx-authored-dynamic-modules) and [NGINX‑certified community](/nginx/admin-guide/installing-nginx/installing-nginx-plus.md#nginx-certified-community-dynamic-modules) dynamic modules are updated automatically together with NGINX Plus. **Note:** For FreeBSD, each NGINX‑authored and NGINX‑certified module must be updated separately using FreeBSD package management tool. - [Community](/nginx/admin-guide/installing-nginx/installing-nginx-plus.md#community-dynamic-modules) dynamic modules must be recompiled against the corresponding NGINX Open Source version. See [Installing NGINX Community Modules](/nginx/admin-guide/installing-nginx/installing-nginx-plus.md#install_modules_oss). -------------------------------------------------------------------------------- title: "Active-Passive HA for NGINX Plus on AWS Using Elastic IP Addresses" url: https://docs.nginx.com/nginx/deployment-guides/amazon-web-services/high-availability-keepalived/index.md description: Create a highly available active-passive deployment of F5 NGINX Plus on AWS with a solution combining keepalived and the AWS Elastic IP address feature. -------------------------------------------------------------------------------- This guide explains how to create a high availability (HA) active‑passive deployment of F5 NGINX Plus in the [Amazon Web Services](https://aws.amazon.com/) (AWS) cloud. It combines the `keepalived`‑based solution for high availability (provided by NGINX for on‑premises HA deployments) with the AWS Elastic IP address feature. NGINX also provides a [solution for active‑active HA of NGINX Plus in AWS](high-availability-network-load-balancer.md), using AWS Network Load Balancer. ## Overview The [supported solution for HA deployment](nginx/admin-guide/high-availability/ha-keepalived.md) of NGINX Plus that uses `keepalived` is designed for on‑premises deployments. It is typically not viable in cloud environments, such as AWS, because of the networking restrictions they impose. One method for deploying NGINX Plus in a highly available manner on AWS is to use ELB in front of NGINX Plus instances. But, the method has several disadvantages: - It increases the cost of your deployment. - It limits the number of protocols NGINX Plus and your applications can support. In particular, ELB does not support UDP load balancing. - It does not provide a single static IP address for NGINX Plus instances, which is a crucial requirement for some applications. This guide explains how to create an active‑passive HA deployment of NGINX Plus on AWS that doesn’t require ELB and thus isn't subject to its disadvantages. It combines the `keepalived`‑based solution with AWS’s [Elastic IP address](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/elastic-ip-addresses-eip.html) feature. Most importantly, this method addresses the requirement for a single IP address: as long as the primary NGINX Plus instance is operating correctly, it has the Elastic IP address. If the primary fails, the backup instance becomes the primary and reassociates the Elastic IP address with itself, as shown in the figure. When two NGINX Plus nodes hosted in AWS share an elastic IP address, the address switches to the backup automatically when the primary goes down, preserving high availability As an alternative to ELB, you can use Route 53 to distribute traffic among NGINX Plus instances, relying only on DNS load balancing. However, clients as well as intermediate DNS servers often cache DNS records as specified by the TTL value in the record, so there can be a delay in propagation of the updated records to the clients. This can lead to increased downtime of your applications as observed by clients. Such an update can happen when Route 53 detects the failure of an NGINX Plus instance and removes the corresponding record. In contrast, when you use the HA solution along with Route 53, the record usually doesn’t change because the IP address stays the same, and there is no TTL‑related problem. **Notes:** - We have successfully tested the instructions on Ubuntu 16.04 LTS (Xenial Xerus) and CentOS 7, with `keepalived` installed from the respective OS vendor repositories. - Except as noted, perform all steps on both the primary and backup instance. - The solution is not covered by your NGINX Plus support contract. - In addition to the [active‑active HA solution](high-availability-network-load-balancer.md) mentioned above, NGINX offers a solution based on [AWS Lambda](https://aws.amazon.com/lambda/) which does not require installation of any additional software on the NGINX Plus instances. The [F5 Professional Services](https://www.f5.com/services) team can deploy and configure the Lambda‑based solution for you and provide support. ## Step 1 – Launch Two NGINX Plus Instances The scripts in the HA solution use the AWS API to associate an Elastic IP address with an NGINX Plus instance, and so must have credentials for accessing the API. AWS uses _IAM roles_ to handle credentials, so you need to create a role and attach it to each NGINX Plus instance. Perform these steps (for step‑by‑step instructions, see the [AWS documentation](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html)): 1. Create an IAM role and attach the following custom policy to it. The instance to which the policy applies can manipulate the Elastic IP address (adopt or release it) as well as perform the two indicated `Describe` actions. ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "ec2:AssociateAddress", "ec2:DescribeInstances", "ec2:DescribeAddresses", "ec2:DisassociateAddress" ], "Resource": "*" } ] } ``` Alternatively, you can use the `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables to provide credentials to the HA scripts, as shown in [Step 5](#ha-aws_keepalived-configure). 2. Launch two instances and [install NGINX Plus](nginx/admin-guide/installing-nginx/installing-nginx-plus-amazon-web-services.md) on each. (As noted, we tested the instructions on Ubuntu 16.04 LTS and CentOS 7.) 3. Attach this IAM role to the instance. ## Step 2 – Allocate an Elastic IP address Allocate an Elastic IP address and remember its ID. For detailed instructions, see the [AWS documentation](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/elastic-ip-addresses-eip.html#using-instance-addressing-eips-allocating). ## Step 3 – Install `keepalived`, `wget`, and the AWS CLI 1. Install two packages from your OS vendor’s repository: **keepalived** and **wget**, which is used by the HA scripts. - On Ubuntu systems: ```shell sudo apt-get install keepalived wget ``` - On CentOS systems: ```shell sudo yum install keepalived wget ``` 2. Follow the instructions in the [AWS documentation](https://docs.aws.amazon.com/cli/latest/userguide/installing.html) to install the AWS CLI. ## Step 4 – Download the HA Scripts The NGINX Plus HA solution uses two scripts, which are invoked by `keepalived`: - **nginx-ha-check** – Determines the health of NGINX Plus. - **nginx-ha-notify** – Moves the Elastic IP address when a state transition happens, for example when the backup instance becomes the primary. 1. Create a directory for the scripts, if it doesn’t already exist. - On Ubuntu systems: ```shell sudo mkdir -p /usr/lib/keepalived ``` - On CentOS systems: ```shell sudo mkdir -p /usr/libexec/keepalived ``` 2. Download the scripts from our [GitHub repository](https://github.com/nginxinc/aws-ha-elastic-ip) into the created directory. ## Step 5 – Configure `keepalived` and the HA Scripts There are two configuration files for the HA solution: - **keepalived.conf** – The main configuration file for `keepalived`, slightly different for each NGINX Plus instance. - **nginx-ha-notify** – The script you downloaded in [Step 4](#ha-aws_ha-scripts), with several user‑defined variables. ### Creating keepalived.conf In the **/etc/keepalived** folder create a file named **keepalived.conf** with the following content. ```nginx vrrp_script chk_nginx_service { script "" interval 3 weight 50 } vrrp_instance VI_1 { interface eth0 priority virtual_router_id 51 advert_int 1 unicast_src_ip unicast_peer { } authentication { auth_type PASS auth_pass } track_script { chk_nginx_service } notify "" } ``` You must change values for the following configuration keywords. As you do so, also remove the angle brackets enclosing the placeholder value: - `script` in the `chk_nginx_service` block – The script that sends health checks to NGINX Plus. - On Ubuntu systems, **/usr/lib/keepalived/nginx-ha-check** - On CentOS systems, **/usr/libexec/keepalived/nginx-ha-check** - `priority` – The value that controls which instance becomes primary, with a higher value meaning a higher priority. Use `101` for the primary instance and `100` for the backup. - `unicast_src_ip` – This instance's IP address. - `unicast_peer` – The other instance's IP address. - `auth_pass` – The password string used for authentication between peers. - `notify` – The script that is invoked during a state transition. - On Ubuntu systems, **/usr/lib/keepalived/nginx-ha-notify** - On CentOS systems, **/usr/libexec/keepalived/nginx-ha-notify** ### Creating nginx-ha-notify Modify the user‑defined variables section of the **nginx-ha-notify** script, replacing each `` placeholder with the value specified in the list below: ```none export AWS_ACCESS_KEY_ID= export AWS_SECRET_ACCESS_KEY= export AWS_DEFAULT_REGION= HA_NODE_1= HA_NODE_2= ALLOCATION_ID= ``` - `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` – The credentials for accessing the AWS API. Set them only when an IAM instance profile isn’t used. Otherwise, delete the corresponding two lines. - `AWS_DEFAULT_REGION` – The AWS region of your deployment. - `HA_NODE_1` and `HA_NODE_2` – The internal or private DNS names of the two NGINX Plus instances. - `ALLOCATION_ID` – The ID of the allocated Elastic IP address. ## Testing Run this command on both instances to start the `keepalived` daemon: ```shell sudo service keepalived start ``` The instance with the higher priority becomes the primary. As a result, the Elastic IP address becomes associated with the primary instance, as confirmed on the AWS Console. To check the instance state, run: ```shell cat /var/run/nginx-ha-keepalived.state ``` The command outputs `STATE=MASTER` on the primary instance and `STATE=BACKUP` otherwise. You can simulate the failure of the primary by stopping the `keepalived` daemon: ```shell sudo service keepalived stop ``` Check the state on the backup instance, confirming that it has transitioned to `MASTER`. Additionally, in the AWS Console the Elastic IP address is now associated with the new primary instance. ## Troubleshooting If the solution doesn’t work as expected, check the `keepalived` logs, which are written to **/var/log/syslog**. Also, you can manually run the commands that invoke the `awscli` utility in the **nginx-ha-notify** script to check that the utility is working properly. ## Caveats - In most of our tests it took 5 to 6 seconds for the Elastic IP address to be reassigned. - Elastic IP address reassignment is not free; see [Amazon EC2 Pricing](https://aws.amazon.com/ec2/pricing/). - Because the solution relies on the AWS APIs to reassociate the Elastic IP address, in some rare scenarios – such as flip‑flopping (the instances change state rapidly) or split‑brain (the instances lose connectivity with each other) – it is possible for the Elastic IP address not to end up associated with the primary. We were not able to reproduce these scenarios in our testing, however. If they occur, restart `keepalived` on both instances. ## Resources [Download the HA solution from the NGINX GitHub repository](https://github.com/nginxinc/aws-ha-elastic-ip). ### Revision History - Version 1 (May 2017) – Initial version (NGINX Plus Release 12) -------------------------------------------------------------------------------- title: "Configuring Active-Active High Availability and Additional Passive Nodes with keepalived" url: https://docs.nginx.com/nginx/admin-guide/high-availability/ha-keepalived-nodes/index.md description: Enable high availability of F5 NGINX Plus instances in on-premises deployments, with active-active pairs and multiple passive instances. -------------------------------------------------------------------------------- ## Introduction F5 NGINX Plus utilizes [keepalived](http://www.keepalived.org/) to provide high availability (HA) in a standard active‑passive fashion. This provides failover redundancy in the event of a problem on the primary NGINX Plus node. We can extend this functionality with additional nodes and changes to the `keepalived` configuration, providing additional redundancy and scalability options. This guide assumes that you have already configured NGINX Plus in an active‑passive implementation with [the NGINX HA solution](nginx/admin-guide/high-availability/ha-keepalived.md). **NOTE:** In a public cloud deployment we recommend using a Layer 4 or TCP load‑balancing service offered by the cloud provider to distribute traffic to NGINX Plus for active‑active functionality. ## Why Add a Passive Node? Many organizations have strict requirements on levels of redundancy and a two node active‑passive system may not meet these requirements. Adding a third node configured to take over in the event that both other nodes are down, provides further redundancy while keeping the configuration simple. This also allows for maintenance on a node without losing redundancy. ## Why Configure Active-Active HA? You can run NGINX Plus in an “active‑active” fashion, where two or more nodes handle traffic at the same time. This is achieved using multiple active IP addresses. Each IP address is hosted on a single NGINX instance, and the Keepalived configuration ensures that these IP addresses are spread across two or more active nodes. - When hosting multiple services, each service’s DNS name should resolve to one of the IP addresses. Share the IP addresses between the services. - Use round‑robin DNS to map a single DNS name to multiple IP addresses. - Use a L3 load‑balancing device such as a datacenter edge load balancer to distribute L3 traffic between the IP addresses. Active‑active may be used to increase the capacity of your load‑balanced cluster, but be aware that if a single node in an active‑active pair were to fail, the capacity would be reduced by half. You can use active‑Active as a form of safety, to provide sufficient resource to absorb unexpected spikes of traffic when both nodes are active, and you can use active‑active in larger clusters to provide more redundancy. Note that NGINX instances in a load‑balanced cluster do not share configuration or state. For best performance in an active‑active scenario, ensure that connections from the same client are routed to the same active IP address, and use session persistence methods such as [`sticky cookie`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#sticky_cookie) that do not rely on server‑side state. ## Configuring keepalived for an Additional Passive Node To configure an additional passive node for your existing NGINX Plus active‑passive HA pair, perform the following steps: 1. Install the **nginx-plus** and **nginx-ha-keepalived** packages on the new node. 2. Copy **/etc/keepalived/keepalived.conf** from the secondary node to the same location on the new node. 3. Edit **keepalived.conf** on the new node: - Lower the `priority` on any `vrrp_instance` blocks so that it is lower than the other nodes. - Change `unicast_src_ip` to match new node's host IP address. - Add the IP address of the secondary node to the `unicast_peer` section so that all other nodes are listed. Below is a sample **keepalived.conf** on an additional passive node with IP address 192.168.10.12. The IP addresses of the other two nodes are 192.168.10.10 and 192.168.10.11. The virtual IP address (VIP) is 192.168.10.100. ```yaml vrrp_script chk_nginx_service { script "/usr/lib/keepalived/nginx-ha-check" interval 3 weight 50 } vrrp_instance VI_1 { interface eth0 state BACKUP priority 99 virtual_router_id 51 advert_int 1 accept unicast_src_ip 192.168.10.12 unicast_peer { 192.168.10.10 192.168.10.11 } virtual_ipaddress { 192.168.10.100 } track_script { chk_nginx_service } notify "/usr/lib/keepalived/nginx-ha-notify" } ``` 4. Edit **keepalived.conf** on the other nodes, adding the IP address of the new passive node to the `unicast_peer` section so that all other nodes are listed: ```yaml unicast_peer { 192.168.10.11 192.168.10.12 } ``` 4. Restart `keepalived` on all nodes. 5. Test by stopping NGINX Plus on the first two nodes. All NGINX Plus nodes must have the identical configuration and SSL certificates. For information about synchronizing NGINX Plus configuration, see [Synchronizing NGINX Configuration in a Cluster](configuration-sharing.md). ## Configuring keepalived for Active-Active HA In order to direct traffic to both nodes at the same time, an additional VIP must be used. This new VIP will be active on the previously passive node, so that each node is active with its own VIP. To configure an existing NGINX Plus HA pair as active‑active, perform the following steps: 1. Edit **keepalived.conf** on the secondary node: - Copy the entire `vrrp_instance block VI_1` section and paste it below the existing block - In the copied `vrrp_instance` section: - Rename the new `vrrp_instance to VI_2` or another unique name - Change the `virtual_router_id` to `61` or another unique value - Change the `virtual_ipaddress` to an available IP address on the same subnet (in this example 192.168.10.101) - Change the `priority` value to `100` ```none vrrp_script chk_nginx_service { script "/usr/lib/keepalived/nginx-ha-check" interval 3 weight 50 } vrrp_instance VI_1 { interface eth0 state BACKUP priority 101 virtual_router_id 51 advert_int 1 accept unicast_src_ip 192.168.10.10 unicast_peer { 192.168.10.11 } virtual_ipaddress { 192.168.10.100 } track_script { chk_nginx_service } notify "/usr/lib/keepalived/nginx-ha-notify" } vrrp_instance VI_2 { interface eth0 state BACKUP priority 100 virtual_router_id 61 advert_int 1 accept unicast_src_ip 192.168.10.10 unicast_peer { 192.168.10.11 } virtual_ipaddress { 192.168.10.101 } track_script { chk_nginx_service } notify "/usr/lib/keepalived/nginx-ha-notify" } ``` 2. Edit **keepalived.conf** on the primary node: - Repeat the edits performed on the secondary node. - Set the `priority` within the new `vrrp_instance` to `99` or a value lower than on the secondary node. 3. Restart `keepalived` on all nodes. Configuration file and SSL certificate file synchronization is out of scope for this document but make sure all nodes have identical NGINX Plus configuration. ## Configuring NGINX Plus for Active-Active HA Now that the two NGINX Plus nodes are active with their own VIPs, NGINX Plus itself must be configured. There are two options for distributing traffic to the active nodes. Option 1 has all nodes active, with each node handling at least one application. Option 2 has all applications active on all nodes. **NOTE:** If the application being load balanced requires session persistence, we recommend that you use [sticky cookie](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#sticky_cookie), [sticky route](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#sticky_route) or [IP hash](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#ip_hash) method, as they function correctly with multiple active nodes. [Sticky learn](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#sticky_learn) creates a session table in memory that is not shared between active nodes. ### Configuring NGINX Plus for Different Applications on Each Node In this configuration, each NGINX Plus node only processes requests to `server` blocks for which it has an active VIP. In the event of a failover, the active node is primary for additional VIPs and processes requests for the associated `server` blocks. Each `server` block includes the [listen](https://nginx.org/en/docs/http/ngx_http_core_module.html#listen) directive to specify which VIP it is listening on. Building on the active‑active `keepalived` configuration from the [previous section](#conf_active), we're using the same two VIPs here. In this example application 1 is active on NGINX Plus node 1 and application 2 is active on NGINX node 2: ```nginx server { listen 192.168.10.100:80; location / { root /application1; } } server { listen 192.168.10.101:80; location / { root /application2; } } ``` ### Configuring NGINX Plus for All Applications on All Nodes In this configuration, NGINX Plus is able to process the traffic for any application on any VIP. In the event of a failure on a node, the VIP for that node moves to the node with the next highest priority. This way the DNS load balancing configuration does not need to change. Each NGINX Plus node listens for all requests. DNS load balancing is used to distribute requests to NGINX Plus nodes. Simple round-robin DNS is sufficient and can be configured according to the documentation for your DNS server. Ensure that there is an `A` for each VIP with the same fully qualified domain name (FQDN). Each time the name is resolved, the DNS server's response includes all VIPs, but in a different order. ```nginx server { listen *:80; location /app1 { root /application1; } location /app2 { root /application2; } } ``` ## Combining and Expanding Methods Both of the methods in [Configuring NGINX Plus for Active-Active HA](#nginx_for_active) can be combined for an active-active-passive configuration, or can even be extended to a configuration with any number of active nodes. ### Configuring All-Active HA on Three or More Nodes The following `keepalived` configuration is for an active-active-active configuration. Here the [steps for adding an active node](#conf_active) are just repeated for the third one. Notice that this node is active for one VIP, secondary for one VIP, and tertiary for one VIP. ```yaml vrrp_script chk_nginx_service { script "/usr/lib/keepalived/nginx-ha-check" interval 3 weight 50 } vrrp_instance VI_1 { interface eth0 state BACKUP priority 101 virtual_router_id 51 advert_int 1 accept unicast_src_ip 192.168.10.10 unicast_peer { 192.168.10.11 192.168.10.12 192.168.10.13 } virtual_ipaddress { 192.168.10.100 } track_script { chk_nginx_service } notify "/usr/lib/keepalived/nginx-ha-notify" } vrrp_instance VI_2 { interface eth0 state BACKUP priority 100 virtual_router_id 61 advert_int 1 accept unicast_src_ip 192.168.10.10 unicast_peer { 192.168.10.11 192.168.10.12 192.168.10.13 } virtual_ipaddress { 192.168.10.101 } track_script { chk_nginx_service } notify "/usr/lib/keepalived/nginx-ha-notify" } vrrp_instance VI_3 { interface eth0 state BACKUP priority 99 virtual_router_id 71 advert_int 1 accept unicast_src_ip 192.168.10.10 unicast_peer { 192.168.10.11 192.168.10.12 192.168.10.13 } virtual_ipaddress { 192.168.10.102 } track_script { chk_nginx_service } notify "/usr/lib/keepalived/nginx-ha-notify" } ``` ### Configuring Active-Active-Passive HA This example `keepalived` configuration is for the passive node in an active-active-passive configuration. It combines the steps in [Configuring keepalived for an Additional Passive Node](#conf_passive) and [Configuring keepalived for Active-Active HA](#conf_active). ```none vrrp_script chk_nginx_service { script "/usr/lib/keepalived/nginx-ha-check" interval 3 weight 50 } vrrp_instance VI_1 { interface eth0 state BACKUP priority 99 virtual_router_id 51 advert_int 1 accept unicast_src_ip 192.168.10.12 unicast_peer { 192.168.10.10 192.168.10.11 } virtual_ipaddress { 192.168.10.100 } track_script { chk_nginx_service } notify "/usr/lib/keepalived/nginx-ha-notify" } vrrp_instance VI_2 { interface eth0 state BACKUP priority 99 virtual_router_id 61 advert_int 1 accept unicast_src_ip 192.168.10.12 unicast_peer { 192.168.10.10 192.168.10.11 } virtual_ipaddress { 192.168.10.101 } track_script { chk_nginx_service } notify "/usr/lib/keepalived/nginx-ha-notify" } ``` -------------------------------------------------------------------------------- title: "Configuring Logging" url: https://docs.nginx.com/nginx/admin-guide/monitoring/logging/index.md description: Capture detailed information about errors and request processing in log files, either locally or via syslog. -------------------------------------------------------------------------------- Logging is essential for troubleshooting and understanding how traffic flows through your infrastructure. This article explains how to configure error and access logs in NGINX and NGINX Plus, customize access log formats, reduce noise with conditional logging, and forward logs to syslog for centralized collection. ## Set up the error log {#error_log} NGINX writes an error log that records issues of different severity levels. The [`error_log`](https://nginx.org/en/docs/ngx_core_module.html#error_log) directive sets up the log location and severity level. The log location can be a particular file, `stderr`, or `syslog`. ### Default path By default, the error log is located at **logs/error.log**, but the absolute path depends on the operating system and installation method. You can find the path to error log by running `nginx -V` command and extracting the value of `--error-log-path=`: ```shell nginx -V 2>&1|sed -n 's/.*--error-log-path=\([^ ]*\).*/\1/p' ``` The output depends on the operating system. - For RHEL-based, Debian, Ubuntu: ```shell /var/log/nginx/error.log ``` - For FreeBSD: ```shell /usr/local/var/log/nginx/error.log ``` - For docker images, it is sent to container logs (stderr) via symlinks `/var/log/nginx/error.log -> /dev/stderr`. ### Severity level The error severity level follows the `syslog` classification system. The log includes messages from all severity levels above the specified level. The following example changes the minimal severity level of error messages to log from `error` to `warn`: ```nginx error_log logs/error.log warn; ``` With this setting, messages above the `warn` level are logged. That includes `warn`, `error`, `crit`, `alert`, and `emerg` levels. For debug logging, see [Debugging NGINX](debugging.md). ### Error log formats The `error_log` directive writes error messages in the standard NGINX text error-log format. In NGINX Plus, error log also supports structured JSON output via a [`json`](https://nginx.org/en/docs/ngx_core_module.html#error_log_json) parameter: ```nginx error_log /var/log/nginx/error.json error json; ``` Example of JSON-formatted error entry: ```json { "level": "error", "timestamp": "2026-05-04T10:30:15.042+00:00", "pid": 12345, "tid": 12345, "cnum": 3, "msg": "connect() failed", "client": "192.168.1.10", "server": "example.com", "request": "GET /api HTTP/1.1", "upstream": "http://127.0.0.1:8080/api", "errno": 111, "errtext": "Connection refused" } ``` ### Error log inheritance The default setting of the error log works globally. To override it, place the [`error_log`](https://nginx.org/en/docs/ngx_core_module.html#error_log) directive in the `main` (top-level) configuration context. Settings in the `main` context are always inherited by other configuration levels (`http`, `server`, `location`). The `error_log` directive can also be specified on the [`http`](https://nginx.org/en/docs/http/ngx_http_core_module.html#http), [`stream`](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#stream), `server` and [`location`](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) levels. Settings at lower levels override the settings inherited from the higher levels. Each error message is written only once to the error log closest to the level where the error has occurred. However, if several `error_log` directives are specified on the same level, the message is written to all specified logs. ## Set up the access log {#access_log} NGINX records client requests in the access log right after the request is processed. The [`access_log`](https://nginx.org/en/docs/http/ngx_http_log_module.html#access_log) directive specifies the location of the log and its format. By default, the access log is located at **logs/access.log**. The format of logged messages is the predefined **combined** format. To change the format of logged messages, use the [log_format](https://nginx.org/en/docs/http/ngx_http_log_module.html#log_format) directive. The log format is defined using variables. The following example extends the predefined **combined** format by specifying the `gzip ratio` keyword. This enables gzip compression of the response in a virtual server. ```nginx http { log_format compression '$remote_addr - $remote_user [$time_local] ' '"$request" $status $body_bytes_sent ' '"$http_referer" "$http_user_agent" "$gzip_ratio"'; server { gzip on; access_log /spool/logs/nginx-access.log compression; ... } } ``` Another example of the log format enables tracking different time values between NGINX and an upstream server. This may help with diagnosing a slowdown of your website. You can use the following variables to log the indicated time values: - [`$upstream_connect_time`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#var_upstream_connect_time) – The time spent on establishing a connection with an upstream server - [`$upstream_header_time`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#var_upstream_header_time) – The time between establishing a connection and receiving the first byte of the response header from the upstream server - [`$upstream_response_time`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#var_upstream_response_time) – The time between establishing a connection and receiving the last byte of the response body from the upstream server - [`$request_time`](https://nginx.org/en/docs/http/ngx_http_log_module.html#var_request_time) – The total time spent processing a request All time values are measured in seconds with millisecond resolution. ```nginx http { log_format upstream_time '$remote_addr - $remote_user [$time_local] ' '"$request" $status $body_bytes_sent ' '"$http_referer" "$http_user_agent"' 'rt="$request_time" uct="$upstream_connect_time" uht="$upstream_header_time" urt="$upstream_response_time"'; server { access_log /spool/logs/nginx-access.log upstream_time; ... } } ``` When reading the resulting time values, keep the following in mind: - When a request is processed through several servers, the variable contains several values separated by commas - When there is an internal redirect from one upstream group to another, the values are separated by semicolons - When a request is unable to reach an upstream server or a full header cannot be received, the variable contains `0` (zero) - In case of internal error while connecting to an upstream or when a reply is taken from the cache, the variable contains `-` (hyphen) The log can be optimized by enabling the `buffer` and the `cache`. With the [buffer](https://nginx.org/en/docs/http/ngx_http_log_module.html#access_log) parameter enabled, messages will be stored in the buffer first. When the buffer is full (or in some other [cases](https://nginx.org/en/docs/http/ngx_http_log_module.html#access_log)), the messages will be written to the log. To enable caching of log file descriptors, use the [open_log_file_cache](https://nginx.org/en/docs/http/ngx_http_log_module.html#open_log_file_cache) directive. Similar to the `error_log` directive, the [access_log](https://nginx.org/en/docs/http/ngx_http_log_module.html#access_log) directive defined on a particular configuration level overrides the settings from the previous levels. When processing of a request is completed, the message is written to the log that is configured on the current level, or inherited from the previous levels. If one level has multiple access log definitions, the message is written to all of them. ## Enabling conditional logging {#conditional} Conditional logging allows excluding trivial or unimportant log entries from the access log. In NGINX, conditional logging is enabled by the `if` parameter to the [access_log](https://nginx.org/en/docs/http/ngx_http_log_module.html#access_log) directive. This example excludes requests with HTTP status codes `2xx` (Success) and `3xx` (Redirection): ```nginx map $status $loggable { ~^[23] 0; default 1; } access_log /path/to/access.log combined if=$loggable; ``` ## Usecase: sampling TLS parameters {#tls_sample} Many clients use TLS versions older than TLS 1.3. Though many ciphers are declared insecure, older implementations still use them; ECC certificates offer greater performance than RSA, but not all clients can accept ECC. Many TLS attacks rely on a “man in the middle” who intercepts the cipher negotiation handshake and forces the client and server to select a less secure cipher. Therefore, it’s important to configure F5 NGINX Plus to not support weak or legacy ciphers, but doing so may exclude legacy clients. You can evaluate the SSL data obtained from the client and determine what proportion of clients get excluded if support for older SSL protocols and ciphers is removed. The following configuration example logs the SSL protocol, cipher, and `User-Agent` header of any connected TLS client, assuming that each client selects the most recent protocol and most secure ciphers it supports. In this example, each client is identified by its unique combination of IP address and User-Agent. 1. Define the custom log format `sslparams` that includes the version of the SSL protocol ([`$ssl_protocol`](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#var_ssl_protocol)), ciphers used in the connection ([`$ssl_cipher`](http://nginx.org/en/docs/http/ngx_http_ssl_module.html#var_ssl_cipher)), the client IP address ([`$remote_addr`](http://nginx.org/ru/docs/http/ngx_http_core_module.html#var_remote_addr)), and the value of standard `User Agent` HTTP request field (`$http_user_agent`): ```nginx log_format sslparams '$ssl_protocol $ssl_cipher ' '$remote_addr "$http_user_agent"'; ``` 2. Define a key-value storage that will keep the IP address of the client and its User Agent, for example, `clients`: ```nginx keyval_zone zone=clients:80m timeout=3600s; ``` 3. Create a variable, for example, `$seen` for each unique combination of `$remote_addr` and `User-Agent` header: ```nginx keyval $remote_addr:$http_user_agent $seen zone=clients; server { listen 443 ssl; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_ciphers HIGH:!aNULL:!MD5; if ($seen = "") { set $seen 1; set $logme 1; } access_log /tmp/sslparams.log sslparams if=$logme; # ... } ``` 4. View the log file generated with this configuration: ```none TLSv1.2 AES128-SHA 1.1.1.1 "Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0" TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 2.2.2.2 "Mozilla/5.0 (iPhone; CPU iPhone OS 9_1 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13B143 Safari/601.1" TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 3.3.3.3 "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:58.0) Gecko/20100101 Firefox/58.0" TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 4.4.4.4 "Mozilla/5.0 (Android 4.4.2; Tablet; rv:65.0) Gecko/65.0 Firefox/65.0" TLSv1 AES128-SHA 5.5.5.5 "Mozilla/5.0 (Android 4.4.2; Tablet; rv:65.0) Gecko/65.0 Firefox/65.0" TLSv1.2 ECDHE-RSA-CHACHA20-POLY1305 6.6.6.6 "Mozilla/5.0 (Linux; U; Android 5.0.2; en-US; XT1068 Build/LXB22.46-28) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/57.0.2987.108 UCBrowser/12.10.2.1164 Mobile Safari/537.36" ``` 5. Process the log file to determine the spread of data: ```shell cat /tmp/sslparams.log | cut -d ' ' -f 2,2 | sort | uniq -c | sort -rn | perl -ane 'printf "%30s %s\n", $F[1], "="x$F[0];' ``` In this output, low‑volume, less secure ciphers are identified: ```shell ECDHE-RSA-AES128-GCM-SHA256 ========================= ECDHE-RSA-AES256-GCM-SHA384 ======== AES128-SHA ==== ECDHE-RSA-CHACHA20-POLY1305 == ECDHE-RSA-AES256-SHA384 == ``` Then you can check the logs to determine which clients are using these ciphers and then make a decision about removing these ciphers from the NGINX Plus configuration. For more information about sampling requests with NGINX conditional logging see the [blog post](https://www.nginx.com/blog/sampling-requests-with-nginx-conditional-logging/#var_request_id). ## Logging to syslog {#syslog} The `syslog` utility is a standard for computer message logging and allows collecting log messages from different devices on a single syslog server. In NGINX, logging to syslog is configured with the `syslog:` prefix in [error_log](https://nginx.org/en/docs/ngx_core_module.html#error_log) and [access_log](https://nginx.org/en/docs/http/ngx_http_log_module.html#access_log) directives. Syslog messages can be sent to a `server=` which can be a domain name, an IP address, or a UNIX-domain socket path. A domain name or IP address can be specified with a port to override the default port, `514`. A UNIX-domain socket path can be specified after the `unix:` prefix: ```nginx error_log syslog:server=unix:/var/log/nginx.sock debug; access_log syslog:server=[2001:db8::1]:1234,facility=local7,tag=nginx,severity=info; ``` In the example, NGINX error log messages are written to a UNIX domain socket at the `debug` logging level, and the access log is written to a syslog server with an IPv6 address and port `1234`. The `facility=` parameter specifies the type of program that is logging the message. The default value is `local7`. Other possible values are: `auth`, `authpriv`, `daemon`, `cron`, `ftp`, `lpr`, `kern`, `mail`, `news`, `syslog`, `user`, `uucp`, `local0 ... local7`. The `tag=` parameter applies a custom tag to syslog messages (`nginx` in our example). The `severity=` parameter sets the severity level of syslog messages for access log. Possible values in order of increasing severity are: `debug`, `info`, `notice`, `warn`, `error` (default), `crit`, `alert`, and `emerg`. Messages are logged at the specified level and all more severe levels. In our example, the severity level `error` also enables `crit`, `alert`, and `emerg` levels to be logged. ## Live activity monitoring {#monitoring} NGINX Plus provides a real-time live activity monitoring interface that shows key load and performance metrics of your [HTTP](nginx/admin-guide/load-balancer/http-load-balancer.md) and [TCP](nginx/admin-guide/load-balancer/tcp-udp-load-balancer.md) upstream servers. See the [Live Activity Monitoring](live-activity-monitoring.md) article for more information. ## Compatibility note {#compat} - Support for multiple `error_log` directives at the same configuration level was introduced in NGINX Open Source [1.5.2](https://nginx.org/en/CHANGES). - Support for JSON-formatted error logs was introduced in NGINX Plus [PLS.37.0.0.1 LTS](/nginx/releases.md#r37.0). -------------------------------------------------------------------------------- title: "Control NGINX Processes at Runtime" url: https://docs.nginx.com/nginx/admin-guide/basic-functionality/runtime-control/index.md description: Understand the NGINX processes that handle traffic, and how to control them at runtime. -------------------------------------------------------------------------------- This section describes the processes that NGINX starts at run time and how to control them. ## Master and worker processes {#processes} NGINX has one master process and one or more worker processes. If [caching](nginx/admin-guide/content-cache/content-caching.md) is enabled, the cache loader and cache manager processes also run at startup. The main purpose of the master process is to read and evaluate configuration files, as well as maintain the worker processes. The worker processes do the actual processing of requests. NGINX relies on OS-dependent mechanisms to efficiently distribute requests among worker processes. The number of worker processes is defined by the [worker_processes](https://nginx.org/en/docs/ngx_core_module.html#worker_processes) directive in the **nginx.conf** configuration file and can either be set to a fixed number or configured to adjust automatically to the number of available CPU cores. ## Control NGINX with signals {#signals} To reload your configuration, you can stop or restart NGINX, or send signals to the master process. A signal can be sent by running the `nginx` command (invoking the NGINX executable) with the `-s` argument. ```none nginx -s ``` where `` can be one of the following: - `quit` – Shut down gracefully (the `SIGQUIT` signal) - `reload` – Reload the configuration file (the `SIGHUP` signal) - `reopen` – Reopen log files (the `SIGUSR1` signal) - `stop` – Shut down immediately (or fast shutdown, the `SIGTERM` signal) The `kill` utility can also be used to send a signal directly to the master process. The process ID of the master process is written, by default, to the **nginx.pid** file, which is located in the **/usr/local/nginx/logs** or **/var/run** directory. For more information about advanced signals (for performing live binary upgrades, for example), see [Control nginx](https://nginx.org/en/docs/control.html) at **nginx.org**. ## Control NGINX with Control API {#control-api} In addition to [signal-based controls](#signals), NGINX Plus and NGINX Open Source can be controlled with the Control REST API, available since [NGINX Plus PLS R37.0.0 LTS](nginx/releases.md#r37.0) and NGINX Open Source 1.31.5. The Control API is implemented in the [NGINX master process](#processes) and provides a REST interface for runtime control and inspection. It allows you to: - view worker process information (process name, PID, and exit state); - get a memory dump of loaded NGINX configuration files; - trigger a configuration reload similar to `nginx -s reload` and review reload logs. The Control API provides the following endpoints: - `/1/control/processes` — return worker process information; - `/1/control/config` — return the in-memory NGINX configuration and trigger reloads with `PATCH`; - `/1/nginx` — return NGINX version and build information. See [NGINX Control REST API reference](nginx/admin-guide/basic-functionality/control-api-reference.md) for details. ### Enable NGINX Control API By default, the Control API is disabled. To enable it, start NGINX Open Source or NGINX Plus with the [`-l` argument](https://nginx.org/en/docs/switches.html#l), specifying a UNIX-domain socket or TCP port: ```shell sudo nginx -l unix:/tmp/nginx.sock ``` For NGINX Open Source 1.31.5 and later, make sure NIGNX is built with the `--with-control-api` configuration parameter: ```shell nginx -V 2>&1 | grep -- '--with-control-api' ``` If the parameter is enabled, the command highlights the `--with-control-api` parameter in the nginx build configuration. ### Security implications - Never expose the Control API to the public Internet. Restrict access to the Control API listener with firewalls or ACLs, where possible, place it on a dedicated interface or VLAN. - Configure the Control API to listen on a UNIX-domain socket. This is currently the most effective way to control access, because authorization can be managed through file permissions, and the created socket file is accessible only to the user running NGINX. - Keep your NGINX binary and the operating system up to date. ### Control API reference documentation See [NGINX Control REST API reference](nginx/admin-guide/basic-functionality/control-api-reference.md) for details on available endpoints, request parameters, and response schemas. [[icon: download]Download Control API OpenAPI YAML specification](/nginx/admin-guide/yaml/nginx-control-api/1/nginx_control_api.yaml) or copy this link to explore the API using standard OpenAPI-compatible tools such as Redoc or Swagger UI. -------------------------------------------------------------------------------- title: "Installing NGINX Open Source" url: https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-open-source/index.md description: Install NGINX Open Source either as a prebuilt package or from source, following step-by-step instructions for all supported Linux distributions. -------------------------------------------------------------------------------- This article explains how to install NGINX Open Source on various operating systems, including an overview of existing NGINX Open Source versions, installation types and methods, modules included in the default package and dynamic modules packages, and the basics of compiling NGINX from the source code. ## Stable and mainline versions NGINX Open Source is available in two versions: Mainline and Stable. The source code and release notes for both versions are available from [NGINX download page](https://nginx.org/en/download.html). ### Mainline version Mainline version (also Mainline release, Mainline branch) is the latest development version, updated approximately every 1 or 2 months, includes the latest features, bug fixes and security fixes. This version is recommended for production unless your organization has strict requirements for stability, in which case the stable version might be the better choice. The Mainline version always has an odd middle number, for example, 1.*27*.X. ### Stable version Stable version (also Stable release, Stable branch) is updated typically once a year or as needed for critical bug fixes or security fixes that are always backported from the Mainline version. This version is recommended for environments with strict requirements for stability. The Stable version is always even-numbered, for example, 1.*28*.X. ## Distribution and installation methods Both the NGINX Open Source Mainline and Stable versions can be obtained and installed in several ways: - A package from the [official NGINX Open Source repository](#official-repository) (recommended for production). This is the most reliable method: you have to set up the repository once, but after that the provided package is always up to date. - A package from your [operating system’s default package repository](#os-default-repository) (suitable for learning or testing). This is the easiest method, but generally the provided package is outdated. - Your own package [compiled from source](#sources) (recommended for advanced and custom builds). This method is the most flexible: you can include non-standard or third‑party modules, apply the latest security patches, or build the binary for almost any Unix-like operating system using different compilers and custom compiler options. - A container: suitable for development, testing, production deployments in container environments such as Docker, Podman, Kubernetes: see [Deploying NGINX with Docker](/nginx/admin-guide/installing-nginx/installing-nginx-docker.md#using-nginx-open-source-docker-images) ## OS default repository Installing from your operating system’s default repository is the simplest and fastest method. It is suitable for demos, learning, or testing environments. However, for production use, there are some considerations: - The package is maintained by the distribution’s repository maintainers, so F5/NGINX cannot verify its contents, build process, or update frequency. - The package may be outdated. To compare release versions and features, see the [Changelog](https://nginx.org/en/CHANGES). - Some Linux distros, especially those focused on long-term support, provide the Stable version only. For the latest official release and better control over updates, it is recommended to configure your package manager to install [from the official NGINX repository](#official-repository). The installation steps include updating the package repository and installing the `nginx` binary. - For MacOS: ```shell # ensure Homebrew is installed, see https://brew.sh/ brew update && \ brew install nginx ``` - For RHEL-based distributions and Amazon Linux 2023: ```shell sudo dnf update -y && \ sudo dnf install nginx ``` If the output of the command is `no nginx package found`, install the [EPEL repository](https://docs.fedoraproject.org/en-US/epel/) that contains the nginx package and run the command again. The EPEL installation steps include installing the repository, clearing the packages cache and updating the repository info: ```shell sudo dnf update -y && \ sudo dnf install epel-release -y && \ sudo dnf clean all && \ sudo dnf update -y ``` - For Debian-based distributions: ```shell sudo apt update -y && \ sudo apt install nginx ``` - For FreeBSD or DragonFly BSD: ```shell sudo pkg update -y && \ sudo pkg install nginx ``` - For SUSE Linux Enterprise or openSUSE: ```shell # ensure Web-Scripting-Module repo is enabled sudo zypper refresh -y && \ sudo zypper install nginx ``` - For Alpine: ```shell # ensure community repo is enabled in /etc/apk/repositories sudo apk update && \ sudo apk add nginx ``` - For Amazon Linux 2: ```shell sudo yum install -y amazon-linux-extras amazon-linux-extras list | grep nginx # get package name, e.g. nginx1 sudo amazon-linux-extras install nginx1 ``` Check the version installed: ```shell sudo nginx -v ``` It is recommended to review the [Changelog](https://nginx.org/en/CHANGES) and compare the features, bugfixes, and security fixes between the installed version and the latest release. The latest NGINX Open Source version is always available from [the official repository](#official-repository). You can find out the path to NGINX configuration file, error log, and access log files using the command: ```shell nginx -V 2>&1 | awk -F: '/configure arguments/ {print $2}' | xargs -n1 ``` where: - `--conf-path=` is the path to the `nginx.conf` configuration file - `--error-log-path=` is the path to the error log - `--http-log-path=` is the path to the access log ## Official repository You can configure your package manager to install NGINX Open Source from the official **nginx.org** repository, which provides both the latest [Mainline and Stable](#stable_vs_mainline) versions for most major production operating systems. See the [Changelog](https://nginx.org/en/CHANGES) for version details. You need to set up the repository once, but after that the provided package will stay up to date with the latest releases. Before installing, ensure your operating system and architecture are [supported](https://nginx.org/en/linux_packages.html#distributions). This installation method is recommended for production environments. ### Repository contents The repository contains the latest versions of the following packages: | Package name | Description | | ---------------------------| ---------------------------------------------| | `nginx` | Main NGINX Open Source package, contains [core modules](#prebuilt_modules) and [statically linked modules](#statically-linked-modules). | | `nginx-module-geoip` | The [`ngx_http_geoip_module`](https://nginx.org/en/docs/http/ngx_http_geoip_module.html) as a [dynamic module](#dynamic-modules). | | `nginx-module-image-filter`| The [`ngx_http_image-filter_module`](https://nginx.org/en/docs/http/ngx_http_image_filter_module.html) as a [dynamic module](#dynamic-modules). | | `nginx-module-njs` | The [`ngx_http_js_module`](https://nginx.org/en/docs/http/ngx_http_js_module.html) and [`ngx_stream_js_module`](https://nginx.org/en/docs/stream/ngx_stream_js_module.html) as [dynamic modules](#dynamic-modules). | | `nginx-module-perl` | The [`ngx_http_perl_module`](https://nginx.org/en/docs/http/ngx_http_perl_module.html) as a [dynamic module](#dynamic-modules). | | `nginx-module-xslt` | The [`ngx_http_xsl_module`](https://nginx.org/en/docs/http/ngx_http_xslt_module.html) as a [dynamic module](#dynamic-modules). | | `nginx-module-otel` | The [`ngx_otel_module`](https://nginx.org/en/docs/ngx_otel_module.html) as a [dynamic module](#dynamic-modules). | | `nginx-module-acme` | The [`ngx_http_acme_module`](https://nginx.org/en/docs/http/ngx_http_acme_module.html) as a [dynamic module](#dynamic-modules). | ### RHEL-based packages RHEL-based operating systems include RHEL, CentOS, Oracle Linux, AlmaLinux, Rocky Linux. Before installing, check if your operating system and architecture are supported, see [Supported distributions and versions](https://nginx.org/en/linux_packages.html#distributions). 1. Set up the `yum` or `dnf` repository. - In the `/etc/yum.repos.d` directory, create the `nginx.repo` file using any text editor, for example, `vi`: ```shell sudo vi /etc/yum.repos.d/nginx.repo ``` - Add the following lines to the file, where `nginx-stable` and `nginx-mainline` point to the latest Stable or Mainline version of NGINX Open Source: ```text [nginx-stable] name=nginx stable repo baseurl=https://nginx.org/packages/centos/$releasever/$basearch/ gpgcheck=1 enabled=1 gpgkey=https://nginx.org/keys/nginx_signing.key module_hotfixes=true [nginx-mainline] name=nginx mainline repo baseurl=https://nginx.org/packages/mainline/centos/$releasever/$basearch/ gpgcheck=1 enabled=0 gpgkey=https://nginx.org/keys/nginx_signing.key module_hotfixes=true ``` - Save the file. 4. If needed, switch to `nginx-mainline` packages instead of `nginx-stable` that are enabled by default (the `enable=1` parameter): ```shell sudo dnf config-manager --enable nginx-mainline ``` 5. Update the repository: ```shell sudo dnf update ``` 6. Install the `nginx` package: ```shell sudo dnf install nginx ``` When prompted to accept the GPG key, verify that the following three fingerprints match, and if so, accept them: ```shell Importing GPG key 0xB49F6B46: UserID : "nginx signing key ``` Then, enable each module in the `nginx.conf` configuration file using the [`load_module`](https://nginx.org/en/docs/ngx_core_module.html#load_module) directive. The resulting `.so` files are located in the `/usr/lib/nginx/modules` directory. ### Debian packages This section covers Debian packages only. For Ubuntu-specific instructions, see [Ubuntu packages](#ubuntu-packages). Before installing, check if your operating system and architecture are supported, see [Supported distributions and versions](https://nginx.org/en/linux_packages.html#distributions). 1. Install the prerequisites: ```shell sudo apt update && \ sudo apt install curl \ gnupg2 \ ca-certificates \ lsb-release \ debian-archive-keyring ``` 2. Import an official nginx signing key to allow `apt` to verify the authenticity of packages. Fetch the key: ```shell curl https://nginx.org/keys/nginx_signing.key | gpg --dearmor \ | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null ``` 3. Verify that the downloaded file contains the correct signing key: ```shell gpg --dry-run --quiet --no-keyring --import --import-options import-show /usr/share/keyrings/nginx-archive-keyring.gpg ``` The output should list the following three full fingerprints: ```none pub rsa4096 2024-05-29 [SC] 8540A6F18833A80E9C1653A42FD21310B49F6B46 uid nginx signing key pub rsa2048 2011-08-19 [SC] [expires: 2027-05-24] 573BFD6B3D8FBC641079A6ABABF5BD827BD9BF62 uid nginx signing key pub rsa4096 2024-05-29 [SC] 9E9BE90EACBCDE69FE9B204CBCDCD8A38D88A2B3 uid nginx signing key ``` If the fingerprints do not match, delete the file immediately. 4. Set up the `apt` repository to fetch packages from either `stable` or `mainline` branch. - For `stable`: ```shell echo "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] \ https://nginx.org/packages/debian `lsb_release -cs` nginx" \ | sudo tee /etc/apt/sources.list.d/nginx.list ``` - For `mainline`: ```shell echo "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] \ https://nginx.org/packages/mainline/debian `lsb_release -cs` nginx" \ | sudo tee /etc/apt/sources.list.d/nginx.list ``` 5. Set up repository pinning to prioritize official nginx packages over those provided by the distribution: ```shell echo -e "Package: *\nPin: origin nginx.org\nPin: release o=nginx\nPin-Priority: 900\n" \ | sudo tee /etc/apt/preferences.d/99nginx ``` 6. Install the `nginx` package: ```shell sudo apt update && \ sudo apt install nginx ``` 7. Start NGINX Open Source: ```shell sudo nginx ``` 8. Verify that NGINX Open Source is up and running using the `curl` command: ```shell curl -I 127.0.0.1 ``` Expected output: ```shell HTTP/1.1 200 OK Server: nginx/1.31.6 ``` After installation, the following files are available for configuration and troubleshooting: - Configuration file: `nginx.conf`, located in `/etc/nginx/` - Log files: `access.log` and `error.log`, located in `/var/log/nginx/` 9. If needed, install one or more [dynamic module packages](#repository-contents): ```shell sudo apt install nginx-module- ``` Then, enable each module in the `nginx.conf` configuration file using the [`load_module`](https://nginx.org/en/docs/ngx_core_module.html#load_module) directive. The resulting `.so` files are located in the `/usr/lib/nginx/modules` directory. ### Ubuntu packages Before installing, check if your operating system and architecture are supported, see [Supported distributions and versions](https://nginx.org/en/linux_packages.html#distributions). 1. Install the prerequisites: ```shell sudo apt update && \ sudo apt install curl \ gnupg2 \ ca-certificates \ lsb-release \ ubuntu-keyring ``` 2. Import an official nginx signing key to allow `apt` to verify the authenticity of packages. Fetch the key: ```shell curl https://nginx.org/keys/nginx_signing.key | gpg --dearmor \ | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null ``` 3. Verify that the downloaded file contains the correct signing key: ```shell gpg --dry-run --quiet --no-keyring --import --import-options import-show /usr/share/keyrings/nginx-archive-keyring.gpg ``` The output should list the following three full fingerprints: ```none pub rsa4096 2024-05-29 [SC] 8540A6F18833A80E9C1653A42FD21310B49F6B46 uid nginx signing key pub rsa2048 2011-08-19 [SC] [expires: 2027-05-24] 573BFD6B3D8FBC641079A6ABABF5BD827BD9BF62 uid nginx signing key pub rsa4096 2024-05-29 [SC] 9E9BE90EACBCDE69FE9B204CBCDCD8A38D88A2B3 uid nginx signing key ``` If the fingerprints do not match, delete the file immediately. 4. Set up the `apt` repository to fetch packages from either `stable` or `mainline` branch. - For `stable`: ```shell echo "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] \ https://nginx.org/packages/ubuntu `lsb_release -cs` nginx" \ | sudo tee /etc/apt/sources.list.d/nginx.list ``` - For `mainline`: ```shell echo "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] \ https://nginx.org/packages/mainline/ubuntu `lsb_release -cs` nginx" \ | sudo tee /etc/apt/sources.list.d/nginx.list ``` 5. Set up repository pinning to prioritize official nginx packages over those provided by the distribution: ```shell echo -e "Package: *\nPin: origin nginx.org\nPin: release o=nginx\nPin-Priority: 900\n" \ | sudo tee /etc/apt/preferences.d/99nginx ``` 6. Install the `nginx` package: ```shell sudo apt update && \ sudo apt install nginx ``` 7. Start NGINX Open Source: ```shell sudo nginx ``` 8. Verify that NGINX Open Source is up and running using the `curl` command: ```shell curl -I 127.0.0.1 ``` Expected output: ```shell HTTP/1.1 200 OK Server: nginx/1.31.6 ``` After installation, the following files are available for configuration and troubleshooting: - Configuration file: `nginx.conf`, located in `/etc/nginx/` - Log files: `access.log` and `error.log`, located in `/var/log/nginx/` 9. If needed, install one or more [dynamic module packages](#repository-contents): ```shell sudo apt install nginx-module- ``` Then, enable each module in the `nginx.conf` configuration file using the [`load_module`](https://nginx.org/en/docs/ngx_core_module.html#load_module) directive. The resulting `.so` files are located in the `/usr/lib/nginx/modules` directory. ### SUSE Linux Enterprise packages Before installing, check if your operating system and architecture are supported, see [Supported distributions and versions](https://nginx.org/en/linux_packages.html#distributions). 1. Install the prerequisites: ```shell sudo zypper refresh && \ sudo zypper install curl \ ca-certificates \ gpg2 ``` 2. Set up the `zypper` repository to fetch packages from either `stable` or `mainline` branch. - For `stable`: ```shell sudo zypper addrepo --gpgcheck --type yum --refresh --check \ 'https://nginx.org/packages/sles/$releasever_major' nginx-stable ``` - For `mainline`: ```shell sudo zypper addrepo --gpgcheck --type yum --refresh --check \ 'https://nginx.org/packages/mainline/sles/$releasever_major' nginx-mainline ``` 3. Import the official nginx signing key so `zypper` and `rpm` could verify the packages authenticity. Fetch the key: ```shell curl -o /tmp/nginx_signing.key https://nginx.org/keys/nginx_signing.key ``` 4. Verify that the downloaded file contains the proper key: ```shell gpg --show-keys /tmp/nginx_signing.key ``` The output should list the following three full fingerprints: ```none pub rsa4096 2024-05-29 [SC] 8540A6F18833A80E9C1653A42FD21310B49F6B46 uid nginx signing key pub rsa2048 2011-08-19 [SC] [expires: 2027-05-24] 573BFD6B3D8FBC641079A6ABABF5BD827BD9BF62 uid nginx signing key pub rsa4096 2024-05-29 [SC] 9E9BE90EACBCDE69FE9B204CBCDCD8A38D88A2B3 uid nginx signing key ``` If the fingerprints do not match, delete the file immediately. 5. Import the key to the `rpm` database: ```shell sudo rpmkeys --import /tmp/nginx_signing.key ``` 6. Install the `nginx` package: ```shell sudo zypper install nginx ``` 7. Verify that NGINX Open Source is up and running using the `curl` command: ```shell curl -I 127.0.0.1 ``` Expected output: ```shell HTTP/1.1 200 OK Server: nginx/1.31.6 ``` After installation, the following files are available for configuration and troubleshooting: - Configuration file: `nginx.conf`, located in `/etc/nginx/` - Log files: `access.log` and `error.log`, located in `/var/log/nginx/` 8. If needed, install one or more [dynamic module packages](#repository-contents): ```shell sudo zypper install nginx-module- ``` Then, enable each module in the `nginx.conf` configuration file using the [`load_module`](https://nginx.org/en/docs/ngx_core_module.html#load_module) directive. The resulting `.so` files are located in the `/usr/lib64/nginx/modules` directory. ### Alpine Linux packages Before installing, check if your operating system and architecture are supported, see [Supported distributions and versions](https://nginx.org/en/linux_packages.html#distributions). 1. Install the prerequisites: ```shell sudo apk add openssl \ curl \ ca-certificates ``` 2. Set up the `apk` repository to fetch packages from either `stable` or `mainline` branch by adding the repository URL to the `/etc/apk/repositories` file and the `@nginx` tag. - For `stable`: ```shell printf "%s%s%s%s\n" \ "@nginx " \ "https://nginx.org/packages/alpine/v" \ `egrep -o '^[0-9]+\.[0-9]+' /etc/alpine-release` \ "/main" \ | sudo tee -a /etc/apk/repositories ``` - For `mainline`: ```shell printf "%s%s%s%s\n" \ "@nginx " \ "https://nginx.org/packages/mainline/alpine/v" \ `egrep -o '^[0-9]+\.[0-9]+' /etc/alpine-release` \ "/main" \ | sudo tee -a /etc/apk/repositories ``` 3. Import the official nginx signing key so that `apk` can verify the authenticity of packages. Fetch the key: ```shell curl -o /tmp/nginx_signing.rsa.pub https://nginx.org/keys/nginx_signing.rsa.pub ``` 4. Verify that the downloaded file contains the proper public key: ```shell openssl rsa -pubin -in /tmp/nginx_signing.rsa.pub -text -noout ``` The output should contain the following modulus: ```none Public-Key: (2048 bit) Modulus: 00:fe:14:f6:0a:1a:b8:86:19:fe:cd:ab:02:9f:58: 2f:37:70:15:74:d6:06:9b:81:55:90:99:96:cc:70: 5c:de:5b:e8:4c:b2:0c:47:5b:a8:a2:98:3d:11:b1: f6:7d:a0:46:df:24:23:c6:d0:24:52:67:ba:69:ab: 9a:4a:6a:66:2c:db:e1:09:f1:0d:b2:b0:e1:47:1f: 0a:46:ac:0d:82:f3:3c:8d:02:ce:08:43:19:d9:64: 86:c4:4e:07:12:c0:5b:43:ba:7d:17:8a:a3:f0:3d: 98:32:b9:75:66:f4:f0:1b:2d:94:5b:7c:1c:e6:f3: 04:7f:dd:25:b2:82:a6:41:04:b7:50:93:94:c4:7c: 34:7e:12:7c:bf:33:54:55:47:8c:42:94:40:8e:34: 5f:54:04:1d:9e:8c:57:48:d4:b0:f8:e4:03:db:3f: 68:6c:37:fa:62:14:1c:94:d6:de:f2:2b:68:29:17: 24:6d:f7:b5:b3:18:79:fd:31:5e:7f:4c:be:c0:99: 13:cc:e2:97:2b:dc:96:9c:9a:d0:a7:c5:77:82:67: c9:cb:a9:e7:68:4a:e1:c5:ba:1c:32:0e:79:40:6e: ef:08:d7:a3:b9:5d:1a:df:ce:1a:c7:44:91:4c:d4: 99:c8:88:69:b3:66:2e:b3:06:f1:f4:22:d7:f2:5f: ab:6d Exponent: 65537 (0x10001) ``` 5. Move the key to `apk` trusted keys storage: ```shell sudo mv /tmp/nginx_signing.rsa.pub /etc/apk/keys/ ``` 6. Install the `nginx` package: ```shell sudo apk add nginx@nginx ``` 7. Start NGINX Open Source: ```shell sudo nginx ``` 8. Verify that NGINX Open Source is up and running using the `curl` command: ```shell curl -I 127.0.0.1 ``` Expected output: ```shell HTTP/1.1 200 OK Server: nginx/1.31.6 ``` After installation, the following files are available for configuration and troubleshooting: - Configuration file: `nginx.conf`, located in `/etc/nginx/` - Log files: `access.log` and `error.log`, located in `/var/log/nginx/` 9. If needed, install one or more [dynamic module packages](#repository-contents). The `@nginx` tag should also be specified: ```shell sudo apk add nginx-module-@nginx nginx-module-@nginx ``` Then, enable each module in the `nginx.conf` configuration file using the [`load_module`](https://nginx.org/en/docs/ngx_core_module.html#load_module) directive. The resulting `.so` files are located in the `/usr/lib/nginx/modules` directory. ### Amazon Linux 2 packages Before installing, check if your operating system and architecture are supported, see [Supported distributions and versions](https://nginx.org/en/linux_packages.html#distributions). 1. Install the prerequisites: ```shell sudo yum install yum-utils ``` 2. Set up the `yum` repository. - In the `/etc/yum.repos.d` directory, create the `nginx.repo` file using any text editor, for example, `vi`: ```shell sudo vi /etc/yum.repos.d/nginx.repo ``` - Add the following lines to file, where the `nginx-stable` and `nginx-mainline` elements point to the latest stable or mainline version of NGINX Open Source: ```none [nginx-stable] name=nginx stable repo baseurl=https://nginx.org/packages/amzn2/$releasever/$basearch/ gpgcheck=1 enabled=1 gpgkey=https://nginx.org/keys/nginx_signing.key module_hotfixes=true [nginx-mainline] name=nginx mainline repo baseurl=https://nginx.org/packages/mainline/amzn2/$releasever/$basearch/ gpgcheck=1 enabled=0 gpgkey=https://nginx.org/keys/nginx_signing.key module_hotfixes=true ``` - Save the file. 3. If needed, switch to `nginx-mainline` packages instead of `nginx-stable` that are enabled by default (the `enable=1` parameter): ```shell sudo yum-config-manager --enable nginx-mainline ``` 4. Update the repository: ```shell sudo yum update ``` 5. Install nginx: ```shell sudo yum install nginx ``` When prompted to accept the GPG key, verify that the following three fingerprints match: `8540 A6F1 8833 A80E 9C16 53A4 2FD2 1310 B49F 6B46`, `573B FD6B 3D8F BC64 1079 A6AB ABF5 BD82 7BD9 BF62`, `9E9B E90E ACBC DE69 FE9B 204C BCDC D8A3 8D88 A2B3`, and if so, accept them. 6. Start NGINX Open Source: ```shell sudo nginx ``` 7. Verify that NGINX Open Source is up and running using the `curl` command: ```shell curl -I 127.0.0.1 ``` Expected output: ```shell HTTP/1.1 200 OK Server: nginx/1.31.6 ``` After installation, the following files are available for configuration and troubleshooting: - Configuration file: `nginx.conf`, located in `/etc/nginx/` - Log files: `access.log` and `error.log`, located in `/var/log/nginx/` 8. If needed, install one or more [dynamic module packages](#repository-contents): ```shell sudo yum install nginx-module- ``` Then, enable each module in the `nginx.conf` configuration file using the [`load_module`](https://nginx.org/en/docs/ngx_core_module.html#load_module) directive. The resulting `.so` files are located in the `/usr/lib64/nginx/modules` directory. ### Amazon Linux 2023 packages Before installing, check if your operating system and architecture are supported, see [Supported distributions and versions](https://nginx.org/en/linux_packages.html#distributions). 1. Install the prerequisites: ```shell sudo yum install yum-utils ``` 2. To set up the `yum` repository for Amazon Linux 2023, create the file named `/etc/yum.repos.d/nginx.repo` with the following contents: ```none [nginx-stable] name=nginx stable repo baseurl=https://nginx.org/packages/amzn/2023/$basearch/ gpgcheck=1 enabled=1 gpgkey=https://nginx.org/keys/nginx_signing.key module_hotfixes=true [nginx-mainline] name=nginx mainline repo baseurl=https://nginx.org/packages/mainline/amzn/2023/$basearch/ gpgcheck=1 enabled=0 gpgkey=https://nginx.org/keys/nginx_signing.key module_hotfixes=true ``` By default, the repository for `stable` nginx packages is used. If you would like to use `mainline` nginx packages, run the following command: ```shell sudo yum-config-manager --enable nginx-mainline ``` 3. Install nginx: ```shell sudo yum install nginx ``` When prompted to accept the GPG key, verify that the following three fingerprints match: `8540 A6F1 8833 A80E 9C16 53A4 2FD2 1310 B49F 6B46`, `573B FD6B 3D8F BC64 1079 A6AB ABF5 BD82 7BD9 BF62`, `9E9B E90E ACBC DE69 FE9B 204C BCDC D8A3 8D88 A2B3` and if so, accept them. 4. Start NGINX Open Source: ```shell sudo nginx ``` 5. Verify that NGINX Open Source is up and running using the `curl` command: ```shell curl -I 127.0.0.1 ``` Expected output: ```shell HTTP/1.1 200 OK Server: nginx/1.31.6 ``` After installation, the following files are available for configuration and troubleshooting: - Configuration file: `nginx.conf`, located in `/etc/nginx/` - Log files: `access.log` and `error.log`, located in `/var/log/nginx/` 6. If needed, install one or more [dynamic module packages](#repository-contents): ```shell sudo dnf install nginx-module- ``` Then, enable each module in the `nginx.conf` configuration file using the [`load_module`](https://nginx.org/en/docs/ngx_core_module.html#load_module) directive. The resulting `.so` files are located in the `/usr/lib64/nginx/modules` directory. ## Package contents NGINX has a modular architecture that allows functionality to be included selectively. Some modules are built into the core and are always included in the package. Others are not part of the core, but are compiled at build time using the `--with-` configuration flag. Additionally, some modules, usually those with external dependencies, are distributed as separate packages that can be loaded at runtime as [dynamic modules](#dynamic-modules). Packaging sources and scripts can be found in the [packaging sources repository](https://github.com/nginx/pkg-oss). ### Core modules NGINX core modules are built-in components that provide essential functionality such as configuration parsing, event handling, process management, and HTTP request processing. They are statically compiled into the NGINX binary and cannot be disabled or excluded. |Module Name | Description | | -------------------------| ---------------------------------------------| | [`ngx_core_module`](https://nginx.org/en/docs/ngx_core_module.html) | Internal core functionality. | | [`ngx_http_core_module`](https://nginx.org/en/docs/http/ngx_http_core_module.html) | Essential HTTP functionality ([`location`](https://nginx.org/en/docs/http/ngx_http_core_module.html#location), [`server`](https://nginx.org/en/docs/http/ngx_http_core_module.html#server), [`listen`](https://nginx.org/en/docs/http/ngx_http_core_module.html#listen)). | | [`ngx_http_access_module`](https://nginx.org/en/docs/http/ngx_http_access_module.html)| Access control ([`allow`](https://nginx.org/en/docs/http/ngx_http_access_module.html#allow), [`deny`](https://nginx.org/en/docs/http/ngx_http_access_module.html#deny)). | | [`ngx_http_auth_basic_module`](https://nginx.org/en/docs/http/ngx_http_auth_basic_module.html) | HTTP Basic Auth. | | [`ngx_http_autoindex_module`](https://nginx.org/en/docs/http/ngx_http_autoindex_module.html) |Directory listing. | | [`ngx_http_browser_module`](https://nginx.org/en/docs/http/ngx_http_browser_module.html) | Browser-based conditional logic.| | [`ngx_http_charset_module`](https://nginx.org/en/docs/http/ngx_http_charset_module.html) | Character set conversion.| | [`ngx_http_empty_gif_module`](https://nginx.org/en/docs/http/ngx_http_empty_gif_module.html) | Serves a 1x1 transparent GIF.| | [`ngx_http_fastcgi_module`](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html) | FastCGI backend support. | | [`ngx_http_geo_module`](https://nginx.org/en/docs/http/ngx_http_geo_module.html) | IP-based variable creation. | | [`ngx_http_gzip_module`](https://nginx.org/en/docs/http/ngx_http_gzip_module.html) | Gzip compression. | | [`ngx_http_headers_module`](https://nginx.org/en/docs/http/ngx_http_headers_module.html) | Add/modify response headers.| | [`ngx_http_index_module`](https://nginx.org/en/docs/http/ngx_http_index_module.html) | Default index file (e.g. index.html). | | [`ngx_http_limit_conn_module`](https://nginx.org/en/docs/http/ngx_http_limit_conn_module.html) | Limits concurrent connections. | | [`ngx_http_limit_req_module`](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html) | Limits request rate. | | [`ngx_http_log_module`](https://nginx.org/en/docs/http/ngx_http_log_module.html) | Logging support. | | [`ngx_http_map_module`](https://nginx.org/en/docs/http/ngx_http_map_module.html) | Variable mapping logic. | | [`ngx_http_memcached_module`](https://nginx.org/en/docs/http/ngx_http_memcached_module.html) | Memcached backend. | | [`ngx_http_mirror_module`](https://nginx.org/en/docs/http/ngx_http_mirror_module.html) | Mirrors requests to another location. | | [`ngx_http_proxy_module`](https://nginx.org/en/docs/http/ngx_http_proxy_module.html) | HTTP reverse proxy. | | [`ngx_http_referer_module`](https://nginx.org/en/docs/http/ngx_http_referer_module.html) | Referer-based access control.| | [`ngx_http_rewrite_module`](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html) | URL rewriting. | | [`ngx_http_scgi_module`](https://nginx.org/en/docs/http/ngx_http_scgi_module.html) | SCGI backend. | | [`ngx_http_split_clients_module`](https://nginx.org/en/docs/http/ngx_http_split_clients_module.html) | A/B testing logic. | | [`ngx_http_ssi_module`](https://nginx.org/en/docs/http/ngx_http_ssi_module.html) | Server Side Includes. | | [`ngx_http_upstream_module`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html) | Upstream group and load balancing logic. | | [`ngx_http_userid_module`](https://nginx.org/en/docs/http/ngx_http_userid_module.html) | Cookie-based user ID generation. | | [`ngx_http_uwsgi_module`](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html) | uWSGI backend. | ### Statically linked modules In addition to core modules, the `nginx` package includes other nginx modules that do not require additional libraries to avoid extra dependencies. These modules are compiled at build time and specified using the` --with-` configure option. Unlike [dynamic modules](#dynamic-modules), they cannot be enabled or disabled dynamically after compilation. You can get the list of these modules in the output of the `nginx -V` command output in `configure arguments`. |Module Name | Description | | -------------------------| ---------------------------------------------| | `--with-compat` |Enables dynamic modules compatibility. | | `--with-file-aio` |Enables the use of [asynchronous file I/O](https://nginx.org/en/docs/http/ngx_http_core_module.html#aio) (AIO) on FreeBSD and Linux. | | `--with-threads` | Enables NGINX to use thread pools. For details, see [Thread Pools in NGINX Boost Performance 9x!](https://www.nginx.com/blog/thread-pools-boost-performance-9x/) on the NGINX blog. | |[`--with-http_addition_module`](https://nginx.org/en/docs/http/ngx_http_addition_module.html)| Adds text before and after a response. | |[`--with-http_auth_request_module`](https://nginx.org/en/docs/http/ngx_http_auth_request_module.html)|Implements client authorization based on the result of a subrequest. | |[`--with-http_dav_module`](https://nginx.org/en/docs/http/ngx_http_dav_module.html)|Enables file management automation using the WebDAV protocol. | |[`--with-http_flv_module`](https://nginx.org/en/docs/http/ngx_http_flv_module.html)|Provides pseudo-streaming server-side support for Flash Video (FLV) files. | |[`--with-http_gunzip_module`](https://nginx.org/en/docs/http/ngx_http_gunzip_module.html)|Decompresses responses with `Content-Encoding: gzip` for clients that do not support the _zip_ encoding method. | |[`--with-http_gzip_static_module`](https://nginx.org/en/docs/http/ngx_http_gzip_static_module.html)| Allows sending precompressed files with the **.gz** filename extension instead of regular files. | |[`--with-http_mp4_module`](https://nginx.org/en/docs/http/ngx_http_mp4_module.html)| Provides pseudo-streaming server-side support for MP4 files. | |[`--with-http_random_index_module`](https://nginx.org/en/docs/http/ngx_http_random_index_module.html) | Processes requests ending with the slash character (‘/’) and picks a random file in a directory to serve as an index file. | |[`--with-http_realip_module`](https://nginx.org/en/docs/http/ngx_http_realip_module.html) | Changes the client address to the one sent in the specified header field. | |[`--with-http_secure_link_module`](https://nginx.org/en/docs/http/ngx_http_secure_link_module.html) | Used to check authenticity of requested links, protect resources from unauthorized access, and limit link lifetime. | |[`--with-http_slice_module`](https://nginx.org/en/docs/http/ngx_http_slice_module.html) | Allows splitting a request into subrequests, each subrequest returns a certain range of response. Provides more effective caching of large files. | |[`--with-http_ssl_module`](https://nginx.org/en/docs/http/ngx_http_ssl_module.html) | Enables HTTPS support. Requires an SSL library such as [OpenSSL](https://www.openssl.org/). | |[`--with-http_stub_status_module`](https://nginx.org/en/docs/http/ngx_http_stub_status_module.html)| Provides access to basic status information. Note that NGINX Plus customers do not require this module as they are already provided with extended status metrics and interactive dashboard. | |[`--with-http_sub_module`](https://nginx.org/en/docs/http/ngx_http_sub_module.html) | Modifies a response by replacing one specified string by another. | |[`--with-http_v2_module`](https://nginx.org/en/docs/http/ngx_http_v2_module.html)| Enable support for [HTTP/2](https://datatracker.ietf.org/doc/html/rfc7540). See [The HTTP/2 Module in NGINX](https://www.nginx.com/blog/http2-module-nginx/) on the NGINX blog for details.| |[`--with-http_v3_module`](https://nginx.org/en/docs/http/ngx_http_v3_module.html)| Provides experimental support for [HTTP/3](https://datatracker.ietf.org/doc/html/rfc9114). | |[`--with-mail`](https://nginx.org/en/docs/mail/ngx_mail_core_module.html)| Enables mail proxy functionality. To compile as a separate [dynamic module](/nginx/admin-guide/dynamic-modules/dynamic-modules.md) instead, change the option to `--with-mail=dynamic`. | |[`--with-mail_ssl_module`](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html)| Provides support for a mail proxy server to work with the SSL/TLS protocol. Requires an SSL library such as [OpenSSL](https://www.openssl.org/). | |[`--with-stream`](https://nginx.org/en/docs/stream/ngx_stream_core_module.html) | Enables the TCP and UDP proxy functionality. To compile as a separate [dynamic module](nginx/admin-guide/dynamic-modules/dynamic-modules.md) instead, change the option to `--with-stream=dynamic`. | |[`--with-stream_realip_module`](https://nginx.org/en/docs/stream/ngx_stream_realip_module.html) | Changes the client address and port to the ones sent in the PROXY protocol header. | |[`--with-stream_ssl_module`](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html)| Provides support for a stream proxy server to work with the SSL/TLS protocol. Requires an SSL library such as [OpenSSL](https://www.openssl.org/). | ## Dynamic modules Some modules, especially those with external dependencies, are not included in the `nginx` package. However, they are available as separate packages from the official NGINX repository. After installation they can be connected as a dynamic modules, or shared object (.so) files via the `load_module` directive in the NGINX configuration. |Module Name | Description | Package name | | -------------------------| ---------------------------------------------|---------------| | [`ngx_http_geoip_module`](https://nginx.org/en/docs/http/ngx_http_geoip_module.html) | Creates variables with values depending on the client IP address, using the precompiled MaxMind databases. | `nginx-module-geoip` | | [`ngx_http_image-filter_module`](https://nginx.org/en/docs/http/ngx_http_image_filter_module.html)| Transforms images in JPEG, GIF, PNG, and WebP formats. | `nginx-module-image-filter` | | [`ngx_http_js_module`](https://nginx.org/en/docs/http/ngx_http_js_module.html), [`ngx_stream_js_module`](https://nginx.org/en/docs/stream/ngx_stream_js_module.html)| Extends the server's functionality through JavaScript scripting, enabling the creation of custom server-side logic. | `nginx-module-njs` | | [`ngx_http_perl_module`](https://nginx.org/en/docs/http/ngx_http_perl_module.html) | Implements location and variable handlers in Perl and inserts Perl calls into SSI. | `nginx-module-perl` | | [`ngx_http_xsl_module`](https://nginx.org/en/docs/http/ngx_http_xslt_module.html) | Transforms XML responses using one or more XSLT stylesheets. | `nginx-module-xslt` | | [`ngx_otel_module`](https://nginx.org/en/docs/ngx_otel_module.html) | Provides OpenTelemetry distributed tracing support. | `nginx-module-otel` | | [`ngx_http_acme_module`](https://nginx.org/en/docs/http/ngx_http_acme_module.html) | Implements the automatic certificate management (ACMEv2) protocol. | `nginx-module-acme` | ## Compile and install from source Compiling from source affords more flexibility than using prebuilt packages: you can add specific modules, both official and third party, and apply the latest security patches. ### Install NGINX dependencies Prior to compiling NGINX Open Source from source, you need to install libraries for its dependencies: - [PCRE](http://pcre.org/) – Supports regular expressions. Required by the NGINX [Core](https://nginx.org/en/docs/ngx_core_module.html) and [Rewrite](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html) modules. ```shell wget https://github.com/PCRE2Project/pcre2/releases/download/pcre2-10.43/pcre2-10.43.tar.gz && \ tar -zxf pcre2-10.43.tar.gz && \ cd pcre2-10.43 && \ ./configure && \ make && \ sudo make install ``` - [zlib](http://www.zlib.net/) – Supports header compression. Required by the NGINX [Gzip](https://nginx.org/en/docs/http/ngx_http_gzip_module.html) module. ```shell wget http://zlib.net/zlib-1.3.1.tar.gz && \ tar -zxf zlib-1.3.1.tar.gz && \ cd zlib-1.3.1 && \ ./configure && \ make && \ sudo make install ``` - [OpenSSL](https://www.openssl.org/) – Supports the HTTPS protocol. Required by the NGINX [SSL](https://nginx.org/en/docs/http/ngx_http_ssl_module.html) module and others. ```shell wget https://www.openssl.org/source/openssl-3.0.13.tar.gz #for Ubuntu and Debian # curl -LO https://www.openssl.org/source/openssl-3.0.13.tar.gz #for RHEL-based: tar -zxf openssl-3.0.13.tar.gz cd openssl-3.0.13 ./config --prefix=/usr/local --openssldir=/usr/local/ssl make -j$(nproc) sudo make install ``` ### Download the sources {#sources_download} Download the source files for both the stable and mainline versions from [**nginx.org**](https://www.nginx.org/en/download.html). To download and unpack the source for the latest _mainline_ version, run: ```shell wget https://nginx.org/download/nginx-1.31.6.tar.gz && \ tar zxf nginx-1.31.6.tar.gz && \ cd nginx-1.31.6 ``` To download and unpack source files for the latest _stable_ version, run: ```shell wget https://nginx.org/download/nginx-1.30.5.tar.gz && \ tar zxf nginx-1.30.5.tar.gz && \ cd nginx-1.30.5 ``` ### Configure the build options {#configure} Configure options are specified with the `./configure` script that sets up various NGINX parameters, including paths to source and configuration files, compiler options, connection processing methods, and the list of modules. The script finishes by creating the `Makefile` required to compile the code and install NGINX Open Source. An example of options to the `configure` script: ```shell ./configure \ --sbin-path=/usr/local/nginx/nginx \ --conf-path=/usr/local/nginx/nginx.conf \ --pid-path=/usr/local/nginx/nginx.pid \ --with-pcre=../pcre2-10.43 \ --with-zlib=../zlib-1.3.1 \ --with-http_ssl_module \ --with-stream \ --with-mail=dynamic \ --add-module=/usr/build/nginx-rtmp-module \ --add-dynamic-module=/usr/build/3party_module ``` ### Configure NGINX paths {#configure_paths} The `configure` script allows you to set paths to NGINX binary and configuration files, and to dependent libraries such as PCRE or SSL, in order to link them statically to the NGINX binary. |Parameter | Description | | ---| --- | |`--prefix=` | Directory for NGINX files, and the base location for all relative paths set by the other `configure` script options (excluding paths to libraries) and for the path to the **nginx.conf** configuration file. Default: **/usr/local/nginx**. | |`--sbin-path=` | Name of the NGINX executable file, which is used only during installation. Default: **/sbin/nginx | |`--conf-path=` | Name of the NGINX configuration file. You can, however, always override this value at startup by specifying a different file with the `-c ` option on the `nginx` command line. Default: **conf/nginx.conf | |`--pid-path=` | Name of the **nginx.pid** file, which stores the process ID of the `nginx` master process. After installation, the path to the filename can be changed with the [pid](https://nginx.org/en/docs/ngx_core_module.html#pid) directive in the NGINX configuration file. Default: **/logs/nginx.pid | |`--error-log-path=` | Name of the primary log file for errors, warnings, and diagnostic data. After installation, the filename can be changed with the [error_log](https://nginx.org/en/docs/ngx_core_module.html#error_log) directive in the NGINX configuration file. Default: **/logs/error.log | |`--http-log-path=` | Name of the primary log file for requests to the HTTP server. After installation, the filename can always be changed with the [access_log](https://nginx.org/en/docs/http/ngx_http_log_module.html#access_log) directive in the NGINX configuration file. Default: **/logs/access.log | |`--user=` | Name of the unprivileged user whose credentials are used by the NGINX worker processes. After installation, the name can be changed with the [user](https://nginx.org/en/docs/ngx_core_module.html#user) directive in the NGINX configuration file. Default: `nobody` | |`--group=` | Name of the group whose credentials are used by the NGINX worker processes. After installation, the name can be changed with the [user](https://nginx.org/en/docs/ngx_core_module.html#user) directive in the NGINX configuration file. Default: the value set by the `--user`` option. | |`--with-pcre=` | Path to the source for the PCRE library, which is required for regular expressions support in the [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) directive and the [Rewrite](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html) module. | |`--with-pcre-jit` | Builds the PCRE library with “just-in-time compilation” support (the [pcre_jit](https://nginx.org/en/docs/ngx_core_module.html#pcre_jit) directive). | |`--with-zlib=` | Path to the source for the `zlib` library, which is required by the [Gzip](https://nginx.org/en/docs/http/ngx_http_gzip_module.html) module. | ### GCC options With the `configure` script you can also specify compiler‑related options. |Parameter | Description | | ---| --- | |`--with-cc-opt=""` | Additional parameters that are added to the ``CFLAGS`` variable. When using the system PCRE library under FreeBSD, the mandatory value is `--with-cc-opt="-I /usr/local/include"`. If the number of files supported by `select()` needs to be increased, it can also specified here as in this example: `--with-cc-opt="-D FD_SETSIZE=2048"`. | |`--with-ld-opt=""` | Additional parameters that are used during linking. When using the system PCRE library under FreeBSD, the mandatory value is `--with-ld-opt="-L /usr/local/lib"`. | ### Connection processing methods {#configure_methods} With the `configure` script you can redefine the method for event‑based polling. For more information, see [Connection processing methods](https://nginx.org/en/docs/events.html) in the NGINX reference documentation. |Module Name | Description | | ---| --- | |`--with-select_module`, `--without-select_module` | Enables or disables building a module that enable NGINX to work with the ``select()`` method. The modules is built automatically if the platform does not appear to support more appropriate methods such as `kqueue`, `epoll`, or `/dev/poll`. | |`--with-poll_module`, `--without-poll_module` | Enables or disables building a module that enables NGINX to work with the `poll()` method. The module is built automatically if the platform does not appear to support more appropriate methods such as `kqueue`, `epoll`, or `/dev/poll`. | ### Select the NGINX modules to build NGINX consists of a set of function‑specific _modules_, which are specified with `configure` script along with other build options. Some modules are built by default – they do not have to be specified with the `configure` script. Default modules can however be explicitly excluded from the NGINX binary with the`--without-`option on the `configure` script. Modules not included by default, as well as third‑party modules, must be explicitly specified in the `configure` script together with other build options. Such modules can be linked to NGINX binary either _statically_ (they are then loaded each time NGINX starts) or _dynamically_ (they are loaded only if associated directives are included in the NGINX configuration file. #### Modules built by default If you do not need a module that is built by default, you can disable it by naming it with the `--without-` option on the `configure` script, as in this example which disables the [Empty GIF](https://nginx.org/en/docs/http/ngx_http_empty_gif_module.html) module (should be typed as a single line): ```shell ./configure --sbin-path=/usr/local/nginx/nginx \ --conf-path=/usr/local/nginx/nginx.conf \ --pid-path=/usr/local/nginx/nginx.pid \ --with-http_ssl_module \ --with-stream \ --with-pcre=../pcre2-10.43 \ --with-zlib=../zlib-1.3.1 \ --without-http_empty_gif_module ``` |Module Name | Description | | ---| --- | |[`http_access_module`](https://nginx.org/en/docs/http/ngx_http_access_module.html) | Accepts or denies requests from specified client addresses. | |[`http_auth_basic_module`](https://nginx.org/en/docs/http/ngx_http_auth_basic_module.html) | Limits access to resources by validating the user name and password using the HTTP Basic Authentication protocol. | |[`http_autoindex_module`](https://nginx.org/en/docs/http/ngx_http_autoindex_module.html) | Processes requests ending with the forward-slash character (_/_) and produces a directory listing. | |[`http_browser_module`](https://nginx.org/en/docs/http/ngx_http_browser_module.html) | Creates variables whose values depend on the value of the ``User-Agent`` request header. | |[`http_charset_module`](https://nginx.org/en/docs/http/ngx_http_charset_module.html) | Adds the specified character set to the ``Content-Type`` response header. Can convert data from one character set to another. | |[`http_empty_gif_module`](https://nginx.org/en/docs/http/ngx_http_empty_gif_module.html) | Emits a single-pixel transparent GIF. | |[`http_fastcgi_module`](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html) | Passes requests to a FastCGI server. | |[`http_geo_module`](https://nginx.org/en/docs/http/ngx_http_geo_module.html) | Creates variables with values that depend on the client IP address. | |[`http_gzip_module`](https://nginx.org/en/docs/http/ngx_http_gzip_module.html) | Compresses responses using `gzip`, reducing the amount of transmitted data by half or more. | |[`http_limit_conn_module`](https://nginx.org/en/docs/http/ngx_http_limit_conn_module.html) | Limits the number of connections per a defined key, in particular, the number of connections from a single IP address. | |[`http_limit_req_module`](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html) | Limits the request processing rate per a defined key, in particular, the processing rate of requests coming from a single IP address. | |[`http_map_module`](https://nginx.org/en/docs/http/ngx_http_map_module.html) | Creates variables whose values depend on the values of other variables. | |[`http_memcached_module`](https://nginx.org/en/docs/http/ngx_http_memcached_module.html) | Passes requests to a memcached server. | |[`http_proxy_module`](https://nginx.org/en/docs/http/ngx_http_proxy_module.html) | Passes HTTP requests to another server. | |[`http_referer_module`](https://nginx.org/en/docs/http/ngx_http_referer_module.html) | Blocks requests with invalid values in the `Referer` header. | |[`http_rewrite_module`](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html) | Changes the request URI using regular expressions and return redirects; conditionally selects configurations. Requires the [PCRE](http://pcre.org/) library. | |[`http_scgi_module`](https://nginx.org/en/docs/http/ngx_http_scgi_module.html) | Passes requests to an SCGI server. | |[`http_ssi_module`](https://nginx.org/en/docs/http/ngx_http_ssi_module.html) | Processes SSI (Server Side Includes) commands in responses passing through it. | |[`http_split_clients_module`](https://nginx.org/en/docs/http/ngx_http_split_clients_module.html) | Creates variables suitable for A/B testing, also known as split testing. | |[`http_upstream_hash_module`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#hash) | Enables the generic Hash load-balancing method. | |[`http_upstream_ip_hash_module`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#ip_hash) | Enables the IP Hash load-balancing method. | |[`http_upstream_keepalive_module`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive) | Enables keepalive connections. | |[`http_upstream_least_conn_module`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#least_conn) | Enables the Least Connections load-balancing method. | |[`http_upstream_zone_module`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#zone) | Enables shared memory zones. | |[`http_userid_module`](https://nginx.org/en/docs/http/ngx_http_userid_module.html) | Sets cookies suitable for client identification. | |[`http_uwsgi_module`](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html) | Passes requests to a uwsgi server. | ### Include modules not built by default Many NGINX modules are not built by default, and must be listed on the `configure` command line to be built. The [mail](https://nginx.org/en/docs/mail/ngx_mail_core_module.html), [stream](https://nginx.org/en/docs/stream/ngx_stream_core_module.html), [geoip](https://nginx.org/en/docs/http/ngx_http_geoip_module.html), [image_filter](https://nginx.org/en/docs/http/ngx_http_image_filter_module.html), [perl](https://nginx.org/en/docs/http/ngx_http_perl_module.html) and [xslt](https://nginx.org/en/docs/http/ngx_http_xslt_module.html) modules can be compiled as dynamic. See [Dynamic Modules](/nginx/admin-guide/dynamic-modules/dynamic-modules.md) for details. An example of the `configure` command that includes nondefault modules (should be typed as a single line): ```shell ./configure \ --sbin-path=/usr/local/nginx/nginx \ --conf-path=/usr/local/nginx/nginx.conf \ --pid-path=/usr/local/nginx/nginx.pid \ --with-pcre=../pcre2-10.43 \ --with-zlib=../zlib-1.3.1 \ --with-http_ssl_module \ --with-stream \ --with-mail ``` } |Module Name | Description | | -------------------------| ---------------------------------------------| |`--with-cpp_test_module` | Tests the C++ compatibility of header files.| |`--with-debug` | Enables the [debugging log](/nginx/admin-guide/monitoring/debugging.md)| | `--with-file-aio` |Enables asynchronous I/O. | | `--with-google-perftools` | Allows using [Google Performance tools](https://github.com/gperftools/gperftools) library. | |[`--with-http_addition_module`](https://nginx.org/en/docs/http/ngx_http_addition_module.html)| Adds text before and after a response. | |[`--with-http_auth_request_module`](https://nginx.org/en/docs/http/ngx_http_auth_request_module.html)|Implements client authorization based on the result of a subrequest. | |[`--with-http_dav_module`](https://nginx.org/en/docs/http/ngx_http_dav_module.html)|Enables file management automation using the WebDAV protocol. | |`--with-http_degradation_module`|Allows returning an error when a memory size exceeds the defined value. | |[`--with-http_flv_module`](https://nginx.org/en/docs/http/ngx_http_flv_module.html)|Provides pseudo-streaming server-side support for Flash Video (FLV) files. | |[`--with-http_geoip_module`](https://nginx.org/en/docs/http/ngx_http_geoip_module.html)|Enables creating variables whose values depend on the client IP address. The module uses [MaxMind](http://www.maxmind.com) GeoIP databases. To compile as a separate [dynamic module](/nginx/admin-guide/dynamic-modules/dynamic-modules.md) instead, change the option to `--with-http_geoip_module=dynamic`. | |[`--with-http_gunzip_module`](https://nginx.org/en/docs/http/ngx_http_gunzip_module.html)|Decompresses responses with `Content-Encoding: gzip` for clients that do not support the _zip_ encoding method. | |[`--with-http_gzip_static_module`](https://nginx.org/en/docs/http/ngx_http_gzip_static_module.html)| Allows sending precompressed files with the **.gz** filename extension instead of regular files. | |[`--with-http_image_filter_module`](https://nginx.org/en/docs/http/ngx_http_image_filter_module.html)|T ransforms images in JPEG, GIF, and PNG formats. The module requires the [LibGD](http://libgd.github.io/) library. To compile as a separate [dynamic module](/nginx/admin-guide/dynamic-modules/dynamic-modules.md) instead, change the option to `--with-http_image_filter_module=dynamic`. | |[`--with-http_mp4_module`](https://nginx.org/en/docs/http/ngx_http_mp4_module.html)| Provides pseudo-streaming server-side support for MP4 files. | |[`--with-http_perl_module`](https://nginx.org/en/docs/http/ngx_http_perl_module.html)| Used to implement location and variable handlers in Perl and insert Perl calls into SSI. Requires the [PERL](https://www.perl.org/get.html) library. To compile as a separate [dynamic module](/nginx/admin-guide/dynamic-modules/dynamic-modules.md) instead, change the option to `--with-http_perl_module=dynamic`. | |[`--with-http_random_index_module`](https://nginx.org/en/docs/http/ngx_http_random_index_module.html) | Processes requests ending with the slash character (‘/’) and picks a random file in a directory to serve as an index file. | |[`--with-http_realip_module`](https://nginx.org/en/docs/http/ngx_http_realip_module.html) | Changes the client address to the one sent in the specified header field. | |[`--with-http_secure_link_module`](https://nginx.org/en/docs/http/ngx_http_secure_link_module.html) | Used to check authenticity of requested links, protect resources from unauthorized access, and limit link lifetime. | |[`--with-http_slice_module`](https://nginx.org/en/docs/http/ngx_http_slice_module.html) | Allows splitting a request into subrequests, each subrequest returns a certain range of response. Provides more effective caching of large files. | |[`--with-http_ssl_module`](https://nginx.org/en/docs/http/ngx_http_ssl_module.html) | Enables HTTPS support. Requires an SSL library such as [OpenSSL](https://www.openssl.org/). | |[`--with-http_stub_status_module`](https://nginx.org/en/docs/http/ngx_http_stub_status_module.html)| Provides access to basic status information. Note that NGINX Plus customers do not require this module as they are already provided with extended status metrics and interactive dashboard. | |[`--with-http_sub_module`](https://nginx.org/en/docs/http/ngx_http_sub_module.html) | Modifies a response by replacing one specified string by another. | |[`--with-http_xslt_module`](https://nginx.org/en/docs/http/ngx_http_xslt_module.html)| Transforms XML responses using one or more XSLT stylesheets. The module requires the [Libxml2](http://xmlsoft.org/) and [XSLT](http://xmlsoft.org/XSLT/) libraries. To compile as a separate [dynamic module](/nginx/admin-guide/dynamic-modules/dynamic-modules.md) instead, change the option to `--with-http_xslt_module=dynamic`. | |[`--with-http_v2_module`](https://nginx.org/en/docs/http/ngx_http_v2_module.html)| Enable support for [HTTP/2](https://datatracker.ietf.org/doc/html/rfc7540). See [The HTTP/2 Module in NGINX](https://www.nginx.com/blog/http2-module-nginx/) on the NGINX blog for details. | | [`--with-mail`](https://nginx.org/en/docs/mail/ngx_mail_core_module.html)| Enables mail proxy functionality. To compile as a separate [dynamic module](/nginx/admin-guide/dynamic-modules/dynamic-modules.md) instead, change the option to `--with-mail=dynamic`. | |[`--with-mail_ssl_module`](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html)| Provides support for a mail proxy server to work with the SSL/TLS protocol. Requires an SSL library such as [OpenSSL](https://www.openssl.org/). | | [`--with-stream`](https://nginx.org/en/docs/stream/ngx_stream_core_module.html) | Enables the TCP and UDP proxy functionality. To compile as a separate [dynamic module](/nginx/admin-guide/dynamic-modules/dynamic-modules.md) instead, change the option to `--with-stream=dynamic`. | | [`--with-stream_ssl_module`](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html)| Provides support for a stream proxy server to work with the SSL/TLS protocol. Requires an SSL library such as [OpenSSL](https://www.openssl.org/). | | `--with-threads` | Enables NGINX to use thread pools. For details, see [Thread Pools in NGINX Boost Performance 9x!](https://www.nginx.com/blog/thread-pools-boost-performance-9x/) on the NGINX blog. | ### Include third-party modules You can extend NGINX functionality by compiling NGINX Open Source with your own module or a third‑party module. Some third‑party modules are listed in the [NGINX Wiki](https://nginx.com/resources/wiki/modules/). Use third‑party modules at your own risk as their stability is not guaranteed. #### Statically linked modules Most modules built into NGINX Open Source are _statically linked_: they are built into NGINX Open Source at compile time and are linked to the NGINX binary statically. These modules can be disabled only by recompiling NGINX. To compile NGINX Open Source with a statically linked third‑party module, include the `--add-module=` option on the `configure` command, where `` is the path to the source code (this example is for the [RTMP](https://github.com/arut/nginx-rtmp-module) module): ```shell ./configure ... --add-module=/usr/build/nginx-rtmp-module ``` #### Dynamically linked modules NGINX modules can also be compiled as a shared object (**\*.so** file) and then dynamically loaded into NGINX Open Source at runtime. This provides more flexibility, as the module can be loaded or unloaded at any time by adding or removing the associated [`load_module`](https://nginx.org/en/docs/ngx_core_module.html#load_module) directive in the NGINX configuration file and reloading the configuration. Note that the module itself must support dynamic linking. To compile NGINX Open Source with a dynamically loaded third‑party module, include the `--add-dynamic-module=` option on the `configure` command, where `` is the path to the source code: ```shell ./configure ... --add-dynamic-module= ``` The resulting **\*.so** files are written to the _prefix_**/modules/** directory, where the _prefix_ is a directory for server files such as **/usr/local/nginx/**. To load a dynamic module, add the [`load_module`](https://nginx.org/en/docs/ngx_core_module.html#load_module) directive to the NGINX configuration after installation: ```nginx load_module modules/ngx_mail_module.so; ``` For more information, see [Compiling Third‑Party Dynamic Modules for NGINX and NGINX Plus](https://www.nginx.com/blog/compiling-dynamic-modules-nginx-plus/) on the NGINX blog and [Extending NGINX](https://nginx.com/resources/wiki/extending/) in the Wiki. ### Complete the installation from source - Compile and install the build: ```shell make sudo make install ``` - After the installation is finished, start NGINX Open Source: ```shell sudo nginx ``` -------------------------------------------------------------------------------- title: "Serve Static Content" url: https://docs.nginx.com/nginx/admin-guide/web-server/serving-static-content/index.md description: Configure NGINX and F5 NGINX Plus to serve static content, with type-specific root directories, checks for file existence, and performance optimizations. -------------------------------------------------------------------------------- This section describes how to configure NGINX and F5 NGINX Plus to serve static content, how to define which paths are searched to find requested files, how to set up index files, and how to tune NGINX and NGINX Plus, as well as the kernel, for optimal performance. ## Root Directory and Index Files The [root](https://nginx.org/en/docs/http/ngx_http_core_module.html#root) directive specifies the root directory that will be used to search for a file. To obtain the path of a requested file, NGINX appends the request URI to the path specified by the `root` directive. The directive can be placed on any level within the `http {}`, `server {}`, or `location {}` contexts. In the example below, the `root` directive is defined for a virtual server. It applies to all `location {}` blocks where the `root` directive is not included to explicitly redefine the root: ```nginx server { root /www/data; location / { } location /images/ { } location ~ \.(mp3|mp4) { root /www/media; } } ``` Here, NGINX searches for a URI that starts with `/images/` in the `/www/data/images/` directory in the file system. But if the URI ends with the `.mp3` or `.mp4` extension, NGINX instead searches for the file in the `/www/media/` directory because it is defined in the matching `location` block. If a request ends with a slash, NGINX treats it as a request for a directory and tries to find an index file in the directory. The [index](https://nginx.org/en/docs/http/ngx_http_index_module.html#index) directive defines the index file’s name (the default value is `index.html`). To continue with the example, if the request URI is `/images/some/path/`, NGINX delivers the file `/www/data/images/some/path/index.html` if it exists. If it does not, NGINX returns HTTP code `404 (Not Found)` by default. To configure NGINX to return an automatically generated directory listing instead, include the `on` parameter to the [autoindex](https://nginx.org/en/docs/http/ngx_http_autoindex_module.html#autoindex) directive: ```nginx location /images/ { autoindex on; } ``` You can list more than one filename in the `index` directive. NGINX searches for files in the specified order and returns the first one it finds. ```nginx location / { index index.$geo.html index.htm index.html; } ``` The `$geo` variable used here is a custom variable set through the [geo](https://nginx.org/en/docs/http/ngx_http_geo_module.html#geo) directive. The value of the variable depends on the client’s IP address. To return the index file, NGINX checks for its existence and then makes an internal redirect to the URI obtained by appending the name of the index file to the base URI. The internal redirect results in a new search of a location and can end up in another location as in the following example: ```nginx location / { root /data; index index.html index.php; } location ~ \.php { fastcgi_pass localhost:8000; #... } ``` Here, if the URI in a request is `/path/`, and `/data/path/index.html` does not exist but `/data/path/index.php` does, the internal redirect to `/path/index.php` is mapped to the second location. As a result, the request is proxied. ## Trying Several Options The [try_files](https://nginx.org/en/docs/http/ngx_http_core_module.html#try_files) directive can be used to check whether the specified file or directory exists; NGINX makes an internal redirect if it does, or returns a specified status code if it doesn’t. For example, to check the existence of a file corresponding to the request URI, use the `try_files` directive and the `$uri` variable as follows: ```nginx server { root /www/data; location /images/ { try_files $uri /images/default.gif; } } ``` The file is specified in the form of the URI, which is processed using the `root` or `alias` directives set in the context of the current location or virtual server. In this case, if the file corresponding to the original URI doesn’t exist, NGINX makes an internal redirect to the URI specified by the last parameter, returning `/www/data/images/default.gif`. The last parameter can also be a status code (directly preceded by the equals sign) or the name of a location. In the following example, a `404` error is returned if none of the parameters to the `try_files` directive resolve to an existing file or directory. ```nginx location / { try_files $uri $uri/ $uri.html =404; } ``` In the next example, if neither the original URI nor the URI with the appended trailing slash resolve into an existing file or directory, the request is redirected to the named location which passes it to a proxied server. ```nginx location / { try_files $uri $uri/ @backend; } location @backend { proxy_pass http://backend.example.com; } ``` For more information, watch the [Content Caching](https://www.nginx.com/resources/webinars/content-caching-nginx-plus/) webinar on‑demand to learn how to dramatically improve the performance of a website, and get a deep‑dive into NGINX’s caching capabilities. ## Optimize Performance for Serving Content Loading speed is a crucial factor of serving any content. Making minor optimizations to your NGINX configuration may boost the productivity and help reach optimal performance. ### Enable `sendfile` By default, NGINX handles file transmission itself and copies the file into the buffer before sending it. Enabling the [sendfile](https://nginx.org/en/docs/http/ngx_http_core_module.html#sendfile) directive eliminates the step of copying the data into the buffer and enables direct copying data from one file descriptor to another. Alternatively, to prevent one fast connection from entirely occupying the worker process, you can use the [sendfile_max_chunk](https://nginx.org/en/docs/http/ngx_http_core_module.html#sendfile_max_chunk) directive to limit the amount of data transferred in a single `sendfile()` call (in this example, to `1` MB): ```nginx location /mp3 { sendfile on; sendfile_max_chunk 1m; #... } ``` ### Enable `tcp_nopush` Use the [tcp_nopush](https://nginx.org/en/docs/http/ngx_http_core_module.html#tcp_nopush) directive together with the [sendfile](https://nginx.org/en/docs/http/ngx_http_core_module.html#sendfile) `on;`directive. This enables NGINX to send HTTP response headers in one packet right after the chunk of data has been obtained by `sendfile()`. ```nginx location /mp3 { sendfile on; tcp_nopush on; #... } ``` ### Enable `tcp_nodelay` The [tcp_nodelay](https://nginx.org/en/docs/http/ngx_http_core_module.html#tcp_nodelay) directive allows override of [Nagle’s algorithm](https://en.wikipedia.org/wiki/Nagle's_algorithm), originally designed to solve problems with small packets in slow networks. The algorithm consolidates a number of small packets into a larger one and sends the packet with a `200` ms delay. Nowadays, when serving large static files, the data can be sent immediately regardless of the packet size. The delay also affects online applications (ssh, online games, online trading, and so on). By default, the [tcp_nodelay](https://nginx.org/en/docs/http/ngx_http_core_module.html#tcp_nodelay) directive is set to `on` which means that the Nagle’s algorithm is disabled. Use this directive only for keepalive connections: ```nginx location /mp3 { tcp_nodelay on; keepalive_timeout 65; #... } ``` ### Optimize the Backlog Queue One of the important factors is how fast NGINX can handle incoming connections. The general rule is when a connection is established, it is put into the “listen” queue of a listen socket. Under normal load, either the queue is small or there is no queue at all. But under high load, the queue can grow dramatically, resulting in uneven performance, dropped connections, and increased latency. #### Display the Listen Queue To display the current listen queue, run this command: ```none netstat -Lan ``` The output might be like the following, which shows that in the listen queue on port `80` there are `10` unaccepted connections against the configured maximum of `128` queued connections. This situation is normal. ```none Current listen queue sizes (qlen/incqlen/maxqlen) Listen Local Address 0/0/128 *.12345 10/0/128 *.80 0/0/128 *.8080 ``` In contrast, in the following command the number of unaccepted connections (`192`) exceeds the limit of `128`. This is quite common when a web site experiences heavy traffic. To achieve optimal performance, you need to increase the maximum number of connections that can be queued for acceptance by NGINX in both your operating system and the NGINX configuration. ```none Current listen queue sizes (qlen/incqlen/maxqlen) Listen Local Address 0/0/128 *.12345 192/0/128 *.80 0/0/128 *.8080 ``` #### Tune the Operating System Increase the value of the `net.core.somaxconn` kernel parameter from its default value (`128`) to a value high enough for a large burst of traffic. In this example, it's increased to `4096`. - For FreeBSD, run the command: ```none sudo sysctl kern.ipc.somaxconn=4096 ``` - For Linux: 1. Run the command: ```none sudo sysctl -w net.core.somaxconn=4096 ``` 2. Use a text editor to add the following line to `/etc/sysctl.conf`: ```none net.core.somaxconn = 4096 ``` #### Tune NGINX If you set the `somaxconn` kernel parameter to a value greater than `512`, change the `backlog` parameter to the NGINX [listen](https://nginx.org/en/docs/http/ngx_http_core_module.html#listen) directive to match: ```nginx server { listen 80 backlog=4096; # ... } ``` -------------------------------------------------------------------------------- title: "Single Sign-On with Amazon Cognito" url: https://docs.nginx.com/nginx/deployment-guides/single-sign-on/cognito/index.md description: Enable OpenID Connect-based single sign-on (SSO) for applications proxied by NGINX Plus, using Amazon Cognito as the identity provider (IdP). -------------------------------------------------------------------------------- This guide explains how to enable single sign-on (SSO) for applications being proxied by F5 NGINX Plus. The solution uses OpenID Connect as the authentication mechanism, with [Amazon Cognito](https://aws.amazon.com/cognito/) as the Identity Provider (IdP), and NGINX Plus as the Relying Party, or OIDC client application that verifies user identity. **Note:** This guide applies to [NGINX Plus Release 36](nginx/releases.md#r36) and later. In earlier versions, NGINX Plus relied on an [njs-based solution](#legacy-njs-guide), which required NGINX JavaScript files, key-value stores, and advanced OpenID Connect logic. In the latest NGINX Plus version, the new [OpenID Connect module](https://nginx.org/en/docs/http/ngx_http_oidc_module.html) simplifies this process to just a few directives. ## Prerequisites - An [AWS account](https://aws.amazon.com/premiumsupport/knowledge-center/create-and-activate-aws-account/) - A Cognito **User Pool** - An NGINX Plus [subscription](https://www.f5.com/products/nginx/nginx-plus) and NGINX Plus [Release 36](nginx/releases.md#r36) or later. For installation instructions, see [Installing NGINX Plus](https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-plus/). - A domain name pointing to your NGINX Plus instance, for example, `demo.example.com`. ## Configure Amazon Cognito {#cognito-setup} 1. Open the Amazon Cognito console in the AWS Management Console 2. In the Cognito dashboard, select **Create or open a User Pool**. 3. **Create an App Client** (for example, "nginx-demo-app") and **enable the "Generate client secret"** option. 4. In the App client settings, select **Enable Cognito User Pool** as an **Identity Provider**: - Add a **Callback URL**: `https://demo.example.com/oidc_callback`. - Enable **Authorization code grant**. - In the **OAuth scopes**, check the values of **openid**, **profile**, and **email**. 5. Configure logout URLs to support RP-initiated logout: - Add a **Sign out URL**: `https://demo.example.com/post_logout/`. 6. Copy the following values — you will need them later when configuring NGINX Plus. - **User Pool ID**, for example, `us-east-2_abCdEfGhI` - **App client id** and **App client secret** - **AWS region**, for example, `us-east-2` - **Issuer**, for example, `https://cognito-idp..amazonaws.com/` ### Get the OpenID Connect Discovery URL Check the OpenID Connect Discovery URL. By default, Amazon Cognito publishes the `.well-known/openid-configuration` document at the following address: `https://cognito-idp..amazonaws.com//.well-known/openid-configuration`. 1. Run the following `curl` command in a terminal: ```shell curl https://cognito-idp.us-east-2.amazonaws.com/us-east-2_abCdEfGhI/.well-known/openid-configuration | jq ``` Where: - the `cognito-idp.us-east-2.amazonaws.com` is your Amazon Cognito server address - the `us-east-2_abCdEfGhI` is your User Pool ID - the `/.well-known/openid-configuration` is the default address for Amazon Cognito for document location - the `jq` command (optional) is used to format the JSON output for easier reading and requires the [jq](https://jqlang.github.io/jq/) JSON processor to be installed. The configuration metadata is returned in the JSON format: ```json { ... "issuer": "https://cognito-idp.us-east-2.amazonaws.com/us-east-2_abCdEfGhI", "authorization_endpoint": "https://cognito-idp.us-east-2.amazonaws.com/us-east-2_abCdEfGhI/oauth2/authorize", "token_endpoint": "https://cognito-idp.us-east-2.amazonaws.com/us-east-2_abCdEfGhI/oauth2/token", "jwks_uri": "https://cognito-idp.us-east-2.amazonaws.com/us-east-2_abCdEfGhI/.well-known/jwks.json", "userinfo_endpoint": "https://cognito-idp.us-east-2.amazonaws.com/us-east-2_abCdEfGhI/oauth2/userInfo", "end_session_endpoint": "https://your-cognito-domain.auth.us-east-2.amazoncognito.com/logout", ... } ``` 2. Copy the **issuer** value, you will need it later when configuring NGINX Plus. Typically, the OpenID Connect Issuer for Amazon Cognito is `https://cognito-idp..amazonaws.com/`. **Note:** You will need the values of **Client ID**, **Client Secret**, and **Issuer** in the next steps. ## Set up NGINX Plus {#nginx-plus-setup} With Cognito configured, you can enable OIDC on NGINX Plus. NGINX Plus serves as the Rely Party (RP) application — a client service that verifies user identity. 1. Ensure that you are using the latest version of NGINX Plus by running the `nginx -v` command in a terminal: ```shell nginx -v ``` The output should match NGINX Plus Release 36 or later: ```none nginx version: nginx/1.29.3 (nginx-plus-r36) ``` 2. Ensure that you have the values of the **Client ID**, **Client Secret**, and **Issuer** obtained during [Cognito Configuration](#cognito-setup). 3. In your preferred text editor, open the NGINX configuration file (`/etc/nginx/nginx.conf` for Linux or `/usr/local/etc/nginx/nginx.conf` for FreeBSD). 4. In the [`http {}`](https://nginx.org/en/docs/http/ngx_http_core_module.html#http) context, make sure your public DNS resolver is specified with the [`resolver`](https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver) directive: By default, NGINX Plus re‑resolves DNS records at the frequency specified by time‑to‑live (TTL) in the record, but you can override the TTL value with the `valid` parameter: ```nginx http { resolver 10.0.0.1 ipv4=on valid=300s; # ... } ``` 5. In the [`http {}`](https://nginx.org/en/docs/http/ngx_http_core_module.html#http) context, define the Amazon Cognito provider named `cognito` by specifying the [`oidc_provider {}`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#oidc_provider) context: ```nginx http { resolver 10.0.0.1 ipv4=on valid=300s; oidc_provider cognito { # ... } # ... } ``` 6. In the [`oidc_provider {}`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#oidc_provider) context, specify: - your actual Amazon Cognito **Client ID** obtained in [Amazon Cognito Configuration](#cognito-setup) with the [`client_id`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_id) directive - your **Client Secret** obtained in [Amazon Cognito Configuration](#cognito-setup) with the [`client_secret`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_secret) directive - the **Issuer** URL obtained in [Amazon Cognito Configuration](#cognito-setup) with the [`issuer`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_secret) directive The `issuer` is typically your Amazon Cognito OIDC URL. As a rule, Cognito uses a unique issuer for each User Pool, for example: `https://cognito-idp.us-east-2.amazonaws.com/us-east-2_abCdEfGhI` By default, NGINX Plus creates the metadata URL by appending the `/.well-known/openid-configuration` part to the Issuer URL. If your Issuer is different, you can explicitly specify the metadata document with the [`config_url`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#config_url) directive. - The **logout_uri** is URI that a user visits to start an RP‑initiated logout flow. - The **post_logout_uri** is absolute HTTPS URL where Amazon Cognito should redirect the user after a successful logout. This value **must also be configured** in the Cognito App client sign out URLs. - If the **userinfo** directive is set to `on`, NGINX Plus will fetch `/oauth2/userInfo` from the Amazon Cognito and append the claims from userinfo to the `$oidc_claims_` variables. - PKCE (Proof Key for Code Exchange) is automatically enabled when Cognito's OpenID Connect discovery document advertises the `S256` code challenge method in the `code_challenge_methods_supported` field. You can override this behavior with the [`pkce`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#pkce) directive: set `pkce off;` to disable PKCE even when `S256` is advertised, or `pkce on;` to force PKCE even if the IdP metadata does not list `S256`. - The module automatically selects the client authentication method for the token endpoint based on the provider metadata `token_endpoint_auth_methods_supported`. When only `client_secret_post` is advertised, NGINX Plus uses the `client_secret_post` method and sends the client credentials in the POST body. When both `client_secret_basic` and `client_secret_post` are present, the module prefers HTTP Basic (`client_secret_basic`). - **Note:** All interaction with the IdP is secured exclusively over SSL/TLS, so NGINX must trust the certificate presented by the IdP. By default, this trust is validated against your system’s CA bundle (the default CA store for your Linux or FreeBSD distribution). If the IdP’s certificate is not included in the system CA bundle, you can explicitly specify a trusted certificate or chain with the [`ssl_trusted_certificate`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#ssl_trusted_certificate) directive so that NGINX can validate and trust the IdP’s certificate. ```nginx http { resolver 10.0.0.1 ipv4=on valid=300s; oidc_provider cognito { issuer https://cognito-idp.us-east-2.amazonaws.com/us-east-2_abCdEfGhI; client_id ; client_secret ; logout_uri /logout; post_logout_uri https://demo.example.com/post_logout/; userinfo on; # Optional: PKCE configuration. By default, PKCE is automatically # enabled when the IdP advertises the S256 code challenge method. # pkce on; } # ... } ``` 7. Make sure you have configured a [server](https://nginx.org/en/docs/http/ngx_http_core_module.html#server) that corresponds to `demo.example.com`, and there is a [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) that [points](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass) to your application (see [Step 10](#oidc_app)) at `http://127.0.0.1:8080` that is going to be OIDC-protected: ```nginx http { # ... server { listen 443 ssl; server_name demo.example.com; ssl_certificate /etc/ssl/certs/fullchain.pem; ssl_certificate_key /etc/ssl/private/key.pem; location / { # ... proxy_pass http://127.0.0.1:8080; } } # ... } ``` 8. Protect this [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) with Amazon Cognito OIDC by specifying the [`auth_oidc`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#auth_oidc) directive that will point to the `cognito` configuration specified in the [`oidc_provider {}`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#oidc_provider) context in [Step 5](#cognito-setup-oidc-provider): ```nginx # ... location / { auth_oidc cognito; # ... proxy_pass http://127.0.0.1:8080; } # ... ``` 9. Pass the OIDC claims as headers to the application ([Step 10](#oidc_app)) with the [`proxy_set_header`](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_header) directive. These claims are extracted from the ID token returned by Amazon Cognito: - [`$oidc_claim_sub`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) - a unique `Subject` identifier assigned for each user by Amazon Cognito - [`$oidc_claim_email`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) the e-mail address of the user - [`$oidc_claim_name`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) - the full name of the user - any other OIDC claim using the [`$oidc_claim_ `](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) variable ```nginx # ... location / { auth_oidc cognito; proxy_set_header sub $oidc_claim_sub; proxy_set_header email $oidc_claim_email; proxy_set_header name $oidc_claim_name; proxy_pass http://127.0.0.1:8080; } # ... ``` 10. Provide endpoint for completing logout: ```nginx # ... location /post_logout/ { return 200 "You have been logged out.\n"; default_type text/plain; } # ... ``` 11. Create a simple test application referenced by the `proxy_pass` directive which returns the authenticated user's full name and email upon successful authentication: ```nginx # ... server { listen 8080; location / { return 200 "Hello, $http_name!\nEmail: $http_email\nCognito sub: $http_sub\n"; default_type text/plain; } } ``` 12. Save the NGINX configuration file and reload the configuration: ```nginx nginx -s reload ``` ### Complete Example This configuration example summarizes the steps outlined above. It includes only essential settings such as specifying the DNS resolver, defining the OIDC provider, configuring SSL, and proxying requests to an internal server. ```nginx http { # Use a public DNS resolver for Issuer discovery, etc. resolver 10.0.0.1 ipv4=on valid=300s; oidc_provider cognito { # Typically your Cognito issuer is something like: # https://cognito-idp..amazonaws.com/ issuer https://cognito-idp.us-east-2.amazonaws.com/us-east-2_abCdEfGhI; # Your Cognito "App client id" and "App client secret" client_id ; client_secret ; # RP‑initiated logout logout_uri /logout; post_logout_uri https://demo.example.com/post_logout/; # Fetch userinfo claims userinfo on; # Optional: PKCE configuration # pkce on; } server { listen 443 ssl; server_name demo.example.com; ssl_certificate /etc/ssl/certs/fullchain.pem; ssl_certificate_key /etc/ssl/private/key.pem; location / { # Protect this path with Cognito OIDC auth_oidc cognito; # Forward OIDC claims as headers if desired proxy_set_header sub $oidc_claim_sub; proxy_set_header email $oidc_claim_email; proxy_set_header name $oidc_claim_name; proxy_pass http://127.0.0.1:8080; } location /post_logout/ { return 200 "You have been logged out.\n"; default_type text/plain; } } server { # Simple test upstream server listen 8080; location / { return 200 "Hello, $http_name!\nEmail: $http_email\nCognito sub: $http_sub\n"; default_type text/plain; } } } ``` ### Testing 1. Open https://demo.example.com/ in a browser. You will be automatically redirected to Amazon Cognito login page for your realm. 2. Enter valid Cognito user credentials (those in the assigned user pool). Upon successful sign-in, Cognito redirects you back to NGINX Plus, and you will see the proxied application content (for example, "Hello, Jane Doe!"). 3. Navigate to `https://demo.example.com/logout`. NGINX Plus initiates an RP‑initiated logout; Amazon Cognito ends the session and redirects back to `https://demo.example.com/post_logout/`. 4. Refresh `https://demo.example.com/` again. You should be redirected to Amazon Cognito for a fresh sign‑in, proving the session has been terminated. ## Legacy njs-based Amazon Cognito Solution {#legacy-njs-guide} If you are running NGINX Plus R33 and earlier or if you still need the njs-based solution, refer to the [Legacy njs-based Cognito Guide](nginx/deployment-guides/single-sign-on/oidc-njs/cognito.md) for details. The solution uses the [`nginx-openid-connect`](https://github.com/nginxinc/nginx-openid-connect) GitHub repository and NGINX JavaScript files. ## See Also - [NGINX Plus Native OIDC Module Reference documentation](https://nginx.org/en/docs/http/ngx_http_oidc_module.html) - [Release Notes for NGINX Plus R36](nginx/releases.md#r36) ## Revision History - Version 3 (November 2025) – Updated for NGINX Plus R36; added PKCE configuration (`pkce` directive) and the `client_secret_post` token endpoint authentication method. - Version 2 (August 2025) – Updated for NGINX Plus R35; added RP‑initiated logout (`logout_uri`, `post_logout_uri`, `logout_token_hint`) and `userinfo` support. - Version 1 (March 2025) – Initial version (NGINX Plus Release 34). -------------------------------------------------------------------------------- title: "SSL Termination for TCP Upstream Servers" url: https://docs.nginx.com/nginx/admin-guide/security-controls/terminating-ssl-tcp/index.md description: Terminate SSL/TLS-encrypted traffic from clients, relieving your upstream TCP servers of the computational load. -------------------------------------------------------------------------------- This article explains how to set up SSL termination for F5 NGINX Plus and a load-balanced group of servers that accept TCP connections. ## What is SSL Termination? SSL termination means that NGINX Plus acts as the server-side SSL endpoint for connections with clients: it performs the decryption of requests and encryption of responses that backend servers would otherwise have to do. The operation is called termination because NGINX Plus closes the client connection and forwards the client data over a newly created, unencrypted connection to the servers in an upstream group. In release R6 and later, NGINX Plus performs SSL termination for TCP connections as well as HTTP connections. ## Prerequisites - [NGINX Plus R6](nginx/releases.md#r6 ) or later - A load-balanced [upstream group](nginx/admin-guide/load-balancer/tcp-udp-load-balancer.md) with several TCP servers - SSL certificates and a private key (obtained or self-generated) ## Obtaining SSL Certificates First, you will need to obtain server certificates and a private key and put them on the server. A certificate can be obtained from a trusted certificate authority (CA) or generated using an SSL library such as [OpenSSL](https://www.openssl.org/). ## Configuring NGINX Plus To configure SSL termination, add the following directives to the NGINX Plus configuration: ### Enabling SSL To enable SSL, specify the `ssl` parameter of the [listen](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#listen) directive for the TCP server that passes connections to an upstream server group: ```nginx stream { server { listen 12345 ssl; proxy_pass backend; #... } } ``` ### Adding SSL Certificates To add SSL certificates, specify the path to the certificates (which must be in the PEM format) with the [ssl_certificate](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_certificate) directive, and specify the path to the private key in the [ssl_certificate_key](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_certificate_key) directive: ```nginx server { #... ssl_certificate /etc/ssl/certs/server.crt; ssl_certificate_key /etc/ssl/certs/server.key; } ``` Additionally, the [ssl_protocols](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_protocols) and [ssl_ciphers](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_ciphers) directives can be used to limit connections and to include only the strong versions and ciphers of SSL/TLS: ```nginx server { #... ssl_protocols TLSv1.2 TLSv1.3; ssl_ciphers HIGH:!aNULL:!MD5; } ``` The [ssl_ciphers](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_ciphers) directive tells NGINX to inform the SSL library which ciphers it prefers. ## Speeding up Secure TCP Connections Implementing SSL/TLS can significantly impact server performance, because the SSL handshake operation (a series of messages the client and server exchange to verify that the connection is trusted) is quite CPU-intensive. The default timeout for the SSL handshake is 60 seconds and it can be redefined with the [ssl_handshake_timeout](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_handshake_timeout) directive. We do not recommend setting this value too low or too high, as that might result either in handshake failure or a long time to wait for the handshake to complete: ```nginx server { #... ssl_handshake_timeout 10s; } ``` ### Optimizing the SSL Session Cache Creating a cache of the session parameters that apply to each SSL/TLS connection reduces the number of handshakes and thus can significantly improve performance. Caching is set with the [ssl_session_cache](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_session_cache) directive: ```nginx ssl_session_cache; ``` By default, NGINX Plus uses the `built-in` type of the session cache, which means the cache built in your SSL library. This is not optimal, because such a cache can be used by only one worker process and can cause memory fragmentation. Set the `ssl_session_cache` directive to `shared` to share the cache among all worker processes, which speeds up later connections because the connection setup information is already known: ```nginx ssl_session_cache shared:SSL:1m; ``` As a reference, a 1-MB shared cache can hold approximately 4,000 sessions. By default, NGINX Plus retains cached session parameters for five minutes. Increasing the value of the [ssl_session_timeout](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_session_timeout) to several hours can improve performance because reusing cached session parameters reduces the number of time-consuming handshakes. When you increase the timeout, the cache needs to be bigger to accommodate the larger number of cached parameters that results. For the 4-hour timeout in the following example, a 20-MB cache is appropriate: ```nginx ssl_session_timeout 4h; ``` If the timeout length is increased, you need a larger cache to store sessions, for example, 20 MB: ```nginx server { #... ssl_session_cache shared:SSL:20m; ssl_session_timeout 4h; } ``` These lines create an in-memory cache of 20 MB to store session information, and instruct NGINX Plus to reuse session parameters from the cache for 4 hours after the moment they were added. ### Session Tickets Session tickets are an alternative to the session cache. Session information is stored on the client side, eliminating the need for a server-side cache to store session information. When a client resumes interaction with the backend server, it presents the session ticket and re-negotiation is not necessary. Set the [ssl_session_tickets](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_session_tickets) directive to `on`: ```nginx server { #... ssl_session_tickets on; } ``` When using session tickets for an upstream group, each upstream server must be initialized with the same session key. It’s a best practice to change session keys frequently, we recommend that you implement a mechanism to rotate the shared key across all upstream servers: ```nginx server { #... ssl_session_tickets on; ssl_session_ticket_key /etc/ssl/session_ticket_keys/current.key; ssl_session_ticket_key /etc/ssl/session_ticket_keys/previous.key; } ``` ## Complete Example ```nginx stream { upstream stream_backend { server backend1.example.com:12345; server backend2.example.com:12345; server backend3.example.com:12345; } server { listen 12345 ssl; proxy_pass stream_backend; ssl_certificate /etc/ssl/certs/server.crt; ssl_certificate_key /etc/ssl/certs/server.key; ssl_protocols TLSv1.2 TLSv1.3; ssl_ciphers HIGH:!aNULL:!MD5; ssl_session_cache shared:SSL:20m; ssl_session_timeout 4h; ssl_handshake_timeout 30s; #... } } ``` In this example, the directives in the `server` block instruct NGINX Plus to terminate and decrypt secured TCP traffic from clients and pass it unencrypted to the upstream group `stream_backend` which consists of three servers. The `ssl` parameter of the [listen](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#listen) directive instructs NGINX Plus to accept SSL connections. When a clent requests a secure TCP connection, NGINX Plus starts the handshake process, which uses the PEM-format certificate specified by the [ssl_certificate](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_certificate) directive, the certificate’s private key specified by the [ssl_certificate_key](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_certificate_key) directive, and the protocols and cyphers listed by the [ssl_protocols](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_protocols) and [ssl_ciphers](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_ciphers) directives. As soon as the secure TCP connection is established, NGINX Plus caches the session parameters according to the [ssl_session_cache](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_session_cache) directive. In the example, the session cache is shared between all worker processes (the `shared` parameter), is 20 MB in size (the `20m` parameter), and retains each SSL session for reuse for 4 hours (the [ssl_session_timeout](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_session_timeout) directive). -------------------------------------------------------------------------------- title: "TCP and UDP Load Balancing" url: https://docs.nginx.com/nginx/admin-guide/load-balancer/tcp-udp-load-balancer/index.md description: This chapter describes how to use F5 NGINX Plus and NGINX Open Source to proxy and load balance TCP and UDP traffic. -------------------------------------------------------------------------------- ## Introduction {#intro} [Load balancing](https://www.f5.com/glossary/load-balancer) is a traffic management technique that efficiently distributes network traffic across multiple backend servers. NGINX Plus and NGINX Open Source provide Layer 4 (transport layer) proxying and load balancing for both Transmission Control Protocol (TCP) and User Datagram Protocol (UDP) traffic. TCP is a connection-oriented protocol used by services such as LDAP, MySQL, and RTMP. UDP is commonly used by non-transactional, latency-sensitive services such as DNS, syslog, and RADIUS. For Layer 7, or application-layer HTTP load balancing, see [HTTP Load Balancing](http-load-balancer.md). ## Prerequisites - NGINX Plus (no extra build steps required) or [NGINX Open Source](https://nginx.org/en/download.html) (either built with the [`--with-stream`](https://nginx.org/en/docs/configure.html) configuration flag or loaded as a dynamic module). - An application, database, or service that communicates over TCP or UDP - Upstream servers, each running the same instance of the application, database, or service ## Configuring reverse proxy {#proxy_pass} First, you will need to configure _reverse proxy_ so that NGINX Plus or NGINX Open Source can forward TCP connections or UDP datagrams from clients to an upstream group or a proxied server. Open the NGINX configuration file and perform the following steps: 1. Create a top‑level [`stream {}`](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#stream) block: ```nginx stream { # ... } ``` 2. Define one or more [`server {}`](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#server) configuration blocks for each virtual server in the top‑level `stream {}` context. 3. Within the `server {}` configuration block for each server, include the [`listen`](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#listen) directive to define the _IP address_ and/or _port_ on which the server listens. For UDP traffic, also include the [`udp`](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#udp) parameter. As TCP is the default protocol for the `stream` context, there is no `tcp` parameter to the `listen` directive: ```nginx stream { server { listen 12345; # ... } server { listen 53 udp; # ... } # ... } ``` 4. Include the [`proxy_pass`](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_pass) directive to define the proxied server or an upstream group to which the server forwards traffic: ```nginx stream { server { listen 12345; #TCP traffic will be forwarded to the "stream_backend" upstream group proxy_pass stream_backend; } server { listen 12346; #TCP traffic will be forwarded to the specified server proxy_pass backend.example.com:12346; } server { listen 53 udp; #UDP traffic will be forwarded to the "dns_servers" upstream group proxy_pass dns_servers; } # ... } ``` 5. If the proxy server has several network interfaces, you can optionally configure NGINX to use a particular source IP address when connecting to an upstream server. This may be useful if a proxied server behind NGINX is configured to accept connections from particular IP networks or IP address ranges. Include the [`proxy_bind`](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_bind) directive and the IP address of the appropriate network interface: ```nginx stream { # ... server { listen 127.0.0.1:12345; proxy_pass backend.example.com:12345; proxy_bind 127.0.0.1:12345; } } ``` 6. Optionally, you can tune the size of two in‑memory buffers where NGINX can put data from both the client and upstream connections. If there is a small volume of data, the buffers can be reduced which may save memory resources. If there is a large volume of data, the buffer size can be increased to reduce the number of socket read/write operations. As soon as data is received on one connection, NGINX reads it and forwards it over the other connection. The buffers are controlled with the [`proxy_buffer_size`](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_buffer_size) directive: ```nginx stream { # ... server { listen 127.0.0.1:12345; proxy_pass backend.example.com:12345; proxy_buffer_size 16k; } } ``` ## Configuring TCP or UDP load balancing {#upstream} To configure load balancing: 1. Create a group of servers, or an _upstream group_ whose traffic will be load balanced. Define one or more [`upstream {}`](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#upstream) configuration blocks in the top‑level [`stream {}`](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#stream) context and set the name for the upstream group, for example, `stream_backend` for TCP servers and `dns_servers` for UDP servers: ```nginx stream { upstream stream_backend { # ... } upstream dns_servers { # ... } # ... } ``` Make sure that the name of the upstream group is referenced by a [`proxy_pass`](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_pass) directive, like those configured [above](#proxy_pass) for reverse proxy. 2. Populate the upstream group with _upstream servers_. Within the [`upstream {}`](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#upstream) block, add a [`server`](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#server) directive for each upstream server, specifying its IP address or hostname (which can resolve to multiple IP addresses) and an _obligatory_ port number. Note that you do not define the protocol for each server, because that is defined for the entire upstream group by the parameter you include on the `listen` directive in the `server` block, which you have created [earlier](#proxy_pass). ```nginx stream { upstream stream_backend { server backend1.example.com:12345; server backend2.example.com:12345; server backend3.example.com:12346; # ... } upstream dns_servers { server 192.168.136.130:53; server 192.168.136.131:53; # ... } # ... } ``` 3. Configure the load‑balancing method used by the upstream group. You can specify one of the following methods: - Round Robin – By default, NGINX uses the Round Robin algorithm to load balance traffic, directing it sequentially to the servers in the configured upstream group. Because it is the default method, there is no `round‑robin` directive; simply create an `upstream {}` configuration block in the top‑level `stream {}` context and add `server` directives as described in the previous step. - [Least Connections](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#least_conn) – NGINX selects the server with the smaller number of current active connections. - [Least Time](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#least_time) – NGINX selects the server with the lowest average latency and the least number of active connections. The method used to calculate lowest average latency depends on which of the following parameters is included on the `least_time` directive: - `connect` – Time to connect to the upstream server - `first_byte` – Time to receive the first byte of data - `last_byte` – Time to receive the full response from the server ```nginx upstream stream_backend { least_time first_byte; server backend1.example.com:12345; server backend2.example.com:12345; server backend3.example.com:12346; } ``` - [Hash](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#hash) – NGINX selects the server based on a user‑defined key, for example, the source IP address ([`$remote_addr`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_remote_addr)): ```nginx upstream stream_backend { hash $remote_addr; server backend1.example.com:12345; server backend2.example.com:12345; server backend3.example.com:12346; } ``` The `Hash` load‑balancing method is also used to configure _session persistence_. As the hash function is based on client IP address, connections from a given client are always passed to the same server unless the server is down or otherwise unavailable. Specify an optional `consistent` parameter to apply the [ketama](http://www.last.fm/user/RJ/journal/2007/04/10/rz_libketama_-_a_consistent_hashing_algo_for_memcache_clients) consistent hashing method: ```nginx hash $remote_addr consistent; ``` - [Random](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#random) – Each connection will be passed to a randomly selected server. If the `two` parameter is specified, first, NGINX randomly selects two servers taking into account server weights, and then chooses one of these servers using the specified method: - `least_conn` – The least number of active connections - `least_time=connect` (NGINX Plus) – The time to connect to the upstream server ([`$upstream_connect_time`](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#var_upstream_connect_time)) - `least_time=first_byte` (NGINX Plus) – The least average time to receive the first byte of data from the server ([`$upstream_first_byte_time`](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#var_upstream_first_byte_time)) - `least_time=last_byte` (NGINX Plus) – The least average time to receive the last byte of data from the server ([`$upstream_session_time`](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#var_upstream_session_time)) ```nginx upstream stream_backend { random two least_time=last_byte; server backend1.example.com:12345; server backend2.example.com:12345; server backend3.example.com:12346; server backend4.example.com:12346; } ``` The **Random** load balancing method should be used for distributed environments where multiple load balancers are passing requests to the same set of backends. For environments where the load balancer has a full view of all requests, use other load balancing methods, such as round robin, least connections and least time. 5. Optionally, for each upstream server specify server‑specific parameters including [maximum number of connections](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#max_conns), [server weight](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#weight), and so on: ```nginx upstream stream_backend { hash $remote_addr consistent; server backend1.example.com:12345 weight=5; server backend2.example.com:12345; server backend3.example.com:12346 max_conns=3; } upstream dns_servers { least_conn; server 192.168.136.130:53; server 192.168.136.131:53; # ... } ``` An alternative approach is to proxy traffic to a single server instead of an upstream group. If you identify the server by hostname, and configure the hostname to resolve to multiple IP addresses, then NGINX load balances traffic across the IP addresses using the `Round Robin` algorithm. In this case, you _must_ specify the server’s port number in the `proxy_pass` directive and _must not_ specify the protocol before IP address or hostname: ```nginx stream { # ... server { listen 12345; proxy_pass backend.example.com:12345; } } ``` ## Configuring health checks {#health} NGINX can continually test your TCP or UDP upstream servers, avoid the servers that have failed, and gracefully add the recovered servers into the load‑balanced group. See [TCP Health Checks](nginx/admin-guide/load-balancer/tcp-health-check.md) for instructions how to configure health checks for TCP. See [UDP Health Checks](nginx/admin-guide/load-balancer/udp-health-check.md) for instructions how to configure health checks for UDP. ## On-the-fly configuration Upstream server groups can be easily reconfigured on-the-fly using NGINX Plus REST API. Using this interface, you can view all servers in an upstream group or a particular server, modify server parameters, and add or remove upstream servers. To enable on-the-fly configuration: 1. Create the top-level `http {}` block or make sure it is present in your configuration: ```nginx http { # ... } ``` 2. Create a location for configuration requests, for example, _api_: ```nginx http { server { location /api { # ... } } } ``` 3. In this location specify the [`api`](https://nginx.org/en/docs/http/ngx_http_api_module.html#api) directive: ```nginx http { server { location /api { api; # ... } } } ``` 4. By default, the NGINX Plus API provides read-only access to data. The `write=on` parameter enables read/write access so that changes can be made to upstreams: ```nginx http { server { location /api { api write=on; # ... } } } ``` 5. Limit access to this location with [`allow`](https://nginx.org/en/docs/http/ngx_http_access_module.html#allow) and [`deny`](https://nginx.org/en/docs/http/ngx_http_access_module.html#deny) directives: ```nginx http { server { location /api { api write=on; allow 127.0.0.1; # permit access from localhost deny all; # deny access from everywhere else } } } ``` 6. When the API is enabled in the write mode, it is recommended restricting access to `PATCH`, `POST`, and `DELETE` methods to particular users. This can be done by implementing [HTTP basic authentication](https://nginx.org/en/docs/http/ngx_http_auth_basic_module.html): ```nginx http { server { location /api { limit_except GET { auth_basic "NGINX Plus API"; auth_basic_user_file /path/to/passwd/file; } api write=on; allow 127.0.0.1; deny all; } } } ``` 7. Create a _shared memory zone_ for the group of upstream servers so that all worker processes can use the same configuration. To do this, in the top-level [`stream {}`](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#stream) block, find the target upsteam group, add the [`zone`](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#zone) directive to the upstream server group and specify the zone name (here, **stream_backend**) and the amount of memory (64 KB): ```nginx stream { upstream stream_backend { zone backend 64k; # ... } } ``` ### On-the-fly configuration example ```nginx stream { # ... # Configuration of an upstream server group upstream appservers { zone appservers 64k; server appserv1.example.com:12345 weight=5; server appserv2.example.com:12345 fail_timeout=5s; server backup1.example.com:12345 backup; server backup2.example.com:12345 backup; } server { # Server that proxies connections to the upstream group proxy_pass appservers; health_check; } } http { # ... server { # Location for API requests location /api { limit_except GET { auth_basic "NGINX Plus API"; auth_basic_user_file /path/to/passwd/file; } api write=on; allow 127.0.0.1; deny all; } } } ``` Here, access to the location is allowed only from the localhost address (`127.0.0.1`). Access from all other IP addresses is denied. To pass a configuration command to NGINX, send an API command by any method, for example, with curl. For example, to add a new server to the server group, send a `POST` request: ```shell curl -X POST -d '{ \ "server": "appserv3.example.com:12345", \ "weight": 4 \ }' -s 'http://127.0.0.1/api/9/stream/upstreams/appservers/servers' ``` To remove a server from the server group, send a `DELETE` request: ```shell curl -X DELETE -s 'http://127.0.0.1/api/9/stream/upstreams/appservers/servers/0' ``` To modify a parameter for a specific server, send a `PATCH` request: ```shell curl -X PATCH -d '{ "down": true }' -s 'http://127.0.0.1/api/9/http/upstreams/appservers/servers/0' ``` ## Example of TCP and UDP load-balancing configuration {#example} This is a configuration example of TCP and UDP load balancing with NGINX: ```nginx stream { upstream stream_backend { least_conn; server backend1.example.com:12345 weight=5; server backend2.example.com:12345 max_fails=2 fail_timeout=30s; server backend3.example.com:12345 max_conns=3; } upstream dns_servers { least_conn; server 192.168.136.130:53; server 192.168.136.131:53; server 192.168.136.132:53; } server { listen 12345; proxy_pass stream_backend; proxy_timeout 3s; proxy_connect_timeout 1s; } server { listen 53 udp; proxy_pass dns_servers; } server { listen 12346; proxy_pass backend4.example.com:12346; } } ``` In this example, all TCP and UDP proxy‑related functionality is configured inside the [`stream`](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#stream) block, just as settings for HTTP requests are configured in the [`http`](https://nginx.org/en/docs/http/ngx_http_core_module.html#http) block. There are two named [`upstream`](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#upstream) blocks, each containing three servers that host the same content as one another. In the [`server`](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#server) for each server, the server name is followed by the obligatory port number. Connections are distributed among the servers according to the [Least Connections](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#least_conn) load‑balancing method: a connection goes to the server with the fewest number of active connections. The three [`server`](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#server) blocks define three virtual servers: - The first server listens on port 12345 and proxies all TCP connections to the **stream_backend** group of upstream servers. Note that the [`proxy_pass`](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_pass) directive defined in the context of the `stream` module must not contain a protocol. Two optional timeout parameters are specified: the [`proxy_connect_timeout`](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_connect_timeout) directive sets the timeout required for establishing a connection with a server in the **stream_backend** group. The [`proxy_timeout`](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_timeout) directive sets a timeout used after proxying to one of the servers in the **stream_backend** group has started. - The second server listens on port 53 and proxies all UDP datagrams (the `udp` parameter to the `listen` directive) to an upstream group called **dns_servers**. If the `udp` parameter is not specified, the socket listens for TCP connections. - The third virtual server listens on port 12346 and proxies TCP connections to **backend4.example.com**, which can resolve to several IP addresses that are load balanced with the Round Robin method. ## Compatibility note {#compat} - [TCP proxying](https://nginx.org/en/docs/stream/ngx_stream_core_module.html) and [load balancing](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html) are supported in NGINX Plus since [Release 5](nginx/releases.md#r5) and in NGINX Open Source since 1.9.0. - [UDP proxying](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#udp) and load balancing are supported in NGINX Plus since [Release 9](nginx/releases.md#r9) and in NGINX Open Source since 1.9.13. - The [`least_time`](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#least_time) load-balancing method is supported in NGINX Plus since [Release 6](nginx/releases.md#r6) and in NGINX Open Source since 1.31.0. ## See also - [TCP Health Checks](tcp-health-check.md) - [UDP Health Checks](udp-health-check.md) - [Load Balancing DNS Traffic with NGINX and NGINX Plus](https://www.f5.com/company/blog/nginx/load-balancing-dns-traffic-nginx-plus) - [TCP/UDP Load Balancing with NGINX: Overview, Tips, and Tricks](https://blog.nginx.org/blog/tcp-load-balancing-udp-load-balancing-nginx-tips-tricks) -------------------------------------------------------------------------------- title: "Debugging NGINX" url: https://docs.nginx.com/nginx/admin-guide/monitoring/debugging/index.md description: Troubleshoot problems and track down bugs in an NGINX or F5 NGINX Plus deployment, with the debugging binary, debug logging, and core dumps. -------------------------------------------------------------------------------- ## Introduction Debugging helps to identify a bug in the program code if something goes wrong. It is generally used in developing or testing third-party or experimental modules. NGINX debugging features include the debugging log and creation of a core dump file with its further backtrace. ### Configuring NGINX Binary For Debugging First, you will need to enable debugging in NGINX binary. NGINX Plus already provides you with _nginx-debug_ binary while NGINX Open Source requires recompilation. ### Configuring F5 NGINX Plus Binary Starting from [Release 8](nginx/releases.md), NGINX Plus ships the _nginx-debug_ binary together with the standard binary. To enable debugging in NGINX Plus, you will need to switch from _nginx_ to _nginx-debug_ binary. Open terminal and run the command: ```shell service nginx stop && service nginx-debug start ``` When finished, [enable](#error_log) the debugging log in the configuration file. ### Compiling NGINX Open Source Binary To enable debugging in NGINX Open Source, you will need to recompile it with the `--with-debug` flag specified in the configure script. To compile NGINX Open Source with the debug support: 1. Download and unpack NGINX source files, go to the directory with the source files. See [Downloading the Sources](nginx/admin-guide/installing-nginx/installing-nginx-open-source.md). 2. Get the list of NGINX configure arguments. Run the command: ```shell nginx -V 2>&1 | grep arguments ``` 3. Add the `--with-debug` option to the list of configure commands and run the configure script: ```shell ./configure --with-debug ``` 4. Compile and install NGINX: ```shell sudo make sudo make install ``` 5. Restart NGINX. ### NGINX and Debugging Symbols Debug symbols helps obtain additional information for debugging, such as functions, variables, data structures, source file and line number information. NGINX by default is compiled with the “_-g_” flag that includes debug symbols. However, if you get the “No symbol table info available” error when you run a [backtrace](#backtrace), then debugging symbols are missing and you will need to recompile NGINX with support of debugging symbols. The exact set of compiler flags depends on the compiler. For example, for the GCC compiler system: - include debugging symbols with the “_-g_” flag - make the debugger output easier to understand by disabling compiler optimization with the “_-O0_” flag: ```shell ./configure --with-debug --with-cc-opt='-O0 -g' ... ``` ## Enabling Debug Logging in NGINX Configuration The debugging log records errors and any debugging-related information and is disabled by default. To enable it, make sure NGINX is compiled to support debugging (see [Configuring NGINX Binary For Debugging](#enable)) and then enable it in NGINX configuration file with the `debug` parameter of the [error_log](https://nginx.org/en/docs/ngx_core_module.html#error_log) directive. The debugging log may be written to a file, an allocated _buffer_ in memory, _stderr_ output, or to _syslog_. It is recommended enabling the debugging log on the ”_main_“ level of NGINX configuration to get the full picture of what’s going on. ### Writing the Debugging Log to a File Writing the debugging log to a file may slow down performance under high load. Also note that the file can grow very large and quickly eat up disk space. To reduce the negative impact, you can configure the debugging log to be written into a memory buffer, or set the debugging log for particular IP addresses. See [Writing the Debugging Log to Memory](#error_log_memory) and [Debug Log for Selected IPs](#error_log_ip) for details. To enable writing the debugging log to a file: 1. Make sure your NGINX is configured with the `--with-debug` configuration option. Run the command and check if the output contains the `--with-debug` line: ```shell nginx -V 2>&1 | grep -- '--with-debug' ``` 2. Open NGINX configuration file: ```shell sudo vi /etc/nginx/nginx.conf ``` 3. Find the [error_log](https://nginx.org/en/docs/ngx_core_module.html#error_log) directive which is by default located in the `main` context, and change the logging level to `debug`. If necessary, change the path to the log file: ```shell error_log /var/log/nginx/error.log debug; ``` 4. Save the configuration and exit the configuration file. ### Writing the Debugging Log to Memory The debugging log can be written to a memory using a cyclic buffer. The advantage is that logging on the debug level will not have significant impact on performance under high load. To enable writing the debug log to memory: 1. Make sure your NGINX is configured with the `--with-debug` configuration option. Run the command and check if the output contains the `--with-debug` line: ```shell nginx -V 2>&1 | grep -- '--with-debug' ``` 2. In NGINX configuration file, enable a memory buffer for debug logging with the [error_log](https://nginx.org/en/docs/ngx_core_module.html#error_log) directive specified in the `main` context: ```nginx error_log memory:32m debug; ... http { ... } ``` #### Extracting Debug Log From Memory The log can be extracted from the memory buffer using a script executed in the GDB debugger. To extract the debugging log from memory: 1. Obtain the PID of NGINX worker process: ```shell ps axu |grep nginx ``` 2. Launch the GDB debugger: ```shell sudo gdb -p ``` 3. Copy the script, paste it to GDB and press “Enter”. The script will save the log in the _debug_log.txt_ file located in the current directory: ```nginx set $log = ngx_cycle->log while $log->writer != ngx_log_memory_writer set $log = $log->next end set $buf = (ngx_log_memory_buf_t *) $log->wdata dump binary memory debug_log.txt $buf->start $buf->end ``` 4. Quit GDB by pressing CTRL+D. 5. Open the file “_debug_log.txt_” located in the current directory: ```shell sudo less debug_log.txt ``` ### Debug Log for Selected IPs It is possible to enable the debugging log for a particular IP address or a range of IP addresses. Logging particular IPs may useful in a production environment as it will not negatively affect performance. The IP address is specified in the [debug_connection](https://nginx.org/en/docs/ngx_core_module.html#debug_connection) directive within the [events](https://nginx.org/en/docs/ngx_core_module.html#events) block; the directive can be defined more than once: ```nginx error_log /path/to/log; ... events { debug_connection 192.168.1.1; debug_connection 192.168.10.0/24; } ``` ### Debug Log for Each Virtual Host Generally, the [error_log](https://nginx.org/en/docs/ngx_core_module.html#error_log) directive is specified in the `main` context and thus is applied to all other contexts including [server](https://nginx.org/en/docs/http/ngx_http_core_module.html#server) and [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location). But if there is another `error_log` directive specified inside a particular `server` or a `location` block, the global settings will be overridden and such `error_log` directive will set its own path to the log file and the level of logging. To set up the debugging log for a particular virtual host, add the `error_log` directive inside a particular `server` block, in which set a new path to the log file and the `debug` logging level: ```nginx error_log /path1/to/log debug; ... http { ... server { error_log /path2/to/log debug; ... } } ``` To disable the debugging log per a particular virtual host, specify the `error_log` directive inside a particular `server` block, and specify a path to the log file only: ```nginx error_log /path/to/log debug; ... http { ... server { error_log /path/to/log; ... } } ``` ## Enabling Core Dumps A core dump file can help identify and fix problems that are causing NGINX to crash. A core dump file may contain sensitive information such as passwords and private keys, so ensure that they are treated securely. In order to create a core dump file, they must be enabled in both the operating system and the NGINX configuration file. ### Enabling Core Dumps in the Operating System Perform the following steps in your operating system: 1. Specify a working directory in which a core dump file will be saved, for example, “_/tmp/cores_”: ```shell mkdir /tmp/cores ``` 2. Make sure the directory is writable by NGINX worker process: ```shell sudo chown root:root /tmp/cores sudo chmod 1777 /tmp/cores ``` 3. Disable the limit for the maximum size of a core dump file: ```shell sudo prlimit --core=unlimited:unlimited --pid $(cat /run/nginx.pid) ``` If the operation ends up with “Cannot modify limit: operation not permitted”, run the command: ```shell sudo sh -c "ulimit -c unlimited && exec su $LOGNAME" ``` 4. Enable core dumps for the _setuid_ and _setgid_ processes. For CentOS 7.0, Debian 8.2, Ubuntu 14.04, run the commands: ```shell echo "/tmp/cores/core.%e.%p" | sudo tee /proc/sys/kernel/core_pattern sudo sysctl -w fs.suid_dumpable=2 sysctl -p ``` For FreeBSD, run the commands: ```shell sudo sysctl kern.sugid_coredump=1 sudo sysctl kern.corefile=/tmp/cores/%N.core.%P ``` ### Enabling Core Dumps in NGINX Configuration To enable core dumps in the NGINX configuration file: 1. Open the NGINX configuration file: ```shell sudo vi /usr/local/etc/nginx/nginx.conf ``` 2. Define a directory that will keep core dump files with the [working_directory](https://nginx.org/en/docs/ngx_core_module.html#working_directory) directive. The directive is specified on the _main_ configuration level: ```nginx working_directory /tmp/cores/; ``` 3. Make sure the directory exists and is writable by NGINX worker process. Open terminal and run the commands: ```shell sudo chown root:root /tmp/cores sudo chmod 1777 /tmp/cores ``` 4. Specify the maximum possible size of the core dump file with the [worker_rlimit_core](https://nginx.org/en/docs/ngx_core_module.html#worker_rlimit_core) directive. The directive is also specified on the `main` configuration level. If the core dump file size exceeds the value, the core dump file will not be created. ```nginx worker_rlimit_core 500M; ``` Example: ```nginx worker_processes auto; error_log /var/log/nginx/error.log debug; working_directory /tmp/cores/; worker_rlimit_core 500M; events { ... } http { ... } ``` With these settings, a core dump file will be created in the “_/tmp/cores/_” directory, and only if its size does not exceed 500 megabytes. ## Obtaining Backtrace From a Core Dump File Backtraces provide information from a core dump file about what was wrong when a program crashed. To get a backtrace from a core dump file: 1. Open a core dump file with the GDB debugger using the pattern: ```shell sudo gdb ``` 2. Type-in the “_backtrace_ command to get a stack trace from the time of the crash: ```nginx (gdb) backtrace ``` If the “_backtrace_” command resulted with the “No symbol table info available” message, you will need to recompile NGINX binary to include debugging symbols. See [NGINX and Debugging Symbols](#compile_symbols). ## Dumping NGINX Configuration From a Running Process You can extract the current NGINX configuration from the master process in memory. This can be useful when you need to: - verify which configuration has been loaded - restore a previous configuration if the version on disk has been accidentally removed or overwritten The configuration dump can be obtained with a GDB script provided that your NGINX has the debug support. 1. Make sure your NGINX is built with the debug support (the `--with-debug` configure option in the list of the configure arguments). Run the command and check if the output contains the `--with-debug` line: ```shell nginx -V 2>&1 | grep -- '--with-debug' ``` 2. Obtain the PID of NGINX worker process: ```shell ps axu | grep nginx ``` 3. Launch the GDB debugger: ```shell sudo gdb -p ``` 4. Copy and paste the script to GDB and press “Enter”. The script will save the configuration in the _nginx_conf.txt_ file in the current directory: ```nginx set $cd = ngx_cycle->config_dump set $nelts = $cd.nelts set $elts = (ngx_conf_dump_t*)($cd.elts) while ($nelts-- > 0) set $name = $elts[$nelts]->name.data printf "Dumping %s to nginx_conf.txt\n", $name append memory nginx_conf.txt \ $elts[$nelts]->buffer.start $elts[$nelts]->buffer.end end ``` 5. Quit GDB by pressing _CTRL+D_. 6. Open the file _nginx_conf.txt_ located in the current directory: ```shell sudo vi nginx.conf.txt ``` ## Asking for help When asking for help with debugging, please provide the following information: 1. NGINX version, compiler version, and configure parameters. Run the command: ```shell nginx -V ``` 2. Current full NGINX configuration. See [Dumping NGINX Configuration From a Running Process](#configdump) 3. The debugging log. See [Enabling Debug Logging in NGINX Configuration](#error_log) 4. The obtained backtrace. See [Enabling Core Dumps](#coredump), [Obtaining Backtrace](#backtrace) -------------------------------------------------------------------------------- title: "HTTP Health Checks" url: https://docs.nginx.com/nginx/admin-guide/load-balancer/http-health-check/index.md description: Monitor the health of HTTP servers in an upstream group by sending periodic health checks, including customizable active health checks in F5 NGINX Plus. -------------------------------------------------------------------------------- ## Introduction NGINX and F5 NGINX Plus can continually test your upstream servers, avoid the servers that have failed, and gracefully add the recovered servers into the load‑balanced group. ## Prerequisites - For passive health checks, [NGINX Open Source](https://nginx.org/en/) or [NGINX Plus](https://www.f5.com/products/nginx/nginx-plus) - For active health checks and the [live activity monitoring dashboard](nginx/admin-guide/monitoring/live-activity-monitoring.md), NGINX Plus - A load‑balanced group of [HTTP upstream servers](http-load-balancer.md) ## Passive Health Checks For passive health checks, NGINX and NGINX Plus monitor transactions as they happen, and try to resume failed connections. If the transaction still cannot be resumed, NGINX Open Source and NGINX Plus mark the server as unavailable and temporarily stop sending requests to it until it is marked active again. The conditions under which an upstream server is marked unavailable are defined for each upstream server with parameters to the [`server`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#server) directive in the `upstream` block: - [`fail_timeout`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#fail_timeout) – Sets the time during which a number of failed attempts must happen for the server to be marked unavailable, and also the time for which the server is marked unavailable (default is 10 seconds). - [`max_fails`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#max_fails) – Sets the number of failed attempts that must occur during the `fail_timeout` period for the server to be marked unavailable (default is 1 attempt). In the following example, if NGINX fails to send a request to a server or does not receive a response from it 3 times in 30 seconds, it marks the server as unavailable for 30 seconds: ```nginx upstream backend { server backend1.example.com; server backend2.example.com max_fails=3 fail_timeout=30s; } ``` Note that if there is only a single server in a group, the `fail_timeout` and `max_fails` parameters are ignored and the server is never marked unavailable. ### Server Slow Start A recently recovered server can be easily overwhelmed by connections, which may cause the server to be marked as unavailable again. Slow start allows an upstream server to gradually recover its weight from zero to its nominal value after it has been recovered or became available. This can be done with the [`slow_start`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#slow_start) parameter of the upstream [`server`](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#match) directive: ```nginx upstream backend { server backend1.example.com slow_start=30s; server backend2.example.com; server 192.0.0.1 backup; } ``` Note that if there is only a single server in a group, the `slow_start` parameter is ignored and the server is never marked unavailable. Slow start is exclusive to NGINX Plus. ## Active Health Checks NGINX Plus can periodically check the health of upstream servers by sending special health‑check requests to each server and verifying the correct response. To enable active health checks: 1. In the [`location`](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) that passes requests ([`proxy_pass`](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass)) to an upstream group, include the [`health_check`](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#health_check) directive: ```nginx server { location / { proxy_pass http://backend; health_check; } } ``` This snippet defines a server that passes all requests (`location /`) to the upstream group called `backend`. It also enables advanced health monitoring with the [`health_check`](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#health_check) directive: by default, every five seconds NGINX Plus sends a request for "**/**" to each server in the `backend` group. If any communication error or timeout occurs (the server responds with a status code outside the range from `200` through `399`) the health check fails. The server is marked as unhealthy, and NGINX Plus does not send client requests to it until it once again passes a health check. Optionally you can specify another port for health checks, for example, for monitoring health of many services on the same host. Specify a new port with the [`port`](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#health_check_port) parameter of the [`health_check`](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#health_check) directive: ```nginx server { location / { proxy_pass http://backend; health_check port=8080; } } ``` 2. In the upstream server group, define a shared memory zone with the [`zone`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#zone) directive: ```nginx http { upstream backend { zone backend 64k; server backend1.example.com; server backend2.example.com; server backend3.example.com; server backend4.example.com; } } ``` The zone is shared among all worker processes and stores the configuration of the upstream group. This [enables](/nginx/admin-guide/load-balancer/http-load-balancer.md#sharing-data-with-multiple-worker-processes) the worker processes to use the same set of counters to keep track of responses from the servers in the group. The defaults for active health checks can be overridden with parameters to the `health_check` directive: ```nginx location / { proxy_pass http://backend; health_check interval=10 fails=3 passes=2; } ``` Here, the [`interval`](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#health_check_interval) parameter increases the delay between health checks from the default 5 seconds to 10 seconds. The [`fails`](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#health_check_fails) parameter requires the server to fail three health checks to be marked as unhealthy (up from the default one). Finally, the [`passes`](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#health_check_passes) parameter means the server must pass two consecutive checks to be marked as healthy again instead of the default one. You can also enable connection caching with the [`keepalive_time`](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#health_check_keepalive_time) parameter - in case of TLS upstreams the full TLS handshake won't happen for every health check probe and the connection can be reused during the specified period of time: ```nginx location / { proxy_http_version 1.1; proxy_set_header Connection ""; proxy_pass https://backend; health_check interval=1 keepalive_time=60s; } ``` ### Specifying the Requested URI Use the [`uri`](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#health_check_uri) parameter of the [`health_check`](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#health_check) directive to set the URI to request in a health check: ```nginx location / { proxy_pass http://backend; health_check uri=/some/path; } ``` The specified URI is appended to the server domain name or IP address set for the server in the `upstream` block. For the first server in the sample `backend` group declared above, a health check requests the URI **"http://backend1.example.com/some/path"**. ### Defining Custom Conditions You can set custom conditions that the response must satisfy for the server to pass the health check. The conditions are defined in a [`match`](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#match) block, which is referenced in the [`match`](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#health_check_match) parameter of the [`health_check`](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#health_check) directive. 1. On the `http {}` level, specify the `match` `{}` block and name it, for example, `server_ok`: ```nginx http { #... match server_ok { # tests are here } } ``` 2. Refer to the block from the [`health_check`](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#health_check) directive by specifying the [`match`](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#health_check_match) parameter and the name of the [`match`](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#match) block: ```nginx http { #... match server_ok { status 200-399; body !~ "maintenance mode"; } server { #... location / { proxy_pass http://backend; health_check match=server_ok; } } } ``` Here the health check is passed if the status code of the response is in the range `200`–`399`, and its body does not contain the string `maintenance mode`. The `match` directive enables NGINX Plus to check the status code, header fields, and the body of a response. Using this directive it is possible to verify whether the status is in a specified range, whether a response includes a header, or whether the header or body matches a regular expression. The `match` directive can contain one status condition, one body condition, and multiple header conditions. A response must satisfy all conditions defined in `match` block for the server to pass the health check. For example, the following `match` directive matches responses that have status code `200`, the exact value `text/html` in the `Content-Type` header, and the text `Welcome to nginx!` in the body: ```nginx match welcome { status 200; header Content-Type = text/html; body ~ "Welcome to nginx!"; } ``` The following example uses the exclamation point (`!`) to define characteristics the response must not have to pass the health check. In this case, the health check passes when the status code is something other than `301`, `302`, `303`, or `307`, and there is no `Refresh` header. ```nginx match not_redirect { status ! 301-303 307; header ! Refresh; } ``` ### Mandatory Health Checks By default, when a new server is added to an upstream group, NGINX Plus considers it healthy and sends traffic to it immediately. But for some servers, particularly if they were added through the API interface or through DNS resolution, it would be good to perform health check first before allowing them to handle traffic. The [`mandatory`](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#health_check_mandatory) parameter requires every newly added server to pass all configured health checks before NGINX Plus sends traffic to it. When combined with [`slow start`](#slow_start), it gives a new server more time to connect to databases and “warm up” before being asked to handle their full share of traffic. Mandatory health checks can be marked as persistent, so that the previous state is remembered when reloading configuration. Specify the [`persistent`](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#health_check_persistent) parameter together with the [`mandatory`](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#health_check_mandatory) parameter: ```nginx upstream my_upstream { zone my_upstream 64k; server backend1.example.com slow_start=30s; } server { location / { proxy_pass http://my_upstream; health_check mandatory persistent; } } ``` Here the `mandatory` and `persistent` parameters of the `health_check` directive and the `slow_start` parameter of the `server` directive are specified. Servers that are added to the upstream group using the API or DNS interfaces are marked as unhealthy and receive no traffic until they pass the health check; at that point they start receiving a gradually increasing amount of traffic over a span of 30 seconds. If NGINX Plus configuration is reloaded and before reload the server was marked as healthy, mandatory health check are not performed and the server state is considered to be `up`. Health checks can also be enabled for non-HTTP protocols, such as [FastCGI](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html), [memcached](https://nginx.org/en/docs/http/ngx_http_memcached_module.html), [SCGI](https://nginx.org/en/docs/http/ngx_http_scgi_module.html), [uwsgi](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html), and also for [TCP and UDP](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#health_check). -------------------------------------------------------------------------------- title: "Installing NGINX Plus AMIs on Amazon EC2" url: https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-plus-amazon-web-services/index.md description: Install F5 NGINX Plus on Amazon Web Services (AWS), to provide sophisticated Layer 7 load balancing for your apps running on Amazon Linux, RHEL, and Ubuntu. -------------------------------------------------------------------------------- NGINX, Inc. participates in the Amazon Web Services (AWS) Partner Network as a Standard Technology Partner. We offer Amazon Machine Images (AMIs) for use in the Amazon Elastic Compute Cloud (EC2), available at the AWS Marketplace for several operating systems, including Amazon Linux, Red Hat Enterprise Linux, and Ubuntu. The AMIs contain the following components: - Latest version of [NGINX Plus](https://www.f5.com/products/nginx/nginx-plus), optimized for use on Amazon EC2 - Pre-packaged software for building highly available (HA) NGINX Plus configurations ## Install the F5 NGINX Plus AMI To quickly set up an NGINX Plus environment on AWS: 1. Follow the instructions in [Getting Started with Amazon EC2 Linux Instances](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EC2_GetStarted.html) to sign up on AWS and get more information about EC2 itself. 2. Proceed to the product page for the appropriate AMI at the AWS Marketplace, and launch the AMI. - [NGINX Plus – Amazon Linux AMI HVM](https://aws.amazon.com/marketplace/seller-profile?id=741df81b-dfdc-4d36-b8da-945ea66b522c) - [NGINX Plus – Red Hat Enterprise Linux 7 AMI HVM](https://aws.amazon.com/marketplace/seller-profile?id=741df81b-dfdc-4d36-b8da-945ea66b522c) - [NGINX Plus – Red Hat Enterprise Linux 6 AMI HVM](https://aws.amazon.com/marketplace/seller-profile?id=741df81b-dfdc-4d36-b8da-945ea66b522c) - [NGINX Plus – Ubuntu AMI HVM](https://aws.amazon.com/marketplace/seller-profile?id=741df81b-dfdc-4d36-b8da-945ea66b522c) Click the **Continue to Subscribe** button to proceed to the **Launch on EC2** page. 3. Select the type of launch by clicking the appropriate tab (**1-Click Launch**, **Manual Launch**, or **Service Catalog**). Choose the desired options for billing, instance size, and so on, and click the **Accept Software Terms…** button. 4. When configuring the firewall rules, add a rule to accept web traffic on TCP ports 80 and 443 (this happens automatically if you launch from the **1-Click Launch** tab). 5. As soon as the new EC2 instance launches, NGINX Plus starts automatically and serves a default **index.html** page. To view the page, use a web browser to access the public DNS name of the new instance. You can also check the status of the NGINX Plus server by logging into the EC2 instance and running this command: ```nginx /etc/init.d/nginx status ``` See [NGINX Plus on the AWS Cloud Quick Start](https://aws.amazon.com/about-aws/whats-new/2017/08/quick-start-update-deploy-nginx-plus-on-the-aws-cloud/) deployment guide for details. ## What If I Need Help? If you encounter any problems with NGINX Plus configuration, documentation is available at [nginx.org](https://nginx.org/en/docs/) and in the [NGINX Plus Admin Guide](/nginx/admin-guide/installing-nginx/). Customers who purchase an NGINX Plus AMI at the AWS Marketplace are eligible for the AWS support provided by the NGINX, Inc. engineering team. To activate support, submit the [AMI Support Activation](https://www.nginx.com/ami-support-activation/) form (you need your AWS account number). When you request support, we’ll ask you to provide the AWS account number that you registered, along with the IDs of your EC2 instances in some cases. -------------------------------------------------------------------------------- title: "NGINX Reverse Proxy" url: https://docs.nginx.com/nginx/admin-guide/web-server/reverse-proxy/index.md description: Configure NGINX as a reverse proxy for HTTP and other protocols, with support for modifying request headers and fine-tuned buffering of responses. -------------------------------------------------------------------------------- This article describes the basic configuration of a proxy server. You will learn how to pass a request from NGINX to proxied servers over different protocols, modify client request headers that are sent to the proxied server, and configure buffering of responses coming from the proxied servers. ## Introduction Proxying is typically used to distribute the load among several servers, seamlessly show content from different websites, or pass requests for processing to application servers over protocols other than HTTP. ## Passing a Request to a Proxied Server When NGINX proxies a request, it: 1. Sends the request to a specified proxy server 1. Fetches the response 1. Sends the response back to the client. It is possible to proxy requests to an HTTP server (another NGINX server or any other server) or a non-HTTP server (which can run an application developed with a specific framework, such as PHP or Python) using a specified protocol. Supported protocols include [FastCGI](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html), [uwsgi](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html), [SCGI](https://nginx.org/en/docs/http/ngx_http_scgi_module.html), and [memcached](https://nginx.org/en/docs/http/ngx_http_memcached_module.html). To pass a request to an HTTP proxied server, the [proxy_pass](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass) directive is specified inside a [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location). For example: ```nginx location /some/path/ { proxy_pass http://www.example.com/link/; } ``` This example configuration results in passing all requests processed in this location to the proxied server at the specified address. This address can be specified as a domain name or an IP address. The address may also include a port: ```nginx location ~ \.php { proxy_pass http://127.0.0.1:8000; } ``` Note that in the first example above, the address of the proxied server is followed by a URI, `/link/`. If the URI is specified along with the address, it replaces the part of the request URI that matches the location parameter. For example, the request with the `/some/path/page.html` URI will be proxied to `http://www.example.com/link/page.html`. However, if the address is specified without a URI, or it is not possible to determine the part of URI to be replaced, the full request URI is passed (possibly, modified). To pass a request to a non-HTTP proxied server, the appropriate `**_pass` directive should be used: - [fastcgi_pass](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_pass) passes a request to a FastCGI server - [uwsgi_pass](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_pass) passes a request to a uwsgi server - [scgi_pass](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_pass) passes a request to an SCGI server - [memcached_pass](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_pass) passes a request to a memcached server Note that in these cases, the rules for specifying addresses may be different. You may also need to pass additional parameters to the server (see the [reference documentation](https://nginx.org/en/docs/) for more detail). The [proxy_pass](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass) directive can also be used to pass requests to a [named group](https://nginx.org/en/docs/http/load_balancing.html#algorithms) of servers. In this case, requests are distributed among the servers in the group according to the [specified method](https://www.nginx.com/resources/admin-guide/load-balancer/). ## Passing Request Headers By default, NGINX modifies two header fields in proxied requests, “Host” and “Connection”, and eliminates the header fields whose values are empty strings. “Host” is set to the value of the `$proxy_host` variable, and “Connection” is set to `close`. To change these settings, as well as modify other header fields, use the [proxy_set_header](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_header) directive. This directive can be specified in a [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) or higher. It can also be specified in a particular [server](https://nginx.org/en/docs/http/ngx_http_core_module.html#server) context or in the [http](https://nginx.org/en/docs/http/ngx_http_core_module.html#http) block. For example: ```nginx location /some/path/ { proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_pass http://localhost:8000; } ``` In this configuration the “Host” field is set to the [$host](https://nginx.org/en/docs/http/ngx_http_core_module.html#variables) variable. To prevent a header field from being passed to the proxied server, set it to an empty string as follows: ```nginx location /some/path/ { proxy_set_header Accept-Encoding ""; proxy_pass http://localhost:8000; } ``` ## Configuring Buffers By default NGINX buffers responses from proxied servers. A response is stored in the internal buffers and is not sent to the client until the whole response is received. Buffering helps to optimize performance with slow clients, which can waste proxied server time if the response is passed from NGINX to the client synchronously. However, when buffering is enabled NGINX allows the proxied server to process responses quickly, while NGINX stores the responses for as much time as the clients need to download them. The directive that is responsible for enabling and disabling buffering is [proxy_buffering](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffering). By default it is set to `on` and buffering is enabled. The [proxy_buffers](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffers) and [proxy_buffer_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffer_size) directives control how NGINX stores and buffers data. The [proxy_buffers](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffers) directive controls the size and the number of buffers allocated for a request. The first part of the response from a proxied server is stored in a separate buffer, the size of which is set with the [proxy_buffer_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffer_size) directive. This part usually contains a comparatively small response header and can be made smaller than the buffers for the rest of the response. In the following example, the default number of buffers is increased and the size of the buffer for the first portion of the response is made smaller than the default. ```nginx location /some/path/ { proxy_buffers 16 4k; proxy_buffer_size 2k; proxy_pass http://localhost:8000; } ``` If buffering is disabled, the response is sent to the client synchronously while it is receiving it from the proxied server. This behavior may be desirable for fast interactive clients that need to start receiving the response as soon as possible. To disable buffering in a specific location, place the [proxy_buffering](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffering) directive in the [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) with the `off` parameter, as follows: ```nginx location /some/path/ { proxy_buffering off; proxy_pass http://localhost:8000; } ``` In this case NGINX uses only the buffer configured by [proxy_buffer_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffer_size) to store the current part of a response. A common use of a reverse proxy is to provide load balancing. See [HTTP Load Balancing](nginx/admin-guide/load-balancer/http-load-balancer.md) and [TCP and UDP Load Balancing](nginx/admin-guide/load-balancer/tcp-udp-load-balancer.md). ## Choosing an Outgoing IP Address If your proxy server has several network interfaces, sometimes you might need to choose a particular source IP address for connecting to a proxied server or an upstream. This may be useful if a proxied server behind NGINX is configured to accept connections from particular IP networks or IP address ranges. In such cases, you can specify the [proxy_bind](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_bind) directive and the IP address of the necessary network interface: ```nginx location /app1/ { proxy_bind 127.0.0.1; proxy_pass http://example.com/app1/; } location /app2/ { proxy_bind 127.0.0.2; proxy_pass http://example.com/app2/; } ``` The IP address can be also specified with a variable. For example, the [`$server_addr`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_server_addr) variable passes the IP address of the network interface that accepted the request: ```nginx location /app3/ { proxy_bind $server_addr; proxy_pass http://example.com/app3/; } ``` -------------------------------------------------------------------------------- title: "Restricting Access with HTTP Basic Authentication" url: https://docs.nginx.com/nginx/admin-guide/security-controls/configuring-http-basic-authentication/index.md description: Control access using HTTP Basic authentication, and optionally in combination with IP address-based access control. -------------------------------------------------------------------------------- ## Introduction You can restrict access to your website or some parts of it by implementing a username/password authentication. Usernames and passwords are taken from a file created and populated by a password file creation tool, for example, `apache2-utils`. HTTP Basic authentication can also be combined with other access restriction methods, for example restricting access by [IP address](denylisting-ip-addresses.md) or [geographical location](controlling-access-by-geoip.md). ## Prerequisites - F5 NGINX Plus or NGINX Open Source - Password file creation utility such as `apache2-utils` (Debian, Ubuntu) or `httpd-tools` (RHEL/CentOS/Oracle Linux). ## Creating a Password File To create username-password pairs, use a password file creation utility, for example, `apache2-utils` or `httpd-tools` 1. Verify that `apache2-utils` (Debian, Ubuntu) or `httpd-tools` (RHEL/CentOS/Oracle Linux) is installed. 2. Create a password file and a first user. Run the `htpasswd` utility with the `-c` flag (to create a new file), the file pathname as the first argument, and the username as the second argument: ```shell sudo htpasswd -c /etc/apache2/.htpasswd user1 ``` Press Enter and type the password for **user1** at the prompts. 3. Create additional user-password pairs. Omit the `-c` flag because the file already exists: ```shell sudo htpasswd /etc/apache2/.htpasswd user2 ``` 4. You can confirm that the file contains paired usernames and hashed passwords: ```shell $ cat /etc/apache2/.htpasswd user1: user2: user3: ``` ## Configuring NGINX and NGINX Plus for HTTP Basic Authentication 1. Inside a location that you are going to protect, specify the [auth_basic](https://nginx.org/en/docs/http/ngx_http_auth_basic_module.html#auth_basic) directive and give a name to the password-protected area. The name of the area will be shown in the username/password dialog window when asking for credentials: ```nginx location /api { auth_basic "Administrator’s Area"; #... } ``` 2. Specify the [auth_basic_user_file](https://nginx.org/en/docs/http/ngx_http_auth_basic_module.html#auth_basic_user_file) directive with a path to the _.htpasswd_ file that contain user/password pairs: ```nginx location /api { auth_basic "Administrator’s Area"; auth_basic_user_file /etc/apache2/.htpasswd; } ``` Alternatively, you you can limit access to the whole website with basic authentication but still make some website areas public. In this case, specify the `off` parameter of the [auth_basic](https://nginx.org/en/docs/http/ngx_http_auth_basic_module.html#auth_basic) directive that cancels inheritance from upper configuration levels: ```nginx server { ... auth_basic "Administrator’s Area"; auth_basic_user_file conf/htpasswd; location /public/ { auth_basic off; } } ``` ## Combining Basic Authentication with Access Restriction by IP Address HTTP basic authentication can be effectively combined with access restriction by IP address. You can implement at least two scenarios: - a user must be both authenticated and have a valid IP address - a user must be either authenticated, or have a valid IP address 1. Allow or deny access from particular IP addresses with the [allow](https://nginx.org/en/docs/http/ngx_http_access_module.html#allow) and [deny](https://nginx.org/en/docs/http/ngx_http_access_module.html#deny) directives: ```nginx location /api { #... deny 192.168.1.2; allow 192.168.1.1/24; allow 127.0.0.1; deny all; } ``` Access will be granted only for the `192.168.1.1/24` network excluding the `192.168.1.2` address. Note that the `allow` and `deny` directives will be applied in the order they are defined. 2. Combine restriction by IP and HTTP authentication with the [satisfy](https://nginx.org/en/docs/http/ngx_http_core_module.html#satisfy) directive. If you set the directive to `all`, access is granted if a client satisfies both conditions. If you set the directive to `any`, access is granted if if a client satisfies at least one condition: ```nginx location /api { #... satisfy all; deny 192.168.1.2; allow 192.168.1.1/24; allow 127.0.0.1; deny all; auth_basic "Administrator’s Area"; auth_basic_user_file conf/htpasswd; } ``` ## Complete Example The example shows how to protect your status area with simple authentication combined with access restriction by IP address: ```nginx http { server { listen 192.168.1.23:8080; root /usr/share/nginx/html; location /api { api; satisfy all; deny 192.168.1.2; allow 192.168.1.1/24; allow 127.0.0.1; deny all; auth_basic "Administrator’s Area"; auth_basic_user_file /etc/apache2/.htpasswd; } } } ``` When you access your status page, you are prompted to log in: ![The "Authentication Required" prompt](/nginx/images/auth-required.png) If the provided name and password do not match the password file, you get the `401 (Authorization Required)` error. -------------------------------------------------------------------------------- title: "Single Sign-On with Microsoft Active Directory FS" url: https://docs.nginx.com/nginx/deployment-guides/single-sign-on/active-directory-federation-services/index.md description: Enable OpenID Connect-based single sign-on (SSO) for applications proxied by NGINX Plus, using Microsoft AD FS as the identity provider (IdP). -------------------------------------------------------------------------------- This guide explains how to enable single sign-on (SSO) for applications being proxied by F5 NGINX Plus. The solution uses OpenID Connect as the authentication mechanism, with [Microsoft Active Directory Federation Services](https://docs.microsoft.com/en-us/windows-server/identity/active-directory-federation-services) (AD FS) as the Identity Provider (IdP) and NGINX Plus as the Relying Party (RP), or OIDC client application that verifies user identity. **Note:** This guide applies to [NGINX Plus Release 36](nginx/releases.md#r36) and later. In earlier versions, NGINX Plus relied on an [njs-based solution](#legacy-njs-guide), which required NGINX JavaScript files, key-value stores, and advanced OpenID Connect logic. In the latest NGINX Plus version, the new [OpenID Connect module](https://nginx.org/en/docs/http/ngx_http_oidc_module.html) simplifies this process to just a few directives. ## Prerequisites - A Microsoft AD FS instance, either on-premises or in [Azure](https://learn.microsoft.com/en-us/windows-server/identity/ad-fs/deployment/how-to-connect-fed-azure-adfs), with administrator privileges. - An NGINX Plus [subscription](https://www.f5.com/products/nginx/nginx-plus) and NGINX Plus [Release 36](nginx/releases.md#r36) or later. For installation instructions, see [Installing NGINX Plus](https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-plus/). - A domain name pointing to your NGINX Plus instance, for example, `demo.example.com`. ## Configure the AD FS Server {#adfs-setup} [Microsoft Active Directory Federation Services](https://docs.microsoft.com/en-us/windows-server/identity/active-directory-federation-services) (AD FS) serves as the Identity Provider. ### Create an AD FS Application 1. In AD FS, open the Server Manager. 2. In Server Manager, select **Tools**, and then select **AD FS Management**. 3. In **AD FS Management**, right-click on **Application Groups** and select **Add Application Group**. 4. On the Application Group Wizard **Welcome** screen: - Enter the Name of your application, for example, `NGINX Demo App`. - Under **Standalone applications**, select **Server application**. 5. On the Application Group Wizard **Server application** screen: - Copy the **Client Identifier** value generated by AD FS. The client identifier is your AD FS Application ID, you will need it later when configuring NGINX Plus. - In **Redirect URI**, enter the Redirect URI for your NGINX Plus instance, for example, `https://demo.example.com/oidc_callback`, and then click **Add**. 6. On the Application Group Wizard **Configure Application Credentials** screen: - Select **Generate a shared secret**. - Copy and save the generated **Client Secret**, you will need it later when configuring NGINX Plus. You will not be able to view the secret after the application group is created. - Select **Next** to complete the steps for adding the application group. ### Configure Logout URLs After creating the application group, you need to configure the logout URLs to support RP-initiated logout: 1. In **AD FS Management**, navigate to **Application Groups** and select your application group. 2. Right-click on the application group and select **Properties**. 3. In the **Properties** dialog, add the post logout redirect URI to the application configuration: - Add the post logout URL, for example: `https://demo.example.com/post_logout/`. 4. To enable OpenID Connect front-channel logout (single sign-out when the user signs out of another application): - Use PowerShell to configure a **LogoutUri** for the AD FS client (there is no GUI option for this). For example, run: ```powershell Set-AdfsClient -TargetClientId -LogoutUri https://demo.example.com/front_logout/ ``` Replace `` with the Client Identifier from [Step 5](#adfs-setup-id) above (the AD FS Application ID) and substitute the domain name of your NGINX Plus instance for `demo.example.com`. This registers a front-channel logout URL (`LogoutUri`) for the client in AD FS. When a user signs out of this or any other application in AD FS, the AD FS server sends a GET request to this URL (typically via a hidden iframe) with the user's session ID (`sid`) as a query parameter, instructing NGINX Plus to clear the user's session. According to the OpenID Connect front-channel logout specification, the identity provider is supposed to send both an issuer (`iss`) and a session ID; AD FS provides only the `sid` parameter, but the NGINX Plus OIDC module supports both the fully compliant `iss+sid` and the `sid`-only variants and will clear the session in either case. ### Get the OpenID Connect Discovery URL Check the OpenID Connect endpoint URL. By default, AD FS publishes the `.well-known/openid-configuration` document at the following address: `https://adfs-server-address/adfs/.well-known/openid-configuration`. 1. Run the following `curl` command in a terminal: ```shell curl https://adfs-server-address/adfs/.well-known/openid-configuration | jq . ``` Where: - the `adfs-server-address` is your AD FS server address - the `/adfs/.well-known/openid-configuration` is the default address for AD FS for document location - the `jq` command (optional) is used to format the JSON output for easier reading and requires the [jq](https://jqlang.github.io/jq/) JSON processor to be installed. The configuration metadata is returned in the JSON format: ```json { ... "issuer": "https://adfs-server-address/adfs", "authorization_endpoint": "https://adfs-server-address/adfs/oauth2/authorize/", "token_endpoint": "https://adfs-server-address/adfs/oauth2/token/", "jwks_uri": "https://adfs-server-address/adfs/discovery/keys", "userinfo_endpoint": "https://adfs-server-address/adfs/userinfo", "end_session_endpoint": "https://adfs-server-address/adfs/oauth2/logout", "frontchannel_logout_supported": true, "frontchannel_logout_session_supported": true ... } ``` 2. Copy the **issuer** value, you will need it later when configuring NGINX Plus. Typically, the OpenID Connect Issuer for AD FS is: `https://adfs-server-address/adfs`. **Note:** You will need the values of **Client ID**, **Client Secret**, and **Issuer** in the next steps. ## Set up NGINX Plus {#nginx-plus-setup} With AD FS configured, you can enable OIDC on NGINX Plus. NGINX Plus serves as the Rely Party (RP) application — a client service that verifies user identity. 1. Ensure that you are using the latest version of NGINX Plus by running the `nginx -v` command in a terminal: ```shell nginx -v ``` The output should match NGINX Plus Release 36 or later: ```text nginx version: nginx/1.29.3 (nginx-plus-r36) ``` 2. Ensure that you have the values of the **Client ID**, **Client Secret**, and **Issuer** obtained during [AD FS Configuration](#adfs-setup). 3. In your preferred text editor, open the NGINX configuration file (`/etc/nginx/nginx.conf` for Linux or `/usr/local/etc/nginx/nginx.conf` for FreeBSD). 4. In the [`http {}`](https://nginx.org/en/docs/http/ngx_http_core_module.html#http) context, make sure your public DNS resolver is specified with the [`resolver`](https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver) directive. By default, NGINX Plus re‑resolves DNS records at the frequency specified by time‑to‑live (TTL) in the record, but you can override the TTL value with the `valid` parameter: ```nginx http { resolver 10.0.0.1 ipv4=on valid=300s; # ... } ``` 5. In the [`http {}`](https://nginx.org/en/docs/http/ngx_http_core_module.html#http) context, define the AD FS OIDC provider named `adfs` by specifying the [`oidc_provider {}`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#oidc_provider) context: ```nginx http { resolver 10.0.0.1 ipv4=on valid=300s; oidc_provider adfs { # ... } # ... } ``` 6. In the [`oidc_provider {}`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#oidc_provider) context, specify: - Your actual AD FS **Client ID** from [Step 5](#adfs-setup-id) of AD FS Configuration with the [`client_id`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_id) directive - Your **Client Secret** from [Step 6](#adfs-setup-secret) of AD FS Configuration with the [`client_secret`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_secret) directive - The **Issuer** URL from [Step 2](#adfs-setup-issuer) of AD FS Configuration with the [`issuer`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_secret) directive The `issuer` is typically your AD FS OIDC URL. By default, NGINX forms the provider metadata endpoint by appending `.well-known/openid-configuration` to the issuer. For AD FS, this often resolves to `https://adfs-server-address/adfs/.well-known/openid-configuration`. If your AD FS issuer differs from `https://adfs-server-address/adfs` (for example, a custom path), you can explicitly specify the metadata document with the [`config_url`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#config_url) directive. - The **logout_uri** is URI that a user visits to start an RP‑initiated logout flow. - The **frontchannel_logout_uri** directive defines the URI that receives OpenID Connect front-channel logout requests from AD FS. This URI must be an HTTPS path and must match the LogoutUri configured for the client in AD FS. When AD FS triggers a front-channel logout (for example, when a user signs out of another application), it sends a GET request to this URI (typically via a hidden iframe) with the session ID (sid) as a query parameter. The OIDC module clears the corresponding user session on NGINX Plus. - The **post_logout_uri** is absolute HTTPS URL where AD FS should redirect the user after a successful logout. This value **must also be configured** in the AD FS application properties. - If the **logout_token_hint** directive set to `on`, NGINX Plus sends the user's ID token as a *hint* to AD FS. This directive is **optional**, however, if it is omitted the AD FS may display an extra confirmation page asking the user to approve the logout request. - If the **userinfo** directive is set to `on`, NGINX Plus will fetch `/userinfo` from the AD FS and append the claims from userinfo to the `$oidc_claims_` variables. - PKCE (Proof Key for Code Exchange) is automatically enabled when the provider's OpenID Connect discovery document advertises the S256 code challenge method in the code_challenge_methods_supported field. You can override this behavior with the pkce directive: set `pkce off;` to disable PKCE even when S256 is advertised, or `pkce on;` to force PKCE even if the IdP's metadata does not list S256. - **Note:** All interaction with the IdP is secured exclusively over SSL/TLS, so NGINX must trust the certificate presented by the IdP. By default, this trust is validated against your system's CA bundle (the default CA store for your Linux or FreeBSD distribution). If the IdP's certificate is not included in the system CA bundle, you can explicitly specify a trusted certificate or chain with the [`ssl_trusted_certificate`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#ssl_trusted_certificate) directive so that NGINX can validate and trust the IdP's certificate. ```nginx http { resolver 10.0.0.1 ipv4=on valid=300s; oidc_provider adfs { issuer https://adfs.example.com/adfs; client_id ; client_secret ; logout_uri /logout; post_logout_uri https://demo.example.com/post_logout/; frontchannel_logout_uri /front_logout; logout_token_hint on; userinfo on; # Optional: PKCE configuration. By default, PKCE is automatically # enabled when the IdP advertises the S256 code challenge method. # pkce on; } # ... } ``` 7. Make sure you have configured a [server](https://nginx.org/en/docs/http/ngx_http_core_module.html#server) that corresponds to `demo.example.com`, and there is a [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) that [points](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass) to your application (see [Step 10](#oidc_app)) at `http://127.0.0.1:8080` that is going to be OIDC-protected: ```nginx http { # ... server { listen 443 ssl; server_name demo.example.com; ssl_certificate /etc/ssl/certs/fullchain.pem; ssl_certificate_key /etc/ssl/private/key.pem; location / { # ... proxy_pass http://127.0.0.1:8080; } } # ... } ``` 8. Protect this [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) with AD FS OIDC by specifying the [`auth_oidc`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#auth_oidc) directive that will point to the `adfs` configuration specified in the [`oidc_provider {}`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#oidc_provider) context in [Step 5](#adfs-setup-oidc-provider): ```nginx # ... location / { auth_oidc adfs; # ... proxy_pass http://127.0.0.1:8080; } # ... ``` 9. Pass the OIDC claims as headers to the application ([Step 10](#oidc_app)) with the [`proxy_set_header`](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_header) directive. These claims are extracted from the ID token returned by AD FS: - [`$oidc_claim_sub`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) - a unique `Subject` identifier assigned for each user by AD FS - [`$oidc_claim_email`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) the e-mail address of the user - [`$oidc_claim_name`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) - the full name of the user - any other OIDC claim using the [`$oidc_claim_ `](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) variable ```nginx # ... location / { auth_oidc adfs; proxy_set_header sub $oidc_claim_sub; proxy_set_header email $oidc_claim_email; proxy_set_header name $oidc_claim_name; proxy_pass http://127.0.0.1:8080; } # ... ``` 10. Provide endpoint for completing logout: ```nginx # ... location /post_logout/ { return 200 "You have been logged out.\n"; default_type text/plain; } # ... ``` 11. Create a simple test application referenced by the `proxy_pass` directive which returns the authenticated user's full name and email upon successful authentication: ```nginx # ... server { listen 8080; location / { return 200 "Hello, $http_name!\nEmail: $http_email\nAD FS sub: $http_sub\n"; default_type text/plain; } } ``` 12. Save the NGINX configuration file and reload the configuration: ```nginx nginx -s reload ``` ### Complete Example This configuration example summarizes the steps outlined above. It includes only essential settings such as specifying the DNS resolver, defining the OIDC provider, configuring SSL, and proxying requests to an internal server. ```nginx http { # Use a public DNS resolver for Issuer discovery, etc. resolver 10.0.0.1 ipv4=on valid=300s; oidc_provider adfs { # The 'issuer' is typically your AD FS OIDC URL # e.g. https://adfs.example.com/adfs issuer https://adfs.example.com/adfs; # Replace with your actual AD FS Client ID and Secret client_id ; client_secret ; # RP‑initiated logout logout_uri /logout; post_logout_uri https://demo.example.com/post_logout/; logout_token_hint on; # Front-channel logout frontchannel_logout_uri /front_logout; # Fetch userinfo claims userinfo on; # Optional: PKCE configuration (enabled automatically when supported by the IdP) # pkce on; } server { listen 443 ssl; server_name demo.example.com; ssl_certificate /etc/ssl/certs/fullchain.pem; ssl_certificate_key /etc/ssl/private/key.pem; location / { # Protect this location with AD FS OIDC auth_oidc adfs; # Forward OIDC claims as headers if desired proxy_set_header sub $oidc_claim_sub; proxy_set_header email $oidc_claim_email; proxy_set_header name $oidc_claim_name; proxy_pass http://127.0.0.1:8080; } location /post_logout/ { return 200 "You have been logged out.\n"; default_type text/plain; } } server { # Simple test upstream server listen 8080; location / { return 200 "Hello, $http_name!\nEmail: $http_email\nAD FS sub: $http_sub\n"; default_type text/plain; } } } ``` ### Testing 1. Open `https://demo.example.com/` in a browser. You will be automatically redirected to the AD FS sign-in page. 2. Enter valid AD FS credentials of a user who has access the application. Upon successful sign-in, AD FS redirects you back to NGINX Plus, and you will see the proxied application content (for example, "Hello, Jane Doe!"). 3. Navigate to `https://demo.example.com/logout`. NGINX Plus initiates an RP‑initiated logout; AD FS ends the session and redirects back to `https://demo.example.com/post_logout/`. 4. Refresh `https://demo.example.com/` again. You should be redirected to AD FS for a fresh sign‑in, proving the session has been terminated. ## Legacy njs-based AD FS Solution {#legacy-njs-guide} If you are running NGINX Plus R33 and earlier or if you still need the njs-based solution, refer to the [Legacy njs-based Microsoft AD FS Guide](nginx/deployment-guides/single-sign-on/oidc-njs/active-directory-federation-services.md) for details. The solution uses the [`nginx-openid-connect`](https://github.com/nginxinc/nginx-openid-connect) GitHub repository and NGINX JavaScript files. ## See Also - [NGINX Plus Native OIDC Module Reference documentation](https://nginx.org/en/docs/http/ngx_http_oidc_module.html) - [Release Notes for NGINX Plus R36](nginx/releases.md#r36) ## Revision History - Version 3 (November 2025) – Updated for NGINX Plus R36; added front-channel logout support (`frontchannel_logout_uri`), PKCE configuration (`pkce` directive), and the `client_secret_post` token endpoint authentication method. - Version 2 (August 2025) – Updated for NGINX Plus R35; added RP‑initiated logout (`logout_uri`, `post_logout_uri`, `logout_token_hint`) and `userinfo` support. - Version 1 (March 2025) – Initial version (NGINX Plus Release 34). -------------------------------------------------------------------------------- title: "Synchronizing NGINX Configuration in a Cluster" url: https://docs.nginx.com/nginx/admin-guide/high-availability/configuration-sharing/index.md description: Sychronize configuration across a cluster of F5 NGINX Plus instances. Though designed for high-availability deployments, the solution works for any cluster. -------------------------------------------------------------------------------- ## Overview NGINX Plus is often deployed in a high‑availability (HA) cluster of two or more devices. The configuration sharing feature enables you to push configuration from one machine in the cluster (the primary) to its peers: ![nginx-sync.sh](/nginx/images/nginx-plus-config-synchronization.png) To configure this feature: 1. [Install](#in_detail1) the **nginx-sync** package on the primary machine 2. [Grant](#in_detail2) the primary machine ssh access as `root` to the peer machines 3. [Create](#in_detail3) the configuration file **/etc/nginx-sync.conf** on the primary machine: ```none NODES="node2.example.com node3.example.com node4.example.com" CONFPATHS="/etc/nginx/nginx.conf /etc/nginx/conf.d" EXCLUDE="default.conf" ``` 4. [Run](#in_detail4) the `nginx-sync.sh` command on the primary node to push the configuration files name in `CONFPATHS` to the specified `NODES`, omitting configuration files named in `EXCLUDE`. `nginx-sync.sh` includes a number of safety checks: - Verifies system prerequisites before proceeding - Validates the local (primary) configuration (`nginx -t`) and exits if that fails - Creates remote backup of the configuration on each peer - Pushes the primary configuration to the peers using `rsync`, validates configuration on the peers (`nginx -t`), and if successful reloads NGINX Plus on the peers (`service nginx reload`) - If any step fails, rolls back to the backup on the peers ## Instructions ### Installing nginx-sync on the Primary Machine Install the NGINX Synchronization module package `nginx-sync` on the Primary machine. Check the [Technical Specifications](nginx/technical-specs.md) page to verify that the module is supported by your operating system. - For Amazon Linux 2, CentOS, Oracle Linux, and RHEL: ```shell sudo yum install nginx-sync ``` - For Amazon Linux 2023, AlmaLinux, Rocky Linux: ```shell sudo dnf install nginx-sync ``` - For Ubuntu or Debian: ```shell sudo apt-get install nginx-sync ``` - For SLES: ```shell sudo zypper install nginx-sync ``` ### Configuring root SSH Access to the Peers This procedure enables the `root` user on the primary node to ssh to the `root` account on each peer, which is required to `rsync` files to the peers and run commands on the peers to validate the configuration, reload NGINX Plus, and so on. 1. On the primary node, generate an SSH authentication key pair for `root` and view the public part of the key: ```shell sudo ssh-keygen -t rsa -b 2048 sudo cat /root/.ssh/id_rsa.pub ssh-rsa AAAAB3Nz4rFgt...vgaD root@node1 ``` 2. Get the IP address of the primary node (in the following example, `192.168.1.2`): ```shell ip addr 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 52:54:00:34:6c:35 brd ff:ff:ff:ff:ff:ff inet 192.168.1.2/24 brd 192.168.1.255 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::5054:ff:fe34:6c35/64 scope link valid_lft forever preferred_lft forever ``` 3. On each peer node, append the public key to `root`’s **authorized_keys** file. The `from=192.168.1.2` prefix restricts access to only the IP address of the primary node: ```shell sudo mkdir /root/.ssh sudo echo 'from="192.168.1.2" ssh-rsa AAAAB3Nz4rFgt...vgaD root@node1' >> /root/.ssh/authorized_keys ``` 4. Add the following line to **/etc/ssh/sshd_config**: ```none PermitRootLogin without-password ``` 5. Reload `sshd` on each peer (but not the primary) to allow SSH key authentication - for Amazon Linux, AlmaLinux, CentOS, Oracle Linux, RHEL, Rocky Linux: ```shell sudo systemctl restart sshd ``` - for Ubuntu, Debian, SLES: ```shell sudo systemctl restart ssh ``` 6. Verify that the `root` user can `ssh` to each of the other nodes without providing a password: ```shell sudo ssh root@node2.example.com ``` ### Creating the nginx-sync.conf Configuration File on the Primary Node On the primary node, create the file **/etc/nginx-sync.conf** with these contents: ```none NODES="node2.example.com node3.example.com node4.example.com" CONFPATHS="/etc/nginx/nginx.conf /etc/nginx/conf.d" EXCLUDE="default.conf" ``` #### Common Parameters Use a space or newline character to separate the items in each list: | Parameter | Description | | ------------------------ | -------------------------------------------------------------------------------------| | `NODES` | List of peers that receive the configuration from the primary. | | `CONFPATHS` | List of files and directories to distribute from the primary to the peers. | | `EXCLUDE` | (Optional) List of configuration files on the primary not to distribute to the peers.| #### Advanced Parameters | Parameter | Description | Default | | ------------------------ | ---------------------------------------------------------------------------------------|-------------------------| | `BACKUPDIR` | Location of backup on each peer | **/var/lib/nginx-sync** | | `DIFF` | Location of `diff` binary | **/usr/bin/diff** | | `LOCKFILE` | Location of the lock file used to ensure only one `nginx-sync` operation runs at a time| **/tmp/nginx-sync.lock**| | `NGINX` | Location of the **nginx-plus** binary | **/usr/sbin/nginx** | | `POSTSYNC` | Space-separated list of file substitutions to make on each remote node in the format:
`'\\|\'`
The substitution is applied in place:
`sed -i' ' \ \`
For example, to substitute the IP address of *node2.example.com* (*192.168.2.2*) for the IP address of *node1.example.com* (*192.168.2.1*) in *keepalived.conf*:
`POSTSYNC="/etc/keepalived/keepalived.conf\|'s/192\.168\.2\.1/192.168.2.2/'"` | | | `RSYNC` | Location of the `rsync` binary | **/usr/bin/rsync** | | `SSH` | Location of the `ssh` binary | **/usr/bin/ssh** | ### Testing the Configuration Back up the configuration before testing. - Synchronize configuration and reload F5 NGINX Plus on the peers – `nginx-sync.sh` - Display usage information – `nginx-sync.sh -h` - Compare configuration between the primary and a peer – `nginx-sync.sh -c ` - Compare configuration on the primary to all peers – `nginx-sync.sh -C` ## Frequently Asked Questions ### Why Do I Need to Grant SSH Access to `root`? The primary node needs to be able to remotely run commands on the peer as the `root` user (for example, `service nginx reload`), and needs to be able to update configuration files (for example, in **/etc/nginx/**) that are owned by `root`. It might seem that granting SSH access to `root` is giving away too many privileges, but it is important to remember that any process that can write remote NGINX Plus configuration and reload the remote NGINX Plus process can subvert this process to gain remote `root` access to the server. Therefore, assume that users who gain `root` access on the primary node also have `root` access on the peer nodes. ### How Do I Synchronize Configuration if the Primary Fails? If the primary fails and will not soon return to service, you need to promote a peer to operate as primary by following the instructions in [Installation](#instructions). This involves 1. [Installing the `nginx-sync.sh` script](#in_detail1) 2. [Granting SSH access to the remaining peers](#in_detail2) 3. [Creating the configuration file](#in_detail3) You can preconfigure several machines to operate as primary, but must ensure that only one node actually runs as primary at a given time. ### What Happens if a Peer Node Fails? If a peer node fails, it no longer receives configuration updates. The `nginx-sync.sh` script returns an error but continues to distribute the configuration to the remaining peers. When the node recovers, its configuration is out of date. You can display the configuration differences by running `nginx-sync.sh -c -d`: ```shell nginx-sync.sh -c node2.example.com -d ``` The output of the command: ```diff diff -ru /tmp/localconf.1XrIqP7f/etc/nginx/conf.d/responder.conf /tmp/remoteconf.Xq5LWGKU/etc/nginx/conf.d/responder.conf --- /tmp/localconf.1XrIqP7f/etc/nginx/conf.d/responder.conf 2020-09-25 10:29:36.988064021 -0800 +++ /tmp/remoteconf.Xq5LWGKU/etc/nginx/conf.d/responder.conf 2020-09-25 10:28:39.764066539 -0800 @@ -4,6 +4,6 @@ listen 80; location / { - return 200 "Received request on $server_addr on host $hostname blue\n"; + return 200 "Received request on $server_addr on host $hostname red\n"; } } * Synchronization ended at Fri Sep 25 18:30:49 UTC 2020 ``` The next time you run `nginx-sync.sh`, the node gets updated with the current primary configuration. -------------------------------------------------------------------------------- title: "Authentication Based on Subrequest Result" url: https://docs.nginx.com/nginx/admin-guide/security-controls/configuring-subrequest-authentication/index.md description: Authenticate clients during request processing by making a subrequest to an external authentication service, such as LDAP or OAuth. -------------------------------------------------------------------------------- ## Introduction NGINX and F5 NGINX Plus can authenticate each request to your website with an external server or service. To perform authentication, NGINX makes an HTTP subrequest to an external server where it is verified. Such type of authentication allows implementing various authentication schemes, such as multifactor authentication, or allows implementing LDAP or OAuth authentication. ## Response Codes Subrequest Response Codes: - 2xx - access is allowed - 401, 403 - access is denied ## Prerequisites - NGINX Plus or NGINX Open Source - External authentication server or service ## Configuring NGINX and NGINX Plus 1. Make sure your NGINX Open Source is compiled with the `with-http_auth_request_module` configuration option. Run this command and verify that the output includes `--with-http_auth_request_module`: ```none nginx -V 2>&1 | grep -- 'http_auth_request_module' ``` Skip this step for NGINX Plus as it already includes the auth_request module. 2. In the location that requires request authentication, specify the [auth_request](https://nginx.org/en/docs/http/ngx_http_auth_request_module.html#auth_request) directive and specify an internal location where an authorization subrequest will be forwarded to: ```nginx location /private/ { auth_request /auth; #... } ``` Here, for each request to **/private**, a subrequest to the internal **/auth** location will be made. 3. Specify an internal location and the [proxy_pass](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass) directive inside this location that will proxy authentication subrequests to an authentication server or service: ```nginx location = /auth { internal; proxy_pass http://auth-server; #... } ``` 4. As the request body is discarded for authentication subrequests, set the [proxy_pass_request_body](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass_request_body) directive to `off` and also set the `Content-Length` header to a null string: ```nginx location = /auth { internal; proxy_pass http://auth-server; proxy_pass_request_body off; proxy_set_header Content-Length ""; #... } ``` 5. Pass the full original request URI with arguments with the [proxy_set_header](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_header) directive: ```nginx location = /auth { internal; proxy_pass http://auth-server; proxy_pass_request_body off; proxy_set_header Content-Length ""; proxy_set_header X-Original-URI $request_uri; } ``` 6. As an option, you can set a variable value basing on the result of the subrequest with the [auth_request_set](https://nginx.org/en/docs/http/ngx_http_auth_request_module.html#auth_request_set) directive: ```nginx location /private/ { auth_request /auth; auth_request_set $auth_status $upstream_status; } ``` ## Complete Example This example sums up the previous steps into one configuration: ```nginx http { #... server { #... location /private/ { auth_request /auth; auth_request_set $auth_status $upstream_status; } location = /auth { internal; proxy_pass http://auth-server; proxy_pass_request_body off; proxy_set_header Content-Length ""; proxy_set_header X-Original-URI $request_uri; } } } ``` -------------------------------------------------------------------------------- title: "Compression and Decompression" url: https://docs.nginx.com/nginx/admin-guide/web-server/compression/index.md description: Compress server responses, or decompress them for clients that don't support compression, to improve delivery speed and reduce overhead on the server. -------------------------------------------------------------------------------- This section describes how to configure compression or decompression of responses, as well as sending compressed files. ## Introduction Compressing responses often significantly reduces the size of transmitted data. However, since compression happens at runtime it can also add considerable processing overhead which can negatively affect performance. NGINX performs compression before sending responses to clients, but does not “double compress” responses that are already compressed (for example, by a proxied server). ## Enabling Compression To enable compression, include the [gzip](https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip) directive with the `on` parameter. ```nginx gzip on; ``` By default, NGINX compresses responses only with MIME type `text/html`. To compress responses with other MIME types, include the [gzip_types](https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_types) directive and list the additional types. ```nginx gzip_types text/plain application/xml; ``` To specify the minimum length of the response to compress, use the [gzip_min_length](https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_min_length) directive. The default is 20 bytes (here adjusted to 1000): ```nginx gzip_min_length 1000; ``` By default, NGINX does not compress responses to proxied requests (requests that come from the proxy server). The fact that a request comes from a proxy server is determined by the presence of the `Via` header field in the request. To configure compression of these responses, use the [gzip_proxied](https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_proxied) directive. The directive has a number of parameters specifying which kinds of proxied requests NGINX should compress. For example, it is reasonable to compress responses only to requests that will not be cached on the proxy server. For this purpose the `gzip_proxied` directive has parameters that instruct NGINX to check the `Cache-Control` header field in a response and compress the response if the value is `no-cache`, `no-store`, or `private`. In addition, you must include the `expired` parameter to check the value of the `Expires` header field. These parameters are set in the following example, along with the `auth` parameter, which checks for the presence of the `Authorization` header field (an authorized response is specific to the end user and is not typically cached): ```nginx gzip_proxied no-cache no-store private expired auth; ``` As with most other directives, the directives that configure compression can be included in the `http` context or in a `server` or `location` configuration block. The overall configuration of gzip compression might look like this. ```nginx server { gzip on; gzip_types text/plain application/xml; gzip_proxied no-cache no-store private expired auth; gzip_min_length 1000; ... } ``` ## Enabling Decompression Some clients do not support responses with the `gzip` encoding method. At the same time, it might be desirable to store compressed data, or compress responses on the fly and store them in the cache. To successfully serve both clients that do and do not accept compressed data, NGINX can decompress data on the fly when sending it to the latter type of client. To enable runtime decompression, use the [gunzip](https://nginx.org/en/docs/http/ngx_http_gunzip_module.html#gunzip) directive. ```nginx location /storage/ { gunzip on; ... } ``` The `gunzip` directive can be specified in the same context as the `gzip` directive: ```nginx server { gzip on; gzip_min_length 1000; gunzip on; ... } ``` Note that this directive is defined in a separate [module](https://nginx.org/en/docs/http/ngx_http_gunzip_module.html) that might not be included in an NGINX Open Source build by default. ## Sending Compressed Files To send a compressed version of a file to the client instead of the regular one, set the [gzip_static](https://nginx.org/en/docs/http/ngx_http_gzip_static_module.html#gzip_static) directive to `on` within the appropriate context. ```nginx location / { gzip_static on; } ``` In this case, to service a request for **/path/to/file**, NGINX tries to find and send the file **/path/to/file.gz**. If the file doesn’t exist, or the client does not support gzip, NGINX sends the uncompressed version of the file. Note that the `gzip_static` directive does not enable on-the-fly compression. It merely uses a file compressed beforehand by any compression tool. To compress content (and not only static content) at runtime, use the `gzip` directive. This directive is defined in a separate [module](https://nginx.org/en/docs/http/ngx_http_gzip_static_module.html) that might not be included in an NGINX Open Source build by default. -------------------------------------------------------------------------------- title: "Install NGINX Plus on the Google Cloud Platform" url: https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-plus-google-cloud-platform/index.md description: Install F5 NGINX Plus on the Google Cloud Platform, to provide sophisticated Layer 7 load balancing for your apps running on Google Compute Engine. -------------------------------------------------------------------------------- [NGINX Plus](https://www.f5.com/products/nginx/nginx-plus), the high‑performance application delivery platform, load balancer, and web server, is available on the Google Cloud Platform as a virtual machine (VM) image. The VM image contains the latest version of NGINX Plus, optimized for use with the Google Cloud Platform Compute Engine. ## Install the NGINX Plus VM To quickly set up an NGINX Plus environment on the Google Cloud Platform, perform the following steps. > **Note:** The Google Cloud Platform interface is under active development. Although we make every effort to provide accurate instructions, they are subject to change. Also, the exact options you see depend on whether or not you have existing projects. 1. Access the [NGINX Plus page](https://console.cloud.google.com/marketplace/details/nginx-public/nginx-plus) in Google Marketplace and click the **LAUNCH ON COMPUTE ENGINE** button. screenshot F5 NGINX Plus on Google Marketplace 2. The **Select or create a project** window opens. Enter a project name and select a value from the **Organization** drop‑down, then click the **Create** button. screenshot nginx plus to google computer engine 3. In the **Configure & Deploy** window, enter or select appropriate values for zone, machine type, and so on. Click the **Deploy** button. > **Note:** In the **Firewall** section, be sure the **Allow HTTP traffic** checkbox is checked. For more information on controlling incoming traffic, see the [Firewall Rules Overview](https://cloud.google.com/vpc/docs/firewalls) in the Google Cloud Platform documentation. Screenshot New NGINX Plus Deployment 4. Click the **Create** button. The Google Developers Console confirms that NGINX Plus was deployed. Screenshot NGINX plus deployed on Google Cloud Platform As soon as the project deploys and the new virtual machine (VM) instance starts running, NGINX Plus starts automatically and serves a default **index.html** page. To verify that NGINX Plus is working properly, use a web browser to access the public DNS name of the new VM and view the page. You can also check the status of the NGINX Plus server by logging into the VM and running this command: ```shell /etc/init.d/nginx status ``` ## What If I Need Help? If you encounter any problems with NGINX Plus configuration, documentation is available at [nginx.org](https://nginx.org/en/docs/) and in the [NGINX Plus Admin Guide](/nginx/admin-guide/installing-nginx/). Customers who purchase an NGINX Plus VM image on the Google Cloud Platform are eligible for the Google Cloud Platform VM support provided by the NGINX, Inc. engineering team. To activate support, submit the [Google Cloud Platform Support Activation](https://www.nginx.com/gcp-support-activation/) form. ### Access the Open Source Licenses for NGINX Plus NGINX Plus includes open source software written by NGINX, Inc. and other contributors. The text of the open source licenses is provided in Appendix B of the _NGINX Plus Reference Guide_. To access the guide included with the NGINX Plus VM instance, run this command: ```shell less /usr/share/nginx/html/nginx-modules-reference.pdf ``` The _NGINX Plus Reference Guide_ is also [available online](http://www.nginx.com/wp-content/uploads/2023/08/nginx-modules-reference.pdf). -------------------------------------------------------------------------------- title: "NGINX Diagnostic Package" url: https://docs.nginx.com/nginx/admin-guide/monitoring/diagnostic-package/index.md description: This page describes how to trigger the automatic collection of data required to troubleshoot issues in a NGINX or F5 NGINX Plus deployment. -------------------------------------------------------------------------------- ## Overview NGINX Diagnostic Package is used to obtain additional information needed by [F5 Technical Support](https://account.f5.com/myf5) when troubleshooting your issue. The package is created by a script that can be [downloaded](https://nginx.org/download/nginx-supportpkg.sh) from the [nginx.org](https://nginx.org/download/) website. The script collects the following information: - host commands such as `ps`, `lsof`, `vmstat` - NGINX configuration files - NGINX log files - NGINX service information - NGINX process information - NGINX versions, dynamically linked libraries - NGINX Plus API endpoints - NGINX Agent logs and configs if NGINX Agent is present - F5 WAF for NGINX logs and configs if F5 WAF for NGINX is present The script does not collect or create: - njs scripts - Lua scripts - Core dumps **Note:** It is highly recommended that you review the script and the created resources and verify that they conform with your organization's data sharing policies. ## Supported Operating Systems The script can be run on most [operating systems supported by NGINX](https://docs.nginx.com/nginx/technical-specs/) and has been tested on the following operating systems: - AlmaLinux 9.1 - Amazon Linux 2 - CentOS 7 - Debian 11 - RHEL 9.1 - Rocky Linux 9.1 - SUSE Linux Enterprise Server 15 - Ubuntu 20.04 ## Usage To create NGINX Diagnostic Package: 1. [Download](https://nginx.org/download/nginx-supportpkg.sh) the `nginx-supportpkg.sh` script: ```shell wget https://nginx.org/download/nginx-supportpkg.sh ``` 2. Grant execution permissions to the script: ```shell chmod +x nginx-supportpkg.sh ``` 3. Run the script. The script requires root privileges to run. The script can be run with optional arguments, see [Arguments](#arguments) for details. ```shell sudo ./nginx-supportpkg.sh ``` The created package will be located in the same directory as the current script. It is a `.tar.gz` archive named according to the file name pattern: `support-pkg-.tar.gz`. 4. After the package has been created, it is recommended to extract and review its contents. Use the `tar` command to extract the archive: ```shell tar -xvf support-pkg-1682457903.tar.gz ``` The archive contains textual output of all the commands run by the script to make it easier to review the collected data. ## Arguments The following table lists the arguments you can use to customize the data that needs to be collected based on your NGINX deployment. | Short | Long | Description | Example | Default | | ----- | ------------------------ | ----------------------------------------------------------------------| -------------------------| -----------------| | `-h` | `--help` | Prints information about the script arguments to stdout. | `--help` | N/A | | `-d` | `--debug` | Sets bash debug flag. | `--debug` | N/A | | `-o` | `--output_dir` | The output directory where the tar archive is saved. | `-o ~/output` | `$(pwd)` | | `-n` | `--nginx_log_path` | The directory where the NGINX log files are located. | `-n /var/log/nginx` | `/var/log/nginx` | | `-xc` | `--exclude_nginx_configs`| Excludes all NGINX configuration files from the support package. | `--exclude_nginx_configs`| N/A | | `-xl` | `--exclude_nginx_logs` | Excludes all NGINX log files from the support package. | `--exclude_nginx_logs` | N/A | | `-ac` | `--exclude_agent_configs`| Excludes all NGINX Agent configuration files from the support package.| `--exclude_agent_configs`| N/A | | `-al` | `--exclude_agent_logs` | Excludes all NGINX Agent logs from the support package. | `--exclude_agent_logs` | N/A | | `-nc` | `--exclude_nap_configs` | Excludes all F5 WAF for NGINX config files from the support package. | `--exclude_nap_configs` | N/A | | `-nl` | `--exclude_nap_logs` | Excludes all F5 WAF for NGINX log files from the support package. | `--exclude_nap_logs` | N/A | | `-ea` | `--exclude_api_stats` | Excludes NGINX Plus API stats from the support package. | `--exclude_api_stats` | N/A | | `-pi` | `--profile_interval` | Profiling interval in seconds. | `-pi 20` | 15 | -------------------------------------------------------------------------------- title: "Runtime State Sharing in a Cluster" url: https://docs.nginx.com/nginx/admin-guide/high-availability/zone_sync/index.md description: This article describes how to use F5 NGINX Plus to synchronize shared memory zones across NGINX cluster nodes including sticky learn session persistence, requests limiting, and key-value store data. -------------------------------------------------------------------------------- ## Introduction If several F5 NGINX Plus instances are organized in a cluster, they can share some state data between them, including: - [sticky learn](https://docs.nginx.com/nginx/admin-guide/load-balancer/http-load-balancer/#sticky) session persistence - [requests limiting](https://docs.nginx.com/nginx/admin-guide/security-controls/controlling-access-proxied-http/#limit_req) - [key-value storage](https://nginx.org/en/docs/http/ngx_http_keyval_module.html#keyval_zone) All NGINX Plus instances can exchange state data with all other members in a cluster, provided that the shared memory zone has the same name on all cluster members. ## Prerequisites - NGINX Plus [R16](nginx/releases.md#r16) for sharing limits of requests processing and key-value data across the cluster - NGINX Plus [R15](nginx/releases.md#r15) for sharing limits of sticky learn data across the cluster State sharing across a cluster is eventually consistent by nature. It is strongly recommended using data-center grade networks for clustering traffic, as latency, low bandwidth, and packet loss will have a significant negative impact on state consistency. We do not recommend stretching clusters over the Internet, regions, or availability zones. ## Configuring zone synchronization For each NGINX instance in a cluster, open the NGINX configuration file and perform the following steps: 1. Enable synchronization between cluster nodes: in the top-level [`stream`](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#stream) block, create a [`server`](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#server) with the [`zone_sync`](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync) directive: ```nginx stream { #... server { zone_sync; #... } } ``` 2. Specify all other NGINX instances in a cluster with the [`zone_sync_server`](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_server) directive. Cluster nodes can be added dynamically using the DNS service if the [`resolver`](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#resolver) is used: ```nginx stream { resolver 10.0.0.53 valid=20s; server { zone_sync; zone_sync_server nginx-cluster.example.com:9000 resolve; } } ``` Otherwise, each cluster node can be added statically as a separate line of the [`zone_sync_server`](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_server) directive: ```nginx stream { server { zone_sync; zone_sync_server nginx-node1.example.com:9000; zone_sync_server nginx-node2.example.com:9000; zone_sync_server nginx-node3.example.com:9000; } } ``` 3. Enable SSL by specifying the `ssl` parameter of the [`listen`](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#listen) directive for the TCP server: ```nginx stream { resolver 10.0.0.53 valid=20s; server { listen 10.0.0.1:9000 ssl; #... zone_sync; zone_sync_server nginx-cluster.example.com:9000 resolve; #... } } ``` 4. Specify the path to the certificates with the [`ssl_certificate`](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_certificate) directive, and to the private key with the [`ssl_certificate_key`](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_certificate_key) directive. Both the certificate and key must be in the PEM format: ```nginx stream { resolver 10.0.0.53 valid=20s; server { listen 10.0.0.1:9000 ssl; ssl_certificate /etc/ssl/nginx-1.example.com.server_cert.pem; ssl_certificate_key /etc/ssl/nginx-1.example.com.key.pem; zone_sync; zone_sync_server nginx-cluster.example.com:9000 resolve; #... } } ``` 5. Enable SSL connections between cluster servers with the [`zone_sync_ssl`](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_ssl) directive, and enable verification of another cluster server certificate with [`zone_sync_ssl_verify`](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_ssl_verify) and [`zone_sync_ssl_trusted_certificate`](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_ssl_trusted_certificate) directives: ```nginx stream { resolver 10.0.0.53 valid=20s; server { listen 10.0.0.1:9000 ssl; ssl_certificate /etc/ssl/nginx-1.example.com.server_cert.pem; ssl_certificate_key /etc/ssl/nginx-1.example.com.key.pem; zone_sync; zone_sync_server nginx-cluster.example.com:9000 resolve; zone_sync_ssl on; zone_sync_ssl_verify on; zone_sync_ssl_trusted_certificate /etc/ssl/server_ca.pem; #... } } ``` 6. Set up certificate-based authentication across cluster nodes. Add the [`zone_sync_ssl_certificate`](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_ssl_certificate) and [`zone_sync_ssl_certificate_key`](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_ssl_certificate_key`) directives to send the client certificate for outgoing connections. Then configure NGINX to require client certificates by setting the [`ssl_verify_client`](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_verify_client) directive to `on` and specifying the location of your client certificates CA with the [`ssl_trusted_certificate`](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_trusted_certificate) directive: ```nginx stream { resolver 10.0.0.53 valid=20s; server { listen 10.0.0.1:9000 ssl; ssl_certificate /etc/ssl/nginx-1.example.com.server_cert.pem; ssl_certificate_key /etc/ssl/nginx-1.example.com.key.pem; zone_sync; zone_sync_server nginx-cluster.example.com:9000 resolve; zone_sync_ssl on; zone_sync_ssl_verify on; zone_sync_ssl_trusted_certificate /etc/ssl/server_ca.pem; zone_sync_ssl_certificate localhost.crt; zone_sync_ssl_certificate_key localhost.key; ssl_verify_client on; ssl_trusted_certificate /etc/ssl/client_ca.pem; #... } } ``` ### Fine-tuning Synchronization Generally you do not have to tune sync options, but in some cases it can be useful to adjust some of these values: ```nginx #... zone_sync; zone_sync_server nginx-cluster.example.com:9000 resolve; zone_sync_buffers 256 4k; zone_sync_connect_retry_interval 1s; zone_sync_connect_timeout 5s; zone_sync_interval 1s; zone_sync_timeout 5s; #... ``` [`zone_sync_buffers`](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_buffers) - controls the number of buffers and their size. Increasing the number of buffers will increase the number of information stored in them. [`zone_sync_connect_retry_interval`](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_connect_retry_interval) - sets the timeout between connection attempts to a cluster node. [`zone_sync_connect_timeout`](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_connect_timeout) - sets the time required to connect to a cluster node. [`zone_sync_interval`](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_interval) - sets an interval for polling updates in a shared memory zone. Increasing this value may result in data inconsistency between cluster nodes, decreasing the value may lead to high consumption of cpu and memory resources. [`zone_sync_timeout`](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_timeout) - sets the lifetime of a TCP stream between cluster nodes. If a TCP stream is idle for longer than the value, the connection will be closed. ## Managing Cluster Nodes ### Starting a Node To start a new node: - in case of DNS, update a DNS record of a cluster hostname with the IP address of the new node and start an instance - in case of statically added nodes, add the node's address to nginx configuration file and reload all other nodes When the node is started, it discovers other nodes from DNS or static configuration and starts sending updates. Other nodes eventually discover the new node using DNS and start pushing updates to it. ### Stopping a Node To stop a node, send the 'QUIT' signal: ```shell nginx -s quit ``` As soon as the node receives the signal, it finishes zone synchronization and gracefully closes open connections. ### Removing a Node To remove a node: - in case of DNS, update a DNS record of a cluster hostname and remove the node's IP address - in case of statically added nodes, remove the node's address from nginx configuration file on each node and reload each node. When the node is removed, other nodes close connections to the removed node and will no longer try to connect to it. After the node is removed, it can be stopped. ## Using synchronization in a cluster ### Sticky learn zone synchronization If your existing configuration already uses the [sticky learn](https://docs.nginx.com/nginx/admin-guide/load-balancer/http-load-balancer/#sticky) feature, existing state can be simply shared across a cluster by adding the [`sync`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#sticky_learn_sync) parameter to the existing `sticky` directive in the configuration file of each NGINX instance in a cluster. Note that the zone name must be the same in all other NGINX nodes in the cluster: ```nginx upstream my_backend { zone my_backend 64k; server backends.example.com resolve; sticky learn zone=sessions:1m create=$upstream_cookie_session lookup=$cookie_session sync; } server { listen 80; location / { proxy_pass http://my_backend; } } ``` See [Enabling Session Persistence](https://docs.nginx.com/nginx/admin-guide/load-balancer/http-load-balancer/#sticky) for information how to configure the "sticky learn" session persistence method. ### Request limits zone synchronization If your existing configuration already uses rate limiting, these limits can be applied across a cluster by simply adding the [`sync`](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_zone) parameter to the [`limit_req_zone`](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_zone) directive in the configuration file of each NGINX instance in a cluster: ```nginx limit_req_zone $remote_addr zone=req:1M rate=100r/s sync; server { listen 80; location / { limit_req zone=req; proxy_pass http://my_backend; } } ``` The zone name also must be the same in all other NGINX nodes in the cluster. See [Limiting the Request Rate](https://docs.nginx.com/nginx/admin-guide/security-controls/controlling-access-proxied-http/#limit_req) for more information. ### Key-value storage zone synchronization Similar to rate limiting and sticky learn, the contents of the key-value shared memory zone can be shared across NGINX machines in a cluster with the `sync` parameter of the [`keyval_zone`](https://nginx.org/en/docs/http/ngx_http_keyval_module.html#keyval_zone) directive: ```nginx keyval_zone zone=one:32k state=/var/lib/nginx/state/one.keyval sync; keyval $arg_text $text zone=one; #... server { #... location / { return 200 $text; } location /api { api write=on; } } ``` See [Dynamic Denylisting of IP Addresses](/nginx/admin-guide/security-controls/denylisting-ip-addresses.md) for information how to configure and manage the key-value storage. ## Monitoring Cluster State Cluster state data can be monitored with [NGINX Plus API metrics](https://nginx.org/en/docs/http/ngx_http_api_module.html#stream_zone_sync_): - names of shared memory zones - total number of records on node - number of records that needs to be sent - sync status per each node in the cluster ## Configuring the API In order to get access to API metrics, you will need to configure the API: 1. Enable the NGINX Plus API in read‑write mode with the [`api`](https://nginx.org/en/docs/http/ngx_http_api_module.html#api) directive: ```nginx # ... server { listen 80; server_name www.example.com; location /api { api write=on; } } ``` 2. It is highly recommended to [restrict access](/nginx/admin-guide/security-controls/controlling-access-proxied-http.md) to this location, for example by allowing access only from localhost (`127.0.0.1`), and by restricting access to `PATCH`, `POST`, and `DELETE` methods to some users with HTTP basic authentication: ```nginx # ... server { listen 80; server_name www.example.com; location /api { limit_except GET { auth_basic "NGINX Plus API"; auth_basic_user_file /path/to/passwd/file; } api write=on; allow 127.0.0.1; deny all; } } ``` See [Using the API for Dynamic Configuration](https://docs.nginx.com/nginx/admin-guide/load-balancer/dynamic-configuration-api/#api_use) for instructions how to configure and use NGINX Plus API. ## Polling Sync Status with the API To get the synchronization status of the shared memory zone, send the API command, for example, with `curl`: ```shell curl -s '127.0.0.1/api/9/stream/zone_sync' | jq ``` The output will be: ```json { "zones" : { "zone1" : { "records_pending" : 2061, "records_total" : 260575 }, "zone2" : { "records_pending" : 0, "records_total" : 14749 } }, "status" : { "bytes_in" : 1364923761, "msgs_in" : 337236, "msgs_out" : 346717, "bytes_out" : 1402765472, "nodes_online" : 15 } } ``` If all nodes have approximately the same number of records ([`records_total`](https://nginx.org/en/docs/http/ngx_http_api_module.html#def_nginx_stream_zone_sync_zone)) and almost empty outgoing queue ([`records_pending`](https://nginx.org/en/docs/http/ngx_http_api_module.html#def_nginx_stream_zone_sync_zone)), the cluster may be considered healthy. -------------------------------------------------------------------------------- title: "Single Sign-On with Microsoft Entra ID" url: https://docs.nginx.com/nginx/deployment-guides/single-sign-on/entra-id/index.md description: Enable OpenID Connect-based single sign-on (SSO) for applications proxied by NGINX Plus, using Microsoft Entra ID (formerly Azure Active Directory) as the identity provider (IdP). -------------------------------------------------------------------------------- This guide explains how to enable single sign-on (SSO) for applications being proxied by F5 NGINX Plus. The solution uses OpenID Connect as the authentication mechanism, with [Microsoft Entra ID](https://www.microsoft.com/en-us/security/business/identity-access/microsoft-entra-id) as the Identity Provider (IdP), and NGINX Plus as the Relying Party, or OIDC client application that verifies user identity. **Note:** This guide applies to [NGINX Plus Release 36](nginx/releases.md#r36) and later. In earlier versions, NGINX Plus relied on an [njs-based solution](#legacy-njs-guide), which required NGINX JavaScript files, key-value stores, and advanced OpenID Connect logic. In the latest NGINX Plus version, the new [OpenID Connect module](https://nginx.org/en/docs/http/ngx_http_oidc_module.html) simplifies this process to just a few directives. ## Prerequisites - A Microsoft Entra tenant with admin access. - Azure CLI. For installation instructions, see [How to install the Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli). - An NGINX Plus [subscription](https://www.f5.com/products/nginx/nginx-plus) and NGINX Plus [Release 36](nginx/releases.md#r36) or later. For installation instructions, see [Installing NGINX Plus](https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-plus/). - A domain name pointing to your NGINX Plus instance, for example, `demo.example.com`. ## Configure Entra ID {#entra-setup} Register a new application in Microsoft Entra ID that will represent NGINX Plus as an OIDC client. This is necessary to obtain unique identifiers and secrets for OIDC, as well as to specify where Azure should return tokens. Ensure you have access to the Azure Portal with Entra ID app administrator privileges. ### Register new Azure Web Application 1. Log in to Azure CLI: ```shell az login ``` This command will open your default browser for authentication. 2. Register a New Application. - Create a new application, for example, "Nginx Demo App", with NGINX callback URI `/oidc_callback`: ```shell az ad app create --display-name "Nginx Demo App" --web-redirect-uris "https://demo.example.com/oidc_callback" ``` - From the command output, copy the `appId` value which represents your **Client ID**. You will need it later when configuring NGINX Plus. 3. Generate a new Client Secret. - Create a client secret for your application by running: ```bash az ad app credential reset --id ``` - Replace the `` with the value obtained in the previous step. - From the command output, copy the the `password` value which represents your **Client Secret**. You will need it later when configuring NGINX Plus. Make sure to securely save the generated client secret, as it will not be displayed again. - From the same command output, copy the the `tenant` value which represents your **Tenant ID**. You will need it later when configuring NGINX Plus. 4. Configure logout URLs to support RP-initiated and front-channel logout: - Add a logout URL for your application by running: ```bash az ad app update --id --web-logout-urls "https://demo.example.com/post_logout/" ``` - Replace the `` with the value obtained in step 2. - To enable OpenID Connect front-channel logout (single sign-out when the user signs out of another application), configure the *Front-channel logout URL* for your application in the Microsoft Entra admin center. Set it to the absolute HTTPS URL that matches the `frontchannel_logout_uri` you will configure in NGINX Plus, for example `https://demo.example.com/front_logout`. According to the OpenID Connect front-channel logout specification, the Identity Provider sends both the issuer (`iss`) and the session identifier (`sid`) as query parameters. Microsoft Entra ID currently sends only the `sid` value; the NGINX Plus OIDC module for NGINX Plus Release 36 supports both the fully compliant (`iss` + `sid`) and the Entra-specific (`sid`‑only) variants and will clear the corresponding user session in either case. ### Get the OpenID Connect Discovery URL Check the OpenID Connect Discovery URL. By default, Microsoft Entra ID publishes the `.well-known/openid-configuration` document at the following address: `https://login.microsoftonline.com//v2.0/.well-known/openid-configuration`. 1. Run the following `curl` command in a terminal: ```shell curl https://login.microsoftonline.com//v2.0/.well-known/openid-configuration | jq ``` Where: - the `` is your Microsoft Entra Tenant ID - the `login.microsoftonline.com` is your Microsoft Entra server address - the `/v2.0/.well-known/openid-configuration` is the default address for Microsoft Entra ID for document location - the `jq` command (optional) is used to format the JSON output for easier reading and requires the [jq](https://jqlang.github.io/jq/) JSON processor to be installed. The configuration metadata is returned in the JSON format: ```json { ... "issuer": "https://login.microsoftonline.com/{tenant_id}/v2.0", "authorization_endpoint": "https://login.microsoftonline.com/{tenant_id}/oauth2/v2.0/authorize", "token_endpoint": "https://login.microsoftonline.com/{tenant_id}/oauth2/v2.0/token", "jwks_uri": "https://login.microsoftonline.com/{tenant_id}/discovery/v2.0/keys", "userinfo_endpoint": "https://graph.microsoft.com/oidc/userinfo", "end_session_endpoint": "https://login.microsoftonline.com/{tenant_id}/oauth2/v2.0/logout", ... } ``` 2. Copy the **issuer** value, you will need it later when configuring NGINX Plus. Typically, the OpenID Connect Issuer for Microsoft Entra ID is `https://login.microsoftonline.com//v2.0`. **Note:** You will need the values of **Client ID**, **Client Secret**, and **Tenant ID** in the next steps. ## Set up NGINX Plus {#nginx-plus} With Microsoft Entra ID configured, you can enable OIDC on NGINX Plus. NGINX Plus serves as the Rely Party (RP) application — a client service that verifies user identity. 1. Ensure that you are using the latest version of NGINX Plus by running the `nginx -v` command in a terminal: ```shell nginx -v ``` The output should match NGINX Plus Release 36 or later: ```none nginx version: nginx/1.29.0 (nginx-plus-r36) ``` 2. Ensure that you have the values of the **Client ID**, **Client Secret**, and **Tenant ID** obtained during [Microsoft Entra ID Configuration](#entra-setup). 3. In your preferred text editor, open the NGINX configuration file (`/etc/nginx/nginx.conf` for Linux or `/usr/local/etc/nginx/nginx.conf` for FreeBSD). 4. In the [`http {}`](https://nginx.org/en/docs/http/ngx_http_core_module.html#http) context, make sure your public DNS resolver is specified with the [`resolver`](https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver) directive: By default, NGINX Plus re‑resolves DNS records at the frequency specified by time‑to‑live (TTL) in the record, but you can override the TTL value with the `valid` parameter: ```nginx http { resolver 10.0.0.1 ipv4=on valid=300s; # ... } ``` 5. In the [`http {}`](https://nginx.org/en/docs/http/ngx_http_core_module.html#http) context, define the Entra ID provider named `entra` by specifying the [`oidc_provider {}`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#oidc_provider) context: ```nginx http { resolver 10.0.0.1 ipv4=on valid=300s; oidc_provider entra { # ... } # ... } ``` 6. In the [`oidc_provider {}`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#oidc_provider) context, specify: - your **Client ID** obtained in [Entra ID Configuration](#entra-setup) with the [`client_id`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_id) directive - your **Client Secret** obtained in [Entra ID Configuration](#entra-setup) with the [`client_secret`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_secret) directive - the **Issuer** URL obtained in [Entra ID Configuration](#entra-setup) with the [`issuer`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_secret) directive The `issuer` is typically: `https://login.microsoftonline.com//v2.0`. By default, NGINX Plus creates the metadata URL by appending the `/.well-known/openid-configuration` part to the Issuer URL. If your metadata URL is different, you can explicitly specify it with the [`config_url`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#config_url) directive. - The **logout_uri** is URI that a user visits to start an RP‑initiated logout flow. - The **post_logout_uri** is absolute HTTPS URL where Microsoft Entra ID should redirect the user after a successful logout. This value **must also be configured** in the Entra ID application logout URLs. - If the **logout_token_hint** directive set to `on`, NGINX Plus sends the user's ID token as a *hint* to Microsoft Entra ID. This directive is **optional**, however, if it is omitted the Microsoft Entra ID may display an extra confirmation page asking the user to approve the logout request. If the “Require ID token in logout requests” option is enabled in your tenant (commonly the case in Azure AD B2C), then the token hint becomes **mandatory**. - The **frontchannel_logout_uri** directive defines the URI that receives OpenID Connect front-channel logout requests from Microsoft Entra ID. This URI must be an HTTPS path hosted by NGINX Plus and must match the *Front-channel logout URL* configured in the Entra ID application registration. When a front-channel logout GET request is received at this URI (typically in a hidden iframe), the OIDC module clears the local session for the affected user. - If the **userinfo** directive is set to `on`, NGINX Plus will fetch userinfo from Microsoft Graph API and append the claims from userinfo to the `$oidc_claims_` variables. - PKCE (Proof Key for Code Exchange) is automatically enabled when the provider metadata advertises the `S256` code challenge method in the `code_challenge_methods_supported` field of the discovery document. You can override this behavior with the [`pkce`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#pkce) directive: set `pkce off;` to disable PKCE even when `S256` is advertised, or `pkce on;` to force PKCE even if the IdP metadata does not list `S256`. - The module automatically selects the client authentication method for the token endpoint based on the provider metadata `token_endpoint_auth_methods_supported`. When only `client_secret_post` is advertised, NGINX Plus uses the `client_secret_post` method and sends the client credentials in the POST body. When both `client_secret_basic` and `client_secret_post` are present, the module prefers HTTP Basic (`client_secret_basic`), which remains the default for Microsoft Entra ID. - **Note:** All interaction with the IdP is secured exclusively over SSL/TLS, so NGINX must trust the certificate presented by the IdP. By default, this trust is validated against your system’s CA bundle (the default CA store for your Linux or FreeBSD distribution). If the IdP’s certificate is not included in the system CA bundle, you can explicitly specify a trusted certificate or chain with the [`ssl_trusted_certificate`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#ssl_trusted_certificate) directive so that NGINX can validate and trust the IdP’s certificate. ```nginx http { resolver 10.0.0.1 ipv4=on valid=300s; oidc_provider entra { issuer https://login.microsoftonline.com//v2.0; client_id ; client_secret ; logout_uri /logout; post_logout_uri https://demo.example.com/post_logout/; logout_token_hint on; frontchannel_logout_uri /front_logout; userinfo on; # Optional: PKCE configuration. By default, PKCE is automatically # enabled when the IdP advertises the S256 code challenge method. # pkce on; } # ... } ``` 7. Make sure you have configured a [server](https://nginx.org/en/docs/http/ngx_http_core_module.html#server) that corresponds to `demo.example.com`, and there is a [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) that [points](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass) to your application (see [Step 10](#oidc_app)) at `http://127.0.0.1:8080` that is going to be OIDC-protected: ```nginx http { # ... server { listen 443 ssl; server_name demo.example.com; ssl_certificate /etc/ssl/certs/fullchain.pem; ssl_certificate_key /etc/ssl/private/key.pem; location / { # ... proxy_pass http://127.0.0.1:8080; } } # ... } ``` 8. Protect this [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) with Entra ID OIDC by specifying the [`auth_oidc`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#auth_oidc) directive that will point to the `entra` configuration specified in the [`oidc_provider {}`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#oidc_provider) context in [Step 5](#entra-setup-oidc-provider): ```nginx # ... location / { auth_oidc entra; # ... proxy_pass http://127.0.0.1:8080; } # ... ``` 9. Pass the OIDC claims as headers to the application ([Step 10](#oidc_app)) with the [`proxy_set_header`](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_header) directive. These claims are extracted from the ID token returned by Entra ID: - [`$oidc_claim_sub`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) - a unique `Subject` identifier assigned for each user by Entra ID - [`$oidc_claim_email`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) - the e-mail address of the user - [`$oidc_claim_name`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) - the full name of the user - any other OIDC claim using the [`$oidc_claim_ `](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) variable ```nginx # ... location / { auth_oidc entra; proxy_set_header sub $oidc_claim_sub; proxy_set_header email $oidc_claim_email; proxy_set_header name $oidc_claim_name; proxy_pass http://127.0.0.1:8080; } # ... ``` 10. Provide endpoint for completing logout: ```nginx # ... location /post_logout/ { return 200 "You have been logged out.\n"; default_type text/plain; } # ... ``` 11. Create a simple test application referenced by the `proxy_pass` directive which returns the authenticated user's full name and email upon successful authentication: ```nginx # ... server { listen 8080; location / { return 200 "Hello, $http_name!\nEmail: $http_email\nEntra ID sub: $http_sub\n"; default_type text/plain; } } ``` 12. Save the NGINX configuration file and reload the configuration: ```nginx nginx -s reload ``` ### Complete Example This configuration example summarizes the steps outlined above. It includes only essential settings such as specifying the DNS resolver, defining the OIDC provider, configuring SSL, and proxying requests to an internal server. ```nginx http { # Use a public DNS resolver for Issuer discovery, etc. resolver 10.0.0.1 ipv4=on valid=300s; oidc_provider entra { # The issuer is typically something like: # https://login.microsoftonline.com//v2.0 issuer https://login.microsoftonline.com//v2.0; # Replace with your actual Entra client_id and client_secret client_id ; client_secret ; # RP‑initiated logout logout_uri /logout; post_logout_uri https://demo.example.com/post_logout/; logout_token_hint on; # Front-channel logout (OP‑initiated single sign-out) frontchannel_logout_uri /front_logout; # Fetch userinfo claims userinfo on; # Optional: PKCE configuration (enabled automatically when supported by the IdP) # pkce on; } server { listen 443 ssl; server_name demo.example.com; ssl_certificate /etc/ssl/certs/fullchain.pem; ssl_certificate_key /etc/ssl/private/key.pem; location / { # Protect this location with Entra OIDC auth_oidc entra; # Forward OIDC claims as headers if desired proxy_set_header sub $oidc_claim_sub; proxy_set_header email $oidc_claim_email; proxy_set_header name $oidc_claim_name; proxy_pass http://127.0.0.1:8080; } location /post_logout/ { return 200 "You have been logged out.\n"; default_type text/plain; } } server { # Simple test upstream server listen 8080; location / { return 200 "Hello, $http_name!\nEmail: $http_email\nEntra ID sub: $http_sub\n"; default_type text/plain; } } } ``` ### Testing 1. Open `https://demo.example.com/` in a browser. You will be automatically redirected to the Entra ID sign-in page. 2. Enter valid Entra ID credentials of a user who has access the application. Upon successful sign-in, Entra ID redirects you back to NGINX Plus, and you will see the proxied application content (for example, "Hello, Jane Doe!"). 3. Navigate to `https://demo.example.com/logout`. NGINX Plus initiates an RP‑initiated logout; Microsoft Entra ID ends the session and redirects back to `https://demo.example.com/post_logout/`. 4. Refresh `https://demo.example.com/` again. You should be redirected to Microsoft Entra ID for a fresh sign‑in, proving the session has been terminated. **Note:** If you restricted access to a group of users, be sure to select a user who has access to the application. ## See Also - [Microsoft identity platform documentation](https://learn.microsoft.com/en-us/entra/identity-platform/) - [NGINX Plus Native OIDC Module Reference documentation](https://nginx.org/en/docs/http/ngx_http_oidc_module.html) - [Release Notes for NGINX Plus R36](nginx/releases.md#r36) ## Revision History - Version 3 (November 2025) – Updated for NGINX Plus R36; added front-channel logout support (`frontchannel_logout_uri`), PKCE configuration (`pkce` directive), and the `client_secret_post` token endpoint authentication method. - Version 2 (August 2025) – Updated for NGINX Plus R35; added RP‑initiated logout (`logout_uri`, `post_logout_uri`, `logout_token_hint`) and `userinfo` support. - Version 1 (March 2025) – Initial version (NGINX Plus Release 34). -------------------------------------------------------------------------------- title: "TCP Health Checks" url: https://docs.nginx.com/nginx/admin-guide/load-balancer/tcp-health-check/index.md description: Monitor the health of TCP servers in an upstream group by sending periodic health checks, including customizable active health checks in F5 NGINX Plus. -------------------------------------------------------------------------------- ## Introduction NGINX and F5 NGINX Plus can continually test your TCP upstream servers, avoid the servers that have failed, and gracefully add the recovered servers into the load‑balanced group. ## Prerequisites - You have configured an upstream group of TCP servers in the [`stream`](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#stream) context, for example: ```nginx stream { #... upstream stream_backend { server backend1.example.com:12345; server backend2.example.com:12345; server backend3.example.com:12345; } #... } ``` - You have configured a server that passes TCP connections to the server group: ```nginx stream { #... server { listen 12345; proxy_pass stream_backend; } #... } ``` ## Passive TCP Health Checks If an attempt to connect to an upstream server times out or results in an error, NGINX Open Source or NGINX Plus can mark the server as unavailable and stop sending requests to it for a defined amount of time. To define the conditions under which NGINX considers an upstream server unavailable, include the following parameters to the [`server`](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#server) directive - [`fail_timeout`](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#fail_timeout) – The amount of time within which a specified number of connection attempts must fail for the server to be considered unavailable. Also, the amount of time that NGINX considers the server unavailable after marking it so. - [`max_fails`](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#max_fails) – The number of failed attempts that happen during the specified time for NGINX to consider the server unavailable. The default values are `10` seconds and `1` attempt. So if a connection attempt times out or fails at least once in a 10‑second period, NGINX marks the server as unavailable for 10 seconds. The example shows how to set these parameters to 2 failures within 30 seconds: ```nginx upstream stream_backend { server backend1.example.com:12345 weight=5; server backend2.example.com:12345 max_fails=2 fail_timeout=30s; server backend3.example.com:12346 max_conns=3; } ``` ### Server Slow Start A recently recovered upstream server can be easily overwhelmed by connections, which may cause the server to be marked as unavailable again. Slow start allows an upstream server to gradually recover its weight from zero to its nominal value after it has been recovered or became available. This can be done with the [`slow_start`](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#slow_start) parameter of the upstream [`server`](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#server) directive: ```nginx upstream backend { server backend1.example.com:12345 slow_start=30s; server backend2.example.com; server 192.0.0.1 backup; } ``` Note that if there is only a single server in a group, the `slow_start` parameter is ignored and the server is never marked unavailable. Slow start is exclusive to NGINX Plus. ## Active TCP Health Checks Health checks can be configured to test a wide range of failure types. For example, NGINX Plus can continually test upstream servers for responsiveness and avoid servers that have failed. NGINX Plus sends special health check requests to each upstream server and checks for a response that satisfies certain conditions. If a connection to the server cannot be established, the health check fails, and the server is considered unhealthy. NGINX Plus does not proxy client connections to unhealthy servers. If several health checks are configured for an upstream group, the failure of any check is enough to consider the corresponding server unhealthy. To enable active health checks: 1. Specify a _shared memory zone_ – a special area where the NGINX Plus worker processes share state information about counters and connections. Add the [`zone`](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#zone) directive to the upstream server group and specify the zone name (here, **stream_backend**) and the amount of memory (64 KB): ```nginx stream { #... upstream stream_backend { zone stream_backend 64k; server backend1.example.com:12345; server backend2.example.com:12345; server backend3.example.com:12345; } #... } ``` 2. Enable active health checks for the upstream group with the [`health_check`](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#health_check) directive: ```nginx stream { #... server { listen 12345; proxy_pass stream_backend; health_check; #... } } ``` 3. If necessary, reduce a timeout between two consecutive health checks with the [`health_check_timeout`](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#health_check_timeout) directive. This directive overrides the [`proxy_timeout`](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_timeout) value for health checks, as for health checks this timeout needs to be significantly shorter: ```nginx stream { #... server { listen 12345; proxy_pass stream_backend; health_check; health_check_timeout 5s; } } ``` 4. By default, NGINX Plus sends health check messages to the port specified by the [`server`](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#server) directive in the [`upstream`](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#upstream) block. You can specify another port for health checks, which is particularly helpful when monitoring the health of many services on the same host. To override the port, specify the [`port`](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#health_check_port) parameter of the [`health_check`](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#health_check) directive: ```nginx stream { #... server { listen 12345; proxy_pass stream_backend; health_check port=12346; health_check_timeout 5s; } } ``` ### Fine-Tuning TCP Health Checks By default, NGINX Plus tries to connect to each server in an upstream server group every `5` seconds. If the connection cannot be established, NGINX Plus considers the health check failed, marks the server as unhealthy, and stops forwarding client connections to the server. To change the default behavior, include parameters to the [`health_check`](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#health_check) directive: - [`interval`](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#health_check_interval) – How often (in seconds) NGINX Plus sends health check requests (default is `5` seconds) - [`passes`](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#health_check_passes) – Number of consecutive health checks the server must respond to to be considered healthy (default is `1`) - [`fails`](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#health_check_fails) – Number of consecutive health checks the server must fail to respond to to be considered unhealthy (default is `1`) ```nginx stream { #... server { listen 12345; proxy_pass stream_backend; health_check interval=10 passes=2 fails=3; } #... } ``` In the example, the time between TCP health checks is increased to `10` seconds, the server is considered unhealthy after `3` consecutive failed health checks, and the server needs to pass `2` consecutive checks to be considered healthy again. ### The “match {}” Configuration Block You can create your own tests to verify server responses to health checks. These tests are defined with the [`match {}`](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#match) configuration block placed in the [`stream {}`](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#stream) context. 1. On the [`stream {}`](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#stream) level, specify the [`match {}`](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#match) block and name it, for example, `tcp_test`: ```nginx stream { #... match tcp_test { #... } } ``` This block will contain tests described in **Step 3**. 2. Refer to the block from the [`health_check`](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#health_check) directive by specifying the [`match`](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#health_check_match) parameter and the name of the [`match`](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#match) block: ```nginx stream { #... server { listen 12345; health_check match=tcp_test; proxy_pass stream_backend; } #... } ``` 3. Within the [`match`](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#match) block, specify the conditions or tests under which a health check succeed. The block can accept the following parameters: - [`send`](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#match_send) – The text string or hexadecimal literals (“\x” followed by two hex digits) to send to the server - [`expect`](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#match_expect) – Literal string or regular expression that the data returned by the server needs to match These parameters can be used in different combinations, but no more than one `send` and one `expect` parameter can be specified at a time: - If no `send` or `expect` parameters are specified, the ability to connect to the server is tested. - If the `expect` parameter is specified, the server is expected to unconditionally send data first: ```nginx match pop3 { expect ~* "\+OK"; } ``` - If the `send` parameter is specified, it is expected that the connection will be successfully established and the specified string will be sent to the server: ```nginx match pop_quit { send QUIT; } ``` - If both the `send` and `expect` parameters are specified, then the string from the `send` parameter must match the regular expression from the `expect` parameter: ```nginx stream { #... upstream stream_backend { zone upstream_backend 64k; server backend1.example.com:12345; } match http { send "GET / HTTP/1.0\r\nHost: localhost\r\n\r\n"; expect ~* "200 OK"; } server { listen 12345; health_check match=http; proxy_pass stream_backend; } } ``` The example shows that in order for a health check to pass, the HTTP request must be sent to the server, and the expected result from the server contains `200` `OK` to indicate a successful HTTP response. -------------------------------------------------------------------------------- title: "Technical Specs" url: https://docs.nginx.com/nginx/technical-specs/index.md description: Platforms supported by F5 NGINX Plus and dynamically loaded modules, supported SSL/TLS versions, supported deployment environments, and list of modules built into NGINX Plus. -------------------------------------------------------------------------------- NGINX Plus is available only as a binary; it is not distributed as source code. For additional platforms and modules, [contact us](https://www.f5.com/products/get-f5). ## Supported Distributions {#supported-distributions} | Distribution | Supported on [PLS.37.1](nginx/releases.md#r37.0) | Supported on [PLS.37.0](nginx/releases.md#r36) | |------------------------------|---------------------------------------------------------------------------------|---------------------------------------------------------| | AlmaLinux | 8.1+ (x86_64, aarch64)
9.7+ (x86_64, aarch64)
10 (x86_64, aarch64) | 8.1+ (x86_64, aarch64)
9.7+ (x86_64, aarch64)
10 (x86_64, aarch64) | | Alpine Linux | 3.21 (x86_64, aarch64)
3.22 (x86_64, aarch64)
3.23 (x86_64, aarch64)
3.24 (x86_64, aarch64) | 3.21 (x86_64, aarch64)
3.22 (x86_64, aarch64)
3.23 (x86_64, aarch64) | | Amazon Linux | 2023 (x86_64, aarch64) | 2 (x86_64, aarch64)
2023 (x86_64, aarch64) | | Debian | 11 (x86_64, aarch64)
12 (x86_64, aarch64)
13 (x86_64, aarch64) | 11 (x86_64, aarch64)
12 (x86_64, aarch64)
13 (x86_64, aarch64) | | FreeBSD | 14.3+ (amd64)
15.0+ (amd64) | 13.5+ (amd64)
14.3+ (amd64)
15.0+ (amd64) | | Oracle Linux | 8.1+ (x86_64, aarch64)
9.7+ (x86_64, aarch64) | 8.1+ (x86_64, aarch64)
9.7+ (x86_64, aarch64) | | Red Hat Enterprise Linux | 8.1+ (x86_64, aarch64)
9.7+ (x86_64, aarch64)
10.1+ (x86_64, aarch64) | 8.1+ (x86_64, aarch64)
9.7+ (x86_64, aarch64)
10.1+ (x86_64, aarch64) | | Rocky Linux | 8.1+ (x86_64, aarch64)
9.7+ (x86_64, aarch64)
10.1+ (x86_64, aarch64) | 8.1+ (x86_64, aarch64)
9.7+ (x86_64, aarch64)
10.1+ (x86_64, aarch64) | | SUSE Linux Enterprise Server | 15 SP7+ (x86_64)
16 (x86_64, aarch64) | 15 SP7+ (x86_64)
16 (x86_64, aarch64) | | Ubuntu | 22.04 (x86_64, aarch64)
24.04 (x86_64, aarch64)
26.04 (x86_64, aarch64) | 22.04 (x86_64, aarch64)
24.04 (x86_64, aarch64)
26.04 (x86_64, aarch64) | ## Dynamic Modules [Dynamic modules](/nginx/admin-guide/dynamic-modules/dynamic-modules.md#nginx-authored-and-nginx-certified-dynamic-modules) are supported on the [same distributions as NGINX Plus](#supported-distributions), unless noted otherwise in the table below. | | [F5 WAF for NGINX](/waf/changelog.md#packages) | [GeoIP](/nginx/admin-guide/dynamic-modules/geoip.md) | [HA-Keepalived](/nginx/admin-guide/high-availability/ha-keepalived-nodes.md#configuring-keepalived-for-an-additional-passive-node) | [NGINX sync](/nginx/admin-guide/high-availability/configuration-sharing.md#installing-nginx-sync-on-the-primary-machine) | |------------------------|----------------------|------------------|-----------------------------|---------------| | **RHEL-based*** | See [Packages](/waf/changelog/_index.md#packages) | ❌ | ✓ | ✓ | | **Alpine Linux** | See [Packages](/waf/changelog/_index.md#packages) | ✓ | ❌ | ❌ | | **Amazon Linux 2023** | See [Packages](/waf/changelog/_index.md#packages) | ❌ | ❌ | ✓ | | **Debian**, **Ubuntu** | See [Packages](/waf/changelog/_index.md#packages) | ✓ | ✓ | ✓ | | **FreeBSD** | ❌ | ❌ | ❌ | ❌ | | **SLES** | ❌ | 15 only| ✓ | ✓ | *RHEL-based operating systems include Red Hat Enterprise Linux, AlmaLinux, Oracle Linux, Rocky Linux. ## Supported SSL/TLS versions NGINX Plus supports the following SSL/TLS protocols: - SSLv2 - SSLv3 - TLSv1 - TLSv1.1 - TLSv1.2 - TLSv1.3 You can configure which protocols to enable or disable with the [ssl_protocols](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_protocols) directive. TLSv1.2 and earlier are supported on all operating systems listed in [Supported Distributions](#supported-distributions). TLSv1.3 is supported starting from NGINX Plus R17 and is enabled by default in NGINX Plus R29 and later. It requires OpenSSL 1.1.1 or higher. Note that not all operating systems supported by NGINX Plus include OpenSSL 1.1.1. Check your operating system's documentation to confirm TLSv1.3 compatibility. TLSv1.2 and TLSv1.3 are the default SSL protocols starting from NGINX Plus R34 (if supported by the OpenSSL library). If OpenSSL 1.0.0 or older is used, the default SSL protocols are TLSv1 and TLSv1.1. **Note:** NGINX Plus is built on the latest minor release of each supported operating system platform. In many cases, the latest revisions of these operating systems are adapting their platforms to support OpenSSL 3.5 (for example, RHEL 9.7 and 10.1). In these situations, NGINX Plus requires that OpenSSL 3.5.0 or later is installed for proper operation. ## Supported Deployment Environments - Bare metal - [Containers](/nginx/admin-guide/installing-nginx/installing-nginx-docker.md) and Kubernetes - Public clouds: AWS, Google Cloud Platform, Microsoft Azure - Virtual machines ## Recommended Hardware See [Sizing Guide for Deploying NGINX Plus](https://www.f5.com/pdf/deployment-guide/Sizing-Guide-for-Deploying-NGINX-Plus-in-Virtualized-Environments-2021-06-03.pdf) ## Modules in the NGINX Plus Package ### Core - [Core](https://nginx.org/en/docs/ngx_core_module.html) – Control basic functioning (mutexes, events, thread pools, workers, and so on) ### Clustering - [Zone Sync](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html) – Synchronize [shared memory zones](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#zone) among nodes in a cluster ### HTTP Core - [HTTP Core](https://nginx.org/en/docs/http/ngx_http_core_module.html) – Process HTTP traffic - [Addition](https://nginx.org/en/docs/http/ngx_http_addition_module.html) – Prepend and append data to a response - [Auto Index](https://nginx.org/en/docs/http/ngx_http_autoindex_module.html) – Generate directory listings - [Charset](https://nginx.org/en/docs/http/ngx_http_charset_module.html) – Add character set in `Content-Type` field of HTTP response header, and define or convert between character sets - [Empty GIF](https://nginx.org/en/docs/http/ngx_http_empty_gif_module.html) – Generate empty image response - [Gunzip](https://nginx.org/en/docs/http/ngx_http_gunzip_module.html) – Decompress responses for clients that don’t support compression - [Gzip](https://nginx.org/en/docs/http/ngx_http_gzip_module.html) – Use GZIP to compress HTTP responses - [Gzip Static](https://nginx.org/en/docs/http/ngx_http_gzip_static_module.html) – Serve pre-compressed files from disk - [Headers](https://nginx.org/en/docs/http/ngx_http_headers_module.html) – Add fields to HTTP response headers, including `Cache-Control` and `Expires` - [Index](https://nginx.org/en/docs/http/ngx_http_index_module.html) – Specify index files used in directory requests - [Internal Redirect](https://nginx.org/en/docs/http/ngx_http_internal_redirect_module.html) – Allow internal redirects after checking request or connection processing limits, and access limits - [Random Index](https://nginx.org/en/docs/http/ngx_http_random_index_module.html) – Select random index file for directory request - [Real IP](https://nginx.org/en/docs/http/ngx_http_realip_module.html) – Determine true origin IP address for proxied traffic - [SSI](https://nginx.org/en/docs/http/ngx_http_ssi_module.html) – Process Server Side Includes (SSI) commands - [User ID](https://nginx.org/en/docs/http/ngx_http_userid_module.html) – Set cookies that uniquely identify clients - [WebDAV](https://nginx.org/en/docs/http/ngx_http_dav_module.html) – Implement WebDAV file management ### HTTP Access Control and Authentication - [Access](https://nginx.org/en/docs/http/ngx_http_access_module.html) – Control access based on client IP address (support access control lists [ACLs]) - [Auth Basic](https://nginx.org/en/docs/http/ngx_http_auth_basic_module.html) – Implement HTTP Basic Authentication scheme - [Auth JWT](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html) – Validate JSON Web Tokens - [Auth Request](https://nginx.org/en/docs/http/ngx_http_auth_request_module.html) – Determine client authorization using subrequests to external authentication server - [Auth Require](https://nginx.org/en/docs/http/ngx_http_auth_require_module.html) – Variable-based access control support - [OIDC](https://nginx.org/en/docs/http/ngx_http_oidc_module.html) – Implement authentication as a Relying Party in OpenID Connect solution - [Referer](https://nginx.org/en/docs/http/ngx_http_referer_module.html) – Control access based on `Referer` field in HTTP request header - [Secure Link](https://nginx.org/en/docs/http/ngx_http_secure_link_module.html) – Process encrypted, time-limited links to content ### HTTP Advanced Configuration - [Browser](https://nginx.org/en/docs/http/ngx_http_browser_module.html) – Create variables based on `User-Agent` field in HTTP request header - [Cache Slice](https://nginx.org/en/docs/http/ngx_http_slice_module.html) – Create byte-range segments of large files, for more efficient caching - [Geo](https://nginx.org/en/docs/http/ngx_http_geo_module.html) – Create variables based on client IP address - [Map](https://nginx.org/en/docs/http/ngx_http_map_module.html) – Create variables based on other variables in requests - [Num_map](https://nginx.org/en/docs/http/ngx_http_num_map_module.html) – Create variables whose values depend on numeric values or numeric value ranges - [Rewrite](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html) – Test and change URI of request - [Split Clients](https://nginx.org/en/docs/http/ngx_http_split_clients_module.html) – Partition clients for A/B testing - [Sub](https://nginx.org/en/docs/http/ngx_http_sub_module.html) – Replace text string in response (rewrite content) ### HTTP Logging - [Log](https://nginx.org/en/docs/http/ngx_http_log_module.html) – Log HTTP transactions locally or to `syslog` - [Session Log](https://nginx.org/en/docs/http/ngx_http_session_log_module.html) – Log HTTP transactions aggregated per session ### HTTP Media Delivery - [F4F](https://nginx.org/en/docs/http/ngx_http_f4f_module.html) – Stream HDS (Adobe HTTP Dynamic Streaming; filename extensions **.f4f**, **.f4m**, **.f4x**) - [FLV](https://nginx.org/en/docs/http/ngx_http_flv_module.html) – Stream FLV (Flash Video; filename extension **.flv**) - [HLS](https://nginx.org/en/docs/http/ngx_http_hls_module.html) – Stream HLS (Apple HTTP Live Streaming; filename extensions **.m3u8**, **.ts**) dynamically generated from MP4 or MOV (filename extensions **.m4a**, **.m4v**, **.mov**, **.mp4**, and **.qt**) - [MP4](https://nginx.org/en/docs/http/ngx_http_mp4_module.html) – Stream MP4 (filename extensions **.m4a**, **.m4v**, **.mp4**) - Streaming of RTMP and DASH is provided by the third-party [RTMP](https://github.com/arut/nginx-rtmp-module) module ### HTTP Proxying - [FastCGI](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html) – Proxy and cache requests to FastCGI server - [gRPC](https://nginx.org/en/docs/http/ngx_http_grpc_module.html) – Proxy requests to gRPC server - [Memcached](https://nginx.org/en/docs/http/ngx_http_memcached_module.html) – Proxy requests to memcached server - [Mirror](https://nginx.org/en/docs/http/ngx_http_mirror_module.html) – Send copy of requests to one or more additional servers - [Proxy](https://nginx.org/en/docs/http/ngx_http_proxy_module.html) – Proxy and cache requests to HTTP server - [SCGI](https://nginx.org/en/docs/http/ngx_http_scgi_module.html) – Proxy and cache requests to SCGI server - [Tunnel](https://nginx.org/en/docs/http/ngx_http_tunnel_module.html) – Handle CONNECT requests and establishes an end-to-end virtual connection for forward proxying - [Upstream](https://nginx.org/en/docs/http/ngx_http_upstream_module.html) – Proxy and cache requests to load-balanced pool of servers - [Upstream Health Checks](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html) – Verify servers in load-balanced pool are operational - [uwsgi](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html) – Proxy and cache requests to uwsgi server ### HTTP Transaction Shaping - [Limit Connections](https://nginx.org/en/docs/http/ngx_http_limit_conn_module.html) – Limit concurrent connections from a client IP address or other keyed value - [Limit Requests](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html) – Limit rate of request processing for a client IP address or other keyed value - [Limit Responses](https://nginx.org/en/docs/http/ngx_http_core_module.html#limit_rate) – Limit rate of responses per client connection ### HTTP/2, HTTP/3 and SSL/TLS - [HTTP/2](https://nginx.org/en/docs/http/ngx_http_v2_module.html) – Process HTTP/2 traffic - [HTTP/3](https://nginx.org/en/docs/http/ngx_http_v3_module.html) – Process HTTP/3 traffic - [SSL/TLS](https://nginx.org/en/docs/http/ngx_http_ssl_module.html) – Process HTTPS traffic ### Mail - [Mail Core](https://nginx.org/en/docs/mail/ngx_mail_core_module.html) – Proxy mail traffic - [Auth HTTP](https://nginx.org/en/docs/mail/ngx_mail_auth_http_module.html) – Offload authentication processing from HTTP server - [IMAP](https://nginx.org/en/docs/mail/ngx_mail_imap_module.html) – Implement capabilities and authentication methods for IMAP - [POP3](https://nginx.org/en/docs/mail/ngx_mail_pop3_module.html) – Implement authentication methods for POP3 traffic - [Proxy](https://nginx.org/en/docs/mail/ngx_mail_proxy_module.html) – Support proxy-related parameters for mail protocols - [SMTP](https://nginx.org/en/docs/mail/ngx_mail_smtp_module.html) – Define accepted SASL authentication methods for SMTP clients - [SSL/TLS](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html) – Implement SSL, STARTTLS, and TLS for mail protocols ### Programmability and Monitoring - [NGINX Plus API](https://nginx.org/en/docs/http/ngx_http_api_module.html) – Provide REST API for accessing metrics, configuring upstream server groups dynamically, and managing key-value pairs, without the need to reload NGINX configuration - [Key-Value Store](https://nginx.org/en/docs/http/ngx_http_keyval_module.html) – Create variables with values taken from key-value pairs managed by the [NGINX Plus API](https://nginx.org/en/docs/http/ngx_http_api_module.html#http_keyvals_) - [Management](https://nginx.org/en/docs/ngx_mgmt_module.html) – [Configure licensing and usage reporting](solutions/about-subscription-licenses.md) of NGINX Plus installation to F5 licensing endpoint or [NGINX Instance Manager](nim/index.md) ### TCP and UDP Proxying and Load Balancing - [Stream](https://nginx.org/en/docs/stream/ngx_stream_module.html) – Process TCP and UDP traffic - [Access](https://nginx.org/en/docs/stream/ngx_stream_access_module.html) – Support IP-based access control lists (ACLs) - [Geo](https://nginx.org/en/docs/stream/ngx_stream_geo_module.html) – Create variables based on client IP address - [Limit Conn](https://nginx.org/en/docs/stream/ngx_stream_limit_conn_module.html) – Limit concurrent connections by key - [Log](https://nginx.org/en/docs/stream/ngx_stream_log_module.html) – Log TCP and UDP transactions - [Map](https://nginx.org/en/docs/stream/ngx_stream_map_module.html) – Create variables based on other variables in requests - [Num_map](https://nginx.org/en/docs/stream/ngx_stream_num_map_module.html) – Create variables whose values depend on numeric values or numeric value ranges - [MQTT Preread](https://nginx.org/en/docs/stream/ngx_stream_mqtt_preread_module.html) – Forward MQTT traffic without processing it - [MQTT Filter](https://nginx.org/en/docs/stream/ngx_stream_mqtt_filter_module.html) – Process Message Queuing Telemetry Transport protocol (MQTT) protocol - [Proxy](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html) – Proxy requests to TCP and UDP servers - [Pass](https://nginx.org/en/docs/stream/ngx_stream_pass_module.html) – Pass any accepted client connection to any configured listening socket in http, stream, mail, and other similar modules - [Real IP](https://nginx.org/en/docs/stream/ngx_stream_realip_module.html) – Determine true origin IP address for proxied traffic - [Return](https://nginx.org/en/docs/stream/ngx_stream_return_module.html) – Return specified value to client and close connection - [Split Clients](https://nginx.org/en/docs/stream/ngx_stream_split_clients_module.html) – Partition clients for A/B testing - [SSL/TLS](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html) – Process TCP traffic secured with SSL/TLS - [SSL/TLS Preread](https://nginx.org/en/docs/stream/ngx_stream_ssl_preread_module.html) – Forward TCP traffic secured with SSL/TLS without decrypting it - [Upstream](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html) – Proxy and cache traffic to load-balanced pool of servers - [Upstream Health Checks](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html) – Verify servers in load-balanced pool are operational -------------------------------------------------------------------------------- title: "Using NGINX or NGINX Plus as the Ingress Controller for Amazon Elastic Kubernetes Services" url: https://docs.nginx.com/nginx/deployment-guides/amazon-web-services/ingress-controller-elastic-kubernetes-services/index.md description: Use NGINX or F5 NGINX Plus as the Ingress Controller for Amazon Elastic Kubernetes Services. -------------------------------------------------------------------------------- This guide explains how to use NGINX Open Source or F5 NGINX Plus with NGINX Ingress Controller for Amazon Elastic Kubernetes Services (EKS). **Note:** These instructions apply to NGINX Ingress Controller with NGINX Open Source or NGINX Plus. For ease of reading, the document refers to NGINX Plus only. ## Prerequisites - [An AWS account](https://docs.aws.amazon.com/AmazonSimpleDB/latest/DeveloperGuide/AboutAWSAccounts.html). - A prebuilt image of the NGINX or NGINX Plus Ingress Controller for Kubernetes. - For NGINX Open Source you can use the pre-built image [on DockerHub](https://hub.docker.com/r/nginx/nginx-ingress/). You can also build your own image. - For NGINX Plus, you must [build an image](https://docs.nginx.com/nginx-ingress-controller/installation/build-nginx-ingress-controller/). **Note:** Never upload your NGINX Plus Ingress Controller images to a public repository such as Docker Hub. Doing so violates your license agreement. Run the `make container` command below. ```shell make container DOCKERFILE=DockerfileForPlus PREFIX=nginx/nginx-plus-ingress ``` The `PREFIX` argument specifies the repo name in your private container registry. In this example, we set it to `nginx/nginx-plus-ingress`. You can later use that name to reference the image instead of its numerical ID. ## Create an Amazon EKS Cluster You can create an Amazon EKS cluster with: - the AWS Management Console - the AWS CLI - the `eksctl` command line utility. This guide covers the `eksctl` command as it is the simplest option. 1. Follow the instructions in the [eksctl.io documentation](https://eksctl.io/installation/) to install or update the `eksctl` command. 2. Create an Amazon EKS cluster by following the instructions in the [AWS documentation](https://docs.aws.amazon.com/eks/latest/userguide/getting-started-eksctl.html). Select the **Managed nodes – Linux** option for each step. Note that the `eksctl create cluster` command in the first step can take ten minutes or more. ## Push the NGINX Plus Ingress Controller Image to AWS ECR This step is only required if you do not plan to use the prebuilt NGINX Open Source image. 1. Use the [AWS documentation](https://docs.aws.amazon.com/AmazonECR/latest/userguide/repository-create.html) to create a repository in the Amazon Elastic Container Registry (ECR). In Step 4 of the AWS instructions, name the repository **nginx-plus-ic** as that is what we use in this guide. 2. Run the following AWS CLI command. It generates an auth token for your AWS ECR registry, then pipes it into the `docker login` command. This lets AWS ECR authenticate and authorize the upcoming Docker requests. For details about the command, see the [AWS documentation](https://docs.aws.amazon.com/AmazonECR/latest/userguide/registry_auth.html). ```shell aws ecr get-login-password --region | docker login --username AWS --password-stdin .dkr.ecr..amazonaws.com ``` - `` is the same region name you specified in Step 2 above. - `` is your AWS account number. For instructions on retrieving the ID, see the [AWS documentation](https://docs.aws.amazon.com/IAM/latest/UserGuide/console_account-alias.html). 3. Run the following command to apply the tag `edge` to your NGINX Ingress Controller image: ```shell docker tag :edge .dkr.ecr..amazonaws.com/:edge ``` - `` is the repo name you set with the `PREFIX` parameter to the `make container` command (see [Prerequisites](#prereqs)). In this guide it is `nginx/nginx-plus-ingress`. - `` is the AWS ECR repository you created in Step 1 above. In this guide it is called `nginx-plus-ic`. The final command is: ```shell docker tag nginx/nginx-plus-ingress:edge .dkr.ecr..amazonaws.com/nginx-plus-ic:edge ``` 5. Push the NGINX Plus Ingress Controller image to AWS ECR: ```shell docker push .dkr.ecr..amazonaws.com/:edge ``` ## Install the NGINX Plus Ingress Controller Use [our documentation](https://docs.nginx.com/nginx-ingress-controller/installation/installation-with-manifests/) to install the NGINX Plus Ingress Controller in your Amazon EKS cluster. Complete the steps up to and including [Confirm NGINX Ingress Controller is running](https://docs.nginx.com/nginx-ingress-controller/installation/installing-nic/installation-with-manifests/#confirm-nginx-ingress-controller-is-running). Next, follow the instructions below to create a Network Load Balancer to route traffic to NGINX Plus Ingress Controller. --- ## Use a Network Load Balancer in front of NGINX Ingress Controller These steps assume you've cloned the [kubernetes-ingress](https://github.com/nginx/kubernetes-ingress) repository in the previous step. You need a Kubernetes `LoadBalancer` service to route traffic to the NGINX Ingress Controller. By default, Amazon EKS will create a [Classic Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/introduction.html) for Kubernetes services of type `LoadBalancer`. However, we recommend that you create a [Network Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/introduction.html) (NLB). It operates at the transport layer and is optimized for high performance and low latency. We also recommend enabling the PROXY Protocol for both the NGINX Plus Ingress Controller and your NLB target groups. This is used to forward client connection information. If you choose not to enable the PROXY protocol, see the [Appendix](#appendix). ### Configure a `LoadBalancer` Service to Use NLB Apply the manifest `deployments/service/loadbalancer-aws-elb.yaml` to create a `LoadBalancer` of type NLB: ```shell kubectl apply -f deployments/service/loadbalancer-aws-elb.yaml ``` ### Enable the PROXY Protocol 1. Add the following keys to the `deployments/common/nginx-config.yaml` config map file: ```yaml proxy-protocol: "True" real-ip-header: "proxy_protocol" set-real-ip-from: "0.0.0.0/0" ``` 2. Run the following command to update the config map. This will enable the PROXY protocol for the NGINX Plus Ingress Controller: ```shell kubectl apply -f deployments/common/nginx-config.yaml ``` 3. Enable the PROXY Protocol for the target groups linked to the NLB. Follow the steps in the **Enable proxy protocol** section of the [AWS documentation](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/edit-target-group-attributes.html#proxy-protocol) to do this. ## Testing 1. Get the DNS name of NGINX Ingress Controller with the command below. Find the value of the "LoadBalancer Ingress" in the output. ```shell kubectl describe svc nginx-ingress --namespace=nginx-ingress ``` 2. Resolve the DNS name into an IP address using `nslookup`: ```shell nslookup ``` 3. Follow the [instructions](https://github.com/nginx/kubernetes-ingress/tree/main/examples/ingress-resources/complete-example) to deploy the Cafe demo app into the EKS cluster. It will be load balanced by NGINX Ingress Controller. * In Step 1 of deploying the demo app, save the public IP address into the `IC_IP` shell variable. Set `IC_HTTPS_PORT` to 443. * The `kubectl` commands are relative to the `deployment/examples/ingress-resources/complete-example` directory of the [kubernetes-ingress](https://github.com/nginx/kubernetes-ingress) repository. * Run the `curl` command listed in the instructions. It will access the demo app and populate the NGINX Plus Ingress Controller logs. 4. Run the following commands to check if the PROXY Protocol is enabled: 1. Display the pod of NGINX Ingress Controller: ```shell kubectl get pods -n nginx-ingress ``` 2. Display the logs from NGINX Ingress Controller. Replace `` with the name from the previous step. If the logged IP address matches the one you used to access the demo app, then the PROXY Protocol is enabled. ```shell kubectl logs -n nginx-ingress ``` ## Appendix: Disable the PROXY Protocol If you want to disable the PROXY Protocol, perform these steps. 1. Disable the PROXY Protocol for the target groups linked to the NLB. Undo the steps in the **Enable proxy protocol** section of the [AWS documentation](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/edit-target-group-attributes.html#proxy-protocol). 2. Remove the following keys from `deployments/common/nginx-config.yaml` the config map file: ```yaml proxy-protocol: "True" real-ip-header: "proxy_protocol" set-real-ip-from: "0.0.0.0/0" ``` 2. Run the following command to update the config map: ```shell kubectl apply -f deployments/common/nginx-config.yaml ``` 3. In the `deployments/service/loadbalancer-aws-elb.yaml` service file, add the `externalTrafficPolicy` key in the `spec` section. Set it to `Local`, as in this example: ```yaml apiVersion: v1 kind: Service metadata: name: nginx-ingress-nlb namespace: nginx-ingress annotations: service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "tcp" service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*" service.beta.kubernetes.io/aws-load-balancer-type: nlb spec: externalTrafficPolicy: Local type: LoadBalancer ports: - port: 80 targetPort: 80 protocol: TCP name: http - port: 443 targetPort: 443 protocol: TCP name: https selector: app: nginx-ingress ``` 4. Run the following command to update the service: ```shell kubectl apply -f deployments/service/loadbalancer-aws-elb.yaml ``` ### Revision History - Version 2 (February 2025) – Initial version (NGINX Plus Release 20) -------------------------------------------------------------------------------- title: "How NGINX Plus Performs Zone Synchronization" url: https://docs.nginx.com/nginx/admin-guide/high-availability/zone_sync_details/index.md description: Design and limitations of the zone synchronization feature that allows synchronizing data in an nginx cluster. -------------------------------------------------------------------------------- This chapter describes the design and limitations of the zone synchronization feature that allows synchronizing data in an nginx cluster. ## Introduction ### Use Cases Given the specific functionality of the feature, namely: - processing requests on each node separately without consulting other nodes - eventual delivery of changes - expirable records The following use cases are considered: - **Session caching**: the first request creates a session which is shared with other nodes. Subsequent requests use the same session on any cluster node. A missing session on a node is more of a performance penalty rather than a fatal error. - **Resource limiting**: each node assesses local resources and notifies others. In average, the cluster limit is properly imposed. - **Dynamic configuration**: dynamic configuration entries (for example, temporary redirection rules, access rules and limits) are automatically shared within the cluster after being entered on one node. As soon as other nodes receive them, new rules are applied. ## Modules The feature is complex, and its support is split between the nginx `core`, the [ngx_stream_zone_sync_module](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html) module, and the modules that actually use shared zones (for example, [sticky](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#sticky) or [limit_req](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html) modules; here, we will refer to such modules as `functional`). Here, the [zone synchronization module](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html) is responsible for: - all network-related functionality, including: - cluster nodes discovery - managing network connections to cluster nodes - providing transport layer security (using the [ssl module](https://nginx.org/en/docs/http/ngx_http_ssl_module.html)) - pushing data provided by the functional modules to remote nodes - accepting data from remote nodes and dispatching it to the functional modules - tracking changes in zones and processing the zone’s output queue On the server side, the [zone synchronization module](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html) is a regular stream module, so access control, transport layer security, and limiting are supported. The functional modules are responsible for: - serializing zone records so they can be sent over the network along with key and timestamp metadata - deserializing messages and updating the local zone with remote data - maintaining a queue of records that were updated locally and have to be distributed within the cluster The core provides necessary interfaces between the zone synchronization module and the functional modules. ## Synchronization ### Data Model Each functional module represents its data as a queue of records. Records are opaque for the zone synchronization module; it deals only with a serialized representation of the records while sending them to other nodes. The queue contains records that need to be sent to the cluster. Only records created locally on a node may appear in the output queue. The queue is intrusive and may include all records stored in a zone. The zone synchronization module polls the output queues of configured zones at a constant [interval](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_interval) which controls how large the inconsistency window of the cluster is. Also, it allows reducing the amount of data transferred in case of frequent updates of a single record (a typical use case with sessions, when a session is created and later updated multiple times). Synchronized records may be `created` or `updated`. Currently, the `delete` operation is not supported; instead, each record has a configuration-defined lifetime. When a record expires, it is deleted locally, and no updates are sent to the cluster. The functional modules control whether the record lifetime is extended upon access/update or not. The data becomes consistent `eventually`: there is always a time span when an updated value varies between the cluster nodes. The span can be reduced (but not eliminated) by tuning the [zone synchronization interval](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_interval). In terms of the [CAP theorem](https://en.wikipedia.org/wiki/CAP_theorem), the zone sync cluster can be categorized as an AP system. Consistency is ensured by the fact that each node eventually receives all events that have happened in the cluster ordered by timestamp, processes them using a common algorithm, and arrives at the same state as other nodes. Obviously, the algorithm depends on proper temporal synchronization of all cluster nodes, so it is important to setup NTP or similar technology on all nodes. ### Data flow 1. The functional module creates a new record (for example, a new sticky session) 2. The record is inserted into the output queue 3. The functional module may continue its normal operation. As a result, the record can be updated multiple times or even deleted (for example, locally on the node after a direct request) before being shared with other nodes. 4. Periodic timer triggers, and the zone synchronization module inspects the output queue of the functional module. 5. If the queue is not empty, the zone synchronization module asks the functional module to serialize the outgoing records and writes them into its own [buffer](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_buffers). 6. The functional module serializes a record using its own serialization format. Typically, it includes `key`, `timestamp`, and `payload`. 7. The zone synchronization module consumes the output queue until it has no buffer space available or the queue is empty. 8. Finally, the zone synchronization module sends all information it has read to nodes it has established connections with, framing the records into messages that contain zone name and version details. If an error occurs and a record cannot be delivered to a node, the connection to the node is closed to be reestablished. See the explanation of the initial state of a cluster portion below. 9. The remote node receives the message and dispatches it to the appropriate functional module and zone. The functional module now has to refresh its local state using the information from the remote node (`key`, `timestamp`, and `payload`) by: - Inserting the new record at the key if no such key exists - Updating an existing record if the remote timestamp is newer - Ignoring the update if the local timestamp is newer 10. The functional module, that is capable to serialize zone state to disk (i.e. keyval with the “state” enabled) saves record timestamps to ensure proper handling of records lifetime after server restart. ### Topology Topologically, the cluster is a full mesh where each node connects to all the others and any changes are pushed to other nodes. All cluster nodes are equal. There are no special `central` or `main` nodes; each node processes requests without waiting for other nodes. Nodes commit any changes to local memory and continue processing requests. Other nodes will be eventually notified about local changes. A node does not know whether its local values are "best/latest"; it relies on them until updates from other nodes are applied. A node listens for changes from other nodes and applies them as they arrive. ### Initial State When a node is connected to the cluster, it already has a state and it is not known whether any parts of this state were sent to the cluster previously. The ultimate solution is to merge the entire node state with the cluster. This process is called `snapshotting`: when the network connection is established with the new node, the cluster and the node exchange snapshots. The most recent versions of records are selected to form the new joint state. This process runs in parallel with regular operations, so the node does not need to wait until snapshotting ends. The same process happens after a cluster brain split: new nodes appear in the cluster and exchange their current state with peers. If the state is large and there are many nodes in the cluster, synchronization can take some time and introduce additional sporadic system load. When a node goes offline, it temporarily affects other nodes: next data read from the zones is delayed until the write timeout expires, or a connection write error is detected, and the faulty node is disconnected. Then, synchronization continues among the remaining nodes until the connection to the offline node is reestablished. ### Wire Format The module uses a set of [buffers](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_buffers) pre-allocated per node connection to deliver changes. A buffer contains a message header and serialized records. The message header contains: - full message length, including its payload - protocol version (currently `1`) - zone name length - module tag - unique module id - module version - destination zone name The size of the fixed-length header section is `12` bytes. For a message to be successfully dispatched, the signature (module tag and version) must match the destination. Messages with unknown signatures are ignored (for example, adding a new zone to another node can yield such a message). Length of the serialized record depends on the specific functional module; for example, the sticky module generates a `21`-byte fixed-length header and a session ID of up to `32` bytes. If a single serialized record is too large to fit in the buffer, an error is logged and the record is ignored during synchronization. This also suggests that the buffer size should be increased. ### Monitoring The standard functionality of access log allows watching stream sessions with other cluster nodes and log supported variables such as [`$remote_addr`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_remote_addr), [`$status`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_status), [`$upstream_bytes_sent`](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#var_upstream_bytes_sent) or [`$bytes_received`](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#var_bytes_received). The error log registers `NOTICE`-level entries with details of various events: node discovery, node connection, accepted client, and others. Occurring errors are reported at the appropriate level. The zone synchronization module exports several counters via the [API](https://nginx.org/en/docs/http/ngx_http_api_module.html) module: the [`stream/zone_sync`](https://nginx.org/en/docs/http/ngx_http_api_module.html#stream_zone_sync_) endpoint includes the [status](https://nginx.org/en/docs/http/ngx_http_api_module.html#def_nginx_stream_zone_sync) endpoint with per-instance information; [`zones/`](https://nginx.org/en/docs/http/ngx_http_api_module.html#def_nginx_stream_zone_sync_zone) endpoints list statistics per zone. The [`nodes online`](https://nginx.org/en/docs/http/ngx_http_api_module.html#def_nginx_stream_zone_sync_zone) metric displays the count of established connections to other nodes. Usually, it is expected to equal `N` minus `1` where `N` is the number of all cluster nodes. Additionally, the endpoint contains cumulative counters of inbound and outbound messages and bytes. The per-zone statistics include the total number of records in the shared memory zone and the number of records in the output queue. ### Scaling Note that with synchronization your nodes start receiving multiple updates from other nodes which typically requires locking a certain amount of shared memory to insert/update records. With many workers and nodes, this can become a bottleneck. If the number of events is high, consumed network bandwidth and CPU usage increase accordingly. ## FAQ **Q.** What happens when a node is isolated from the cluster? **A.** When a node is isolated from the cluster, it continues to operate as a single node: i.e. it manages sessions locally and continues to respond to clients, while continually trying to connect to others. **Q.** What happens when an isolated node reconnects to the cluster? **A.** When a node reconnects, it performs full resync: it sends all local sessions to cluster, and, in turn, receives data from other nodes.Thus, when all nodes are connected, they will eventually reach a consistent state. **Q.** If multiple nodes sync with the same data, how are conflicts resolved? **A.** The conflict resolution is based on time, thus it is critical to have synchronized clocks across the cluster. If two nodes created a record with same key, the newest record wins. **Q.** Is there a state file for cluster sync data? **A.** No. Cluster sync data is shared across the cluster so there is no need to write it out to disk. The keyval module is capable of saving local node state to disk. **Q.** Can F5 NGINX Plus be configured to reject connections before sync has completed? **A.** No, because there is no such state. Each node just receives stream of updates from others and sends own updates to others. If no new data is arriving to cluster, all nodes will have an empty backlog queue and this can be counted as 'complete sync', but in reality such situation is usually never met: there are always clients. **Q.** How do I deliberately put a node offline? **A.** The documentation describes how to control cluster nodes. **Q.** How can we monitor cluster state data? **A.** There are 2 counters available for each synchronized zone in the NGINX Plus API: Total number of records on node, and a number of records that needs to be sent. If all nodes have approximately the same number of records and almost empty outgoing queue, we may consider the cluster to be healthy. **Q.** How can we monitor cluster health? **A.** There are 2 further metrics exposed by the NGINX Plus API: 1. Number of connected nodes: this is expected to be equal on all nodes and be equal to total number of nodes in cluster minus one. That is, you can monitor cluster connectivity: if some nodes are dead/disconnected, this counters will change 2. Length of send queue: ideally, this is zero (low load) or some constant number (average load). If a backlog is growing and you did not made changes (i.e. adding or removing nodes to cluster), this indicates problems (network connection issues, dead nodes, etc) 3. As usual, monitor error logs: all related events are logged. By default, only errors are logged. It is possible to view more cluster events at INFO level. ## History In NGINX Plus [R15](nginx/releases.md#r15), the ability to synchronize memory zones between instances was introduced; only sticky sessions with [learn](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#sticky_learn_sync) method were supported at the moment. In NGINX Plus [R16](nginx/releases.md#r16), shared zones synchronization was extended to support [keyval](https://nginx.org/en/docs/http/ngx_http_keyval_module.html) and [limit_req](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req) modules. In NGINX Plus [R18](nginx/releases.md#r18), a single zone_sync configuration can be applied between all instances in a cluster using wildcards in the [listen](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#listen) directive. -------------------------------------------------------------------------------- title: "Installing NGINX Plus on Microsoft Azure" url: https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-plus-microsoft-azure/index.md description: Install F5 NGINX Plus as a virtual machine image on Microsoft Azure, to provide sophisticated Layer 7 load balancing for your apps. -------------------------------------------------------------------------------- [F5 NGINX Plus](https://www.f5.com/products/nginx/nginx-plus), the high-performance application delivery platform, load balancer, and web server, is available at the Microsoft Azure Marketplace as a virtual machine (VM) image. The VM image contains the latest version of NGINX Plus, optimized for use with Azure. ## Install the NGINX Plus VM To quickly set up an NGINX Plus environment on Microsoft Azure: 1. Follow the instructions in [Create a Virtual Machine Running Linux](https://learn.microsoft.com/en-us/azure/virtual-machines/linux/quick-create-portal) to sign up on Azure and get more information about Azure itself. 2. Search for “NGINX Plus” in the [Azure Marketplace](https://marketplace.microsoft.com/en-us/search/products?search=nginx%2C%20inc&page=1), open the VM image, and follow the installation instructions. 3. Create an Azure _availability set_ of two or more NGINX Plus virtual machines, which adds redundancy to your NGINX Plus setup by ensuring that at least one virtual machine remains available during a planned or unplanned maintenance event on the Azure platform. For more information, see [Manage the availability of Linux virtual machines](https://docs.microsoft.com/en-us/azure/virtual-machines/linux/manage-availability?) in the Azure documentation. 4. Create _endpoints_ to enable clients outside the NGINX Plus virtual machine’s cloud or virtual network to access it. Sign in to the Azure Management Portal and add endpoints manually to handle the inbound network traffic on port 80 (HTTP) and port 443 (HTTPS). For more information, see [How to set up endpoints on a Linux classic virtual machine in Azure](https://docs.microsoft.com/en-us/azure/virtual-machines/linux/classic/setup-endpoints) in the Azure documentation. 5. As soon as the new virtual machine launches, NGINX Plus starts automatically and serves a default **index.html** page. To verify that NGINX Plus is working properly, use a web browser to access the public DNS name of the new virtual machine and view the page. You can also check the status of the NGINX Plus server by logging into the virtual machine and running this command: ```none /etc/init.d/nginx status ``` ## What If I Need Help? If you encounter any problems with NGINX Plus configuration, documentation is available at [nginx.org](https://nginx.org/en/docs/) and in the [NGINX Plus Admin Guide](/nginx/admin-guide/installing-nginx/). Customers who purchase an NGINX Plus VM image at the Azure Marketplace are eligible for the Azure VM support provided by the NGINX, Inc. engineering team. To activate support, submit the [Azure support activation](https://www.nginx.com/azure-support-activation/) form (you need your Azure subscription ID). When you request support, we’ll ask you to provide the Azure subscription ID that you registered, along with the deployment IDs of your Azure virtual machines in some cases. -------------------------------------------------------------------------------- title: "Monitoring NGINX and NGINX Plus with the New Relic Plug-In" url: https://docs.nginx.com/nginx/admin-guide/monitoring/new-relic-plugin/index.md description: Find what's new in version 2 of the NGINX and F5 NGINX Plus plug‑in for New Relic – more convenience, more metrics, richer visualization. -------------------------------------------------------------------------------- In March, 2013 we released the first version of the [“nginx web server” plug‑in](https://newrelic.com/instant-observability/nginx) for New Relic monitoring of the NGINX Open Source software and F5 NGINX Plus. Since then, we’ve received lots of interest from users – we greatly appreciate it! The plug‑in continues to be one of the most popular New Relic plug‑ins, in spite of the few things that (we believe!) could be improved. If you don’t already have a New Relic account, [sign up](http://newrelic.com/). We selected Ruby as the base language for the original plug‑in, because the New Relic platform itself is written in Ruby, the API was not yet published, and many things were changing quickly. Later, after the launch of the New Relic platform, we continued to work on adding a number of improvements based on users’ feedback and our own vision of how to improve the overall usability of the NGINX monitoring plug‑in, and realized the need for further changes. ## What’s New? Today, we are pleased to announce a major update of the NGINX plug‑in for New Relic, **version 2.0**, which includes the following changes: - The plug‑in is rewritten in Python. You no longer need to install Ruby. - The plug‑in is finally packaged. There are prebuilt packages for RHEL/CentOS‑based and Debian/Ubuntu‑based systems. An included init script enables you to easily set up autostart of the plug‑in. - There are two new sections in the live activity monitoring dashboard, for NGINX Plus customers: - **Servers** – Additional summary counters for virtual servers whose `server` configuration block includes the [status_zone](https://nginx.org/en/docs/http/ngx_http_api_module.html#status_zone) directive - **Cache** – Cumulative stats for all configured caches - Verbose logging is enabled by default. ## Installation Download the [plug‑in and installation instructions](https://docs.newrelic.com/install/nginx/). ## Configuring the Plug‑In The configuration file for the NGINX plug‑in is **/etc/nginx-nr-agent/nginx-nr-agent.ini**. The minimal configuration includes: - Your New Relic license key in the `newrelic_license_key` statement in the `global` section. - At least one `source` section. The name of the section is used in log entries only, and can contain almost any character string you want. Two parameters are required: - `name` – NGINX instance name in the New Relic UI. - `url` – Full URL to the corresponding instance. The plug‑in accepts source data in the format generated when the [stub_status](https://nginx.org/en/docs/http/ngx_http_stub_status_module.html#stub_status) directive is included in the configuration of NGINX or NGINX Plus. It also supports the JSON‑formatted output generated when the [api](https://nginx.org/en/docs/http/ngx_http_api_module.html#api) directive is included in an NGINX Plus configuration. You can include the optional `http_user` and `http_pass` statements to set HTTP basic authentication credentials in cases where the corresponding location is protected by the NGINX [auth_basic](https://nginx.org/en/docs/http/ngx_http_auth_basic_module.html#auth_basic) directive. The default log file is **/var/log/nginx-nr-agent.log**. ## Running the Plug‑In You can manage the plug‑in with an init script. Run the `service` `nginx-nr-agent` command with the `start`, `stop`, or `status` argument to start, stop, or display the status of the plug‑in, respectively. When started from an init script, the plug‑in daemon runs as the `nobody` user. You can start the plug‑in directly by running `/usr/bin/nginx-nr-agent.py`, with the following optional parameters: - `-c`, `--config` – Path to configuration file - `-p`, `--pidfile`> – Path to PID file - `-f`, `--foreground` – Do not detach from terminal (useful for debugging) If everything is working as expected, the plug‑in appears as a tab in the New Relic UI: [![image of NGINX instance in New Relic instances list](/nginx/images/0s-instances.png)](/nginx/images/0s-instances.png) ## Dashboard Examples ### Overview The **Overview** tab displays the most basic metrics for the whole instance: number of active and idle connections, and request rate. [![image of Overview tab in New Relic UI](/nginx/images/1s-overview.png)](/nginx/images/1s-overview.png) ### Connections The **Connections** tab displays various metrics about client connections. [![image of Connections tab in New Relic UI](/nginx/images/2s-connections.png)](/nginx/images/2s-connections.png) ### Requests The **Requests** tab displays the number of requests currently being processed and the overall request rate. [![image of Requests tab in New Relic UI](/nginx/images/3s-requests.png)](/nginx/images/3s-requests.png) ### Upstreams (NGINX Plus only) The **Upstreams** tab displays detailed statistics about all configured upstream groups. [![image of Upstreams tab in New Relic UI](/nginx/images/4s-upstreams.png)](/nginx/images/4s-upstreams.png) ### Servers (NGINX Plus only) The **Servers** tab displays detailed statistics about all configured virtual server zones. [![image of Servers tab in New Relic UI](/nginx/images/5s-servers.png)](/nginx/images/5s-servers.png) ### Cache (NGINX Plus only) The **Cache** tab displays summary statistics for all configured caches. [![image of Caches tab in New Relic UI](/nginx/images/6s-cache.png)](/nginx/images/6s-cache.png) ## What’s Next? We plan to extend the plug‑in’s functionality with additional metrics for both NGINX and NGINX Plus, and your feedback is very important to us. Ideas, thoughts, questions? Let us know by commenting on this post. Thanks for using NGINX and NGINX Plus! To try NGINX Plus, start your [free 30-day trial](https://www.nginx.com/free-trial-request/) today or [contact us](https://www.nginx.com/contact-sales/) for a demo. -------------------------------------------------------------------------------- title: "Open Source Components" url: https://docs.nginx.com/nginx/open-source-components/index.md description: License information for open source components included in the NGINX Plus software. -------------------------------------------------------------------------------- Open source components included in the F5 NGINX Plus (package name is `nginx-plus`) are: - nginx/OSS 1.29.3, distributed under 2-clause BSD license. Homepage: Copyright © 2002-2021 Igor Sysoev Copyright © 2011-2025 NGINX, Inc. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: - Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. - Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - Internal MD5 implementation based on Alexander Peslyak's public domain implementation: This is an OpenSSL-compatible implementation of the RSA Data Security, Inc. MD5 Message-Digest Algorithm (RFC 1321). Homepage: Author: Alexander Peslyak, better known as Solar Designer This software was written by Alexander Peslyak in 2001. No copyright is claimed, and the software is hereby placed in the public domain. In case this attempt to disclaim copyright and place the software in the public domain is deemed null and void, then the software is Copyright © 2001 Alexander Peslyak and it is hereby released to the general public under the following terms: - Redistribution and use in source and binary forms, with or without modification, are permitted. - There's ABSOLUTELY NO WARRANTY, express or implied. (This is a heavily cut-down "BSD license".) - MurmurHash algorithm (version 2), distributed under MIT license. Homepage: Copyright © Austin Appleby Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: - The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. - Components used in status monitoring dashboard v2 only (`dashboard.html` in `nginx-plus` package) and distributed under MIT license: - `@babel-core`, Babel compiler core (7.23.2) Homepage: Copyright © 2014-present Sebastian McKenzie and other contributors - `@babel/plugin-proposal-object-rest-spread`, produces spec-compliant code by using Babel's objectSpread helper (7.22.15). Homepage: - `@babel/plugin-transform-runtime`, makes helpers reference the module babel-runtime to avoid duplication across your compiled output (7.22.15). Homepage: - `@babel/preset-env`, a Babel preset for each environment (7.22.15). Homepage: - `@babel/preset-react`, a Babel preset for all React plugins (7.22.15). Homepage: - `autoprefixer`, a PostCSS plugin to parse CSS and add vendor prefixes to CSS rules (10.4.7) Homepage: Copyright © 2013 Andrey Sitnik - `babel-loader`, allows transpiling JavaScript files using Babel and webpack (9.1.3). Homepage: Copyright © 2014-2019 Luís Couto - `babel-plugin-istanbul`, a babel plugin that adds istanbul instrumentation to ES6 code (6.1.1). Homepage: Copyright © 2016, Istanbul Code Coverage - `core-js`, a modular standard library for JavaScript (3.22.8). Homepage: Copyright © 2014-2022 Denis Pushkarev - `css-loader`, interprets `@import` and `url()` like `import/require()` and will resolve them (6.8.1). Homepage: Copyright © JS Foundation and other contributors - `cssnano`, a modular minifier, built on top of the PostCSS ecosystem (5.1.11). Homepage: Copyright © Ben Briggs - `eslint`, a tool for identifying and reporting on patterns found in ECMAScript/JavaScript code (8.49.0). Homepage: Copyright OpenJS Foundation and other contributors, - `eslint-config-airbnb`, exports some ESLint configurations (19.0.4). Homepage: Copyright © 2012 Airbnb - `eslint-plugin-import`, supports linting of ES2015+ (ES6+) import/export syntax and prevent issues with misspelling of file paths and import names (2.28.1). Homepage: Copyright © 2015 Ben Mosher - `eslint-plugin-jsx-a11y`, static AST checker for accessibility rules on JSX elements (6.5.1). Homepage: Copyright © 2016 Ethan Cohen - `eslint-plugin-react`, react specific linting rules for eslint (7.33.2). Homepage: Copyright © 2014 Yannick Croissant - `eslint-webpack-plugin`, uses eslint to find and fix problems in the JavaScript code (4.0.1). Homepage: Copyright JS Foundation and other contributors - `history`, manage session history with JavaScript (4.10.1). Homepage: Copyright © React Training 2016-2020, Copyright © Remix Software 2020-2021 - `html-inline-css-webpack-plugin`, converts external stylesheet to embedded stylesheet, aka document stylesheet (1.11.1). Homepage: Copyright © 2018 Huang - `html-webpack-plugin`, simplifies creation of HTML files to serve your webpack bundles (5.5.0). Homepage: Copyright © JS Foundation and other contributors - `mini-css-extract-plugin`, extracts CSS into separate files (2.6.0). Homepage: Copyright © JS Foundation and other contributors - `postcss`, a tool for transforming styles with JS plugins (8.4.31). Homepage: Copyright 2013 Andrey Sitnik <> - `postcss-loader`, PostCSS loader for webpack (7.3.3). Homepage: Copyright © JS Foundation and other contributors - `postcss-url`, PostCSS plugin to rebase url(), inline or copy asset (10.1.3). Homepage: Copyright © 2014 Maxime Thirouin - `preact`, fast 3kb React alternative with the same ES6 API (10.7.3). Homepage: Copyright © 2015-present Jason Miller - `react-dev-utils`, utilities used by Create React App (12.0.1). Homepage: Copyright © 2013-present, Facebook, Inc. - `regenerator-runtime`, standalone runtime for Regenerator-compiled generator and async functions (0.14.1). Homepage: Copyright © 2014-present, Facebook, Inc. - `style-loader`, injects CSS into the DOM (3.3.1). Homepage: Copyright © JS Foundation and other contributors - `webpack`, a bundler for javascript and friends (5.88.2). Homepage: Copyright © JS Foundation and other contributors - `webpack-cli`, provides the interface of options webpack uses in its configuration file (4.10.0). Homepage: Copyright © JS Foundation and other contributors - `whatwg-fetch`, a window.fetch JavaScript polyfill (2.0.4). Homepage: Copyright © 2014-2016 GitHub, Inc. The MIT License Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the 'Software'), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: - The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. - Components used in status monitoring dashboard v2 only (`dashboard.html` in `nginx-plus` package) and distributed under 3-clause BSD license and Apache 2.0 license: - `babel-plugin-react-css-modules`, transforms styleName to className using compile time CSS module resolution (3.4.2), distributed under 3-clause BSD license. Homepage: Copyright © 2016, Gajus Kuizinas () All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: - Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. - Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. - Neither the name of the Gajus Kuizinas () nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL ANUARY BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - `npm-font-open-sans`, Open Sans font family - incl. usage of CSS, SCSS, LESS (1.1.0), distributed under Apache 2.0 license. Homepage: Copyright © Steve Matteson Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. Optional add-on and third-party modules provided with NGINX Plus may include additional open-source components. The licenses for these components are included in the installation package for each module. -------------------------------------------------------------------------------- title: "Setting up JWT Authentication" url: https://docs.nginx.com/nginx/admin-guide/security-controls/configuring-jwt-authentication/index.md description: This article explains how to control authentication of your web resources using JWT authentication. -------------------------------------------------------------------------------- ## Introduction With F5 NGINX Plus it is possible to control access to your resources using JWT authentication. JWT is data format for user information in the OpenID Connect standard, which is the standard identity layer on top of the OAuth 2.0 protocol. Deployers of APIs and microservices are also turning to the JWT standard for its simplicity and flexibility. With JWT authentication, a client provides a JSON Web Token, and the token will be validated against a local key file or a remote service. ## Prerequisites - NGINX Plus [Release 10](nginx/releases.md#r10) (R10) for native [JWT support](https://www.nginx.com/blog/nginx-plus-r10-released/#r10-jwt) - NGINX Plus [Release 14](nginx/releases.md#r14) for access to [nested JWT claims and longer signing keys](https://www.nginx.com/blog/nginx-plus-r14-released/#jwt) - NGINX Plus [Release 17](nginx/releases.md#r17) for [getting JSON Web keys from a remote location](https://www.nginx.com/blog/nginx-plus-r17-released/#r17-openid) - NGINX Plus [Release 24](nginx/releases.md#r24) (R24) for support of encrypted tokens (JWE) - NGINX Plus [Release 25](nginx/releases.md#r25) (R25) for support of Nested JWT, multiple sources of JSON Web keys, condition-based JWT authentication - NGINX Plus [Release 26](nginx/releases.md#r26) (R26) for support of [JWT key caching](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_cache) - An identity provider (IdP) or service that creates JWT. For manual JWT generation, see "Issuing a JWT to API Clients" section of the [Authenticating API Clients with JWT and NGINX Plus](https://www.nginx.com/blog/authenticating-api-clients-jwt-nginx-plus/) blog post. NGINX Plus supports the following types of JWT: - JSON Web Signature (JWS) - JWT content is digitally signed. The following algorithms can be used for signing: - HS256, HS384, HS512 - RS256, RS384, RS512 - ES256, ES384, ES512 - EdDSA (Ed25519 and Ed448 signatures) - JSON Web Encryption (JWE) - the contents of JWT is encrypted. The following content encryption algorithms (the "enc" field of JWE header) are supported: - A128CBC-HS256, A192CBC-HS384, A256CBC-HS512 - A128GCM, A192GCM, A256GCM The following key management algorithms (the "alg" field of JWE header) are supported: - A128KW, A192KW, A256KW - A128GCMKW, A192GCMKW, A256GCMKW - dir - direct use of a shared symmetric key as the content encryption key - RSA-OAEP, RSA-OAEP-256, RSA-OAEP-384, RSA-OAEP-512 - Nested JWT - support for JWS enclosed into JWE ## Configuring NGINX Plus to Authenticate API Let's assume that NGINX Plus serves as a gateway (`proxy_pass http://api_server`) to a number of API servers (the [`upstream {}`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#upstream) block), and requests passed to the API servers should be authenticated: ```nginx upstream api_server { server 10.0.0.1; server 10.0.0.2; } server { listen 80; location /products/ { proxy_pass http://api_server; #... } } ``` To implement JWT for authentication: 1. First, it is necessary to create a JWT that will be issued to a client. You can use your identity provider (IdP) or your own service to create JWTs. For testing purposes, you can create your own JWT, see [Authenticating API Clients with JWT and NGINX Plus](https://www.nginx.com/blog/authenticating-api-clients-jwt-nginx-plus/) blog post for details. 2. Configure NGINX Plus to accept JWT: specify the [`auth_jwt`](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt) directive that enables JWT authentication and also defines the authentication area (or "realm", "API" in the example): ```nginx server { listen 80; location /products/ { proxy_pass http://api_server; auth_jwt "API"; #... } } ``` NGINX Plus can also obtain the JWT from a query string parameter. To configure this, include the `token=` parameter to the [`auth_jwt`](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt) directive: ```nginx #... auth_jwt "API" token=$arg_apijwt; #... ``` 3. Specify the type of JWT - `signed` (JWS), `encrypted` (JWE) or `nested` (Nested JWT) - with the [`auth_jwt_type`](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_type) directive. The default value of the directive is `signed`, so for JWS, the directive can be omitted. ```nginx server { listen 80; location /products/ { proxy_pass http://api_server; auth_jwt "API"; auth_jwt_type encrypted; #... } } ``` 4. Specify the path to the [JSON Web Key file](#jwk_create) that will be used to verify JWT signature or decrypt JWT content, depending on what you are using. This can be done with the [`auth_jwt_key_file`](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_file) and/or [`auth_jwt_key_request`](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_request) directives. Specifying both directives at the same time will allow you to specify more than one source for keys. If none of the directives are specified, JWS signature verification will be skipped. In this scenario, the keys will be taken from two files: the `key.jwk` file and the `keys.json` file: ```nginx server { listen 80; location /products/ { proxy_pass http://api_server; auth_jwt "API"; auth_jwt_type encrypted; auth_jwt_key_file conf/key.jwk; auth_jwt_key_file conf/keys.json; } } ``` In this scenario, there are also two sources for the keys, but the private keys will be taken from the local file `private_jwe_keys.jwk`, while the public keys will be taken from the external identity provider service `https://idp.example.com` in a [subrequest](#auth_jwt_key_request): ```nginx server { listen 80; location /products/ { proxy_pass http://api_server; auth_jwt "API"; auth_jwt_type encrypted; auth_jwt_key_file private_jwe_keys.jwk; auth_jwt_key_request /public_jws_keys; } location /public_jws_keys { proxy_pass "https_//idp.example.com/keys"; } } ``` It is recommended to enable JWT key caching to get the optimal performance from the JWT module. For example, you can use the [`auth_jwt_key_cache`](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_cache) directive for the above configuration, and enable the JWT key caching for one hour. Note that if the [`auth_jwt_key_request`](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_request) or [`auth_jwt_key_file`](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_file) are configured dynamically with variables, [`auth_jwt_key_cache`](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_cache) cannot be used. ```nginx server { listen 80; location /products/ { proxy_pass http://api_server; auth_jwt "API"; auth_jwt_type encrypted; auth_jwt_key_file private_jwe_keys.jwk; auth_jwt_key_request /public_jws_keys; auth_jwt_key_cache 1h; } location /public_jws_keys { proxy_pass "https_//idp.example.com/keys"; } } ``` ## How NGINX Plus Validates a JWT A JWT is considered to be valid when the following conditions are met: - The signature can be verified (for JWS) or payload can be decrypted (for JWE) with the key found in the [`auth_jwt_key_file`](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_file) or [`auth_jwt_key_request`](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_request) (matching on the `kid` ("key ID"), if present, and `alg` ("algorithm") header fields). - The JWT is presented inside the validity period, when defined by one or both of the `nbf` ("not before") and `exp` ("expires") claims. ## Creating a JSON Web Key File In order to validate the signature with a key or to decrypt data, a JSON Web Key (`key.jwk`) should be created. The file format is defined by [JSON Web Key specification](https://tools.ietf.org/html/rfc7517): ```json {"keys": [{ "k":"ZmFudGFzdGljand0", "kty":"oct", "kid":"0001" }] } ``` where: - the `k` field is the generated symmetric key (base64url-encoded) basing on a `secret` (`fantasticjwt` in the example). The secret can be generated with the following command: ```shell echo -n fantasticjwt | base64 | tr '+/' '-_' | tr -d '=' ZmFudGFzdGljand0 ``` - the `kty` field defines the key type as a symmetric key (octet sequence) - the `kid` (Key ID) field defines a serial number for this JSON Web Key ## Getting JWKs from Subrequest NGINX Plus can be configured to fetch JSON Web Keys from the remote location - usually an identity provider, especially when using OpenID Connect. The IdP URI where the subrequest will be sent to is configured with the [`auth_jwt_key_request`](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_request) directive: ```nginx http { #... server { listen 80; #... location / { auth_jwt "closed site"; auth_jwt_key_request /_jwks_uri; # Keys will be fetched by subrequest proxy_pass http://my_backend; } } } ``` The URI may refer to an internal location (`_jwks_uri`) so that the JSON Web Key Set can be cached ([`proxy_cache`](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache) and [`proxy_cache_path`](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_path) directives) to avoid validation overhead. Turning on caching is recommended for high-load API gateways even if [JWT key caching](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_cache) is used as it will help to avoid overwhelming a key server with key requests when a JWT key cache expires. ```nginx http { proxy_cache_path /var/cache/nginx/jwk levels=1 keys_zone=jwk:1m max_size=10m; #... server { listen 80; #... location = /_jwks_uri { internal; proxy_method GET; proxy_cache jwk; # Cache responses proxy_cache_valid 200 12h; proxy_pass https://idp.example.com/oauth2/keys; # Obtain keys from here } } } ``` The full example of getting JWKs from a subrequest: ```nginx # proxy_cache_path /var/cache/nginx/jwk levels=1 keys_zone=jwk:1m max_size=10m; server { listen 80; # Use SSL/TLS in production location / { auth_jwt "closed site"; auth_jwt_key_cache 1h; auth_jwt_key_request /_jwks_uri; # Keys will be fetched by subrequest proxy_pass http://my_backend; } location = /_jwks_uri { internal; proxy_method GET; proxy_cache jwk; # Cache responses proxy_cache_valid 200 12h; proxy_pass https://idp.example.com/oauth2/keys; # Obtain keys from here } } ``` ## Arbitrary JWT Claims Validation During JWT verification, NGINX Plus automatically validates only `nbf` ("not before") and `exp` ("expires") claims. However, in some cases you need to set more conditions for a successful JWT validation, in particular when dealing with application-specific or protocol level claims. For example, OpenID Connect Core requires validation of `iss` ("issuer"), `aud` ("audience"), `sub` ("subject") claims for `ID` token. Additional conditions for JWT validation can be set as variables with the [`map`](https://nginx.org/en/docs/http/ngx_http_map_module.html) module and then evaluated with the [`auth_jwt_require`](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_require) directive. In this scenario, we are verifying that: - the recipient of the token (audience) is our APIs (map rule 1) - the token was issued by a trusted identity provider (map rule 2) - scopes in APIs called on behalf of administrators (map rule 3) The values of three resulting variables are evaluated in the [`auth_jwt_require`](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_require) directive, and if the value of each variable is `1`, the JWT will be accepted: ```nginx upstream api_server { server 10.0.0.1; server 10.0.0.2; } map $jwt_claim_aud $valid_app_id { #map rule 1: "~api\d.example.com" 1; #token issued only for target apps } map $jwt_claim_iss $valid_issuer { #map rule 2: "https://idp.example.com/sts" 1; #token issued by trusted CA } map $jwt_claim_scope $valid_scope { #map rule 3: "access_as_admin" 1; #access as admin only } server { listen 80; location /products/ { auth_jwt "API"; auth_jwt_key_file conf/api_secret.jwk; auth_jwt_require $valid_app_id $valid_issuer $valid_scope; proxy_pass http://api_server; } } ``` In some cases the [`auth_jwt_require`](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_require) directive can be specified multiple times, for example, for the purpose of authentication and then for authorization. In case of an error, the `401` code will be displayed. Assigning the custom error code `403` to another `auth_jwt_require` directive makes ti possible to differentiate authentication and authorization usecases and handle corresponding failures appropriately: ```nginx location /products/ { auth_jwt "API"; auth_jwt_key_file conf/api_secret.jwk; auth_jwt_require $valid_app_id $valid_issuer $valid_scope; auth_jwt_require $valid_scope error=403; proxy_pass http://api_server; } ``` ## Nested JWT Extraction A Nested JWT is a JWS token enclosed into JWE. In a Nested JWT, the sensitive information from JWS is protected with extra encryption of JWE. Using Nested JWT may be preferable over JWE because: - in case of JWE, the target application/service needs to decrypt the token first, then verify the signature. Decrypt operation on the application side may be time and resource consuming. - in case of Nested JWT, as NGINX Plus resides in the same trusted network with the target application, there is no need for token encryption between NGINX Plus and the application. NGINX Plus decrypts the JWE, checks the enclosed JWS, and sends the Bearer Token to the application. This will offload JWE decryption from the application to NGINX Plus. - if your application doesn't support JWE, using Nested JWT enables full protection for JWS. To enable Nested tokens: 1. Specify the `nested` type of JWT with the [`auth_jwt_type`](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_type) directive. ```nginx auth_jwt_type nested; ``` 2. Pass the decrypted payload (the [`$jwt_payload`](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#var_jwt_payload) variable) to the application as the Bearer token value in the `Authorization` header: ```nginx proxy_set_header Authorization "Bearer $jwt_payload"; ``` This example sums up the previous steps into one configuration: ```nginx upstream api_server { server 10.0.0.1; server 10.0.0.2; } http { server { listen 80; auth_jwt "API"; auth_jwt_type nested; auth_jwt_key_file conf/api_secret.jwk; proxy_pass http://api_server; proxy_set_header Authorization "Bearer $jwt_payload"; } } ``` ## See Also - [Authenticating API Clients with JWT and NGINX Plus](https://www.nginx.com/blog/authenticating-api-clients-jwt-nginx-plus/) -------------------------------------------------------------------------------- title: "Single Sign-On with Keycloak" url: https://docs.nginx.com/nginx/deployment-guides/single-sign-on/keycloak/index.md description: Enable OpenID Connect-based single sign-on (SSO) for applications proxied by NGINX Plus, using Keycloak as the identity provider (IdP). -------------------------------------------------------------------------------- This guide explains how to enable single sign-on (SSO) for applications being proxied by F5 NGINX Plus. The solution uses OpenID Connect as the authentication mechanism, with [Keycloak](https://www.keycloak.org/) as the Identity Provider (IdP), and NGINX Plus as the Relying Party, or OIDC client application that verifies user identity. **Note:** This guide applies to [NGINX Plus Release 36](nginx/releases.md#r36) and later. In earlier versions, NGINX Plus relied on an [njs-based solution](#legacy-njs-guide), which required NGINX JavaScript files, key-value stores, and advanced OpenID Connect logic. Starting from NGINX Plus version R34, the new [OpenID Connect module](https://nginx.org/en/docs/http/ngx_http_oidc_module.html) simplifies this process to just a few directives. ## Prerequisites - An NGINX Plus [subscription](https://www.f5.com/products/nginx/nginx-plus) and NGINX Plus [Release 36](nginx/releases.md#r36) or later. For installation instructions, see [Installing NGINX Plus](https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-plus/). - A domain name pointing to your NGINX Plus instance, for example, `demo.example.com`. ## Configure Keycloak {#keycloak-setup} 1. Log in to your Keycloak admin console, for example, `https:///admin/master/console/`. 2. In the left navigation, go to **Clients**, then 3. Select **Create client** and provide the following details: - Set **Client type** to **OpenID Connect**. - Enter a **Client ID**, for example, `nginx-demo-app`. You will need it later when configuring NGINX Plus. - Select **Next**. 4. In the **Capability Config** section: - Set **Client Authentication** to **On**. This sets the client type to **confidential**. - Select **Next**. 5. In the **Login Settings** section: Add a **Redirect URI**, for example: ```text https://demo.example.com/oidc_callback ``` Add a **Post Logout Redirect URI** to support RP-initiated logout, for example: ```text https://demo.example.com/post_logout/ ``` - Select **Save**. 6. In the **Credentials** tab, make note of the **Client Secret**. You will need it later when configuring NGINX Plus. ### Configure Logout and Front-Channel Single Logout (optional) {#keycloak-frontchannel-logout} Front-channel logout allows Keycloak to notify NGINX Plus when a user signs out of the realm or another application that participates in Single Logout (SLO). Keycloak sends a front-channel HTTP request (typically loaded in a hidden iframe) to a logout URL that you configure for the client. 1. In the Keycloak admin console, go to **Clients** and select your `nginx-demo-app` client. 2. On the **Settings** tab, scroll to the **Logout settings** section. 3. Configure front-channel logout for this client: - Set **Front channel logout** to **On**. - In **Front-channel Logout URL**, enter the NGINX Plus front-channel logout endpoint, for example: ```text https://demo.example.com/front_logout ``` - Enable **Front-channel logout session required**. With this option enabled, Keycloak includes both the session identifier (`sid`) and issuer (`iss`) parameters in the front-channel logout request that it sends to NGINX Plus, as defined by the OpenID Connect Front-Channel Logout specification. NGINX Plus uses these parameters to identify and clear the corresponding user session. ### Assign Users or Groups This step is optional, and is necessary if you need to restrict or organize user permissions. 1. In the **Roles** tab, add a **Client Role**, for example, `nginx-keycloak-role`. 2. Under **Users**, create a new user or select a user. 3. In **Role Mappings**, assign a role to the user within the `nginx-demo-app` client. ### Get the OpenID Connect Discovery URL Check the OpenID Connect Discovery URL. By default, Keycloak publishes the `.well-known/openid-configuration` document at the following address: `https:///realms//.well-known/openid-configuration`. 1. Run the following `curl` command in a terminal: ```shell curl https:///realms//.well-known/openid-configuration | jq ``` Where: - the `` is your Keycloak server address - the `` is your Keycloak realm name - the `/.well-known/openid-configuration` is the default address for Keycloak for document location - the `jq` command (optional) is used to format the JSON output for easier reading and requires the [jq](https://jqlang.github.io/jq/) JSON processor to be installed. The configuration metadata is returned in the JSON format: ```json { ... "issuer": "https:///realms/", "authorization_endpoint": "https:///realms//protocol/openid-connect/auth", "token_endpoint": "https:///realms//protocol/openid-connect/token", "jwks_uri": "https:///realms//protocol/openid-connect/certs", "userinfo_endpoint": "https:///realms//protocol/openid-connect/userinfo", "end_session_endpoint": "https:///realms//protocol/openid-connect/logout", ... } ``` 2. Copy the **issuer** value, you will need it later when configuring NGINX Plus. Typically, the OpenID Connect Issuer for Keycloak is `https:///realms/`. **Note:** You will need the values of **Client ID**, **Client Secret**, and **Issuer** in the next steps. ## Set up NGINX Plus {#nginx-plus-setup} With Keycloak configured, you can enable OIDC on NGINX Plus. NGINX Plus serves as the Relying Party (RP) application — a client service that verifies user identity. 1. Ensure that you are using the latest version of NGINX Plus by running the `nginx -v` command in a terminal: ```shell nginx -v ``` The output should match NGINX Plus Release 36 or later: ```none nginx version: nginx/1.29.3 (nginx-plus-r36) ``` 2. Ensure that you have the values of the **Client ID**, **Client Secret**, and **Issuer** obtained during [Keycloak Configuration](#keycloak-setup). 3. In your preferred text editor, open the NGINX configuration file (`/etc/nginx/nginx.conf` for Linux or `/usr/local/etc/nginx/nginx.conf` for FreeBSD). 4. In the [`http {}`](https://nginx.org/en/docs/http/ngx_http_core_module.html#http) context, make sure your public DNS resolver is specified with the [`resolver`](https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver) directive: By default, NGINX Plus re‑resolves DNS records at the frequency specified by time‑to‑live (TTL) in the record, but you can override the TTL value with the `valid` parameter: ```nginx http { resolver 10.0.0.1 ipv4=on valid=300s; # ... } ``` 5. In the [`http {}`](https://nginx.org/en/docs/http/ngx_http_core_module.html#http) context, define the Keycloak provider named `keycloak` by specifying the [`oidc_provider {}`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#oidc_provider) context: ```nginx http { resolver 10.0.0.1 ipv4=on valid=300s; oidc_provider keycloak { # ... } # ... } ``` 6. In the [`oidc_provider {}`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#oidc_provider) context, specify: - your actual Keycloak **Client ID** obtained in [Keycloak Configuration](#keycloak-setup) with the [`client_id`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_id) directive - (if not using PKCE) your **Client Secret** obtained in [Keycloak Configuration](#keycloak-setup) with the [`client_secret`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_secret) directive - the **Issuer** URL obtained in [Keycloak Configuration](#keycloak-setup) with the [`issuer`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_secret) directive The `issuer` is typically your Keycloak OIDC URL: `https:///realms/`. By default, NGINX Plus creates the metadata URL by appending the `/.well-known/openid-configuration` part to the Issuer URL. If your metadata URL is different, you can explicitly specify it with the [`config_url`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#config_url) directive. - The **logout_uri** is URI that a user visits to start an RP‑initiated logout flow. - The **post_logout_uri** is absolute HTTPS URL where Keycloak should redirect the user after a successful logout. This value **must also be configured** in the Keycloak client's Post Logout Redirect URIs. - If the **logout_token_hint** directive set to `on`, NGINX Plus sends the user's ID token as a *hint* to Keycloak. This directive is **optional**, however, if it is omitted the Keycloak may display an extra confirmation page asking the user to approve the logout request. - The **frontchannel_logout_uri** directive defines the URI on NGINX Plus that receives OpenID Connect front-channel logout requests from Keycloak. This path must be an HTTPS endpoint and must match the **Front-channel Logout URL** configured for the client in Keycloak. When a front-channel logout request is sent (typically in a hidden iframe), Keycloak includes the session identifier and issuer (the `sid` and `iss` parameters, when **Front-channel logout session required** is enabled); the OIDC module uses these values to locate and clear the corresponding local session. - If the **userinfo** directive is set to `on`, NGINX Plus will fetch `/protocol/openid-connect/userinfo` from the Keycloak and append the claims from userinfo to the `$oidc_claims_` variables. - PKCE (Proof Key for Code Exchange) is automatically enabled when Keycloak's OpenID Connect discovery document advertises the `S256` code challenge method in the `code_challenge_methods_supported` field. You can override this behavior with the [`pkce`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#pkce) directive: set `pkce off;` to disable PKCE even when `S256` is advertised, or `pkce on;` to force PKCE even if the IdP metadata does not list `S256`. - The module automatically selects the client authentication method for the token endpoint based on the provider metadata `token_endpoint_auth_methods_supported`. When only `client_secret_post` is advertised, NGINX Plus uses the `client_secret_post` method and sends the client credentials in the POST body. When both `client_secret_basic` and `client_secret_post` are present, the module prefers HTTP Basic (`client_secret_basic`), which remains the default for Keycloak. - **Note:** All interaction with the IdP is secured exclusively over SSL/TLS, so NGINX must trust the certificate presented by the IdP. By default, this trust is validated against your system’s CA bundle (the default CA store for your Linux or FreeBSD distribution). If the IdP’s certificate is not included in the system CA bundle, you can explicitly specify a trusted certificate or chain with the [`ssl_trusted_certificate`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#ssl_trusted_certificate) directive so that NGINX can validate and trust the IdP’s certificate. ```nginx http { resolver 10.0.0.1 ipv4=on valid=300s; oidc_provider keycloak { issuer https:///realms/; client_id ; client_secret ; logout_uri /logout; post_logout_uri https://demo.example.com/post_logout/; logout_token_hint on; frontchannel_logout_uri /front_logout; userinfo on; # Optional: PKCE configuration. By default, PKCE is automatically # enabled when the IdP advertises the S256 code challenge method. # pkce on; } # ... } ``` 7. Make sure you have configured a [server](https://nginx.org/en/docs/http/ngx_http_core_module.html#server) that corresponds to `demo.example.com`, and there is a [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) that [points](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass) to your application (see [Step 10](#oidc_app)) at `http://127.0.0.1:8080` that is going to be OIDC-protected: ```nginx http { # ... server { listen 443 ssl; server_name demo.example.com; ssl_certificate /etc/ssl/certs/fullchain.pem; ssl_certificate_key /etc/ssl/private/key.pem; location / { # ... proxy_pass http://127.0.0.1:8080; } } # ... } ``` 8. Protect this [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) with Keycloak OIDC by specifying the [`auth_oidc`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#auth_oidc) directive that will point to the `keycloak` configuration specified in the [`oidc_provider {}`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#oidc_provider) context in [Step 5](#keycloak-setup-oidc-provider): ```nginx # ... location / { auth_oidc keycloak; # ... proxy_pass http://127.0.0.1:8080; } # ... ``` 9. Pass the OIDC claims as headers to the application ([Step 10](#oidc_app)) with the [`proxy_set_header`](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_header) directive. These claims are extracted from the ID token returned by Keycloak: - [`$oidc_claim_sub`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) - a unique `Subject` identifier assigned for each user by Keycloak - [`$oidc_claim_email`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) the e-mail address of the user - [`$oidc_claim_name`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) - the full name of the user - any other OIDC claim using the [`$oidc_claim_ `](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) variable ```nginx # ... location / { auth_oidc keycloak; proxy_set_header sub $oidc_claim_sub; proxy_set_header email $oidc_claim_email; proxy_set_header name $oidc_claim_name; proxy_pass http://127.0.0.1:8080; } # ... ``` 10. Provide endpoint for completing logout: ```nginx # ... location /post_logout/ { return 200 "You have been logged out.\n"; default_type text/plain; } # ... ``` 11. Create a simple test application referenced by the `proxy_pass` directive which returns the authenticated user's full name and email upon successful authentication: ```nginx # ... server { listen 8080; location / { return 200 "Hello, $http_name!\nEmail: $http_email\nKeycloak sub: $http_sub\n"; default_type text/plain; } } ``` 12. Save the NGINX configuration file and reload the configuration: ```nginx nginx -s reload ``` ### Complete Example This configuration example summarizes the steps outlined above. It includes only essential settings such as specifying the DNS resolver, defining the OIDC provider, configuring SSL, and proxying requests to an internal server. ```nginx http { # Use a public DNS resolver for Issuer discovery, etc. resolver 10.0.0.1 ipv4=on valid=300s; oidc_provider keycloak { # The 'issuer' typically matches your Keycloak realm's base URL: # For example: https:///realms/ issuer https:///realms/master; # Replace with your actual Keycloak client_id and secret client_id ; client_secret ; # RP‑initiated logout logout_uri /logout; post_logout_uri https://demo.example.com/post_logout/; logout_token_hint on; # Front-channel logout (OP‑initiated single sign-out) frontchannel_logout_uri /front_logout; # Fetch userinfo claims userinfo on; # Optional: PKCE configuration # pkce on; # If the .well-known endpoint can't be derived automatically, # specify config_url: # config_url https:///realms/master/.well-known/openid-configuration; } server { listen 443 ssl; server_name demo.example.com; ssl_certificate /etc/ssl/certs/fullchain.pem; ssl_certificate_key /etc/ssl/private/key.pem; location / { # Protect this location with Keycloak OIDC auth_oidc keycloak; # Forward OIDC claims as headers if desired proxy_set_header sub $oidc_claim_sub; proxy_set_header email $oidc_claim_email; proxy_set_header name $oidc_claim_name; proxy_pass http://127.0.0.1:8080; } location /post_logout/ { return 200 "You have been logged out.\n"; default_type text/plain; } } server { # Simple test upstream server listen 8080; location / { return 200 "Hello, $http_name!\nEmail: $http_email\nKeycloak sub: $http_sub\n"; default_type text/plain; } } } ``` ### Testing 1. Open https://demo.example.com/ in a browser. You should be redirected to Keycloak's login page for your realm. 2. Enter valid Keycloak credentials for a user assigned to the `nginx-demo-app` client. Upon successful sign-in, Keycloak redirects you back to NGINX Plus, and you will see the proxied application content (for example, "Hello, Jane Doe!"). 3. Navigate to `https://demo.example.com/logout`. NGINX Plus initiates an RP‑initiated logout; Keycloak ends the session and redirects back to `https://demo.example.com/post_logout/`. 4. Refresh `https://demo.example.com/` again. You should be redirected to Keycloak for a fresh sign‑in, proving the session has been terminated. ## Legacy njs-based Keycloak Solution {#legacy-njs-guide} If you are running NGINX Plus R33 and earlier or if you still need the njs-based solution, refer to the [Legacy njs-based Keycloak Guide](nginx/deployment-guides/single-sign-on/oidc-njs/keycloak.md) for details. The solution uses the [`nginx-openid-connect`](https://github.com/nginxinc/nginx-openid-connect) GitHub repository and NGINX JavaScript files. ## See Also - [NGINX Plus Native OIDC Module Reference documentation](https://nginx.org/en/docs/http/ngx_http_oidc_module.html) - [Release Notes for NGINX Plus R36](nginx/releases.md#r36) ## Revision History - Version 3 (November 2025) – Updated for NGINX Plus R36; added front-channel logout support (`frontchannel_logout_uri`), PKCE configuration (`pkce` directive), and the `client_secret_post` token endpoint authentication method. - Version 2 (August 2025) – Updated for NGINX Plus R35; added RP‑initiated logout (`logout_uri`, `post_logout_uri`, `logout_token_hint`) and `userinfo` support. - Version 1 (March 2025) – Initial version (NGINX Plus Release 34). -------------------------------------------------------------------------------- title: "UDP Health Checks" url: https://docs.nginx.com/nginx/admin-guide/load-balancer/udp-health-check/index.md description: This chapter describes how to configure different types of health checks for UDP servers in a load‑balanced upstream server group. -------------------------------------------------------------------------------- NGINX Plus can continually test your upstream servers that handle UDP network traffic (DNS, RADIUS, syslog), avoid the servers that have failed, and gracefully add the recovered servers into the load‑balanced group. ## Prerequisites {#prereq} - You have [configured an upstream group of servers](nginx/admin-guide/load-balancer/tcp-udp-load-balancer.md) that handles UDP network traffic in the [`stream {}`](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#stream) context, for example: ```nginx stream { #... upstream dns_upstream { server 192.168.136.130:53; server 192.168.136.131:53; server 192.168.136.132:53; } #... } ``` - You have configured a server that passes UDP datagrams to the upstream server group: ```nginx stream { #... server { listen 53 udp; proxy_pass dns_upstream; proxy_timeout 1s; proxy_responses 1; error_log logs/dns.log; } #... } ``` See [TCP and UDP Load Balancing](nginx/admin-guide/load-balancer/tcp-udp-load-balancer.md) for details. ## Passive UDP health checks {#hc_passive} NGINX Open Source or F5 NGINX Plus can mark the server as unavailable and stop sending UDP datagrams to it for some time if the server replies with an error or times out. The number of consecutive failed connection attempts within a certain time period is set with the [`max_fails`](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#max_fails) parameter for an [`upstream server`](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#server) (default value is `1`). The time period is set with the [`fail_timeout`](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#fail_timeout) parameter (default value is `10` seconds). The parameter also sets the amount of time that NGINX considers the server unavailable after marking it so. So if a connection attempt times out or fails at least once in a 10‑second period, NGINX marks the server as unavailable for 10 seconds. The example shows how to set these parameters to 2 failures within 60 seconds: ```nginx upstream dns_upstream { server 192.168.136.130:53 fail_timeout=60s; server 192.168.136.131:53 fail_timeout=60s; } ``` ## Active UDP health checks {#hc_active} Active Health Checks allow testing a wider range of failure types and are available only for NGINX Plus. For example, instead of waiting for an actual TCP request from a DNS client to fail before marking the DNS server as down (as in passive health checks), NGINX Plus will send special health check requests to each upstream server and check for a response that satisfies certain conditions. If a connection to the server cannot be established, the health check fails, and the server is considered unhealthy. NGINX Plus does not proxy client connections to unhealthy servers. If more than one health check is defined, the failure of any check is enough to consider the corresponding upstream server unhealthy. To enable active health checks: 1. In the upstream group, specify a _shared memory zone_ with the [`zone`](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#zone) directive – a special area where the NGINX Plus worker processes share state information about counters and connections. In the `zone` directive, specify the zone _name_ (`dns_zone` in the example) and the zone _size_ (`64k` in the example): ```nginx stream { #... upstream dns_upstream { zone dns_zone 64k; server 192.168.136.130:53; server 192.168.136.131:53; server 192.168.136.132:53; } #... } ``` 2. In the `server` block that forwards traffic to the upstream group (via [`proxy_pass`](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_pass)), specify the [`udp`](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#health_check_udp) parameter to the `health_check` directive: ```nginx stream { #... server { listen 53 udp; proxy_pass dns_upstream; health_check udp; } #... } ``` A basic UDP health check assumes that NGINX Plus sends the “nginx health check” string to an upstream server and expects the absence of ICMP “Destination Unreachable” message in response. You can configure your own health check tests in the `match {}` block. See [The “match {}” Configuration Block](#hc_active_match) for details. ### Fine-Tuning UDP Health Checks {#hc_active_finetune} You can fine‑tune the health check by specifying the following parameters to the [`health_check`](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#health_check) directive: - `interval`– How often (in seconds) NGINX Plus sends health check requests (default is `5` seconds) - `passes`– Number of consecutive health checks the server must respond to to be considered healthy (default is `1`) - `fails`– Number of consecutive health checks the server must fail to respond to to be considered unhealthy (default is `1`) ```nginx server { listen 53 udp; proxy_pass dns_upstream; health_check interval=20 passes=2 fails=2 udp; } ``` In the example, the time between UDP health checks is increased to 20 seconds, the server is considered unhealthy after 2 consecutive failed health checks, and the server needs to pass 2 consecutive checks to be considered healthy again. ### The “match {}” configuration block {#hc_active_match} A basic UDP health check assumes that NGINX Plus sends the “nginx health check” string to an upstream server and expects the absence of ICMP “Destination Unreachable” message in response. You can configure your own health check tests that will verify server responses. These tests are defined within the [`match {}`](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#match) configuration block. 1. In the top‑level `stream {}` context, specify the [`match {}`](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#match) block and set its name, for example, `udp_test`: ```nginx stream { #... match udp_test { #... } } ``` 2. Refer to the block from the `health_check` directive by including the [`match`](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#health_check_match) parameter to specify the name of the `match {}` block: ```nginx stream { #... server { listen 53 udp; proxy_pass dns_upstream; health_check match=udp_test udp; } #... } ``` 3. In the `match {}` block, specify conditions or tests under which a health check succeeds. This is done with `send` and `expect` parameters: - [`send`](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#match_send)– The text string or hexadecimal literals (“/x” followed by two hex digits) to send to the server - [`expect`](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#match_expect)– Literal string or regular expression that the data returned by the server needs to match These parameters can be used in different combinations, but no more than one `send` and one `expect` parameter can be specified at a time. ## Usage scenarios ### NTP health checks {#example_ntp} To fine‑tune health checks for NTP, you should specify both `send` and `expect` parameters with the following text strings: ```nginx match ntp { send \xe3\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00; expect ~* \x24; } ``` #### Complete NTP health check configuration example ```nginx stream { upstream ntp_upstream { zone ntp_zone 64k; server 192.168.136.130:53; server 192.168.136.131:53; server 192.168.136.132:53; } server { listen 53 udp; proxy_pass ntp_upstream; health_check match=ntp udp; proxy_timeout 1s; proxy_responses 1; error_log logs/ntp.log; } match ntp { send \xe3\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00; expect ~* \x24; } } ``` ### DNS health checks {#example_dns} [DNS health checks](#hc_active) can be enhanced to perform real DNS lookup queries. You can craft a valid DNS query packet, send it to the upstream server, and inspect the response to determine health. The process includes three steps: - [Creating a CNAME test record](#create-a-cname-record) on your DNS server. - [Crafting a raw DNS query packet](#construct-a-raw-dns-query-packet) to be sent by NGINX Plus. - [Validating the expected response](#configure-the-match-block-for-dns-lookup) using the `match` block, where the `send` parameter represents a raw DNS query packet, and `expect` represents the value of the CNAME record. #### Create a CNAME record First, create a CNAME record on your DNS server for a health check that points to the target website. For example, if you are using BIND self-hosted DNS solution on a Linux server: - Open the zone file in a text editor: ```shell sudo nano /etc/bind/zones/db.example.com ``` - Add a CNAME record, making `healthcheck.example.com` resolve to `healthy.svcs.example.com`: ```none healthcheck IN CNAME healthy.svcs.example.com. ``` - Save the file and reload the DNS service: ```shell sudo systemctl reload bind9 ``` Once the CNAME record is live and resolvable, you can craft a DNS query packet that represents a DNS lookup and can be used in the `send` directive. #### Construct a raw DNS query packet The `send` parameter of the `match` block allows you to send raw UDP packets for health checks. To query your CNAME record, you need to construct a valid DNS query packet according to the [DNS protocol message format](https://datatracker.ietf.org/doc/html/rfc1035#section-4.1), including a header and question section. The DNS Query packet can be created using DNS packet builders, such as Python Scapy or dnslib, or packet analyzers, such as tcpdump or Wireshark. If using a packet analyzer, extract only the DNS layer, removing Ethernet, IP, and UDP-related headers. This is the raw DNS query of `healthcheck.example.com`, represented as one line in Hex with `\x` prefixes: ```none \x00\x01\x01\x00\x00\x01\x00\x00\x00\x00\x00\x00\x0b\x68\x65\x61\x6c\x74\x68\x63\x68\x65\x63\x6b\x07\x65\x78\x61\x6d\x70\x6c\x65\x03\x63\x6f\x6d\x00\x00\x01\x00\x01 ``` where: | HEX | Description | |------------------|------------------------| | \x00\x01 | Transaction ID: 0x0001 | | \x01\x00 | Flags: Standard query, recursion desired | | \x00\x01 | Questions: 1 | | \x00\x00 | Answer RRs: 0 | | \x00\x00 | Authority RRs: 0 | | \x00\x00 | Additional RRs: 0 | | \x0b\x68\x65\x61\x6c\x74\x68\x63\x68\x65\x63\x6b | "healthcheck" | | \x07\x65\x78\x61\x6d\x70\x6c\x65 | "example" | | \x03\x63\x6f\x6d | "com" | | \x00 | end of name | | \x00\x01 | Type: A | | \x00\x01 | Class: IN | #### Configure the match block for DNS lookup Finally, specify the `match` block in the NGINX configuration file to pair the raw query with an expected response. The `send` directive should contain the DNS query packet, while `expect` directive should contain a matching DNS record in the DNS server's response: ```nginx match dns { send \x00\x01\x01\x00\x00\x01\x00\x00\x00\x00\x00\x00\x0b\x68\x65\x61\x6c\x74\x68\x63\x68\x65\x63\x6b\x07\x65\x78\x61\x6d\x70\x6c\x65\x03\x63\x6f\x6d\x00\x00\x01\x00\x01; expect ~* "healthy.svcs.example.com"; } ``` #### Complete DNS health check configuration example ```nginx stream { upstream dns_upstream { zone dns_zone 64k; server 192.168.136.130:53; server 192.168.136.131:53; server 192.168.136.132:53; } server { listen 53 udp; proxy_pass dns_upstream; health_check match=dns udp; proxy_timeout 1s; proxy_responses 1; error_log logs/dns.log; } match dns { # make sure appropriate CNAME record exists send \x00\x01\x01\x00\x00\x01\x00\x00\x00\x00\x00\x00\x0b\x68\x65\x61\x6c\x74\x68\x63\x68\x65\x63\x6b\x07\x65\x78\x61\x6d\x70\x6c\x65\x03\x63\x6f\x6d\x00\x00\x01\x00\x01; expect ~* "healthy.svcs.example.com"; } } ``` ## See also - [Load Balancing DNS Traffic with NGINX and NGINX Plus](https://www.f5.com/company/blog/nginx/load-balancing-dns-traffic-nginx-plus) - [TCP/UDP Load Balancing with NGINX: Overview, Tips, and Tricks](https://blog.nginx.org/blog/tcp-load-balancing-udp-load-balancing-nginx-tips-tricks#activeHealthCheck) -------------------------------------------------------------------------------- title: "Using NGINX and NGINX Plus as an Application Gateway with uWSGI and Django" url: https://docs.nginx.com/nginx/admin-guide/web-server/app-gateway-uwsgi-django/index.md description: Configure NGINX and F5 NGINX Plus as an application gateway for uWSGI and Django -------------------------------------------------------------------------------- This article explains how to use NGINX or F5 NGINX Plus as an application gateway with uWSGI and Django. ## Introduction NGINX is a high‑performance, scalable, secure, and reliable web server and a reverse proxy. NGINX uses web acceleration techniques to manage HTTP connections and traffic. Features like [load balancing](../load-balancer/http-load-balancer.md), [SSL termination](/nginx/admin-guide/security-controls/terminating-ssl-http.md), connection and request [policing](/nginx/admin-guide/security-controls/controlling-access-proxied-http.md), static [content offload](/nginx/admin-guide/web-server/serving-static-content.md), and [content caching](/nginx/admin-guide/content-cache/content-caching.md) help users build reliable websites. NGINX acts as a secure application gateway, passing traffic from users to applications. In this regard, not only can NGINX proxy HTTP and HTTPS traffic to an HTTP‑enabled application container, it can also connect to most of the popular application servers and web frameworks via optimized app‑gateway interfaces implemented in modules like [FastCGI](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html), [Memcached](https://nginx.org/en/docs/http/ngx_http_memcached_module.html), [scgi](https://nginx.org/en/docs/http/ngx_http_scgi_module.html), and [uwsgi](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html). Many application containers have embedded external HTTP interfaces with some routing capabilities. NGINX offers an all‑in‑one solution. It handles HTTP connection management, load balancing, content caching, and traffic security. The application backend sits behind NGINX for better scalability and performance. You can group app instances behind NGINX to ensure high availability. ## About uWSGI and Django A few words about "specialized interfaces." As useful as it is, HTTP has never been designed for modern, lightweight application‑deployment scenarios. Over time, standardized interfaces have evolved for use with various application frameworks and application containers. One of these interfaces is the Web Server Gateway Interface ([WSGI](http://wsgi.readthedocs.org/en/latest/)), an interface between a web server/proxy and Python‑based applications. One common application server is the [uWSGI application server container](https://github.com/unbit/uwsgi). It offers [uwsgi](http://uwsgi-docs.readthedocs.org/en/latest/Protocol.html) - its own implementation of the WSGI protocol. Other than that, the uWSGI application server supports HTTP, FastCGI, and SCGI – with the uwsgi protocol recommended as the fastest way to talk to applications. ## Configure NGINX and NGINX Plus for Use with uWSGI and Django This document provides an example of how to configure NGINX and NGINX Plus for use with a [uWSGI](http://uwsgi-docs.readthedocs.org/en/latest/) server and a Python development environment. NGINX 0.8.40 and later (and all releases of NGINX Plus) includes native support for passing traffic from users to Python applications via the uwsgi protocol. If you downloaded [NGINX Open Source binaries or source](https://nginx.org/en/download.html) from our official repositories, or [NGINX Plus from the customer portal](https://account.f5.com/myf5), no action is needed to enable support for the uwsgi protocol – NGINX and NGINX Plus support uswgi by default. Configuring the uWSGI application container itself is outside the scope of this document; refer to the excellent [Quickstart for Python/WSGI applications](http://uwsgi-docs.readthedocs.org/en/latest/WSGIquickstart.html) for more information. [Django](https://www.djangoproject.com/) is a common Python web framework. For simplicity the example uses a Django‑based setup for the Python app. The [Django documentation](https://docs.djangoproject.com/en/1.11/) provides extensive information on how to configure a Django environment. This example is illustrative only, and one way you might invoke your uWSGI server with Django: ```shell uwsgi \ --chdir=/var/django/projects/myapp \ --module=myapp.wsgi:application \ --env DJANGO_SETTINGS_MODULE=myapp.settings \ --master --pidfile=/usr/local/var/run/uwsgi/project-master.pid \ --socket=127.0.0.1:29000 \ --processes=5 \ --uid=505 --gid=505 \ --harakiri=20 \ --max-requests=5000 \ --vacuum \ --daemonize=/usr/local/var/log/uwsgi/myapp.log ``` With these options in place, here's a sample NGINX configuration for use with a Django project: ```nginx http { # ... upstream django { server 127.0.0.1:29000; } server { listen 80; server_name myapp.example.com; root /var/www/myapp/html; location / { index index.html; } location /static/ { alias /var/django/projects/myapp/static/; } location /main { include /etc/nginx/uwsgi_params; uwsgi_pass django; uwsgi_param Host $host; uwsgi_param X-Real-IP $remote_addr; uwsgi_param X-Forwarded-For $proxy_add_x_forwarded_for; uwsgi_param X-Forwarded-Proto $http_x_forwarded_proto; } } } ``` This configuration defines an upstream named `django`. The port number `29000` specified for the server in this upstream matches the port that the uWSGI server binds to, as specified by the `--socket=` argument in the sample `uwsgi` command. Serving static content is offloaded to NGINX or NGINX Plus, which serves it directly from `/var/django/projects/myapp/static`. Application traffic to the `/main` location is proxied and bridged from HTTP to the uwsgi protocol, and then passed to the Django app that runs within the uWSGI application container. ## Conclusion Lightweight, heterogeneous application environments are becoming a popular way of building and deploying modern web applications. Newer, standardized application interface protocols like uwsgi and FastCGI enable faster communication between users and applications. Using NGINX and NGINX Plus in front of an application container has become a common way to free applications from the burden of HTTP traffic management, and to protect the application from unexpected spikes of user traffic, malicious behavior, denial‑of‑service (DoS) attacks, and more. This allows developers to fully focus on the application logic, and leave the web acceleration and fundamental HTTP traffic security tasks to NGINX or NGINX Plus. ## Resources - [NGINX support](https://uwsgi-docs.readthedocs.io/en/latest/Nginx.html) in the uWSGI project documentation - [How to use Django with uWSGI](https://docs.djangoproject.com/en/1.11/howto/deployment/wsgi/uwsgi/) in the Django project documentation -------------------------------------------------------------------------------- title: "Single Sign-On with OpenID Connect and Identity Providers" url: https://docs.nginx.com/nginx/admin-guide/security-controls/configuring-oidc/index.md description: Enable OpenID Connect-based single sign-on (SSO) for applications proxied by NGINX Plus, using an Identity Provider (IdP). -------------------------------------------------------------------------------- This guide explains how to enable single sign-on (SSO) for applications being proxied by F5 NGINX Plus using: - OpenID Connect as the authentication mechanism. - An external Identity Provider (IdP) such as AD FS, Auth0, Cognito, Entra ID, Keycloak, OneLogin, Okta, Ping Identity and others. - NGINX Plus as an OIDC client application that verifies user identity (Relying Party). OpenID Connect is an identity protocol that utilizes the authorization and authentication mechanisms of OAuth 2.0. With it, NGINX Plus can provide a layer of authentication for protected applications that do not natively support it. NGINX Plus consumes authorization and claims from OpenID Connect Identity Providers by utilizing JWT-based identity tokens that are delivered via the OAuth 2.0 framework. It supports several specific flows suitable for browser-based and desktop/mobile applications. OpenID Connect allows the client (NGINX Plus) to retrieve an ID token in addition to an access token. The ID token provides information about the authenticated user. For the target client application, OIDC authentication can be enabled with great flexibility on different levels. It can be enabled globally, or more granular at a per-server or a per-location level. Additionally, OIDC supports auto discovery and retrieval of the OpenID provider configuration metadata while also allowing the definition of additional metadata if needed. ## OpenID Connect Workflow and NGINX Plus {#oidc-workflow} 1. A user accesses a protected resource. 2. NGINX Plus redirects the user to the IdP for user authentication and authorization. 3. The IdP collects user credentials and authenticates the user. 4. The IdP redirects the user back to NGINX Plus with an authorization code. 5. NGINX Plus retrieves an `id_token` and access token using the authorization code from the IdP. 6. NGINX Plus validates the `id_token` and retrieves profile data for the user using the `UserInfo` endpoint. The retrieved profile data is validated and the content of the `id_token` and the profile data is used for providing access control to the client application. 7. Upon successful validation, the resource access request is sent to the client application along with the access token. 8. The client application validates the access token and based on the token validation, the resource access request is allowed or denied. ![image](nginx/images/oidc.png) ## Tested Identity Providers {#deployment-guildes} NGINX Plus has tested support with the following Identity Providers: | IdP Provider | Resource | |-----------------|-------------------------------------------------------------------------------------------------------------| | Amazon Cognito | [Deployment Guide for Amazon Cognito](nginx/deployment-guides/single-sign-on/cognito.md) | | Auth0 | [Deployment Guide for Auth0](nginx/deployment-guides/single-sign-on/auth0.md) | | Microsoft AD FS | [Deployment Guide for AD FS](nginx/deployment-guides/single-sign-on/active-directory-federation-services.md) | | Microsoft Entra / Azure ID| [Deployment Guide for Entra ID](nginx/deployment-guides/single-sign-on/entra-id.md) | | Keycloak | [Deployment Guide for Keycloak](nginx/deployment-guides/single-sign-on/keycloak.md) | | OneLogin | [Deployment Guide for OneLogin](nginx/deployment-guides/single-sign-on/onelogin.md) | | Okta | [Deployment Guide for Okta](nginx/deployment-guides/single-sign-on/okta.md) | | Ping Identity | [Deployment Guide for Ping Identity](nginx/deployment-guides/single-sign-on/ping-identity.md) | ## Prerequisites {#prerequisites} - An identity provider (IdP) set up on your network or in the cloud. You need admin access to the IdP. - An [NGINX Plus subscription](https://www.f5.com/products/nginx/nginx-plus) and NGINX Plus [Release 34](nginx/releases.md#r34) or later. To install NGINX Plus, follow the steps in [Installing NGINX Plus](https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-plus/). - A domain name that points to your NGINX Plus instance (for example, `demo.example.com`). ## Set up your identity provider (IdP) {#idp-setup} The setup steps are similar for most identity providers, but some details may differ. 1. Log in to your IdP's admin console. 2. Create a new OpenID Connect (OIDC) application. - Give the app a name. - Add the users or groups who need access. 3. Find the **Client ID** and **Client Secret** for your app. You'll need these later when you [set up NGINX Plus as the relying party](#setup-oidc-provider2). 4. Find the **issuer** value. You can find the issuer value in your IdP app settings or at the standard discovery URL: `https://your-idp-domain/.well-known/openid-configuration` where: - the `your-idp-domain` is your IdP's server address - the `.well-known/openid-configuration` is the default address for IdPs for the `.well-known` document - the `userinfo_endpoint` is URI that a user visits to start an RP‑initiated logout flow. - the `end_session_endpoint` is the URL where your IdP should redirect the user after a successful logout. The IdP configuration metadata is returned in the JSON format, for example: ```json { ... "issuer": "https://your-idp-domain/idp", "authorization_endpoint": "https://your-idp-domain/idp/oauth2/authorize/", "token_endpoint": "https://your-idp-domain/idp/oauth2/token/", "jwks_uri": "https://your-idp-domain/idp/discovery/keys", "userinfo_endpoint": "https://your-idp-domain/idp/userinfo", "end_session_endpoint": "https://your-idp-domain/idp/oauth2/logout", ... } ``` Copy the **issuer** value. You will need it later when [configuring NGINX Plus as the Relying Party](#setup-oidc-provider2). 5. Configure a logout URI - a URI that a user visits to initiate an RP‑initiated logout flow, for example, `https://demo.example.com/post_logout/`. (Optional, supported since [NGINX Plus R35](nginx/releases.md#r35)). ## Configure the Relying Party (NGINX Plus) {#rp-setup} With your IdP configured, you can enable OIDC on NGINX Plus. NGINX Plus serves as the Rely Party (RP) client service that verifies user identity. 1. Ensure that you are using the latest version of NGINX Plus by running the `nginx -v` command in a terminal: ```shell nginx -v ``` The output should match NGINX Plus Release 35: ```none nginx version: nginx/1.29.0 (nginx-plus-r35) ``` 2. Ensure that you have the values of the **Client ID**, **Client Secret**, and **Issuer** obtained from your IdP Provider. 3. In your preferred text editor, open the NGINX configuration file (`/etc/nginx/nginx.conf` for Linux or `/usr/local/etc/nginx/nginx.conf` for FreeBSD). 4. In the [`http {}`](https://nginx.org/en/docs/http/ngx_http_core_module.html#http) context, make sure your public DNS resolver is specified with the [`resolver`](https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver) directive: By default, NGINX Plus re‑resolves DNS records at the frequency specified by time‑to‑live (TTL) in the record, but you can override the TTL value with the `valid` parameter: ```nginx http { resolver 10.0.0.1 ipv4=on valid=300s; # ... } ``` 5. In the [`http {}`](https://nginx.org/en/docs/http/ngx_http_core_module.html#http) context, define the IdP provider named `my_idp` by specifying the [`oidc_provider {}`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#oidc_provider) context: ```nginx http { resolver 10.0.0.1 ipv4=on valid=300s; oidc_provider my_idp { # ... } # ... } ``` 6. In the [`oidc_provider {}`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#oidc_provider) context, specify: - Your actual **Client ID** obtained from your IdP with the [`client_id`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_id) directive - Your **Client Secret** obtained from your IdP with the [`client_secret`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_secret) directive - The **Issuer** URL obtained from your IdP with the [`issuer`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_secret) directive By default, NGINX Plus creates the metadata URL by appending the `/.well-known/openid-configuration` part to the Issuer URL. If your Issuer is different, you can explicitly specify the metadata document with the [`config_url`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#config_url) directive. - A valid system CA bundle with the [`ssl_trusted_certificate`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#ssl_trusted_certificate) so that NGINX Plus could validate the IdP TLS certificates: ```nginx http { resolver 10.0.0.1 ipv4=on valid=300s; oidc_provider my_idp { issuer https://your-idp-domain/idp; client_id ; client_secret ; ssl_trusted_certificate /etc/ssl/certs/ca-certificates.crt; } # ... } ``` 7. Make sure you have configured a [server](https://nginx.org/en/docs/http/ngx_http_core_module.html#server) that corresponds to `demo.example.com`, and there is a [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) that [points](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass) to your application (see [Step 10](#oidc_app)) at `http://127.0.0.1:8080` that is going to be OIDC-protected: ```nginx http { # ... server { listen 443 ssl; server_name demo.example.com; ssl_certificate /etc/ssl/certs/fullchain.pem; ssl_certificate_key /etc/ssl/private/key.pem; location / { # ... proxy_pass http://127.0.0.1:8080; } } # ... } ``` 8. Protect this [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) with IdP OIDC by specifying the [`auth_oidc`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#auth_oidc) directive that will point to the `my_idp` configuration specified in the [`oidc_provider {}`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#oidc_provider) context in [Step 5](#setup-oidc-provider): ```nginx # ... location / { auth_oidc my_idp; # ... proxy_pass http://127.0.0.1:8080; } # ... ``` 9. Pass the OIDC claims as headers to the application ([Step 10](#oidc_app)) with the [`proxy_set_header`](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_header) directive. These claims are extracted from the ID token returned by the IdP: - [`$oidc_claim_sub`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) - a unique `Subject` identifier assigned for each user by the IdP - [`$oidc_claim_email`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) the e-mail address of the user - [`$oidc_claim_name`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) - the full name of the user - Any other OIDC claim using the [`$oidc_claim_ `](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) variable ```nginx # ... location / { auth_oidc my_idp; proxy_set_header sub $oidc_claim_sub; proxy_set_header email $oidc_claim_email; proxy_set_header name $oidc_claim_name; proxy_pass http://127.0.0.1:8080; } # ... ``` 10. Create a simple test application referenced by the `proxy_pass` directive which returns the authenticated user's full name and email upon successful authentication: ```nginx # ... server { listen 8080; location / { return 200 "Hello, $http_name!\nEmail: $http_email\nIdP sub sub: $http_sub\n"; default_type text/plain; } } ``` 11. Save the NGINX configuration file and reload the configuration: ```nginx nginx -s reload ``` ### Complete Example {#example} This configuration example summarizes the steps outlined above. It includes only essential settings such as specifying the DNS resolver, defining the OIDC provider, configuring SSL, and proxying requests to an internal server. ```nginx http { # Use a public DNS resolver for Issuer discovery, etc. resolver 10.0.0.1 ipv4=on valid=300s; oidc_provider my_idp { # The 'issuer' typically matches your IdP's base URL issuer https:///idp; # Provide a CA bundle for certificate validation ssl_trusted_certificate /etc/ssl/certs/ca-certificates.crt; # Replace with your actual IdP's client_id and secret client_id ; client_secret ; # RP‑initiated logout logout_uri /logout; post_logout_uri https://demo.example.com/post_logout/; logout_token_hint on; # Fetch userinfo claims userinfo on; # If the .well-known endpoint cannot be derived automatically, # specify config_url: # config_url https:///auth/realms/main/.well-known/openid-configuration; } server { listen 443 ssl; server_name demo.example.com; ssl_certificate /etc/ssl/certs/fullchain.pem; ssl_certificate_key /etc/ssl/private/key.pem; location / { # Protect this location with OIDC auth_oidc my_idp; # Forward OIDC claims as headers if desired proxy_set_header sub $oidc_claim_sub; proxy_set_header email $oidc_claim_email; proxy_set_header name $oidc_claim_name; proxy_pass http://127.0.0.1:8080; } location /post_logout/ { return 200 "You have been logged out.\n"; default_type text/plain; } } server { # simple test oidc-protected application listen 8080; location / { return 200 "Hello, $http_name!\nEmail: $http_email\nIdP sub: $http_sub\n"; default_type text/plain; } } } ``` ### Testing {#testing} 1. Open https://demo.example.com/ in a browser. You should be redirected to your IdP's login page. 2. Enter valid IdP credentials for a user assigned to the `nginx-demo-app` client. Upon successful sign-in, the IdP redirects you back to NGINX Plus, and you will see the proxied application content (for example, “Hello, Jane Doe!”). ## Glossary {#glossary} | Term | Description | |-------------------------|----------------------------------------------------| | Identity Provider (IdP) | A service that authenticates users and verifies their identity for client applications. | | Protected Resource | A resource that is hosted by the resource server and requires an access token to be accessed. | | Relying Party (RP) | A client service required to verify user identity. | | JSON Web Token (JWT) | An open standard (RFC 7519) that defines a compact and self-contained way for securely transmitting information between parties as a JSON object. This information can be verified and trusted because it is digitally signed. | | ID Token | Specific to OIDC, the primary use of the token in JWT format is to provide information about the authentication operation's outcome. | | Access Token | Defined in OAuth2, this (optional) short lifetime token provides access to specific user resources as defined in the scope values in the request to the authorization server (can be a JSON token as well). | | Refresh Token | Coming from OAuth2 specs, the token is usually long-lived and may be used to obtain new access tokens. | ## See Also {#see-also} - [Simplifying OIDC and SSO with the New NGINX Plus R34 OIDC Module blog post](https://community.f5.com/kb/technicalarticles/simplifying-oidc-and-sso-with-the-new-nginx-plus-r34-oidc-module/340552) - [NGINX Plus Native OIDC Module Reference documentation](https://nginx.org/en/docs/http/ngx_http_oidc_module.html) - [Release Notes for NGINX Plus R34](nginx/releases.md#r34) -------------------------------------------------------------------------------- title: "Create Amazon EC2 Instances for NGINX Open Source and NGINX Plus" url: https://docs.nginx.com/nginx/deployment-guides/amazon-web-services/ec2-instances-for-nginx/index.md description: Create Amazon Elastic Compute Cloud (EC2) instances for running NGINX Open Source and F5 NGINX Plus. -------------------------------------------------------------------------------- This guide explains how to create instances in the Amazon Elastic Compute Cloud (EC2) that can run NGINX Open Source and F5 NGINX Plus. For NGINX Plus, you can buy a prebuilt Amazon Machine Image (AMI) from the AWS Marketplace for a faster option. You can find AMIs for various operating systems, such as Amazon Linux, Red Hat Enterprise Linux, and Ubuntu. For instructions, see [Installing NGINX Plus AMIs on Amazon EC2](/nginx/admin-guide/installing-nginx/installing-nginx-plus-amazon-web-services.md). ## Before you begin To complete this guide, you need the following: - An [AWS account](http://docs.aws.amazon.com/AmazonSimpleDB/latest/DeveloperGuide/AboutAWSAccounts.html). - Basic Linux system administration skills, including installing software, managing files and folders, and using the command line, to follow the [Ansible instructions](#automate-ansible). - A paid or trial subscription for NGINX Plus. - Root privileges on the hosts where NGINX Open Source or NGINX Plus will be installed, with `sudo` access as needed. ## Create an Amazon EC2 Instance {#create-ec2-instances} 1. Log in to the EC2 dashboard in the AWS Management Console: **[https://console.aws.amazon.com/ec2](https://console.aws.amazon.com/ec2)**. 1. In the left navigation bar, choose **Instances**. Then select **Launch Instances** in the top right corner. 1. On the **Launch an Instance** page, give your new instance a name in the **Name and Tags** section. This name will show in the Name column of the summary table on the EC2 Instances dashboard. This guide is using "instance-name." 1. In the **Application and OS Images (Amazon Machine Image)** section select the image of the Linux distribution of your choice. ![Screenshot of AMI section on the EC2 Launch Instance page](/img/aws/aws-nlb-instance-choose-ami.png) 1. In the **Instance Type** section, choose an appropriate instance type. The screenshot shows the **t2.micro** instance type selected by default. This type is sufficient for demo purposes. **Note:** At the time of publication, AWS offered 750 hours of free usage each month for this instance type. This applies during your first year with an AWS account. Keep in mind, though, that several NGINX instances running all day will use the free 750 hours up quickly. For example, 6 instances will use them in just over 5 days. If you use 8 instances, you'll hit the limit in under 4 days. ![Screenshot of Instance Type on the EC2 Launch Instance page](/img/aws/aws-nlb-instance-choose-type.png) 1. In the **Key pair (login)** section, select an existing key pair or create a new one. If you choose **Create new key pair**, a window appears, allowing you to download the key pair. **Note:** It's best practice — and necessary in production — to create a separate key for each EC2 instance. This way, if a key is compromised, only that one instance is at risk. 1. Scroll to the **Network settings** section. You can keep the default **VPC** and **Subnet** settings. Under **Firewall (Security Groups),** either create a new security group or use an existing one. - If this is your first setup, keep **Create security group** selected. - Select **Allow HTTP traffic from the internet.** - (Optional) Select **Allow HTTPS traffic from the internet** if needed. This creates the following inbound security group rules: 1. Accept SSH connections from all sources - **Type** – SSH - **Protocol** – TCP - **Port Range** – 22 - **Source** – Custom `0.0.0.0/0` 1. Accept unencrypted HTTP connections from all sources - **Type** – HTTP - **Protocol** – TCP - **Port Range** – 80 - **Source** – Custom `0.0.0.0/0` 1. Accept encrypted HTTPS connections from all soruces (optional) - **Type** – HTTPS - **Protocol** – TCP - **Port Range** – 443 - **Source** – Custom `0.0.0.0/0` ![Screenshot of Network Settings on the EC2 Launch Instance page](/img/aws/aws-generic-instance-details.png) If you are deploying multiple instances, it makes sense to use the same security group for all. In that case, choose "Select existing security group." This will allow you to pick a security group from a list. ![Screenshot of Network Settings on the EC2 Launch Instance page with the option of selecting an existing security group](/img/aws/aws-instance-select-sg.png) 1. In the **Configure Storage** section, leave the defaults unchanged. 1. In the right panel, verify the settings in the **Summary** section. If everything is correct, select **Launch Instance**. ![Screenshot of the Summary panel on the EC2 Launch Instance page with button to launch instance](/img/aws/aws-instance-summary.png) 1. After launching the instance, you are redirected to a confirmation page with a success message. Select the instance ID to return to the **EC2 Instances** page. ![Screenshot of the Summary panel on the EC2 Launch Instance page with button to launch instance](/img/aws/aws-instance-launch-success.png) 1. On the **EC2 Instances** page, you can view all your instances, including the new one. The following screenshot shows a single instance: ![Screenshot of the EC2 Instances page with a single instance](/img/aws/aws-generic-instance-display-first.png) 1. Finalize your security group rules. You only need to do this for the first instance in a set. All other instances in that set can use the same security group. - In the left navigation bar, select **Security Groups**. - Select the security group by clicking its radio button in the leftmost column of the table. A panel opens in the lower part of the window displaying details about the group. - In the **Inbound** tab, verify that the rules you created in Step 7 are listed. - Open the **Outbound** tab and select **Edit outbound rules** to create a rule for outbound traffic. The rules depend on the ports used for traffic handled by NGINX Open Source or NGINX Plus instances: - By default, AWS adds an outbound rule that allows all traffic to all destinations. You can remove this rule by selecting **Delete**. - If you use port 80 for client traffic and health checks from a load balancer, like [AWS Network Load Balancer](high-availability-network-load-balancer.md), you only need one rule. - If you set up different ports for various tasks, or if you use ports like 443 for HTTPS, adjust them accordingly. In the **Destination** field, start typing your security group's name or ID. It should appear under **Security Groups**. The example below shows **sg-0dd4d3c5284052f99**. ![Screenshot of the EC2 Security Group outbound rules page](/img/aws/aws-generic-instance-security-outbound.png) 1. To install NGINX software on the instance, first [connect](#connect-to-an-ec2-instance) to it. Then follow the instructions in the NGINX Plus Admin Guide for [NGINX Open Source](/nginx/admin-guide/installing-nginx/installing-nginx-open-source#prebuilt) and [NGINX Plus](/nginx/admin-guide/installing-nginx/installing-nginx-plus.md). --- ## Connect to an EC2 Instance To install and configure NGINX Open Source or NGINX Plus on an instance, open a terminal window and connect to the instance over SSH. 1. Navigate to the **Instances** tab on the EC2 Dashboard if you are not there already. 1. Select the row for an instance to highlight it. 1. Select **Connect** above the list of instances. You are redirected to the **Connect to Instance** page, with the **SSH client** tab selected by default. 1. Follow the instructions on the page, which are customized for the selected instance. A sample `ssh` command includes the key file name and the instance hostname. ![Screenshot of the EC2 Instance Connect with SSH page](/img/aws/aws-nlb-instance-connect.png) --- ## Install NGINX software Once you have established a connection with an instance, you can install the NGINX software on it. Follow the instructions in the NGINX Plus Admin Guide for [NGINX Open Source](/nginx/admin-guide/installing-nginx/installing-nginx-open-source#prebuilt) and [NGINX Plus](/nginx/admin-guide/installing-nginx/installing-nginx-plus.md). The [Admin Guide](/nginx/admin-guide/_index.md) also provides instructions for many maintenance tasks. ### Automate installation with a configuration manager You can automate the installation of NGINX Open Source and NGINX Plus. Instructions for Ansible are provided below. #### Automate installation with Ansible {#automate-ansible} NGINX, Inc. releases a combined Ansible role for NGINX Open Source and NGINX Plus on [Ansible Galaxy](https://galaxy.ansible.com/nginxinc/nginx/) and [GitHub](https://github.com/nginxinc/ansible-role-nginx). Perform these steps to install and run it. 1. [Connect to the EC2 instance](#connect-instance). 1. Install Ansible following the [instructions](https://docs.ansible.com/ansible/latest/installation_guide/installation_distros.html) for the operating system on your EC2 instance. 1. Install the official Ansible role from NGINX: ```shell ansible-galaxy install nginxinc.nginx ``` 1. (NGINX Plus only) Copy the **nginx-repo.key** and **nginx-repo.crt** files provided by NGINX, Inc. to **~/.ssh/ngx-certs/**. 1. Create a file called **playbook.yml** with the following contents: ```none --- - hosts: localhost become: true roles: - role: nginxinc.nginx ``` 1. Run the playbook: ```shell ansible-playbook playbook.yml ``` 1. Confirm that NGINX is installed by running `nginx -v`. ## Optional: Create an NGINX Open Source AMI To simplify installing NGINX Open Source on several instances, create an AMI from an existing NGINX instance. You can then create additional virtual instances based on the AMI. 1. Follow the instructions in [Create Amazon EC2 Instance](#create-ec2-instances) and [Install NGINX software](#install-nginx-software). 1. Go to the **Instances** tab on the Amazon EC2 Dashboard. 1. Select the base instance with NGINX installed by clicking its row in the table. 1. Select **Actions**, then choose **Image and templates** > **Create Image**. ![Screenshot of the EC2 Create Image button menu](/img/aws/aws-generic-create-image-menu.png) 1. On the **Create Image** page, enter the **Image name** and optionally add an **Image description**. Then select **Create image**. You are returned to the **Instances** page with a green alert confirming that the image is being created. 1. To verify that the image was created, go to the **AMIs** tab and find the new image by its name. ### Revision History - Version 3 (February 2025) - Update guide to match new AWS instance creation flow, update screenshots, and links. - Version 2 (July 2018) – Substitute links to NGINX Plus Admin Guide for sample installation instructions. - Version 1 (April 2018) – Initial version (NGINX Plus Release 14) -------------------------------------------------------------------------------- title: "Deploying NGINX and NGINX Plus with Docker" url: https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-docker/index.md description: Deploy NGINX and F5 NGINX Plus as the Docker container. -------------------------------------------------------------------------------- [F5 NGINX Plus](https://www.nginx.com/products/nginx/), the high‑performance application delivery platform, load balancer, and web server, is available as the Docker container. ## Prerequisites - The [Docker Engine](https://docs.docker.com/engine/install/) command-line tool - for NGINX Plus: * the JSON Web Token (JWT) from [MyF5](https://my.f5.com) customer portal * the *nginx-repo.crt* and *nginx-repo.key* files from [MyF5](https://my.f5.com) customer portal * your private Docker registry is configured and running - for NGINX Open Source: [Docker Hub](https://hub.docker.com/) account ## Use official NGINX Plus Docker images Since NGINX Plus [Release 31](nginx/releases.md#r31) you can get an NGINX Plus image from the official NGINX Plus Docker registry and upload it to your private registry. The NGINX Plus Docker registry is available at `https://private-registry.nginx.com/v2/`. The registry contains the following image types: - [NGINX Plus](/nginx/releases.md):
`https://private-registry.nginx.com/v2/nginx-plus/base` - [Unprivileged](/nginx/admin-guide/installing-nginx/installing-nginx-plus.md#unpriv_install) installation of NGINX Plus:
`https://private-registry.nginx.com/v2/nginx-plus/rootless-base` - NGINX Plus bundled with [NGINX Agent](https://docs.nginx.com/nginx-agent/overview/):
`https://private-registry.nginx.com/v2/nginx-plus/agent` - Unprivileged installation of NGINX Plus and NGINX Agent:
`https://private-registry.nginx.com/v2/nginx-plus/rootless-agent` - [NGINX Plus dynamic modules](/nginx/admin-guide/dynamic-modules/dynamic-modules.md):
`https://private-registry.nginx.com/v2/nginx-plus/modules` The images can be targeted for a particular operating system and NGINX Plus release using tags. ### Tags for operating systems | Operating system | Basic OS tag | Tag examples | |--------------------------------------------|--------------|-----------------------------------------------| | Alpine (x86_64, aarch64) | `alpine` | `r36-alpine`, `r36-alpine-3.20` | | Debian (x86_64, aarch64) | `debian` | `r36-debian`, `r36-debian-bookworm` | | Red Hat Enterprise Linux (x86_64, aarch64) | `ubi` | `r36-ubi`, `r36-ubi-9`, `r36-ubi-9-20251201` | ### Tags for NGINX Plus versions The NGINX Plus registry contains images for the two most recent versions of NGINX Plus. The basic operating system tag returns the latest version of NGINX Plus built for the latest version of this operating system. **Note:** `nginx-plus-r36-ubi-9`, `nginx-plus-r36-alpine-3.22`. The image may contain a particular version of NGINX Plus or contain a bundle of NGINX Plus and NGINX Agent, and can be targeted for a specific architecture. ### List all tags For a complete tag list for NGINX Plus bundled with NGINX Agent images, use the command: ```shell curl https://private-registry.nginx.com/v2/nginx-plus//tags/list --key --cert | jq ``` where: - the `` is the location of images in NGINX Plus private registry: - `base` is NGINX Plus only - `rootless-base` is NGINX Plus run from `nginx` user - `agent` is NGINX Plus along with NGINX Agent in a single image - `rootless-agent` is NGINX Plus with NGINX Agent both run from `nginx` user - `modules` is NGINX Plus dynamic modules - the `` is a local path to your client key from MyF5, for example, `/etc/ssl/nginx/nginx-repo-x12345.key` - the `` is a local path to your client certificate from MyF5, for example,`/etc/ssl/nginx/nginx-repo-x12345.crt` - the `jq` command is used to format the JSON output for easier reading and requires the [jq](https://jqlang.github.io/jq/) JSON processor to be installed. ### Download your subscription credential files In order to obtain a container image, you will need the JSON Web Token file or SSL certificate and private key files provided with your NGINX Plus subscription. These files grant access to the package repository from which the script will download the NGINX Plus package: #### JSON Web Token #### SSL ### Set up Docker for the F5 Container Registry This step describes how to use Docker to communicate with the F5 Container Registry located at `private-registry.nginx.com`. **Note:** The steps provided are for Linux. For Mac or Windows, see the [Docker for Mac](https://docs.docker.com/docker-for-mac/#add-client-certificates) or [Docker for Windows](https://docs.docker.com/docker-for-windows/#how-do-i-add-client-certificates) documentation. For more details on Docker Engine security, you can refer to the [Docker Engine Security documentation](https://docs.docker.com/engine/security/). #### JSON Web Token Open the JSON Web Token file previously downloaded from [MyF5](https://my.f5.com) customer portal (for example, `nginx-repo-12345abc.jwt`) and copy its contents. Log in to the Docker registry using the contents of the JSON Web Token file: ```shell docker login private-registry.nginx.com --username= --password=none ``` #### SSL Create a directory and copy your certificate and key to this directory: ```shell mkdir -p /etc/docker/certs.d/private-registry.nginx.com cp /etc/docker/certs.d/private-registry.nginx.com/client.cert cp /etc/docker/certs.d/private-registry.nginx.com/client.key ``` ### Pull the image Next, pull the image you need from `private-registry.nginx.com`. To pull an image, replace `` with the specific NGINX Plus version or the NGINX Plus version and OS version you need. For NGINX Plus, run: ```shell docker pull private-registry.nginx.com/nginx-plus/base: ``` For NGINX Plus with NGINX Agent version 2, run: ```shell docker pull private-registry.nginx.com/nginx-plus/agent: ``` For NGINX Plus with NGINX Agent version 3, run: ```shell docker pull private-registry.nginx.com/nginx-plus/agentv3: ``` For NGINX Plus installed from `nginx` user (rootless installation), run: ```shell docker pull private-registry.nginx.com/nginx-plus/rootless-base: ``` For NGINX Plus with NGINX Agent version 2 installed from `nginx` user (rootless installation), run: ```shell docker pull private-registry.nginx.com/nginx-plus/rootless-agent: ``` For NGINX Plus with NGINX Agent version 3 installed from `nginx` user (rootless installation), run: ```shell docker pull private-registry.nginx.com/nginx-plus/rootless-agentv3: ``` For NGINX modules, run: ```shell docker pull private-registry.nginx.com/nginx-plus/modules: ``` **Note:** To protect your system and data, follow these security practices: 1. **JWTs:** Treat JSON Web Tokens (JWTs) as sensitive data. Store them securely and delete them after use to prevent unauthorized access. 2. **Shell history:** Commands with JWTs or passwords are saved in plain text in your shell history. After running such commands, clear the history to protect credentials. For example: - Edit your shell history file (such as ~/.bash_history or ~/.zsh_history) to remove specific commands. - Use `history -c` to clear all shell history in bash or zsh. ### Push the image to your private registry After pulling the image, tag it and upload it to your private registry. > **Note:** Never upload your NGINX Plus images to a public repository such as Docker Hub. Doing so violates your license agreement. Log in to your private registry: ```shell docker login ``` Tag and push the image. Replace `` with your registry’s path and `` with the your NGINX Plus version and/or OS version: ```shell docker tag private-registry.nginx.com/nginx-plus/base: /nginx-plus/base: ``` ```shell docker push /nginx-plus/base: ``` ### Run the NGINX Plus container **Note:** Starting from [NGINX Plus Release 33](nginx/releases.md#r33), the JWT file is required for each NGINX Plus instance. For more information, see [About Subscription Licenses](/solutions/about-subscription-licenses.md). To start the Docker container with NGINX Plus, you will need to pass your JWT license file named `license.jwt` as the `NGINX_LICENSE_JWT` environment variable. If the license file needs to be located in a non-default directory, specify its full path using the `NGINX_LICENSE_PATH` variable (default path: `/etc/nginx/license.jwt`). To start the Docker container with NGINX Plus only: ```sh sudo docker run \ --env=NGINX_LICENSE_JWT=$(cat license.jwt) \ --restart=always \ --runtime=runc \ -d /nginx-plus/base: ``` To start the Docker container with NGINX Plus and NGINX Agent, you will need to additionally pass the NGINX One data plane key as the `NGINX_AGENT_SERVER_TOKEN` environment variable. For more information, see [Create and manage data plane keys](/nginx-one-console/connect-instances/create-manage-data-plane-keys.md): ```sh sudo docker run \ --env=NGINX_LICENSE_JWT=$(cat license.jwt) \ --env=NGINX_AGENT_SERVER_GRPCPORT=443 \ --env=NGINX_AGENT_SERVER_HOST=agent.connect.nginx.com \ --env=NGINX_AGENT_SERVER_TOKEN="YOUR_NGINX_ONE_DATA_PLANE_KEY" \ --env=NGINX_AGENT_TLS_ENABLE=true \ --restart=always \ --runtime=runc \ -d /nginx-plus/agent: ``` where: - `NGINX_LICENSE_JWT` is your JWT license file from MyF5. The file name should be `license.jwt`. - `NGINX_AGENT_SERVER_GRPCPORT` sets a GRPC port used by NGINX Agent to communicate with NGINX Instance Manager. - `NGINX_AGENT_SERVER_HOST` sets the domain name or IP address of NGINX Instance Manager. Note that for production environments it is not recommended to expose NGINX Instance Manager to public networks. - `NGINX_AGENT_SERVER_TOKEN` sets NGINX One data plane key. See [Create and manage data plane keys](/nginx-one-console/connect-instances/create-manage-data-plane-keys.md) for details. - `NGINX_AGENT_TLS_ENABLE` enables mutual TLS, server-side TLS, or insecure mode (not recommended for production environments). See [Encrypt communication](agent/configuration/encrypt-communication.md) for details. - `YOUR_REGISTRY` is the path to your private registry. - `VERSION_TAG` is the tag assigned when pushing to your registry.
**Note:** To start the Docker Container with NGINX Plus and NGINX Agent on Debian or Ubuntu: ```sh sudo docker run \ --env=NGINX_LICENSE_JWT=$(cat license.jwt) \ --env=NGINX_AGENT_SERVER_GRPCPORT=443 \ --env=NGINX_AGENT_SERVER_HOST=agent.connect.nginx.com \ --env=NGINX_AGENT_SERVER_TOKEN="YOUR_NGINX_ONE_DATA_PLANE_KEY" \ --env=NGINX_AGENT_TLS_ENABLE=true \ --restart=always \ --runtime=runc \ -d private-registry.nginx.com/nginx-plus/agent:debian ## Using NGINX Open Source Docker Images You can create an NGINX instance in a Docker container using the NGINX Open Source image from the Docker Hub. 1. Launch an instance of NGINX running in a container and using the default NGINX configuration with the following command: ```shell docker run --name mynginx1 -p 80:80 -d nginx ``` where: - `mynginx1` is the name of the created container based on the NGINX image - the `-d` option specifies that the container runs in detached mode: the container continues to run until stopped but does not respond to commands run on the command line. - the `-p` option tells Docker to map the ports exposed in the container by the NGINX image (port `80`) to the specified port on the Docker host. The first parameter specifies the port in the Docker host, the second parameter is mapped to the port exposed in the container The command returns the long form of the container ID: `fcd1fb01b14557c7c9d991238f2558ae2704d129cf9fb97bb4fadf673a58580d`. This form of ID is used in the name of log files. 2. Verify that the container was created and is running with the `docker ps` command: ```shell $ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS ... fcd1fb01b145 nginx:latest "nginx -g 'daemon of 16 seconds ago Up 15 seconds ... ... PORTS NAMES ... 0.0.0.0:80->80/tcp mynginx1 ``` This command also allows viewing the port mappings set in the previous step: the `PORTS` field in the output reports that port `80` on the Docker host is mapped to port `80` in the container. ## Creating custom NGINX Plus Docker Image As NGINX Plus is a commercial offering, NGINX Plus Docker images are not available at Docker Hub, so first you will need to create an NGINX Plus Docker image. > **Note:** Never upload your NGINX Plus images to a public repository such as Docker Hub. Doing so violates your license agreement. To generate a custom NGINX Plus image: 1. Create the Docker build context, or a Dockerfile, for example: 2. As with NGINX Open Source, default NGINX Plus image has the same default settings: - access and error logs are linked to the Docker log collector - no volumes are specified: a Dockerfile can be used to create base images from which you can create new images with volumes specified, or volumes can be specified manually: ```dockerfile VOLUME /usr/share/nginx/html VOLUME /etc/nginx ``` - no files are copied from the Docker host as a container is created: you can add `COPY` definitions to each Dockerfile, or the image you create can be used as the basis for another image 3. Log in to [MyF5 Customer Portal](https://account.f5.com/myf5). As noted in the [Prerequisites](#prerequisites], download your *nginx-repo.crt*, *nginx-repo.key*, and **JSON Web Token** files. For a trial of NGINX Plus, the files are provided with your trial package. 4. Copy the files to the directory where the Dockerfile is located. 5. Create a Docker image, for example, `nginxplus` (note the final period in the command). ```shell docker build --no-cache --secret id=nginx-key,src=nginx-repo.key --secret id=nginx-crt,src=nginx-repo.crt --secret id=nginx-jwt,src=license.jwt -t nginxplus . ``` The `--no-cache` option tells Docker to build the image from scratch and ensures the installation of the latest version of NGINX Plus. If the Dockerfile was previously used to build an image without the `--no-cache` option, the new image uses the version of NGINX Plus from the previously built image from the Docker cache. 6. Verify that the `nginxplus` image was created successfully with the `docker images` command: ```shell $ docker images nginxplus REPOSITORY TAG IMAGE ID CREATED SIZE nginxplus latest ef2bf65931cf 6 seconds ago 91.2 MB ``` 7. Create a container based on this image, for example, `mynginxplus` container: ```shell docker run --name mynginxplus -p 80:80 -d nginxplus ``` 8. Verify that the `mynginxplus` container is up and running with the `docker ps` command: ```shell $ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS ... eb7be9f439db nginxplus:latest "nginx -g 'daemon of 1 minute ago Up 15 seconds ... ... PORTS NAMES ... 0.0.0.0:80->80/tcp mynginxplus ``` NGINX Plus containers are controlled and managed in the same way as NGINX Open Source containers. ## Managing Content and Configuration Files Content served by NGINX and NGINX configuration files can be managed in several ways: - files are maintained on the Docker host - files are copied from the Docker host to a container - files are maintained in the container ### Maintaining Content and Configuration Files on the Docker Host When the container is created, you can mount a local directory on the Docker host to a directory in the container. The NGINX image uses the default NGINX configuration, which uses `/usr/share/nginx/html` as the container’s root directory and puts configuration files in `/etc/nginx`. For a Docker host with content in the local directory `/var/www` and configuration files in `/var/nginx/conf`, run the command: ```shell docker run --name mynginx2 \ --mount type=bind,source=/var/www,target=/usr/share/nginx/html,readonly \ --mount type=bind,source=/var/nginx/conf,target=/etc/nginx/conf,readonly \ -p 80:80 \ -d nginxplus ``` Any change made to the files in the local directories `/var/www and /var/nginx/conf` on the Docker host are reflected in the directories `/usr/share/nginx/html` and `/etc/nginx` in the container. The `readonly` option means these directories can be changed only on the Docker host, not from within the container. ### Copy content and configuration files from the Docker host Docker can copy the content and configuration files from a local directory on the Docker host during container creation. Once a container is created, the files are maintained by creating a new container when files change or by modifying the files in the container. A simple way to copy the files is to create a Dockerfile with commands that are run during generation of a new Docker image based on the NGINX image. For the file‑copy (COPY) commands in the Dockerfile, the local directory path is relative to the build context where the Dockerfile is located. Let's assume that the content directory is `content` and the directory for configuration files is `conf`, both subdirectories of the directory where the Dockerfile is located. The NGINX image has the default NGINX configuration files, including `default.conf`, in the `/etc/nginx/conf.d` directory. To use the configuration files from the Docker host only, delete the default files with the `RUN` command: ```dockerfile FROM nginx RUN rm /etc/nginx/conf.d/default.conf COPY content /usr/share/nginx/html COPY conf /etc/nginx ``` Create NGINX image by running the command from the directory where the Dockerfile is located. The period (“.”) at the end of the command defines the current directory as the build context, which contains the Dockerfile and the directories to be copied: ```shell docker build -t mynginx_image1 . ``` Create a container `mynginx3` based on the `mynginx_image1` image: ```shell docker run --name mynginx3 -p 80:80 -d mynginx_image1 ``` To make changes to the files in the container, use a helper container as described in the next section. ### Maintain content and configuration files in the container As SSH cannot be used to access the NGINX container, to edit the content or configuration files directly you need to create a helper container that has shell access. For the helper container to have access to the files, create a new image that has the proper Docker data volumes defined for the image: 1. Copy nginx content and configuration files and define the volume for the image with the Dockerfile: ```dockerfile FROM nginx COPY content /usr/share/nginx/html COPY conf /etc/nginx VOLUME /usr/share/nginx/html VOLUME /etc/nginx ``` 2. Create the new NGINX image by running the following command: ```shell docker build -t mynginx_image2 . ``` 3. Create an NGINX container `mynginx4` based on the `mynginx_image2` image: ```shell docker run --name mynginx4 -p 80:80 -d mynginx_image2 ``` 4. Start a helper container `mynginx4_files` that has a shell, providing access the content and configuration directories of the `mynginx4` container we just created: ```shell $ docker run -i -t --volumes-from mynginx4 --name mynginx4_files debian /bin/bash root@b1cbbad63dd1:/# ``` where: - the new `mynginx4_files` helper container runs in the foreground with a persistent standard input (the `-i` option) and a tty (the `-t` option). All volumes defined in `mynginx4` are mounted as local directories in the helper container. - the `debian` argument means that the helper container uses the Debian image from Docker Hub. Because the NGINX image also uses Debian, it is most efficient to use Debian for the helper container, rather than having Docker load another operating system - the `/bin/bash` argument means that the bash shell runs in the helper container, presenting a shell prompt that you can use to modify files as needed To start and stop the container, run the commands: ```shell docker start mynginx4_files docker stop mynginx4_files ``` To exit the shell but leave the container running, press `Ctrl+p` followed by `Ctrl+q`. To regain shell access to a running container, run this command: ```shell docker attach mynginx4_files ``` To exit the shell and terminate the container, run the `exit` command. ## Manage logging You can use default logging or customize logging. ### Use default logging By default, the NGINX image is configured to send NGINX [access log](https://nginx.org/en/docs/http/ngx_http_log_module.html#access_log) and [error log](https://nginx.org/en/docs/ngx_core_module.html#error_log) to the Docker log collector. This is done by linking them to `stdout` and `stderr`: all messages from both logs are then written to the file `/var/lib/docker/containers/container-ID/container-ID-json.log` on the Docker host. The container‑ID is the long‑form ID returned when you [create a container](#docker_oss_image). To display the long form ID, run the command: ```shell docker inspect --format '{{ .Id }}' container-name ``` You can use both the Docker command line and the Docker Engine API to extract the log messages. To extract log messages from the command line, run the command: ```shell docker logs container-name ``` To extract log messages using the Docker Remote API, send a `GET` request using the Docker Unix sock: ```shell curl --unix-sock /var/run/docker-sock http://localhost/containers/container-name/logs?stdout=1&stderr=1 ``` To include only access log messages in the output, include only `stdout=1`. To limit the output to error log messages, include only `stderr=1`. For other available options, see [Get container logs](https://docs.docker.com/reference/api/engine/version/v1.45/#tag/Container/operation/ContainerLogs) section of the [Docker Engine API](https://docs.docker.com/reference/api/engine/version/v1.45/) documentation. ### Use customized logging If you want to configure logging differently for certain configuration blocks (such as `server {}` and `location {}`), define a Docker volume for the directory in which to store the log files in the container, create a helper container to access the log files, and use any logging tools. To implement this, create a new image that contains the volume or volumes for the logging files. For example, to configure NGINX to store log files in `/var/log/nginx/log`, add a `VOLUME` definition for this directory to the Dockerfile (provided that content and configuration Files are [managed in the container](#manage_container)): ```dockerfile FROM nginx COPY content /usr/share/nginx/html COPY conf /etc/nginx VOLUME /var/log/nginx/log ``` Then you can [create an image](#docker_plus_image) and use it to create an NGINX container and a helper container that have access to the logging directory. The helper container can have any desired logging tools installed. ## Control NGINX Since there is no direct access to the command line of the NGINX container, NGINX commands cannot be sent to a container directly. Instead, [signals](https://nginx.org/en/docs/control.html) can be sent to a container via Docker `kill` command. To reload the NGINX configuration, send the `HUP` signal to Docker: ```shell docker kill -s HUP container-name ``` To restart NGINX, run this command to restart the container: ```shell docker restart container-name ``` -------------------------------------------------------------------------------- title: "gRPC Health Checks" url: https://docs.nginx.com/nginx/admin-guide/load-balancer/grpc-health-check/index.md description: Monitor the health of gRPC servers in an upstream group by sending periodic health checks. -------------------------------------------------------------------------------- ## Introduction F5 NGINX Plus can monitor the health of upstream servers by making active health checks. NGINX Plus R23 supports the [gRPC health checking protocol](https://github.com/grpc/grpc/blob/master/doc/health-checking.md#grpc-health-checking-protocol) so that upstream gRPC services can be tested for their ability to handle new requests. This is particularly important in dynamic and containerized environments. When adding a new instance of a gRPC service, it is important that requests are sent to the fully operating service. ## Prerequisites - You have configured an upstream group of servers that handles gRPC network traffic and specified a shared memory zone that keeps the state of these servers, for example: ```nginx stream { #... upstream grpc_backend { zone grpc_backend 64k; server 10.0.0.1:50051; server 10.0.0.2:50051; } #... } ``` - You have configured a server that routes gRPC requests to the upstream server group: ```nginx location /grpc { grpc_pass grpc://grpc_backend; } ``` ## gRPC Servers that accept health checking protocol If your gRPC services support the gRPC health checking protocol, specify the [`type=grpc`](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#health_check_grpc) parameter of the `health_check` directive: ```nginx location / { grpc_pass grpc://grpc_backend; health_check mandatory type=grpc; } ``` In this example, according to health checking protocol, the `Check` method of the `Health` service will be invoked, and the gRPC server that respond with `SERVING` are considered healthy. The `mandatory` parameter ensures that the health check must pass before traffic is sent on an instance, for example, when it is introduced or reloaded. Note that the `type=grpc` must be specified after all other `health_check` parameters. If there are several gRPC services exposed on each upstream server then the most significant service can be monitored by specifying the service name with the `grpc_service` parameter: ```nginx location / { grpc_pass grpc://grpc_backend; health_check mandatory type=grpc grpc_service=MyStatus; } ``` ## gRPC Servers that do not accept health checking protocol If your gRPC services do not implement the gRPC health checking protocol, it is still possible to perform health check by sending the `Check` method and expecting a particular [`status code`](https://github.com/grpc/grpc/blob/master/doc/statuscodes.md#status-codes-and-their-use-in-grpc). This will test that the upstream server is responding to gRPC requests. Specify the gRPC error response code with the `grpc_status` parameter: ```nginx location / { grpc_pass grpc://grpc_backend; health_check type=grpc grpc_status=12; # 12=unimplemented } ``` In this case the response with the gRPC status of `12` / `UNIMPLEMENTED` will be considered healthy. Note that the `type=grpc` parameter is not compatible with `uri` or `match` parameters of the `health_check` directive. The `type=grpc` parameter must be specified after all other directive parameters: `grpc_service` and `grpc_status` must follow `type=grpc`. ## See also - [Deploying NGINX Plus as an API Gateway, Part 3: Publishing gRPC Services](https://www.nginx.com/blog/deploying-nginx-plus-as-an-api-gateway-part-3-publishing-grpc-services/) -------------------------------------------------------------------------------- title: "HTTP CONNECT forward proxy" url: https://docs.nginx.com/nginx/admin-guide/web-server/http-connect-proxy/index.md description: HTTP traffic tunneling via the HTTP CONNECT method, enabling centralized egress control through a trusted NGINX Plus server. -------------------------------------------------------------------------------- In corporate networks, NGINX Plus R36 and later can be configured as a forward proxy server, facilitating client access to external resources. A forward proxy operates between internal clients and the global network, enabling centralized traffic control. Unlike a reverse proxy that protects servers, upstreams, or services, forward proxy serves client requests and regulates their access to the external resources. To enable this functionality, the HTTP `CONNECT` method is used to establish a secure tunnel between the client and the proxy server (NGINX Plus). This tunnel permits the transmission of HTTPS traffic and other protocols, such as SSH or FTP, through the proxy. ## Enable HTTP CONNECT proxy To enable the HTTP `CONNECT` forward proxy, add the [`tunnel_pass`](https://nginx.org/en/docs/http/ngx_http_tunnel_module.html#tunnel_pass) directive that turns on the forward proxy functionality within the `server` or `location` blocks. The `tunnel_pass` directive can be used without any parameters. The default value is [`$host`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_host):[`$request_port`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_request_port) variables: all requests are automatically forwarded to external resources based on the host and the port which we are trying to access. The `tunnel_pass` directive is a content handling directive (similar to `proxy_pass`), and if specified for the `server` block, will not be inherited by corresponding `location` blocks. If you have a location that implements any other NGINX Plus capabilities within the forward proxy, you will need to enable the `tunnel_pass` for this `location` as well. See [Disable other HTTP methods](#disable-other-http-methods). ```nginx server { listen 10.10.1.11:3128; tunnel_pass; } ``` In this basic configuration, the client establishes a tunnel with NGINX Plus, sends the HTTP `CONNECT` method to the specified port `3128` which is standard for forward proxying, and if successful, responds with `200 OK`. The tunnel is established and the client can perform the TLS handshake and get the response. ## Disable other HTTP methods As soon as the `CONNECT` method is enabled, all other methods such as `GET`, `POST` are not applicable in this mode. Rejecting them ensures that only tunnel requests are allowed thus improving security. You can create a rule with the [`if`](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#if) directive to deny all HTTP methods except `CONNECT`, If a client uses any other method, NGINX Plus returns the `403` error code: ```nginx server { listen 10.10.1.11:3128; # Handle other methods location / { if ($request_method != CONNECT) { return 403 "Forbidden: allows only CONNECT method"; } # allow CONNECT requests tunnel_pass; } } ``` Note that in the example the [`tunnel_pass`](https://nginx.org/en/docs/http/ngx_http_tunnel_module.html#tunnel_pass) directive is specified inside the `location` block. Since `tunnel_pass` is a content handling directive, when it is specified for the `server` block, it is not inherited by nested locations. ## Enable logging For testing, debugging and monitoring purposes you can configure error and access logs. 1. On the `http` level, specify the [`log_format`](https://nginx.org/en/docs/http/ngx_http_log_module.html#log_format) directive, give the log a name (for example, `http_connect`), and include the metrics: ```nginx log_format http_connect '$remote_addr [$time_local] ' '$request_method $host:$request_port $status ' '$upstream_addr $bytes_sent $upstream_connect_time ' '$request_time "$http_user_agent"'; ``` See the [NGINX variables index](https://nginx.org/en/docs/varindex.html) for the list of all supported variables. 2. Specify the [`error_log`](https://nginx.org/en/docs/ngx_core_module.html#error_log) and [`access_log`](https://nginx.org/en/docs/http/ngx_http_log_module.html#access_log) directives on the same configuration level where the [`tunnel_pass`](https://nginx.org/en/docs/http/ngx_http_tunnel_module.html#tunnel_pass) directive is defined: ```nginx access_log logs/connect_access.log http_connect; error_log logs/connect_error.log; ``` ## Test the configuration This example sums up the steps above: it enables logging for the `location` block which denies all HTTP methods except `CONNECT` and where the `tunnel_pass` directive is specified: ```nginx log_format http_connect '$remote_addr [$time_local] ' '$request_method $host:$request_port $status ' '$upstream_addr $bytes_sent $upstream_connect_time ' '$request_time "$http_user_agent"'; server { listen 10.10.1.11:3128; location / { if ($request_method != CONNECT) { return 403 "Forbidden: allows only CONNECT method"; } tunnel_pass; access_log logs/connect_access.log http_connect; error_log logs/connect_error.log; } } ``` 1. Test NGINX configuration for syntax validity and apply the changes: ```nginx nginx -t && nginx -s reload ``` 2. Send a test request with the `curl` command: ```shell curl -v -x 10.10.1.11:3128 https://example.com ``` 3. Check the access log to see the result of the command: ```shell tail -n 100 /logs/connect_access.log ``` The result of the command: ```none logs > connect_access.log 1 10.10.1.240 [01/Dec/2025:10:46:27 +0000] CONNECT example.com:443 200 127.456.789.0:443 4275 0.014 0.258 "curl/8.7.1" ``` The result shows that the client (`10.10.1.240`) successfully established a connection (`200`) to the destination server (`example.com:443` resolved to `127.456.789.0:443`). ## Access control It is highly recommended to restrict access to the proxy servers. Access control can be managed in several ways: - by ports and port ranges with the [`num_map`](https://nginx.org/en/docs/http/ngx_http_num_map_module.html) module (NGINX Plus R36) - by IP addresses with the [`geo`](https://nginx.org/en/docs/http/ngx_http_geo_module.html) module - by hostnames with the [`map`](https://nginx.org/en/docs/http/ngx_http_map_module.html) module All of these access methods can be used together and combined with the the [`proxy_allow_upstream`](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_allow_upstream) directive (NGINX Plus R36). ### Restrict by ports and port ranges The [`num_map`](https://nginx.org/en/docs/http/ngx_http_num_map_module.html) module introduced in NGINX Plus R36 allows defining ports and port ranges and uses the same approach as the `map` block. For example, if the value of the [`$request_port`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_request_port) variable is `443`, the `$allowed_port` variable is assigned the value `ssl`. It also restricts the allowed port range for the upstream connections or proxy requests: ```nginx num_map $request_port $allowed_port { default 0; 443 ssl; <=1023 less-eq; 8080-8090 range; >8092 more; } ``` ### Restrict by IP addresses Access by IP addresses can be restricted with the [`geo`](https://nginx.org/en/docs/http/ngx_http_geo_module.html) module: ```nginx geo $allowed_networks { 10.10.1.0/24 allow; 10.20.11.0/24 allow; default deny; } ``` ### Restrict by hostnames Restriction by hostnames can be configured with the [`map`](https://nginx.org/en/docs/http/ngx_http_map_module.html) module, which works mainly with strings and regular expressions. Although it is possible to match port numbers using `map`, it is recommended to use the [`num_map`](https://nginx.org/en/docs/http/ngx_http_num_map_module.html) module for more port-based restrictions, as it provides additional functionality such as defining ranges. In this example, access is allowed to `example.com` and its subdomains only: ```nginx map $host $allowed_host { hostnames; default 0; example.com 1; *.example.com 1; } ``` ### Combine access control methods These access methods can be combined in the [`tunnel_allow_upstream`](https://nginx.org/en/docs/http/ngx_http_tunnel_module.html#tunnel_allow_upstream) directive, which performs access checks when the upstream server to which the request will be sent is selected. If any variable passed to [`tunnel_allow_upstream`](https://nginx.org/en/docs/http/ngx_http_tunnel_module.html#tunnel_allow_upstream) evaluates to non-zero or non-empty value, it will be calculated as `1` and access is granted. If all variables evaluate to `0` or an empty value, access is denied. ### Access control example ```nginx log_format connect_debug '$remote_addr [$time_local] ' '$request_method $host:$request_port $status ' '$upstream_addr $bytes_sent $upstream_connect_time ' '$request_time "$http_user_agent"'; num_map $request_port $allowed_port { default 0; 443 ssl; <=1023 less-eq; 8080-8090 range; >8092 more; } geo $allowed_networks { 10.10.1.0/24 allow; 10.20.11.0/24 allow; default deny; } map $host $allowed_host { hostnames; default 0; example.com 1; *.example.com 1; } server { listen 10.10.1.11:3128; #Allow CONNECT only to certain ports/nets/hosts tunnel_allow_upstream $allowed_networks $allowed_port $allowed_host; error_page 403; satisfy all; allow 10.10.0.0/16; allow 127.0.0.1; deny all; tunnel_pass; location ^~ /errors/ { internal; root html; allow all; } access_log logs/connect_access.log connect_debug; error_log logs/connect_debug.log debug; } ``` Access can also be limited using other modules, for example with the [`satisfy`](https://nginx.org/en/docs/http/ngx_http_core_module.html#satisfy) directive, or the [`allow`/`deny`](https://nginx.org/en/docs/http/ngx_http_access_module.html#allow) directives. ## mTLS authentication Mutual TLS (mTLS) ensures that both the client and server authenticate each other by exchanging and validating trusted certificates. Once the authentication succeeds, a secure, encrypted connection is established. For an HTTP CONNECT proxy, mutual TLS (mTLS) is currently the only supported authentication method. Other methods, such as [basic authentication](https://nginx.org/en/docs/http/ngx_http_auth_basic_module.html) are currently not supported because NGINX Plus processes only the `Authorization` header, not the `Proxy-Authorization` header. To enable mTLS, you need: - a server certificate and private key, used by NGINX Plus to authenticate itself to clients - a client Certificate Authority (CA) certificate, used by NGINX Plus to verify client certificates - one or more client certificates, signed by the trusted client CA, used by clients to authenticate to the proxy - NGINX Plus configuration that enables TLS and enforces client certificates verification Certificates can be created using tools such as OpenSSL or obtained from certificate providers such as Let's Encrypt. ### Configure mTLS In your NGINX Plus configuration file: 1. Enable the TLS layer on proxy listener by adding the `ssl` parameter to the `listen` directive, port `3128` is a commonly used port for forward proxies: ```nginx listen 10.10.1.11:3128 ssl; #... ``` 2. Specify the proxy server SSL certificate and private key (the certificate that NGINX Plus presents to clients): ```nginx #... ssl_certificate /etc/ssl/certs/forward_proxy_server.crt; ssl_certificate_key /etc/ssl/certs/forward_proxy_server.key; #... ``` 3. Configure client certificate verification. Specify the trusted Client CA certificate used to validate client certificates with the [`ssl_client_certificate`](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_client_certificate) directive. With [`ssl_verify_client`](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_verify_client) set to on, every client is required to present a valid certificate during the TLS handshake, otherwise the connection will be rejected: ```nginx #... ssl_client_certificate /etc/ssl/certs/forward_proxy_client_ca.crt; ssl_verify_client on; ssl_verify_depth 1; ssl_protocols TLSv1.2 TLSv1.3; ssl_prefer_server_ciphers on; #... ``` ### Conditional forward proxy access based on mTLS Based on the result of client certificate verification, you can implement different access policies. This scenario enables conditional access: if a client provides a valid certificate, access is granted to all resources through the proxy. If not, access is limited to particular resources or blocked. 1. Set the [`ssl_verify_client`](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_verify_client) directive value to `optional` to allow clients to connect with or without a client certificate, making conditional access possible: ```nginx #... ssl_client_certificate /etc/ssl/certs/forward_proxy_client_ca.crt; ssl_verify_client optional; #... ``` 2. Verify the status of the client certificate using the [`map`](https://nginx.org/en/docs/http/ngx_http_map_module.html#map) block and the [`$ssl_client_verify`](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#var_ssl_client_verify) variable. The result will be saved to the `$ssl_ok` variable: ```nginx map $ssl_client_verify $ssl_ok { default 0; SUCCESS 1; } ``` In the `map` block, the value of the `$ssl_ok` variable depends on the status of [`$ssl_client_verify`](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#var_ssl_client_verify) variable. If the result of client certificate verification is “SUCCESS”, the `$ssl_ok` variable gets the value `1`. 3. Check the hostname the client is going to access based on the `Host` header of the request (the [$host](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_host) variable). Access will be allowed only for `example.com` and its subdomains. The `hostnames` parameter of the [`map`](https://nginx.org/en/docs/http/ngx_http_map_module.html#map) directive is required to recognize hostnames and wildcards in the hostnames: ```nginx map $host $allowed_host { hostnames; default 0; example.com 1; *.example.com 1; } ``` 4. Define access rules by combining the results of client certificate verification (`$ssl_ok`) and hostname validation (`$allowed_host`). To allow access to all websites, the client must present a valid client certificate. If no client certificate is provided, access is restricted to `example.com` and its subdomains. The result is stored in the `$connect_ok` variable: ```nginx map "$ssl_ok:$allowed_host" $connect_ok { default 0; "1:0" 1; "1:1" 1; "0:1" 1; } ``` 5. Restrict upstream connections to valid `CONNECT` requests using the [`tunnel_allow_upstream`](https://nginx.org/en/docs/http/ngx_http_tunnel_module.html#tunnel_allow_upstream) directive based on the result in the `$connect_ok` variable: ```nginx tunnel_allow_upstream $connect_ok; tunnel_pass; ``` ### Complete example This configuration sums up the steps and enables authorization based on client certificate verification result. Clients with valid client certificates are granted full access through the proxy. Clients without a certificate are restricted to `example.com` and its subdomains, all other `CONNECT` requests are denied. ```nginx log_format http_connect '$remote_addr [$time_local] ' '$request_method $host:$request_port $status ' '$upstream_addr $bytes_sent $upstream_connect_time ' '$request_time "$http_user_agent"'; map $ssl_client_verify $ssl_ok { default 0; SUCCESS 1; } map $host $allowed_host { hostnames; default 0; example.com 1; *.example.com 1; } map "$ssl_ok:$allowed_host" $connect_ok { default 0; "1:0" 1; "1:1" 1; "0:1" 1; } server { listen 10.10.1.11:3128 ssl; ssl_certificate /etc/ssl/certs/forward_proxy_server.crt; ssl_certificate_key /etc/ssl/certs/forward_proxy_server.key; ssl_client_certificate /etc/ssl/certs/forward_proxy_client_ca.crt; ssl_verify_client optional; ssl_verify_depth 1; ssl_protocols TLSv1.2 TLSv1.3; ssl_prefer_server_ciphers on; tunnel_allow_upstream $connect_ok; tunnel_pass; access_log logs/connect_access.log http_connect; error_log logs/connect_error.log; } ``` The first `map` block checks if the client passed the mTLS verification. It sets the `$ssl_ok` variable to `1` if NGINX Plus successfully verified the client certificate during the TLS handshake, and to `0` otherwise. The second `map` block checks if the requested host is allowlisted: if the `CONNECT` target hostname is `example.com` or any of its subdomains, the `$allowed_host` variable is set to `1` and to `0` otherwise. The third `map` block builds the final decision variable, `$connect_ok`. Its result will be evaluated by the `tunnel_allow_upstream` directive: if it has a non-empty value, access will be granted, with restrictions above implemented. If the client cert is valid (`$ssl_ok`=`1`), access is allowed regardless of host. If the client cert is not valid (`$ssl_ok`=`0`), access is allowed only if the host is allowlisted. Otherwise access is denied (default is `0`). The [`ssl_verify_client`](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_verify_client) directive is set to `optional` which allows clients to connect with or without client certificate, making conditional access possible. #### Testing 1. Test the nginx configuration for syntax validity and apply the configuration: ```nginx nginx -t && \ nginx -s reload ``` 2. Test the proxy authentication layer for requests to verify that the proxy is protected as expected under different scenarios. - Client certificate is not provided: ```shell curl -v -x https://myproxy:3128 \ https://www.example.com \ --proxy-cacert ./conf/tls/ca.crt ``` where: - `myproxy:3128` is the proxy address and port - `https://www.example.com` is the destination URL requested through the proxy - `--proxy-cacert ./conf/tls/ca.crt` - the CA used to verify the proxy’s server certificate. Expected result: requests to `example.com` and its subdomains succeed (for example, `301 Moved Permanently`), requests to other domains are denied (for example, `502 Bad Gateway`). - Correct client certificate is provided (the `--proxy-cert` and `--proxy-key` parameters): ```shell curl -v -x https://myproxy:3128 \ https://www.example.com \ --proxy-cert ./conf/tls/client.crt \ --proxy-key ./conf/tls/client.key \ --proxy-cacert ./conf/tls/ca.crt ``` For requests to all resources, the request should succeed (for example, `301 Moved Permanently`). ## See also - [Secure and harden forward proxies in NGINX Plus](https://community.f5.com/kb/technicalarticles/secure-and-harden-forward-proxies-in-nginx-plus/344989) - [F5 NGINX Plus R36 release blogpost](https://community.f5.com/kb/technicalarticles/f5-nginx-plus-r36-release-now-available/344514) - [`ngx_http_tunnel_module` module reference](https://nginx.org/en/docs/http/ngx_http_tunnel_module.html) -------------------------------------------------------------------------------- title: "Limiting Access to Proxied HTTP Resources" url: https://docs.nginx.com/nginx/admin-guide/security-controls/controlling-access-proxied-http/index.md description: Protect your upstream web and application servers by limiting connections, rate of requests, or bandwidth, based on client IP address or other variables. -------------------------------------------------------------------------------- This article explains how to set the maximum number of requests for a connection, or the maximum rate of downloading content from the server. ## Introduction Using NGINX and F5 NGINX Plus, it is possible to limit: - The number of connections per key value (for example, per IP address) - The request rate per key value (the number of requests that are allowed to be processed during a second or minute) - The download speed for a connection Note that IP addresses can be shared behind NAT devices, so limiting by IP address should be used judiciously. ## Limiting the Number of Connections To limit the number of connections: 1. Use the [limit_conn_zone](https://nginx.org/en/docs/http/ngx_http_limit_conn_module.html#limit_conn_zone) directive to define the key and set the parameters of the shared memory zone (the worker processes will use this zone to share counters for key values). As the first parameter, specify the expression evaluated as a key. In the second parameter `zone`, specify the name of the zone and its size: ```nginx limit_conn_zone $binary_remote_addr zone=addr:10m; ``` 2. Use the [limit_conn](https://nginx.org/en/docs/http/ngx_http_limit_conn_module.html#limit_conn) directive to apply the limit within the `location {}`, `server {}`, or `http {}` context. Specify the name of the shared memory zone as the first parameter, and the number of allowed connection per key as the second parameter: ```nginx location /download/ { limit_conn addr 1; } ``` The number of connections is limited on an IP address basis because the [`$binary_remote_addr`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_binary_remote_addr) variable is used as a key. Another way to limit the number of connections for a given server is by using the [`$server_name`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_server_name) variable: ```nginx http { limit_conn_zone $server_name zone=servers:10m; server { limit_conn servers 1000; } } ``` ## Limiting the Request Rate Rate limiting can be used to prevent DDoS attacks, or prevent upstream servers from being overwhelmed by too many requests at the same time. The method is based on the [`leaky bucket`](https://en.wikipedia.org/wiki/Leaky_bucket) algorithm: requests arrive at the bucket at various rates and leave the bucket at fixed rate. Before using rate limiting, you will need to configure global parameters of the "leaky bucket": - key - a parameter used to differentiate one client from another, generally a variable - shared memory zone - the name and size of the zone that keeps states of these keys (the "leaky bucket") - rate - the request rate limit specified in requests per second (`r/s`) or requests per minute (`r/m`) ("leaky bucket draining"). Requests per minute are used to specify a rate less than one request per second. These parameters are set with the [limit_req_zone](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_zone) directive. The directive is defined on the `http {}` level - such approach allows applying different zones and request overflow parameters to different contexts: ```nginx http { #... limit_req_zone $binary_remote_addr zone=one:10m rate=1r/s; } ``` With this configuration, the shared memory zone `one` with the size of 10 megabytes is created. The zone keeps states of client IP addresses set with the [`$binary_remote_addr`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_binary_remote_addr) variable. Note that in comparison to [`$remote_addr`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_remote_addr) which also holds a client’s IP address, [`$binary_remote_addr`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_binary_remote_addr) holds the binary representation of IP address which is shorter. The optimal size of the shared memory zone can be counted using the following data: the size of [`$binary_remote_addr`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_binary_remote_addr) value is 4 bytes for IPv4 addresses, stored state occupies 128 bytes on 64-bit platforms. Thus, state information for about 16,000 IP addresses occupies 1 megabyte of the zone. If storage is exhausted when NGINX needs to add a new entry, it removes the oldest entry. If the space freed is still not enough to accommodate the new record, NGINX returns status code `503 Service Unavailable`. The status code can be redefined with the [limit_req_status](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_status) directive. Once the zone is set, you can use requests limiting anywhere in the NGINX configuration with the [limit_req](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req) specified for a `server {}`, `location {}`, or `http {}` context: ```nginx http { #... limit_req_zone $binary_remote_addr zone=one:10m rate=1r/s; server { #... location /search/ { limit_req zone=one; } } } ``` With this configuration, NGINX will process no more than `1` request per second within the `/search/` location. Processing of these requests is delayed in such a way that the overall rate is not greater than specified. If the number of requests exceeds the specified rate, NGINX will delay processing of such requests until the "bucket" (shared memory zone `one`) is full. For requests that arrive at the full bucket, NGINX will respond with the `503 Service Unavailable` error (if not redefined with [limit_req_status](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_status)). ### Testing the Request Rate Limit Before configuring real-life rate limiting, you can try the “dry run” mode that does not limit the requests processing rate. However, such excessive requests are still accounted in the shared memory zone and logged. The “dry run” mode can be enabled with the [limit_req_dry_run](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_dry_run) directive: ```nginx http { #... limit_req_zone $binary_remote_addr zone=one:10m rate=1r/s; server { #... location /search/ { limit_req zone=one; limit_req_dry_run on; } } } ``` Every request that would exceed the defined rate limit will be logged with the “dry run” mark: ```none 2019/09/03 10:28:45 [error] 142#142: *13246 limiting requests, dry run, excess: 1.000 by zone "one", client: 172.19.0.1, server: www.example.com, request: "GET / HTTP/1.0", host: "www.example.com:80" ``` ### Handling Excessive Requests Requests are limited to fit the rate defined in the [limit_req_zone](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_zone) directive. If the number of requests exceeds the specified rate and the shared memory zone becomes full, NGINX will respond with an error. As traffic tends to be bursty, returning the error in response to a client request during traffic bursts is not the best case. Such excessive requests in NGINX can be buffered and processed. The `burst` parameter of the [limit_req](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req) directive sets the maximum number of excessive requests that await to be processed at the specified rate: ```nginx http { #... limit_req_zone $binary_remote_addr zone=one:10m rate=1r/s; server { #... location /search/ { limit_req zone=one burst=5; } } } ``` With this configuration, if request rate exceeds `1` request per second, requests beyond the rate will be put into the zone `one`. When the zone is full, excessive requests will be queued (`burst`), the size of this queue is `5` requests. Request processing in the queue is delayed in such a way that the overall rate is not greater than specified. Requests above the burst limit will be rejected with the `503` error. If delaying of request is not desired during traffic burst, add the `nodelay` parameter: ```nginx http { #... limit_req_zone $binary_remote_addr zone=one:10m rate=1r/s; server { #... location /search/ { limit_req zone=one burst=5 nodelay; } } } ``` With this configuration, excessive requests within the `burst` limit will be served immediately regardless of the specified `rate`, requests above the burst limit will be rejected with the `503` error. ### Delaying Excessive Requests Another way to handle excessive requests is to serve some number of these requests without delay, then apply rate limiting up to the point when excessive requests will be rejected. This can be achieved with the `delay` and `burst` parameters. The `delay` parameter defines the point at which excessive requests are delayed to comply with the defined rate limit: ```nginx http { #... limit_req_zone $binary_remote_addr zone=one:10m rate=1r/s; server { #... location /search/ { limit_req zone=one burst=5 delay=3; } } } ``` With this configuration, first 3 requests (`delay`) are passed without delay, next 2 requests (`burst` - `delay`) are delayed in such a way that the overall rate is not greater than specified, further excessive requests will be rejected because the total burst size has been exceeded, subsequent requests will be delayed. ### Synchronizing Contents of Many Shared Memory Zones If you have a computer cluster with several NGINX instances and these instances use the `limit_req` method, it is possible to sync the contents of their shared memory zones on conditions that: - the [zone_sync](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync) functionality is configured for each instance - shared memory zones set in the [limit_req_zone](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_zone) directive for each instance have the same name - the [sync](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_zone_sync) parameter of the [limit_req_zone](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_zone) directive is specified for each instance: ```nginx http { #... limit_req_zone $binary_remote_addr zone=one:10m rate=1r/s sync; } ``` See [Runtime State Sharing in a Cluster](nginx/admin-guide/high-availability/zone_sync.md) for details. ## Limiting the Bandwidth To limit the bandwidth per connection, use the [limit_rate](https://nginx.org/en/docs/http/ngx_http_core_module.html#limit_rate) directive: ```nginx location /download/ { limit_rate 50k; } ``` With this setting a client will be able to download content through a single connection at a maximum speed of `50` kilobytes per second. However, the client can open several connections. So if the goal is to prevent a speed of downloading greater than the specified value, the number of connections should also be limited. For example, one connection per IP address (if the shared memory zone specified above is used): ```nginx location /download/ { limit_conn addr 1; limit_rate 50k; } ``` To impose the limit only after the client downloads a certain amount of data, use the [limit_rate_after](https://nginx.org/en/docs/http/ngx_http_core_module.html#limit_rate_after) directive. It may be reasonable to allow a client to quickly download a certain amount of data (for example, a file header — film index) and limit the rate for downloading the rest of the data (to make users watch a film, not download). ```nginx limit_rate_after 500k; limit_rate 20k; ``` The following example shows the combined configuration for limiting the number of connections and the bandwidth. The maximum allowed number of connections is set to `5` connections per client address, which fits most common cases since modern browsers typically open up to 3 connections at a time. Meanwhile the location that serves downloads allows only one connection: ```nginx http { limit_conn_zone $binary_remote_addr zone=addr:10m; server { root /www/data; limit_conn addr 5; location / { } location /download/ { limit_conn addr 1; limit_rate_after 1m; limit_rate 50k; } } } ``` ### Dynamic Bandwidth Control The [limit_rate](https://nginx.org/en/docs/http/ngx_http_core_module.html#limit_rate) value can also be specified as a variable - this enables dynamic bandwidth use cases, for example, allow a higher bandwidth limit to modern browsers: ```nginx map $ssl_protocol $response_rate { "TLSv1.1" 10k; "TLSv1.2" 100k; "TLSv1.3" 1000k; } server { listen 443 ssl; ssl_protocols TLSv1.1 TLSv1.2 TLSv1.3; ssl_certificate www.example.com.crt; ssl_certificate_key www.example.com.key; location / { limit_rate $response_rate; # Limit bandwidth based on TLS version limit_rate_after 512; # Apply limit after headers have been sent proxy_pass http://my_backend; } } ``` ## See Also - [Rate Limiting with NGINX and NGINX Plus](https://www.nginx.com/blog/rate-limiting-nginx/) -------------------------------------------------------------------------------- title: "NGINX Plus FIPS Compliance" url: https://docs.nginx.com/nginx/fips-compliance-nginx-plus/index.md -------------------------------------------------------------------------------- ## What is FIPS The Federal Information Processing Standard (FIPS), issued by the [U.S. National Institute of Standards and Technology](https://www.nist.gov/) (NIST), defines mandatory security requirements for cryptographic modules used in federal IT systems. [FIPS 140-2](https://csrc.nist.gov/pubs/fips/140-2/upd2/final), and its successor [FIPS 140-3](https://csrc.nist.gov/pubs/fips/140-3/final), establish strict standards to protect sensitive information, including government communications and citizen data. ## Why FIPS-140 matters FIPS 140 is a mandatory cryptographic standard in the United States and Canada for federal agencies, their contractors, and many regulated industries. Non-compliance can result to contract loss, restricted project access, fines, or, in severe cases, data breaches compromising personal information or national security. Some industries such as finance, healthcare, energy, also adopt FIPS to enhance data protection and operational security. ### FIPS compliance in U.S. Currently, both FIPS 140-2 and FIPS 140-3 certifications are accepted. However, FIPS 140-2 is being phased out as part of the [FIPS 140-3 transition plan](https://csrc.nist.gov/projects/fips-140-3-transition-effort). After September 22, 2026, only FIPS 140-3 certifications will be recognized. Organizations are encouraged to migrate to FIPS 140-3 to meet updated cryptographic security requirements. | **Sector / Program** | **Version** | **Status** | |--------------------------------|----------------|---------------| | **Federal Programs** | | | | CJIS | 140-2 or 140-3 | Mandatory | | FedRAMP | 140-2 or 140-3 | Mandatory | | FISMA | 140-2 or 140-3 | Mandatory | | DFARS | 140-2 or 140-3 | Mandatory | | DoDIN APL | 140-2 or 140-3 | Mandatory | | FAA | 140-2 to 140-3 | Transitioning | | TSA | 140-2 or 140-3 | Recommended | | **Defense & Intelligence** | | | | CMMC | 140-2 or 140-3 | Mandatory | | Intelligence Community | 140-2 to 140-3 | Transitioning | | NSA CSfC | 140-2 to 140-3 | Transitioning | | Military & Tactical Systems | 140-2 to 140-3 | Transitioning | | **Healthcare & Education** | | | | HIPAA | 140-2 or 140-3 | Mandatory | | HITECH | 140-2 or 140-3 | Mandatory | | Department of Veterans Affairs | 140-2 or 140-3 | Mandatory | | FERPA | 140-2 or 140-3 | Recommended | | **Commercial/Private Sector** | | | | PCI DSS | 140-2 or 140-3 | Recommended | | Common Criteria | 140-2 or 140-3 | Recommended | | **Infrastructure & Critical Systems** | | | | Critical Infrastructure | 140-2 or 140-3 | Recommended | | Nuclear Regulatory Commission | 140-2 or 140-3 | Recommended | | **State & Local Government** | | | | State and Local Gov Programs | 140-2 or 140-3 | Mandatory | ### FIPS compliance in other countries Although FIPS 140 is primarily a North American government cryptographic standard, it is widely recognized as a global benchmark for cryptographic security. Numerous countries outside North America align their cryptographic requirements with FIPS, especially in regulated sectors such as finance, defense, healthcare, and critical infrastructure. | Country/Region | FIPS Use | |----------------|-----------------------------------------------------------------------------| | Australia | Referenced for government, defense, and cryptography systems. | | Canada | Mandatory for federal and sensitive systems. | | Denmark | Referenced in finance, healthcare, and NATO communications. | | Estonia | Adopted for e-governance and critical systems. | | Finland | Relied on for defense and NATO communications. | | France | Relied on for defense and secure systems. | | Germany | Relied on for defense, critical infrastructure, and NATO communications. | | Israel | Trusted in defense, government, and financial systems. | | Italy | Relied on for defense and financial cryptography. | | Japan | Referenced in government and financial cryptographic practices. | | Netherlands | Referenced in finance, healthcare, and NATO communications. | | New Zealand | Referenced for government and national cryptography. | | Poland | Relied on for secure government and NATO communications. | | Spain | Referenced in NATO communications and critical systems. | | Sweden | Relied on for defense and secure NATO communications. | | UAE | Trusted in finance, energy, and interoperability with the U.S. cryptography.| | United Kingdom | Referenced for defense, health, and procurement standards. | | United States | Mandatory for federal government systems and contractors. | ## FIPS compliant vs FIPS validated FIPS validation is a formal multistep process that certifies cryptographic modules through testing under the [Cryptographic Module Validation Program](https://csrc.nist.gov/Projects/cryptographic-module-validation-program/cmvp-flow) (CMVP). The process is managed by the [NIST](https://csrc.nist.gov/) and requires accredited third-party laboratories to evaluate the cryptographic module. Once a module passes validation, it is officially recognized as FIPS-validated (FIPS-certified). FIPS compliance indicates that a system or a module claims to meet the FIPS requirements, however, it has not been officially tested or certified under the CMVP program. ## FIPS compliance with NGINX Plus NGINX Plus is **FIPS 140-2 Level 1** and **FIPS 140-3 Level 1 compliant**, provided that the operating system and the OpenSSL library are operating in FIPS mode. NGINX Plus relies exclusively on the operating system’s FIPS-validated OpenSSL library for all SSL/TLS, HTTP/2, and HTTP/3 encryption and decryption operations. ## FIPS compliance with NGINX Open Source While NGINX Plus is tested to work on FIPS-enabled operating systems in FIPS mode, NGINX Open Source is not verified for such environments, especially when third-party builds or modules implementing custom cryptographic functions are used. Compiling NGINX Open Source for FIPS mode may also require additional OS-level dependencies beyond its core requirements, potentially introducing unintended risks. Organizations should consult their security and compliance teams to ensure their configurations meet FIPS requirements. ## FIPS validation of operating systems Several operating system vendors have obtained FIPS 140-2 Level 1 and 140-3 Level 1 validation for the OpenSSL Cryptographic Module included with their respective operating systems: - RedHat: [RedHat FIPS Certifications](https://access.redhat.com/compliance/fips) - Ubuntu: [Overview of FIPS-certified modules](https://documentation.ubuntu.com/security/docs/compliance/fips/fips-overview/) - Oracle: [Oracle FIPS Certifications](https://www.oracle.com/corporate/security-practices/assurance/development/external-security-evaluations/fips/certifications/) - SUSE: [SUSE FIPS 140-3 cryptographic certificates](https://www.suse.com/c/suse-has-received-first-fips-140-3-cryptographic-certificates/) - AWS: [FIPS 140-3 Compliance](https://aws.amazon.com/compliance/fips/) - Amazon Linux: [Achieving FIPS 140-3 validation](https://aws.amazon.com/blogs/compute/amazon-linux-2023-achieves-fips-140-3-validation/) You also can verify whether your operating system or cryptographic module is FIPS-validated using the [NIST database search tool](https://csrc.nist.gov/Projects/cryptographic-module-validation-program/validated-modules/search). ## FIPS validation of OpenSSL OpenSSL 3.0 and later versions introduced a FIPS provider that enables cryptographic operations in a FIPS-compliant mode. FIPS 140-3 validation: starting with OpenSSL 3.1.2, the library has [achieved FIPS 140-3 validation](https://openssl-library.org/post/2025-03-11-fips-140-3/) under certification [#4985](https://csrc.nist.gov/projects/cryptographic-module-validation-program/certificate/4985). FIPS 140-2 validation: the FIPS provider for OpenSSL 3.0.x has been validated for FIPS 140-2 under certifications [#4811](https://csrc.nist.gov/projects/cryptographic-module-validation-program/certificate/4811) and [#4282](https://csrc.nist.gov/projects/cryptographic-module-validation-program/certificate/4282). ## Verification of correct operation of NGINX Plus The following process describes how to deploy NGINX Plus in a FIPS‑compliant environment and verify that the FIPS operations are functioning correctly. It involves three basic steps: - [Verify](#os-fips-check) if the operating system is running in FIPS mode. If not, [configure](#os-fips-setup) it to enable FIPS mode. - [Verify](#openssl-fips-check) that the OpenSSL library is operating in FIPS mode. - Run basic checks for [OpenSSL](#openssl-fips-check) and [NGINX Plus](#nginx-plus-fips-check) to confirm deployment in FIPS mode. The process uses Red Hat Enterprise Linux (RHEL) release 9.6 as an example and can be adapted for other Linux operating systems that can be configured in FIPS mode. ### Step 1: Configure the operating system to use FIPS mode {#os-fips-setup} For the purposes of the following demonstration, we installed and configured a RHEL 9.6 server. The [Red Hat FIPS documentation](https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/9/html/security_hardening/switching-rhel-to-fips-mode_security-hardening) explains how to switch the operating system between FIPS mode and non‑FIPS mode by editing the boot options and restarting the system. For instructions for enabling FIPS mode on other FIPS‑compliant Linux operating systems, see the operating system documentation, for example: - RHEL 9: [Switching to FIPS mode](https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/9/html/security_hardening/switching-rhel-to-fips-mode_security-hardening) - Ubuntu: [Switching to FIPS mode](https://documentation.ubuntu.com/security/docs/compliance/fips/how-to-install-ubuntu-with-fips/) - SLES: [How to enable FIPS](https://www.suse.com/support/kb/doc/?id=000019432) - Oracle Linux 9: [Configuring FIPS mode](https://docs.oracle.com/en/operating-systems/oracle-linux/9/security/configuring_fips_mode.html#configuring-fips-mode) - Amazon Linux 2023: [Enabling FIPS mode](https://docs.aws.amazon.com/linux/al2023/ug/fips-mode.html) - Amazon Linux 2: [Enabling FIPS mode](https://docs.aws.amazon.com/linux/al2/ug/fips-mode.html) - AlmaLinux: [FIPS Validation for AlmaLinux](https://almalinux.org/blog/2023-09-19-fips-validation-for-almalinux/) ### Step 2: Verify the operating system is in FIPS mode {#os-fips-check} You can verify that the operating system is in FIPS mode and that the version of OpenSSL provided by the operating system vendor is FIPS‑compliant by using the following tests. **Check operating system flags**: When the operating system is in FIPS mode, `crypto.fips_enabled` kernlel flag is `1`; otherwise, it is `0`: ```shell sudo sysctl -a | grep fips ``` The output of the command shows that FIPS is enabled at the kernel level: ```none crypto.fips_enabled = 1 crypto.fips_name = Red Hat Enterprise Linux 9 - Kernel Cryptographic API crypto.fips_version = 5.14.0-570.39.1.el9_6.aarch64 ``` Check kernel logs for FIPS algorithm registration: ```shell journalctl -k -o cat -g alg: ``` The output of the command verifies the status of algorithm self-tests and whether certain algorithms are registered, passed FIPS self-tests, or are disabled due to FIPS mode being active: ```none alg: self-tests for pkcs1pad(rsa-generic,sha512) (pkcs1pad(rsa,sha512)) passed alg: self-tests for pkcs1pad(rsa-generic,sha256) (pkcs1pad(rsa,sha256)) passed alg: self-tests for cbc-aes-ce (cbc(aes)) passed alg: self-tests for jitterentropy_rng (jitterentropy_rng) passed alg: poly1305 (poly1305-neon) is disabled due to FIPS alg: xchacha12 (xchacha12-neon) is disabled due to FIPS alg: xchacha20 (xchacha20-neon) is disabled due to FIPS ``` Beyond kernel-level verification, you can ensure that the whole operating system environment is configured for FIPS compliance: ```shell sudo fips-mode-setup --check ``` The output of the command shows that FIPS is running: ```none FIPS mode is enabled. ``` ### Step 3: Verify the OpenSSL is in FIPS mode {#openssl-fips-check} **Determine the OpenSSL FIPS Provider is active**: This test verifies the correct version of OpenSSL and that the OpenSSL FIPS Provider is active: ```shell openssl list -providers | grep -A3 fips ``` The output of the command shows the FIPS provider status: ```none fips name: Red Hat Enterprise Linux 9 - OpenSSL FIPS Provider version: 3.0.7-395c1a240fbfffd8 status: active ``` **Determine whether OpenSSL can perform SHA1 hashes**: This test verifies the correct operation of OpenSSL. The SHA-1 hash algorithm, while considered weak, is still permitted in FIPS mode as it is included in FIPS-approved standards for certain legacy use cases. Failure of this command indicates that the OpenSSL implementation is not working properly: ```shell openssl sha1 /dev/null ``` The result of the command, showing the SHA1 checksum of `/dev/null`: ```none SHA1(/dev/null)= da39a3ee5e6b4b0d3255bfef95601890afd80709 ``` **Determine whether OpenSSL allows MD5 hashes**: This test verifies that OpenSSL is running in FIPS mode. MD5 is not a permitted hash algorithm in FIPS mode, so an attempt to use it fails: ```shell openssl md5 /dev/null ``` The result of the command: ```none Error setting digest 200458BAFFFF0000:error:0308010C:digital envelope routines:inner_evp_generic_fetch:unsupported:crypto/evp/evp_fetch.c:355:Global default library context, Algorithm (MD5 : 95), Properties () 200458BAFFFF0000:error:03000086:digital envelope routines:evp_md_init_internal:initialization error:crypto/evp/digest.c:272: ``` If OpenSSL is not running in FIPS mode, the MD5 hash functions normally: ```shell openssl md5 /dev/null ``` The result of the command, showing the MD5 checksum of `/dev/null`: ```none MD5(/dev/null)= d41d8cd98f00b204e9800998ecf8427e ``` ### Step 4: Install NGINX Plus on the operating system {#nginx-plus-instll} Follow the [F5 NGINX Plus Installation guide](https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-plus/) to install NGINX Plus on the host operating system, either directly from the [NGINX Plus repository](https://account.f5.com/myf5), or by downloading the **nginx-plus** package (**rpm** or **deb** package) onto another system and manually installing it on the host operating system. **Verify that NGINX Plus is correctly installed**: Run the following command to confirm that NGINX Plus is installed and is using the expected OpenSSL cryptographic module: ```shell nginx -V ``` Sample output from the command: ```shell nginx version: nginx/1.29.3 (nginx-plus-r36) built by gcc 11.5.0 20240719 (Red Hat 11.5.0-5) (GCC) built with OpenSSL 3.2.2 4 Jun 2024 ``` **Configure NGINX Plus to serve a simple SSL/TLS‑protected website**: Add the following simple configuration to NGINX Plus: ```nginx server { listen 443 ssl; ssl_certificate /etc/nginx/ssl/test.crt; ssl_certificate_key /etc/nginx/ssl/test.key; location / { root /usr/share/nginx/html; index index.html index.htm; } } ``` If necessary, you can generate a self‑signed certificate for test purposes: ```shell mkdir -p /etc/nginx/ssl && \ openssl req -newkey rsa:2048 -nodes -keyout /etc/nginx/ssl/test.key -x509 -days 365 -out /etc/nginx/ssl/test.crt ``` Verify that you can access the website using HTTPS from a remote host. Connect to the NGINX IP address using the `openssl s_client` command, and enter the HTTP message `GET /`: ```shell (echo "GET /" ; sleep 1) | openssl s_client -connect :443 ``` Use `openssl s_client` for this test because it unambiguously confirms which SSL/TLS cipher was negotiated in the connection. After some debugging information (including the cipher selected), the body of the default “Welcome to nginx!” greeting page is displayed. ### Step 5: Verify compliance with FIPS {#nginx-plus-fips-check} FIPS 140-2 and 140-3 disallows the use of some cryptographic algorithms, including the Camellia block cipher. In addition to FIPS 140-2, FIPS 140-3 disallows the use of several ciphers and algorithms that were once allowed or still allowed under FIPS 140-2. You can test compliance with FIPS 140-2 / 140-3 by issuing SSL/TLS requests with known ciphers on another (non-FIPS-mode) server: #### RC4-MD5 `RC4-MD5` is considered insecure and deprecated across all modern cryptographic standards. It is disallowed and disabled by default in FIPS-compliant OpenSSL, and TLS 1.2 and 1.3. The SSL handshake always fails. ```shell (echo "GET /" ; sleep 1) | openssl s_client -connect :443 -cipher RC4-MD5 ``` For FIPS compliance, alternative cipher suites can be used such as: - `TLS_RSA_WITH_AES_128_GCM_SHA256` - `TLS_RSA_WITH_AES_256_GCM_SHA384` - `TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256` - `TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384` #### CAMELLIA-SHA ```shell (echo "GET /" ; sleep 1) | openssl s_client -connect :443 -cipher CAMELLIA256-SHA ``` This cipher is considered secure but is not permitted by the FIPS standard. The SSL handshake fails if the target system is compliant with FIPS 140-2 /140-3, and succeeds otherwise. #### AES256-SHA The cipher is permitted under FIPS 140-2 as it combines AES encryption with SHA-1. However, under FIPS 140-3, SHA-1 is explicitly disallowed due to its vulnerabilities, such as susceptibility to collision attacks. As a result, the SSL handshake fails under FIPS 140-3 and succeeds under FIPS 140-2: ```shell (echo "GET /" ; sleep 1) | openssl s_client -connect :443 -cipher AES256-SHA ``` For FIPS 140-3 compliance, alternative cipher suites that leverage SHA-2 or SHA-3 for hashing can be used: - AES-GCM-Based Cipher Suites (TLS 1.2): - `TLS_RSA_WITH_AES_256_GCM_SHA384` - `TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384` - ChaCha20-Based Cipher Suites (TLS 1.2 or 1.3): - `TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256` - TLS 1.3 Cipher Suites: - `TLS_AES_256_GCM_SHA384` - `TLS_AES_128_GCM_SHA256` - `TLS_CHACHA20_POLY1305_SHA256` #### 3DES The `3DES` (Triple DES) cipher is allowed under FIPS 140-2, but disallowed under FIPS 140-3. NIST deprecated its use [starting January 1, 2024](https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-131Ar2.pdf) due to its reduced security strength (112 bits) and vulnerability to brute-force attacks. As a result, the SSL handshake always fails in FIPS-3 compliant environment: ```shell (echo "GET /" ; sleep 1) | openssl s_client -connect :443 -cipher DES-CBC3 ``` For FIPS 140-3 compliance, AES-Based or ChaCha20-Based cipher suites can be used: - `TLS_RSA_WITH_AES_128_GCM_SHA256` - `TLS_RSA_WITH_AES_256_GCM_SHA384` - `TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256` - `TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384` - `TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256` #### DH and DSA Under FIPS 140-2, Diffie-Hellman (DH) and Digital Signature Algorithm (DSA) were permitted with a minimum key size of 1024 bits. However, under FIPS 140-3, the minimum key size for both DH and DSA has been increased to 2048 bits. For example, the `TLS_DH_RSA_WITH_AES_128_CBC_SHA` algorithm is FIPS 140-2 compliant, but not FIPS 140-3 compliant due to its use of DH with a key size of less than 2048 bits, CBC mode encryption, and SHA-1 hashing: ```shell (echo "GET /" ; sleep 1) | openssl s_client -connect :443 -cipher TLS_DH_RSA_WITH_AES_128_CBC_SHA ``` The `TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256` algorithm is FIPS 140-3 compliant as it uses Elliptic Curve Diffie-Hellman Ephemeral (ECDHE), AES-GCM for encryption, and SHA-256 for hashing: ```shell (echo "GET /" ; sleep 1) | openssl s_client -connect :443 -cipher TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 ``` ## Ciphers disabled in FIPS Mode The FIPS 140-2 standard only permits a [subset of the typical SSL and TLS ciphers](https://csrc.nist.gov/csrc/media/publications/fips/140/2/final/documents/fips1402annexa.pdf), while FIPS 140-3 [extends this requirements](https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.140-3.pdf) to enforce stricter cryptographic algorithms. In the following test, the ciphers presented by NGINX Plus are surveyed using the `nmap` utility (installed separately). In its default configuration, with the [`ssl_ciphers HIGH:!aNULL:!MD5`](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_ciphers) directive, NGINX Plus presents the following ciphers to SSL/TLS clients: ```shell nmap --script ssl-enum-ciphers -p 443 ``` The output of the command for NGINX Plus running on Red Hat Enterprise Linux 9 without FIPS enabled: ```shell PORT STATE SERVICE 443/tcp open https | ssl-enum-ciphers: | TLSv1.2: | ciphers: | TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp256r1) - A | TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (secp256r1) - A | TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (secp256r1) - A | TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (secp256r1) - A | TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (secp256r1) - A | TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (secp256r1) - A | TLS_ECDHE_RSA_WITH_ARIA_128_GCM_SHA256 (secp256r1) - A | TLS_ECDHE_RSA_WITH_ARIA_256_GCM_SHA384 (secp256r1) - A | TLS_ECDHE_RSA_WITH_CAMELLIA_128_CBC_SHA256 (secp256r1) - A | TLS_ECDHE_RSA_WITH_CAMELLIA_256_CBC_SHA384 (secp256r1) - A | TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (secp256r1) - A | TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A | TLS_RSA_WITH_AES_128_CBC_SHA256 (rsa 2048) - A | TLS_RSA_WITH_AES_128_CCM (rsa 2048) - A | TLS_RSA_WITH_AES_128_GCM_SHA256 (rsa 2048) - A | TLS_RSA_WITH_AES_256_CBC_SHA (rsa 2048) - A | TLS_RSA_WITH_AES_256_CBC_SHA256 (rsa 2048) - A | TLS_RSA_WITH_AES_256_CCM (rsa 2048) - A | TLS_RSA_WITH_AES_256_GCM_SHA384 (rsa 2048) - A | TLS_RSA_WITH_ARIA_128_GCM_SHA256 (rsa 2048) - A | TLS_RSA_WITH_ARIA_256_GCM_SHA384 (rsa 2048) - A | TLS_RSA_WITH_CAMELLIA_128_CBC_SHA (rsa 2048) - A | TLS_RSA_WITH_CAMELLIA_128_CBC_SHA256 (rsa 2048) - A | TLS_RSA_WITH_CAMELLIA_256_CBC_SHA (rsa 2048) - A | TLS_RSA_WITH_CAMELLIA_256_CBC_SHA256 (rsa 2048) - A | compressors: | NULL | cipher preference: client | TLSv1.3: | ciphers: | TLS_AKE_WITH_AES_128_CCM_SHA256 (ecdh_x25519) - A | TLS_AKE_WITH_AES_128_GCM_SHA256 (ecdh_x25519) - A | TLS_AKE_WITH_AES_256_GCM_SHA384 (ecdh_x25519) - A | TLS_AKE_WITH_CHACHA20_POLY1305_SHA256 (ecdh_x25519) - A | cipher preference: client |_ least strength: A ``` When FIPS 140-3 mode is enabled, NGINX Plus presents the following ciphers to SSL/TLS clients: ```shell PORT STATE SERVICE 443/tcp open https | ssl-enum-ciphers: | TLSv1.2: | ciphers: | TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp256r1) - A | TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (secp256r1) - A | TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (secp256r1) - A | TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (secp256r1) - A | TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (secp256r1) - A | TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (secp256r1) - A | compressors: | NULL | cipher preference: client | TLSv1.3: | ciphers: | TLS_AKE_WITH_AES_128_CCM_SHA256 (secp256r1) - A | TLS_AKE_WITH_AES_128_GCM_SHA256 (secp256r1) - A | TLS_AKE_WITH_AES_256_GCM_SHA384 (secp256r1) - A | cipher preference: client |_ least strength: A ``` Based on the results above, the following ciphers are disallowed under FIPS 140-3 compliance: - Camellia-Based Ciphers: FIPS compliance requires cryptographic algorithms to be validated by NIST, and Camellia is not NIST-approved despite being recognized by ISO/IEC standards. - `TLS_ECDHE_RSA_WITH_CAMELLIA_128_CBC_SHA256` - `TLS_ECDHE_RSA_WITH_CAMELLIA_256_CBC_SHA384` - `TLS_RSA_WITH_CAMELLIA_128_CBC_SHA` - `TLS_RSA_WITH_CAMELLIA_128_CBC_SHA256` - `TLS_RSA_WITH_CAMELLIA_256_CBC_SHA` - `TLS_RSA_WITH_CAMELLIA_256_CBC_SHA256` - ARIA-Based Ciphers: similar to Camellia, ARIA is not a NIST-approved algorithm and is therefore excluded from FIPS compliance. - `TLS_ECDHE_RSA_WITH_ARIA_128_GCM_SHA256` - `TLS_ECDHE_RSA_WITH_ARIA_256_GCM_SHA384` - `TLS_RSA_WITH_ARIA_128_GCM_SHA256` - `TLS_RSA_WITH_ARIA_256_GCM_SHA384` - RSA Key Exchange Ciphers: static RSA key exchange lacks Forward Secrecy, allowing decryption of past traffic if the private key is compromised, thus disallowed in FIPS mode. - `TLS_RSA_WITH_AES_128_CBC_SHA` - `TLS_RSA_WITH_AES_128_CBC_SHA256` - `TLS_RSA_WITH_AES_128_GCM_SHA256` - `TLS_RSA_WITH_AES_256_CBC_SHA` - `TLS_RSA_WITH_AES_256_CBC_SHA256` - `TLS_RSA_WITH_AES_256_GCM_SHA384` - CBC Mode Ciphers (Non-AEAD: CBC is vulnerable to padding oracle attacks (e.g., POODLE, Lucky13), making it insecure. FIPS 140-3 prioritizes AEAD modes like AES-GCM and AES-CCM. - `TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA` - `TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256` - `TLS_RSA_WITH_AES_128_CBC_SHA` - `TLS_RSA_WITH_AES_128_CBC_SHA256` - `TLS_RSA_WITH_AES_256_CBC_SHA` - `TLS_RSA_WITH_AES_256_CBC_SHA256` - ChaCha20-Poly1305: it is not a NIST-approved algorithm and is excluded from FIPS compliance. FIPS exclusively permits algorithms such as `AES-GCM` and `AES-CCM`. - `TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256` - AES-CCM Variants: - `TLS_RSA_WITH_AES_128_CCM` - `TLS_RSA_WITH_AES_256_CCM` You can also use the [Qualys SSL server test](https://www.ssllabs.com/ssltest) to verify the ciphers presented by NGINX Plus to SSL/TLS clients. ## Conclusion NGINX Plus can be used to decrypt and encrypt SSL/TLS‑encrypted network traffic in deployments that require FIPS 140-2 Level 1 or FIPS 140-3 Level 1 compliance. The process described above may be used to verify that NGINX Plus is operating in conformance with the FIPS 140-2 Level 1 and FIPS 140-3 Level 1 standards. ## Definition of terms - **Cryptographic module**: The OpenSSL software, comprised of libraries of FIPS‑validated algorithms that can be used by other applications. - **Cryptographic boundary**: The operational functions that use FIPS‑validated algorithms. For NGINX Plus, the cryptographic boundary includes all functionality that is implemented by the [`http_auth_jwt`](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html), [`http_ssl`](https://nginx.org/en/docs/http/ngx_http_ssl_module.html), [`http_v2`](https://nginx.org/en/docs/http/ngx_http_v2_module.html), [`http_v3`](https://nginx.org/en/docs/http/ngx_http_v3_module.html), [`mail_ssl`](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html), and [`stream_ssl`](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html) modules. These modules implement SSL and TLS operations for inbound and outbound connections which use HTTP, HTTP/2, HTTP/3, TCP, and mail protocols. - **NGINX Plus**: The NGINX Plus software application developed by F5, Inc. and delivered in binary format from F5 servers. - **FIPS mode**: When the operating system is configured to run in FIPS mode, the OpenSSL cryptographic module operates in a mode that has been validated to be in compliance with FIPS 140-2 Level 1 or FIPS 140-3 Level 1. Most operating systems do not run in FIPS mode by default, so explicit configuration is necessary to enable FIPS mode. - **FIPS validated**: A component of the OpenSSL cryptographic module (the OpenSSL FIPS Object Module) is formally validated by an authorized certification laboratory. The validation holds if the module is built from source with no modifications to the source or build process. The implementation of FIPS mode that is present in operating system vendors’ distributions of OpenSSL contains this validated module. - **FIPS compliant**: NGINX Plus is compliant with FIPS 140-2 Level 1 and FIPS 140-3 Level 1 within the cryptographic boundary when used with a FIPS‑validated OpenSSL cryptographic module on an operating system running in FIPS mode. ## See also [FIPS 140-3 Standard in the PDF format](https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.140-3.pdf) [FIPS compliance with NGINX Plus and Red Hat Enterprise Linux](https://www.f5.com/pdf/technology-alliances/fips-compliance-made-simple-with-f5-and-red-hat-one-pager.pdf) [F5 NGINX Plus running on Red Hat Enterprise Linux is now FIPS 140-3 compliant](https://www.redhat.com/en/blog/f5-nginx-plus-running-red-hat-enterprise-linux-now-fips-140-3-compliant) ## Revision history - Version 2 (September 2025) - Added information about FIPS 140-3 Level 1 compliance, updated test procedures, updated product versions, updated URLs to operating systems, NIST certificates and other relevant resources. - Version 1 (August 2019) - Initial version with FIPS 140-2 Level 1 compliance. -------------------------------------------------------------------------------- title: "Single Sign-On with OneLogin" url: https://docs.nginx.com/nginx/deployment-guides/single-sign-on/onelogin/index.md description: Enable OpenID Connect-based single sign-on (SSO) for applications proxied by NGINX Plus, using OneLogin as the identity provider (IdP). -------------------------------------------------------------------------------- This guide explains how to enable single sign-on (SSO) for applications being proxied by F5 NGINX Plus. The solution uses OpenID Connect as the authentication mechanism, with [OneLogin](https://www.onelogin.com/) as the Identity Provider (IdP) and NGINX Plus as the Relying Party (RP), or OIDC client application that verifies user identity. **Note:** This guide applies to [NGINX Plus Release 36](nginx/releases.md#r36) and later. In earlier versions, NGINX Plus relied on an [njs-based solution](#legacy-njs-guide), which required NGINX JavaScript files, key-value stores, and advanced OpenID Connect logic. In the latest NGINX Plus version, the new [OpenID Connect module](https://nginx.org/en/docs/http/ngx_http_oidc_module.html) simplifies this process to just a few directives. ## Prerequisites - An [OneLogin](https://www.onelogin.com/) account with administrator privileges. - An NGINX Plus [subscription](https://www.f5.com/products/nginx/nginx-plus) and NGINX Plus [Release 36](nginx/releases.md#r36) or later. For installation instructions, see [Installing NGINX Plus](https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-plus/). - A domain name pointing to your NGINX Plus instance, for example, `demo.example.com`. ## Configure OneLogin {#onelogin-setup} ### Create a OneLogin OIDC Application 1. Log in to your OneLogin admin console, for example, `https://.onelogin.com`. 2. In the navigation bar, select **Applications**. 3. Click the **Add App** button. - On the **Find Applications** page, search for **OpenID Connect (OIDC)** and then select it. - Enter a **Display Name**, for example, `NGINX Demo App`. - Select **Save**. 4. In the app navigation, select **Configuration**. - In **Redirect URIs**, add the callback URI for your NGINX Plus instance, for example, `https://demo.example.com/oidc_callback`. - In **Logout URL**, add the post logout redirect URI, for example, `https://demo.example.com/post_logout/`. - Select **Save**. 5. In the app navigation, select **SSO**. - Copy the **Client ID**. You will need it later when configuring NGINX Plus. - Select **Show client secret** and copy the **Client Secret**. You will need it later when configuring NGINX Plus. - Copy the **Issuer** URL, or OpenID Connect Discovery URL. You will need it later when configuring NGINX Plus. For OneLogin, the Issuer ID generally structured as: `https://.onelogin.com/oidc/2` See [Provider Configuration](https://developers.onelogin.com/openid-connect/api/provider-config) for details. ### Get the OpenID Connect Discovery URL Check the OpenID Connect Discovery URL. By default, OneLogin publishes the `.well-known/openid-configuration` document at the following address: `https://.onelogin.com/oidc/2/.well-known/openid-configuration`. 1. Run the following `curl` command in a terminal: ```shell curl https://.onelogin.com/oidc/2/.well-known/openid-configuration | jq ``` Where: - the `.onelogin.com` is your OneLogin subdomain - the `/oidc/2` is the OneLogin OIDC endpoint version - the `/.well-known/openid-configuration` is the default address for OneLogin for document location - the `jq` command (optional) is used to format the JSON output for easier reading and requires the [jq](https://jqlang.github.io/jq/) JSON processor to be installed. The configuration metadata is returned in the JSON format: ```json { ... "issuer": "https://.onelogin.com/oidc/2", "authorization_endpoint": "https://.onelogin.com/oidc/2/auth", "token_endpoint": "https://.onelogin.com/oidc/2/token", "jwks_uri": "https://.onelogin.com/oidc/2/certs", "userinfo_endpoint": "https://.onelogin.com/oidc/2/me", "end_session_endpoint": "https://.onelogin.com/oidc/2/logout", ... } ``` 2. Copy the **issuer** value, you will need it later when configuring NGINX Plus. Typically, the OpenID Connect Issuer for OneLogin is `https://.onelogin.com/oidc/2`. **Note:** You will need the values of **Client ID**, **Client Secret**, and **Issuer** in the next steps. ### Assign Users and Groups 1. In the app navigation, select **Users** > **Roles**. 2. Add users and groups who should have access to this application. ## Set up NGINX Plus {#nginx-plus-setup} With Onelogin configured, you can enable OIDC on NGINX Plus. NGINX Plus serves as the Rely Party (RP) application — a client service that verifies user identity. 1. Ensure that you are using the latest version of NGINX Plus by running the `nginx -v` command in a terminal: ```shell nginx -v ``` The output should match NGINX Plus Release 36 or later: ```none nginx version: nginx/1.29.3 (nginx-plus-r36) ``` 2. Ensure that you have the values of the **Client ID**, **Client Secret**, and **Issuer** obtained during [Onelogin Configuration](#onelogin-setup). 3. In your preferred text editor, open the NGINX configuration file (`/etc/nginx/nginx.conf` for Linux or `/usr/local/etc/nginx/nginx.conf` for FreeBSD). 4. In the [`http {}`](https://nginx.org/en/docs/http/ngx_http_core_module.html#http) context, make sure your public DNS resolver is specified with the [`resolver`](https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver) directive: By default, NGINX Plus re‑resolves DNS records at the frequency specified by time‑to‑live (TTL) in the record, but you can override the TTL value with the `valid` parameter: ```nginx http { resolver 10.0.0.1 ipv4=on valid=300s; # ... } ``` 5. In the [`http {}`](https://nginx.org/en/docs/http/ngx_http_core_module.html#http) context, define the OneLogin provider named `onelogin` by specifying the [`oidc_provider {}`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#oidc_provider) context: ```nginx http { resolver 10.0.0.1 ipv4=on valid=300s; oidc_provider onelogin { # ... } # ... } ``` 6. In the [`oidc_provider {}`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#oidc_provider) context, specify: - your actual OneLogin **Client ID** obtained in [OneLogin Configuration](#onelogin-setup) with the [`client_id`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_id) directive - your **Client Secret** obtained in [OneLogin Configuration](#onelogin-setup) with the [`client_secret`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_secret) directive - the **Issuer** URL obtained in [OneLogin Configuration](#onelogin-setup) with the [`issuer`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_secret) directive The `issuer` is typically your OneLogin OIDC URL: `https://.onelogin.com/oidc/2`. - The **logout_uri** is URI that a user visits to start an RP‑initiated logout flow. - The **post_logout_uri** is absolute HTTPS URL where OneLogin should redirect the user after a successful logout. This value **must also be configured** in the OneLogin application's Logout URL setting. - If the **logout_token_hint** directive set to `on`, NGINX Plus sends the user's ID token as a *hint* to OneLogin. This directive is **required** by OneLogin when `post_logout_redirect_uri` is used. - If the **userinfo** directive is set to `on`, NGINX Plus will fetch `/oidc/2/me` from the OneLogin and append the claims from userinfo to the `$oidc_claims_` variables. - PKCE (Proof Key for Code Exchange) is automatically enabled when OneLogin's OpenID Connect discovery document advertises the `S256` code challenge method in the `code_challenge_methods_supported` field. You can override this behavior with the [`pkce`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#pkce) directive: set `pkce off;` to disable PKCE even when `S256` is advertised, or `pkce on;` to force PKCE even if the IdP metadata does not list `S256`. - The module automatically selects the client authentication method for the token endpoint based on the provider metadata `token_endpoint_auth_methods_supported`. When only `client_secret_post` is advertised, NGINX Plus uses the `client_secret_post` method and sends the client credentials in the POST body. When both `client_secret_basic` and `client_secret_post` are present, the module prefers HTTP Basic (`client_secret_basic`). - **Note:** All interaction with the IdP is secured exclusively over SSL/TLS, so NGINX must trust the certificate presented by the IdP. By default, this trust is validated against your system’s CA bundle (the default CA store for your Linux or FreeBSD distribution). If the IdP’s certificate is not included in the system CA bundle, you can explicitly specify a trusted certificate or chain with the [`ssl_trusted_certificate`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#ssl_trusted_certificate) directive so that NGINX can validate and trust the IdP’s certificate. ```nginx http { resolver 10.0.0.1 ipv4=on valid=300s; oidc_provider onelogin { issuer https://.onelogin.com/oidc/2; client_id ; client_secret ; logout_uri /logout; post_logout_uri https://demo.example.com/post_logout/; logout_token_hint on; userinfo on; # Optional: PKCE configuration. By default, PKCE is automatically # enabled when the IdP advertises the S256 code challenge method. # pkce on; } # ... } ``` 7. Make sure you have configured a [server](https://nginx.org/en/docs/http/ngx_http_core_module.html#server) that corresponds to `demo.example.com`, and there is a [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) that [points](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass) to your application (see [Step 10](#oidc_app)) at `http://127.0.0.1:8080` that is going to be OIDC-protected: ```nginx http { # ... server { listen 443 ssl; server_name demo.example.com; ssl_certificate /etc/ssl/certs/fullchain.pem; ssl_certificate_key /etc/ssl/private/key.pem; location / { # ... proxy_pass http://127.0.0.1:8080; } } # ... } ``` 8. Protect this [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) with OneLogin OIDC by specifying the [`auth_oidc`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#auth_oidc) directive that will point to the `onelogin` configuration specified in the [`oidc_provider {}`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#oidc_provider) context in [Step 5](#onelogin-setup-oidc-provider): ```nginx # ... location / { auth_oidc onelogin; # ... proxy_pass http://127.0.0.1:8080; } # ... ``` 9. Pass the OIDC claims as headers to the application ([Step 10](#oidc_app)) with the [`proxy_set_header`](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_header) directive. These claims are extracted from the ID token returned by OneLogin: - [`$oidc_claim_sub`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) - a unique `Subject` identifier assigned for each user by OneLogin - [`$oidc_claim_email`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) the e-mail address of the user - [`$oidc_claim_name`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) - the full name of the user - any other OIDC claim using the [`$oidc_claim_ `](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) variable ```nginx # ... location / { auth_oidc onelogin; proxy_set_header sub $oidc_claim_sub; proxy_set_header email $oidc_claim_email; proxy_set_header name $oidc_claim_name; proxy_pass http://127.0.0.1:8080; } # ... ``` 10. Provide endpoint for completing logout: ```nginx # ... location /post_logout/ { return 200 "You have been logged out.\n"; default_type text/plain; } # ... ``` 11. Create a simple test application referenced by the `proxy_pass` directive which returns the authenticated user's full name and email upon successful authentication: ```nginx # ... server { listen 8080; location / { return 200 "Hello, $http_name!\nEmail: $http_email\nOneLogin sub: $http_sub\n"; default_type text/plain; } } ``` 12. Save the NGINX configuration file and reload the configuration: ```nginx nginx -s reload ``` ### Complete Example This configuration example summarizes the steps outlined above. It includes only essential settings such as specifying the DNS resolver, defining the OIDC provider, configuring SSL, and proxying requests to an internal server. ```nginx http { # Use a public DNS resolver for Issuer discovery, etc. resolver 10.0.0.1 ipv4=on valid=300s; oidc_provider onelogin { # The 'issuer' is typically your OneLogin OIDC base URL # e.g. https://.onelogin.com/oidc/2 issuer https://.onelogin.com/oidc/2; # Replace with your actual OneLogin Client ID and Secret client_id ; client_secret ; # RP‑initiated logout logout_uri /logout; post_logout_uri https://demo.example.com/post_logout/; logout_token_hint on; # Fetch userinfo claims userinfo on; # Optional: PKCE configuration # pkce on; } server { listen 443 ssl; server_name demo.example.com; ssl_certificate /etc/ssl/certs/fullchain.pem; ssl_certificate_key /etc/ssl/private/key.pem; location / { # Protect this path with OneLogin OIDC auth_oidc onelogin; # Forward OIDC claims to the upstream as headers if desired proxy_set_header sub $oidc_claim_sub; proxy_set_header email $oidc_claim_email; proxy_set_header name $oidc_claim_name; proxy_pass http://127.0.0.1:8080; } location /post_logout/ { return 200 "You have been logged out.\n"; default_type text/plain; } } server { # Simple test upstream server listen 8080; location / { return 200 "Hello, $http_name!\nEmail: $http_email\nOneLogin sub: $http_sub\n"; default_type text/plain; } } } ``` ### Testing 1. Open `https://demo.example.com/` in a browser. You will be automatically redirected to the OneLogin sign-in page. 2. Enter valid OneLogin credentials of a user who has access the application. Upon successful sign-in, OneLogin redirects you back to NGINX Plus, and you will see the proxied application content (for example, "Hello, Jane Doe!"). 3. Navigate to `https://demo.example.com/logout`. NGINX Plus initiates an RP‑initiated logout; OneLogin ends the session and redirects back to `https://demo.example.com/post_logout/`. 4. Refresh `https://demo.example.com/` again. You should be redirected to OneLogin for a fresh sign‑in, proving the session has been terminated. **Note:** If you restricted access to a group of users, be sure to select a user who has access to the application. ## Legacy njs-based OneLogin Solution {#legacy-njs-guide} If you are running NGINX Plus R33 and earlier or if you still need the njs-based solution, refer to the [Legacy njs-based OneLogin Guide](nginx/deployment-guides/single-sign-on/oidc-njs/onelogin.md) for details. The solution uses the [`nginx-openid-connect`](https://github.com/nginxinc/nginx-openid-connect) GitHub repository and NGINX JavaScript files. ## See Also - [NGINX Plus Native OIDC Module Reference documentation](https://nginx.org/en/docs/http/ngx_http_oidc_module.html) - [Release Notes for NGINX Plus R36](nginx/releases.md#r36) ## Revision History - Version 3 (November 2025) – Updated for NGINX Plus R36; added PKCE configuration (`pkce` directive) and the `client_secret_post` token endpoint authentication method. - Version 2 (August 2025) – Updated for NGINX Plus R35; added RP‑initiated logout (`logout_uri`, `post_logout_uri`, `logout_token_hint`) and `userinfo` support. - Version 1 (March 2025) – Initial version (NGINX Plus Release 34). -------------------------------------------------------------------------------- title: "Dynamic Configuration of Upstreams with the NGINX Plus API" url: https://docs.nginx.com/nginx/admin-guide/load-balancer/dynamic-configuration-api/index.md description: Dynamically reconfigure the servers in an F5 NGINX Plus upstream group using the NGINX Plus API, without reloading configuration or restarting processes. -------------------------------------------------------------------------------- ## Overview With F5 NGINX Plus, configuration of upstream servers in a server group can be modified on-the-fly without reloading the servers and NGINX configuration. This is useful for: - autoscaling, when you need to add more servers - maintenance, when you need to remove a server, specify a backup server, or take a server down temporarily - quick setup, when you need to change upstream server settings such as server weight, active connections, slow start, failure timeouts. - monitoring, when you get the state of the server or server group with one command These changes are made with the NGINX Plus REST API interface with API commands. > **Note:** In NGINX Plus [Release 12](nginx/releases.md#r12) and earlier, dynamic configuration was performed with the `upstream_conf` handler. That API (and the extended `status` API) are now deprecated in favor of the NGINX Plus API. ## Prerequisites Prior to using the dynamic configuration feature, make sure that you have the following environment: 1. NGINX Plus [R13](nginx/releases.md#r13) or later 2. You have created upstream groups of application or web servers, as described in [HTTP Load Balancing](http-load-balancer.md) and [TCP/UDP Load Balancing](tcp-udp-load-balancer.md) 3. Upstream server groups reside in the shared memory zone, as described in [Sharing Data with Multiple Worker Processes](http-load-balancer.md) ## Enabling Dynamic Configuration 1. Create an upstream server group as described in Proxying Traffic to a Group of Servers. ```nginx http { # ... upstream appservers { server appserv1.example.com weight=5; server appserv2.example.com:8080 fail_timeout=5s; server reserve1.example.com:8080 backup; server reserve2.example.com:8080 backup; } server { # Location that proxies requests to the upstream group location / { proxy_pass http://appservers; health_check; } } } ``` 1. Include the [`zone`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#zone) directive in the `upstream` block. The `zone` directive configures a zone in the shared memory and sets the zone name and size. The configuration of the server group is kept in this zone, so all worker processes use the same configuration: ```nginx http { # ... upstream appservers { zone appservers 64k; server appserv1.example.com weight=5; server appserv2.example.com:8080 fail_timeout=5s; server reserve1.example.com:8080 backup; server reserve2.example.com:8080 backup; } } ``` 2. Enable the NGINX API in read‑write mode by including the [`api`](https://nginx.org/en/docs/http/ngx_http_api_module.html#api) directive in a dedicated [`location`](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) block in a [`server`](https://nginx.org/en/docs/http/ngx_http_core_module.html#server) block. We strongly recommend restricting access to the location and to `PATCH`/`POST`/`DELETE` methods. This example uses the [`allow`](https://nginx.org/en/docs/http/ngx_http_access_module.html#allow) and [`deny`](https://nginx.org/en/docs/http/ngx_http_access_module.html#deny) directives to grant access from the `localhost` address (`127.0.0.1`) and deny access from all other addresses. It also restricts access to `PATCH`/`POST`/`DELETE` methods with [HTTP basic authentication](https://nginx.org/en/docs/http/ngx_http_auth_basic_module.html): ```nginx server { location /api { limit_except GET { auth_basic "NGINX Plus API"; auth_basic_user_file /path/to/passwd/file; } api write=on; allow 127.0.0.1; deny all; } } ``` Complete example: ```nginx http { # ... # Configuration of the server group upstream appservers { zone appservers 64k; server appserv1.example.com weight=5; server appserv2.example.com:8080 fail_timeout=5s; server reserve1.example.com:8080 backup; server reserve2.example.com:8080 backup; } server { # Location that proxies requests to the upstream group location / { proxy_pass http://appservers; health_check; } # Location for dynamic configuration requests location /api { limit_except GET { auth_basic "NGINX Plus API"; auth_basic_user_file /path/to/passwd/file; } api write=on; allow 127.0.0.1; deny all; } } } ``` ## Using the API for Dynamic Configuration The NGINX Plus REST API supports the following HTTP methods: - `GET` – Display information about an upstream group or individual server in it - `POST` – Add a server to the upstream group - `PATCH` – Modify the parameters of a particular server - `DELETE` – Delete a server from the upstream group The endpoints and methods for the NGINX Plus API are described in the [NGINX Modules Reference](https://nginx.org/en/docs/http/ngx_http_api_module.html). In addition, the API has a built‑in a Swagger specification that can be used to explore the API and understand the capabilities of each resource. The Swagger documentation can be accessed at `http://_NGINX-host_/swagger-ui/`. To change the configuration of an upstream group dynamically, send an HTTP request with the appropriate API method. The following examples use the `curl` command, but any mechanism for making HTTP requests is supported. All request bodies and responses are in JSON format. The URI specifies the following information in this order: - The hostname or IP address of the node that handles the request (in the following examples, `127.0.0.1`) - The location where the `api` directive appears (`api`) - The API version (`9`) - The name of the upstream group, complete its place in the NGINX Plus configuration hierarchy represented as a slash‑separated path (`http/upstreams/appservers`) For example, to add a new server to the `appservers` upstream group, send the following `curl` command: ```shell curl -X POST -d '{ \ "server": "10.0.0.1:8089", \ "weight": 4, \ "max_conns": 0, \ "max_fails": 0, \ "fail_timeout": "10s", \ "slow_start": "10s", \ "backup": true, \ "down": true \ }' -s 'http://127.0.0.1/api/9/http/upstreams/appservers/servers' ``` To remove a server from the upstream group: ```shell curl -X DELETE -s 'http://127.0.0.1/api/9/http/upstreams/appservers/servers/0' ``` To set the `down` parameter for the first server in the group (with ID `0`): ```shell curl -X PATCH -d '{ "down": true }' -s 'http://127.0.0.1/api/9/http/upstreams/appservers/servers/0' ``` ### Interactive Example You can explore the Swagger interface to the NGINX Plus API in read‑only mode at [https://demo.nginx.com/swagger-ui/](https://demo.nginx.com/swagger-ui/). ## Configuring Persistence of Dynamic Configuration With the basic configuration in [Enabling the API](#api_setup), changes made with the API are stored only in the shared memory zone. The changes are discarded when the NGINX Plus configuration file is reloaded. To make the changes persist across configuration reloads, move the list of upstream servers from the `upstream` block to a special file for storing server state, defined with the [`state`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#state) directive. The recommended path for Linux distributions is `/var/lib/nginx/state/`, and for FreeBSD distributions is `/var/db/nginx/state/`. ```nginx http { # ... upstream appservers { zone appservers 64k; state /var/lib/nginx/state/appservers.conf; # All servers are defined in the state file # server appserv1.example.com weight=5; # server appserv2.example.com:8080 fail_timeout=5s; # server reserve1.example.com:8080 backup; # server reserve2.example.com:8080 backup; } } ``` Keep in mind that the state file can be modified only with configuration commands from the [API](https://nginx.org/en/docs/http/ngx_http_api_module.html) interface; do not modify the file directly (for example, using a text editor). -------------------------------------------------------------------------------- title: "NGINX Directives Index" url: https://docs.nginx.com/nginx/directives/index.md -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- title: "Restricting Access to Proxied TCP Resources" url: https://docs.nginx.com/nginx/admin-guide/security-controls/controlling-access-proxied-tcp/index.md description: Protect your upstream TCP application servers by limiting connections or bandwidth, based on client IP address or other variables. -------------------------------------------------------------------------------- This chapter provides scenarios for restricting access to a database or media server that communicates over TCP. Access can be limited by IP address, the number of simultaneous connections, or bandwidth. ## Restricting Access by IP Address NGINX can allow or deny access based on a particular IP address or the range of IP addresses of client computers. To allow or deny access, use the [allow](https://nginx.org/en/docs/stream/ngx_stream_access_module.html#allow) and [deny](https://nginx.org/en/docs/stream/ngx_stream_access_module.html#deny) directives inside the [stream](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#stream) context or a [server](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#server) block: ```nginx stream { #... server { listen 12345; deny 192.168.1.2; allow 192.168.1.1/24; allow 2001:0db8::/32; deny all; } } ``` The rules are processed in sequence, from top to bottom: if the first directive in the sequence is `deny all`, then all further `allow` directives have no effect. In this example, the subnet `192.168.1.1/24` is allowed access, with the exception of `192.168.1.2`. The `2001:0db8::/32` range of IPv6 addresses is also allowed, and access to any other IP addresses is denied. ## Limiting the Number of TCP Connections You can limit the number of simultaneous TCP connections from one IP address. This can be useful in preventing denial-of-service (DoS) attacks. First, let’s define the _zone_ that will store the maximum number of TCP connections to one server, and a key to identify the connection. This can be done with the [limit_conn_zone](https://nginx.org/en/docs/http/ngx_http_limit_conn_module.html#limit_conn_zone) directive in the [stream](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#stream) context: ```nginx stream { #... limit_conn_zone $binary_remote_addr zone=ip_addr:10m; #... } ``` The key that identifies the connection is defined as `$binary_remote_addr`, which represents the IP address of the client in binary format. The name of the shared memory zone is `ip_addr` and the zone size is 10 megabytes. After the zone is defined, limit connections with the [limit_conn](https://nginx.org/en/docs/stream/ngx_stream_limit_conn_module.html#limit_conn) directive. Its first parameter specifies the name of the shared memory zone previously defined by [limit_conn_zone](https://nginx.org/en/docs/http/ngx_http_limit_conn_module.html#limit_conn_zone). As the second parameter, specify the maximum number of allowed connections for each IP address, in either the [stream](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#stream) context or a [server](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#server) block (as in this example, which also shows the prerequisite `limit_conn_zone` directive): ```nginx stream { #... limit_conn_zone $binary_remote_addr zone=ip_addr:10m; server { #... limit_conn ip_addr 1; } } ``` When limiting the number of connections per IP address, be aware that multiple hosts behind a Network Address Translation (NAT) device share the same IP address. ## Limiting the Bandwidth You can configure the maximum download or upload speed for TCP connections. Include the [proxy_download_rate](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_download_rate) or [proxy_upload_rate](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_upload_rate) directive, respectively: ```nginx server { #... proxy_download_rate 100k; proxy_upload_rate 50k; } ``` With these settings a client can download data through a single connection at a maximum speed of 100 kilobytes per second, and upload data through a single connection at a maximum speed of 50 kilobytes per second. However, the client can open several connections. So if the goal is to limit overall speed of loading for each client, the number of connections must also be limited to `1` as described in the previous section. ```nginx stream { #... limit_conn_zone $binary_remote_addr zone=ip_addr:10m; server { #... limit_conn ip_addr 1; proxy_download_rate 100k; proxy_upload_rate 50k; } } ``` -------------------------------------------------------------------------------- title: "Single Sign-On with Okta" url: https://docs.nginx.com/nginx/deployment-guides/single-sign-on/okta/index.md description: Enable OpenID Connect-based single sign-on (SSO) for applications proxied by NGINX Plus, using Okta as the identity provider (IdP). -------------------------------------------------------------------------------- This guide explains how to enable single sign-on (SSO) for applications being proxied by F5 NGINX Plus. The solution uses OpenID Connect as the authentication mechanism, with [Okta](https://www.okta.com/) as the Identity Provider (IdP), and NGINX Plus as the Relying Party, or OIDC client application that verifies user identity. **Note:** This guide applies to [NGINX Plus Release 36](nginx/releases.md#r36) and later. In earlier versions, NGINX Plus relied on an [njs-based solution](#legacy-njs-guide), which required NGINX JavaScript files, key-value stores, and advanced OpenID Connect logic. Starting from NGINX Plus version R34, the new [OpenID Connect module](https://nginx.org/en/docs/http/ngx_http_oidc_module.html) simplifies this process to just a few directives. ## Prerequisites - An [Okta](https://www.okta.com/) administrator account with privileges to create and manage applications. - An NGINX Plus [subscription](https://www.f5.com/products/nginx/nginx-plus) and NGINX Plus [Release 36](nginx/releases.md#r36) or later. For installation instructions, see [Installing NGINX Plus](https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-plus/). - A domain name pointing to your NGINX Plus instance, for example, `demo.example.com`. ## Configure Okta {#okta-setup} In Okta, register a new application for NGINX Plus as the OIDC client to obtain the Client ID, Client Secret, and required OIDC endpoints. 1. Log in to your Okta admin console. 2. In the Admin Console, go to **Applications** > **Applications**. 3. Select **Create App Integration**. 4. In **Create a new app integration**, select: - **Sign-in method**: `OIDC - OpenID Connect`. - **Application type**: `Web Application`. - Select **Next**. 5. In **New Web App Integration**: - Enter the **Name** for your new application, for example, **Nginx Demo App**. - Add a URI for the OIDC callback in **Sign-in redirect URIs**, for example, `https://demo.example.com/oidc_callback`. - Add a URI for post logout redirect in **Sign-out redirect URIs**, for example, `https://demo.example.com/post_logout/`. - Select **Save**. 6. In **Applications**, select **Nginx Demo App**. 7. In the **General** tab: - Copy the **Client ID**. You will need it later when configuring NGINX Plus. - Copy the **Client secret**. You will need it later when configuring NGINX Plus. 8. In the **Sign On** tab: - Copy the **Okta Issuer (Authorization Server)**, for example: `https://dev-123456.oktapreview.com/oauth2/default` You will need it later when configuring NGINX Plus. ### Get the OpenID Connect Discovery URL Check the OpenID Connect Discovery URL. By default, Okta publishes the `.well-known/openid-configuration` document at the following address: `https:///oauth2/default/.well-known/openid-configuration`. 1. Run the following `curl` command in a terminal: ```shell curl https://dev-123456.okta.com/oauth2/default/.well-known/openid-configuration | jq ``` Where: - the `dev-123456.okta.com` is your Okta domain - the `/oauth2/default` is the default authorization server - the `/.well-known/openid-configuration` is the default address for Okta for document location - the `jq` command (optional) is used to format the JSON output for easier reading and requires the [jq](https://jqlang.github.io/jq/) JSON processor to be installed. The configuration metadata is returned in the JSON format: ```json { ... "issuer": "https://dev-123456.okta.com/oauth2/default", "authorization_endpoint": "https://dev-123456.okta.com/oauth2/default/v1/authorize", "token_endpoint": "https://dev-123456.okta.com/oauth2/default/v1/token", "jwks_uri": "https://dev-123456.okta.com/oauth2/default/v1/keys", "userinfo_endpoint": "https://dev-123456.okta.com/oauth2/default/v1/userinfo", "end_session_endpoint": "https://dev-123456.okta.com/oauth2/default/v1/logout", ... } ``` 2. Copy the **issuer** value, you will need it later when configuring NGINX Plus. Typically, the OpenID Connect Issuer for Okta is `https:///oauth2/default`. **Note:** You will need the values of **Client ID**, **Client Secret**, and **Issuer** in the next steps. ### Assign Users or Groups By default, Okta might limit application access to certain users or groups. To add or remove users in Okta: 1. Log in to your Okta admin console. 2. In **Applications**, choose **Nginx Demo App**. 3. Go to **Assignments**. 4. Add or remove users and groups that can access this application. ### Enable Front-Channel Single Logout (optional) {#okta-frontchannel-logout} Front-channel logout allows Okta to notify NGINX Plus when a user signs out of other Single Logout (SLO)-participating applications or from the Okta dashboard itself. Okta sends a front-channel HTTP request (typically loaded in a hidden iframe) to a logout URL that you configure in the application settings. This feature is optional and requires NGINX Plus Release 36 or later, together with the `frontchannel_logout_uri` directive in the NGINX OIDC provider configuration. 1. In the Okta Admin Console, go to **Settings** > **Features**. 2. Locate **Front-channel Single Logout** and enable it for your org. 3. Go to **Applications** > **Applications** and open your **Nginx Demo App** (or the OIDC app that you created earlier). 4. On the **General** tab, click **Edit**. 5. In the **Logout** section, under **Single Logout**, enable **User logs out of other logout-initiating apps or Okta**. This option enables front-channel SLO for the app. 6. In **Logout request URL**, enter the NGINX Plus front-channel logout endpoint, for example: ```text https://demo.example.com/front_logout ``` 7. Select **Include user session details**. With this option enabled, Okta includes the user session identifier (sid) and issuer (iss) in the front-channel logout request that it sends to NGINX Plus. 8. Click **Save**. ## Set up NGINX Plus {#nginx-plus-setup} With Okta configured, you can enable OIDC on NGINX Plus. NGINX Plus serves as the Rely Party (RP) application — a client service that verifies user identity. 1. Ensure that you are using the latest version of NGINX Plus by running the `nginx -v` command in a terminal: ```shell nginx -v ``` The output should match NGINX Plus Release 36 or later: ```none nginx version: nginx/1.29.3 (nginx-plus-r36) ``` 2. Ensure that you have the values of the **Client ID**, **Client Secret**, and **Issuer** obtained during [Okta Configuration](#okta-setup). 3. In your preferred text editor, open the NGINX configuration file (`/etc/nginx/nginx.conf` for Linux or `/usr/local/etc/nginx/nginx.conf` for FreeBSD). 4. In the [`http {}`](https://nginx.org/en/docs/http/ngx_http_core_module.html#http) context, make sure your public DNS resolver is specified with the [`resolver`](https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver) directive: By default, NGINX Plus re‑resolves DNS records at the frequency specified by time‑to‑live (TTL) in the record, but you can override the TTL value with the `valid` parameter: ```nginx http { resolver 10.0.0.1 ipv4=on valid=300s; # ... } ``` 5. In the [`http {}`](https://nginx.org/en/docs/http/ngx_http_core_module.html#http) context, define the Okta provider named `okta` by specifying the [`oidc_provider {}`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#oidc_provider) context: ```nginx http { resolver 10.0.0.1 ipv4=on valid=300s; oidc_provider okta { # ... } # ... } ``` 6. In the [`oidc_provider {}`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#oidc_provider) context, specify: - your actual Okta **Client ID** obtained in [Okta Configuration](#okta-setup) with the [`client_id`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_id) directive - your **Client Secret** obtained in [Okta Configuration](#okta-setup) with the [`client_secret`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_secret) directive - the **Issuer** URL obtained in [Okta Configuration](#okta-setup) with the [`issuer`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_secret) directive The `issuer` is typically your Okta OIDC URL: `https://dev-123456.okta.com/oauth2/default`. - The **logout_uri** is URI that a user visits to start an RP‑initiated logout flow. - The **post_logout_uri** is absolute HTTPS URL where Okta should redirect the user after a successful logout. This value **must also be configured** in the Okta application's Sign-out redirect URIs. - If the **logout_token_hint** directive set to `on`, NGINX Plus sends the user's ID token as a *hint* to Okta. This directive is **optional**, however, if it is omitted the Okta may display an extra confirmation page asking the user to approve the logout request. - The **frontchannel_logout_uri** directive defines the URI that receives OpenID Connect front-channel logout requests from Okta. This URI must be an HTTPS path hosted by NGINX Plus and must match the *Front-channel logout URL* configured in the Okta application registration. When a front-channel logout GET request is received at this URI (typically in a hidden iframe), the OIDC module clears the local session for the affected user. - If the **userinfo** directive is set to `on`, NGINX Plus will fetch `/oauth2/default/v1/userinfo` from the Okta and append the claims from userinfo to the `$oidc_claims_` variables. - PKCE (Proof Key for Code Exchange) is automatically enabled when the provider metadata advertises the `S256` code challenge method in the `code_challenge_methods_supported` field of the discovery document. You can override this behavior with the [`pkce`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#pkce) directive: set `pkce off;` to disable PKCE even when `S256` is advertised, or `pkce on;` to force PKCE even if the IdP metadata does not list `S256`. - The module automatically selects the client authentication method for the token endpoint based on the provider metadata `token_endpoint_auth_methods_supported`. When only `client_secret_post` is advertised, NGINX Plus uses the `client_secret_post` method and sends the client credentials in the POST body. When both `client_secret_basic` and `client_secret_post` are present, the module prefers HTTP Basic (`client_secret_basic`). - **Note:** All interaction with the IdP is secured exclusively over SSL/TLS, so NGINX must trust the certificate presented by the IdP. By default, this trust is validated against your system’s CA bundle (the default CA store for your Linux or FreeBSD distribution). If the IdP’s certificate is not included in the system CA bundle, you can explicitly specify a trusted certificate or chain with the [`ssl_trusted_certificate`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#ssl_trusted_certificate) directive so that NGINX can validate and trust the IdP’s certificate. ```nginx http { resolver 10.0.0.1 ipv4=on valid=300s; oidc_provider okta { issuer https://dev-123456.okta.com/oauth2/default; client_id ; client_secret ; logout_uri /logout; post_logout_uri https://demo.example.com/post_logout/; logout_token_hint on; frontchannel_logout_uri /front_logout; userinfo on; # Optional: PKCE configuration. By default, PKCE is automatically # enabled when the IdP advertises the S256 code challenge method. # pkce on; } # ... } ``` 7. Make sure you have configured a [server](https://nginx.org/en/docs/http/ngx_http_core_module.html#server) that corresponds to `demo.example.com`, and there is a [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) that [points](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass) to your application (see [Step 10](#oidc_app)) at `http://127.0.0.1:8080` that is going to be OIDC-protected: ```nginx http { # ... server { listen 443 ssl; server_name demo.example.com; ssl_certificate /etc/ssl/certs/fullchain.pem; ssl_certificate_key /etc/ssl/private/key.pem; location / { # ... proxy_pass http://127.0.0.1:8080; } } # ... } ``` 8. Protect this [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) with Okta OIDC by specifying the [`auth_oidc`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#auth_oidc) directive that will point to the `okta` configuration specified in the [`oidc_provider {}`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#oidc_provider) context in [Step 5](#okta-setup-oidc-provider): ```nginx # ... location / { auth_oidc okta; # ... proxy_pass http://127.0.0.1:8080; } # ... ``` 9. Pass the OIDC claims as headers to the application ([Step 10](#oidc_app)) with the [`proxy_set_header`](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_header) directive. These claims are extracted from the ID token returned by Okta: - [`$oidc_claim_sub`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) - a unique `Subject` identifier assigned for each user by Okta - [`$oidc_claim_email`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) the e-mail address of the user - [`$oidc_claim_name`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) - the full name of the user - any other OIDC claim using the [`$oidc_claim_ `](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) variable ```nginx # ... location / { auth_oidc okta; proxy_set_header sub $oidc_claim_sub; proxy_set_header email $oidc_claim_email; proxy_set_header name $oidc_claim_name; proxy_pass http://127.0.0.1:8080; } # ... ``` 10. Provide endpoint for completing logout: ```nginx # ... location /post_logout/ { return 200 "You have been logged out.\n"; default_type text/plain; } # ... ``` 11. Create a simple test application referenced by the `proxy_pass` directive which returns the authenticated user's full name and email upon successful authentication: ```nginx # ... server { listen 8080; location / { return 200 "Hello, $http_name!\nEmail: $http_email\nOkta sub: $http_sub\n"; default_type text/plain; } } ``` 12. Save the NGINX configuration file and reload the configuration: ```nginx nginx -s reload ``` ### Complete Example This configuration example summarizes the steps outlined above. It includes only essential settings such as specifying the DNS resolver, defining the OIDC provider, configuring SSL, and proxying requests to an internal server. ```nginx http { # Use a public DNS resolver for Issuer discovery, etc. resolver 10.0.0.1 ipv4=on valid=300s; # Define the OIDC provider block for Okta oidc_provider okta { # The 'issuer' is your Okta issuer URL # For okta dev it looks like: https://dev-123456.okta.com/oauth2/default issuer https://dev-123456.okta.com/oauth2/default; # Your Okta "Client ID" from the application settings client_id ; # Your Okta "Client Secret" from the application settings client_secret ; # RP‑initiated logout logout_uri /logout; post_logout_uri https://demo.example.com/post_logout/; logout_token_hint on; # Front-channel logout (OP‑initiated single sign-out) frontchannel_logout_uri /front_logout; # Fetch userinfo claims userinfo on; # Optional: PKCE configuration # pkce on; } server { listen 443 ssl; server_name demo.example.com; ssl_certificate /etc/ssl/certs/fullchain.pem; ssl_certificate_key /etc/ssl/private/key.pem; location / { # Enforce OIDC authentication with Okta auth_oidc okta; # Pass OIDC claims as HTTP headers to the backend proxy_set_header sub $oidc_claim_sub; proxy_set_header email $oidc_claim_email; proxy_set_header name $oidc_claim_name; proxy_pass http://127.0.0.1:8080; } location /post_logout/ { return 200 "You have been logged out.\n"; default_type text/plain; } } server { # Simple test upstream server listen 8080; location / { return 200 "Hello, $http_name!\nEmail: $http_email\nOkta sub: $http_sub\n"; default_type text/plain; } } } ``` ### Testing 1. Open `https://demo.example.com/` in a browser. You will be automatically redirected to the Okta sign-in page. 2. Enter valid Okta credentials of a user who has access the application. Upon successful sign-in, Okta redirects you back to NGINX Plus, and you will see the proxied application content (for example, "Hello, Jane Doe!"). 3. Navigate to `https://demo.example.com/logout`. NGINX Plus initiates an RP‑initiated logout; Okta ends the session and redirects back to `https://demo.example.com/post_logout/`. 4. Refresh `https://demo.example.com/` again. You should be redirected to Okta for a fresh sign‑in, proving the session has been terminated. **Note:** If you restricted access to a group of users, be sure to select a user who has access to the application. ## Legacy njs-based Okta Solution {#legacy-njs-guide} If you are running NGINX Plus R33 and earlier or if you still need the njs-based solution, refer to the [Legacy njs-based Okta Guide](nginx/deployment-guides/single-sign-on/oidc-njs/okta.md) for details. The solution uses the [`nginx-openid-connect`](https://github.com/nginxinc/nginx-openid-connect) GitHub repository and NGINX JavaScript files. ## See Also - [NGINX Plus Native OIDC Module Reference documentation](https://nginx.org/en/docs/http/ngx_http_oidc_module.html) - [Release Notes for NGINX Plus R36](nginx/releases.md#r36) ## Revision History - Version 3 (November 2025) – Updated for NGINX Plus R36; added front-channel logout support (`frontchannel_logout_uri`), PKCE configuration (`pkce` directive), and the `client_secret_post` token endpoint authentication method. - Version 2 (August 2025) – Updated for NGINX Plus R35; added RP‑initiated logout (`logout_uri`, `post_logout_uri`, `logout_token_hint`) and `userinfo` support. - Version 1 (March 2025) – Initial version (NGINX Plus Release 34). -------------------------------------------------------------------------------- title: "Accepting the PROXY Protocol" url: https://docs.nginx.com/nginx/admin-guide/load-balancer/using-proxy-protocol/index.md -------------------------------------------------------------------------------- This article explains how to configure NGINX and F5 NGINX Plus to accept the PROXY protocol, rewrite the IP address of a load balancer or proxy to the one received in the PROXY protocol header, configure simple logging of a client’s IP address, and enable the PROXY protocol between NGINX and a TCP upstream server. ## Introduction The [PROXY protocol](https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt) enables NGINX and NGINX Plus to receive client connection information passed through proxy servers and load balancers such as HAproxy and Amazon Elastic Load Balancer (ELB). With the PROXY protocol, NGINX can learn the originating IP address from HTTP, SSL, HTTP/2, SPDY, WebSocket, and TCP. Knowing the originating IP address of a client may be useful for setting a particular language for a website, keeping a denylist of IP addresses, or simply for logging and statistics purposes. The information passed via the PROXY protocol is the client IP address, the proxy server IP address, and both port numbers. Using this data, NGINX can get the originating IP address of the client in several ways: - With the [`$proxy_protocol_addr`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_proxy_protocol_addr) and [`$proxy_protocol_port`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_proxy_protocol_port) variables which capture the original client IP address and port. The [`$remote_addr`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_remote_addr) and [`$remote_port`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_remote_port) variables capture the IP address and port of the load balancer. - With the [RealIP](https://nginx.org/en/docs/http/ngx_http_realip_module.html) module which rewrites the values in the [`$remote_addr`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_remote_addr) and [`$remote_port`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_remote_port) variables, replacing the IP address and port of the load balancer with the original client IP address and port. The [`$realip_remote_addr`](https://nginx.org/en/docs/http/ngx_http_realip_module.html#var_realip_remote_addr) and [`$realip_remote_port`](https://nginx.org/en/docs/http/ngx_http_realip_module.html#var_realip_remote_port) variables retain the address and port of the load balancer, and the [`$proxy_protocol_addr`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_proxy_protocol_addr) and [`$proxy_protocol_port`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_proxy_protocol_port) variables retain the original client IP address and port anyway. ## Prerequisites - To accept the PROXY protocol v2, NGINX Plus [R16](nginx/releases.md#r16) and later or NGINX Open Source [1.13.11](https://nginx.org/en/CHANGES) and later - To accept the PROXY protocol for HTTP, NGINX Plus [R3](nginx/releases.md#r3) and later or NGINX Open Source [1.5.12](https://nginx.org/en/CHANGES) and later - For TCP client‑side PROXY protocol support, NGINX Plus [R7](nginx/releases.md#r7) and later or NGINX Open Source [1.9.3](https://nginx.org/en/CHANGES) and later - To accept the PROXY protocol for TCP, NGINX Plus [R11](nginx/releases.md#r11) and later or NGINX Open Source [1.11.4](https://nginx.org/en/CHANGES) and later - The Real‑IP modules for [HTTP](https://nginx.org/en/docs/http/ngx_http_realip_module.html) and [Stream TCP](https://nginx.org/en/docs/stream/ngx_stream_realip_module.html) are not included in NGINX Open Source by default; see [Installing NGINX Open Source](nginx/admin-guide/installing-nginx/installing-nginx-open-source.md) for details. No extra steps are required for NGINX Plus. ## Configure NGINX to Accept the PROXY Protocol To configure NGINX to accept PROXY protocol headers, add the `proxy_protocol` parameter to the `listen` directive in a `server` block in the [`http {}`](https://nginx.org/en/docs/http/ngx_http_core_module.html#listen) or [`stream {}`](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#listen) block. ```nginx http { #... server { listen 80 proxy_protocol; listen 443 ssl proxy_protocol; #... } } stream { #... server { listen 12345 proxy_protocol; #... } } ``` Now you can use the [`$proxy_protocol_addr`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_proxy_protocol_addr) and [`$proxy_protocol_port`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_proxy_protocol_port) variables for the client IP address and port and additionally configure the [HTTP](https://nginx.org/en/docs/http/ngx_http_realip_module.html) and [`stream`](https://nginx.org/en/docs/stream/ngx_stream_realip_module.html) RealIP modules to replace the IP address of the load balancer in the [`$remote_addr`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_remote_addr) and [`$remote_port`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_remote_port) variables with the IP address and port of the client. ## Change the Load Balancer's IP Address To the Client IP Address You can replace the address of the load balancer or TCP proxy with the client IP address received from the PROXY protocol. This can be done with the [HTTP](https://nginx.org/en/docs/http/ngx_http_realip_module.html) and [`stream`](https://nginx.org/en/docs/stream/ngx_stream_realip_module.html) RealIP modules. With these modules, the [`$remote_addr`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_remote_addr) and [`$remote_port`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_remote_port) variables retain the real IP address and port of the client, while the [`$realip_remote_addr`](https://nginx.org/en/docs/http/ngx_http_realip_module.html#var_realip_remote_addr) and [`$realip_remote_port`](https://nginx.org/en/docs/http/ngx_http_realip_module.html#var_realip_remote_port) variables retain the IP address and port of the load balancer. To change the IP address from the load balancer's IP address to the client's IP address: 1. Make sure you've configured NGINX to accept the PROXY protocol headers. See [Configuring NGINX to Accept the PROXY Protocol](#listen). 2. Make sure that your NGINX installation includes the [HTTP](https://nginx.org/en/docs/http/ngx_http_realip_module.html) and [Stream](https://nginx.org/en/docs/stream/ngx_stream_realip_module.html) Real‑IP modules: ```shell nginx -V 2>&1 | grep -- 'http_realip_module' nginx -V 2>&1 | grep -- 'stream_realip_module' ``` If not, recompile NGINX with these modules. See [Installing NGINX Open Source](nginx/admin-guide/installing-nginx/installing-nginx-open-source.md) for details. No extra steps are required for NGINX Plus. 3. In the `set_real_ip_from` directive for [HTTP](https://nginx.org/en/docs/http/ngx_http_realip_module.html#set_real_ip_from), [Stream](https://nginx.org/en/docs/stream/ngx_stream_realip_module.html#set_real_ip_from), or both, specify the IP address or the CIDR range of addresses of the TCP proxy or load balancer: ```nginx server { #... set_real_ip_from 192.168.1.0/24; #... } ``` 4. In the `http {}` context, change the IP address of the load balancer to the IP address of the client received from the PROXY protocol header, by specifying the `proxy_protocol` parameter to the [`real_ip_header`](https://nginx.org/en/docs/http/ngx_http_realip_module.html#real_ip_header) directive: ```nginx http { server { #... real_ip_header proxy_protocol; } } ``` ## Log the Original IP Address When you know the original IP address of the client, you can configure the correct logging: 1. For HTTP, configure NGINX to pass the client IP address to upstream servers using the [`$proxy_protocol_addr`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_proxy_protocol_addr) variable with the [`proxy_set_header`](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_header) directive: ```nginx http { proxy_set_header X-Real-IP $proxy_protocol_addr; proxy_set_header X-Forwarded-For $proxy_protocol_addr; } ``` 2. Add the [`$proxy_protocol_addr`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_proxy_protocol_addr) variable to the `log_format` directive ([HTTP](https://nginx.org/en/docs/http/ngx_http_log_module.html#log_format) or [Stream](https://nginx.org/en/docs/stream/ngx_stream_log_module.html#log_format)): - In the `http` block: ```nginx http { #... log_format combined '$proxy_protocol_addr - $remote_user [$time_local] ' '"$request" $status $body_bytes_sent ' '"$http_referer" "$http_user_agent"'; } ``` - In the `stream` block: ```nginx stream { #... log_format basic '$proxy_protocol_addr - $remote_user [$time_local] ' '$protocol $status $bytes_sent $bytes_received ' '$session_time'; } ``` ## PROXY Protocol for a TCP Connection to an Upstream For a TCP stream, the PROXY protocol can be enabled for connections between NGINX and an upstream server. To enable the PROXY protocol, include the [`proxy_protocol`](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_protocol) directive in a `server` block at the `stream {}` level: ```nginx stream { server { listen 12345; proxy_pass example.com:12345; proxy_protocol on; } } ``` ## Example ```nginx http { log_format combined '$proxy_protocol_addr - $remote_user [$time_local] ' '"$request" $status $body_bytes_sent ' '"$http_referer" "$http_user_agent"'; #... server { server_name localhost; listen 80 proxy_protocol; listen 443 ssl proxy_protocol; ssl_certificate /etc/nginx/ssl/public.example.com.pem; ssl_certificate_key /etc/nginx/ssl/public.example.com.key; location /app/ { proxy_pass http://backend1; proxy_set_header Host $host; proxy_set_header X-Real-IP $proxy_protocol_addr; proxy_set_header X-Forwarded-For $proxy_protocol_addr; } } } stream { log_format basic '$proxy_protocol_addr - $remote_user [$time_local] ' '$protocol $status $bytes_sent $bytes_received ' '$session_time'; #... server { listen 12345 ssl proxy_protocol; ssl_certificate /etc/nginx/ssl/cert.pem; ssl_certificate_key /etc/nginx/ssl/cert.key; proxy_pass backend.example.com:12345; proxy_protocol on; } } ``` The example assumes that there is a load balancer in front of NGINX to handle all incoming HTTPS traffic, for example Amazon ELB. NGINX accepts HTTPS traffic on port 443 (`listen 443 ssl;`), TCP traffic on port 12345, and accepts the client’s IP address passed from the load balancer via the PROXY protocol as well (the `proxy_protocol` parameter to the `listen` directive in both the `http {}` and `stream {}` blocks. NGINX terminates HTTPS traffic (the [`ssl_certificate`](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_certificate) and [`ssl_certificate_key`](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_certificate_key) directives) and proxies the decrypted data to a backend server: - For HTTP: `proxy_pass http://backend1;` - For TCP: `proxy_pass backend.example.com:12345` It includes the client IP address and port with the [`proxy_set_header`](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_header) directives. The [`$proxy_protocol_addr`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_proxy_protocol_addr) variable specified in the [`log_format`](https://nginx.org/en/docs/http/ngx_http_log_module.html#log_format) directive also passes the client’s IP address to the log for both HTTP and TCP. Additionally, a TCP server (the `stream {}` block) sends its own PROXY protocol data to its backend servers (the `proxy_protocol on` directive). -------------------------------------------------------------------------------- title: "NGINX Variables Index" url: https://docs.nginx.com/nginx/variables/index.md -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- title: "Restricting Access by Geographical Location" url: https://docs.nginx.com/nginx/admin-guide/security-controls/controlling-access-by-geoip/index.md description: Control access or forward traffic to different upstream servers based on the client's geographical location, using the GeoIP2 dynamic module. -------------------------------------------------------------------------------- ## Introduction F5 NGINX Plus can differentiate users based on their geographical location. For example, you can have different website content for different countries, or you can restrict content distribution to a particular country or city. NGINX Plus uses third-party MaxMind databases to match the IP address of the user and its location. As soon as the geoposition is known, it is then possible to use geoip-based variables in the [map](https://nginx.org/en/docs/http/ngx_http_map_module.html) or the [split_clients](https://nginx.org/en/docs/http/ngx_http_split_clients_module.html) module. > **Note** MaxMind GeoLite Legacy databases are currently [discontinued](https://blog.maxmind.com/2018/01/discontinuation-of-the-geolite-legacy-databases), MaxMind GeoIP2 or GeoLite2 databases and NGINX Plus [GeoIP2 module](nginx/admin-guide/dynamic-modules/geoip2.md) should be used instead. Restricting by geographical location works both for HTTP and TCP/UDP protocols. ## Prerequisites - NGINX Plus [GeoIP2 dynamic module](nginx/admin-guide/dynamic-modules/geoip2.md) - [GeoIP2](https://www.maxmind.com/en/geoip2-databases) or [GeoLite2](https://dev.maxmind.com/geoip/geoip2/geolite2/) databases from MaxMind - (optional) [mmdblookup](http://maxmind.github.io/libmaxminddb/mmdblookup.html) utility that looks up an IP address in a MaxMind Database file ## Getting the Databases The GeoIP2 or GeoLite2 databases can be obtained from the [MaxMind download page](https://www.maxmind.com/en/geoip2-databases). In this example, the GeoLite2 free downloadable databases are used. To get and unpack GeoLite2 Country database: ```shell wget http://geolite.maxmind.com/download/geoip/database/GeoLite2-Country.mmdb.gz gunzip GeoLite2-Country.mmdb.gz ``` To get and unpack GeoLite2 City database: ```shell wget http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.mmdb.gz gunzip GeoLite2-City.mmdb.gz ``` ## Understanding Database Structure To see the available geodata, you can query the GeoLite2-Country and GeoLite2-City databases with the [mmdblookup](http://maxmind.github.io/libmaxminddb/mmdblookup.html) utility. The geodata is represented as the JSON tree. Install the [libmaxminddb](http://maxmind.github.io/libmaxminddb/index.html) database utility: - For Amazon Linux, CentOS, Oracle Linux, and RHEL: ```shell yum install libmaxminddb-devel ``` - For Debian and Ubuntu: ```shell apt-get install libmaxminddb-dev ``` - For SLES: ```shell zypper install libmaxminddb-devel ``` A query to the databases can be sent in the following format: ```none mmdblookup –file [FILE PATH] –ip [IP ADDRESS] [DATA PATH] ``` For example, to get all available geodata for the `8.8.8.8` IP address, send the following command: ```shell mmdblookup --file /usr/local/etc/geoip2/GeoLite2-Country.mmdb --ip 8.8.8.8 ``` The output will be: ```json { "continent": { "code": "NA" "geoname_id": 6255149 "names": { "de": "Nordamerika" "en": "North America" "es": "Norteamérica" "fr": "Amérique du Nord" "ja": "北アメリカ" "pt-BR": "América do Norte" "ru": "Северная Америка" "zh-CN": "北美洲" } } "country": { "geoname_id": 6252001 "iso_code": "US" "names": { "de": "USA" "en": "United States" "es": "Estados Unidos" "fr": "États-Unis" "ja": "アメリカ合衆国" "pt-BR": "Estados Unidos" "ru": "США" "zh-CN": "美国" } } "registered_country": { "geoname_id": 6252001 "iso_code": "US" "names": { "de": "USA" "en": "United States" "es": "Estados Unidos" "fr": "États-Unis" "ja": "アメリカ合衆国" "pt-BR": "Estados Unidos" "ru": "США" "zh-CN": "美国" } } } ``` To get particular geodata, for example, only the ISO code of a particular country, add the `country iso_code` parameters to the end of the command: ```shell mmdblookup --file /usr/local/etc/geoip2/GeoLite2-Country.mmdb --ip 8.8.8.8 country iso_code ``` These parameters are also used when creating variables in the GeoIP2 module for NGINX. ## Configuring GeoIP2 in NGINX Plus 1. Install the GeoIP2 dynamic module for NGINX Plus: For Amazon Linux, CentOS, Oracle Linux, and RHEL: ```shell yum install nginx-plus-module-geoip2 ``` For Debian and Ubuntu: ```shell apt-get install nginx-plus-module-geoip2 ``` For SLES: ```shell zypper install nginx-plus-module-geoip2 ``` 2. Enable the GeoIP2 dynamic module in the NGINX Plus configuration file with the [load_module](https://nginx.org/en/docs/ngx_core_module.html#load_module) directive specified in the `main` configuration level: ```nginx load_module modules/ngx_http_geoip2_module.so; load_module modules/ngx_stream_geoip2_module.so; http { # ... } ``` 3. Add the paths to the country and city databases to the NGINX configuration with the `geoip2 {}` block for [`http {}`](https://nginx.org/en/docs/http/ngx_http_core_module.html#http), [`stream {}`](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#stream), or both: ```nginx http { #... geoip2 GeoIP2/GeoLite2-Country.mmdb { #... } geoip2 GeoIP2/GeoLite2-City.mmdb { #... } } stream { #... geoip2 GeoIP2/GeoLite2-Country.mmdb { #... } geoip2 GeoIP2/GeoLite2-City.mmdb { #... } } ``` 4. Basing on the [GeoIP database structure](#mmdblookup), create custom variables that will keep the data from the GeoIP2 database and then later pass the data to the [map](https://nginx.org/en/docs/http/ngx_http_map_module.html) or [split_clients](https://nginx.org/en/docs/http/ngx_http_split_clients_module.html) directives (can be applied in both the [`http {}`](https://nginx.org/en/docs/http/ngx_http_core_module.html#http) and [`stream {}`](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#stream) contexts): ```nginx geoip2 GeoIP2/GeoLite2-City.mmdb { $geoip2_data_city_name city names en; $geoip2_data_postal_code postal code; $geoip2_data_latitude location latitude; $geoip2_data_longitude location longitude; $geoip2_data_state_name subdivisions 0 names en; $geoip2_data_state_code subdivisions 0 iso_code; } geoip2 GeoIP2/GeoLite2-Country.mmdb { $geoip2_data_continent_code continent code; $geoip2_data_country_iso_code country iso_code; } #... ``` ## Scenario: Choosing the Nearest Server Using the geolocation data from the created variables, a client connection can be redirected to the closest server, thus reducing network latency and improving connection speed. This can be achieved by using the continent code from the GeoIP2 database in a variable and the [map](https://nginx.org/en/docs/http/ngx_http_map_module.html) module that will create another variable whose value will be the closest server basing on a continent location. Basing on this value, NGINX will pass the request to the corresponding upstream server group. 1. Make sure you have configured the servers or [upstream server groups](nginx/admin-guide/load-balancer/http-load-balancer.md) for each continent, for example, `eu` for Europe, `na` for North America, `all` for cases when the IP address cannot be matched against the GeoIP database: ```nginx upstream all { server all1.example.com:12345; server all2.example.com:12345; } upstream eu { server eu1.example.com:12345; server eu2.example.com:12345; } upstream na { server na1.example.com:12345; server na2.example.com:12345; } ``` 2. Add the `geoip2 {}` block with a variable of any name (for example, `$geoip2_data_continent_code`) that obtains the continent code of the GeoIP2 database: ```nginx geoip2 GeoIP2/GeoLite2-Country.mmdb { $geoip2_data_continent_code continent code; } #... ``` 3. Create the [map](https://nginx.org/en/docs/http/ngx_http_map_module.html) block that will create the `$nearest_server` variable: ```nginx #... map $geoip2_data_continent_code $nearest_server { default all; EU eu; NA na; AS as; AF af; } #... ``` 3. Create the [`server {}`](https://nginx.org/en/docs/http/ngx_http_core_module.html#server) block which will pass the requests to one of the upstream server groups according to the value passed in the `$nearest_server` variable: ```nginx server { listen 12346; proxy_pass http://$nearest_server; } ``` If the continent is Europe, then the value of the `$nearest_server` will be `eu`, and the connection will be passed to the `eu` upstream via the [proxy_pass](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass) directive: ```nginx #... server { listen 12346; proxy_pass http://$nearest_server; } upstream all { server all1.example.com:12345; server all2.example.com:12345; upstream eu { server eu1.example.com:12345; server eu2.example.com:12345; } upstream na { server na1.example.com:12345; server na2.example.com:12345; } #... ``` ### Example This example can be applied in both the [http](https://nginx.org/en/docs/http/ngx_http_core_module.html#http) and [stream](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#stream) contexts. ```nginx # can be either "http {}" or "stream {}" #... geoip2 GeoIP2/GeoLite2-Country.mmdb { $geoip2_data_continent_code continent code; } map $geoip2_data_continent_code $nearest_server { default all; EU eu; NA na; AS as; AF af; } server { listen 12346; proxy_pass http://$nearest_server; } upstream all { server all1.example.com:12345; server all2.example.com:12345; } upstream eu { server eu1.example.com:12345; server eu2.example.com:12345; } upstream na { server na1.example.com:12345; server na2.example.com:12345; } ``` In this example, the IP address will be checked in the `GeoLite2-Country.mmdb` database, the result will be written to the `$geoip2_data_continent_code` variable. NGINX Plus will match the value of the variable against values in the [map](https://nginx.org/en/docs/http/ngx_http_map_module.html#map) directive and write the result in the custom variable, in our example `$nearest_server`. Basing on the value of the `$nearest_server`, the [proxy_pass](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass) directive will choose a corresponding upstream server. ## More Info - [GeoIP2 Dynamic Module Installation Instructions](/nginx/admin-guide/dynamic-modules/geoip2.md) - [MaxMind GeoIP2 Databases](https://www.maxmind.com/en/geoip2-databases) - [MaxMind Geolite2 Free Downloadable Databases](https://dev.maxmind.com/geoip/geoip2/geolite2/) -------------------------------------------------------------------------------- title: "Single Sign-On with Ping Identity" url: https://docs.nginx.com/nginx/deployment-guides/single-sign-on/ping-identity/index.md description: Enable OpenID Connect-based single sign-on (SSO) for applications proxied by NGINX Plus, using Ping Identity as the identity provider (IdP). -------------------------------------------------------------------------------- This guide explains how to enable single sign-on (SSO) for applications being proxied by F5 NGINX Plus. The solution uses OpenID Connect as the authentication mechanism, with [Ping Identity](https://www.pingidentity.com/en.html) (PingFederate or PingOne) as the Identity Provider (IdP), and NGINX Plus as the Relying Party. **Note:** This guide applies to [NGINX Plus Release 36](nginx/releases.md#r36) and later. In earlier versions, NGINX Plus relied on an [njs-based solution](#legacy-njs-guide), which required NGINX JavaScript files, key-value stores, and advanced OpenID Connect logic. In the latest NGINX Plus version, the new [OpenID Connect module](https://nginx.org/en/docs/http/ngx_http_oidc_module.html) simplifies this process to just a few directives. ## Prerequisites - [PingFederate](https://docs.pingidentity.com/pingfederate/latest/pf_pf_landing_page.html) Enterprise Federation Server or [PingOne](https://docs.pingidentity.com/pingone/p1_cloud__platform_main_landing_page.html) Cloud deployment with a Ping Identity account. - An NGINX Plus [subscription](https://www.f5.com/products/nginx/nginx-plus) and NGINX Plus [Release 36](nginx/releases.md#r36) or later. For installation instructions, see [Installing NGINX Plus](https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-plus/). - A domain name pointing to your NGINX Plus instance, for example, `demo.example.com`. ## Configure PingFederate or PingOne for Enterprise {#ping-create} **Note:** These steps outline an example with the cloud offering of **PingOne**. If you are using the on‑premises PingFederate, the user interface might slightly differ. Create a new application for NGINX Plus: 1. Log in to your Ping Identity admin console. 2. Go to **Applications** > **Applications**. 3. Click the **+** (plus) symbol to create a new OIDC Application. 4. On the New Application screen: - Enter the Name of your application, for example, `nginx-demo-app`. - Select the Application Type **OIDC Web App**. - Select **Save**. 5. In your OIDC application, Select the **Overview** tab: - in the **General** section, copy your **Client ID** and **Client Secret** values. You will need then later when configuring NGINX Plus. - In the **Connection Details** section, copy your **Issuer ID**. You will need it later when configuring NGINX Plus. For PingOne Cloud, the Issuer ID generally structured as `https://auth.pingone.com//as`. For PingFederate, the Issuer ID generally structured as `https://pingfederate.example.com:9031` appended with the realm path of your environment. 6. On the **Configuration** tab of your OIDC application: - In the **Redirect URIs** field, add the NGINX Plus callback URI, for example: `https://demo.example.com/oidc_callback`. - In the **Post Logout Redirect URIs** field, add the post logout redirect URI, for example: `https://demo.example.com/post_logout/`. - Select **Save**. 7. Assign the application to the appropriate **Groups** or **Users** who will be allowed to log in. ### Enable Front-Channel Logout (optional) {#ping-frontchannel-logout} Front-channel logout allows Ping Identity to notify NGINX Plus when a user signs out of other Single Logout (SLO)-participating applications or from their Ping portal. Ping sends a front-channel HTTP request (typically loaded in a hidden iframe) to a logout URL that you configure in the application settings. This feature is optional and requires NGINX Plus Release 36 or later, together with the `frontchannel_logout_uri` directive in the NGINX OIDC provider configuration. 1. In the Ping Identity admin console, open the OIDC application that you created earlier (for example, `nginx-demo-app`). 2. Locate the **Front-channel logout** or **Single Logout** settings for the application. In PingOne, these are available under the **Configuration** or **Policies** sections; in PingFederate they are available in the OAuth / OIDC application connection settings. The exact location and labels can vary between product versions. 3. Configure the front-channel logout URL to point to the NGINX Plus front-channel logout endpoint, for example: ```text https://demo.example.com/front_logout ``` 4. Ensure that the front-channel logout request includes both the issuer (`iss`) and the session identifier (`sid`) as query parameters, in accordance with the OpenID Connect front-channel logout specification. If your Ping Identity admin console exposes an option to **include session details** or **send session ID and issuer** in logout requests, enable it. 5. Save the application configuration. ### Get the OpenID Connect Discovery URL Check the OpenID Connect Discovery URL. By default, Ping Identity publishes the `.well-known/openid-configuration` document at the following address: For PingOne: `https://auth.pingone.com//as/.well-known/openid-configuration` For PingFederate: `https://pingfederate.example.com:9031//.well-known/openid-configuration` 1. Run the following `curl` command in a terminal: ```shell curl https://auth.pingone.com//as/.well-known/openid-configuration | jq ``` Where: - the `auth.pingone.com` is your PingOne server address (or your PingFederate server for on-premises) - the `` is your PingOne environment ID - the `/as` is the authorization server path - the `/.well-known/openid-configuration` is the default address for Ping Identity for document location - the `jq` command (optional) is used to format the JSON output for easier reading and requires the [jq](https://jqlang.github.io/jq/) JSON processor to be installed. The configuration metadata is returned in the JSON format: ```json { ... "issuer": "https://auth.pingone.com//as", "authorization_endpoint": "https://auth.pingone.com//as/authorize", "token_endpoint": "https://auth.pingone.com//as/token", "jwks_uri": "https://auth.pingone.com//as/jwks", "userinfo_endpoint": "https://auth.pingone.com//as/userinfo", "end_session_endpoint": "https://auth.pingone.com//as/signoff", ... } ``` 2. Copy the **issuer** value, you will need it later when configuring NGINX Plus. Typically, the OpenID Connect Issuer for PingOne is `https://auth.pingone.com//as`. **Note:** You will need the values of **Client ID**, **Client Secret**, and **Issuer** in the next steps. ## Set up NGINX Plus {#nginx-plus-setup} With PingOne or PingFederate configured, you can enable OIDC on NGINX Plus. NGINX Plus serves as the Rely Party (RP) application — a client service that verifies user identity. 1. Ensure that you are using the latest version of NGINX Plus by running the `nginx -v` command in a terminal: ```shell nginx -v ``` The output should match NGINX Plus Release 36 or later: ```none nginx version: nginx/1.29.3 (nginx-plus-r36) ``` 2. Ensure that you have the values of the **Client ID**, **Client Secret**, and **Issuer** obtained during [PingOne or PingFederate Configuration](#ping-create). 3. In your preferred text editor, open the NGINX configuration file (`/etc/nginx/nginx.conf` for Linux or `/usr/local/etc/nginx/nginx.conf` for FreeBSD). 4. In the [`http {}`](https://nginx.org/en/docs/http/ngx_http_core_module.html#http) context, make sure your public DNS resolver is specified with the [`resolver`](https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver) directive: By default, NGINX Plus re‑resolves DNS records at the frequency specified by time‑to‑live (TTL) in the record, but you can override the TTL value with the `valid` parameter: ```nginx http { resolver 10.0.0.1 ipv4=on valid=300s; # ... } ``` 5. In the [`http {}`](https://nginx.org/en/docs/http/ngx_http_core_module.html#http) context, define the PingOne or PingFederate provider named `ping` by specifying the [`oidc_provider {}`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#oidc_provider) context: ```nginx http { resolver 10.0.0.1 ipv4=on valid=300s; oidc_provider ping { # ... } # ... } ``` 6. In the [`oidc_provider {}`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#oidc_provider) context, specify: - your actual Ping **Client ID** obtained in [Ping Configuration](#ping-create) with the [`client_id`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_id) directive - your **Client Secret** obtained in [Ping Configuration](#ping-create) with the [`client_secret`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_secret) directive - the **Issuer** URL obtained in [Ping Configuration](#ping-create) with the [`issuer`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_secret) directive The `issuer` is typically your Ping Identity OIDC URL. For PingOne Cloud, the URL is `https://auth.pingone.com//as`. For PingFederate, the URL is `https://pingfederate.example.com:9031` followed by your environment's realm path. By default, NGINX Plus creates the OpenID metadata URL by appending the `/.well-known/openid-configuration` part to the Issuer URL. If your metadata URL is different, you can explicitly specify the metadata document with the [`config_url`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#config_url) directive. - The **logout_uri** is URI that a user visits to start an RP‑initiated logout flow. - The **post_logout_uri** is absolute HTTPS URL where Ping Identity should redirect the user after a successful logout. This value **must also be configured** in the Ping Identity application's Post Logout Redirect URIs. - If the **logout_token_hint** directive set to `on`, NGINX Plus sends the user's ID token as a *hint* to Ping Identity. This directive is **required** by PingOne. - The **frontchannel_logout_uri** directive defines the URI that receives OpenID Connect front-channel logout requests from Ping Identity. This URI must be an HTTPS path and must match the front-channel logout URL configured for the application in Ping Identity. When Ping sends a front-channel logout GET request (typically via a hidden iframe) to this URI with the `iss` and `sid` query parameters, the OIDC module clears the corresponding user session on NGINX Plus. - If the **userinfo** directive is set to `on`, NGINX Plus will fetch userinfo from Ping Identity and append the claims from userinfo to the `$oidc_claims_` variables. - PKCE (Proof Key for Code Exchange) is automatically enabled when the provider's OpenID Connect discovery document advertises the `S256` code challenge method in the `code_challenge_methods_supported` field. You can override this behavior with the [`pkce`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#pkce) directive: set `pkce off;` to disable PKCE even when `S256` is advertised, or `pkce on;` to force PKCE even if the IdP's metadata does not list `S256`. - The OIDC module automatically selects the client authentication method for the token endpoint based on the provider metadata `token_endpoint_auth_methods_supported`. When only `client_secret_post` is advertised, NGINX Plus uses the `client_secret_post` method and sends the client credentials in the POST body. When both `client_secret_basic` and `client_secret_post` are present, the module prefers HTTP Basic (`client_secret_basic`), which remains the default for Ping Identity. - **Note:** All interaction with the IdP is secured exclusively over SSL/TLS, so NGINX must trust the certificate presented by the IdP. By default, this trust is validated against your system’s CA bundle (the default CA store for your Linux or FreeBSD distribution). If the IdP’s certificate is not included in the system CA bundle, you can explicitly specify a trusted certificate or chain with the [`ssl_trusted_certificate`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#ssl_trusted_certificate) directive so that NGINX can validate and trust the IdP’s certificate. ```nginx http { resolver 10.0.0.1 ipv4=on valid=300s; oidc_provider ping { issuer https://auth.pingone.com//as; client_id ; client_secret ; logout_uri /logout; logout_token_hint on; post_logout_uri https://demo.example.com/post_logout/; frontchannel_logout_uri /front_logout; userinfo on; # Optional: PKCE configuration. By default, PKCE is automatically # enabled when the IdP advertises the S256 code challenge method. # pkce on; } # ... } ``` 7. Make sure you have configured a [server](https://nginx.org/en/docs/http/ngx_http_core_module.html#server) that corresponds to `demo.example.com`, and there is a [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) that [points](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass) to your application (see [Step 10](#oidc_app)) at `http://127.0.0.1:8080` that is going to be OIDC-protected: ```nginx http { # ... server { listen 443 ssl; server_name demo.example.com; ssl_certificate /etc/ssl/certs/fullchain.pem; ssl_certificate_key /etc/ssl/private/key.pem; location / { # ... proxy_pass http://127.0.0.1:8080; } } # ... } ``` 8. Protect this [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) with Ping Identity OIDC by specifying the [`auth_oidc`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#auth_oidc) directive that will point to the `ping` configuration specified in the [`oidc_provider {}`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#oidc_provider) context in [Step 5](#ping-setup-oidc-provider): ```nginx # ... location / { auth_oidc ping; # ... proxy_pass http://127.0.0.1:8080; } # ... ``` 9. Pass the OIDC claims as headers to the application ([Step 10](#oidc_app)) with the [`proxy_set_header`](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_header) directive. These claims are extracted from the ID token returned by Ping: - [`$oidc_claim_sub`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) - a unique `Subject` identifier assigned for each user by Ping Identity - [`$oidc_claim_email`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) - the e-mail address of the user - [`$oidc_claim_name`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) - the full name of the user - any other OIDC claim using the [`$oidc_claim_ `](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#var_oidc_claim_) variable **Note:** Ensure the `openid`, `profile`, `email` Scopes are enabled in Ping Identity. ```nginx # ... location / { auth_oidc ping; proxy_set_header sub $oidc_claim_sub; proxy_set_header email $oidc_claim_email; proxy_set_header name $oidc_claim_name; proxy_pass http://127.0.0.1:8080; } # ... ``` 10. Provide endpoint for completing logout: ```nginx # ... location /post_logout/ { return 200 "You have been logged out.\n"; default_type text/plain; } # ... ``` 11. Create a simple test application referenced by the `proxy_pass` directive which returns the authenticated user's full name and email upon successful authentication: ```nginx # ... server { listen 8080; location / { return 200 "Hello, $http_name!\nEmail: $http_email\nPing Identity sub: $http_sub\n"; default_type text/plain; } } ``` 12. Save the NGINX configuration file and reload the configuration: ```nginx nginx -s reload ``` ### Complete Example This configuration example summarizes the steps outlined above. It includes only essential settings such as specifying the DNS resolver, defining the OIDC provider, configuring SSL, and proxying requests to an internal server. ```nginx http { # Use a public DNS resolver for Issuer discovery, etc. resolver 10.0.0.1 ipv4=on valid=300s; oidc_provider ping { # The issuer is typically something like: # https://auth.pingone.com//as issuer https://auth.pingone.com//as; # Your Ping Identity Client ID and Secret client_id ; client_secret ; # RP‑initiated logout logout_uri /logout; post_logout_uri https://demo.example.com/post_logout/; logout_token_hint on; # Front-channel logout (OP‑initiated single sign-out) frontchannel_logout_uri /front_logout; # Fetch userinfo claims userinfo on; # Optional: PKCE configuration (enabled automatically when supported by the IdP) # pkce on; } server { listen 443 ssl; server_name demo.example.com; ssl_certificate /etc/ssl/certs/fullchain.pem; ssl_certificate_key /etc/ssl/private/key.pem; location / { # Enforce OIDC with Ping Identity auth_oidc ping; # Forward OIDC claims as headers if desired proxy_set_header sub $oidc_claim_sub; proxy_set_header email $oidc_claim_email; proxy_set_header name $oidc_claim_name; proxy_pass http://127.0.0.1:8080; } location /post_logout/ { return 200 "You have been logged out.\n"; default_type text/plain; } } server { # Simple test upstream server listen 8080; location / { return 200 "Hello, $http_name!\nEmail: $http_email\nPing Identity sub: $http_sub\n"; default_type text/plain; } } } ``` ### Testing 1. Open `https://demo.example.com/` in a browser. You will be automatically redirected to the PingOne sign-in page. 2. Enter valid Ping Identity credentials of a user who has access the application. Upon successful sign-in, PingOne redirects you back to NGINX Plus, and you will see the proxied application content (for example, "Hello, Jane Doe!"). 3. Navigate to `https://demo.example.com/logout`. NGINX Plus initiates an RP‑initiated logout; Ping Identity ends the session and redirects back to `https://demo.example.com/post_logout/`. 4. Refresh `https://demo.example.com/` again. You should be redirected to Ping Identity for a fresh sign‑in, proving the session has been terminated. ## Legacy njs-based Ping Identity Solution {#legacy-njs-guide} If you are running NGINX Plus R33 and earlier or if you still need the njs-based solution, refer to the [Legacy njs-based Ping Identity Guide](nginx/deployment-guides/single-sign-on/oidc-njs/ping-identity.md) for details. The solution uses the [`nginx-openid-connect`](https://github.com/nginxinc/nginx-openid-connect) GitHub repository and NGINX JavaScript files. ## See Also - [NGINX Plus Native OIDC Module Reference documentation](https://nginx.org/en/docs/http/ngx_http_oidc_module.html) - [Release Notes for NGINX Plus R36](nginx/releases.md#r36) ## Revision History - Version 3 (November 2025) – Updated for NGINX Plus R36; added front-channel logout support (`frontchannel_logout_uri`), PKCE configuration (`pkce` directive), and the `client_secret_post` token endpoint authentication method. - Version 2 (August 2025) – Updated for NGINX Plus R35; added RP‑initiated logout (`logout_uri`, `post_logout_uri`, `logout_token_hint`) and `userinfo` support. - Version 1 (March 2025) – Initial version for NGINX Plus Release 34. -------------------------------------------------------------------------------- title: "Securing HTTP Traffic to Upstream Servers" url: https://docs.nginx.com/nginx/admin-guide/security-controls/securing-http-traffic-upstream/index.md description: Secure HTTP traffic between NGINX or F5 NGINX Plus and upstream servers, using SSL/TLS encryption. -------------------------------------------------------------------------------- This article explains how to encrypt HTTP traffic between NGINX and an upstream group or a proxied server. ## Prerequisites - [NGINX Open Source](https://nginx.org/en/download.html) or [F5 NGINX Plus](https://nginx.com/products/) - A [proxied server](nginx/admin-guide/web-server/reverse-proxy.md) or an [upstream group of servers](nginx/admin-guide/load-balancer/http-load-balancer.md) - SSL certificates and a private key ## Obtain SSL server certificates {#serever_certs} You can purchase a server certificate from a trusted certificate authority (CA). You can create own internal CA with an [OpenSSL](https://www.openssl.org/) library and generate your own certificate instead. The server certificate, along with a private key, should be placed on each upstream server. ## Obtain an SSL client certificate {#client_certs} NGINX will identify itself to the upstream servers with an SSL client certificate. A trusted CA must sign the client certificate, which is configured on NGINX together with the corresponding private key. You will also need to configure the upstream servers to require client certificates for all incoming SSL connections, and to trust the CA that issued NGINX client certificate. Then, when NGINX connects to the upstream, it will provide its client certificate and the upstream server will accept it. ## Configure NGINX {#config} First, change the URL to an upstream group to support SSL connections. In the NGINX configuration file, specify the `https` protocol in the [proxy_pass](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass) directive: ```nginx location /upstream { proxy_pass https://backend.example.com; } ``` Add the client certificate and the key that will be used to authenticate NGINX on each upstream server with [proxy_ssl_certificate](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_certificate) and [proxy_ssl_certificate_key](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_certificate_key) directives: ```nginx location /upstream { proxy_pass https://backend.example.com; proxy_ssl_certificate /etc/nginx/client.pem; proxy_ssl_certificate_key /etc/nginx/client.key; } ``` If you use a self-signed certificate for an upstream or your own CA, also include the [proxy_ssl_trusted_certificate](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_trusted_certificate). The file must be in the PEM format. Optionally, include the [proxy_ssl_verify](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_verify) and [proxy_ssl_verfiy_depth](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_verify_depth) directives to have NGINX check the validity of the security certificates: ```nginx location /upstream { #... proxy_ssl_trusted_certificate /etc/nginx/trusted_ca_cert.crt; proxy_ssl_verify on; proxy_ssl_verify_depth 2; #... } ``` Each new SSL connection requires a full SSL handshake between the client and server, which is quite CPU-intensive. To have NGINX proxy previously negotiated connection parameters and use a so-called abbreviated handshake, include the [proxy_ssl_session_reuse](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_session_reuse) directive: ```nginx location /upstream { #... proxy_ssl_session_reuse on; #... } ``` Optionally, you can specify which SSL protocols and ciphers are used: ```nginx location /upstream { #... proxy_ssl_protocols TLSv1.2 TLSv1.3; proxy_ssl_ciphers HIGH:!aNULL:!MD5; } ``` ## Configure upstream servers {#config_upstream} Each upstream server should be configured to accept HTTPS connections. For each upstream server, specify a path to the server certificate and the private key with [ssl_certificate](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_certificate) and [ssl_certificate_key](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_certificate_key) directives: ```nginx server { listen 443 ssl; server_name backend1.example.com; ssl_certificate /etc/ssl/certs/server.crt; ssl_certificate_key /etc/ssl/certs/server.key; #... location /yourapp { proxy_pass https://url_to_app.com; #... } } ``` Specify the path to a client certificate with the [ssl_client_certificate](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_client_certificate) directive: ```nginx server { #... ssl_client_certificate /etc/ssl/certs/ca.crt; ssl_verify_client optional; #... } ``` ## Complete example {#example} ```nginx http { #... upstream backend.example.com { server backend1.example.com:443; server backend2.example.com:443; } server { listen 80; server_name www.example.com; #... location /upstream { proxy_pass https://backend.example.com; proxy_ssl_certificate /etc/nginx/client.pem; proxy_ssl_certificate_key /etc/nginx/client.key; proxy_ssl_protocols TLSv1.2 TLSv1.3; proxy_ssl_ciphers HIGH:!aNULL:!MD5; proxy_ssl_trusted_certificate /etc/nginx/trusted_ca_cert.crt; proxy_ssl_verify on; proxy_ssl_verify_depth 2; proxy_ssl_session_reuse on; } } server { listen 443 ssl; server_name backend1.example.com; ssl_certificate /etc/ssl/certs/server.crt; ssl_certificate_key /etc/ssl/certs/server.key; ssl_client_certificate /etc/ssl/certs/ca.crt; ssl_verify_client optional; location /yourapp { proxy_pass https://url_to_app.com; #... } server { listen 443 ssl; server_name backend2.example.com; ssl_certificate /etc/ssl/certs/server.crt; ssl_certificate_key /etc/ssl/certs/server.key; ssl_client_certificate /etc/ssl/certs/ca.crt; ssl_verify_client optional; location /yourapp { proxy_pass https://url_to_app.com; #... } } } ``` In this example, the “`https`” protocol in the [proxy_pass](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass) directive specifies that the traffic forwarded by NGINX to upstream servers be secured. When a secure connection is passed from NGINX to the upstream server for the first time, the full handshake process is performed: - The [proxy_ssl_certificate](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_certificate) directive defines the location of the PEM-format certificate required by the upstream server. - The [proxy_ssl_certificate_key](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_certificate_key) directive defines the location of the certificate’s private key. - The [proxy_ssl_protocols](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_protocols) and [proxy_ssl_ciphers](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_ciphers) directives control which protocols and ciphers are used. The next time NGINX passes a connection to the upstream server, session parameters will be reused because of the [proxy_ssl_session_reuse](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_session_reuse) directive, and the secured connection is established faster. The trusted CA certificates in the file named by the [proxy_ssl_trusted_certificate](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_trusted_certificate) directive are used to verify the certificate on the upstream. The [proxy_ssl_verify_depth](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_verify_depth) directive specifies that two certificates in the certificates chain are checked. The [proxy_ssl_verify](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_verify) directive verifies the validity of certificates. -------------------------------------------------------------------------------- title: "Securing TCP Traffic to Upstream Servers" url: https://docs.nginx.com/nginx/admin-guide/security-controls/securing-tcp-traffic-upstream/index.md description: Secure TCP traffic between NGINX or F5 F5 NGINX Plus and upstream servers, using SSL/TLS encryption. -------------------------------------------------------------------------------- This article explains how to secure TCP traffic between NGINX and an upstream TCP server or group of upstream TCP servers. ## Prerequisites - F5 NGINX Plus [R6](nginx/releases.md) or later, or the latest NGINX Open Source compiled with the `--with-stream` and `with-stream_ssl_module` configuration parameters - A proxied TCP server or an [upstream group of TCP servers](nginx/admin-guide/load-balancer/tcp-udp-load-balancer.md) - SSL certificates and a private key ## Obtaining SSL Server Certificates First, obtain SSL server certificates and a private key. Obtain an SSL server certificate from a trusted certificate authority (CA). Alternatively, generate one using an SSL library such as [OpenSSL](http://www.openssl.org/). Place the server certificates and private key on each of the upstream servers. Self-signed server certificates encrypt the connection between NGINX and the upstream server. However, these connections are vulnerable to a man-in-the-middle attack. (If an imposter impersonates the upstream server, NGINX will not know it is talking to a fake server.) To lessen the risk, obtain server certificates signed by a trusted CA. (You can create your own internal CA using OpenSSL.) Afterward, configure NGINX to only trust certificates signed by that CA. This makes it much more difficult for an attacker to impersonate an upstream server. ## Obtaining an SSL Client Certificate NGINX can identify itself to the upstream servers using an SSL client certificate. A trusted CA must sign this client certificate. It must be stored on NGINX along with the corresponding private key. Configure the upstream servers to require client certificates for all incoming SSL connections and trust the CA that issued the client certificate to NGINX. By doing so, NGINX provides its client certificate when it connects to an upstream server, and the server will accept the certificate. ## Configuring NGINX Open the NGINX configuration file. Then, include the [proxy_ssl](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl) directive in the `server` block on the `stream` level: ```nginx stream { server { ... proxy_pass backend; proxy_ssl on; } } ``` Specify the path to the SSL client certificate required by the upstream server. Also, specify the certificate’s private key: ```nginx server { ... proxy_ssl_certificate /etc/ssl/certs/backend.crt; proxy_ssl_certificate_key /etc/ssl/certs/backend.key; } ``` Optionally, specify which SSL protocols and ciphers to use: ```nginx server { ... proxy_ssl_protocols TLSv1.2 TLSv1.3; proxy_ssl_ciphers HIGH:!aNULL:!MD5; } ``` The [proxy_ssl_trusted_certificate](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_trusted_certificate) directive specifies the file containing the trusted CA certificates. These trusted certificates are used to validate the upstream server’s security certificates. The file must be in PEM format. Optionally, include the [proxy_ssl_verify](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_verify) and [proxy_ssl_verfiy_depth](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_verify_depth) directives so NGINX vaidates the security certificates: ```nginx server { ... proxy_ssl_trusted_certificate /etc/ssl/certs/trusted_ca_cert.crt; proxy_ssl_verify on; proxy_ssl_verify_depth 2; } ``` New SSL connections require a full SSL handshake between the client and server. This is quite CPU-intensive. NGINX can proxy previously negotiated connection parameters and use a so-called abbreviated handshake. This lessens the CPU load for new SSL connections. To enable this, include the [proxy_ssl_session_reuse](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_session_reuse) directive: ```nginx proxy_ssl_session_reuse on; ``` ## Complete Example ```nginx stream { upstream backend { server backend1.example.com:12345; server backend2.example.com:12345; server backend3.example.com:12345; } server { listen 12345; proxy_pass backend; proxy_ssl on; proxy_ssl_certificate /etc/ssl/certs/backend.crt; proxy_ssl_certificate_key /etc/ssl/certs/backend.key; proxy_ssl_protocols TLSv1.2 TLSv1.3; proxy_ssl_ciphers HIGH:!aNULL:!MD5; proxy_ssl_trusted_certificate /etc/ssl/certs/trusted_ca_cert.crt; proxy_ssl_verify on; proxy_ssl_verify_depth 2; proxy_ssl_session_reuse on; } } ``` In this example, the [proxy_ssl](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl) directive specifies to secure the TCP traffic NGINX forwards to upstream servers. A full handshake occurs when NGINX first secures a TCP connection to an upstream server. The upstream server asks NGINX to present a security certificate specified in the [proxy_ssl_certificate](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_certificate) directive. The [proxy_ssl_protocols](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_protocols) and [proxy_ssl_ciphers](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_ciphers) directives specify the respective protocols and ciphers. The [proxy_ssl_session_reuse](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_session_reuse) directive causes subsequent upstream connections to reuse the session parameters. This makes establishing a secured TCP connection faster. The [proxy_ssl_trusted_certificate](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_trusted_certificate) directive names a file containing trusted CA certificates. Use these certificates to verify the certificate on the upstream server. The [proxy_ssl_verify_depth](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_verify_depth) directive specifies to check two certificates in the certificates chain. The [proxy_ssl_verify](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_verify) directive verifies the validity of certificates. To learn more about NGINX Plus, please see our [commercial subscriptions](https://nginx.com/products/). -------------------------------------------------------------------------------- title: "Dynamic Denylisting of IP Addresses" url: https://docs.nginx.com/nginx/admin-guide/security-controls/denylisting-ip-addresses/index.md description: Control access to your site or apps from specific client IP addresses, using dynamic denylists built with the F5 NGINX Plus key-value store and API. -------------------------------------------------------------------------------- This section describes how to create a denylist or allowlist of specific client IP addresses, which denies or allows them access to your site, and how to dynamically maintain the list of addresses. ## Overview In F5 NGINX Plus [Release 13](nginx/releases.md#r13) (R13) and later, you can denylist some IP addresses as well as create and maintain a database of denylisted IP addresses. You can also explicitly allowlist other IP addresses. The IP addresses database is managed with the NGINX Plus API and keyval modules. NGINX Plus [Release 19](nginx/releases.md#r19) extends this capability by matching an IP address to any address within a subnet or network range. ## Prerequisites NGINX Plus [Release 13](nginx/releases.md#r13) and later, NGINX Plus [Release 19](nginx/releases.md#r19) and later for network ranges support. ## Setup First, enable the database for storing the list of denylisted and allowlisted IP addresses. 1. In NGINX Plus configuration file, include the [keyval_zone](https://nginx.org/en/docs/http/ngx_http_keyval_module.html#keyval_zone) directive in the [http](https://nginx.org/en/docs/http/ngx_http_core_module.html#http) context to create a memory zone for storing keys and values. This sample directive creates a 1‑MB zone called **one**. ```nginx http { # ... keyval_zone zone=one:1m; } ``` To perform matching of an IP address against subnets (for example, `192.168.13.0/24`), specify the `type=ip` parameter of the [keyval_zone](https://nginx.org/en/docs/http/ngx_http_keyval_module.html#keyval_zone) directive: ```nginx http { # ... keyval_zone zone=one:1m type=ip; } ``` Note that the size of [keyval_zone](https://nginx.org/en/docs/http/ngx_http_keyval_module.html#keyval_zone) should also be increased as the `type=ip` parameter also enables an extra index stored in the zone. You can optionally include the `state` parameter to create a file where the key‑value database is stored and so persists across NGINX Plus reloads and restarts; in this example, **one.keyval**: ```nginx keyval_zone zone=one:1m state=one.keyval; ``` 2. Enable the NGINX Plus API in read‑write mode with the [api](https://nginx.org/en/docs/http/ngx_http_api_module.html#api) directive: ```nginx # ... server { listen 80; server_name www.example.com; location /api { api write=on; } } ``` We strongly recommend [restricting access](/nginx/admin-guide/security-controls/controlling-access-proxied-http.md) to this location, for example by allowing access only from `localhost` (`127.0.0.1`), and by using HTTP basic authentication to restrict use of the `PATCH`, `POST`, and `DELETE` methods to a specified set of users: ```nginx # ... server { listen 80; server_name www.example.com; location /api { api write=on; allow 127.0.0.1; deny all; limit_except GET { auth_basic "NGINX Plus API"; auth_basic_user_file /path/to/passwd/file; } } } ``` 3. Populate the key‑value database with the API's [POST](https://nginx.org/en/docs/http/ngx_http_api_module.html#postHttpKeyvalZoneData) method, supplying the data in JSON format. You can use the `curl` command as in the following example. If the zone is empty, you can enter several key‑value pairs at once; otherwise, pairs must be added one at a time. ```shell $ curl -X POST -d '{ "10.0.0.1": "1", "10.0.0.2": "1", "10.0.0.3": "0", "10.0.0.4": "0" }' -s http://www.example.com/api/6/http/keyvals/one ``` If you have specified matching of IP addresses against network ranges (with the `type=ip` parameter of the [keyval_zone](https://nginx.org/en/docs/http/ngx_http_keyval_module.html#keyval_zone) directive), send the `POST` command with the network range specified in CIDR notation: ```shell $ curl -X POST -d '{ "192.168.13.0/24": "1" }' -s http://www.example.com/api/6/http/keyvals/one ``` 4. Define how client IP addresses are evaluated against the key‑value database, by including the [keyval](https://nginx.org/en/docs/http/ngx_http_keyval_module.html#keyval) directive in the [http](https://nginx.org/en/docs/http/ngx_http_core_module.html#http) context. The directive takes advantage of the standard NGINX and NGINX Plus variable [`$remote_addr`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_remote_addr), which is set to the client IP address automatically for every request. As it processes each request, NGINX Plus: - Looks up the first parameter (here, `$remote_addr`, preset to the client's IP address) in the key‑value database specified by the `zone=` parameter (here, **one**). - If a key in the database exactly matches `$remote_addr`, sets the second parameter (here, `$target`) to the value corresponding to the key. In our example, the value is `1` for denylisted addresses or `0` for allowlisted addresses. ```nginx http { # ... keyval_zone zone=one:1m type=ip state=one.keyval; keyval $remote_addr $target zone=one; # Client address is the key, # $target is the value; } ``` 5. Create a rule with the [if](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#if) directive that either allows or denies access depending on the client IP address. With this rule, access is allowed when `$target` is `0` and denied when it is `1`: ```nginx if ($target) { return 403; } ``` ## Managing the Key-Value Database You can use API methods to update a key‑value database dynamically, without requiring a reload of NGINX Plus. All of the following examples operate on the **one** zone, which is accessible at ****. - To get the list of all database entries for a zone: ```shell curl -X GET 'http://www.example.com/api/6/http/keyvals/one' ``` - To update the value for an existing entry (in this example to change the access status for IP address `10.0.0.4` from allowlisted to denylisted): ```shell curl -X PATCH -d '{"10.0.0.4": "1"}' -s 'http://www.example.com/api/6/http/keyvals/one' ``` - To add an entry to a populated zone: ```shell curl -X POST -d '{"10.0.0.5": "1"}' -s 'http://www.example.com/api/6/http/keyvals/one' ``` - To delete an entry: ```shell curl -X PATCH -d '{"10.0.0.4":null}' -s 'http://www.example.com/api/6/http/keyvals/one' ``` ## Full Example The full NGINX Plus configuration: ```nginx http { # ... keyval_zone zone=one:1m type=ip state=one.keyval; keyval $remote_addr $target zone=one; server { listen 80; server_name www.example.com; location /api { api write=on; allow 127.0.0.1; deny all; limit_except GET { auth_basic "NGINX Plus API"; auth_basic_user_file /path/to/passwd/file; } } if ($target) { return 403; } } } ``` This configuration: - Creates a 1 MB keyval zone **one** that accepts network ranges and also creates the file **one.keyval** to make the database of key‑value pairs persists across reloads and restarts of NGINX Plus. - Enables the NGINX Plus API in write mode so that the zone can populated with IP addresses. - Enables lookup of the IP address `$remote_addr` in the key-value database as the key, and puts the value of the found key into the `$target` variable. - Enables a simple rule to check for the resulting value: if the value of `$target` is `1` (address is denylisted), return `403 (Forbidden)` to the client. The following `curl` command populates the empty keyval zone **one** with IP addresses that are denylisted (value is `1`) or allowlisted (value is `0`): ```shell curl -X POST -d '{ "10.0.0.1": "1", "192.168.13.0/24": "1", "10.0.0.3": "0", "10.0.0.4": "0" }' -s 'http://www.example.com/api/6/http/keyvals/one' ``` ## See Also - [Dynamic IP Denylisting with NGINX Plus and fail2ban](https://www.nginx.com/blog/dynamic-ip-denylisting-with-nginx-plus-and-fail2ban/) -------------------------------------------------------------------------------- title: "NGINX Control REST API reference" url: https://docs.nginx.com/nginx/admin-guide/basic-functionality/control-api-reference/index.md -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- title: "NGINX Plus REST API reference" url: https://docs.nginx.com/nginx/admin-guide/monitoring/nginx-plus-api-reference/index.md -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- title: "F5 DoS for NGINX" url: https://docs.nginx.com/nginx-app-protect-dos/index.md -------------------------------------------------------------------------------- ## About Achieve comprehensive protection against DoS and DDoS attacks for your apps and APIs with a multi-layered, adaptive, automated mitigation strategy for DevOps environments. Running natively on NGINX Plus and NGINX Ingress Controller, F5 DoS for NGINX is platform-agnostic and supports deployment options ranging from edge load balancers to individual pods in Kubernetes clusters. ## Featured content [//]: # "You can add a maximum of three cards: any extra will not display." [//]: # "One card will take full width page: two will take half width each. Three will stack like an inverse pyramid." [//]: # "Some examples of content could be the latest release note, the most common install path, and a popular new feature." ### [Deployment](/nginx-app-protect-dos/deployment-guide/learn-about-deployment/) Read how to install and upgrade F5 DoS for NGINX ### [Troubleshooting](/nginx-app-protect-dos/troubleshooting/how-to-troubleshoot/) Learn how to debug F5 DoS for NGINX ### [Releases](/nginx-app-protect-dos/releases/) Review changelogs for F5 DoS for NGINX -------------------------------------------------------------------------------- title: "F5 DoS for NGINX 4.10" url: https://docs.nginx.com/nginx-app-protect-dos/releases/about-4.10/index.md description: Release notes for F5 DoS for NGINX 4.10, including support for NGINX Plus R37.1. -------------------------------------------------------------------------------- F5 DoS for NGINX provides behavioral protection against Denial of Service (DoS) attacks for your web applications. ## Release 4.10 August 26, 2026 ### New features - NGINX Plus R37.1 support ### Supported packages | Distribution name | Package file | |----------------------------|-------------------------------------------------------| | Alpine 3.21 | _app-protect-dos-37+4.10.0-r1.apk_ | | Alpine 3.22 | _app-protect-dos-37+4.10.0-r1.apk_ | | Amazon Linux 2023 | _app-protect-dos-37+4.10.0-1.amzn2023.ngx.x86_64.rpm_ | | RHEL 8 and Rocky Linux 8 | _app-protect-dos-37+4.10.0-1.el8.ngx.x86_64.rpm_ | | RHEL 9 and Rocky Linux 9 | _app-protect-dos-37+4.10.0-1.el9.ngx.x86_64.rpm_ | | RHEL 10 and Rocky Linux 10 | _app-protect-dos-37+4.10.0-1.el10.ngx.x86_64.rpm_ | | Debian 11 | _app-protect-dos_37+4.10.0-1\~bullseye_amd64.deb_ | | Debian 12 | _app-protect-dos_37+4.10.0-1\~bookworm_amd64.deb_ | | Debian 13 | _app-protect-dos_37+4.10.0-1\~trixie_amd64.deb_ | | Ubuntu 22.04 | _app-protect-dos_37+4.10.0-1\~jammy_amd64.deb_ | | Ubuntu 24.04 | _app-protect-dos_37+4.10.0-1\~noble_amd64.deb_ | | NGINX Plus | _NGINX Plus R37.1_ | ### Important notes - F5 DoS for NGINX 4.10 requires NGINX Plus **R37.1**, whereas 4.9 requires **R37.0**. Each `nginx-plus-module-appprotectdos` package depends on a virtual package named `nginx-plus-r` that only its own NGINX Plus release provides, so F5 DoS for NGINX and NGINX Plus must be upgraded together. - On Debian and Ubuntu, `apt` considers only the newest `nginx-plus` available and will not select an older one on its own. When installing a specific F5 DoS for NGINX version, pin `nginx-plus` as well, or the install fails with `Unable to correct problems, you have held broken packages`. See the [F5 DoS for NGINX Deployment Guide](/nap-dos/deployment-guide/learn-about-deployment.md) for the exact commands. - If you use the L4 accelerated mitigation feature, pin `app-protect-dos-ebpf-manager` to the same version as `app-protect-dos`. Left unpinned, it installs the most recent version, which may not match the rest of your installation. -------------------------------------------------------------------------------- title: "F5 DoS for NGINX 4.9" url: https://docs.nginx.com/nginx-app-protect-dos/releases/about-4.9/index.md description: Release notes for F5 DoS for NGINX 4.9, including new platform support for Debian 13, RHEL 10, Rocky Linux 10, and NGINX Plus R37.0. -------------------------------------------------------------------------------- F5 DoS for NGINX provides behavioral protection against Denial of Service (DoS) attacks for your web applications. ## Release 4.9 May 13, 2026 ### New features - NGINX Plus R37.0 support - Debian 13 support - RHEL 10 and Rocky Linux 10 support - Bug fixes ### Supported packages | Distribution name | Package file | |----------------------------|------------------------------------------------------| | Alpine 3.21 | _app-protect-dos-37+4.9.6-r1.apk_ | | Alpine 3.22 | _app-protect-dos-37+4.9.6-r1.apk_ | | Amazon Linux 2023 | _app-protect-dos-37+4.9.6-1.amzn2023.ngx.x86_64.rpm_ | | RHEL 8 and Rocky Linux 8 | _app-protect-dos-37+4.9.6-1.el8.ngx.x86_64.rpm_ | | RHEL 9 and Rocky Linux 9 | _app-protect-dos-37+4.9.6-1.el9.ngx.x86_64.rpm_ | | RHEL 10 and Rocky Linux 10 | _app-protect-dos-37+4.9.6-1.el10.ngx.x86_64.rpm_ | | Debian 11 | _app-protect-dos_37+4.9.6-1\~bullseye_amd64.deb_ | | Debian 12 | _app-protect-dos_37+4.9.6-1\~bookworm_amd64.deb_ | | Debian 13 | _app-protect-dos_37+4.9.6-1\~trixie_amd64.deb_ | | Ubuntu 22.04 | _app-protect-dos_37+4.9.6-1\~jammy_amd64.deb_ | | Ubuntu 24.04 | _app-protect-dos_37+4.9.6-1\~noble_amd64.deb_ | | NGINX Plus | _NGINX Plus R37.0_ | -------------------------------------------------------------------------------- title: "F5 DoS for NGINX 4.8" url: https://docs.nginx.com/nginx-app-protect-dos/releases/about-4.8/index.md -------------------------------------------------------------------------------- Here you can find the release information for F5 DoS for NGINX v4.8 F5 DoS for NGINX provides behavioral protection against Denial of Service (DoS) for your web applications. ## Release 4.8 December 1, 2025 ### New features - R36 support - Add support for Alpine 3.22 - Remove support for Alpine 3.19 - Bugs fixing ### Supported packages | Distribution name | Package file | |--------------------------|------------------------------------------------------| | Alpine 3.21 | _app-protect-dos-36+4.8.3-r1.apk_ | | Alpine 3.22 | _app-protect-dos-36+4.8.3-r1.apk_ | | Amazon Linux 2023 | _app-protect-dos-36+4.8.3-1.amzn2023.ngx.x86_64.rpm_ | | RHEL 8 and Rocky Linux 8 | _app-protect-dos-36+4.8.3-1.el8.ngx.x86_64.rpm_ | | RHEL 9 and Rocky Linux 9 | _app-protect-dos-36+4.8.3-1.el9.ngx.x86_64.rpm_ | | Debian 11 | _app-protect-dos_36+4.8.3-1\~bullseye_amd64.deb_ | | Debian 12 | _app-protect-dos_36+4.8.3-1\~bookworm_amd64.deb_ | | Ubuntu 22.04 | _app-protect-dos_36+4.8.3-1\~jammy_amd64.deb_ | | Ubuntu 24.04 | _app-protect-dos_36+4.8.3-1\~noble_amd64.deb_ | | NGINX Plus | _NGINX Plus R36_ | -------------------------------------------------------------------------------- title: "F5 DoS for NGINX 4.7" url: https://docs.nginx.com/nginx-app-protect-dos/releases/about-4.7/index.md -------------------------------------------------------------------------------- Here you can find the release information for F5 DoS for NGINX v4.7 F5 DoS for NGINX provides behavioral protection against Denial of Service (DoS) for your web applications. ## Release 4.7 August 13, 2025 ### New features - R35 support - Add support for Alpine 3.21 - Add support for Rocky 9 - Remove support for Ubuntu 20.04 (EOL) - Bugs fixing ### Supported packages | Distribution name | Package file | |--------------------------|------------------------------------------------------| | Alpine 3.19 | _app-protect-dos-35+4.7.3-r1.apk_ | | Alpine 3.21 | _app-protect-dos-35+4.7.3-r1.apk_ | | Amazon Linux 2023 | _app-protect-dos-35+4.7.3-1.amzn2023.ngx.x86_64.rpm_ | | RHEL 8 and Rocky Linux 8 | _app-protect-dos-35+4.7.3-1.el8.ngx.x86_64.rpm_ | | RHEL 9 and Rocky Linux 9 | _app-protect-dos-35+4.7.3-1.el9.ngx.x86_64.rpm_ | | Debian 11 | _app-protect-dos_35+4.7.3-1\~bullseye_amd64.deb_ | | Debian 12 | _app-protect-dos_35+4.7.3-1\~bookworm_amd64.deb_ | | Ubuntu 22.04 | _app-protect-dos_35+4.7.3-1\~jammy_amd64.deb_ | | Ubuntu 24.04 | _app-protect-dos_35+4.7.3-1\~noble_amd64.deb_ | | NGINX Plus | _NGINX Plus R35_ | -------------------------------------------------------------------------------- title: "F5 DoS for NGINX 4.6" url: https://docs.nginx.com/nginx-app-protect-dos/releases/about-4.6/index.md -------------------------------------------------------------------------------- Here you can find the release information for F5 DoS for NGINX v4.6 F5 DoS for NGINX provides behavioral protection against Denial of Service (DoS) for your web applications. --- ## Release 4.6 April 1, 2025 --- ### New features - Support for NGINX Plus R34 - Add support for AmazonLinux 2023 - Remove support for Alpine 3.17 - Bug fixes --- ### Supported packages | Distribution name | Package file | |--------------------------|------------------------------------------------------| | Alpine 3.19 | _app-protect-dos-34+4.6.1-r1.apk_ | | Amazon Linux 2023 | _app-protect-dos-34+4.6.1-1.amzn2023.ngx.x86_64.rpm_ | | RHEL 8 and Rocky Linux 8 | _app-protect-dos-34+4.6.1-1.el8.ngx.x86_64.rpm_ | | RHEL 9 | _app-protect-dos-34+4.6.1-1.el9.ngx.x86_64.rpm_ | | Debian 11 | _app-protect-dos_34+4.6.1-1\~bullseye_amd64.deb_ | | Debian 12 | _app-protect-dos_34+4.6.1-1\~bookworm_amd64.deb_ | | Ubuntu 20.04 | _app-protect-dos_34+4.6.1-1\~focal_amd64.deb_ | | Ubuntu 22.04 | _app-protect-dos_34+4.6.1-1\~jammy_amd64.deb_ | | Ubuntu 24.04 | _app-protect-dos_34+4.6.1-1\~noble_amd64.deb_ | | NGINX Plus | _NGINX Plus R34_ | -------------------------------------------------------------------------------- title: "F5 DoS for NGINX Arbitrator 1.2.0" url: https://docs.nginx.com/nginx-app-protect-dos/releases/about-1.2.0/index.md -------------------------------------------------------------------------------- Here you can find the release information for F5 DoS for NGINX Arbitrator v1.2.0 ## Arbitrator Service Release 1.2.0 Mar 24, 2025 ### Enhanced Sync Mechanism in ADMD Based on Arbitrator Version In this release, ADMD now dynamically selects the appropriate synchronization method based on the Arbitrator's version: Arbitrator 1.2.0 and above – ADMD uses Batch Sync, allowing all VSs to be synchronized in a single request, improving efficiency and reducing overhead. Arbitrator versions below 1.2.0 – ADMD continues to use per-VS Sync, ensuring compatibility with earlier versions. This enhancement optimizes synchronization performance while maintaining backward compatibility. -------------------------------------------------------------------------------- title: "F5 DoS for NGINX 4.5" url: https://docs.nginx.com/nginx-app-protect-dos/releases/about-4.5/index.md -------------------------------------------------------------------------------- Here you can find the release information for F5 DoS for NGINX v4.5 F5 DoS for NGINX provides behavioral protection against Denial of Service (DoS) for your web applications. --- ## Release 4.5 Nov 19, 2024 F5 DoS for NGINX 4.5 adds support for NGINX Plus R33. --- ### New features - Support for NGINX Plus R33 - Add support for Alpine 3.19 - Add support for Ubuntu 24.04 - Remove support for CentOS 7 / RHEL 7 - *eBPF Manager - Privileged Process for Secure Command Handling* This feature introduces the eBPF Manager, a process that securely handles eBPF commands on behalf of other processes. By allowing certain processes to operate without elevated privileges, it enhances system security. As part of this enhancement, NGINX now runs under a non-root user account, which is required for deploying specific security solutions. --- ### Supported packages | Distribution name | Package file | |--------------------------|--------------------------------------------------| | Alpine 3.17 / 3.19 | _app-protect-dos-33+4.5.2-r1.apk_ | | RHEL 8 and Rocky Linux 8 | _app-protect-dos-33+4.5.2-1.el8.ngx.x86_64.rpm_ | | RHEL 9 | _app-protect-dos-33+4.5.2-1.el9.ngx.x86_64.rpm_ | | Debian 11 | _app-protect-dos_33+4.5.2-1\~bullseye_amd64.deb_ | | Debian 12 | _app-protect-dos_33+4.5.2-1\~bookworm_amd64.deb_ | | Ubuntu 20.04 | _app-protect-dos_33+4.5.2-1\~focal_amd64.deb_ | | Ubuntu 22.04 | _app-protect-dos_33+4.5.2-1\~jammy_amd64.deb_ | | Ubuntu 24.04 | _app-protect-dos_33+4.5.2-1\~noble_amd64.deb_ | | NGINX Plus | _NGINX Plus R33_ | -------------------------------------------------------------------------------- title: "F5 DoS for NGINX 4.4" url: https://docs.nginx.com/nginx-app-protect-dos/releases/about-4.4/index.md -------------------------------------------------------------------------------- Here you can find the release information for F5 DoS for NGINX v4.4. F5 DoS for NGINX provides behavioral protection against Denial of Service (DoS) for your web applications. ## Release 4.4 May 29, 2024 In this release, F5 DoS for NGINX supports NGINX Plus R32. ### New Features - Support for NGINX Plus R32 - Support for Debian 12 - Support for RHEL 9 - [Allowlisted IPs](/nap-dos/directives-and-policy/learn-about-directives-and-policy.md#access-file-directive-app_protect_dos_access_file) - Support for up to 1000 protected objects, ensuring high scalability and efficient resource utilization. ### Supported Packages #### App Protect DoS ##### Alpine 3.17 - app-protect-dos-32+4.4.1-r1.apk ##### CentOS 7.4+ / RHEL 7.4+ - app-protect-dos-32+4.4.1-1.el7.ngx.x86_64.rpm ##### RHEL 8 and Rocky Linux 8 - app-protect-dos-32+4.4.1-1.el8.ngx.x86_64.rpm ##### RHEL 9 - app-protect-dos-32+4.4.1-1.el9.ngx.x86_64.rpm ##### Debian 11 - app-protect-dos_32+4.4.1-1~bullseye_amd64.deb ##### Debian 12 - app-protect-dos_32+4.4.1-1~bookworm_amd64.deb ##### Ubuntu 20.04 - app-protect-dos_32+4.4.1-1~focal_amd64.deb ##### Ubuntu 22.04 - app-protect-dos_32+4.4.1-1~jammy_amd64.deb #### NGINX Plus - NGINX Plus R32 -------------------------------------------------------------------------------- title: "Deployment" url: https://docs.nginx.com/nginx-app-protect-dos/deployment-guide/learn-about-deployment/index.md description: Learn about F5 DoS for NGINX Deployment. -------------------------------------------------------------------------------- ## Overview F5 DoS for NGINX provides behavioral protection against DoS for your web applications.

This guide explains how to deploy F5 DoS for NGINX as well as upgrade App Protect DoS. ## Prerequisites F5 DoS for NGINX is available to the customers as a downloadable dynamic module at an additional cost. To purchase or add F5 DoS for NGINX to an existing NGINX Plus subscription, contact the NGINX sales team. NGINX Plus Release 24 and later supports F5 DoS for NGINX. F5 DoS for NGINX supports the following operating systems: - [RHEL 8.1+ / Rocky Linux 8](#rhel-8--rocky-linux-8-installation) - [RHEL 9.0+ / Rocky Linux 9](#rhel-9--rocky-linux-9-installation) - [Debian 11 (Bullseye)](#debian--ubuntu-installation) - [Debian 12 (Bookworm)](#debian--ubuntu-installation) - [Ubuntu 22.04 (Jammy)](#debian--ubuntu-installation) - [Ubuntu 24.04 (Noble)](#debian--ubuntu-installation) - [Alpine 3.21](#alpine-installation) - [Alpine 3.22](#alpine-installation) - [AmazonLinux 2023](#amazon-linux-2023-installation) The F5 DoS for NGINX package has the following dependencies: 1. **nginx-plus-module-appprotectdos** - NGINX Plus dynamic module for App Protect DoS 2. **libcurl** - Software library for HTTP access 3. **zeromq4** - Software library for fast, message-based applications 4. **boost** - The free peer-reviewed portable C++ source libraries 5. **openssl** - Toolkit for the Transport Layer Security (TLS) and Secure Sockets Layer (SSL) protocol 6. **libelf** - Software library for ELF access See the NGINX Plus full list of prerequisites for more details. F5 DoS for NGINX can be installed as a module to an existing NGINX Plus installation or as a complete NGINX Plus with App Protect DoS installation in a clean environment or to a system with F5 WAF for NGINX. **Note:** - gRPC, HTTP/2 and WebSocket protection require active monitoring of the protected service. The directive `app_protect_dos_monitor` is mandatory for the attack to be detected. - Monitor directive `app_protect_dos_monitor` with proxy_protocol parameter can not be configured on Ubuntu 18.04. As a result, gRPC and HTTP/2 DoS protection for proxy_protocol configuration is not supported. - Regularly update the Operating System (OS) to avoid known OS vulnerabilities which may impact the service. ## Platform Security Considerations When deploying App Protect DoS on NGINX Plus take the following precautions to secure the platform. This avoids the risk of causing a Denial of Service condition or compromising the platform security. - Restrict permissions to the files on the F5 DoS for NGINX platform to user **nginx** and group **nginx**, especially for the sensitive areas containing the configuration. - Remove unnecessary remote access services on the platform. - Configure a Syslog destination on the same machine as App Protect DoS and proxy to an external destination. This avoids eavesdropping and [man-in-the-middle](https://en.wikipedia.org/wiki/Man-in-the-middle_attack) attacks on the Syslog channel. ## Virtual machine or bare metal Deployment ### RHEL 8+ / Rocky Linux 8 Installation 1. If you already have NGINX packages in your system, back up your configs and logs: ```shell sudo cp -a /etc/nginx /etc/nginx-plus-backup sudo cp -a /var/log/nginx /var/log/nginx-plus-backup ``` 1. Create the **/etc/ssl/nginx** directory: ```shell sudo mkdir -p /etc/ssl/nginx ``` 1. Create the **/etc/nginx/** directory for the JWT license file: ```shell sudo mkdir -p /etc/nginx/ ``` 1. Download the SSL certificate, private key, and the JWT license file associated with your NGINX Plus subscription from the MyF5 Customer Portal: - Log in to [MyF5](https://my.f5.com/manage/s/). - Go to **My Products & Plans > Subscriptions** to see your active subscriptions. - Find your NGINX products or services subscription, and select the **Subscription ID** for details. - Download the **.crt** and **.key** from the subscription page. - Download the **JSON Web Token** (JWT) from the subscription page. See [About Subscription Licenses](/solutions/about-subscription-licenses.md) for details. 1. Copy the downloaded **.crt** and **.key** files to the **/etc/ssl/nginx/** directory and make sure they are named **nginx-repo.crt** and **nginx-repo.key**: ```shell sudo cp .crt /etc/ssl/nginx/nginx-repo.crt sudo cp .key /etc/ssl/nginx/nginx-repo.key ``` 1. Copy the downloaded JWT file to the **/etc/nginx/** directory and make sure it is named **license.jwt**: ```shell sudo cp .jwt /etc/nginx/license.jwt ``` 5. Install prerequisite packages: ```shell sudo dnf install ca-certificates wget 6. Enable Yum repositories to pull F5 DoS for NGINX dependencies: For RHEL subscription: ```shell sudo subscription-manager repos --enable=rhel-8-for-x86_64-baseos-rpms sudo subscription-manager repos --enable=rhel-8-for-x86_64-appstream-rpms sudo dnf -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm ``` For RockyLinux: ```shell sudo dnf -y install epel-release ``` 8. Add NGINX Plus and NGINX App Protect DoS repository: ```shell sudo wget -P /etc/yum.repos.d https://cs.nginx.com/static/files/nginx-plus-8.repo sudo wget -P /etc/yum.repos.d https://cs.nginx.com/static/files/app-protect-dos-8.repo ``` 9. In case of fresh installation, update the repository and install the most recent version of the NGINX Plus App Protect DoS package (which includes NGINX Plus): ```shell sudo dnf install app-protect-dos ``` For L4 accelerated mitigation feature (RHEL 8.6+): ```shell sudo dnf install app-protect-dos-ebpf-manager ``` **Note:** L4 accelerated mitigation feature (RHEL 8.6+): - `app-protect-dos-ebpf-manager` run with root privileges. Alternatively, you can use the following command to list available versions: ```shell sudo dnf --showduplicates list app-protect-dos ``` Then, install a specific version from the output of command above. For example: ```shell # F5 DoS for NGINX 4.10.0, which requires NGINX Plus R37.1 sudo dnf install app-protect-dos-37+4.10.0 # F5 DoS for NGINX 4.9.6, which requires NGINX Plus R37.0 sudo dnf install app-protect-dos-37+4.9.6 ``` If you use the L4 accelerated mitigation feature, pin `app-protect-dos-ebpf-manager` to the matching version. Left unpinned, it installs the most recent version, which may not match the rest of your installation: ```shell sudo dnf --showduplicates list app-protect-dos-ebpf-manager sudo dnf install app-protect-dos-ebpf-manager-37+4.9.6 ``` 10. In case of upgrading from previously installed NGINX Plus App Protect DoS package (which includes NGINX Plus): ```shell sudo dnf remove nginx-plus sudo dnf install app-protect-dos sudo systemctl start nginx ``` **Note:** Make sure to restore configuration from `/etc/nginx-plus-backup` back to `/etc/nginx-plus`. 1. Check the `nginx` version to verify that NGINX Plus is installed correctly: ```shell nginx -v ``` 11. Check the App Protect DoS binary version to ensure that you have the right version installed correctly: ```shell sudo admd -v ``` 12. Load the F5 DoS for NGINX module on the main context in the `nginx.conf` file: ```nginx load_module modules/ngx_http_app_protect_dos_module.so; ``` 13. Enable F5 DoS for NGINX in an `http/server/location` context in the `nginx.conf` file: ```nginx app_protect_dos_enable on; app_protect_dos_name "App1"; app_protect_dos_monitor uri=serv:80/; # Assuming server_name "serv" on port 80, with the root path "/" ``` 14. Enable the L4 accelerated mitigation feature (RHEL 8.6+) in an `http` context in the `nginx.conf` file: ```nginx app_protect_dos_accelerated_mitigation on; ``` 15. Configure the SELinux to allow App Protect DoS: a. Using the vi editor, create a file: ```shell vi app-protect-dos.te ``` b. Insert the following contents into the file that you have created: ```shell module app-protect-dos 2.0; require { type unconfined_t; type unconfined_service_t; type httpd_t; type tmpfs_t; type initrc_t; type initrc_state_t; class capability sys_resource; class shm { associate read unix_read unix_write write }; class file { read write }; } allow httpd_t initrc_state_t:file { read write }; allow httpd_t self:capability sys_resource; allow httpd_t tmpfs_t:file { read write }; allow httpd_t unconfined_service_t:shm { associate read unix_read unix_write write }; allow httpd_t unconfined_t:shm { associate read write unix_read unix_write }; allow httpd_t initrc_t:shm { associate read unix_read unix_write write }; ``` c. Run the following chain of commands: ```shell sudo checkmodule -M -m -o app-protect-dos.mod app-protect-dos.te && \ sudo semodule_package -o app-protect-dos.pp -m app-protect-dos.mod && \ sudo semodule -i app-protect-dos.pp; ``` For L4 accelerated mitigation feature: a. Using the vi editor, create a file: ```shell vi app-protect-dos-ebpf-manager.te ``` b. Insert the following contents into the file you have created: ```shell module app-protect-dos-ebpf-manager 1.0; require { type root_t; type httpd_t; type unconfined_service_t; class sock_file write; class unix_stream_socket connectto; class shm { unix_read unix_write }; } allow httpd_t root_t:sock_file write; allow httpd_t unconfined_service_t:shm { unix_read unix_write }; allow httpd_t unconfined_service_t:unix_stream_socket connectto; ``` c. Run the following chain of commands: ```shell sudo checkmodule -M -m -o app-protect-dos-ebpf-manager.mod app-protect-dos-ebpf-manager.te && \ sudo semodule_package -o app-protect-dos-ebpf-manager.pp -m app-protect-dos-ebpf-manager.mod && \ sudo semodule -i app-protect-dos-ebpf-manager.pp; ``` If you encounter any issues, refer to the [Troubleshooting Guide](/nap-dos/troubleshooting/how-to-troubleshoot.md). **Note:** Additional SELinux configuration may be required to allow NGINX Plus to listen on specific network ports, connect to upstreams, and send syslog entries to remote systems. Refer to the practices outlined in the [Using NGINX and NGINX Plus with SELinux](https://www.f5.com/company/blog/nginx/using-nginx-plus-with-selinux) article for details. 16. To enable the NGINX/App-Protect-DoS service to start at boot, run the command: ```shell sudo systemctl enable nginx.service ``` 17. Start the NGINX service: ```shell sudo systemctl start nginx ``` 18. L4 mitigation To enable the `app-protect-dos-ebpf-manager` service to start at boot, run the command: ```shell sudo systemctl enable nginx.service ``` Start the `app-protect-dos-ebpf-manager` service: ``` sudo systemctl start app-protect-dos-ebpf-manager ``` ### RHEL 9+ / Rocky Linux 9 Installation 1. If you already have NGINX packages on your system, back up your configs and logs: ```shell sudo cp -a /etc/nginx /etc/nginx-plus-backup sudo cp -a /var/log/nginx /var/log/nginx-plus-backup ``` 1. Create the **/etc/ssl/nginx** directory: ```shell sudo mkdir -p /etc/ssl/nginx ``` 1. Create the **/etc/nginx/** directory for the JWT license file: ```shell sudo mkdir -p /etc/nginx/ ``` 1. Download the SSL certificate, private key, and the JWT license file associated with your NGINX Plus subscription from the MyF5 Customer Portal: - Log in to [MyF5](https://my.f5.com/manage/s/). - Go to **My Products & Plans > Subscriptions** to see your active subscriptions. - Find your NGINX products or services subscription, and select the **Subscription ID** for details. - Download the **.crt** and **.key** from the subscription page. - Download the **JSON Web Token** (JWT) from the subscription page. See [About Subscription Licenses](/solutions/about-subscription-licenses.md) for details. 1. Copy the downloaded **.crt** and **.key** files to the **/etc/ssl/nginx/** directory and make sure they are named **nginx-repo.crt** and **nginx-repo.key**: ```shell sudo cp .crt /etc/ssl/nginx/nginx-repo.crt sudo cp .key /etc/ssl/nginx/nginx-repo.key ``` 1. Copy the downloaded JWT file to the **/etc/nginx/** directory and make sure it is named **license.jwt**: ```shell sudo cp .jwt /etc/nginx/license.jwt ``` 5. Install prerequisite packages: ```shell sudo dnf install ca-certificates wget ``` 6. Enable the yum repositories to pull F5 DoS for NGINX dependencies: For RHEL subscription: ```shell sudo subscription-manager repos --enable=rhel-9-for-x86_64-baseos-rpms sudo subscription-manager repos --enable=rhel-9-for-x86_64-appstream-rpms sudo dnf -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm ``` For RockyLinux: ```shell sudo dnf -y install epel-release ``` 7. Add the NGINX Plus and NGINX App Protect DoS repositories: ```shell sudo wget -P /etc/yum.repos.d https://cs.nginx.com/static/files/plus-9.repo sudo wget -P /etc/yum.repos.d https://cs.nginx.com/static/files/app-protect-dos-9.repo ``` 8. If you are performing a fresh installation, update the repository and install the most recent version of the NGINX Plus App Protect DoS package (which includes NGINX Plus): ```shell sudo dnf install app-protect-dos ``` For L4 accelerated mitigation feature (RHEL 9): ```shell sudo dnf install app-protect-dos-ebpf-manager ``` **Note:** L4 accelerated mitigation feature (RHEL 9): - `app-protect-dos-ebpf-manager` run with root privileges. Alternatively, you can use the following command to list available versions: ```shell sudo dnf --showduplicates list app-protect-dos ``` Then, install a specific version from the output of command above. For example: ```shell # F5 DoS for NGINX 4.10.0, which requires NGINX Plus R37.1 sudo dnf install app-protect-dos-37+4.10.0 # F5 DoS for NGINX 4.9.6, which requires NGINX Plus R37.0 sudo dnf install app-protect-dos-37+4.9.6 ``` If you use the L4 accelerated mitigation feature, pin `app-protect-dos-ebpf-manager` to the matching version. Left unpinned, it installs the most recent version, which may not match the rest of your installation: ```shell sudo dnf --showduplicates list app-protect-dos-ebpf-manager sudo dnf install app-protect-dos-ebpf-manager-37+4.9.6 ``` 9. In you are upgrading from previously installed NGINX Plus App Protect DoS package (which includes NGINX Plus): ```shell sudo dnf remove nginx-plus sudo dnf install app-protect-dos sudo systemctl start nginx ``` **Note:** Make sure to restore configuration from `/etc/nginx-plus-backup` back to `/etc/nginx-plus`. 10. Check the NGINX binary version to ensure that you have NGINX Plus installed correctly: ```shell sudo nginx -v ``` 11. Check the App Protect DoS binary version to ensure that you have the right version installed correctly: ```shell sudo admd -v ``` 12. Load the F5 DoS for NGINX module on the main context in the `nginx.conf`: ```nginx load_module modules/ngx_http_app_protect_dos_module.so; ``` 13. Enable F5 DoS for NGINX on an `http/server/location` context in the `nginx.conf` file: ```nginx app_protect_dos_enable on; app_protect_dos_name "App1"; app_protect_dos_monitor uri=serv:80/; # Assuming server_name "serv" on port 80, with the root path "/" ``` 14. Enable the L4 accelerated mitigation feature (RHEL 8.6+) in the `http` context of the `nginx.conf` file: ```nginx app_protect_dos_accelerated_mitigation on; ``` 15. Configure the SELinux to allow App Protect DoS: a. Using the vi editor, create a file: ```shell vi app-protect-dos.te ``` b. Insert the following contents into the file created above: ```shell module app-protect-dos 2.0; require { type unconfined_t; type unconfined_service_t; type httpd_t; type tmpfs_t; type initrc_t; type initrc_state_t; class capability sys_resource; class shm { associate read unix_read unix_write write }; class file { read write }; } allow httpd_t initrc_state_t:file { read write }; allow httpd_t self:capability sys_resource; allow httpd_t tmpfs_t:file { read write }; allow httpd_t unconfined_service_t:shm { associate read unix_read unix_write write }; allow httpd_t unconfined_t:shm { associate read write unix_read unix_write }; allow httpd_t initrc_t:shm { associate read unix_read unix_write write }; ``` c. Run the following chain of commands: ```shell sudo checkmodule -M -m -o app-protect-dos.mod app-protect-dos.te && \ sudo semodule_package -o app-protect-dos.pp -m app-protect-dos.mod && \ sudo semodule -i app-protect-dos.pp; ``` For L4 accelerated mitigation feature:
a. Using the vi editor, create a file: ```shell vi app-protect-dos-ebpf-manager.te ``` b. Insert the following contents into the file created above: ```shell module app-protect-dos-ebpf-manager 1.0; require { type root_t; type httpd_t; type unconfined_service_t; class sock_file write; class unix_stream_socket connectto; class shm { unix_read unix_write }; } allow httpd_t root_t:sock_file write; allow httpd_t unconfined_service_t:shm { unix_read unix_write }; allow httpd_t unconfined_service_t:unix_stream_socket connectto; ``` c. Run the following chain of commands: ```shell sudo checkmodule -M -m -o app-protect-dos-ebpf-manager.mod app-protect-dos-ebpf-manager.te && \ sudo semodule_package -o app-protect-dos-ebpf-manager.pp -m app-protect-dos-ebpf-manager.mod && \ sudo semodule -i app-protect-dos-ebpf-manager.pp; ``` If you encounter any issues, refer to the [Troubleshooting Guide](/nap-dos/troubleshooting/how-to-troubleshoot.md). **Note:** Additional SELinux configuration may be required to allow NGINX Plus to listen on specific network ports, connect to upstreams, and send syslog entries to remote systems. Refer to the practices outlined in the [Using NGINX and NGINX Plus with SELinux](https://www.f5.com/company/blog/nginx/using-nginx-plus-with-selinux/) article for details. 16. To enable the NGINX/App-Protect-DoS service to start at boot, run the command: ```shell sudo systemctl enable nginx.service ``` 17. Start the NGINX service: ```shell sudo systemctl start nginx ``` 18. L4 mitigation To enable the `app-protect-dos-ebpf-manager` service to start at boot, run the command: ```shell sudo systemctl enable nginx.service ``` Start the `app-protect-dos-ebpf-manager` service: ``` sudo systemctl start app-protect-dos-ebpf-manager ``` ### Debian / Ubuntu Installation 1. If you already have NGINX packages in your system, back up your configs and logs: ```shell sudo cp -a /etc/nginx /etc/nginx-plus-backup sudo cp -a /var/log/nginx /var/log/nginx-plus-backup ``` 1. Create the **/etc/ssl/nginx** directory: ```shell sudo mkdir -p /etc/ssl/nginx ``` 1. Create the **/etc/nginx/** directory for the JWT license file: ```shell sudo mkdir -p /etc/nginx/ ``` 1. Download the SSL certificate, private key, and the JWT license file associated with your NGINX Plus subscription from the MyF5 Customer Portal: - Log in to [MyF5](https://my.f5.com/manage/s/). - Go to **My Products & Plans > Subscriptions** to see your active subscriptions. - Find your NGINX products or services subscription, and select the **Subscription ID** for details. - Download the **.crt** and **.key** from the subscription page. - Download the **JSON Web Token** (JWT) from the subscription page. See [About Subscription Licenses](/solutions/about-subscription-licenses.md) for details. 1. Copy the downloaded **.crt** and **.key** files to the **/etc/ssl/nginx/** directory and make sure they are named **nginx-repo.crt** and **nginx-repo.key**: ```shell sudo cp .crt /etc/ssl/nginx/nginx-repo.crt sudo cp .key /etc/ssl/nginx/nginx-repo.key ``` 1. Copy the downloaded JWT file to the **/etc/nginx/** directory and make sure it is named **license.jwt**: ```shell sudo cp .jwt /etc/nginx/license.jwt ``` 5. Install appropriate packages with `apt`: For Debian: ```shell sudo apt-get install apt-transport-https lsb-release ca-certificates wget gnupg2 debian-archive-keyring ``` For Ubuntu: ```shell sudo apt-get install apt-transport-https lsb-release ca-certificates wget gnupg2 ubuntu-keyring ``` **Note:** In case the apt installation or database update fails due to release info change, run the below command before you install. ```shell sudo apt-get update --allow-releaseinfo-change ``` 6. Download and add the NGINX signing key: ```shell sudo wget -qO - https://cs.nginx.com/static/keys/nginx_signing.key | sudo gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null ``` 7. Add NGINX Plus and F5 DoS for NGINX repository: For Debian: ```shell printf "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] https://pkgs.nginx.com/plus/debian `lsb_release -cs` nginx-plus\n" | sudo tee /etc/apt/sources.list.d/nginx-plus.list printf "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] https://pkgs.nginx.com/app-protect-dos/debian `lsb_release -cs` nginx-plus\n" | sudo tee /etc/apt/sources.list.d/nginx-app-protect-dos.list ``` For Ubuntu: ```shell printf "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] https://pkgs.nginx.com/plus/ubuntu `lsb_release -cs` nginx-plus\n" | sudo tee /etc/apt/sources.list.d/nginx-plus.list printf "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] https://pkgs.nginx.com/app-protect-dos/ubuntu `lsb_release -cs` nginx-plus\n" | sudo tee /etc/apt/sources.list.d/nginx-app-protect-dos.list ``` 8. Download the apt configuration to `/etc/apt/apt.conf.d`: ```shell sudo wget -P /etc/apt/apt.conf.d https://cs.nginx.com/static/files/90pkgs-nginx ``` 9. In case of fresh Installation, update the repository and install the most recent version of the NGINX Plus App Protect DoS package (which includes NGINX Plus): ```shell sudo apt-get update sudo apt-get install app-protect-dos ``` For L4 accelerated mitigation feature (Debian 11 / Debian 12 / Ubuntu 22.04 / Ubuntu 24.04): ```shell sudo apt-get install app-protect-dos-ebpf-manager ``` **Note:** L4 accelerated mitigation feature (Debian 11 / Debian 12 / Ubuntu 22.04 / Ubuntu 24.04): - `app-protect-dos-ebpf-manager` run with root privileges. Alternatively, to install a specific version, use the following commands to update and list the available versions: ```shell sudo apt-get update sudo apt-cache policy app-protect-dos ``` Each `nginx-plus-module-appprotectdos` package is built for one NGINX Plus release and depends on a virtual package named `nginx-plus-r`, which only that release of `nginx-plus` provides. To find which NGINX Plus release a version requires, list the dependencies of each available module version: ```shell sudo apt-cache show nginx-plus-module-appprotectdos | grep -E '^Version:|nginx-plus-r' ``` To find which `nginx-plus` versions provide that release, list its reverse provides. Replace `r37.0` with the release reported by the previous command: ```shell sudo apt-cache showpkg nginx-plus-r37.0 ``` Finally, install `app-protect-dos`, `nginx-plus-module-appprotectdos`, and `nginx-plus` in a single command, pinning all three. **Note:** `apt` considers only the newest `nginx-plus` available and does not select an older one to satisfy another package's dependency. If the version you are installing was built for an earlier NGINX Plus release, the install fails with `Unable to correct problems, you have held broken packages` unless you pin `nginx-plus` as well. The unpinned `apt-get install app-protect-dos` shown earlier needs no pin only while the most recent `app-protect-dos` is built for the most recent NGINX Plus release. If a new NGINX Plus release is published before the F5 DoS for NGINX version that supports it, pin both packages until that version is available. The two most recent versions and the NGINX Plus release each one requires: | F5 DoS for NGINX package | NGINX Plus release | `nginx-plus` version | |--------------------------|--------------------|----------------------| | 37+4.10.0 | R37.1 | 37.1.* | | 37+4.9.6 | R37.0 | 37.0.* | Quote any version that contains a `*` so that the shell does not expand it as a filename pattern. For example for Debian 11: ```shell # F5 DoS for NGINX 4.10.0, which requires NGINX Plus R37.1 sudo apt-get install app-protect-dos=37+4.10.0-1~bullseye nginx-plus-module-appprotectdos=37+4.10.0-1~bullseye 'nginx-plus=37.1.*-1~bullseye' # F5 DoS for NGINX 4.9.6, which requires NGINX Plus R37.0 sudo apt-get install app-protect-dos=37+4.9.6-1~bullseye nginx-plus-module-appprotectdos=37+4.9.6-1~bullseye 'nginx-plus=37.0.*-1~bullseye' ``` For example, for Debian 12: ```shell # F5 DoS for NGINX 4.10.0, which requires NGINX Plus R37.1 sudo apt-get install app-protect-dos=37+4.10.0-1~bookworm nginx-plus-module-appprotectdos=37+4.10.0-1~bookworm 'nginx-plus=37.1.*-1~bookworm' # F5 DoS for NGINX 4.9.6, which requires NGINX Plus R37.0 sudo apt-get install app-protect-dos=37+4.9.6-1~bookworm nginx-plus-module-appprotectdos=37+4.9.6-1~bookworm 'nginx-plus=37.0.*-1~bookworm' ``` For example for Ubuntu 22.04: ```shell # F5 DoS for NGINX 4.10.0, which requires NGINX Plus R37.1 sudo apt-get install app-protect-dos=37+4.10.0-1~jammy nginx-plus-module-appprotectdos=37+4.10.0-1~jammy 'nginx-plus=37.1.*-1~jammy' # F5 DoS for NGINX 4.9.6, which requires NGINX Plus R37.0 sudo apt-get install app-protect-dos=37+4.9.6-1~jammy nginx-plus-module-appprotectdos=37+4.9.6-1~jammy 'nginx-plus=37.0.*-1~jammy' ``` For example for Ubuntu 24.04: ```shell # F5 DoS for NGINX 4.10.0, which requires NGINX Plus R37.1 sudo apt-get install app-protect-dos=37+4.10.0-1~noble nginx-plus-module-appprotectdos=37+4.10.0-1~noble 'nginx-plus=37.1.*-1~noble' # F5 DoS for NGINX 4.9.6, which requires NGINX Plus R37.0 sudo apt-get install app-protect-dos=37+4.9.6-1~noble nginx-plus-module-appprotectdos=37+4.9.6-1~noble 'nginx-plus=37.0.*-1~noble' ``` If you use the L4 accelerated mitigation feature, pin `app-protect-dos-ebpf-manager` to the matching version. Left unpinned, it installs the most recent version, which may not match the rest of your installation. The `lsb_release` call fills in the suite of the system you are running on: ```shell sudo apt-cache policy app-protect-dos-ebpf-manager sudo apt-get install app-protect-dos-ebpf-manager=37+4.9.6-1~`lsb_release -cs` ``` 10. In the case of upgrading from a previously installed NGINX Plus App Protect DoS package (which includes NGINX Plus): ```shell sudo apt-get update sudo apt-get remove nginx-plus sudo apt-get install app-protect-dos sudo systemctl start nginx ``` 11. Check the NGINX binary version to ensure that you have NGINX Plus installed correctly: ```shell sudo nginx -v ``` 12. Check the App Protect DoS binary version to ensure that you have the right version installed correctly: ```shell sudo admd -v ``` 13. Load the F5 DoS for NGINX module on the main context in the `nginx.conf` file: ```nginx load_module modules/ngx_http_app_protect_dos_module.so; ``` 14. Enable F5 DoS for NGINX on an `http/server/location` context in the `nginx.conf` via: ```nginx app_protect_dos_enable on; app_protect_dos_name "App1"; app_protect_dos_monitor uri=serv:80/; # Assuming server_name "serv" on port 80, with the root path "/" ``` 15. Enable the L4 accelerated mitigation feature (Debian 11 / Debian 12 / Ubuntu 22.04 / Ubuntu 24.04) on the `http` context of the `nginx.conf` file: ```nginx app_protect_dos_accelerated_mitigation on; ``` 16. Start the NGINX service: ```shell sudo systemctl start nginx ``` 17. Start the L4 service: ```shell sudo systemctl start app-protect-dos-ebpf-manager ``` ### Alpine Installation 1. If you already have NGINX packages in your system, back up your configs and logs: ```shell sudo cp -a /etc/nginx /etc/nginx-plus-backup sudo cp -a /var/log/nginx /var/log/nginx-plus-backup ``` 1. Create the **/etc/ssl/nginx** directory: ```shell sudo mkdir -p /etc/ssl/nginx ``` 1. Create the **/etc/nginx/** directory for the JWT license file: ```shell sudo mkdir -p /etc/nginx/ ``` 1. Download the SSL certificate, private key, and the JWT license file associated with your NGINX Plus subscription from the MyF5 Customer Portal: - Log in to [MyF5](https://my.f5.com/manage/s/). - Go to **My Products & Plans > Subscriptions** to see your active subscriptions. - Find your NGINX products or services subscription, and select the **Subscription ID** for details. - Download the **.crt** and **.key** from the subscription page. - Download the **JSON Web Token** (JWT) from the subscription page. See [About Subscription Licenses](/solutions/about-subscription-licenses.md) for details. 3. Upload `nginx-repo.key` to `/etc/apk/cert.key` and `nginx-repo.crt` to `/etc/apk/cert.pem`. Make sure that files do not contain other certificates and keys, as Alpine Linux does not support mixing client certificates for different repositories. 1. Copy the downloaded JWT file to the **/etc/nginx/** directory and make sure it is named **license.jwt**: ```shell sudo cp .jwt /etc/nginx/license.jwt ``` 4. Add the NGINX public signing key to the directory `/etc/apk/keys`: ```shell sudo wget -O /etc/apk/keys/nginx_signing.rsa.pub https://cs.nginx.com/static/keys/nginx_signing.rsa.pub ``` 5. Remove any previously configured NGINX Plus repository: ```shell sed "/plus-pkgs.nginx.com/d" /etc/apk/repositories ``` 6. Add NGINX Plus repository to `/etc/apk/repositories` file: ```shell printf "https://pkgs.nginx.com/plus/alpine/v`egrep -o '^[0-9]+\.[0-9]+' /etc/alpine-release`/main\n" | sudo tee -a /etc/apk/repositories ``` 7. Add F5 DoS for NGINX repository to `/etc/apk/repositories` file: ```shell printf "https://pkgs.nginx.com/app-protect-dos/alpine/v`egrep -o '^[0-9]+\.[0-9]+' /etc/alpine-release`/main\n" | sudo tee -a /etc/apk/repositories ``` 8. It is recommended to remove all community-supported NGINX packages. Note that all NGINX modules will be removed as well. ```shell sudo apk del -r app-protect-dos sudo apk del -r nginx ``` 9. Update the repository and install the most recent version of the NGINX Plus and F5 DoS for NGINX: ```shell sudo apk update sudo apk add nginx-plus app-protect-dos ``` For L4 accelerated mitigation feature: ```shell sudo apk add app-protect-dos-ebpf-manager ``` **Note:** L4 accelerated mitigation feature: - `app-protect-dos-ebpf-manager` run with root privileges. Alternatively, to install a specific version, use the following commands to update and list available versions: ```shell sudo apk update sudo apk info app-protect-dos ``` Finally, install a specific version from the output of command above. For example: ```shell # F5 DoS for NGINX 4.10.0, which requires NGINX Plus R37.1 sudo apk add nginx-plus app-protect-dos=37+4.10.0-r1 # F5 DoS for NGINX 4.9.6, which requires NGINX Plus R37.0 sudo apk add nginx-plus app-protect-dos=37+4.9.6-r1 ``` If you use the L4 accelerated mitigation feature, pin `app-protect-dos-ebpf-manager` to the matching version. Left unpinned, it installs the most recent version, which may not match the rest of your installation: ```shell sudo apk info app-protect-dos-ebpf-manager sudo apk add app-protect-dos-ebpf-manager=37+4.9.6-r1 ``` 10. In case of upgrading from previously installed NGINX Plus App Protect DoS package (which includes NGINX Plus): ```shell sudo apk update sudo apk del -r app-protect-dos sudo apk del -r nginx-plus sudo apk add nginx-plus app-protect-dos rc-service nginx-app-protect-dos start ``` 11. Check the NGINX binary version to ensure that you have NGINX Plus installed correctly: ```shell sudo nginx -v ``` 12. Check the App Protect DoS binary version to ensure that you have the right version installed correctly: ```shell sudo admd -v ``` 13. Load the F5 DoS for NGINX module on the main context in the `nginx.conf` file: ```nginx load_module modules/ngx_http_app_protect_dos_module.so; ``` 14. Enable F5 DoS for NGINX on an `http/server/location` context in the `nginx.conf` via: ```nginx app_protect_dos_enable on; app_protect_dos_name "App1"; app_protect_dos_monitor uri=serv:80/; # Assuming server_name "serv" on port 80, with the root path "/" ``` 15. Enable the L4 accelerated mitigation feature on the `http` context of the `nginx.conf` file: ```nginx app_protect_dos_accelerated_mitigation on; ``` 16. Start the NGINX service: ```shell rc-service nginx-app-protect-dos start ``` 17. Start the L4 service: ```shell rc-service app-protect-dos-ebpf-manager start ``` ### Amazon Linux 2023 Installation 1. If you already have NGINX packages in your system, back up your configs and logs: ```shell sudo cp -a /etc/nginx /etc/nginx-plus-backup sudo cp -a /var/log/nginx /var/log/nginx-plus-backup ``` 1. Create the **/etc/ssl/nginx** directory: ```shell sudo mkdir -p /etc/ssl/nginx ``` 1. Create the **/etc/nginx/** directory for the JWT license file: ```shell sudo mkdir -p /etc/nginx/ ``` 1. Download the SSL certificate, private key, and the JWT license file associated with your NGINX Plus subscription from the MyF5 Customer Portal: - Log in to [MyF5](https://my.f5.com/manage/s/). - Go to **My Products & Plans > Subscriptions** to see your active subscriptions. - Find your NGINX products or services subscription, and select the **Subscription ID** for details. - Download the **.crt** and **.key** from the subscription page. - Download the **JSON Web Token** (JWT) from the subscription page. See [About Subscription Licenses](/solutions/about-subscription-licenses.md) for details. 1. Copy the downloaded **.crt** and **.key** files to the **/etc/ssl/nginx/** directory and make sure they are named **nginx-repo.crt** and **nginx-repo.key**: ```shell sudo cp .crt /etc/ssl/nginx/nginx-repo.crt sudo cp .key /etc/ssl/nginx/nginx-repo.key ``` 1. Copy the downloaded JWT file to the **/etc/nginx/** directory and make sure it is named **license.jwt**: ```shell sudo cp .jwt /etc/nginx/license.jwt ``` 5. Install prerequisite packages: ```shell sudo dnf install ca-certificates wget 6. Add NGINX Plus and F5 DoS for NGINX repository: ```shell sudo wget -P /etc/yum.repos.d https://cs.nginx.com/static/files/plus-amazonlinux2023.repo sudo wget -P /etc/yum.repos.d https://cs.nginx.com/static/files/app-protect-dos-amazonlinux2023.repo ``` 7. In case of fresh installation, update the repository and install the most recent version of the NGINX Plus App Protect DoS package (which includes NGINX Plus): ```shell sudo dnf install app-protect-dos ``` For L4 accelerated mitigation feature: ```shell sudo dnf install app-protect-dos-ebpf-manager ``` **Note:** L4 accelerated mitigation feature: - `app-protect-dos-ebpf-manager` run with root privileges. Alternatively, you can use the following command to list available versions: ```shell sudo dnf --showduplicates list app-protect-dos ``` Then, install a specific version from the output of command above. For example: ```shell # F5 DoS for NGINX 4.10.0, which requires NGINX Plus R37.1 sudo dnf install app-protect-dos-37+4.10.0 # F5 DoS for NGINX 4.9.6, which requires NGINX Plus R37.0 sudo dnf install app-protect-dos-37+4.9.6 ``` If you use the L4 accelerated mitigation feature, pin `app-protect-dos-ebpf-manager` to the matching version. Left unpinned, it installs the most recent version, which may not match the rest of your installation: ```shell sudo dnf --showduplicates list app-protect-dos-ebpf-manager sudo dnf install app-protect-dos-ebpf-manager-37+4.9.6 ``` 8. In case of upgrading from previously installed NGINX Plus App Protect DoS package (which includes NGINX Plus): ```shell sudo dnf remove nginx-plus sudo dnf install app-protect-dos sudo systemctl start nginx ``` **Note:** Make sure to restore configuration from `/etc/nginx-plus-backup` back to `/etc/nginx-plus`. 9. Confirm the NGINX binary version to make sure that you have NGINX Plus installed correctly: ```shell sudo nginx -v ``` 10. Check the App Protect DoS binary version to ensure that you have the right version installed correctly: ```shell sudo admd -v ``` 11. Load the F5 DoS for NGINX module on the main context in the `nginx.conf` file: ```nginx load_module modules/ngx_http_app_protect_dos_module.so; ``` 12. Enable F5 DoS for NGINX in an `http/server/location` context in the `nginx.conf` file: ```nginx app_protect_dos_enable on; app_protect_dos_name "App1"; app_protect_dos_monitor uri=serv:80/; # Assuming server_name "serv" on port 80, with the root path "/" ``` 13. Enable the L4 accelerated mitigation feature in an `http` context in the `nginx.conf` file: ```nginx app_protect_dos_accelerated_mitigation on; ``` 14. To enable the NGINX/App-Protect-DoS service to start at boot, run the command: ```shell sudo systemctl enable nginx.service ``` 15. Start the NGINX service: ```shell sudo systemctl start nginx ``` 16. L4 mitigation To enable the `app-protect-dos-ebpf-manager` service to start at boot, run the command: ```shell sudo systemctl enable nginx.service ``` Start the `app-protect-dos-ebpf-manager` service: ``` sudo systemctl start app-protect-dos-ebpf-manager ``` ## Docker Deployment ### Docker Deployment Instructions You need root permissions to execute the following steps. 1. Create a Dockerfile (see examples below) which copies the following files into the docker image: - `nginx-repo.crt`: Certificate for NGINX repository access - `nginx-repo.key`: Private key for NGINX repository access - `license.jwt`: JWT license file for NGINX Plus license management - `nginx.conf`: User defined `nginx.conf` with `app-protect-dos` enabled - `entrypoint.sh`: Docker startup script which spins up all App Protect DoS processes, must have executable permissions - custom_log_format.json: Optional user-defined security log format file (if not used - remove its references from the nginx.conf and Dockerfile) 2. Log in to NGINX Plus Customer Portal and download your `nginx-repo.crt`, `nginx-repo.key` and `license.jwt` files. 3. Copy the files to the directory where the Dockerfile is located. 4. Add F5 DoS for NGINX to your `nginx.conf`. The configuration below is an example for an `http` and `grpc+tls` servers which has F5 DoS for NGINX enabled. Note that every F5 DoS for NGINX related directive starts with `app_protect_dos_`. `nginx.conf` ```nginx user nginx; worker_processes auto; error_log /var/log/nginx/error.log error; worker_rlimit_nofile 65535; working_directory /tmp/cores; load_module modules/ngx_http_app_protect_dos_module.so; # F5 DoS for NGINX module events { worker_connections 65535; } http { include /etc/nginx/mime.types; log_format log_napd ', vs_name_al=$app_protect_dos_vs_name, ip=$remote_addr, tls_fp=$app_protect_dos_tls_fp, ' 'outcome=$app_protect_dos_outcome, reason=$app_protect_dos_outcome_reason, ' 'ip_tls=$remote_addr:$app_protect_dos_tls_fp, '; app_protect_dos_security_log_enable on; # Enable F5 DoS for NGINX's security logger app_protect_dos_security_log "/etc/app_protect_dos/log-default.json" /var/log/adm/logger.log; # Security logger outputs to a file # app_protect_dos_security_log "/etc/app_protect_dos/log-default.json" syslog:server=1.2.3.4:5261; # Security logger outputs to a syslog destination # HTTP/1 server server { default_type application/octet-stream; listen 80 reuseport; server_name serv80; set $loggable '0'; access_log /var/log/nginx/access.log log_napd if=$loggable; # Access log with rate limiting and additional information # access_log syslog:server=1.1.1.1:5561 log_napd if=$loggable; app_protect_dos_policy_file "/etc/app_protect_dos/BADOSDefaultPolicy.json"; # Policy configuration for F5 DoS for NGINX location / { app_protect_dos_enable on; # Enable F5 DoS for NGINX in this block app_protect_dos_name "App80"; # PO name app_protect_dos_monitor uri=http://serv80/; # Health monitoring proxy_pass http://1.2.3.4:80; } } # gRPC server with ssl server { default_type application/grpc; listen 443 http2 ssl reuseport; server_name serv_grpc; # TLS config ssl_certificate /etc/ssl/certs/grpc.example.com.crt; ssl_certificate_key /etc/ssl/private/grpc.example.com.key; ssl_session_cache shared:SSL:10m; ssl_session_timeout 5m; ssl_ciphers HIGH:!aNULL:!MD5; ssl_protocols TLSv1.2 TLSv1.3; set $loggable '0'; access_log /var/log/nginx/access.log log_napd if=$loggable; #access_log syslog:server=1.1.1.1:5561 log_napd if=$loggable; location / { app_protect_dos_enable on; app_protect_dos_name "AppGRPC"; app_protect_dos_monitor uri=https://serv_grpc:443/service/method protocol=grpc; # mandatory for gRPC grpc_pass grpc://1.2.3.4:1001; } } sendfile on; tcp_nopush on; keepalive_timeout 65; } ``` **Note:** Make sure to replace upstream and proxy pass directives in this example with relevant application backend settings. 5. In the same directory create an `entrypoint.sh` file with executable permissions, with the following content: ```shell #!/usr/bin/env bash USER=nginx LOGDIR=/var/log/adm # prepare environment mkdir -p /var/run/adm /tmp/cores ${LOGDIR} chmod 755 /var/run/adm /tmp/cores ${LOGDIR} chown ${USER}:${USER} /var/run/adm /tmp/cores ${LOGDIR} # run processes /bin/su -s /bin/bash -c "/usr/bin/adminstall > ${LOGDIR}/adminstall.log 2>&1" ${USER} /bin/su -s /bin/bash -c "/usr/bin/admd -d --log info > ${LOGDIR}/admd.log 2>&1 &" ${USER} /usr/sbin/nginx -g 'daemon off;' ``` 6. Create a Docker image: ```shell DOCKER_BUILDKIT=1 docker build --no-cache --platform linux/amd64 --secret id=nginx-crt,src=nginx-repo.crt --secret id=nginx-key,src=nginx-repo.key --secret id=license-jwt,src=./license.jwt -t app-protect-dos . ``` The `--no-cache` option tells Docker to build the image from scratch and ensures the installation of the latest version of NGINX Plus and F5 DoS for NGINX. If the Dockerfile was previously used to build an image without the `--no-cache` option, the new image uses versions from the previously built image from the Docker cache. For RHEL8/9 with subctiption manager setup add build arguments: ```shell DOCKER_BUILDKIT=1 docker build --build-arg RHEL_ORG=... --build-arg RHEL_ACTIVATION_KEY=... --no-cache --platform linux/amd64 --secret id=nginx-crt,src=nginx-repo.crt --secret id=nginx-key,src=nginx-repo.key --secret id=license-jwt,src=./license.jwt -t app-protect-dos . ``` 8. Verify that the `app-protect-dos` image was created successfully with the docker images command: ```shell docker images app-protect-dos ``` 9. Create a container based on this image, for example, `my-app-protect-dos` container: ```shell docker run --name my-app-protect-dos -p 80:80 -d app-protect-dos ``` 10. Verify that the `my-app-protect-dos` container is up and running with the `docker ps` command: ```shell docker ps ``` 11. L4 Accelerated Mitigation Deployment Options:
There are three different ways to deploy the L4 accelerated mitigation feature:
1. Deploy in a Dedicated Container.
Create a shared folder on the host: ```shell mkdir /shared ``` This folder will be used to share data between containers. Modify the `entrypoint.sh` to run the L4 mitigation: ```shell # run processes /usr/bin/ebpf_manager_dos ``` Create and run the L4 container: ```shell docker run --privileged --network host --mount type=bind,source=/sys/fs/bpf,target=/sys/fs/bpf -v /shared:/shared --name my-app-protect-dos-ebpf-manager -d app-protect-dos-ebpf-manager ``` Create and run the main `app-protect-dos` container: ```shell docker run --name my-app-protect-dos -v /shared:/shared -p 80:80 -d app-protect-dos ``` 2. Deploy Directly on the Host.
To run L4 mitigation directly on the host:
1. Install the L4 mitigation on the host, as described in the OS-specific instructions. 2. Run the app-protect-dos container: ```shell docker run --name my-app-protect-dos -v /shared:/shared -p 80:80 -d app-protect-dos ``` 3. Run L4 Mitigation Inside the Same Container as `app-protect-dos`.
To run both L4 mitigation and the main application within the same container:
1. Modify the `entrypoint.sh`: ```shell ... # run processes /usr/bin/ebpf_manager_dos & ... ``` 2. run the container: ```shell docker run --name my-app-protect-dos -p 80:80 -d app-protect-dos ``` **Note:** L4 accelerated mitigation feature: - `app-protect-dos-ebpf-manager` need to run with root privileges. ### Alpine Docker Deployment Example ```Dockerfile # syntax=docker/dockerfile:1 # For Alpine 3.22: FROM alpine:3.22 # Download and add the NGINX signing keys: RUN wget -O /etc/apk/keys/nginx_signing.rsa.pub https://cs.nginx.com/static/keys/nginx_signing.rsa.pub # Add NGINX Plus/F5 DoS for NGINX repository: RUN printf "https://pkgs.nginx.com/plus/alpine/v`egrep -o '^[0-9]+\.[0-9]+' /etc/alpine-release`/main\n" | tee -a /etc/apk/repositories && \ printf "https://pkgs.nginx.com/app-protect-dos/alpine/v`egrep -o '^[0-9]+\.[0-9]+' /etc/alpine-release`/main\n" | tee -a /etc/apk/repositories # Update the repository and install the most recent version of the F5 DoS for NGINX package (which includes NGINX Plus): RUN --mount=type=secret,id=nginx-crt,dst=/etc/apk/cert.pem,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/apk/cert.key,mode=0644 \ --mount=type=secret,id=license-jwt,dst=license.jwt,mode=0644 \ apk update && apk add app-protect-dos && \ cat license.jwt > /etc/nginx/license.jwt # Forward request logs to Docker log collector: RUN ln -sf /dev/stdout /var/log/nginx/access.log && \ ln -sf /dev/stderr /var/log/nginx/error.log # Forward request logs to Docker log collector: RUN ln -sf /dev/stdout /var/log/nginx/access.log && \ ln -sf /dev/stderr /var/log/nginx/error.log # Copy configuration files: COPY nginx.conf custom_log_format.json /etc/nginx/ COPY entrypoint.sh /root/ RUN chmod +x /root/entrypoint.sh EXPOSE 80 STOPSIGNAL SIGQUIT CMD ["sh", "/root/entrypoint.sh"] ``` ### AmazonLinux 2023 Docker Deployment Example ```Dockerfile # For AmazonLinux 2023: FROM amazonlinux:2023 # Install prerequisite packages: RUN dnf -y install ca-certificates # Add NGINX Plus/F5 DoS for NGINX repository: RUN curl -o /etc/yum.repos.d/plus-amazonlinux2023.repo https://cs.nginx.com/static/files/plus-amazonlinux2023.repo && \ curl -o /etc/yum.repos.d/app-protect-dos-amazonlinux2023.repo https://cs.nginx.com/static/files/app-protect-dos-amazonlinux2023.repo # Install F5 DoS for NGINX: RUN --mount=type=secret,id=nginx-crt,dst=/etc/ssl/nginx/nginx-repo.crt,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/ssl/nginx/nginx-repo.key,mode=0644 \ --mount=type=secret,id=license-jwt,dst=license.jwt,mode=0644 \ dnf install -y app-protect-dos && \ cat license.jwt > /etc/nginx/license.jwt && \ rm /etc/yum.repos.d/plus-amazonlinux2023.repo && \ rm /etc/yum.repos.d/app-protect-dos-amazonlinux2023.repo && \ dnf clean all && \ rm -rf /var/cache/dnf # Forward request logs to Docker log collector: RUN ln -sf /dev/stdout /var/log/nginx/access.log && \ ln -sf /dev/stderr /var/log/nginx/error.log # Copy configuration files: COPY nginx.conf custom_log_format.json /etc/nginx/ COPY entrypoint.sh /root/ RUN chmod +x /root/entrypoint.sh EXPOSE 80 STOPSIGNAL SIGQUIT CMD ["sh", "/root/entrypoint.sh"] ``` ### Debian 11 (Bullseye) / Debian 12 (Bookworm) Docker Deployment Example ```Dockerfile # Where can be bullseye/bookworm FROM debian:bullseye # Setup repository keys RUN mkdir -p /etc/ssl/nginx/ /etc/nginx/ && \ apt-get update && \ apt-get install -y --no-install-recommends apt-transport-https lsb-release ca-certificates wget gnupg2 debian-archive-keyring && \ wget -qO - https://cs.nginx.com/static/keys/nginx_signing.key | gpg --dearmor | tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null && \ printf "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] https://pkgs.nginx.com/plus/debian $(lsb_release -cs) nginx-plus\n" > /etc/apt/sources.list.d/nginx-plus.list && \ printf "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] https://pkgs.nginx.com/app-protect-dos/debian $(lsb_release -cs) nginx-plus\n" > /etc/apt/sources.list.d/nginx-app-protect-dos.list && \ wget -P /etc/apt/apt.conf.d https://cs.nginx.com/static/files/90pkgs-nginx # Install F5 DoS for NGINX RUN --mount=type=secret,id=nginx-crt,dst=/etc/ssl/nginx/nginx-repo.crt,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/ssl/nginx/nginx-repo.key,mode=0644 \ --mount=type=secret,id=license-jwt,dst=license.jwt,mode=0644 \ apt-get update && DEBIAN_FRONTEND="noninteractive" apt-get install -y app-protect-dos && \ cat license.jwt > /etc/nginx/license.jwt && \ apt-get remove --purge --auto-remove -y && rm -rf /var/lib/apt/lists/* /etc/apt/sources.list.d/nginx-plus.list /etc/apt/sources.list.d/nginx-app-protect-dos.list && \ rm -rf /etc/apt/apt.conf.d/90nginx /var/lib/apt/lists/* # Forward request logs to Docker log collector: RUN ln -sf /dev/stdout /var/log/nginx/access.log && \ ln -sf /dev/stderr /var/log/nginx/error.log COPY nginx.conf /etc/nginx/ COPY entrypoint.sh /root/ RUN chmod +x /root/entrypoint.sh EXPOSE 80 STOPSIGNAL SIGQUIT CMD ["sh", "/root/entrypoint.sh"] ``` ### Ubuntu 22.04 (Jammy) / 24.04 (Noble) Docker Deployment Example ```Dockerfile # Where version can be: jammy/noble FROM ubuntu:noble # Setup repository keys RUN apt-get update && \ apt-get install -y --no-install-recommends apt-transport-https lsb-release ca-certificates wget gnupg2 ubuntu-keyring && \ wget -qO - https://cs.nginx.com/static/keys/nginx_signing.key | gpg --dearmor | tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null && \ printf "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] https://pkgs.nginx.com/plus/ubuntu $(lsb_release -cs) nginx-plus\n" > /etc/apt/sources.list.d/nginx-plus.list && \ printf "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] https://pkgs.nginx.com/app-protect-dos/ubuntu $(lsb_release -cs) nginx-plus\n" > /etc/apt/sources.list.d/nginx-app-protect-dos.list && \ wget -P /etc/apt/apt.conf.d https://cs.nginx.com/static/files/90pkgs-nginx # Install F5 DoS for NGINX RUN --mount=type=secret,id=nginx-crt,dst=/etc/ssl/nginx/nginx-repo.crt,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/ssl/nginx/nginx-repo.key,mode=0644 \ --mount=type=secret,id=license-jwt,dst=license.jwt,mode=0644 \ apt-get update && DEBIAN_FRONTEND="noninteractive" apt-get install -y app-protect-dos && \ cat license.jwt > /etc/nginx/license.jwt && \ apt-get remove --purge --auto-remove -y && rm -rf /var/lib/apt/lists/* /etc/apt/sources.list.d/nginx-plus.list /etc/apt/sources.list.d/nginx-app-protect-dos.list && \ rm -rf /etc/apt/apt.conf.d/90nginx /var/lib/apt/lists/* # Forward request logs to Docker log collector: RUN ln -sf /dev/stdout /var/log/nginx/access.log && \ ln -sf /dev/stderr /var/log/nginx/error.log COPY nginx.conf /etc/nginx/ COPY entrypoint.sh /root/ RUN chmod +x /root/entrypoint.sh EXPOSE 80 STOPSIGNAL SIGQUIT CMD ["sh", "/root/entrypoint.sh"] ``` ### RHEL 8 Docker Deployment Example ```Dockerfile # For UBI 8 FROM registry.access.redhat.com/ubi8 ARG RHEL_ORG ARG RHEL_ACTIVATION_KEY # Setup repository keys RUN subscription-manager register --org=${RHEL_ORG} --activationkey=${RHEL_ACTIVATION_KEY} && \ subscription-manager refresh && \ subscription-manager attach --auto || true && \ subscription-manager repos --enable=rhel-8-for-x86_64-baseos-rpms && \ subscription-manager repos --enable=rhel-8-for-x86_64-appstream-rpms && \ dnf -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm && \ dnf -y install ca-certificates && \ curl -o /etc/yum.repos.d/plus-8.repo https://cs.nginx.com/static/files/plus-8.repo && \ curl -o /etc/yum.repos.d/app-protect-dos-8.repo https://cs.nginx.com/static/files/app-protect-dos-8.repo # Install F5 DoS for NGINX RUN --mount=type=secret,id=nginx-crt,dst=/etc/ssl/nginx/nginx-repo.crt,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/ssl/nginx/nginx-repo.key,mode=0644 \ --mount=type=secret,id=license-jwt,dst=license.jwt,mode=0644 \ dnf -y install app-protect-dos && \ cat license.jwt > /etc/nginx/license.jwt && \ rm /etc/yum.repos.d/plus-8.repo && \ rm /etc/yum.repos.d/app-protect-dos-8.repo && \ dnf clean all && \ rm -rf /var/cache/yum # Forward request logs to Docker log collector: RUN ln -sf /dev/stdout /var/log/nginx/access.log && \ ln -sf /dev/stderr /var/log/nginx/error.log # Copy configuration files: COPY nginx.conf custom_log_format.json /etc/nginx/ COPY entrypoint.sh /root/ RUN chmod +x /root/entrypoint.sh EXPOSE 80 STOPSIGNAL SIGQUIT CMD ["sh", "/root/entrypoint.sh"] ``` ### Rocky Linux 9 Docker Deployment Example ```Dockerfile # syntax=docker/dockerfile:1 # For Rocky Linux 9: FROM rockylinux:9 # Install prerequisite packages: RUN dnf -y install ca-certificates epel-release 'dnf-command(config-manager)' # Add NGINX App-protect-DoS & NGINX Plus repo to Yum: RUN curl -o /etc/yum.repos.d/plus-9.repo https://cs.nginx.com/static/files/plus-9.repo && \ curl -o /etc/yum.repos.d/app-protect-dos-9.repo https://cs.nginx.com/static/files/app-protect-dos-9.repo && \ dnf config-manager --set-enabled crb && \ dnf clean all # Install F5 DoS for NGINX: RUN --mount=type=secret,id=nginx-crt,dst=/etc/ssl/nginx/nginx-repo.crt,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/ssl/nginx/nginx-repo.key,mode=0644 \ --mount=type=secret,id=license-jwt,dst=license.jwt,mode=0644 \ dnf install -y app-protect-dos && \ cat license.jwt > /etc/nginx/license.jwt && \ rm /etc/yum.repos.d/plus-9.repo && \ rm /etc/yum.repos.d/app-protect-dos-9.repo && \ dnf clean all && \ rm -rf /var/cache/dnf # Forward request logs to Docker log collector: RUN ln -sf /dev/stdout /var/log/nginx/access.log && \ ln -sf /dev/stderr /var/log/nginx/error.log # Copy configuration files: COPY nginx.conf custom_log_format.json /etc/nginx/ COPY entrypoint.sh /root/ RUN chmod +x /root/entrypoint.sh EXPOSE 80 STOPSIGNAL SIGQUIT CMD ["sh", "/root/entrypoint.sh"] ``` ## Docker Deployment with NGINX App Protect ### Docker Deployment Instructions You need root permissions to execute the following steps. 1. Create a Dockerfile (see examples below) which copies the following files into the docker image: - `nginx-repo.crt`: Certificate for NGINX repository access - `nginx-repo.key`: Private key for NGINX repository access - `license.jwt`: JWT license file for NGINX Plus license management - `nginx.conf`: User defined `nginx.conf` with `app-protect-dos` enabled - `entrypoint.sh`: Docker startup script which spins up all App Protect DoS processes, must have executable permissions - `custom_log_format.json`: Optional user-defined security log format file (if not used - remove its references from the nginx.conf and Dockerfile) 2. Log in to NGINX Plus Customer Portal and download your `nginx-repo.crt`, `nginx-repo.key` and `license.jwt` files. 3. Copy the files to the directory where the Dockerfile is located. 4. Optionally, create `custom_log_format.json` in the same directory, for example: ```json { "filter": { "request_type": "all" }, "content": { "format": "splunk", "max_request_size": "any", "max_message_size": "10k" } } ``` 5. In the same directory create the `nginx.conf` file with the following contents: ```nginx user nginx; worker_processes auto; error_log /var/log/nginx/error.log error; worker_rlimit_nofile 65535; working_directory /tmp/cores; load_module modules/ngx_http_app_protect_module.so; load_module modules/ngx_http_app_protect_dos_module.so; events { worker_connections 65535; } http { include /etc/nginx/mime.types; log_format log_napd ', vs_name_al=$app_protect_dos_vs_name, ip=$remote_addr, tls_fp=$app_protect_dos_tls_fp, ' 'outcome=$app_protect_dos_outcome, reason=$app_protect_dos_outcome_reason, ' 'ip_tls=$remote_addr:$app_protect_dos_tls_fp, '; app_protect_dos_security_log_enable on; app_protect_dos_security_log "/etc/app_protect_dos/log-default.json" /var/log/adm/logger.log; #app_protect_dos_security_log "/etc/app_protect_dos/log-default.json" syslog:server=1.2.3.4:5261; # HTTP/1 server server { default_type application/octet-stream; listen 80 reuseport; server_name serv80; proxy_http_version 1.1; app_protect_policy_file "/etc/app_protect/conf/NginxDefaultPolicy.json"; app_protect_security_log_enable on; app_protect_security_log "/etc/nginx/custom_log_format.json" syslog:server=127.0.0.1:514; set $loggable '0'; access_log /var/log/nginx/access.log log_napd if=$loggable; #access_log syslog:server=1.1.1.1:5561 log_napd if=$loggable; app_protect_dos_policy_file "/etc/app_protect_dos/BADOSDefaultPolicy.json"; location / { app_protect_dos_enable on; app_protect_dos_name "App80"; app_protect_dos_monitor uri=http://serv80/; proxy_pass http://1.2.3.4:80; } } # gRPC server with ssl server { default_type application/grpc; listen 443 http2 ssl reuseport; server_name serv_grpc; # TLS config ssl_certificate /etc/ssl/certs/grpc.example.com.crt; ssl_certificate_key /etc/ssl/private/grpc.example.com.key; ssl_session_cache shared:SSL:10m; ssl_session_timeout 5m; ssl_ciphers HIGH:!aNULL:!MD5; ssl_protocols TLSv1.2 TLSv1.3; set $loggable '0'; access_log /var/log/nginx/access.log log_napd if=$loggable; #access_log syslog:server=1.1.1.1:5561 log_napd if=$loggable; location / { app_protect_dos_enable on; app_protect_dos_name "AppGRPC"; app_protect_dos_monitor uri=https://serv_grpc:443/service/method protocol=grpc; # mandatory for gRPC grpc_pass grpc://1.2.3.4:1001; } } sendfile on; tcp_nopush on; keepalive_timeout 65; } ``` **Note:** Make sure to replace upstream and proxy pass directives in this example with relevant application backend settings. 6. For the L4 accelerated mitigation feature:
The following line in the `nginx.conf` file needs to be modified:
Change: ```nginx user nginx; ``` To: ```nginx user root; ``` 7. In the same directory create an `entrypoint.sh` file with executable permissions, with the following content: For Alpine / Debian / Ubuntu / UBI 8 / UBI 9: ```shell #!/usr/bin/env bash USER=nginx LOGDIR=/var/log/adm # prepare environment mkdir -p /var/run/adm /tmp/cores ${LOGDIR} chmod 755 /var/run/adm /tmp/cores ${LOGDIR} chown ${USER}:${USER} /var/run/adm /tmp/cores ${LOGDIR} # run processes /bin/su -s /bin/bash -c "/usr/bin/adminstall > ${LOGDIR}/adminstall.log 2>&1" ${USER} /bin/su -s /bin/bash -c "/opt/app_protect/bin/bd_agent &" ${USER} /bin/su -s /bin/bash -c "/usr/share/ts/bin/bd-socket-plugin tmm_count 4 proc_cpuinfo_cpu_mhz 2000000 total_xml_memory 307200000 total_umu_max_size 3129344 sys_max_account_id 1024 no_static_config 2>&1 > /var/log/app_protect/bd-socket-plugin.log &" ${USER} /bin/su -s /bin/bash -c "/usr/bin/admd -d --log info > ${LOGDIR}/admd.log 2>&1 &" ${USER} /usr/sbin/nginx -g 'daemon off;' ``` 8. Create a Docker image: For Debian/Ubuntu/Alpine/Amazon Linux: ```shell DOCKER_BUILDKIT=1 docker build --no-cache --platform linux/amd64 --secret id=nginx-crt,src=nginx-repo.crt --secret id=nginx-key,src=nginx-repo.key --secret id=license-jwt,src=./license.jwt -t app-protect-dos . ``` For RHEL: ```shell DOCKER_BUILDKIT=1 docker build --build-arg RHEL_ORG=... --build-arg RHEL_ACTIVATION_KEY=... --no-cache --platform linux/amd64 --secret id=nginx-crt,src=nginx-repo.crt --secret id=nginx-key,src=nginx-repo.key --secret id=license-jwt,src=./license.jwt -t app-protect-dos . ``` **Notes:** - The `--no-cache` option tells Docker/Podman to build the image from scratch and ensures the installation of the latest version of NGINX Plus and F5 WAF for NGINX 4.x. If the Dockerfile was previously used to build an image without the `--no-cache` option, the new image uses versions from the previously built image from the cache. - For RHEL:
The subscription-manager is disabled when running inside containers based on Red Hat Universal Base images. You will need a registered and subscribed RHEL system. 9. Verify that the `app-protect-dos` image was created successfully with the docker images command: ```shell docker images app-protect-dos ``` 10. Create a container based on this image, for example, `my-app-protect-dos` container: ```shell docker run --name my-app-protect-dos -p 80:80 -d app-protect-dos ``` 11. Verify that the `my-app-protect-dos` container is up and running with the `docker ps` command: ```shell docker ps ``` ### Alpine Dockerfile example ```dockerfile # syntax=docker/dockerfile:1 # For Alpine 3.22: FROM alpine:3.22 # Download and add the NGINX signing keys: RUN wget -O /etc/apk/keys/nginx_signing.rsa.pub https://cs.nginx.com/static/keys/nginx_signing.rsa.pub && \ wget -O /etc/apk/keys/app-protect-security-updates.rsa.pub https://cs.nginx.com/static/keys/app-protect-security-updates.rsa.pub # Add NGINX Plus repository: RUN printf "https://pkgs.nginx.com/plus/alpine/v`egrep -o '^[0-9]+\.[0-9]+' /etc/alpine-release`/main\n" | tee -a /etc/apk/repositories # Add F5 WAF for NGINX & Dos repositories: RUN printf "https://pkgs.nginx.com/app-protect-dos/alpine/v`egrep -o '^[0-9]+\.[0-9]+' /etc/alpine-release`/main\n" | tee -a /etc/apk/repositories && \ printf "https://pkgs.nginx.com/app-protect/alpine/v`egrep -o '^[0-9]+\.[0-9]+' /etc/alpine-release`/main\n" | tee -a /etc/apk/repositories && \ printf "https://pkgs.nginx.com/app-protect-security-updates/alpine/v`egrep -o '^[0-9]+\.[0-9]+' /etc/alpine-release`/main\n" | tee -a /etc/apk/repositories # Update the repository and install the most recent versions of the F5 WAF and F5 DoS for NGINX packages (which include NGINX Plus): RUN --mount=type=secret,id=nginx-crt,dst=/etc/apk/cert.pem,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/apk/cert.key,mode=0644 \ --mount=type=secret,id=license-jwt,dst=license.jwt,mode=0644 \ apk update && apk add app-protect app-protect-dos && \ cat license.jwt > /etc/nginx/license.jwt # Forward request logs to Docker log collector: RUN ln -sf /dev/stdout /var/log/nginx/access.log && \ ln -sf /dev/stderr /var/log/nginx/error.log # Forward request logs to Docker log collector: RUN ln -sf /dev/stdout /var/log/nginx/access.log && \ ln -sf /dev/stderr /var/log/nginx/error.log # Copy configuration files: COPY nginx.conf custom_log_format.json /etc/nginx/ COPY entrypoint.sh /root/ RUN chmod +x /root/entrypoint.sh EXPOSE 80 STOPSIGNAL SIGQUIT CMD ["sh", "/root/entrypoint.sh"] ``` ### Amazon Linux Dockerfile example ```dockerfile # syntax=docker/dockerfile:1 FROM amazonlinux:2023 # Install prerequisite packages: RUN dnf -y install ca-certificates # Add NGINX/NAP WAF/NAP DOS repositories: RUN curl -o /etc/yum.repos.d/plus-amazonlinux2023.repo https://cs.nginx.com/static/files/plus-amazonlinux2023.repo && \ curl -o /etc/yum.repos.d/app-protect-dos-amazonlinux2023.repo https://cs.nginx.com/static/files/app-protect-dos-amazonlinux2023.repo && \ curl -o /etc/yum.repos.d/app-protect-amazonlinux2023.repo https://cs.nginx.com/static/files/app-protect-amazonlinux2023.repo && \ curl -o /etc/yum.repos.d/dependencies.amazonlinux2023.repo https://cs.nginx.com/static/files/dependencies.amazonlinux2023.repo # Update the repository and install the most recent versions of the F5 WAF and F5 DoS for NGINX packages (which include NGINX Plus): RUN --mount=type=secret,id=nginx-crt,dst=/etc/ssl/nginx/nginx-repo.crt,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/ssl/nginx/nginx-repo.key,mode=0644 \ --mount=type=secret,id=license-jwt,dst=license.jwt,mode=0644 \ dnf -y install app-protect app-protect-dos && \ cat license.jwt > /etc/nginx/license.jwt && \ rm /etc/yum.repos.d/plus-amazonlinux2023.repo && \ rm /etc/yum.repos.d/app-protect-dos-amazonlinux2023.repo && \ dnf clean all && \ rm -rf /var/cache/dnf && \ rm -rf /var/cache/yum # Forward request logs to Docker log collector: RUN ln -sf /dev/stdout /var/log/nginx/access.log && \ ln -sf /dev/stderr /var/log/nginx/error.log # Copy configuration files: COPY nginx.conf custom_log_format.json /etc/nginx/ COPY entrypoint.sh /root/ RUN chmod +x /root/entrypoint.sh EXPOSE 80 STOPSIGNAL SIGQUIT CMD ["sh", "/root/entrypoint.sh"] ``` ### Debian Docker Deployment Example ```Dockerfile # Where version can be: bullseye/bookworm FROM debian:bullseye # Install prerequisite packages: RUN apt-get update && \ apt-get install -y --no-install-recommends apt-transport-https lsb-release ca-certificates wget gnupg2 debian-archive-keyring && \ wget -qO - https://cs.nginx.com/static/keys/nginx_signing.key | gpg --dearmor | tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null && \ wget -qO - https://cs.nginx.com/static/keys/app-protect-security-updates.key | gpg --dearmor | tee /usr/share/keyrings/app-protect-security-updates.gpg > /dev/null # Add NGINX Plus, NGINX App Protect and F5 DoS for NGINX repository: RUN printf "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] https://pkgs.nginx.com/plus/debian `lsb_release -cs` nginx-plus\n" | tee /etc/apt/sources.list.d/nginx-plus.list \ && printf "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] https://pkgs.nginx.com/app-protect-dos/debian `lsb_release -cs` nginx-plus\n" | tee /etc/apt/sources.list.d/nginx-app-protect-dos.list \ && printf "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] https://pkgs.nginx.com/app-protect/debian `lsb_release -cs` nginx-plus\n" | tee /etc/apt/sources.list.d/nginx-app-protect.list \ && printf "deb [signed-by=/usr/share/keyrings/app-protect-security-updates.gpg] https://pkgs.nginx.com/app-protect-security-updates/debian `lsb_release -cs` nginx-plus\n" | tee /etc/apt/sources.list.d/app-protect-security-updates.list # Download the apt configuration to `/etc/apt/apt.conf.d`: RUN wget -P /etc/apt/apt.conf.d https://cs.nginx.com/static/files/90pkgs-nginx # Update the repository and install the most recent versions of the F5 WAF and F5 DoS for NGINX packages (which includes NGINX Plus): RUN --mount=type=secret,id=nginx-crt,dst=/etc/ssl/nginx/nginx-repo.crt,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/ssl/nginx/nginx-repo.key,mode=0644 \ --mount=type=secret,id=license-jwt,dst=license.jwt,mode=0644 \ apt-get update && DEBIAN_FRONTEND="noninteractive" apt-get install -y app-protect app-protect-dos && \ cat license.jwt > /etc/nginx/license.jwt && \ apt-get remove --purge --auto-remove -y && rm -rf /var/lib/apt/lists/* /etc/apt/sources.list.d/nginx-plus.list /etc/apt/sources.list.d/nginx-app-protect.list /etc/apt/sources.list.d/nginx-app-protect-dos.list && \ rm -rf /etc/apt/apt.conf.d/90nginx /var/lib/apt/lists/* # Forward request logs to Docker log collector: RUN ln -sf /dev/stdout /var/log/nginx/access.log && \ ln -sf /dev/stderr /var/log/nginx/error.log COPY nginx.conf /etc/nginx/ COPY entrypoint.sh /root/ RUN chmod +x /root/entrypoint.sh EXPOSE 80 STOPSIGNAL SIGQUIT CMD ["sh", "/root/entrypoint.sh"] ``` ### Ubuntu Docker Deployment Example ```Dockerfile # Where version can be:jammy/noble FROM ubuntu:noble # Install prerequisite packages: RUN apt-get update && \ apt-get install -y --no-install-recommends apt-transport-https lsb-release ca-certificates wget gnupg2 ubuntu-keyring && \ wget -qO - https://cs.nginx.com/static/keys/nginx_signing.key | gpg --dearmor | tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null && \ wget -qO - https://cs.nginx.com/static/keys/app-protect-security-updates.key | gpg --dearmor | tee /usr/share/keyrings/app-protect-security-updates.gpg > /dev/null # Add NGINX Plus, NGINX App Protect and F5 DoS for NGINX repository: RUN printf "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] https://pkgs.nginx.com/plus/ubuntu `lsb_release -cs` nginx-plus\n" | tee /etc/apt/sources.list.d/nginx-plus.list \ && printf "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] https://pkgs.nginx.com/app-protect-dos/ubuntu `lsb_release -cs` nginx-plus\n" | tee /etc/apt/sources.list.d/nginx-app-protect-dos.list \ && printf "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] https://pkgs.nginx.com/app-protect/ubuntu `lsb_release -cs` nginx-plus\n" | tee /etc/apt/sources.list.d/nginx-app-protect.list \ && printf "deb [signed-by=/usr/share/keyrings/app-protect-security-updates.gpg] https://pkgs.nginx.com/app-protect-security-updates/debian `lsb_release -cs` nginx-plus\n" | tee /etc/apt/sources.list.d/app-protect-security-updates.list # Download the apt configuration to `/etc/apt/apt.conf.d`: RUN wget -P /etc/apt/apt.conf.d https://cs.nginx.com/static/files/90pkgs-nginx # Update the repository and install the most recent versions of the F5 WAF and F5 DoS for NGINX packages (which include NGINX Plus): RUN --mount=type=secret,id=nginx-crt,dst=/etc/ssl/nginx/nginx-repo.crt,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/ssl/nginx/nginx-repo.key,mode=0644 \ --mount=type=secret,id=license-jwt,dst=license.jwt,mode=0644 \ apt-get update && DEBIAN_FRONTEND="noninteractive" apt-get install -y app-protect app-protect-dos && \ cat license.jwt > /etc/nginx/license.jwt && \ apt-get remove --purge --auto-remove -y && rm -rf /var/lib/apt/lists/* /etc/apt/sources.list.d/nginx-plus.list /etc/apt/sources.list.d/nginx-app-protect.list /etc/apt/sources.list.d/nginx-app-protect-dos.list && \ rm -rf /etc/apt/apt.conf.d/90nginx /var/lib/apt/lists/* # Forward request logs to Docker log collector: RUN ln -sf /dev/stdout /var/log/nginx/access.log && \ ln -sf /dev/stderr /var/log/nginx/error.log COPY nginx.conf /etc/nginx/ COPY entrypoint.sh /root/ RUN chmod +x /root/entrypoint.sh EXPOSE 80 STOPSIGNAL SIGQUIT CMD ["sh", "/root/entrypoint.sh"] ``` ## F5 DoS for NGINX Arbitrator ### Overview F5 DoS for NGINX arbitrator orchestrates all the running F5 DoS for NGINX instances to synchronize local/global attack start/stop. F5 DoS for NGINX arbitrator serves as a central coordinating component for managing multiple instances of App Protect DoS in a network. It is needed when there are more than one F5 DoS for NGINX instances. Its primary function is to ensure that all instances are aware of and share the same state for each protected object. Here's a clearer breakdown of how it works and why it's necessary: How F5 DoS for NGINX Arbitrator Works: - **Collecting State Periodically**: The arbitrator regularly collects the state information from all running instances of App Protect DoS. This collection occurs at set intervals, typically every 10 seconds. - **State Initialization for New Instances**: When a new App Protect DoS instance is created, it doesn't start with a blank or uninitialized state for a protected object. Instead, it retrieves the initial state for the protected object from the arbitrator. - **Updating State in Case of an Attack**: If an attack is detected by one of the App Protect DoS instances, that instance sends an attack notification to the arbitrator. The arbitrator then updates the state of the affected protected object to indicate that it is under attack. Importantly, this updated state is propagated to all other instances. ### Why F5 DoS for NGINX Arbitrator is Necessary F5 DoS for NGINX Arbitrator is essential for several reasons: - **Global State Management**: Without the arbitrator, each individual instance of App Protect DoS would manage its own isolated state for each protected object. This isolation could lead to inconsistencies. For example, if instance A declared an attack on a protected object named "PO-Example," instance B would remain unaware of this attack, potentially leaving the object vulnerable. - **Uniform Attack Detection**: With the arbitrator in place, when instance A detects an attack on "PO-Example" and reports it to the arbitrator, the state of "PO-Example" is immediately updated to indicate an attack. This means that all instances, including instance B, are aware of the attack and can take appropriate measures to mitigate it. In summary, F5 DoS for NGINX Arbitrator acts as a central coordinator to maintain a consistent and up-to-date global state for protected objects across multiple instances of App Protect DoS. This coordination helps ensure that attacks are properly detected and mitigated, and that knowledge gained by one instance is efficiently shared with others, enhancing the overall security of the network. ### F5 DoS for NGINX Arbitrator Deployment 1. Pull the official F5 DoS for NGINX Arbitrator image with the command: ```shell docker pull docker-registry.nginx.com/nap-dos/app_protect_dos_arb:latest ``` 2. Create a container based on this image, for example, `app-protect-dos-arb` container: ```shell docker run --name app_protect_dos_arb -p 3000:3000 -d docker-registry.nginx.com/nap-dos/app_protect_dos_arb ``` 3. Verify that the `app-protect-dos-arb` container is up and running with the `docker ps` command. 4. DNS records are required for F5 DoS for NGINX Arbitrator to work properly and be accessible by F5 DoS for NGINX servers. Ensure that the `svc-appprotect-dos-arb` or configured Arbitrator FQDN (with `app_protect_dos_arb_fqdn` directive) has a valid DNS resolution. This step is necessary only for VM/Docker deployments with arbitrator. When the arbitrator is in the same Kubernetes namespace as F5 DoS for NGINX, this step is not needed. ### Multi-VM Deployment The Arbitrator service is standalone. Once it is down, it can be seamlessly re-started. It will immediately recover all the needed information from F5 DoS for NGINX instances that communicate to it every 10 sec. It’s downtime is around 10-20 seconds which will not affect the F5 DoS for NGINX working. F5 DoS for NGINX Arbitrator service connects to port 3000 and can be seen under App Protect DoS instances. All modules try to connect to this service automatically. If it’s not accessible, each instance works in standalone mode. There is no such option for authentications between F5 DoS for NGINX servers and Arbitrator service like MTLS or password . Currently Arbitrator service is not exposed outside of the namespace. It is customers responsibility to isolate it from outside. It is applicable to any deployment of Arbitrator, not only to multi-VM. ## Post-Installation Checks You can run the following commands to ensure that F5 DoS for NGINX enforcement is operational. 1. Check that the three processes needed for F5 DoS for NGINX are running using `ps aux`: - admd - nginx: master process - nginx: worker process ```shell USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND nginx 7759 0.0 0.0 113120 1200 ? Ss Sep06 0:00 /bin/sh -c /usr/bin/admd -d --log info > /var/log/adm/admd.log 2>&1 root 7765 0.0 0.0 87964 1464 ? Ss Sep06 0:00 nginx: master process /usr/sbin/nginx -g daemon off; nginx 7767 0.0 0.1 615868 8188 ? Sl Sep06 0:04 nginx: worker process ``` 2. Verify that there are no NGINX errors in the `/var/log/nginx/error.log` and that the policy compiled successfully: ```shell 2020/09/07 15:33:44 [notice] 9307#9307: using the "epoll" event method 2020/09/07 15:33:44 [notice] 9307#9307: nginx/1.19.0 (nginx-plus-r22) 2020/09/07 15:33:44 [notice] 9307#9307: built by gcc 4.8.5 20150623 (Red Hat 4.8.5-39) (GCC) 2020/09/07 15:33:44 [notice] 9307#9307: OS: Linux 3.10.0-327.28.3.el7.x86_64 2020/09/07 15:33:44 [notice] 9307#9307: getrlimit(RLIMIT_NOFILE): 1024:4096 2020/09/07 15:33:44 [notice] 9310#9310: start worker processes 2020/09/07 15:33:44 [notice] 9310#9310: start worker process 9311 PID <9311>, WORKER <0>, Function adm_ngx_init_process, line 684, version: 22+1.19.4-1.el7.ngx ``` 3. Check that by applying an attack, the attacker IP addresses are blocked while the good traffic pass through: a. Simulate good traffic: ```shell echo "Start Good Traffic 2" while true; do curl ${VS}/good1 & curl ${VS}/good2 & curl ${VS}/good3 & curl ${VS}/good4 sleep 0.1 done & ``` b. After 7 minutes start the attack: ```shell while [ true ] do ab -B ${BAD_IP1} -l -r -n 1000000 -c 150 -d -H "Host: evil.net" -H "Pragma: no-cache" -H "Cache-Control: no-cache" -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8" -H "Upgrade-Insecure-Requests: 1" -H "User-Agent: WireXBot" -H "x-requested-with:" -H "Referer: http://10.0.2.1/none.html" -H "Accept-Encoding: gzip, deflate" -H "Accept-Language: en-US" http://${VS}/ & ab -B ${BAD_IP2} -l -r -n 1000000 -c 150 -d -H "Host: evil.net" -H "Pragma: no-cache" -H "Cache-Control: no-cache" -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8" -H "Upgrade-Insecure-Requests: 1" -H "User-Agent: WireXBot" -H "x-requested-with:" -H "Referer: http://10.0.2.1/none.html" -H "Accept-Encoding: gzip, deflate" -H "Accept-Language: en-US" http://${VS}/ & ab -B ${BAD_IP3} -l -r -n 1000000 -c 150 -d -s 10 -H "Host: evil.net" -H "Pragma: no-cache" -H "Cache-Control: no-cache" -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8" -H "Upgrade-Insecure-Requests: 1" -H "User-Agent: WireXBot" -H "x-requested-with:" -H "Referer: http://10.0.2.1/none.html" -H "Accept-Encoding: gzip, deflate" -H "Accept-Language: en-US" http://${VS}/ killall ab done ``` c. See that the good traffic continue as usual while the attackers receive denial of service. To check F5 WAF for NGINX alongside F5 DoS for NGINX, just perform the normal tests as specified at [Admin Guide](https://docs.nginx.com/waf/install/virtual-environment/#post-installation-checks) ### Compatibility with NGINX Plus Releases A threat campaign package is compatible with the NGINX Plus release supported during the time the threat campaign package was released and with all future releases from that point in time on. In other words, it is not compatible with earlier App Protect DoS releases. Those older releases are not supported at this point in time so you will have to upgrade App Protect DoS to benefit from the support which includes Threat campaigns updates. ## Upgrading App Protect DoS You can upgrade to the latest NGINX Plus and App Protect DoS versions by downloading and installing the latest F5 DoS for NGINX package. When upgrading from this package, App Protect DoS will be uninstalled and reinstalled. The old default security policy is deleted and the new default security policy is installed. If you have created a custom security policy, the policy persists and you will need to update `nginx.conf` and point to the custom security policy by referencing the json file (using the full path). If you upgrade your NGINX version outside the App Protect DoS module, App Protect DoS will be uninstalled and you will need to reinstall it. You need to restart NGINX after an upgrade. ## SELinux The default settings for Security-Enhanced Linux (SELinux) on modern Red Hat Enterprise Linux (RHEL) and related Linux distributions can be very strict, erring on the side of security rather than convenience. Although the App Protect DoS applies its SELinux policy module during installation, your specific configuration might be blocked unless you adjust the policy or modify file labels. ### Modifying File Labels For example, if you plan to store your log configuration files in `/etc/logs` - you should change the default SELinux file context for this directory: ```shell semanage fcontext -a -t httpd_config_t /etc/logs restorecon -Rv /etc/logs ``` ### Syslog to Custom Port If you want to send logs to some unreserved port, you can use semanage to add the desired port (here, 35514) to the syslogd_port_t type: ```shell semanage port -a -t syslogd_port_t -p tcp 35514 ``` Review the syslog ports by entering the following command: ```shell semanage port -l | grep syslog ``` ## Kubernetes Deployment Examples ### App Protect DoS `appprotect-dos.yaml`: ```yaml apiVersion: apps/v1 kind: Deployment metadata: name: appprotect-dos namespace: appprotect-dos-wp-diff labels: app: appprotect-dos spec: replicas: 1 revisionHistoryLimit: 10 selector: matchLabels: app: appprotect-dos strategy: type: RollingUpdate rollingUpdate: maxSurge: 2 maxUnavailable: 1 template: metadata: labels: app: appprotect-dos spec: containers: - name: ubuntu-bados image: example.com/ubuntu_app_protect_dos:latest imagePullPolicy: Always resources: requests: cpu: "200m" memory: "500Mi" limits: cpu: "900m" memory: "800Mi" ports: - containerPort: 80 name: web - containerPort: 8090 name: probe - containerPort: 8091 name: probe500 livenessProbe: httpGet: path: /app_protect_dos_liveness port: 8090 initialDelaySeconds: 0 periodSeconds: 10 readinessProbe: httpGet: path: /app_protect_dos_readiness port: 8090 initialDelaySeconds: 0 periodSeconds: 10 volumeMounts: - name: shared mountPath: /shared/ - name: conf mountPath: /etc/nginx/nginx.conf subPath: nginx.conf - name: root-script mountPath: /root/entrypoint.sh subPath: entrypoint.sh - name: log-default mountPath: /etc/app_protect_dos/log-default.json subPath: log-default.json volumes: - name: shared persistentVolumeClaim: claimName: pvc-appprotect-dos-shared - name: conf configMap: name: cm-appprotect-dos-nginx items: - key: nginx.conf path: nginx.conf - name: root-script configMap: name: cm-appprotect-dos-entry defaultMode: 0755 items: - key: entrypoint.sh path: entrypoint.sh - name: log-default configMap: name: cm-appprotect-dos-log-default defaultMode: 0755 items: - key: log-default.json path: log-default.json ``` `svc-appprotect-dos.yaml`: ```yaml apiVersion: v1 kind: Service metadata: name: svc-appprotect-dos namespace: appprotect-dos-wp-diff labels: app: appprotect-dos spec: ports: - name: app port: 80 protocol: TCP nodePort: 80 selector: app: appprotect-dos type: NodePort ``` `log-default.json`: ```json { "filter": { "traffic-mitigation-stats": "all", "bad-actors": "all", "attack-signatures": "all" } } ``` `entrypoint.sh`: ```shell #!/usr/bin/env bash USER=nginx LOGDIR=/var/log/adm # prepare environment mkdir -p /var/run/adm /tmp/cores ${LOGDIR} chmod 755 /var/run/adm /tmp/cores ${LOGDIR} chown ${USER}:${USER} /var/run/adm /tmp/cores ${LOGDIR} # run processes /bin/su -s /bin/bash -c "/usr/bin/adminstall > ${LOGDIR}/adminstall.log 2>&1" ${USER} /usr/sbin/nginx -g 'daemon off;' & /bin/su -s /bin/bash -c "/usr/bin/admd -d --log info > ${LOGDIR}/admd.log 2>&1 &" ${USER} ``` `install.sh`: ```shell #!/bin/bash set -ex DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )" kubectl -n appprotect-dos-wp-diff create cm cm-appprotect-dos-nginx --from-file ${DIR}/nginx.conf kubectl -n appprotect-dos-wp-diff create cm cm-appprotect-dos-entry --from-file ${DIR}/entrypoint.sh kubectl -n appprotect-dos-wp-diff create cm cm-appprotect-dos-log-default --from-file ${DIR}/log-default.json kubectl create -f ${DIR}/appprotect-dos.yaml #kubectl create -f ${DIR}/svc-appprotect-dos.yaml ``` `nginx.conf`: ```nginx user nginx; worker_processes 1; error_log /var/log/nginx/error.log debug; worker_rlimit_nofile 65535; working_directory /tmp/cores; load_module modules/ngx_http_app_protect_dos_module.so; events { worker_connections 65535; } http { include /etc/nginx/mime.types; default_type application/octet-stream; log_format log_napd ', vs_name_al=$app_protect_dos_vs_name, ip=$remote_addr, tls_fp=$app_protect_dos_tls_fp, ' 'outcome=$app_protect_dos_outcome, reason=$app_protect_dos_outcome_reason, ' 'ip_tls=$remote_addr:$app_protect_dos_tls_fp, '; app_protect_dos_security_log_enable on; app_protect_dos_security_log "/etc/app_protect_dos/log-default.json" /var/log/adm/logger.log; # app_protect_dos_security_log "/etc/app_protect_dos/log-default.json" syslog:server=1.2.3.4:5261; app_protect_dos_liveness on; # uri:/app_protect_dos_liveness port:8090 app_protect_dos_readiness on; # uri:/app_protect_dos_readiness port:8090 server { listen 80 reuseport; server_name serv; set $loggable '0'; access_log /var/log/nginx/access.log log_napd if=$loggable; # access_log syslog:server=1.1.1.1:5561 log_napd if=$loggable; app_protect_dos_policy_file "/etc/app_protect_dos/BADOSDefaultPolicy.json"; location / { app_protect_dos_enable on; app_protect_dos_name "App1"; app_protect_dos_monitor uri=http://serv:80/ protocol=http1; proxy_pass http://1.2.3.4:80; } } server { listen 8090; server_name probe; location / { proxy_pass http://localhost:8091; } } server { listen 8091; return 503; } sendfile on; tcp_nopush on; keepalive_timeout 65; } ``` ### App Protect DoS Arbitrator Arbitrator (arb) is an internal service that is essential for the scaling scenarios. The arbitrator service should be deployed in the same namespace as F5 DoS for NGINX. `appprotect-dos-arb.yaml`: ```yaml apiVersion: apps/v1 kind: Deployment metadata: name: appprotect-dos-arb namespace: appprotect-dos-wp-diff spec: replicas: 1 selector: matchLabels: app: appprotect-dos-arb template: metadata: labels: app: appprotect-dos-arb spec: containers: - name: arb-svc image: example.com/app_protect_dos_arb:latest resources: requests: cpu: "200m" memory: "500Mi" limits: cpu: "900m" memory: "800Mi" ports: - containerPort: 3000 ``` `svc-appprotect-dos-arb.yaml`: ```yaml apiVersion: v1 kind: Service metadata: name: svc-appprotect-dos-arb namespace: appprotect-dos-wp-diff spec: selector: app: appprotect-dos-arb ports: - name: arb port: 3000 protocol: TCP targetPort: 3000 clusterIP: None ``` `install_appprotect-arb.sh`: ```shell #!/bin/bash set -ex DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )" kubectl -n appprotect-dos-wp-diff apply -f ${DIR}/appprotect-dos-arb.yaml kubectl -n appprotect-dos-wp-diff apply -f ${DIR}/svc-appprotect-dos-arb.yaml ``` `install F5 DoS for NGINX with ARB service`: ```shell #!/bin/bash set -ex DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )" kubectl create ns appprotect-dos-wp-diff ${DIR}/appprotect-dos/install.sh ${DIR}/appprotect-dos-arb/install_appprotect-dos-arb.sh ``` --- ## App Protect DoS eBPF manager ### Overview The eBPF Manager is a powerful and efficient tool designed to simplify and secure the deployment of eBPF (Extended Berkeley Packet Filter) programs for advanced networking use cases. Its primary responsibilities include program installation and managing client interactions to enable real-time packet processing and mitigation solutions. ### CLI Options for Flexible Configuration The eBPF Manager comes with configurable command-line flags for ease of use and deployment customization. Key options include: * Interface Selection: * -i, --interface [interfaces...]: Specify one or more network interfaces for eBPF XDP program deployment. If omitted, it defaults to all non-virtual, active network devices. * gRPC UDS Ownership: * -u, --user : Set the user ownership for the gRPC Unix Domain Socket (UDS). Defaults to nginx. * -g, --group : Set the group ownership for the gRPC Unix Domain Socket (UDS). Defaults to nginx. -------------------------------------------------------------------------------- title: "F5 DoS for NGINX 4.3" url: https://docs.nginx.com/nginx-app-protect-dos/releases/about-4.3/index.md -------------------------------------------------------------------------------- Here you can find the release information for F5 DoS for NGINX v4.3. F5 DoS for NGINX provides behavioral protection against Denial of Service (DoS) for your web applications. ## Release 4.3 December 19, 2023 In this release, F5 DoS for NGINX supports NGINX Plus R31. ### New Features - Support for Nginx Plus R31 - [Support for Alpine 3.17](learn-about-deployment.md#alpine-315x--317x-installation) ### Supported Packages #### App Protect DoS ##### Alpine 3.17 - app-protect-dos-31.4.3.1-r1.apk ##### CentOS 7.4+ / RHEL 7.4+ / UBI7 - app-protect-dos-31.4.3.1-1.el7.ngx.x86_64.rpm ##### RHEL 8 and Rocky Linux 8 - app-protect-dos-31.4.3.1-1.el8.ngx.x86_64.rpm ##### Debian 11 - app-protect-dos_31.4.3.1-1~bullseye_amd64.deb ##### Ubuntu 20.04 - app-protect-dos_31.4.3.1-1~focal_amd64.deb ##### Ubuntu 22.04 - app-protect-dos_31.4.3.1-1~jammy_amd64.deb #### NGINX Plus - NGINX Plus R31 ### Important Note - Starting with this release, Alpine 3.15 support has been deprecated. -------------------------------------------------------------------------------- title: "Best Practices" url: https://docs.nginx.com/nginx-app-protect-dos/deployment-guide/best-practices/index.md description: F5 DoS for NGINX Best Practices Deployment. -------------------------------------------------------------------------------- This guide shows how to modify your NGINX configuration to enable F5 DoS for NGINX (NGINX App Protect DoS). We will configure F5 DoS For NGINX to protect a proxy server. ## F5 DoS Configuration ### Load Module ```nginx load_module modules/ngx_http_app_protect_dos_module.so; ``` ### Enable Add the directive in the appropriate context, You can set it in location, server, or http blocks: ```nginx app_protect_dos_enable on; ``` #### Set a Protected Object Name Choose a unique name. You can set it in location, server, or http blocks. ```nginx app_protect_dos_name po-example; ``` **Note**: Although optional, we strongly recommend specifying a name for each Protected Object (PO) to improve organization and maintainability. If no name is provided, the virtual server is assigned an auto-generated name using the following syntax: ```nginx line_number-server_name:seq-location_name Example: 30-backend:1-/abc ``` Where: - `line number:` the line number of the server block (`server {`) in the `nginx.conf` file (i.e. `30`)
- `server name:` taken from directive `server_name` (i.e. `backend`)
seq: 0 for server block, increments for each location block. i.e. VS created from server block will have 0 and VS's from location blocks will be 1,2,3,... (i.e. `1`) - `location name:` the name of the location (i.e. `/abc`) Capacity limits - Up to 300 Protected Objects in versions up to 4.3
- Up to 1,000 Protected Objects in versions 4.4 and later
### Set a Monitor directive The `app_protect_dos_monitor` directive monitors the stress level of a Protected Object by generating requests from localhost (127.0.0.1) that traverse your NGINX configuration like normal client traffic (through the same `server` / `location` / `proxy_pass` chain). **This directive is mandatory for optimal accuracy (it may be omitted only when using HTTP/1.1, though it is still strongly recommended).** ```nginx app_protect_dos_monitor uri= [protocol=http1|http2|grpc|websocket] [timeout=] [proxy_protocol=on|off]; ``` - `uri` is the value of `server_name`, optionally followed by `:port`, and then the location path. Examples: `my_server/`, `example_server:81/abc` A complete guide on configuring the Monitor Directive can be found here: [Monitor Directive](https://docs.nginx.com/nginx-app-protect-dos/directives-and-policy/learn-about-directives-and-policy/#monitor-directive-app_protect_dos_monitor). **Monitor directive best practice** - Monitor the same virtual host and path that your users hit. Set `uri=` to the `server_name[:port]/path` that matches the `server_name` and `listen` directives, **not** to the upstream IP:port.
Examples:
For `server_name "my_server"` on port `80` and path `/` (port 80 is default, so it can be omitted): ```nginx app_protect_dos_monitor uri=my_server/; ``` For `server_name "serv"` on port `81` with location path `/abc`: ```nginx app_protect_dos_monitor uri=serv:81/abc protocol=http1 timeout=7; ``` A full example with upstream:
```nginx upstream backend { server 10.197.24.136:3000; } server { listen 80 reuseport; server_name example_srv; location / { app_protect_dos_enable on; app_protect_dos_name "main_app"; # ✅ Good: monitor hits NGINX using server_name and is proxied to the upstream app_protect_dos_monitor uri=example_srv:80/ protocol=http1 timeout=7; # ❌ Bad: do NOT point the monitor directly at the upstream IP # app_protect_dos_monitor uri=10.197.24.136:3000/ protocol=http1 timeout=7; proxy_pass http://backend; } } ``` - Avoid monitors that short-circuit upstreams (for example, `return 200` locally); this will under-estimate stress.
- Choose `timeout` slightly above your upstream’s p95/p99 latency under normal load, but low enough to react quickly under stress.
- Monitor traffic originates from `127.0.0.1`. Exclude it from rate and connection limits as needed.
- Define the monitor inside each protected `location` block.
## Arbitrator It is required when more than one F5 DoS for NGINX instance is deployed. Its primary function is to ensure that all instances are aware of—and share—the same state for each Protected Object.
A complete guide on configuring F5 DoS for NGINX Arbitrator be found here: [F5 DoS for NGINX Arbitrator](https://docs.nginx.com/nginx-app-protect-dos/deployment-guide/learn-about-deployment/#f5-dos-for-nginx-arbitrator)
Enable the F5 DoS for NGINX Arbitrator in the `http` context of the `nginx.conf` file: ```nginx app_protect_dos_arb_fqdn 10.1.10.22; ``` ## EBPF manager The eBPF Manager is a high-performance component that simplifies and secures the deployment of eBPF (Extended Berkeley Packet Filter) programs for advanced networking use cases. Enable the L4-accelerated mitigation feature in the http context of the nginx.conf file: ```nginx app_protect_dos_accelerated_mitigation on; ``` ## ELK Dashboards ELK stands for Elasticsearch, Logstash, and Kibana. Logstash receives logs from F5 DoS, normalizes them, and stores them in the Elasticsearch index. Kibana allows you to visualize and navigate the logs using purpose-built dashboards.
A complete guide on configuring ELK can be found here: [F5 DoS for NGINX ELK Dashboards](https://github.com/f5devcentral/nap-dos-elk-dashboards)
F5 DoS directives should appear in your `nginx.conf` as shown. Replace `ip_kibana` with the hostname of the server running your ELK Docker container:
```nginx http { log_format log_dos ', vs_name_al=$app_protect_dos_vs_name, ip=$remote_addr, tls_fp=$app_protect_dos_tls_fp, outcome=$app_protect_dos_outcome, reason=$app_protect_dos_outcome_reason, ip_tls=$remote_addr:$app_protect_dos_tls_fp, '; ... server { ... app_protect_dos_security_log_enable on; app_protect_dos_security_log "/etc/app_protect_dos/log-default.json" syslog:server=ip_kibana:5261; location / { app_protect_dos_enable on; set $loggable '0'; access_log syslog:server=ip_kibana:5561 log_dos if=$loggable; ... } } ... } ``` ## Live Activity Monitoring F5 DoS for NGINX provides a range of application monitoring tools: - F5 DoS for NGINX Dashboard: A dynamic interface for real-time monitoring and detailed views of Protected Objects.
- F5 DoS for NGINX REST API: An interface that exposes comprehensive metrics for Protected Objects.
A complete guide on configuring F5 DoS for NGINX Live Activity Monitoring be found here: [F5 DoS for NGINX Live Activity Monitoring](https://docs.nginx.com/nginx-app-protect-dos/monitoring/live-activity-monitoring/)
Below is an example configuration that limits API location access to the local network using the allow and deny directives, and uses HTTP Basic Authentication to restrict the PATCH, POST, and DELETE methods to specific users.
To view the dashboard, enter its address in your browser’s address bar. For example, http://192.168.1.23/dashboard-dos.html displays the dashboard page located in `/usr/share/nginx/html`, as specified by the root directive.
```nginx http { # ... server { listen 192.168.1.23; # ... location /api { limit_except GET { auth_basic "NGINX Plus API"; auth_basic_user_file /path/to/passwd/file; } app_protect_dos_api; allow 192.168.1.0/24; deny all; } location = /dashboard-dos.html { root /usr/share/nginx/html; } } } ``` ## Example nginx.conf ```nginx user nginx; worker_processes auto; error_log /var/log/nginx/error.log error; load_module modules/ngx_http_app_protect_dos_module.so; events { worker_connections 1024; } http { app_protect_dos_arb_fqdn 10.1.10.22; app_protect_dos_accelerated_mitigation on; sendfile on; tcp_nopush on; keepalive_timeout 65; log_format log_dos ', vs_name_al=$app_protect_dos_vs_name, ip=$remote_addr, tls_fp=$app_protect_dos_tls_fp, ' 'outcome=$app_protect_dos_outcome, reason=$app_protect_dos_outcome_reason, ' 'ip_tls=$remote_addr:$app_protect_dos_tls_fp, '; server { listen 80 reuseport; server_name example_srv; access_log /var/log/nginx/access.log log_dos if=$loggable; app_protect_dos_security_log_enable on; app_protect_dos_security_log "/etc/app_protect_dos/log-default.json" syslog:server=10.197.30.219:5261; app_protect_dos_policy_file "/etc/app_protect_dos/BADOSDefaultPolicy.json"; location / { app_protect_dos_enable on; app_protect_dos_name "main_app"; set $loggable '0'; access_log syslog:server=10.97.30.219:5561 log_dos if=$loggable; app_protect_dos_monitor uri=example_srv:80/ protocol=http1 timeout=7; proxy_pass http://10.197.24.136:3000; } } server { listen 800; location /api { limit_except GET { auth_basic "NGINX Plus API"; auth_basic_user_file /path/to/passwd/file; } app_protect_dos_api; allow 192.168.1.0/24; deny all; } location = /dashboard-dos.html { root /usr/share/nginx/html; } } } ``` -------------------------------------------------------------------------------- title: "F5 DoS for NGINX 4.2" url: https://docs.nginx.com/nginx-app-protect-dos/releases/about-4.2/index.md -------------------------------------------------------------------------------- Here you can find the release information for F5 DoS for NGINX v4.2. F5 DoS for NGINX provides behavioral protection against Denial of Service (DoS) for your web applications. ## Release 4.2 August 15, 2023 In this release, F5 DoS for NGINX supports NGINX Plus R30. ### New Features - Support for Nginx Plus R30 - [Support for Ubuntu 22.04](learn-about-deployment.md#debian--ubuntu-installation) - Support for HTTP3/QUIC - Improvement of Embedded Server Health mechanism ### Supported Packages #### App Protect DoS ##### Alpine 3.15 - app-protect-dos-30.4.2.0-r1.apk ##### CentOS 7.4+ / RHEL 7.4+ / UBI7 - app-protect-dos-30.4.2.0-1.el7.ngx.x86_64.rpm ##### RHEL 8 and Rocky Linux 8 - app-protect-dos-30.4.2.0-1.el8.ngx.x86_64.rpm ##### Debian 11 - app-protect-dos_30.4.2.0-1~bullseye_amd64.deb ##### Ubuntu 20.04 - app-protect-dos_30.4.2.0-1~focal_amd64.deb ##### Ubuntu 22.04 - app-protect-dos_30.4.2.0-1~jammy_amd64.deb #### NGINX Plus - NGINX Plus R30 ### Important Notes - Installing L4 accelerated mitigation feature (install `app-protect-dos-ebpf`) configures `nginx` and `admd` to run with root privileges. - Support for `proxy_protocol` configuration: `proxy_protocol` monitor parameter should be used when the `listen` directive of the correspondent server block contains the `proxy_protocol` parameter. - If F5 WAF for NGINX is installed, app protect should be disabled for the location of DoS Live Activity Monitoring API. For example: ```shell location /api { app_protect_enable off; app_protect_dos_api; } ``` - Port configuration in `app_protect_dos_monitor` should correspond to the port, the server listens to. Misconfiguration can potentially cause a false attack declaration. For example: ```shell server { listen 8080; server_name myservice.com; location / { app_protect_dos_monitor "myservice.com:8080/"; } } ``` - `proxy_request_buffering` off is not supported. - gRPC and HTTP/2 protection require active monitoring of the protected service. The directive `app_protect_dos_monitor` is mandatory for the attack to be detected. - [TLS fingerprint](/nap-dos/directives-and-policy/learn-about-directives-and-policy.md#policy-directive-app_protect_dos_policy_file) feature is not used in CentOS 7.4 and RHEL 7 / UBI 7 due to the old OpenSSL version. The required OpenSSL version is 1.1.1 or higher. - Slow POST attack always mitigates with block action while other types of attacks can also be mitigated with redirection or JS challenges. - The recommended option of running NGINX Plus in a Docker Container is with the `daemon off` flag. It's mandatory for UBI 8. - The package dependencies for F5 DoS for NGINX have changed in this release, replacing the `curl` dependencies with `libcurl` only. For more information, see the [F5 DoS for NGINX Deployment Guide](/nap-dos/deployment-guide/learn-about-deployment.md#prerequisites). - Starting with this release, Ubuntu 18.04 support has been deprecated. -------------------------------------------------------------------------------- title: "Kubernetes" url: https://docs.nginx.com/nginx-app-protect-dos/deployment-guide/kubernetes/index.md description: Install F5 DoS for NGINX on Kubernetes using manifests to deploy DoS protection as a sidecar container alongside NGINX Plus. -------------------------------------------------------------------------------- This guide explains how to install F5 DoS for NGINX on Kubernetes. It covers the common steps for any Kubernetes-based deployment, then provides the manifest-based installation steps. ## Before you begin Before you start, make sure you have: - A functional Kubernetes cluster - An active F5 DoS for NGINX subscription (purchased or trial) - [Docker](https://docs.docker.com/get-started/get-docker/) To review supported operating systems, read the [Releases](/nap-dos/releases) topic. ## Download your subscription credentials Download the SSL certificate, private key, and the JWT license file associated with your NGINX Plus subscription from the MyF5 Customer Portal: - Log in to [MyF5](https://my.f5.com/manage/s/). - Go to **My Products & Plans > Subscriptions** to see your active subscriptions. - Find your NGINX products or services subscription, and select the **Subscription ID** for details. - Download the **.crt** and **.key** from the subscription page. - Download the **JSON Web Token** (JWT) from the subscription page. See [About Subscription Licenses](/solutions/about-subscription-licenses.md) for details. ## Create a Dockerfile In the same folder as your credential files, create a _Dockerfile_ based on your desired operating system image using an example from the following sections. As written, the examples install the most recent version of F5 DoS for NGINX and NGINX Plus. To build a specific version instead, set the `DOS_VERSION` build argument, using the syntax shown in the comment at the top of the Dockerfile for your operating system. On Debian and Ubuntu, also set `NGINX_PLUS_VERSION`, because `apt` considers only the newest `nginx-plus` available and does not select an older one to satisfy the module's dependency. To review which NGINX Plus release each version requires, read the [Releases](/nap-dos/releases) topic. ### Alpine Linux #### NGINX Plus ```dockerfile # syntax=docker/dockerfile:1 # Supported OS_VER's are 3.21/3.22 ARG OS_VER="3.22" # Base image FROM alpine:${OS_VER} # Leave empty to install the most recent version. To build a specific version, # set the argument, for example: # --build-arg DOS_VERSION="=37+4.9.6-r1" # NGINX Plus needs no pin here, because apk can select an older nginx-plus to # satisfy the module's nginx-plus-r dependency. ARG DOS_VERSION="" # Install F5 DoS for NGINX RUN --mount=type=secret,id=nginx-crt,dst=/etc/apk/cert.pem,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/apk/cert.key,mode=0644 \ --mount=type=secret,id=license-jwt,dst=license.jwt,mode=0644 \ wget -O /etc/apk/keys/nginx_signing.rsa.pub https://cs.nginx.com/static/keys/nginx_signing.rsa.pub \ && printf "https://pkgs.nginx.com/plus/alpine/v`egrep -o '^[0-9]+\.[0-9]+' /etc/alpine-release`/main\n" | tee -a /etc/apk/repositories \ && printf "https://pkgs.nginx.com/app-protect-dos/alpine/v`egrep -o '^[0-9]+\.[0-9]+' /etc/alpine-release`/main\n" | tee -a /etc/apk/repositories \ && apk update \ && apk add "app-protect-dos${DOS_VERSION}" \ && cat license.jwt > /etc/nginx/license.jwt \ && ln -sf /dev/stdout /var/log/nginx/access.log \ && ln -sf /dev/stderr /var/log/nginx/error.log \ && rm -rf /var/cache/apk/* # Copy configuration files: COPY nginx.conf custom_log_format.json /etc/nginx/ COPY entrypoint.sh /root/ RUN chmod +x /root/entrypoint.sh EXPOSE 80 STOPSIGNAL SIGQUIT CMD ["sh", "/root/entrypoint.sh"] ``` ### Amazon Linux #### NGINX Plus ```dockerfile # For AmazonLinux 2023: FROM amazonlinux:2023 # Leave empty to install the most recent version. To build a specific version, # set the argument, for example: # --build-arg DOS_VERSION="-37+4.9.6" # NGINX Plus needs no pin here, because dnf can select an older nginx-plus to # satisfy the module's nginx-plus-r dependency. ARG DOS_VERSION="" # Install F5 DoS for NGINX RUN --mount=type=secret,id=nginx-crt,dst=/etc/ssl/nginx/nginx-repo.crt,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/ssl/nginx/nginx-repo.key,mode=0644 \ --mount=type=secret,id=license-jwt,dst=license.jwt,mode=0644 \ dnf -y install ca-certificates \ && curl -o /etc/yum.repos.d/plus-amazonlinux2023.repo https://cs.nginx.com/static/files/plus-amazonlinux2023.repo \ && curl -o /etc/yum.repos.d/app-protect-dos-amazonlinux2023.repo https://cs.nginx.com/static/files/app-protect-dos-amazonlinux2023.repo \ && dnf install -y "app-protect-dos${DOS_VERSION}" \ && cat license.jwt > /etc/nginx/license.jwt \ && dnf clean all \ && rm -rf /var/cache/dnf \ && ln -sf /dev/stdout /var/log/nginx/access.log \ && ln -sf /dev/stderr /var/log/nginx/error.log # Copy configuration files: COPY nginx.conf custom_log_format.json /etc/nginx/ COPY entrypoint.sh /root/ RUN chmod +x /root/entrypoint.sh EXPOSE 80 STOPSIGNAL SIGQUIT CMD ["sh", "/root/entrypoint.sh"] ``` ### Debian #### NGINX Plus ```dockerfile # Where can be bullseye/bookworm/trixie FROM debian:bullseye # Leave both arguments empty to install the most recent version of each package. # To build a specific version, set both, for example: # --build-arg DOS_VERSION="=37+4.9.6-1~bullseye" --build-arg NGINX_PLUS_VERSION="=37.0.*-1~bullseye" # Pin NGINX Plus as well: apt considers only the newest nginx-plus available and # does not select an older one to satisfy the module's nginx-plus-r dependency. ARG DOS_VERSION="" ARG NGINX_PLUS_VERSION="" # Install F5 DoS for NGINX RUN --mount=type=secret,id=nginx-crt,dst=/etc/ssl/nginx/nginx-repo.crt,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/ssl/nginx/nginx-repo.key,mode=0644 \ --mount=type=secret,id=license-jwt,dst=license.jwt,mode=0644 \ apt-get update \ && DEBIAN_FRONTEND="noninteractive" apt-get install -y --no-install-recommends apt-transport-https lsb-release ca-certificates wget gnupg2 debian-archive-keyring \ && mkdir -p /etc/ssl/nginx/ /etc/nginx/ \ && wget -qO - https://cs.nginx.com/static/keys/nginx_signing.key | gpg --dearmor | tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null \ && printf "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] https://pkgs.nginx.com/plus/debian $(lsb_release -cs) nginx-plus\n" > /etc/apt/sources.list.d/nginx-plus.list \ && printf "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] https://pkgs.nginx.com/app-protect-dos/debian $(lsb_release -cs) nginx-plus\n" > /etc/apt/sources.list.d/nginx-app-protect-dos.list \ && wget -P /etc/apt/apt.conf.d https://cs.nginx.com/static/files/90pkgs-nginx \ && apt-get update \ && DEBIAN_FRONTEND="noninteractive" apt-get install -y "app-protect-dos${DOS_VERSION}" "nginx-plus-module-appprotectdos${DOS_VERSION}" "nginx-plus${NGINX_PLUS_VERSION}" \ && cat license.jwt > /etc/nginx/license.jwt \ && apt-get remove --purge --auto-remove -y apt-transport-https lsb-release gnupg2 wget \ && rm -rf /var/lib/apt/lists/* \ && ln -sf /dev/stdout /var/log/nginx/access.log \ && ln -sf /dev/stderr /var/log/nginx/error.log COPY nginx.conf /etc/nginx/ COPY entrypoint.sh /root/ RUN chmod +x /root/entrypoint.sh EXPOSE 80 STOPSIGNAL SIGQUIT CMD ["sh", "/root/entrypoint.sh"] ``` ### RHEL 8 #### NGINX Plus ```dockerfile # For UBI 8 FROM registry.access.redhat.com/ubi8 ARG RHEL_ORG ARG RHEL_ACTIVATION_KEY # Leave empty to install the most recent version. To build a specific version, # set the argument, for example: # --build-arg DOS_VERSION="-37+4.9.6" # NGINX Plus needs no pin here, because dnf can select an older nginx-plus to # satisfy the module's nginx-plus-r dependency. ARG DOS_VERSION="" # Install F5 DoS for NGINX RUN --mount=type=secret,id=nginx-crt,dst=/etc/ssl/nginx/nginx-repo.crt,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/ssl/nginx/nginx-repo.key,mode=0644 \ --mount=type=secret,id=license-jwt,dst=license.jwt,mode=0644 \ subscription-manager register --org=${RHEL_ORG} --activationkey=${RHEL_ACTIVATION_KEY} \ && subscription-manager refresh \ && subscription-manager attach --auto || true \ && subscription-manager repos --enable=rhel-8-for-x86_64-baseos-rpms \ && subscription-manager repos --enable=rhel-8-for-x86_64-appstream-rpms \ && dnf -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm \ && dnf -y install ca-certificates \ && curl -o /etc/yum.repos.d/plus-8.repo https://cs.nginx.com/static/files/plus-8.repo \ && curl -o /etc/yum.repos.d/app-protect-dos-8.repo https://cs.nginx.com/static/files/app-protect-dos-8.repo \ && dnf -y install "app-protect-dos${DOS_VERSION}" \ && cat license.jwt > /etc/nginx/license.jwt \ && rm /etc/yum.repos.d/plus-8.repo \ && rm /etc/yum.repos.d/app-protect-dos-8.repo \ && dnf clean all \ && rm -rf /var/cache/yum \ && ln -sf /dev/stdout /var/log/nginx/access.log \ && ln -sf /dev/stderr /var/log/nginx/error.log # Copy configuration files: COPY nginx.conf custom_log_format.json /etc/nginx/ COPY entrypoint.sh /root/ RUN chmod +x /root/entrypoint.sh EXPOSE 80 STOPSIGNAL SIGQUIT CMD ["sh", "/root/entrypoint.sh"] ``` ### RHEL 9 #### NGINX Plus ```dockerfile # For UBI 9 FROM registry.access.redhat.com/ubi9 ARG RHEL_ORG ARG RHEL_ACTIVATION_KEY # Leave empty to install the most recent version. To build a specific version, # set the argument, for example: # --build-arg DOS_VERSION="-37+4.9.6" # NGINX Plus needs no pin here, because dnf can select an older nginx-plus to # satisfy the module's nginx-plus-r dependency. ARG DOS_VERSION="" # Install F5 DoS for NGINX RUN --mount=type=secret,id=nginx-crt,dst=/etc/ssl/nginx/nginx-repo.crt,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/ssl/nginx/nginx-repo.key,mode=0644 \ --mount=type=secret,id=license-jwt,dst=license.jwt,mode=0644 \ subscription-manager register --org=${RHEL_ORG} --activationkey=${RHEL_ACTIVATION_KEY} \ && subscription-manager refresh \ && subscription-manager attach --auto || true \ && subscription-manager repos --enable=rhel-9-for-x86_64-baseos-rpms \ && subscription-manager repos --enable=rhel-9-for-x86_64-appstream-rpms \ && dnf -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm \ && dnf -y install ca-certificates \ && curl -o /etc/yum.repos.d/plus-9.repo https://cs.nginx.com/static/files/plus-9.repo \ && curl -o /etc/yum.repos.d/app-protect-dos-9.repo https://cs.nginx.com/static/files/app-protect-dos-9.repo \ && dnf -y install "app-protect-dos${DOS_VERSION}" \ && cat license.jwt > /etc/nginx/license.jwt \ && rm /etc/yum.repos.d/plus-9.repo \ && rm /etc/yum.repos.d/app-protect-dos-9.repo \ && dnf clean all \ && rm -rf /var/cache/yum \ && ln -sf /dev/stdout /var/log/nginx/access.log \ && ln -sf /dev/stderr /var/log/nginx/error.log # Copy configuration files: COPY nginx.conf custom_log_format.json /etc/nginx/ COPY entrypoint.sh /root/ RUN chmod +x /root/entrypoint.sh EXPOSE 80 STOPSIGNAL SIGQUIT CMD ["sh", "/root/entrypoint.sh"] ``` ### RHEL 10 #### NGINX Plus ```dockerfile # For UBI 10 FROM registry.access.redhat.com/ubi10 ARG RHEL_ORG ARG RHEL_ACTIVATION_KEY # Leave empty to install the most recent version. To build a specific version, # set the argument, for example: # --build-arg DOS_VERSION="-37+4.9.6" # NGINX Plus needs no pin here, because dnf can select an older nginx-plus to # satisfy the module's nginx-plus-r dependency. ARG DOS_VERSION="" # Install F5 DoS for NGINX RUN --mount=type=secret,id=nginx-crt,dst=/etc/ssl/nginx/nginx-repo.crt,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/ssl/nginx/nginx-repo.key,mode=0644 \ --mount=type=secret,id=license-jwt,dst=license.jwt,mode=0644 \ subscription-manager register --org=${RHEL_ORG} --activationkey=${RHEL_ACTIVATION_KEY} \ && subscription-manager refresh \ && subscription-manager attach --auto || true \ && subscription-manager repos --enable=rhel-10-for-x86_64-baseos-rpms \ && subscription-manager repos --enable=rhel-10-for-x86_64-appstream-rpms \ && dnf -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-10.noarch.rpm \ && dnf -y install ca-certificates \ && curl -o /etc/yum.repos.d/plus-10.repo https://cs.nginx.com/static/files/plus-10.repo \ && curl -o /etc/yum.repos.d/app-protect-dos-10.repo https://cs.nginx.com/static/files/app-protect-dos-10.repo \ && dnf -y install "app-protect-dos${DOS_VERSION}" \ && cat license.jwt > /etc/nginx/license.jwt \ && rm /etc/yum.repos.d/plus-10.repo \ && rm /etc/yum.repos.d/app-protect-dos-10.repo \ && dnf clean all \ && rm -rf /var/cache/yum \ && ln -sf /dev/stdout /var/log/nginx/access.log \ && ln -sf /dev/stderr /var/log/nginx/error.log # Copy configuration files: COPY nginx.conf custom_log_format.json /etc/nginx/ COPY entrypoint.sh /root/ RUN chmod +x /root/entrypoint.sh EXPOSE 80 STOPSIGNAL SIGQUIT CMD ["sh", "/root/entrypoint.sh"] ``` ### Rocky Linux 9 #### NGINX Plus ```dockerfile # syntax=docker/dockerfile:1 # For Rocky Linux 9 FROM rockylinux:9 # Leave empty to install the most recent version. To build a specific version, # set the argument, for example: # --build-arg DOS_VERSION="-37+4.9.6" # NGINX Plus needs no pin here, because dnf can select an older nginx-plus to # satisfy the module's nginx-plus-r dependency. ARG DOS_VERSION="" # Install F5 DoS for NGINX: RUN --mount=type=secret,id=nginx-crt,dst=/etc/ssl/nginx/nginx-repo.crt,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/ssl/nginx/nginx-repo.key,mode=0644 \ --mount=type=secret,id=license-jwt,dst=license.jwt,mode=0644 \ dnf -y install ca-certificates epel-release 'dnf-command(config-manager)' \ && curl -o /etc/yum.repos.d/plus-9.repo https://cs.nginx.com/static/files/plus-9.repo \ && curl -o /etc/yum.repos.d/app-protect-dos-9.repo https://cs.nginx.com/static/files/app-protect-dos-9.repo \ && dnf config-manager --set-enabled crb \ && dnf install -y "app-protect-dos${DOS_VERSION}" \ && cat license.jwt > /etc/nginx/license.jwt \ && dnf clean all \ && rm -rf /var/cache/dnf \ && ln -sf /dev/stdout /var/log/nginx/access.log \ && ln -sf /dev/stderr /var/log/nginx/error.log # Copy configuration files: COPY nginx.conf custom_log_format.json /etc/nginx/ COPY entrypoint.sh /root/ RUN chmod +x /root/entrypoint.sh EXPOSE 80 STOPSIGNAL SIGQUIT CMD ["sh", "/root/entrypoint.sh"] ``` ### Ubuntu #### NGINX Plus ```dockerfile # syntax=docker/dockerfile:1 # For Ubuntu # Where version can be: jammy/noble FROM ubuntu:noble # Leave both arguments empty to install the most recent version of each package. # To build a specific version, set both, for example: # --build-arg DOS_VERSION="=37+4.9.6-1~noble" --build-arg NGINX_PLUS_VERSION="=37.0.*-1~noble" # Pin NGINX Plus as well: apt considers only the newest nginx-plus available and # does not select an older one to satisfy the module's nginx-plus-r dependency. ARG DOS_VERSION="" ARG NGINX_PLUS_VERSION="" # Install F5 DoS for NGINX RUN --mount=type=secret,id=nginx-crt,dst=/etc/ssl/nginx/nginx-repo.crt,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/ssl/nginx/nginx-repo.key,mode=0644 \ --mount=type=secret,id=license-jwt,dst=license.jwt,mode=0644 \ apt-get update \ && apt-get install -y --no-install-recommends apt-transport-https lsb-release ca-certificates wget gnupg2 ubuntu-keyring \ && mkdir -p /etc/ssl/nginx/ /etc/nginx/ \ && wget -qO - https://cs.nginx.com/static/keys/nginx_signing.key | gpg --dearmor | tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null \ && printf "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] https://pkgs.nginx.com/plus/ubuntu $(lsb_release -cs) nginx-plus\n" > /etc/apt/sources.list.d/nginx-plus.list \ && printf "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] https://pkgs.nginx.com/app-protect-dos/ubuntu $(lsb_release -cs) nginx-plus\n" > /etc/apt/sources.list.d/nginx-app-protect-dos.list \ && wget -P /etc/apt/apt.conf.d https://cs.nginx.com/static/files/90pkgs-nginx \ && apt-get update \ && DEBIAN_FRONTEND="noninteractive" apt-get install -y "app-protect-dos${DOS_VERSION}" "nginx-plus-module-appprotectdos${DOS_VERSION}" "nginx-plus${NGINX_PLUS_VERSION}" \ && cat license.jwt > /etc/nginx/license.jwt \ && apt-get remove --purge --auto-remove -y apt-transport-https lsb-release gnupg2 wget && rm -rf /var/lib/apt/lists/* \ && ln -sf /dev/stdout /var/log/nginx/access.log \ && ln -sf /dev/stderr /var/log/nginx/error.log COPY nginx.conf /etc/nginx/ COPY entrypoint.sh /root/ RUN chmod +x /root/entrypoint.sh EXPOSE 80 STOPSIGNAL SIGQUIT CMD ["sh", "/root/entrypoint.sh"] ``` ## Create DOS entrypoint.sh Docker startup script which spins up all App Protect DoS processes, must have executable permissions ```shell #!/usr/bin/env bash USER=nginx LOGDIR=/var/log/adm # prepare environment mkdir -p /var/run/adm /tmp/cores ${LOGDIR} chmod 755 /var/run/adm /tmp/cores ${LOGDIR} chown ${USER}:${USER} /var/run/adm /tmp/cores ${LOGDIR} # run processes /bin/su -s /bin/bash -c "/usr/bin/adminstall > ${LOGDIR}/adminstall.log 2>&1" ${USER} /bin/su -s /bin/bash -c "/usr/bin/admd -d --log info > ${LOGDIR}/admd.log 2>&1 &" ${USER} /usr/sbin/nginx -g 'daemon off;' ``` ## Build the Docker image Your folder should contain the following files: - _nginx-repo.crt_ - _nginx-repo.key_ - _entrypoint.sh_ - _nginx.conf_ - _Dockerfile_ To build an image, use the following command, replacing `` as appropriate: ```shell sudo docker build --no-cache --platform linux/amd64 \ --secret id=nginx-crt,src=nginx-repo.crt \ --secret id=nginx-key,src=nginx-repo.key \ -t . ``` To build a specific version instead, add the two build arguments. For example, on Debian 11: ```shell sudo docker build --no-cache --platform linux/amd64 \ --secret id=nginx-crt,src=nginx-repo.crt \ --secret id=nginx-key,src=nginx-repo.key \ --build-arg DOS_VERSION="=37+4.9.6-1~bullseye" \ --build-arg NGINX_PLUS_VERSION="=37.0.*-1~bullseye" \ -t . ``` Once you have built the image, push it to your private image repository, which must be accessible to your Kubernetes cluster. ## Use Manifests to install F5 DOS for NGINX The `` argument must be the _contents_ of the file, not the file itself. Ensure there are no additional characters such as extra whitespace. ### Create Manifest files The default configuration provided creates two replicas, each hosting NGINX and DOS services together in a single Kubernetes pod. Create all of these files in a single folder (Such as `/manifests`). On manifest deployment, set the environment variables for image repository and tag. Set environment variable `DOS_IMAGE_REPOSITORY` with your actual nginx-dos image name. Set environment variable `DOS_IMAGE_TAG` with your actual nginx-dos image tag. ```dos-namespace.yaml apiVersion: v1 kind: Namespace metadata: name: app-protect-dos ``` ```dos-nginx-conf-configmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: dos-nginx-conf namespace: app-protect-dos data: nginx.conf: | user nginx; worker_processes auto; error_log /var/log/nginx/error.log error; worker_rlimit_nofile 65535; load_module modules/ngx_http_app_protect_dos_module.so; working_directory /tmp/cores; events { worker_connections 65535; } http { app_protect_dos_arb_fqdn svc-appprotect-dos-arb.arb.svc.cluster.local; sendfile on; tcp_nopush on; keepalive_timeout 65; log_format log_dos ', vs_name_al=$app_protect_dos_vs_name, ip=$remote_addr, tls_fp=$app_protect_dos_tls_fp, ' 'outcome=$app_protect_dos_outcome, reason=$app_protect_dos_outcome_reason, ' 'ip_tls=$remote_addr:$app_protect_dos_tls_fp, '; # Health endpoints for probes app_protect_dos_liveness on; # uri:/app_protect_dos_liveness port:8090 app_protect_dos_readiness on; # uri:/app_protect_dos_readiness port:8090 server { listen 8090; server_name probe; location / { proxy_pass http://localhost:8091; } } server { listen 8091; return 503; } server { listen 80 reuseport; server_name serv; proxy_http_version 1.1; access_log /var/log/nginx/access.log log_dos if=$loggable; app_protect_dos_security_log_enable on; app_protect_dos_security_log "/etc/app_protect_dos/log-default.json" syslog:server=:5261; app_protect_dos_policy_file "/etc/app_protect_dos/BADOSDefaultPolicy.json"; location / { app_protect_dos_enable on; app_protect_dos_name "main_serv"; app_protect_dos_monitor uri=http://serv:80/ protocol=http1; proxy_pass http://127.0.0.1/proxy$request_uri; } location /proxy { app_protect_dos_enable off; client_max_body_size 0; default_type text/html; return 200 "Hello! I got your URI request - $request_uri\n"; } } } ``` ```dos-log-default-configmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: dos-log-default namespace: app-protect-dos data: log-default.json: | { "filter": { "traffic-mitigation-stats": "all", "bad-actors": "all", "attack-signatures": "all" } } ``` ```dos-deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: app-protect-dos namespace: app-protect-dos labels: app: app-protect-dos spec: replicas: 1 revisionHistoryLimit: 10 selector: matchLabels: app: app-protect-dos strategy: type: RollingUpdate rollingUpdate: maxSurge: 2 maxUnavailable: 1 template: metadata: labels: app: app-protect-dos spec: containers: - name: nginx-app-protect-dos image: ${DOS_IMAGE_REPOSITORY}:${DOS_IMAGE_TAG} imagePullPolicy: Always command: ["/bin/bash", "-c"] args: - | /root/entrypoint.sh resources: requests: cpu: "200m" memory: "500Mi" limits: cpu: "900m" memory: "800Mi" ports: - containerPort: 80 name: web - containerPort: 8090 name: probe - containerPort: 8091 name: probe500 livenessProbe: httpGet: path: /app_protect_dos_liveness port: 8090 initialDelaySeconds: 5 periodSeconds: 10 readinessProbe: httpGet: path: /app_protect_dos_readiness port: 8090 initialDelaySeconds: 5 periodSeconds: 10 volumeMounts: - name: shared-dir mountPath: /shared/ - name: conf mountPath: /etc/nginx/nginx.conf subPath: nginx.conf - name: log-default mountPath: /etc/app_protect_dos/log-default.json subPath: log-default.json - name: license-token-volume mountPath: /etc/nginx/license.jwt subPath: license.jwt readOnly: true volumes: - name: shared-dir emptyDir: { } - name: conf configMap: name: dos-nginx-conf items: - key: nginx.conf path: nginx.conf - name: log-default configMap: name: dos-log-default defaultMode: 0644 items: - key: log-default.json path: log-default.json - name: license-token-volume secret: secretName: license-token items: - key: license.jwt path: license.jwt ``` ```dos-service.yaml apiVersion: v1 kind: Service metadata: name: nap-dos namespace: app-protect-dos spec: ports: - name: app port: 80 protocol: TCP selector: app: app-protect-dos type: NodePort ``` ### Start the Manifest deployment From the folder containing the YAML files from the previous step (Suggested as `/manifests`), deploy F5 DOS for NGINX using `kubectl`: ```shell export DOS_IMAGE_REPOSITORY= export DOS_IMAGE_TAG= kubectl apply -f manifests/dos-namespace.yaml kubectl create secret generic license-token --from-file=license.jwt=license.jwt --type=nginx.com/license --namespace app-protect-dos kubectl apply -f dos-manifest/dos-log-default-configmap.yaml kubectl apply -f dos-manifest/dos-nginx-conf-configmap.yaml kubectl apply -f manifests/dos-deployment.yaml kubectl apply -f manifests/dos-service.yaml ``` It will apply all the configuration defined in the files to your Kubernetes cluster. You can then check the status of the deployment with `kubectl get`: ```shell kubectl --namespace app-protect-dos get deployments kubectl --namespace app-protect-dos get pods kubectl --namespace app-protect-dos get services ``` You should see output similar to the following: ```text ~$ kubectl --namespace app-protect-dos get deployments NAME READY UP-TO-DATE AVAILABLE AGE app-protect-dos 1/1 1 1 1m ~$ kubectl --namespace app-protect-dos get pods NAME READY STATUS RESTARTS AGE app-protect-dos-586fb94947-8sjnc 1/1 Running 0 1m ~$ kubectl --namespace app-protect-dos get services NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE nap-dos LoadBalancer 10.43.83.225 80:30307/TCP 1m ``` ## Post-installation checks At this stage, you have finished deploying F5 DOS for NGINX. You can log in to the app-protect-dos pod using the following command: ```text kubectl exec -it app-protect-dos-586fb94947-8sjnc -n app-protect-dos -c nginx-app-protect-dos -- bash ``` Then run the following post-installation checks. You can run the following commands to ensure that F5 DoS for NGINX enforcement is operational. 1. Check that the three processes needed for F5 DoS for NGINX are running using `ps aux`: - admd - nginx: master process - nginx: worker process ```shell USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND nginx 7759 0.0 0.0 113120 1200 ? Ss Sep06 0:00 /bin/sh -c /usr/bin/admd -d --log info > /var/log/adm/admd.log 2>&1 root 7765 0.0 0.0 87964 1464 ? Ss Sep06 0:00 nginx: master process /usr/sbin/nginx -g daemon off; nginx 7767 0.0 0.1 615868 8188 ? Sl Sep06 0:04 nginx: worker process ``` 2. Verify that there are no NGINX errors in the `/var/log/nginx/error.log` and that the policy compiled successfully: ```shell 2025/12/07 09:14:34 [notice] 675#675: APP_PROTECT_DOS { "event": "shared_memory_connected", "worker_pid": 675, "mode": "operational", "mode_changed": true } 2025/12/07 09:14:34 [notice] 675#675: using the "epoll" event method 2025/12/07 09:14:34 [notice] 675#675: APP_PROTECT_DOS { "event": "configuration_load_success", "software_version": "36+4.8.3-1.el8.ngx"} 2025/12/07 09:14:34 [notice] 675#675: nginx/1.29.3 (nginx-plus-r36) 2025/12/07 09:14:34 [notice] 675#675: built by gcc 8.5.0 20210514 (Red Hat 8.5.0-28) (GCC) 2025/12/07 09:14:34 [notice] 675#675: OS: Linux 6.8.0-88-generic 2025/12/07 09:14:34 [notice] 675#675: getrlimit(RLIMIT_NOFILE): 1048576:1048576 2025/12/07 09:14:34 [notice] 675#675: start worker processes 2025/12/07 09:14:34 [notice] 675#675: start worker process 679 2025/12/07 09:14:34 [notice] 679#679: APP_PROTECT_DOS { "event": "shared_memory_connected", "worker_pid": 679, "mode": "operational", "mode_changed": true } ``` 3. Verify that when you simulate an attack, attacker IP addresses are blocked while legitimate traffic passes through: a. Simulate good traffic: ```shell echo "Start Good Traffic 2" while true; do curl ${VS}/good1 & curl ${VS}/good2 & curl ${VS}/good3 & curl ${VS}/good4 sleep 0.1 done & ``` b. After 7 minutes start the attack: ```shell while [ true ] do ab -B ${BAD_IP1} -l -r -n 1000000 -c 150 -d -H "Host: evil.net" -H "Pragma: no-cache" -H "Cache-Control: no-cache" -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8" -H "Upgrade-Insecure-Requests: 1" -H "User-Agent: WireXBot" -H "x-requested-with:" -H "Referer: http://10.0.2.1/none.html" -H "Accept-Encoding: gzip, deflate" -H "Accept-Language: en-US" http://${VS}/ & ab -B ${BAD_IP2} -l -r -n 1000000 -c 150 -d -H "Host: evil.net" -H "Pragma: no-cache" -H "Cache-Control: no-cache" -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8" -H "Upgrade-Insecure-Requests: 1" -H "User-Agent: WireXBot" -H "x-requested-with:" -H "Referer: http://10.0.2.1/none.html" -H "Accept-Encoding: gzip, deflate" -H "Accept-Language: en-US" http://${VS}/ & ab -B ${BAD_IP3} -l -r -n 1000000 -c 150 -d -s 10 -H "Host: evil.net" -H "Pragma: no-cache" -H "Cache-Control: no-cache" -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8" -H "Upgrade-Insecure-Requests: 1" -H "User-Agent: WireXBot" -H "x-requested-with:" -H "Referer: http://10.0.2.1/none.html" -H "Accept-Encoding: gzip, deflate" -H "Accept-Language: en-US" http://${VS}/ killall ab done ``` c. Verify that legitimate traffic continues as usual while the attack traffic is blocked. 4. For DOS with L4 accelerated mitigation enabled Check that the ebpf_manager_dos process needed for F5 DoS for NGINX is running using `ps aux | grep /usr/bin/ebpf_manager_dos`: ``` root 1 0.0 0.0 4324 3072 ? Ss 19:32 0:00 bash -c /usr/bin/ebpf_manager_dos 2>&1 | tee /shared/ebpf_dos.log root 7 0.2 0.0 1722732 14208 ? Sl 19:32 0:01 /usr/bin/ebpf_manager_dos root 46 0.0 0.0 3528 1792 pts/0 S+ 19:44 0:00 grep --color=auto /usr/bin/ebpf_manager_dos ``` Verify that there are no errors in the `/shared/ebpf_dos.log` and that the XDP program uploaded successfully: ```[2025-12-02 19:32:12] INFO: Uninstall old eBPF maps and XDP program [2025-12-02 19:32:13] INFO: Install eBPF maps and XDP program [2025-12-02 19:32:13] INFO: Start ebpf manager [2025-12-02 19:32:13] INFO: Version: 36+4.8.3-1~noble [2025-12-02 19:32:13] INFO: Start Periodic task for update time [2025-12-02 19:32:13] INFO: Owner of the UDS has been changed to user nginx and group nginx. [2025-12-02 19:32:13] INFO: Permissions of the UDS have been changed successfully for user nginx and group nginx. [2025-12-02 19:32:13] INFO: Async Callback Server listening on unix:/shared/ebpf_manager_dos_uds ``` To check F5 WAF for NGINX alongside F5 DoS for NGINX, just perform the normal tests as specified at [Admin Guide](https://docs.nginx.com/waf/install/virtual-environment/#post-installation-checks) ## F5 DoS for NGINX arbitrator ## F5 DoS for NGINX Arbitrator ### Overview F5 DoS for NGINX Arbitrator orchestrates all running F5 DoS for NGINX instances to synchronize local and global attack start and stop. F5 DoS for NGINX Arbitrator is a central coordinating component for managing multiple F5 DoS for NGINX instances in a network. It is needed when there is more than one F5 DoS for NGINX instance. Its primary function is to ensure that all instances are aware of and share the same state for each protected object. ### How the Arbitrator works - **Collecting state periodically**: The Arbitrator regularly collects state information from all running F5 DoS for NGINX instances. This collection occurs at set intervals, typically every 10 seconds. - **State initialization for new instances**: When a new F5 DoS for NGINX instance starts, it retrieves the initial state for each protected object from the Arbitrator rather than starting with an empty state. - **Updating state during an attack**: When an F5 DoS for NGINX instance detects an attack, it sends a notification to the Arbitrator. The Arbitrator updates the state of the affected protected object and propagates that state to all other instances. ### Why F5 DoS for NGINX Arbitrator is necessary F5 DoS for NGINX Arbitrator is essential for several reasons: - **Global state management**: Without the Arbitrator, each F5 DoS for NGINX instance manages its own isolated state for each protected object. This can lead to inconsistencies. For example, if instance A declares an attack on a protected object named "PO-Example," instance B remains unaware of it, potentially leaving the object vulnerable. - **Uniform attack detection**: With the Arbitrator, when instance A detects an attack on "PO-Example" and reports it, the Arbitrator updates the state of "PO-Example" and propagates it to all instances, including instance B. F5 DoS for NGINX Arbitrator maintains a consistent global state for protected objects across all F5 DoS for NGINX instances. This ensures attacks are detected and mitigated uniformly across your deployment. ### F5 DoS for NGINX Arbitrator Deployment 1. Pull the official F5 DoS for NGINX Arbitrator image with the command: ```shell docker pull docker-registry.nginx.com/nap-dos/app_protect_dos_arb:latest ``` 2. Create a container based on this image, for example, `app-protect-dos-arb` container: ```shell docker run --name app_protect_dos_arb -p 3000:3000 -d docker-registry.nginx.com/nap-dos/app_protect_dos_arb ``` 3. Verify that the `app-protect-dos-arb` container is up and running with the `docker ps` command. 4. DNS records are required for F5 DoS for NGINX Arbitrator to work properly and be accessible by F5 DoS for NGINX servers. Ensure that the `svc-appprotect-dos-arb` or configured Arbitrator FQDN (with `app_protect_dos_arb_fqdn` directive) has a valid DNS resolution. This step is necessary only for VM/Docker deployments with arbitrator. When the arbitrator is in the same Kubernetes namespace as F5 DoS for NGINX, this step is not needed. ### Multi-VM Deployment The Arbitrator service is standalone. If it goes down, it can be restarted and immediately recovers all required information from F5 DoS for NGINX instances, which report to it every 10 seconds. Its downtime is around 10 to 20 seconds, which does not affect F5 DoS for NGINX operation. F5 DoS for NGINX Arbitrator connects to port 3000. All modules try to connect to it automatically. If it's not accessible, each instance operates in standalone mode. F5 DoS for NGINX does not support mutual TLS (mTLS) or password authentication between DoS servers and the Arbitrator. Arbitrator is not exposed outside the namespace. It is the customer's responsibility to isolate it from external access. This applies to all Arbitrator deployments, not only multi-VM. ## Next steps -------------------------------------------------------------------------------- title: "F5 DoS for NGINX 4.1" url: https://docs.nginx.com/nginx-app-protect-dos/releases/about-4.1/index.md -------------------------------------------------------------------------------- Here you can find the release information for F5 DoS for NGINX v4.1. F5 DoS for NGINX provides behavioral protection against Denial of Service (DoS) for your web applications. ## Release 4.1 May 2, 2023 In this release, F5 DoS for NGINX supports NGINX Plus R29. ### New Features - Support for NGINX Plus R29 - Support for Rocky Linux 8 ### Supported Packages #### App Protect DoS ##### CentOS 7.4+ / RHEL 7.4+ / UBI7 - app-protect-dos-29+4.1.2-1.el7.ngx.x86_64.rpm ##### RHEL 8 and Rocky Linux 8 - app-protect-dos-29+4.1.2-1.el8.ngx.x86_64.rpm ##### Debian 11 - app-protect-dos_29+4.1.2-1~bullseye_amd64.deb ##### Ubuntu 18.04 - app-protect-dos_29+4.1.2-1~bionic_amd64.deb ##### Ubuntu 20.04 - app-protect-dos_29+4.1.2-1~focal_amd64.deb ##### Alpine 3.15 - app-protect-dos-29.4.1.2-r1.apk #### NGINX Plus - NGINX Plus R29 ### Important Notes - Installing L4 accelerated mitigation feature (install `app-protect-dos-ebpf`) configures `nginx` and `admd` to run with root privileges. - Support for `proxy_protocol` configuration: `proxy_protocol` monitor parameter should be used when the `listen` directive of the correspondent server block contains the `proxy_protocol` parameter. - If F5 WAF for NGINX is installed, app protect should be disabled for the location of DoS Live Activity Monitoring API. For example: ```shell location /api { app_protect_enable off; app_protect_dos_api; } ``` - Port configuration in `app_protect_dos_monitor` should correspond to the port, the server listens to. Misconfiguration can potentially cause a false attack declaration. For example: ```shell server { listen 8080; location / { app_protect_dos_monitor "myservice.com:8080"; } } ``` - `proxy_request_buffering` off is not supported. - gRPC and HTTP/2 protection require active monitoring of the protected service. The directive `app_protect_dos_monitor` is mandatory for the attack to be detected. - [TLS fingerprint](/nap-dos/directives-and-policy/learn-about-directives-and-policy.md#policy-directive-app_protect_dos_policy_file) feature is not used in CentOS 7.4 and RHEL 7 / UBI 7 due to the old OpenSSL version. The required OpenSSL version is 1.1.1 or higher. - Monitor directive `app_protect_dos_monitor` with `proxy_protocol` parameter can not be configured on Ubuntu 18.04. As a result, gRPC and HTTP/2 DoS protection for `proxy_protocol` configuration is not supported. - Slow POST attack always mitigates with block action while other types of attacks can also be mitigated with redirection or JS challenges. - The recommended option of running NGINX Plus in a Docker Container is with the `daemon off` flag. It's mandatory for UBI 8. - The package dependencies for F5 DoS for NGINX have changed in this release, replacing the `curl` dependencies with `libcurl` only. For more information, see the [F5 DoS for NGINX Deployment Guide](/nap-dos/deployment-guide/learn-about-deployment.md#prerequisites). -------------------------------------------------------------------------------- title: "Installing F5 WAF & DoS for NGINX AMIs on Amazon EC2" url: https://docs.nginx.com/nginx-app-protect-dos/deployment-guide/installing-nginx-plus-with-dos-and-waf-on-amazon-web-services/index.md description: Install F5 NGINX Plus, F5 WAF & DoS for NGINX Plus on Amazon Web Services (AWS), to provide sophisticated Layer 7 load balancing, Modern app security solution, behavioral DoS detection and mitigation that works seamlessly in DevOps environments for your apps running on AmazonLinux 2023, RHEL, Debian and Ubuntu Linux OS. -------------------------------------------------------------------------------- NGINX, Inc. participates in the Amazon Web Services (AWS) Partner Network as a Standard Technology Partner. We offer Amazon Machine Images (AMIs) for use in the Amazon Elastic Compute Cloud (EC2), available at the AWS Marketplace for several operating systems, including Amazon Linux, Red Hat Enterprise Linux, and Ubuntu. The AMIs contain combination of the following components: - Latest version of [F5 NGINX Plus](https://www.f5.com/products/nginx/nginx-plus), optimized for use on Amazon EC2 - Latest version of [F5 DoS for NGINX](https://docs.nginx.com/nginx-app-protect-dos/), optimized for use on Amazon EC2 - Latest version of [F5 WAF for NGINX](https://docs.nginx.com/nginx-app-protect-waf/), optimized for use on Amazon EC2 - Pre-packaged software for building highly available (HA) NGINX Plus configurations ## Install To quickly set up an environment with NGINX Plus, F5 WAF for NGINX and F5 DoS for NGINX on AWS: 1. Follow the instructions in [Getting Started with Amazon EC2 Linux Instances](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EC2_GetStarted.html) to sign up on AWS and get more information about EC2 itself. 2. Proceed to the product page for the appropriate AMI at the AWS Marketplace, and launch the AMI. - [NGINX Plus with F5 DoS for NGINX Plus – AmazonLinux 2023 AMI HVM](https://aws.amazon.com/marketplace/pp/prodview-x5nkvphsrg6ag) - [NGINX Plus with F5 DoS for NGINX Plus – RHEL8 Linux AMI HVM](https://aws.amazon.com/marketplace/pp/prodview-e6bifer7o6uzm) - [NGINX Plus with F5 DoS for NGINX Plus – RHEL9 Linux AMI HVM](https://aws.amazon.com/marketplace/pp/prodview-eoddgsaqujgkm) - [NGINX Plus with F5 DoS for NGINX Plus – Debian 11 Linux AMI HVM](https://aws.amazon.com/marketplace/pp/prodview-56oveh2rsxsbq) - [NGINX Plus with F5 DoS for NGINX Plus – Ubuntu 22.04 Linux AMI HVM](https://aws.amazon.com/marketplace/pp/prodview-l6f2q2ykrjufy) - [NGINX Plus with F5 DoS for NGINX Plus – Ubuntu 24.04 Linux AMI HVM](https://aws.amazon.com/marketplace/pp/prodview-osz3wmirx5zqg) - [NGINX Plus with F5 WAF & DoS for NGINX Plus – RHEL 8 Linux AMI HVM](https://aws.amazon.com/marketplace/pp/prodview-6pvnoyr2mp2co) - [NGINX Plus with F5 WAF & DoS for NGINX Plus – RHEL 9 Linux AMI HVM](https://aws.amazon.com/marketplace/pp/prodview-kmtitev2hhrym) - [NGINX Plus with F5 WAF & DoS for NGINX Plus – Debian 11 Linux AMI HVM](https://aws.amazon.com/marketplace/pp/prodview-wbyobl7a55vcu) - [NGINX Plus with F5 WAF & DoS for NGINX Plus – Ubuntu 22.04 Linux AMI HVM](https://aws.amazon.com/marketplace/pp/prodview-os7tw2atlkwpg) - [NGINX Plus with F5 WAF & DoS for NGINX Plus – Ubuntu 24.04 Linux AMI HVM](https://aws.amazon.com/marketplace/pp/prodview-pz64pqetwyrhw) Click the **Continue to Subscribe** button to proceed to the **Launch on EC2** page. 3. Select the type of launch by clicking the appropriate tab (1‑Click Launch, **Manual Launch**, or **Service Catalog**). Choose the desired options for billing, instance size, and so on, and click the Accept Software Terms… button. 4. When configuring the firewall rules, add a rule to accept web traffic on TCP ports 80 and 443 (this happens automatically if you launch from the 1-Click Launch tab). 5. As soon as the new EC2 instance launches, NGINX Plus starts automatically and serves a default **index.html** page. To view the page, use a web browser to access the public DNS name of the new instance. You can also check the status of the NGINX Plus server by logging into the EC2 instance and running this command: ```nginx /etc/init.d/nginx status ``` See [NGINX Plus on the AWS Cloud](https://www.nginx.com/resources/datasheets/nginx-quick-start-guide-for-aws/) deployment guide for details. 6. Verify latest NGINX PLUS / F5 DoS for NGINX / F5 WAF for NGINX packages are installed on EC2 after its first start: Verify F5 WAF for NGINX latest release from is installed by comparing with installed version from following command on the EC2 machine ```shell cat /opt/app_protect/VERSION /opt/app_protect/RELEASE ``` Verify F5 DoS for NGINX latest release from is installed by comparing with installed version from following command on the EC2 machine ```shell admd -v ``` Verify NGINX Plus latest release from is installed by comparing with installed version from following command on the EC2 machine ```shell nginx -v ``` In case NGINX PLUS / F5 DoS for NGINX / F5 WAF for NGINX packages are not latest release then upgrade the following with these commands: For F5 DoS for NGINX solution based on RedHat / AmazonLinux 2023 ```shell sudo service nginx stop sudo yum install app-protect-dos sudo systemctl start nginx ``` For F5 DoS for NGINX solution based on Debian / Ubuntu ```shell sudo service nginx stop sudo apt-get update sudo apt-get install app-protect-dos sudo service nginx start ``` For F5 WAF for NGINX solution based on RedHat / AmazonLinux 2023 ```shell sudo service nginx stop sudo yum install app-protect sudo systemctl start nginx ``` For F5 WAF for NGINX solution based on Debian / Ubuntu ```shell sudo service nginx stop sudo apt-get update sudo apt-get install app-protect sudo service nginx start ``` 7. If AMI includes [F5 WAF for NGINX](https://docs.nginx.com/nginx-app-protect-waf/) To enable F5 WAF for NGINX use the following steps: a. Load the F5 WAF for NGINX module on the main context in the `nginx.conf` file: ```shell load_module modules/ngx_http_app_protect_module.so; ``` b. Enable F5 WAF for NGINX on an `http/server/location` context in the `nginx.conf` file: ```shell app_protect_enable on; ``` c. Restart the NGINX service: ```shell sudo systemctl restart nginx ``` For more configuration information follow [F5 WAF for NGINX Configuration Guide](https://docs.nginx.com/nginx-app-protect-waf/configuration-guide/configuration/). 8. If AMI includes [F5 DoS for NGINX](https://docs.nginx.com/nginx-app-protect-dos/) To enable F5 DoS for NGINX use the following steps: a. Load the F5 DoS for NGINX module on the main context in the `nginx.conf` file: ```shell load_module modules/ngx_http_app_protect_dos_module.so; ``` b. Enable F5 DoS for NGINX on an `http/server/location` context in the `nginx.conf` file: ```shell app_protect_dos_enable on; app_protect_dos_name "App1"; app_protect_dos_policy_file "/etc/app_protect_dos/BADOSDefaultPolicy.json"; app_protect_dos_monitor uri=serv:80/; # Assuming server_name "serv" on port 80, with the root path "/" ``` c. Enable the L4 accelerated mitigation feature in the `http` context of the `nginx.conf` file: ```shell app_protect_dos_accelerated_mitigation on; ``` d. Restart the NGINX service: ```shell sudo systemctl restart nginx ``` For more configuration information follow [F5 DoS for NGINX Directives and Policy](https://docs.nginx.com/nginx-app-protect-dos/directives-and-policy/learn-about-directives-and-policy/). ## What If I Need Help? If you encounter any problems with NGINX Plus configuration, documentation is available at [nginx.org](https://nginx.org/en/docs/) and in the [NGINX Plus Admin Guide](https://docs.nginx.com/nginx/admin-guide/installing-nginx/). If you encounter any problems with F5 DoS for NGINX configuration, documentation is available at the [F5 DoS for NGINX Troubleshooting Guide](https://docs.nginx.com/nginx-app-protect-dos/troubleshooting/how-to-troubleshoot/). If you encounter any problems with F5 WAF for NGINX configuration, documentation is available at the [F5 WAF for NGINX Troubleshooting Guide](https://docs.nginx.com/nginx-app-protect-waf/). Customers who purchase an NGINX Plus AMI at the AWS Marketplace are eligible for the AWS support provided by the NGINX, Inc. engineering team. To activate support, submit the [AMI Support Activation](https://www.nginx.com/ami-support-activation/) form (you need your AWS account number). When you request support, we’ll ask you to provide the AWS account number that you registered, along with the IDs of your EC2 instances in some cases. -------------------------------------------------------------------------------- title: "Kubernetes with L4 accelerated mitigation" url: https://docs.nginx.com/nginx-app-protect-dos/deployment-guide/kubernetes-with-l4-accelerated-mitigation/index.md description: Install F5 DoS for NGINX on Kubernetes with L4 accelerated mitigation using eBPF to offload DoS blocking to the Linux kernel. -------------------------------------------------------------------------------- This guide explains how to install F5 DoS for NGINX on Kubernetes with L4 accelerated mitigation. By enabling the [`app_protect_dos_accelerated_mitigation`](/nap-dos/directives-and-policy/learn-about-directives-and-policy.md#accelerated-mitigation-directive-app_protect_dos_accelerated_mitigation) directive and running the DoS eBPF (Extended Berkeley Packet Filter) Manager as a sidecar container alongside the NGINX container, you can offload Layer 4 DoS mitigation to eBPF programs in the Linux kernel. This improves mitigation performance and reduces CPU usage on the NGINX container. Deployments with L4 accelerated mitigation require the NGINX and DoS containers to run with elevated privileges and additional Linux capabilities. This guide assumes you have a good understanding of Kubernetes security best practices and have secured your cluster accordingly. F5 DoS for NGINX requires the service to run with [`externalTrafficPolicy`](https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/#preserving-the-client-source-ip) set to `Local` to preserve the client source IP address for accurate DoS mitigation: ```yaml spec: externalTrafficPolicy: Local ``` It covers the common steps for any Kubernetes-based deployment, then provides the manifest-based installation steps. ## Before you begin Before you start, make sure you have: - A functional Kubernetes cluster - An active F5 DoS for NGINX subscription (purchased or trial) - [Docker](https://docs.docker.com/get-started/get-docker/) To review supported operating systems, read the [Releases](/nap-dos/releases) topic. ## Download your subscription credentials Download the SSL certificate, private key, and the JWT license file associated with your NGINX Plus subscription from the MyF5 Customer Portal: - Log in to [MyF5](https://my.f5.com/manage/s/). - Go to **My Products & Plans > Subscriptions** to see your active subscriptions. - Find your NGINX products or services subscription, and select the **Subscription ID** for details. - Download the **.crt** and **.key** from the subscription page. - Download the **JSON Web Token** (JWT) from the subscription page. See [About Subscription Licenses](/solutions/about-subscription-licenses.md) for details. ## Create a Dockerfile In the same folder as your credential files, create a _Dockerfile_ based on your desired operating system image using an example from the following sections. As written, the examples install the most recent version of F5 DoS for NGINX and NGINX Plus. To build a specific version instead, set the `DOS_VERSION` build argument, using the syntax shown in the comment at the top of the Dockerfile for your operating system. On Debian and Ubuntu, also set `NGINX_PLUS_VERSION`, because `apt` considers only the newest `nginx-plus` available and does not select an older one to satisfy the module's dependency. To review which NGINX Plus release each version requires, read the [Releases](/nap-dos/releases) topic. **Note:** Build the NGINX Plus image and the eBPF Manager image from the same F5 DoS for NGINX version. They are built separately, so if you pin one and not the other, the eBPF Manager sidecar can end up running a different version to the NGINX container. ### Alpine Linux #### NGINX Plus ```dockerfile # syntax=docker/dockerfile:1 # Supported OS_VER's are 3.21/3.22 ARG OS_VER="3.22" # Base image FROM alpine:${OS_VER} # Leave empty to install the most recent version. To build a specific version, # set the argument, for example: # --build-arg DOS_VERSION="=37+4.9.6-r1" # NGINX Plus needs no pin here, because apk can select an older nginx-plus to # satisfy the module's nginx-plus-r dependency. ARG DOS_VERSION="" # Install F5 DoS for NGINX RUN --mount=type=secret,id=nginx-crt,dst=/etc/apk/cert.pem,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/apk/cert.key,mode=0644 \ --mount=type=secret,id=license-jwt,dst=license.jwt,mode=0644 \ wget -O /etc/apk/keys/nginx_signing.rsa.pub https://cs.nginx.com/static/keys/nginx_signing.rsa.pub \ && printf "https://pkgs.nginx.com/plus/alpine/v`egrep -o '^[0-9]+\.[0-9]+' /etc/alpine-release`/main\n" | tee -a /etc/apk/repositories \ && printf "https://pkgs.nginx.com/app-protect-dos/alpine/v`egrep -o '^[0-9]+\.[0-9]+' /etc/alpine-release`/main\n" | tee -a /etc/apk/repositories \ && apk update \ && apk add "app-protect-dos${DOS_VERSION}" \ && cat license.jwt > /etc/nginx/license.jwt \ && ln -sf /dev/stdout /var/log/nginx/access.log \ && ln -sf /dev/stderr /var/log/nginx/error.log \ && rm -rf /var/cache/apk/* # Copy configuration files: COPY nginx.conf custom_log_format.json /etc/nginx/ COPY entrypoint.sh /root/ RUN chmod +x /root/entrypoint.sh EXPOSE 80 STOPSIGNAL SIGQUIT CMD ["sh", "/root/entrypoint.sh"] ``` #### EBPF Manager ```dockerfile # syntax=docker/dockerfile:1 # Supported OS_VER's are 3.21/3.22 ARG OS_VER="3.22" # Base image FROM alpine:${OS_VER} # Set this to the same version you built the F5 DoS for NGINX image with, for example: # --build-arg DOS_VERSION="=37+4.9.6-r1" # Left empty, the most recent version is installed, which may not match that image. ARG DOS_VERSION="" # Install F5 DoS ebpf manager for NGINX and create required nginx user RUN --mount=type=secret,id=nginx-crt,dst=/etc/apk/cert.pem,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/apk/cert.key,mode=0644 \ set -x \ # Create nginx user/group first, to be consistent throughout Docker variants \ && addgroup -S -g 101 nginx \ && adduser -S -u 101 -G nginx -h /nonexistent -s /sbin/nologin nginx \ && wget -O /etc/apk/keys/nginx_signing.rsa.pub https://cs.nginx.com/static/keys/nginx_signing.rsa.pub \ && printf "https://pkgs.nginx.com/app-protect-dos/alpine/v`egrep -o '^[0-9]+\.[0-9]+' /etc/alpine-release`/main\n" | tee -a /etc/apk/repositories \ && apk update \ && apk add "app-protect-dos-ebpf-manager${DOS_VERSION}" \ && rm -rf /var/cache/apk/* STOPSIGNAL SIGQUIT CMD ["bash", "-c", "/usr/bin/ebpf_manager_dos 2>&1 | tee /shared/ebpf_dos.log"] ``` ### Amazon Linux #### NGINX Plus ```dockerfile # For AmazonLinux 2023: FROM amazonlinux:2023 # Leave empty to install the most recent version. To build a specific version, # set the argument, for example: # --build-arg DOS_VERSION="-37+4.9.6" # NGINX Plus needs no pin here, because dnf can select an older nginx-plus to # satisfy the module's nginx-plus-r dependency. ARG DOS_VERSION="" # Install F5 DoS for NGINX RUN --mount=type=secret,id=nginx-crt,dst=/etc/ssl/nginx/nginx-repo.crt,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/ssl/nginx/nginx-repo.key,mode=0644 \ --mount=type=secret,id=license-jwt,dst=license.jwt,mode=0644 \ dnf -y install ca-certificates \ && curl -o /etc/yum.repos.d/plus-amazonlinux2023.repo https://cs.nginx.com/static/files/plus-amazonlinux2023.repo \ && curl -o /etc/yum.repos.d/app-protect-dos-amazonlinux2023.repo https://cs.nginx.com/static/files/app-protect-dos-amazonlinux2023.repo \ && dnf install -y "app-protect-dos${DOS_VERSION}" \ && cat license.jwt > /etc/nginx/license.jwt \ && dnf clean all \ && rm -rf /var/cache/dnf \ && ln -sf /dev/stdout /var/log/nginx/access.log \ && ln -sf /dev/stderr /var/log/nginx/error.log # Copy configuration files: COPY nginx.conf custom_log_format.json /etc/nginx/ COPY entrypoint.sh /root/ RUN chmod +x /root/entrypoint.sh EXPOSE 80 STOPSIGNAL SIGQUIT CMD ["sh", "/root/entrypoint.sh"] ``` #### EBPF Manager ```dockerfile # For AmazonLinux 2023: FROM amazonlinux:2023 # Set this to the same version you built the F5 DoS for NGINX image with, for example: # --build-arg DOS_VERSION="-37+4.9.6" # Left empty, the most recent version is installed, which may not match that image. ARG DOS_VERSION="" # Install F5 DoS ebpf manager for NGINX and create required nginx user RUN --mount=type=secret,id=nginx-crt,dst=/etc/ssl/nginx/nginx-repo.crt,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/ssl/nginx/nginx-repo.key,mode=0644 \ set -x \ && dnf -y install ca-certificates shadow-utils \ && groupadd --system --gid 101 nginx \ && useradd --system --gid nginx --no-create-home --home /nonexistent --comment "nginx user" --shell /bin/false --uid 101 nginx \ && curl -o /etc/yum.repos.d/app-protect-dos-amazonlinux2023.repo https://cs.nginx.com/static/files/app-protect-dos-amazonlinux2023.repo \ && dnf install -y "app-protect-dos-ebpf-manager${DOS_VERSION}" \ && dnf clean all \ && rm -rf /var/cache/dnf STOPSIGNAL SIGQUIT CMD ["bash", "-c", "/usr/bin/ebpf_manager_dos 2>&1 | tee /shared/ebpf_dos.log"] ``` ### Debian #### NGINX Plus ```dockerfile # Where can be bullseye/bookworm/trixie FROM debian:bullseye # Leave both arguments empty to install the most recent version of each package. # To build a specific version, set both, for example: # --build-arg DOS_VERSION="=37+4.9.6-1~bullseye" --build-arg NGINX_PLUS_VERSION="=37.0.*-1~bullseye" # Pin NGINX Plus as well: apt considers only the newest nginx-plus available and # does not select an older one to satisfy the module's nginx-plus-r dependency. ARG DOS_VERSION="" ARG NGINX_PLUS_VERSION="" # Install F5 DoS for NGINX RUN --mount=type=secret,id=nginx-crt,dst=/etc/ssl/nginx/nginx-repo.crt,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/ssl/nginx/nginx-repo.key,mode=0644 \ --mount=type=secret,id=license-jwt,dst=license.jwt,mode=0644 \ apt-get update \ && DEBIAN_FRONTEND="noninteractive" apt-get install -y --no-install-recommends apt-transport-https lsb-release ca-certificates wget gnupg2 debian-archive-keyring \ && mkdir -p /etc/ssl/nginx/ /etc/nginx/ \ && wget -qO - https://cs.nginx.com/static/keys/nginx_signing.key | gpg --dearmor | tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null \ && printf "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] https://pkgs.nginx.com/plus/debian $(lsb_release -cs) nginx-plus\n" > /etc/apt/sources.list.d/nginx-plus.list \ && printf "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] https://pkgs.nginx.com/app-protect-dos/debian $(lsb_release -cs) nginx-plus\n" > /etc/apt/sources.list.d/nginx-app-protect-dos.list \ && wget -P /etc/apt/apt.conf.d https://cs.nginx.com/static/files/90pkgs-nginx \ && apt-get update \ && DEBIAN_FRONTEND="noninteractive" apt-get install -y "app-protect-dos${DOS_VERSION}" "nginx-plus-module-appprotectdos${DOS_VERSION}" "nginx-plus${NGINX_PLUS_VERSION}" \ && cat license.jwt > /etc/nginx/license.jwt \ && apt-get remove --purge --auto-remove -y apt-transport-https lsb-release gnupg2 wget \ && rm -rf /var/lib/apt/lists/* \ && ln -sf /dev/stdout /var/log/nginx/access.log \ && ln -sf /dev/stderr /var/log/nginx/error.log COPY nginx.conf /etc/nginx/ COPY entrypoint.sh /root/ RUN chmod +x /root/entrypoint.sh EXPOSE 80 STOPSIGNAL SIGQUIT CMD ["sh", "/root/entrypoint.sh"] ``` #### EBPF Manager ```dockerfile # Where can be bullseye/bookworm FROM debian:bullseye # Set this to the same version you built the F5 DoS for NGINX image with, for example: # --build-arg DOS_VERSION="=37+4.9.6-1~bullseye" # Left empty, the most recent version is installed, which may not match that image. ARG DOS_VERSION="" # Install F5 DoS for NGINX RUN --mount=type=secret,id=nginx-crt,dst=/etc/ssl/nginx/nginx-repo.crt,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/ssl/nginx/nginx-repo.key,mode=0644 \ set -x \ # Create nginx user/group first, to be consistent throughout Docker variants \ && groupadd --system --gid 101 nginx \ && useradd --system --gid nginx --no-create-home --home /nonexistent --comment "nginx user" --shell /bin/false --uid 101 nginx \ && DEBIAN_FRONTEND=noninteractive apt-get update \ && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \ apt-transport-https \ lsb-release \ ca-certificates \ wget \ gnupg2 \ debian-archive-keyring \ && wget -qO - https://cs.nginx.com/static/keys/nginx_signing.key \ | gpg --dearmor \ | tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null \ && echo "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] https://pkgs.nginx.com/app-protect-dos/debian $(lsb_release -cs) nginx-plus" \ > /etc/apt/sources.list.d/nginx-app-protect-dos.list \ && wget -P /etc/apt/apt.conf.d https://cs.nginx.com/static/files/90pkgs-nginx \ && DEBIAN_FRONTEND=noninteractive apt-get update \ && DEBIAN_FRONTEND=noninteractive apt-get install -y "app-protect-dos-ebpf-manager${DOS_VERSION}" \ && apt-get clean \ && rm -rf /var/lib/apt/lists/* STOPSIGNAL SIGQUIT CMD ["bash", "-c", "/usr/bin/ebpf_manager_dos 2>&1 | tee /shared/ebpf_dos.log"] ``` ### RHEL 8 #### NGINX Plus ```dockerfile # For UBI 8 FROM registry.access.redhat.com/ubi8 ARG RHEL_ORG ARG RHEL_ACTIVATION_KEY # Leave empty to install the most recent version. To build a specific version, # set the argument, for example: # --build-arg DOS_VERSION="-37+4.9.6" # NGINX Plus needs no pin here, because dnf can select an older nginx-plus to # satisfy the module's nginx-plus-r dependency. ARG DOS_VERSION="" # Install F5 DoS for NGINX RUN --mount=type=secret,id=nginx-crt,dst=/etc/ssl/nginx/nginx-repo.crt,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/ssl/nginx/nginx-repo.key,mode=0644 \ --mount=type=secret,id=license-jwt,dst=license.jwt,mode=0644 \ subscription-manager register --org=${RHEL_ORG} --activationkey=${RHEL_ACTIVATION_KEY} \ && subscription-manager refresh \ && subscription-manager attach --auto || true \ && subscription-manager repos --enable=rhel-8-for-x86_64-baseos-rpms \ && subscription-manager repos --enable=rhel-8-for-x86_64-appstream-rpms \ && dnf -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm \ && dnf -y install ca-certificates \ && curl -o /etc/yum.repos.d/plus-8.repo https://cs.nginx.com/static/files/plus-8.repo \ && curl -o /etc/yum.repos.d/app-protect-dos-8.repo https://cs.nginx.com/static/files/app-protect-dos-8.repo \ && dnf -y install "app-protect-dos${DOS_VERSION}" \ && cat license.jwt > /etc/nginx/license.jwt \ && rm /etc/yum.repos.d/plus-8.repo \ && rm /etc/yum.repos.d/app-protect-dos-8.repo \ && dnf clean all \ && rm -rf /var/cache/yum \ && ln -sf /dev/stdout /var/log/nginx/access.log \ && ln -sf /dev/stderr /var/log/nginx/error.log # Copy configuration files: COPY nginx.conf custom_log_format.json /etc/nginx/ COPY entrypoint.sh /root/ RUN chmod +x /root/entrypoint.sh EXPOSE 80 STOPSIGNAL SIGQUIT CMD ["sh", "/root/entrypoint.sh"] ``` #### EBPF Manager ```dockerfile # For UBI 8 FROM registry.access.redhat.com/ubi8 ARG RHEL_ORG ARG RHEL_ACTIVATION_KEY # Set this to the same version you built the F5 DoS for NGINX image with, for example: # --build-arg DOS_VERSION="-37+4.9.6" # Left empty, the most recent version is installed, which may not match that image. ARG DOS_VERSION="" # Install F5 DoS ebpf manager for NGINX and create required nginx user RUN --mount=type=secret,id=nginx-crt,dst=/etc/ssl/nginx/nginx-repo.crt,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/ssl/nginx/nginx-repo.key,mode=0644 \ set -x \ # Create nginx user/group first, to be consistent throughout Docker variants \ && groupadd --system --gid 101 nginx \ && useradd --system --gid nginx --no-create-home --home /nonexistent --comment "nginx user" --shell /bin/false --uid 101 nginx \ && dnf -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm \ && dnf -y install ca-certificates \ && curl -o /etc/yum.repos.d/app-protect-dos-8.repo https://cs.nginx.com/static/files/app-protect-dos-8.repo \ && dnf -y install "app-protect-dos-ebpf-manager${DOS_VERSION}" \ && rm /etc/yum.repos.d/app-protect-dos-8.repo \ && dnf clean all \ && rm -rf /var/cache/yum STOPSIGNAL SIGQUIT CMD ["bash", "-c", "/usr/bin/ebpf_manager_dos 2>&1 | tee /shared/ebpf_dos.log"] ``` ### RHEL 9 #### NGINX Plus ```dockerfile # For UBI 9 FROM registry.access.redhat.com/ubi9 ARG RHEL_ORG ARG RHEL_ACTIVATION_KEY # Leave empty to install the most recent version. To build a specific version, # set the argument, for example: # --build-arg DOS_VERSION="-37+4.9.6" # NGINX Plus needs no pin here, because dnf can select an older nginx-plus to # satisfy the module's nginx-plus-r dependency. ARG DOS_VERSION="" # Install F5 DoS for NGINX RUN --mount=type=secret,id=nginx-crt,dst=/etc/ssl/nginx/nginx-repo.crt,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/ssl/nginx/nginx-repo.key,mode=0644 \ --mount=type=secret,id=license-jwt,dst=license.jwt,mode=0644 \ subscription-manager register --org=${RHEL_ORG} --activationkey=${RHEL_ACTIVATION_KEY} \ && subscription-manager refresh \ && subscription-manager attach --auto || true \ && subscription-manager repos --enable=rhel-9-for-x86_64-baseos-rpms \ && subscription-manager repos --enable=rhel-9-for-x86_64-appstream-rpms \ && dnf -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm \ && dnf -y install ca-certificates \ && curl -o /etc/yum.repos.d/plus-9.repo https://cs.nginx.com/static/files/plus-9.repo \ && curl -o /etc/yum.repos.d/app-protect-dos-9.repo https://cs.nginx.com/static/files/app-protect-dos-9.repo \ && dnf -y install "app-protect-dos${DOS_VERSION}" \ && cat license.jwt > /etc/nginx/license.jwt \ && rm /etc/yum.repos.d/plus-9.repo \ && rm /etc/yum.repos.d/app-protect-dos-9.repo \ && dnf clean all \ && rm -rf /var/cache/yum \ && ln -sf /dev/stdout /var/log/nginx/access.log \ && ln -sf /dev/stderr /var/log/nginx/error.log # Copy configuration files: COPY nginx.conf custom_log_format.json /etc/nginx/ COPY entrypoint.sh /root/ RUN chmod +x /root/entrypoint.sh EXPOSE 80 STOPSIGNAL SIGQUIT CMD ["sh", "/root/entrypoint.sh"] ``` #### EBPF Manager ```dockerfile # For UBI 9 FROM registry.access.redhat.com/ubi9 ARG RHEL_ORG ARG RHEL_ACTIVATION_KEY # Set this to the same version you built the F5 DoS for NGINX image with, for example: # --build-arg DOS_VERSION="-37+4.9.6" # Left empty, the most recent version is installed, which may not match that image. ARG DOS_VERSION="" # Install F5 DoS ebpf manager for NGINX and create required nginx user RUN --mount=type=secret,id=nginx-crt,dst=/etc/ssl/nginx/nginx-repo.crt,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/ssl/nginx/nginx-repo.key,mode=0644 \ set -x \ # Create nginx user/group first, to be consistent throughout Docker variants \ && groupadd --system --gid 101 nginx \ && useradd --system --gid nginx --no-create-home --home /nonexistent --comment "nginx user" --shell /bin/false --uid 101 nginx \ && dnf -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm \ && dnf -y install ca-certificates \ && curl -o /etc/yum.repos.d/app-protect-dos-9.repo https://cs.nginx.com/static/files/app-protect-dos-9.repo \ && dnf -y install "app-protect-dos-ebpf-manager${DOS_VERSION}" \ && rm /etc/yum.repos.d/app-protect-dos-9.repo \ && dnf clean all \ && rm -rf /var/cache/yum STOPSIGNAL SIGQUIT CMD ["bash", "-c", "/usr/bin/ebpf_manager_dos 2>&1 | tee /shared/ebpf_dos.log"] ``` ### RHEL 10 #### NGINX Plus ```dockerfile # For UBI 10 FROM registry.access.redhat.com/ubi10 ARG RHEL_ORG ARG RHEL_ACTIVATION_KEY # Leave empty to install the most recent version. To build a specific version, # set the argument, for example: # --build-arg DOS_VERSION="-37+4.9.6" # NGINX Plus needs no pin here, because dnf can select an older nginx-plus to # satisfy the module's nginx-plus-r dependency. ARG DOS_VERSION="" # Install F5 DoS for NGINX RUN --mount=type=secret,id=nginx-crt,dst=/etc/ssl/nginx/nginx-repo.crt,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/ssl/nginx/nginx-repo.key,mode=0644 \ --mount=type=secret,id=license-jwt,dst=license.jwt,mode=0644 \ subscription-manager register --org=${RHEL_ORG} --activationkey=${RHEL_ACTIVATION_KEY} \ && subscription-manager refresh \ && subscription-manager attach --auto || true \ && subscription-manager repos --enable=rhel-10-for-x86_64-baseos-rpms \ && subscription-manager repos --enable=rhel-10-for-x86_64-appstream-rpms \ && dnf -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-10.noarch.rpm \ && dnf -y install ca-certificates \ && curl -o /etc/yum.repos.d/plus-10.repo https://cs.nginx.com/static/files/plus-10.repo \ && curl -o /etc/yum.repos.d/app-protect-dos-10.repo https://cs.nginx.com/static/files/app-protect-dos-10.repo \ && dnf -y install "app-protect-dos${DOS_VERSION}" \ && cat license.jwt > /etc/nginx/license.jwt \ && rm /etc/yum.repos.d/plus-10.repo \ && rm /etc/yum.repos.d/app-protect-dos-10.repo \ && dnf clean all \ && rm -rf /var/cache/yum \ && ln -sf /dev/stdout /var/log/nginx/access.log \ && ln -sf /dev/stderr /var/log/nginx/error.log # Copy configuration files: COPY nginx.conf custom_log_format.json /etc/nginx/ COPY entrypoint.sh /root/ RUN chmod +x /root/entrypoint.sh EXPOSE 80 STOPSIGNAL SIGQUIT CMD ["sh", "/root/entrypoint.sh"] ``` #### EBPF Manager ```dockerfile # For UBI 10 FROM registry.access.redhat.com/ubi10 ARG RHEL_ORG ARG RHEL_ACTIVATION_KEY # Set this to the same version you built the F5 DoS for NGINX image with, for example: # --build-arg DOS_VERSION="-37+4.9.6" # Left empty, the most recent version is installed, which may not match that image. ARG DOS_VERSION="" # Install F5 DoS ebpf manager for NGINX and create required nginx user RUN --mount=type=secret,id=nginx-crt,dst=/etc/ssl/nginx/nginx-repo.crt,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/ssl/nginx/nginx-repo.key,mode=0644 \ set -x \ # Create nginx user/group first, to be consistent throughout Docker variants \ && groupadd --system --gid 101 nginx \ && useradd --system --gid nginx --no-create-home --home /nonexistent --comment "nginx user" --shell /bin/false --uid 101 nginx \ && dnf -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-10.noarch.rpm \ && dnf -y install ca-certificates \ && curl -o /etc/yum.repos.d/app-protect-dos-10.repo https://cs.nginx.com/static/files/app-protect-dos-10.repo \ && dnf -y install "app-protect-dos-ebpf-manager${DOS_VERSION}" \ && rm /etc/yum.repos.d/app-protect-dos-10.repo \ && dnf clean all \ && rm -rf /var/cache/yum STOPSIGNAL SIGQUIT CMD ["bash", "-c", "/usr/bin/ebpf_manager_dos 2>&1 | tee /shared/ebpf_dos.log"] ``` ### Rocky Linux 9 #### NGINX Plus ```dockerfile # syntax=docker/dockerfile:1 # For Rocky Linux 9 FROM rockylinux:9 # Leave empty to install the most recent version. To build a specific version, # set the argument, for example: # --build-arg DOS_VERSION="-37+4.9.6" # NGINX Plus needs no pin here, because dnf can select an older nginx-plus to # satisfy the module's nginx-plus-r dependency. ARG DOS_VERSION="" # Install F5 DoS for NGINX: RUN --mount=type=secret,id=nginx-crt,dst=/etc/ssl/nginx/nginx-repo.crt,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/ssl/nginx/nginx-repo.key,mode=0644 \ --mount=type=secret,id=license-jwt,dst=license.jwt,mode=0644 \ dnf -y install ca-certificates epel-release 'dnf-command(config-manager)' \ && curl -o /etc/yum.repos.d/plus-9.repo https://cs.nginx.com/static/files/plus-9.repo \ && curl -o /etc/yum.repos.d/app-protect-dos-9.repo https://cs.nginx.com/static/files/app-protect-dos-9.repo \ && dnf config-manager --set-enabled crb \ && dnf install -y "app-protect-dos${DOS_VERSION}" \ && cat license.jwt > /etc/nginx/license.jwt \ && dnf clean all \ && rm -rf /var/cache/dnf \ && ln -sf /dev/stdout /var/log/nginx/access.log \ && ln -sf /dev/stderr /var/log/nginx/error.log # Copy configuration files: COPY nginx.conf custom_log_format.json /etc/nginx/ COPY entrypoint.sh /root/ RUN chmod +x /root/entrypoint.sh EXPOSE 80 STOPSIGNAL SIGQUIT CMD ["sh", "/root/entrypoint.sh"] ``` #### EBPF Manager ```dockerfile # syntax=docker/dockerfile:1 # For Rocky Linux 9 FROM rockylinux:9 # Set this to the same version you built the F5 DoS for NGINX image with, for example: # --build-arg DOS_VERSION="-37+4.9.6" # Left empty, the most recent version is installed, which may not match that image. ARG DOS_VERSION="" # Install F5 DoS ebpf manager for NGINX and create required nginx user RUN --mount=type=secret,id=nginx-crt,dst=/etc/ssl/nginx/nginx-repo.crt,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/ssl/nginx/nginx-repo.key,mode=0644 \ set -x \ # Create nginx user/group first, to be consistent throughout Docker variants \ && groupadd --system --gid 101 nginx \ && useradd --system --gid nginx --no-create-home --home /nonexistent --comment "nginx user" --shell /bin/false --uid 101 nginx \ && dnf -y install ca-certificates epel-release 'dnf-command(config-manager)' \ && curl -o /etc/yum.repos.d/app-protect-dos-9.repo https://cs.nginx.com/static/files/app-protect-dos-9.repo \ && dnf config-manager --set-enabled crb \ && dnf install -y "app-protect-dos-ebpf-manager${DOS_VERSION}" \ && dnf clean all \ && rm -rf /var/cache/dnf STOPSIGNAL SIGQUIT CMD ["bash", "-c", "/usr/bin/ebpf_manager_dos 2>&1 | tee /shared/ebpf_dos.log"] ``` ### Ubuntu #### NGINX Plus ```dockerfile # syntax=docker/dockerfile:1 # For Ubuntu # Where version can be: jammy/noble FROM ubuntu:noble # Leave both arguments empty to install the most recent version of each package. # To build a specific version, set both, for example: # --build-arg DOS_VERSION="=37+4.9.6-1~noble" --build-arg NGINX_PLUS_VERSION="=37.0.*-1~noble" # Pin NGINX Plus as well: apt considers only the newest nginx-plus available and # does not select an older one to satisfy the module's nginx-plus-r dependency. ARG DOS_VERSION="" ARG NGINX_PLUS_VERSION="" # Install F5 DoS for NGINX RUN --mount=type=secret,id=nginx-crt,dst=/etc/ssl/nginx/nginx-repo.crt,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/ssl/nginx/nginx-repo.key,mode=0644 \ --mount=type=secret,id=license-jwt,dst=license.jwt,mode=0644 \ apt-get update \ && apt-get install -y --no-install-recommends apt-transport-https lsb-release ca-certificates wget gnupg2 ubuntu-keyring \ && mkdir -p /etc/ssl/nginx/ /etc/nginx/ \ && wget -qO - https://cs.nginx.com/static/keys/nginx_signing.key | gpg --dearmor | tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null \ && printf "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] https://pkgs.nginx.com/plus/ubuntu $(lsb_release -cs) nginx-plus\n" > /etc/apt/sources.list.d/nginx-plus.list \ && printf "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] https://pkgs.nginx.com/app-protect-dos/ubuntu $(lsb_release -cs) nginx-plus\n" > /etc/apt/sources.list.d/nginx-app-protect-dos.list \ && wget -P /etc/apt/apt.conf.d https://cs.nginx.com/static/files/90pkgs-nginx \ && apt-get update \ && DEBIAN_FRONTEND="noninteractive" apt-get install -y "app-protect-dos${DOS_VERSION}" "nginx-plus-module-appprotectdos${DOS_VERSION}" "nginx-plus${NGINX_PLUS_VERSION}" \ && cat license.jwt > /etc/nginx/license.jwt \ && apt-get remove --purge --auto-remove -y apt-transport-https lsb-release gnupg2 wget && rm -rf /var/lib/apt/lists/* \ && ln -sf /dev/stdout /var/log/nginx/access.log \ && ln -sf /dev/stderr /var/log/nginx/error.log COPY nginx.conf /etc/nginx/ COPY entrypoint.sh /root/ RUN chmod +x /root/entrypoint.sh EXPOSE 80 STOPSIGNAL SIGQUIT CMD ["sh", "/root/entrypoint.sh"] ``` #### EBPF Manager ```dockerfile # syntax=docker/dockerfile:1 # For Ubuntu # Where version can be: jammy/noble FROM ubuntu:noble # Set this to the same version you built the F5 DoS for NGINX image with, for example: # --build-arg DOS_VERSION="=37+4.9.6-1~noble" # Left empty, the most recent version is installed, which may not match that image. ARG DOS_VERSION="" # Install F5 DoS ebpf manager for NGINX and create required nginx user RUN --mount=type=secret,id=nginx-crt,dst=/etc/ssl/nginx/nginx-repo.crt,mode=0644 \ --mount=type=secret,id=nginx-key,dst=/etc/ssl/nginx/nginx-repo.key,mode=0644 \ set -x \ # Create nginx user/group first, to be consistent throughout Docker variants \ && groupadd --system --gid 101 nginx \ && useradd --system --gid nginx --no-create-home --home /nonexistent --comment "nginx user" --shell /bin/false --uid 101 nginx \ && DEBIAN_FRONTEND=noninteractive apt-get update \ && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \ apt-transport-https \ lsb-release \ ca-certificates \ wget \ gnupg2 \ ubuntu-keyring \ && wget -qO - https://cs.nginx.com/static/keys/nginx_signing.key \ | gpg --dearmor \ | tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null \ && echo "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] https://pkgs.nginx.com/app-protect-dos/ubuntu $(lsb_release -cs) nginx-plus" \ > /etc/apt/sources.list.d/nginx-app-protect-dos.list \ && wget -P /etc/apt/apt.conf.d https://cs.nginx.com/static/files/90pkgs-nginx \ && DEBIAN_FRONTEND=noninteractive apt-get update \ && DEBIAN_FRONTEND=noninteractive apt-get install -y "app-protect-dos-ebpf-manager${DOS_VERSION}" \ && apt-get clean \ && rm -rf /var/lib/apt/lists/* STOPSIGNAL SIGQUIT # Idle forever CMD ["bash", "-c", "/usr/bin/ebpf_manager_dos 2>&1 | tee /shared/ebpf_dos.log"] ``` ## Create DOS entrypoint.sh Docker startup script which spins up all App Protect DoS processes, must have executable permissions ```shell #!/usr/bin/env bash USER=nginx LOGDIR=/var/log/adm # prepare environment mkdir -p /var/run/adm /tmp/cores ${LOGDIR} chmod 755 /var/run/adm /tmp/cores ${LOGDIR} chown ${USER}:${USER} /var/run/adm /tmp/cores ${LOGDIR} # run processes /bin/su -s /bin/bash -c "/usr/bin/adminstall > ${LOGDIR}/adminstall.log 2>&1" ${USER} /bin/su -s /bin/bash -c "/usr/bin/admd -d --log info > ${LOGDIR}/admd.log 2>&1 &" ${USER} /usr/sbin/nginx -g 'daemon off;' ``` ## Build the DOS Docker image Your folder should contain the following files: - _nginx-repo.crt_ - _nginx-repo.key_ - _license.jwt_ - _entrypoint.sh_ - _nginx.conf_ - _Dockerfile_ To build an image, use the following command, replacing `` as appropriate: ```shell sudo docker build --no-cache --platform linux/amd64 \ --secret id=nginx-crt,src=nginx-repo.crt \ --secret id=nginx-key,src=nginx-repo.key \ -t . ``` To build a specific version instead, add the two build arguments. For example, on Debian 11: ```shell sudo docker build --no-cache --platform linux/amd64 \ --secret id=nginx-crt,src=nginx-repo.crt \ --secret id=nginx-key,src=nginx-repo.key \ --build-arg DOS_VERSION="=37+4.9.6-1~bullseye" \ --build-arg NGINX_PLUS_VERSION="=37.0.*-1~bullseye" \ -t . ``` ## Build the EBPF Manager Docker image Your folder should contain the following files: - _nginx-repo.crt_ - _nginx-repo.key_ - _Dockerfile_ To build an image, use the following command, replacing `` as appropriate: ```shell sudo docker build --no-cache --platform linux/amd64 \ --secret id=nginx-crt,src=nginx-repo.crt \ --secret id=nginx-key,src=nginx-repo.key \ -t . ``` If you pinned a version when building the NGINX Plus image, pass the same `DOS_VERSION` here: ```shell sudo docker build --no-cache --platform linux/amd64 \ --secret id=nginx-crt,src=nginx-repo.crt \ --secret id=nginx-key,src=nginx-repo.key \ --build-arg DOS_VERSION="=37+4.9.6-1~bullseye" \ -t . ``` Once you have built the DOS and EBPF images, push them to your private image repository, which should be accessible to your Kubernetes cluster. ## Use Manifests to install F5 DOS for NGINX ### Create Manifest files The default configuration provided creates two replicas, each hosting NGINX and DOS services together in a single Kubernetes pod. Create all of these files in a single folder (Such as `/manifests`). On manifest deployment environment variables need to be set for image repository and tag. `set enviorment variable DOS_IMAGE_REPOSITORY` with your actual nginx-dos image anmae. `set enviorment variable DOS_IMAGE_TAG` with your actual nginx-dos image tag. `set enviorment variable EBPF_IMAGE_REPOSITORY` with your actual ebpf-manager image name. `set enviorment variable EBPF_IMAGE_TAG` with your actual ebpf-manager image tag. ```dos-namespace.yaml apiVersion: v1 kind: Namespace metadata: name: app-protect-dos ``` ```dos-nginx-conf-configmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: dos-nginx-conf namespace: app-protect-dos data: nginx.conf: | user nginx; worker_processes auto; error_log /var/log/nginx/error.log error; worker_rlimit_nofile 65535; load_module modules/ngx_http_app_protect_dos_module.so; working_directory /tmp/cores; events { worker_connections 65535; } http { app_protect_dos_arb_fqdn svc-appprotect-dos-arb.arb.svc.cluster.local; sendfile on; tcp_nopush on; keepalive_timeout 65; log_format log_dos ', vs_name_al=$app_protect_dos_vs_name, ip=$remote_addr, tls_fp=$app_protect_dos_tls_fp, ' 'outcome=$app_protect_dos_outcome, reason=$app_protect_dos_outcome_reason, ' 'ip_tls=$remote_addr:$app_protect_dos_tls_fp, '; app_protect_dos_accelerated_mitigation on syn_drop=on; # Health endpoints for probes app_protect_dos_liveness on; # uri:/app_protect_dos_liveness port:8090 app_protect_dos_readiness on; # uri:/app_protect_dos_readiness port:8090 server { listen 8090; server_name probe; location / { proxy_pass http://localhost:8091; } } server { listen 8091; return 503; } server { listen 80 reuseport; server_name serv; access_log /var/log/nginx/access.log log_dos if=$loggable; app_protect_dos_security_log_enable on; app_protect_dos_security_log "/etc/app_protect_dos/log-default.json" syslog:server=:5261; app_protect_dos_policy_file "/etc/app_protect_dos/BADOSDefaultPolicy.json"; location / { app_protect_dos_enable on; app_protect_dos_name "main_serv"; app_protect_dos_monitor uri=http://serv:80/ protocol=http1; proxy_pass http://127.0.0.1/proxy$request_uri; } location /proxy { app_protect_dos_enable off; client_max_body_size 0; default_type text/html; return 200 "Hello! I got your URI request - $request_uri\n"; } } } ``` ```dos-log-default-configmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: dos-log-default namespace: app-protect-dos data: log-default.json: | { "filter": { "traffic-mitigation-stats": "all", "bad-actors": "all", "attack-signatures": "all" } } ``` ```dos-deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: app-protect-dos namespace: app-protect-dos labels: app: app-protect-dos spec: replicas: 1 revisionHistoryLimit: 10 selector: matchLabels: app: app-protect-dos strategy: type: RollingUpdate rollingUpdate: maxSurge: 2 maxUnavailable: 1 template: metadata: labels: app: app-protect-dos spec: containers: - name: nginx-app-protect-dos image: ${DOS_IMAGE_REPOSITORY}:${DOS_IMAGE_TAG} imagePullPolicy: Always command: ["/bin/bash", "-c"] args: - | /root/entrypoint.sh resources: requests: cpu: "200m" memory: "500Mi" limits: cpu: "900m" memory: "800Mi" ports: - containerPort: 80 name: web - containerPort: 8090 name: probe - containerPort: 8091 name: probe500 livenessProbe: httpGet: path: /app_protect_dos_liveness port: 8090 initialDelaySeconds: 5 periodSeconds: 10 readinessProbe: httpGet: path: /app_protect_dos_readiness port: 8090 initialDelaySeconds: 5 periodSeconds: 10 volumeMounts: - name: shared-dir mountPath: /shared/ - name: bpf mountPath: /sys/fs/bpf - name: conf mountPath: /etc/nginx/nginx.conf subPath: nginx.conf - name: log-default mountPath: /etc/app_protect_dos/log-default.json subPath: log-default.json - name: license-token-volume mountPath: /etc/nginx/license.jwt subPath: license.jwt readOnly: true - name: dos-ebpf-manager image: ${EBPF_IMAGE_REPOSITORY}:${EBPF_IMAGE_TAG} securityContext: privileged: true env: - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name # This is unique for every Pod volumeMounts: - name: shared-dir mountPath: /shared/ - name: bpf mountPath: /sys/fs/bpf volumes: - name: shared-dir emptyDir: {} - name: bpf hostPath: path: /sys/fs/bpf type: DirectoryOrCreate - name: conf configMap: name: dos-nginx-conf items: - key: nginx.conf path: nginx.conf - name: log-default configMap: name: dos-log-default defaultMode: 0644 items: - key: log-default.json path: log-default.json - name: license-token-volume secret: secretName: license-token items: - key: license.jwt path: license.jwt ``` ```dos-service.yaml apiVersion: v1 kind: Service metadata: name: nap-dos namespace: app-protect-dos spec: externalTrafficPolicy: Local ports: - name: app port: 80 protocol: TCP selector: app: app-protect-dos type: LoadBalancer ``` ### Start the Manifest deployment From the folder containing the YAML files from the previous step (Suggested as `/manifests`), deploy F5 DOS for NGINX using `kubectl`: ```shell export DOS_IMAGE_REPOSITORY= export DOS_IMAGE_TAG= export EBPF_IMAGE_REPOSITORY= export EBPF_IMAGE_TAG= kubectl apply -f manifests/dos-namespace.yaml kubectl apply -f manifests/dos-nginx-conf-configmap.yaml kubectl apply -f manifests/dos-log-default-configmap.yaml kubectl apply -f manifests/dos-deployment.yaml kubectl apply -f manifests/dos-service.yaml ``` It will apply all the configuration defined in the files to your Kubernetes cluster. You can then check the status of the deployment with `kubectl get`: ```shell kubectl -n app-protect-dos get deployments kubectl -n app-protect-dos get pods kubectl -n app-protect-dos get services ``` You should see output similar to the following: ```text ~$ kubectl -n app-protect-dos get deployments NAME READY UP-TO-DATE AVAILABLE AGE app-protect-dos 1/1 1 1 33s ~$ kubectl -n app-protect-dos get pods NAME READY STATUS RESTARTS AGE app-protect-dos-7f9798654c-7ncbl 2/2 Running 0 68s $ kubectl -n app-protect-dos get pods -o jsonpath='{range .items[*]}Pod: {.metadata.name} -> Containers: {.spec.containers[*].name}{"\n"}{end}' Pod: app-protect-dos-7f9798654c-7ncbl -> Containers: dos-ebpf-manager nginx-app-protect-dos ~$ kubectl -n app-protect-dos get services NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE nap-dos LoadBalancer 10.43.212.232 80:32586/TCP 93s ``` ## Post-Installation Checks At this stage, you have finished deploying F5 DOS for NGINX with EBPF L4 accelerated mitigation enabled You can login to dos-ebpf-manager container like following command ```text kubectl exec -it app-protect-dos-586fb94947-8sjnc -n app-protect-dos -c nginx-app-protect-dos -- bash kubectl exec -it app-protect-dos-586fb94947-8sjnc -n app-protect-dos -c dos-ebpf-manager -- bash ``` and can look at . You can run the following commands to ensure that F5 DoS for NGINX enforcement is operational. 1. Check that the three processes needed for F5 DoS for NGINX are running using `ps aux`: - admd - nginx: master process - nginx: worker process ```shell USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND nginx 7759 0.0 0.0 113120 1200 ? Ss Sep06 0:00 /bin/sh -c /usr/bin/admd -d --log info > /var/log/adm/admd.log 2>&1 root 7765 0.0 0.0 87964 1464 ? Ss Sep06 0:00 nginx: master process /usr/sbin/nginx -g daemon off; nginx 7767 0.0 0.1 615868 8188 ? Sl Sep06 0:04 nginx: worker process ``` 2. Verify that there are no NGINX errors in the `/var/log/nginx/error.log` and that the policy compiled successfully: ```shell 2025/12/07 09:14:34 [notice] 675#675: APP_PROTECT_DOS { "event": "shared_memory_connected", "worker_pid": 675, "mode": "operational", "mode_changed": true } 2025/12/07 09:14:34 [notice] 675#675: using the "epoll" event method 2025/12/07 09:14:34 [notice] 675#675: APP_PROTECT_DOS { "event": "configuration_load_success", "software_version": "36+4.8.3-1.el8.ngx"} 2025/12/07 09:14:34 [notice] 675#675: nginx/1.29.3 (nginx-plus-r36) 2025/12/07 09:14:34 [notice] 675#675: built by gcc 8.5.0 20210514 (Red Hat 8.5.0-28) (GCC) 2025/12/07 09:14:34 [notice] 675#675: OS: Linux 6.8.0-88-generic 2025/12/07 09:14:34 [notice] 675#675: getrlimit(RLIMIT_NOFILE): 1048576:1048576 2025/12/07 09:14:34 [notice] 675#675: start worker processes 2025/12/07 09:14:34 [notice] 675#675: start worker process 679 2025/12/07 09:14:34 [notice] 679#679: APP_PROTECT_DOS { "event": "shared_memory_connected", "worker_pid": 679, "mode": "operational", "mode_changed": true } ``` 3. Verify that when you simulate an attack, attacker IP addresses are blocked while legitimate traffic passes through: a. Simulate good traffic: ```shell echo "Start Good Traffic 2" while true; do curl ${VS}/good1 & curl ${VS}/good2 & curl ${VS}/good3 & curl ${VS}/good4 sleep 0.1 done & ``` b. After 7 minutes start the attack: ```shell while [ true ] do ab -B ${BAD_IP1} -l -r -n 1000000 -c 150 -d -H "Host: evil.net" -H "Pragma: no-cache" -H "Cache-Control: no-cache" -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8" -H "Upgrade-Insecure-Requests: 1" -H "User-Agent: WireXBot" -H "x-requested-with:" -H "Referer: http://10.0.2.1/none.html" -H "Accept-Encoding: gzip, deflate" -H "Accept-Language: en-US" http://${VS}/ & ab -B ${BAD_IP2} -l -r -n 1000000 -c 150 -d -H "Host: evil.net" -H "Pragma: no-cache" -H "Cache-Control: no-cache" -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8" -H "Upgrade-Insecure-Requests: 1" -H "User-Agent: WireXBot" -H "x-requested-with:" -H "Referer: http://10.0.2.1/none.html" -H "Accept-Encoding: gzip, deflate" -H "Accept-Language: en-US" http://${VS}/ & ab -B ${BAD_IP3} -l -r -n 1000000 -c 150 -d -s 10 -H "Host: evil.net" -H "Pragma: no-cache" -H "Cache-Control: no-cache" -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8" -H "Upgrade-Insecure-Requests: 1" -H "User-Agent: WireXBot" -H "x-requested-with:" -H "Referer: http://10.0.2.1/none.html" -H "Accept-Encoding: gzip, deflate" -H "Accept-Language: en-US" http://${VS}/ killall ab done ``` c. Verify that legitimate traffic continues as usual while the attack traffic is blocked. 4. For DOS with L4 accelerated mitigation enabled Check that the ebpf_manager_dos process needed for F5 DoS for NGINX is running using `ps aux | grep /usr/bin/ebpf_manager_dos`: ``` root 1 0.0 0.0 4324 3072 ? Ss 19:32 0:00 bash -c /usr/bin/ebpf_manager_dos 2>&1 | tee /shared/ebpf_dos.log root 7 0.2 0.0 1722732 14208 ? Sl 19:32 0:01 /usr/bin/ebpf_manager_dos root 46 0.0 0.0 3528 1792 pts/0 S+ 19:44 0:00 grep --color=auto /usr/bin/ebpf_manager_dos ``` Verify that there are no errors in the `/shared/ebpf_dos.log` and that the XDP program uploaded successfully: ```[2025-12-02 19:32:12] INFO: Uninstall old eBPF maps and XDP program [2025-12-02 19:32:13] INFO: Install eBPF maps and XDP program [2025-12-02 19:32:13] INFO: Start ebpf manager [2025-12-02 19:32:13] INFO: Version: 36+4.8.3-1~noble [2025-12-02 19:32:13] INFO: Start Periodic task for update time [2025-12-02 19:32:13] INFO: Owner of the UDS has been changed to user nginx and group nginx. [2025-12-02 19:32:13] INFO: Permissions of the UDS have been changed successfully for user nginx and group nginx. [2025-12-02 19:32:13] INFO: Async Callback Server listening on unix:/shared/ebpf_manager_dos_uds ``` To check F5 WAF for NGINX alongside F5 DoS for NGINX, just perform the normal tests as specified at [Admin Guide](https://docs.nginx.com/waf/install/virtual-environment/#post-installation-checks) ## F5 DoS for NGINX Arbitrator ## F5 DoS for NGINX Arbitrator ### Overview F5 DoS for NGINX Arbitrator orchestrates all running F5 DoS for NGINX instances to synchronize local and global attack start and stop. F5 DoS for NGINX Arbitrator is a central coordinating component for managing multiple F5 DoS for NGINX instances in a network. It is needed when there is more than one F5 DoS for NGINX instance. Its primary function is to ensure that all instances are aware of and share the same state for each protected object. ### How the Arbitrator works - **Collecting state periodically**: The Arbitrator regularly collects state information from all running F5 DoS for NGINX instances. This collection occurs at set intervals, typically every 10 seconds. - **State initialization for new instances**: When a new F5 DoS for NGINX instance starts, it retrieves the initial state for each protected object from the Arbitrator rather than starting with an empty state. - **Updating state during an attack**: When an F5 DoS for NGINX instance detects an attack, it sends a notification to the Arbitrator. The Arbitrator updates the state of the affected protected object and propagates that state to all other instances. ### Why F5 DoS for NGINX Arbitrator is necessary F5 DoS for NGINX Arbitrator is essential for several reasons: - **Global state management**: Without the Arbitrator, each F5 DoS for NGINX instance manages its own isolated state for each protected object. This can lead to inconsistencies. For example, if instance A declares an attack on a protected object named "PO-Example," instance B remains unaware of it, potentially leaving the object vulnerable. - **Uniform attack detection**: With the Arbitrator, when instance A detects an attack on "PO-Example" and reports it, the Arbitrator updates the state of "PO-Example" and propagates it to all instances, including instance B. F5 DoS for NGINX Arbitrator maintains a consistent global state for protected objects across all F5 DoS for NGINX instances. This ensures attacks are detected and mitigated uniformly across your deployment. ### F5 DoS for NGINX Arbitrator Deployment 1. Pull the official F5 DoS for NGINX Arbitrator image with the command: ```shell docker pull docker-registry.nginx.com/nap-dos/app_protect_dos_arb:latest ``` 2. Create a container based on this image, for example, `app-protect-dos-arb` container: ```shell docker run --name app_protect_dos_arb -p 3000:3000 -d docker-registry.nginx.com/nap-dos/app_protect_dos_arb ``` 3. Verify that the `app-protect-dos-arb` container is up and running with the `docker ps` command. 4. DNS records are required for F5 DoS for NGINX Arbitrator to work properly and be accessible by F5 DoS for NGINX servers. Ensure that the `svc-appprotect-dos-arb` or configured Arbitrator FQDN (with `app_protect_dos_arb_fqdn` directive) has a valid DNS resolution. This step is necessary only for VM/Docker deployments with arbitrator. When the arbitrator is in the same Kubernetes namespace as F5 DoS for NGINX, this step is not needed. ### Multi-VM Deployment The Arbitrator service is standalone. If it goes down, it can be restarted and immediately recovers all required information from F5 DoS for NGINX instances, which report to it every 10 seconds. Its downtime is around 10 to 20 seconds, which does not affect F5 DoS for NGINX operation. F5 DoS for NGINX Arbitrator connects to port 3000. All modules try to connect to it automatically. If it's not accessible, each instance operates in standalone mode. F5 DoS for NGINX does not support mutual TLS (mTLS) or password authentication between DoS servers and the Arbitrator. Arbitrator is not exposed outside the namespace. It is the customer's responsibility to isolate it from external access. This applies to all Arbitrator deployments, not only multi-VM. ## Next steps -------------------------------------------------------------------------------- title: "F5 DoS for NGINX Arbitrator 1.1.1" url: https://docs.nginx.com/nginx-app-protect-dos/releases/about-1.1.1/index.md -------------------------------------------------------------------------------- Here you can find the release information for F5 DoS for NGINX Arbitrator v1.1.1. ## Arbitrator Service Release 1.1.1 May 2, 2023 This release is focused on security. New image is backward compatible with the previous version. ### Resolved Issues - Removed unused REDIS module -------------------------------------------------------------------------------- title: "Directives and Policy" url: https://docs.nginx.com/nginx-app-protect-dos/directives-and-policy/learn-about-directives-and-policy/index.md -------------------------------------------------------------------------------- ## Introduction NGINX directives are specified in the `nginx.conf` file and are used to configure various modules of NGINX.
F5 DoS for NGINX has its own set of directives, which follow the same rules as other NGINX directives, and are used to enable and configure its features.
The table below provides a summary of all the F5 DoS for NGINX directives.
While only the first directive is mandatory for enabling F5 DoS for NGINX, it is recommended to use as many directives as possible to leverage the product’s full range of monitoring and application health detection capabilities. After adding these directives, ensure you reload NGINX and check the error log for any errors or warnings.
## Directives table Below is a summary of all F5 DoS for NGINX directives. Detailed descriptions of each directive can be found in the following sections. | Directive syntax | Options | Context | Description | Mandatory | Default | |-----------------------------------------------------------------------------------------------------------|----------|----------|--------------|------------|-------------------------------------------------------------------------------------------------------------------------------------| | [app_protect_dos_enable](#enable-directive-app_protect_dos_enable) | [on\|off] | http,
server,
location | Enable/Disable DoS protection | Yes | off | | [app_protect_dos_policy_file](#policy-directive-app_protect_dos_policy_file) | [FILE-PATH] | http,
server,
location | Load DoS configuration from a policy file | No | `/etc/app_protect_dos/BADOSDefaultPolicy.json` | | [app_protect_dos_name](#service-name-directive-app_protect_dos_name) | [SERVICE-NAME] | http,
server,
location | Name of protected object | No | **line_num-server_name**:*seq*-location_name

(i.e. `30-backend:1-/abc`) | | [app_protect_dos_monitor](#monitor-directive-app_protect_dos_monitor) | [uri=X] [protocol=Y] [timeout=Z] [proxy_protocol \| proxy_protocol=on\|off] | http,
server,
location | URI to monitor server's stress. Protocol and timeout are optional | Yes, unless its regular http1 traffic | uri - None
protocol - http1
timeout - 10 seconds for http1/websocket ; 5 seconds for http2/grpc
proxy_protocol - off | | [app_protect_dos_security_log_enable](#security-log-enable-directive-app_protect_dos_security_log_enable) | [on\|off] | http,
server,
location | Enable/Disable security logger | No | off | | [app_protect_dos_security_log](#security-log-directive-app_protect_dos_security_log) | [LOG-CONFIG-FILE] [DESTINATION] | http,
server,
location | Security logger configuration. Second argument:
"syslog:server={ip}:{port}" or
"stderr" or
"{absolute_file_path}" | No | `/etc/app_protect_dos/log-default.json stderr` | | [app_protect_dos_liveness](#liveness-probe-directive-app_protect_dos_liveness) | [on\|off] [uri:URI] [port:PORT] | http | Liveness prob. Second and third arguments are optional | No | `off uri:/app_protect_dos_liveness port:8090` | | [app_protect_dos_readiness](#readiness-probe-directive-app_protect_dos_readiness) | [on\|off] [uri:URI] [port:PORT] | http | Readiness prob. Second and third arguments are optional | No | `off uri:/app_protect_dos_readiness port:8090` | | [app_protect_dos_arb_fqdn](#arbitrator-fqdn-directive-app_protect_dos_arb_fqdn) | [FQDN\|IP address] | http | Arbitrator FQDN/IP address | No | `svc-appprotect-dos-arb` | | [app_protect_dos_api](#api-directive-app_protect_dos_api) | No arguments | location | Monitoring via Rest API (also includes the dashboard) | No | off | | [app_protect_dos_accelerated_mitigation](#api-directive-app_protect_dos_api) | [on\|off] [syn_drop=on\|off]| http | Enable/Disable L4 accelerated mitigation. Second argument is optional | No | off syn_drop=off | | [app_protect_dos_access_file](#access-file-directive-app_protect_dos_access_file) | [FILE-PATH] | http,
server,
location | Define allowlist policy from a file | No | None / disabled | ## Directives Info ### Enable directive (`app_protect_dos_enable`) Enables/disables App Protect DoS module in the relevant block/s.
It can be written in the following contexts: `location/server/http`. The derived blocks/contexts also inherit the directive. **For example:** A directive written in `http` context will be considered as if written also in all of the http's server blocks and their location blocks. In case of multiple directives in different contexts, the derived overwrites the base's directive. | Config | Expected | |------- | -------- | | Http block: directive is **on**
Server block: none is written
Location-1 block: none is written
Location-2 block: none is written | VS1: the server block
VS2: location-1 block
VS3: location-2 block | | Server block: directive is **on**
Location-1 block: directive is **off**
Location-2 block: none is written | VS1: the server block
VS2: location-2 block | | Http block: directive is **on**
Server block: directive is **off**
Location-1 block: directive is **on**
Location-2 block: none is written | VS1: location-1 block | **Example:** ```nginx app_protect_dos_enable on; ``` ### Policy directive (`app_protect_dos_policy_file`) This is the path to the JSON policy file which includes the product's configuration parameters. It can be written in the following contexts: `location/server/http`. The directive is optional. If not inserted then default path will be used, which is `/etc/app_protect_dos/BADOSDefaultPolicy.json`. If the configuration file doesn't exist or its attributes are invalid, default values will be used. `BADOSDefaultPolicy.json`: ```json { "mitigation_mode": "standard", "signatures": "on", "bad_actors": "on", "automation_tools_detection": "on", "tls_fingerprint" : "on" } ``` | Parameter name | Values | Default | Description | |:--------------- |:------- |:--------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | mitigation_mode | standard / conservative / none | standard| **Standard** - module allowed to use global rate mitigation
**Conservative** - module is not allowed to use global rate but only Signatures/Bad Actors mitigation
**None** - module is not allowed to mitigate. Only to learn and report. | | signatures | [on\|off] | on| Enable Signatures mechanism | | bad_actors | [on\|off] | on| Enable Bad Actors mechanism | | automation_tools_detection | [on\|off] | on |Enable the usage of automation tools detection (via cookies and redirect) | | tls_fingerprint| [on\|off] | on | Enable source identification using TLS fingerprinting| | Scenario | Result | |:--------- |:-------- | | Directive is not written | Default path is used: "/etc/app_protect_dos/BADOSDefaultPolicy.json" | | Directive is written | Path from the directive is used | | File not found / file syntax is invalid | Default values are used | **Example:** ```nginx app_protect_dos_policy_file /etc/app_protect_dos/BADOSPolicy.json; ``` ### Service Name directive (`app_protect_dos_name`) This is the Protected Object (VS) name, which should be unique and is used to identify the Protected Object in the logs.
It can be utilized within `location`, `server`, and `http` blocks.

Directive is optional. If not written, then each protected object (VS) will have an auto-generated name according to the following syntax: `line_number-server_name:seq-location_name` **For example:** `30-backend:1-/abc` - `line number:` the line number of the server block (`server {`) in the `nginx.conf` file (i.e. `30`)
- `server name:` taken from directive `server_name` (i.e. `backend`)
seq: 0 for server block, increments for each location block. i.e. VS created from server block will have 0 and VS's from location blocks will be 1,2,3,... (i.e. `1`) - `location name:` the name of the location (i.e. `/abc`) F5 DoS for NGINX supports up to 300 Protected Objects for versions up to 4.3, and 1,000 Protected Objects in version 4.4 and above.

**Example:** ```nginx app_protect_dos_name po-example; ``` ### Monitor directive (`app_protect_dos_monitor`) The `app_protect_dos_monitor` directive is used to monitor the stress level of the Protected Object.
Requests for this monitoring are sent from localhost (127.0.0.1) and pass through the NGINX configuration to simulate regular client traffic.
This directive is mandatory, except when using the `http1` protocol, where it is still strongly recommended for optimal performance.
**Syntax:**
app_protect_dos_monitor uri=path [protocol=http1|http2|grpc|websocket] [timeout=number] [proxy_protocol=on|off]; **Arguments**
Monitor directive has four arguments - **uri**, **protocol**, **timeout** and **proxy_protocol**. The first is mandatory and the rest are optional. - **URI** - The URI of the Protected Object as defined in the `nginx.conf`. This must point to a location block that proxies traffic to the backend (upstream) to ensure accurate monitoring.
Format: **scheme://server_name:port/location**. **Note:** For gRPC, the URI must specify a valid gRPC method (for example, /RouteGuide/GetFeature).
The health check is not a true gRPC client, so its requests do not conform to the gRPC wire protocol. As a result, the backend responds with grpc-status: 12 (UNIMPLEMENTED), which is expected and treated as a successful health check. Regular gRPC client traffic is unaffected by this behavior. - **Protocol** - determines the protocol type of the service. Options are `http1 / http2 / grpc / websocket`.
Default: `http1`.
**Note:** HTTP2 and gRPC are supported from F5 DoS for NGINX v2, while WebSocket is supported from F5 DoS for NGINX v4. - **Timeout** - determines how long (in seconds) should F5 DoS for NGINX wait for a response.
Default: 10 seconds for `http1/http2/websocket` and 5 seconds for `grpc`.
- **Proxy Protocol** - Should be used when the listen directive of the corresponding server block contains the proxy_protocol parameter. It adds an HAProxy PROXY protocol header to the monitor request.
Format is **proxy_protocol | proxy_protocol=on**.
Default: off.
**Note:** The proxy_protocol is supported from F5 DoS for NGINX v3.1. #### For Older Versions (F5 DoS for NGINX v1) In F5 DoS for NGINX v1, the app_protect_dos_monitor directive has only one argument: uri. Only HTTP1 is supported.

**Examples:** 1. HTTP/1 on Port 80: ```nginx listen 80; server_name serv; location / { # Protected Object is defined here app_protect_dos_monitor uri=http://serv:80/; } ``` **Note:** For F5 DoS for NGINX v1, use: app_protect_dos_monitor ; 2. HTTP/2 Over SSL ```nginx listen 443 http2 reuseport ssl; server_name serv; location / { # Protected Object is defined here app_protect_dos_monitor uri=https://serv:443/ protocol=http2 timeout=5; } ``` 3. gRPC Service on Port 50051 ```nginx listen 50051 http2 reuseport; server_name my_grpc; location /routeguide. { # Protected Object is defined here # Note: The URI must include a valid gRPC method (for example, /routeguide.RouteGuide/GetFeature). # The health check will expect a grpc-status of 12 (UNIMPLEMENTED) because it is not a true gRPC client. app_protect_dos_monitor uri=http://my_grpc:50051/routeguide.RouteGuide/GetFeature protocol=grpc; } ``` 4. Server with Proxy Protocol ```nginx listen 443 ssl http2 proxy_protocol; server_name serv; location / { # Protected Object is defined here # Note: Use proxy_protocol=on if the listen directive includes the "proxy_protocol" parameter. app_protect_dos_monitor uri=https://serv:443/ protocol=http2 timeout=5 proxy_protocol=on; } location /abc { # Protected Object is defined here app_protect_dos_monitor uri=https://serv:443/abc protocol=http2 timeout=5 proxy_protocol; } ``` 5. WebSocket service ```nginx listen 80; server_name wsserv; location /app/ { # WebSocket configuration required by NGINX proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "Upgrade"; proxy_set_header Host $host; # Protected Object is defined here app_protect_dos_monitor uri=http://wsserv:80/app/ protocol=websocket; } ``` ### Security log enable directive (`app_protect_dos_security_log_enable`) Enable/Disable App Protect DoS security logger. It can be used in `location/server/http` blocks. Directive is optional. If not written, then logger is disabled.
**Example:** ```nginx app_protect_dos_security_log_enable on; ``` ### Security log directive (`app_protect_dos_security_log`) This directive has two string arguments. First argument is the configuration file path, i.e. `/etc/app_protect_dos/log-default.json`. Second argument is the destination (the location which the events will be sent to). The destination can be one of three options: - `syslog:server={ip}:{port}`, i.e. `syslog:server=1.2.3.4:3000` - `stderr` (**default**) - `{absolute_file_path}`, i.e. `/shared/dos_sec_logger.log` Implemented according to: [F5 DoS for NGINX Security Log](/nap-dos/monitoring/security-log.md) **Note:** - When using stderr, make sure that the process `admd` is not redirecting the stderr output to file. - When using the Docker `entrypoint.sh` startup script from the admin guide, make sure that it doesn’t redirect stderr. **Examples:** - **Syslog:** ```nginx app_protect_dos_security_log "/etc/app_protect_dos/log-default.json" syslog:server=1.2.3.4:5000; ``` - **File:** ```nginx app_protect_dos_security_log "/etc/app_protect_dos/log-default.json" /shared/logger.log; ``` - **Stderr:** ```nginx app_protect_dos_security_log "/etc/app_protect_dos/log-default.json" stderr; ``` While `/etc/app_protect_dos/log-default.json` is: ```json { "filter": { "traffic-mitigation-stats": "all", "bad-actors": "top 10", "attack-signatures": "top 10" } } ``` ### Liveness probe directive (`app_protect_dos_liveness`) This directive has 3 arguments. | First argument | Second argument | Third argument | | :-------------- | :--------------- | :-------------- | | [on\|off] depending if this feature should be enabled or disabled. | URI Syntax is: `uri:___` | Port Syntax is: `port:____` | **Note:** Second and Third arguments are optional; if one or more is not written, the default will take place. If liveness is enabled, a request with URI and PORT that matches the probe configuration (i.e. `/app_protect_dos_liveness:8090`) will be answered with RC 200 "Alive" by our NGINX module, without being counted or pass to other handlers nor the backend server. Any other response will indicate that our NGINX module (F5 DoS for NGINX) has not received the request (possibly means that NGINX is down). **Example:** ```nginx app_protect_dos_liveness on uri:/liveness port:8090; ``` ### Readiness probe directive (`app_protect_dos_readiness`) This directive has 3 arguments. | First argument | Second argument | Third argument | | :-------------- | :--------------- | :-------------- | | [on\|off] depending if this feature should be enabled or disabled. | URI Syntax is: `uri:___` | Port Syntax is: `port:____` | **Note:** Second and Third arguments are optional; if one or more is not written, the default will take place. If readiness is enabled, a request with URI and PORT that matches the probe configuration (i.e. `/app_protect_dos_readiness:8090`) will be answered with RC 200 "Ready" or RC 503 "Not Ready" by our NGINX module, without being counted or pass to other handlers nor the backend server. Any other response will indicate that our NGINX module (F5 DoS for NGINX) has not received the request (possibly means that NGINX is down). RC 200 "Ready" will occur if two conditions are met: 1. NGINX worker successfully connected to the global shared memory block 2. ADMD process is running (and not stuck) **Example:** ```nginx app_protect_dos_readiness on uri:/readiness port:8090; ``` ### Arbitrator FQDN directive (`app_protect_dos_arb_fqdn`) Arbitrator FQDN directive has one argument which is the `FQDN/IP`. The argument is the FQDN to the desired Arbitrator. **Examples:** FQDN: ```nginx app_protect_dos_arb_fqdn svc-appprotect-dos-arb.arb.svc.cluster.local; ``` IP address: ```nginx app_protect_dos_arb_fqdn 192.168.1.10; ``` ### API directive (`app_protect_dos_api`) This directive is used to enable the App Protect DoS monitoring capability via REST API.
The REST API interface provides extended metrics information of the Protected Objects. It can be used by sending REST API requests manually or by using the App Protect DoS dashboard page. For more information refer to [F5 DoS for NGINX Live Activity Monitoring](/nap-dos/monitoring/live-activity-monitoring.md) **Example:** ```nginx http { # ... server { listen 192.168.1.23; # ... location /api { app_protect_dos_api; allow 192.168.1.0/24; deny all; } location = /dashboard-dos.html { root /usr/share/nginx/html; } } } ``` ### Accelerated mitigation directive (`app_protect_dos_accelerated_mitigation`) This directive is used to enable or disable App Protect DoS L4 accelerated mitigation.
syn_drop is an optional parameter; the default value is "off".
syn_drop=on mode is applicable for plane HTTP services or HTTPS when the `tls_fingerprint` feature is disabled. Refer to policy parameter "tls_fingerprint" in [Policy directive](#policy-directive-app_protect_dos_policy_file). In syn_drop mode, the SYN packet of detected bad actors will be dropped. syn_drop mode is recommended for the deployments of F5 DoS for NGINX at the perimeter network or behind L3 load balancer. Using this mode when F5 DoS for NGINX is deployed behind L4/L7 load balancer may result in the load balancer’s starvation during an attack. **Note:** To use this directive you need to install the eBPF package. For more information about eBPF, you can read the [Accelerating DDoS Mitigation with eBPF in F5 DoS for NGINX](https://www.f5.com/company/blog/nginx/accelerating-ddos-mitigation-with-ebpf-in-f5-nginx-app-protect-dos) article. **Example:** ```nginx app_protect_dos_accelerated_mitigation on syn_drop=on; ``` ### Access File directive (`app_protect_dos_access_file`) The `app_protect_dos_access_file` directive defines an allowlist policy from a specified file.
This enables specifying IP addresses or ranges that should never be blocked.
The format of the file is the same as used in F5 WAF for NGINX, making it easy to reuse existing WAF policies with defined allowlist IPs.

The directive is optional. If not written, then the allowlist feature is disabled.

The file should include a list of IP addresses or ranges in JSON format. Both IPv4 and IPv6 addresses are supported.
IPv4 addresses are in the format "a.b.c.d" where each component is a decimal number in the range 0-255.
IPv6 addresses are in the format "h1:h2:h3:h4:h5:h6:h7:h8" where each component is a hex number in the range 0x0-0xffff. Any contiguous range of zero elements can be omitted and replaced by "::".
IPv4 and IPv6 masks are written in the format "IP/xxx" (for example: /24), indicating the number of significant bits.

The JSON file should include the ipAddress field for specifying IP addresses or ranges, and the blockRequests field set to "transparent". The file can also include $ref to reference additional files containing more IP addresses.

Additionally, a second format is supported where the mask is specified in a dedicated field `ipMask`. The mask should be written in the standard subnet notation for IPv4 and IPv6 addresses. In this format, the `blockRequests` field should have a value of "never" instead of "transparent".

**Example:** ```nginx app_protect_dos_access_file "/etc/app_protect_dos/allowlist.json"; ``` **Example content of /etc/app_protect_dos/allowlist.json:** ```nginx { "policy": { "ip-address-lists": [ { "ipAddresses": [ { "ipAddress": "1.1.1.1" }, { "ipAddress": "1.1.1.1/32" }, { "ipAddress": "3.3.3.0/24" }, { "ipAddress": "2023::4ef3/128" }, { "ipAddress": "2034::2300/120" } ], "blockRequests": "transparent" }, { "$ref": "/etc/app_protect_dos/additional_ips.json", "blockRequests": "transparent" } ] } } ``` **Example content of /etc/app_protect_dos/additional_ips.json:** ```nginx { "ipAddresses": [ { "ipAddress": "2.2.2.2/32" }, { "ipAddress": "4.4.4.0/24" } ] } ``` **Example content with second format:** ```nginx { "policy":{ "whitelist-ips":[ { "ipAddress":"2034::2300", "ipMask":"ffff:ffff:ffff:ffff:ffff:ffff:ffff:ff00", "blockRequests":"never" }, { "blockRequests":"never", "ipAddress":"4.4.4.0", "ipMask":"255.255.255.0" } ] } } ``` -------------------------------------------------------------------------------- title: "F5 DoS for NGINX 4.0" url: https://docs.nginx.com/nginx-app-protect-dos/releases/about-4.0/index.md -------------------------------------------------------------------------------- Here you can find the release information for F5 DoS for NGINX v4.0. F5 DoS for NGINX provides behavioral protection against Denial of Service (DoS) for your web applications. ## Release 4.0 January 31, 2023 ### New Features - Distributed Denial of Service (DDoS) protection feature for WebSocket services.
Refer to the [Configuration Example](/nap-dos/directives-and-policy/learn-about-directives-and-policy.md#monitor-directive-app_protect_dos_monitor) for WebSocket services here.
- DDoS protection against slow attacks has been improved using machine learning algorithm on all types of traffic. - `app_protect_dos_monitor` directive, which monitors the proxied server, supports a new type of protocol - "WebSocket". ### Supported Packages #### App Protect DoS ##### CentOS 7.4+ / RHEL 7.4+ / UBI7 - app-protect-dos-28+4.0.1-1.el7.ngx.x86_64.rpm ##### RHEL 8 / UBI8 - app-protect-dos-28+4.0.1-1.el8.ngx.x86_64.rpm ##### Debian 11 - app-protect-dos_28+4.0.1.-1~bullseye_amd64.deb ##### Ubuntu 18.04 - app-protect-dos_28+4.0.1-1~bionic_amd64.deb ##### Ubuntu 20.04 - app-protect-dos_28+4.0.1-1~focal_amd64.deb ##### Alpine 3.15 - app-protect-dos-28.4.0.1-r1.apk #### NGINX Plus - NGINX Plus R28 ### Important Notes - WebSocket protection requires active monitoring of the protected service. The directive `app_protect_dos_monitor` is mandatory for these use cases, otherwise, the attack will not be detected. - Installing accelerated mitigation feature (install `app-protect-dos-ebpf`) configures `nginx` and `admd` to run with root privileges. - Support for `proxy_protocol` configuration: `proxy_protocol` monitor parameter should be used when the `listen` directive of the correspondent server block contains the `proxy_protocol` parameter. - If F5 WAF for NGINX is installed, app protect should be disabled for the location of DoS Live Activity Monitoring API. For example: ```shell location /api { app_protect_enable off; app_protect_dos_api; } ``` - Port configuration in `app_protect_dos_monitor` should correspond to the port, the server listens to. Misconfiguration can potentially cause a false attack declaration. For example: ```shell server { listen 8080; location / { app_protect_dos_monitor "myservice.com:8080"; } } ``` - gRPC and HTTP/2 protection require active monitoring of the protected service. The directive `app_protect_dos_monitor` is mandatory for the attack to be detected. - [TLS fingerprint](/nap-dos/directives-and-policy/learn-about-directives-and-policy.md#policy-directive-app_protect_dos_policy_file) feature is not used in CentOS 7.4 and RHEL 7 / UBI 7 due to the old OpenSSL version. The required OpenSSL version is 1.1.1 or higher. - Monitor directive `app_protect_dos_monitor` with `proxy_protocol` parameter can not be configured on Ubuntu 18.04. As a result, gRPC and HTTP/2 DoS protection for `proxy_protocol` configuration is not supported. - Slow attack always mitigates with block action while other types of attacks can also be mitigated with redirection or JS challenges. - The recommended option of running NGINX Plus in a Docker Container is with the `daemon off` flag. It's mandatory for UBI 8. -------------------------------------------------------------------------------- title: "Logs Overview" url: https://docs.nginx.com/nginx-app-protect-dos/monitoring/types-of-logs/index.md description: Learn about the F5 DoS for NGINX Logs Overview. -------------------------------------------------------------------------------- There are 4 types of logs corresponding to App Protect DoS: - [Security Log](#security-log): The general picture of the site and how App Protect DoS processed it, including anomalies and signatures found. - [Operation Log](#operation-log): Events such as configuration errors or warnings. - [Debug Logs](#debug-log): Technical messages at different levels of severity used to debug and resolve incidents and error behaviors. - [Request Logging](#request-log): F5 DoS for NGINX adds information to each request logged to NGINX's access logging mechanism. **Note:** NGINX does not have audit logs in the sense of *"**who** did **what**"*. This can be done either from the orchestration system controlling NGINX (such as NGINX Controller) or by tracking the configuration files and the systemd invocations using Linux tools. |Type|Log Configuration| Configuration Contexts| File Destination| Syslog Destination | |----|-----------------|-----------------------|-----------------|--------------------| | Debug | Log file name is the redirection in the invocation of the `admd` command line in the start script | Global (not part of `nginx.conf`)|Yes. Log file is in /var/log/adm/admd.log directory. There is currently no file rotation capability available for this log.| No | | Operation | `error_log` directive, part of core NGINX | `nginx.conf` - global | Yes, NGINX error log | Yes, NGINX error log | |Request |NGINX has two directives for the access log:
- **access_log** - to turn [on\|off]
- **log_format** - to specify the required information regarding each request

F5 DoS for NGINX has several variables that can be added to the log_format directive, such as $app_protect_dos_outcome.

For more information refer to [F5 DoS for NGINX Access Log](/nap-dos/monitoring/access-log.md) | `nginx.conf` - global| Yes, NGINX access log | Yes, NGINX access log | | Security | F5 DoS for NGINX has two directives in `nginx.conf`:
- app_protect_dos_security_log_enable to turn logging [on\|off]
- app_protect_dos_security_log to set it's logging configuration and destination

For more information refer:
- **Configuration**: [App Protect DoS - Directives and Policy](/nap-dos/directives-and-policy/learn-about-directives-and-policy.md)
- **Usage**: [F5 DoS for NGINX - Security Log](/nap-dos/monitoring/security-log.md) | `nginx.conf`: http, server, location | Yes, either stderr, or an absolute path to a local file are supported | Yes | ## Security Log The security logs contain information about the status of the protected objects. It gives a general picture about each protected object in terms of traffic intensity, health of the backend server, learning and mitigations. For more information refer to [F5 DoS for NGINX Security Log](/nap-dos/monitoring/security-log.md) documentation. ## Operation Log The operation logs consists of system operational and health events. The events are sent to the NGINX error log and are distinguished by the `APP_PROTECT_DOS` prefix followed by JSON body. The log level depends on the event: success is usually indicated by `notice`, while failure is indicated by `error`. The timestamp is inherent in the error log. For more information refer to [App Protect DoS Operation Log](/nap-dos/monitoring/operation-log.md) documentation. ## Request Log Access log is NGINX’s request log mechanism. It is controlled by two directives. ### log_format This directive determines the format of the log messages using predefined variables. App Protect DoS will enrich this set of variables with several security log attributes that are available to be included in the `log_format`. If `log_format` is not specified then the built-in format `combined` is used but, because that format does not include the extended App Protect DoS variables, this directive must be used when the user wants to add App Protect DoS information to the log. ### access_log This directive determines the destination of the `access_log` and the name of the format. The default is the file `/var/log/nginx/access.log` using the combined format. In order to use the custom format that includes the F5 DoS for NGINX variables, use this directive with the name of the desired format. ### App Protect DoS Variables These are the variables added to Access Log. They are a subset of the Security log attributes. The Security log names are prefixed with `$app_protect_dos`.
For more information refer to [F5 DoS for NGINX Access Log](/nap-dos/monitoring/access-log.md) ## Debug Log - F5 DoS for NGINX The F5 DoS for NGINX Debug log is used to troubleshoot the functionality of the product.
The path of the log is at a fixed location: `/var/log/adm/admd.log`. There are several log levels - `error`, `warning`, `info` and `debug`. The default is `info`. In order to change the log level at run time, the following command can be called: ```shell admd -l DEBUG_LEVEL ``` **Note:** `nginx.conf` does not refer to the F5 DoS for NGINX debug log configuration neither directly nor indirectly. ## NGINX Error log The NGINX Error log is used to troubleshoot the configuration portion of F5 DoS for NGINX. The file is called `error.log` and its path and debug level is determined in `nginx.conf` by the directive `error_log`.
For example: ```shell error_log /var/log/nginx/error.log debug; ``` -------------------------------------------------------------------------------- title: "F5 DoS for NGINX 3.1" url: https://docs.nginx.com/nginx-app-protect-dos/releases/about-3.1/index.md -------------------------------------------------------------------------------- Here you can find the release information for F5 DoS for NGINX v3.1. F5 DoS for NGINX provides behavioral protection against Denial of Service (DoS) for your web applications. ## Release 3.1 November 29, 2022 In this release, F5 DoS for NGINX supports NGINX Plus R28. ### New Features - Support for NGINX Plus R28. - F5 DoS for NGINX can be deployed behind L4/L7 load balancers when L4 protection is enabled. - Fixed the issue: Installing F5 DoS for NGINX with L4 (eBPF) mitigation behind L4/L7 load balancer may result in the load balancer's starvation during an attack. - Support for [proxy_protocol](/nap-dos/directives-and-policy/learn-about-directives-and-policy.md#monitor-directive-app_protect_dos_monitor) configuration for server health monitoring.
Previously, server health monitoring could not be used when the listen directive of the correspondent server block contained the `proxy_protocol` parameter.
This disallowed using F5 DoS for NGINX for Denial of Service (DoS) protection for HTTP2 and gRPC protected objects in the `proxy_protocol` configuration. ### Supported Packages #### App Protect DoS ##### CentOS 7.4+ / RHEL 7.4+ / UBI7 - app-protect-dos-28+3.1.7-1.el7.ngx.x86_64.rpm ##### RHEL 8 / UBI8 - app-protect-dos-28+3.1.7-1.el8.ngx.x86_64.rpm ##### Debian 11 - app-protect-dos_28+3.1.7-1~bullseye_amd64.deb ##### Ubuntu 18.04 - app-protect-dos_28+3.1.7-1~bionic_amd64.deb ##### Ubuntu 20.04 - app-protect-dos_28+3.1.7-1~focal_amd64.deb ##### Alpine 3.15 - app-protect-dos-28.3.1.7-r1.apk #### NGINX Plus - NGINX Plus R28 ### Important Notes - Installing L4 accelerated mitigation feature (install `app-protect-dos-ebpf`) configures `nginx` and `admd` to run with root privileges. - Support for `proxy_protocol` configuration: `proxy_protocol` monitor parameter should be used when the listen directive of the correspondent server block contains the `proxy_protocol` parameter. - If F5 WAF for NGINX is installed, app protect should be disabled for the location of DoS Live Activity Monitoring API. For example: ```shell location /api { app_protect_enable off; app_protect_dos_api; } ``` - Misconfiguration of `app_protect_dos_monitor` potentially can cause a false attack declaration. Port configuration should correspond to the port the server listens to. For example: ```shell server { listen 8080; location / { app_protect_dos_monitor "myservice.com:8080"; } } ``` - `proxy_request_buffering` off is not supported. - gRPC and HTTP/2 protection require active monitoring of the protected service. The directive `app_protect_dos_monitor` is mandatory for these use cases, otherwise, the attack will not be detected. - [TLS fingerprint](/nap-dos/directives-and-policy/learn-about-directives-and-policy.md#policy-directive-app_protect_dos_policy_file) feature is not used in CentOS 7.4 and RHEL 7 / UBI 7 due to the old OpenSSL version. The required OpenSSL version is 1.1.1 or higher. - Monitor directive (app_protect_monitor) with the `proxy_protocol` parameter can not be configured on Ubuntu 18.04. As a result, gRPC and HTTP/2 DoS protection for `proxy_protocol` configuration is not supported. - Slow POST attack always mitigates with block action while other types of attacks can also be mitigated with redirection or JS challenges. - The recommended option of running NGINX Plus in a Docker Container is with the `daemon off` flag. It's mandatory for UBI 8. -------------------------------------------------------------------------------- title: "Live Activity Monitoring" url: https://docs.nginx.com/nginx-app-protect-dos/monitoring/live-activity-monitoring/index.md description: Learn about the F5 DoS for NGINX Live Activity Monitoring. -------------------------------------------------------------------------------- ![F5 DoS for NGINX Dashboard](/dashboard/dos-tab.png) ## Overview F5 DoS for NGINX offers a variety of application monitoring tools: - **App Protect DoS Dashboard Page**: This dynamic interface provides real-time monitoring and details of Protected Objects. - **F5 DoS for NGINX REST API**: This interface offers comprehensive metrics about the Protected Objects. ## Prerequisites - NGINX Plus R26 or later is required for accessing the F5 DoS for NGINX REST API and the DoS Dashboard. ## API Configuration Steps To activate the API: - Define a `server` block within the `http` context to handle the API: ```nginx http { server { # insert your API configuration here } } ``` - Create a `location` for API requests and include the `app_protect_dos_api` directive: ```nginx http { # ... server { listen 192.168.1.23; # ... location /api { app_protect_dos_api; # ... } } } ``` - Limit API location access, for example, permit only local network access using `allow` and `deny` directives: ```nginx http { # ... server { listen 192.168.1.23; # ... location /api { app_protect_dos_api; allow 192.168.1.0/24; deny all; } } } ``` - Implement HTTP basic authentication to restrict access to PATCH, POST, and DELETE methods to certain users: ```nginx http { # ... server { listen 192.168.1.23; # ... location /api { limit_except GET { auth_basic "NGINX Plus API"; auth_basic_user_file /path/to/passwd/file; } app_protect_dos_api; allow 192.168.1.0/24; deny all; } } } ``` - Enable the DoS Dashboard by defining the /dashboard-dos.html location, which is typically in the root directory: ```nginx http { # ... server { listen 192.168.1.23; # ... location /api { limit_except GET { auth_basic "NGINX Plus API"; auth_basic_user_file /path/to/passwd/file; } app_protect_dos_api; allow 192.168.1.0/24; deny all; } location = /dashboard-dos.html { root /usr/share/nginx/html; } } } ``` ## Use the Dashboard ### Access the Dashboard To view the Dashboard, enter the corresponding address in your browser's address bar.
For example, "http://192.168.1.23/dashboard-dos.html" will show the Dashboard page located in /usr/share/nginx/html as indicated in the root directive. ### DoS Tab Overview The **DoS** tab displays real-time statistics, settings, and traffic graphs for each Protected Object.
In multi-instance environments with an arbitrator, these statistics will be combined. | Metric name | Values | Description | |:--------------- |:------- |:-----------| | Name | - | The name of the Protected Object, as defined by the `app_protect_dos_name` directive (or auto-generated if not present) | | Health | [good\|bad] | The health of the backend server, as defined by the `uri` argument of the `app_protect_dos_monitor` directive | | Under Attack | [yes\|no] | Whether the Protected Object is under attack or not | | Req/s | - | Number of incoming requests per second | | Mitigations/s | - | Number of mitigated requests per second | | Requests | - | Total number of incoming requests | | Mitigations | - | Total number of mitigated requests | | Learning | [ready\|ba only\|not ready] | Whether F5 DoS for NGINX collected enough data to protect the Protected Object. Note: `ba only` means that request thresholds were learned, but full request sampling is not complete. In this status, F5 DoS for NGINX will only detect bad actors (by IP or IP+TLS fingerprint) and will not utilize mitigation signatures. | | Protocol | [http1\|http2\|grpc] | As defined by the `protocol` argument of the `app_protect_dos_monitor` directive | | Mitigation Mode | [standard\|conservative\|none] | As defined by the `mitigation_mode` object in the JSON policy file from the `app_protect_dos_policy_file` directive | | Signatures | [on\|off] | As defined by the `signatures` object in the JSON policy file from the `app_protect_dos_policy_file` directive. Values - on/off | | Bad Actors | [on\|off] | As defined by the `bad_actors` object in the JSON policy file from the `app_protect_dos_policy_file` directive | | Automation Tools Detection | [on\|off] | As defined by the `automation_tools_detection` object in the JSON policy file from the `app_protect_dos_policy_file` directive | | TLS Fingerprint| [on\|off] | As defined by the `tls_fingerprint` object in the JSON policy file from the `app_protect_dos_policy_file` directive |
### Graph Representation The displayed graph is a stacked composition, encompassing two specific metrics: `Passthrough Requests` and `Mitigations`, with both metrics measured on a per-second basis. - `Passthrough Requests` represents the count of requests successfully forwarded to the backend server. - `Mitigations` indicates the quantity of requests that have been mitigated.
The sum of these two metrics at any given moment provides the aggregate count of incoming requests. ### Status Color Indicators - **Green**: Indicates a normal state, with no attack and optimal server health. - **Yellow**: Indicates the presence of either an active attack or diminished server health. - **Red**: Indicates a severe scenario, where the object is experiencing an attack combined with poor health. ### Configure Dashboard Options You can configure the dashboard by clicking the Gear button in the Tabs menu.
**Update every N sec** - updates the Dashboard data after the specified number of seconds, default is 1 second.

## Use the REST API Statistics of your server infrastructure can be managed with the REST API interface. The API is based on standard HTTP requests: statistics can be obtained with `GET` requests. The requests are sent in the JSON format that allows you to connect the stats to monitoring tools or dashboards that support JSON. The status information of any element can be accessed with a slash-separated URL. The URL may look as follows: `http://192.168.1.23/api/dos/1/protected_objects` where: - `/api` represents the configured endpoint in your NGINX setup for API access. - `/dos` differentiates the App Protect DoS API from other NGINX Plus APIs. - `/1` indicates the API version you are using, with the current version being 1. - `/protected_objects` points to the specific resource or data you want to access. Responses from the API are in JSON format for easy parsing and analysis. To view all primary endpoints, use a GET request via 'curl' in the terminal: ```shell curl -s 'http://192.168.1.23/api/dos/1/' | json_pp ``` This command returns a JSON array listing the main access points, like: ```json ["protected_objects"] ``` For obtaining detailed statistics about a particular endpoint, execute a similar GET request: ```shell curl -s 'http://192.168.1.23/api/dos/1/protected_objects' | json_pp ``` ### APIs overview #### Endpoints | Path | Description | |:------ |:-----------| | / | Return list of root endpoints | | /protected_objects/ | Return statistics of all Protected Objects | | /protected_objects/{protectedObjectName} | Return statistics of a Protected Object | An example response from the `/protected_objects` endpoint could be: ```json { "po_1": { "attack": false, "health": 0.50, "rps": 12, "mps": 0, "requests": 500123, "passthrough": 260023, "mitigations": 240100, "learning": "ready", "protocol": "http1", "mode": "standard", "sig": true, "ba": true, "auto_tools": true, "tls_fp": true }, "po_http2": { "attack": false, "health": 0.50, "rps": 20, "mps": 0, "requests": 500123, "passthrough": 260023, "mitigations": 240100, "learning": "not_ready", "protocol": "http2", "mode": "conservative", "sig": true, "ba": true, "auto_tools": true, "tls_fp": true }, "po_grpc": { "attack": false, "health": 0.50, "rps": 25, "mps": 0, "requests": 6000123, "passthrough": 2599123, "mitigations": 3401000, "learning": "ready", "protocol": "grpc", "mode": "standard", "sig": true, "ba": true, "auto_tools": true, "tls_fp": true } } ``` -------------------------------------------------------------------------------- title: "Security Log" url: https://docs.nginx.com/nginx-app-protect-dos/monitoring/security-log/index.md description: Learn about the F5 DoS for NGINX Security Log. -------------------------------------------------------------------------------- ## Overview Security logs contain information about the status of the protected objects. It gives a general picture about each protected object in terms of traffic intensity, health of the backend server, learning and mitigations. There are several types of logs, each contains different information and published either periodically or upon an important event. ### Dictionary The following table lists all the possible fields in the logs and their meaning. | Field | Type | Meaning | |----------------|--------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `date_time` | string | the date and time of the event | | `product` | string | always set to `app-protect-dos` | | `product_version` | string | F5 DoS for NGINX version | | `unit_hostname` | string | host name of the app-protect-dos instance | | `instance_id` | string | instance ID: container id from `/proc/self/cgroup`or hostname if container is is not available | | `vs_name` | string | A unique identifier (representing the protected object's name) of the location in the `nginx.conf` file that this request is associated with. It contains the line number of the containing server block in `nginx.conf`, the server name, a numeric discriminator that distinguishes between multiple entries within the same server, and the location name.
For example: `34-mydomain.com:0-~/.*php(2)`. | | `dos_attack_id`| integer | unique attack IP per unit_hostname | | `attack_event` | string | Event name as it appears in remote logger. | | `stress_level` | float | a number from 0 to ... that reflects stress level. | | `learning_confidence` | string | the possible values are **not ready/bad actors only/ready** | | `baseline_dps` | integer | learned datagrams per second (DPS) | | `incoming_dps` | integer | current datagrams per second (DPS) | | `incoming_rps` | integer | current RPS (requests per second) | | `successful_tps` | integer | successful TPS (successful requests per second - Any RC but 5xx) | | `allowlist_rps` | integer | allowlist requests per second | | `unsuccessful_rps` | integer | unsuccessful requests per second (passed to server and not responded: `reset / timeout / 5xx` | | `incoming_datagrams` | integer | incremental number of incoming datagrams | | `incoming_requests` | integer | incremental number of incoming requests | | `allowlist_requests` | integer | incremental number of allowlist requests | | `successful_responses` | integer | incremental number of successful responses | | `unsuccessful_requests` | integer | incremental number of unsuccessful requests (passed to server and not responded: `reset / timeout / 5xx` | | `active_connections`| integer | current number of active server connections | | `threshold_dps` | float | global rate DPS threshold | | `threshold_conns` | float | active connections threshold | | `mitigated_bad_actors`
`redirect_bad_actor`
`challenge_bad_actor`
`block_bad_actor` | integer | incremental number of mitigated bad actors. Increments upon any type of bad actors mitigations.
incremental number of http redirections sent to detected bad actors
incremental number of JS challenges sent to detected bad actors
incremental number of blocked bad actors | | `mitigated_by_signatures`
`redirect_signature`
`challenge_signature`
`block_signature` | integer | incremental number of requests mitigated by signatures. Increments upon any type of signatures mitigations.
incremental number of http redirections sent to clients when requests match a signature.
incremental number of JS challenges sent to clients when requests match a signature.
incremental number of blocked requests when requests match a signature. | | `mitigated_by_global_rate`
`redirect_global`
`challenge_global`
`block_global` | integer | incremental number of requests mitigated by global_rate. Increments upon any type of global rate mitigations.
incremental number of http redirections sent to clients upon global rate mitigation.
incremental number of JS challenges sent to clients upon global rate mitigation.
incremental number of blocked requests upon global rate mitigation. | | `mitigated_slow`
`redirect_slow`
`challenge_slow`
`block_slow`| integer | incremental number of mitigated slow requests. Increments upon any type of slow requests mitigations.
incremental number of http redirections sent to clients upon slow request mitigation.
incremental number of JS challenges sent to clients upon slow request mitigation.
incremental number of blocked slow requests. | | `mitigated_connections` | integer | incremental number of mitigated by connections mitigation | | `mitigated_bad_actors_l4` | integer | incremental number of mitigated by L4 accelerated mitigation | | `mitigated_bad_actors_rps`
`redirect_bad_actor_rps`
`challenge_bad_actor_rps`
`block_bad_actor_rps` | integer | mitigated_bad_actors rps. Includes any type of bad actors mitigations.
http redirections per second sent to detected bad actors.
JS challenges per second sent to detected bad actors.
blocked bad actors per second. | | `mitigated_by_signatures_rps`
`redirect_signature_rps`
`challenge_signature_rps`
`block_signature_rps` | integer | mitigated_signatures rps. Includes any type of signatures mitigations.
http redirections sent per second to clients when requests match a signature.
JS challenges per second sent to clients when requests match a signature.
blocked requests per second when requests match a signature. | | `mitigated_slow_rps`
`redirect_slow_rps`
`challenge_slow_rps`
`block_slow_rps` | integer | mitigated slow requests per second. Includes any type of slow requests mitigations.
http redirections per second sent to clients upon slow request mitigation.
JS challenges per second sent to clients upon slow request mitigation.
blocked slow requests per second. | | `mitigated_by_global_rate_rps`
`redirect_global_rps`
`challenge_global_rps`
`block_global_rps` | integer | mitigated_global_rate rps. Includes any type of global rate mitigations.
http redirections per second sent to clients upon global rate mitigation.
JS challenges per second sent to clients upon global rate mitigation.
blocked requests per second upon global rate mitigation. | | `mitigated_bad_actors_l4_rps` | integer | blocked requests per second when mitigated by L4 accelerated mitigation | | `mitigated_connections_rps` | integer | mitigated_connections rps | | `source_ip`
`tls_fp`
`impact_rps` | string
string
integer | ip address of the detected bad actor `1.1.1.1`
TLS Fingerprint of the bad actor
RPS created by bad actor in the time of the detection **(to be calculated as a max hitcount in AMT / 10)** | | `new_bad_actors_detected`
`bad_actors` | integer | the number of newly detected bad actors
the number of bad actors | | `signature`
`signature_id`
`signature_efficiency`
`signature_accuracy`| string
integer
float
float | signature string `http.request.method eq GET and http.uri_parameters eq 6`
unique signature ID per unit_host
estimated efficiency upon signature detection: percentage of bad traffic covered by the signature
estimated accuracy upon signature detection: percentage of learned good traffic NOT covered by the signature | ## Events ### 1a. Attack notification Reports about the start and end of an attack, as well as major parameters of ongoing attacks. a. Example: **Attack Started** ```shell date_time="Oct 05 2021 08:01:00", product="app-protect-dos", product_version="25+1.78.0-1.el7.ngx", unit_hostname="localhost.localdomain", instance_id="129c76", vs_name="example.com/", dos_attack_id="1", attack_event="Attack started", stress_level="1.00", learning_confidence="Ready", baseline_dps="17", incoming_dps="181", incoming_rps="181", successful_tps="0", allowlist_rps="0", unsuccessful_rps="0", incoming_datagrams="8576", incoming_requests="8576", allowlist_requests="162", successful_responses="5265", unsuccessful_requests="0", active_connections="58", threshold_dps="41.60", threshold_conns="41.60", mitigated_bad_actors="0", mitigated_by_signatures="0", mitigated_by_global_rate="0", mitigated_bad_actors_l4="0", mitigated_slow="0", redirect_global="0", redirect_bad_actor="0", redirect_signature="0", redirect_slow="0", challenge_global="0", challenge_bad_actor="0", challenge_signature="0", challenge_slow="0", block_global="0", block_bad_actor="0", block_signature="0", block_slow="0", mitigated_connections="0", mitigated_bad_actors_rps="0", mitigated_by_signatures_rps="0", mitigated_by_global_rate_rps="0", mitigated_bad_actors_l4_rps="0", mitigated_slow_rps="0", redirect_global_rps="0", redirect_bad_actor_rps="0", redirect_signature_rps="0", redirect_slow_rps="0", challenge_global_rps="0", challenge_bad_actor_rps="0", challenge_signature_rps="0", challenge_slow_rps="0", block_global_rps="0", block_bad_actor_rps="0", block_signature_rps="0", block_slow_rps="0", mitigated_connections_rps="0", ``` b. Example: **Attack Ended** ```shell date_time="Oct 05 2021 08:06:21", product="app-protect-dos", product_version="25+1.78.0-1.el7.ngx", unit_hostname="localhost.localdomain", instance_id="129c76", vs_name="example.com/", dos_attack_id="1", attack_event="Attack ended", stress_level="0.50", learning_confidence="Ready", baseline_dps="12", incoming_dps="0", incoming_rps="0", successful_tps="0", allowlist_rps="0", unsuccessful_rps="0", incoming_datagrams="226566", incoming_requests="226566", allowlist_requests="1632", successful_responses="7760", unsuccessful_requests="0", active_connections="0", threshold_dps="2121.60", threshold_conns="2121.60", mitigated_bad_actors="94488", mitigated_by_signatures="117361", mitigated_by_global_rate="2861", mitigated_bad_actors_l4="62788", mitigated_slow="0", redirect_global="2861", redirect_bad_actor="94488", redirect_signature="117361", redirect_slow="0", challenge_global="0", challenge_bad_actor="0", challenge_signature="0", challenge_slow="0", block_global="0", block_bad_actor="0", block_signature="0", block_slow="0", mitigated_connections="0", mitigated_bad_actors_rps="0", mitigated_by_signatures_rps="0", mitigated_by_global_rate_rps="0", mitigated_bad_actors_l4_rps="0", mitigated_slow_rps="0", redirect_global_rps="0", redirect_bad_actor_rps="0", redirect_signature_rps="0", redirect_slow_rps="0", challenge_global_rps="0", challenge_bad_actor_rps="0", challenge_signature_rps="0", challenge_slow_rps="0", block_global_rps="0", block_bad_actor_rps="0", block_signature_rps="0", block_slow_rps="0", mitigated_connections_rps="0", ``` ### 1b. Traffic/Mitigation summary stats Reported periodically, providing aggregated statistics per protected object.
This corresponds to the metrics reported on the main Grafana screen. a. Example: **No Attack** ```shell date_time="Oct 05 2021 07:54:29", product="app-protect-dos", product_version="25+1.78.0-1.el7.ngx", unit_hostname="localhost.localdomain", instance_id="129c76", vs_name="example.com/", dos_attack_id="0", attack_event="No Attack", stress_level="0.50", learning_confidence="Not ready", baseline_dps="19", incoming_dps="9", incoming_rps="9", successful_tps="10", allowlist_rps="1", unsuccessful_rps="0", incoming_datagrams="678", incoming_requests="678", allowlist_requests="52", successful_responses="678", unsuccessful_requests="0", active_connections="0", threshold_dps="2121.60", threshold_conns="2121.60", mitigated_bad_actors="0", mitigated_by_signatures="0", mitigated_by_global_rate="0", mitigated_bad_actors_l4="0", mitigated_slow="0", redirect_global="0", redirect_bad_actor="0", redirect_signature="0", redirect_slow="0", challenge_global="0", challenge_bad_actor="0", challenge_signature="0", challenge_slow="0", block_global="0", block_bad_actor="0", block_signature="0", block_slow="0", mitigated_connections="0", mitigated_bad_actors_rps="0", mitigated_by_signatures_rps="0", mitigated_by_global_rate_rps="0", mitigated_bad_actors_l4_rps="0", mitigated_slow_rps="0", redirect_global_rps="0", redirect_bad_actor_rps="0", redirect_signature_rps="0", redirect_slow_rps="0", challenge_global_rps="0", challenge_bad_actor_rps="0", challenge_signature_rps="0", challenge_slow_rps="0", block_global_rps="0", block_bad_actor_rps="0", block_signature_rps="0", block_slow_rps="0", mitigated_connections_rps="0", ``` b. Example: **Under Attack** ```shell date_time="Oct 05 2021 08:02:35", product="app-protect-dos", product_version="25+1.78.0-1.el7.ngx", unit_hostname="localhost.localdomain", instance_id="129c76", vs_name="example.com/", dos_attack_id="1", attack_event="Under Attack", stress_level="0.50", learning_confidence="Ready", baseline_dps="12", incoming_dps="893", incoming_rps="893", successful_tps="12", allowlist_rps="1", unsuccessful_rps="0", incoming_datagrams="87823", incoming_requests="87823", allowlist_requests="1523", successful_responses="5736", unsuccessful_requests="0", active_connections="1", threshold_dps="92.40", threshold_conns="92.40", mitigated_bad_actors="0", mitigated_by_signatures="75137", mitigated_by_global_rate="2861", mitigated_bad_actors_l4="62788", mitigated_slow="0", redirect_global="2861", redirect_bad_actor="0", redirect_signature="75137", redirect_slow="0", challenge_global="0", challenge_bad_actor="0", challenge_signature="0", challenge_slow="0", block_global="0", block_bad_actor="0", block_signature="0", block_slow="0", mitigated_connections="0", mitigated_bad_actors_rps="0", mitigated_by_signatures_rps="879", mitigated_by_global_rate_rps="0", mitigated_bad_actors_l4_rps="0", mitigated_slow_rps="0", redirect_global_rps="0", redirect_bad_actor_rps="0", redirect_signature_rps="879", redirect_slow_rps="0", challenge_global_rps="0", challenge_bad_actor_rps="0", challenge_signature_rps="0", challenge_slow_rps="0", block_global_rps="0", block_bad_actor_rps="0", block_signature_rps="0", block_slow_rps="0", mitigated_connections_rps="0", ``` ### 2. Bad actor detection/expiration Reports F5 DoS for NGINX decisions regarding bad actors. a. Example: **Bad Actor Detection** ```shell date_time="Apr 29 2021 14:03:01", product="app-protect-dos", product_version="23+1.54.1-1.el7.ngx", unit_hostname="localhost.localdomain", instance_id="d9a6d8", vs_name="example.com/", dos_attack_id="1", attack_event="Bad actor detection", source_ip="5.5.5.9", impact_rps="30", ``` b. Example: **Bad Actor Expired** ```shell date_time="Apr 29 2021 14:05:29", product="app-protect-dos", product_version="23+1.54.1-1.el7.ngx", unit_hostname="localhost.localdomain", instance_id="d9a6d8", vs_name="example.com/", dos_attack_id="0", attack_event="Bad actor expired", source_ip="5.5.5.10", impact_rps="12", ``` ### 3. Attack signatures Reports F5 DoS for NGINX decisions regarding signatures.
Example: **Attack Signature Detected** ```shell date_time="Apr 29 2021 14:02:56", product="app-protect-dos", product_version="23+1.54.1-1.el7.ngx", unit_hostname="localhost.localdomain", instance_id="d9a6d8", vs_name="example.com/", dos_attack_id="1", attack_event="Attack signature detected", signature="(http.user_agent_header_exists eq true) and (http.accept contains other-than(application|audio|message|text|image|multipart)) and (http.unknown_header_exists eq true) and (http.headers_count neq 10) and (http.x_forwarded_for_header_exists eq false) and (http.uri_parameters eq 1) and (http.uri_len between 48-63) and (http.accept_header_exists eq true) and (http.hdrorder not-hashes-to 55) and (http.connection_header_exists eq true) and (http.accept_encoding_header_exists eq true) and (http.request.method eq reserved) and (http.cookie_header_exists eq true) and (http.uri_file hashes-to 7) and (http.host_header_exists eq true)", signature_id="809655398", signature_efficiency="72.00", signature_accuracy="100.00", ``` ### 4. Bad actors detection information Provides detailed information about bad actors.
Example: **Bad Actors Detected** ```shell date_time="Apr 29 2021 14:02:00", product="app-protect-dos", product_version="23+1.54.1-1.el7.ngx", unit_hostname="localhost.localdomain", instance_id="d9a6d8", vs_name="example.com/", dos_attack_id="1", attack_event="Bad actors detected", new_bad_actors_detected="2", bad_actors="2", ``` ## Security Log Configuration File The file is in JSON format.
### Filter
| Element | Description | Type/Values| Default | |----------|--------------| ---------- | -----------| |traffic-mitigation-stats| This filter element refers to [Traffic/Mitigation summary stats](#1b-trafficmitigation-summary-stats).| **Enumerated values:**
- **all**
- **none**| `all` | |bad-actors| This filter element refers to [Bad actor detection/expiration](#2-bad-actor-detectionexpiration), every 10 seconds.| **Enumerated values:**
- **all**
- **none**
- **top N**| `top 10` | |attack-signatures| This filter element refers to [Attack Signatures](#3-attack-signatures), every 10 seconds.| **Enumerated values:**
- **all**
- **none**
- **top N**| `top 10` | Example: ```json { "filter": { "traffic-mitigation-stats": "all", "bad-actors": "top 100", "attack-signatures": "top 100" } } ``` -------------------------------------------------------------------------------- title: "F5 DoS for NGINX 3.0" url: https://docs.nginx.com/nginx-app-protect-dos/releases/about-3.0/index.md -------------------------------------------------------------------------------- Here you can find the release information for F5 DoS for NGINX v3.0. F5 DoS for NGINX provides behavioral protection against Denial of Service (DoS) for your web applications. ## Release 3.0 September 21, 2022 ### New Features - L4 mitigation (with eBPF) - DoS Live Activity Monitoring with requests mitigation graphs - DoS Live Activity Monitoring support for multi-instances F5 DoS for NGINX setups (multi-VMs, multi-replicas) ### Supported Packages #### App Protect DoS ##### CentOS 7.4+ / RHEL 7.4+ / UBI7 - app-protect-dos-27+3.0.3-1.el7.ngx.el7.ngx.x86_64.rpm ##### RHEL 8 / UBI8 - app-protect-dos-27+3.0.3-1.el8.ngx.el8.ngx.x86_64.rpm ##### Debian 10 - app-protect-dos_27+3.0.3-1~buster_amd64.deb ##### Debian 11 - app-protect-dos_27+3.0.3-1~bullseye_amd64.deb ##### Ubuntu 18.04 - app-protect-dos_27+3.0.3-1~bionic_amd64.deb ##### Ubuntu 20.04 - app-protect-dos_27+3.0.3-1~focal_amd64.deb ##### Alpine 3.15 - app-protect-dos-27.3.0.3-r1.apk #### NGINX Plus - NGINX Plus R27 ### Important Notes - L4 (eBPF) mitigation helps mitigate volumetric attacks by slowing down the opening of TCP connections by the attackers. It is recommended to deploy F5 DoS for NGINX with L4 (eBPF) mitigation at the perimeter network or behind L3 load balancer. Installing F5 DoS for NGINX with L4 (eBPF) mitigation behind L4/L7 load balancer may result in the load balancer's starvation during an attack. - If F5 WAF for NGINX is installed, app protect should be disabled for the location of DoS Live Activity Monitoring API. For example: ```shell location /api { app_protect_enable off; app_protect_dos_api; } ``` - Misconfiguration of `app_protect_dos_monitor` potentially can cause a false attack declaration. Port configuration should correspond to the port the server listens to. For example: ```shell server { listen 8080; location / { app_protect_dos_monitor "myservice.com:8080"; } } ``` - `proxy_request_buffering` off is not supported. - gRPC and HTTP/2 protection require active monitoring of the protected service. The directive `app_protect_dos_monitor` is mandatory for these use cases, otherwise, the attack will not be detected. - [TLS fingerprint](/nap-dos/directives-and-policy/learn-about-directives-and-policy.md#policy-directive-app_protect_dos_policy_file) feature is not used in CentOS 7.4 and RHEL 7 / UBI 7 due to the old OpenSSL version. The required OpenSSL version is 1.1.1 or higher. - Slow POST attack always mitigates with block action while other types of attacks can also be mitigated with redirection or JS challenges. - The recommended option of running NGINX Plus in a Docker Container is with the `daemon off` flag. It's mandatory for UBI 8. -------------------------------------------------------------------------------- title: "Access Log Request Mechanism" url: https://docs.nginx.com/nginx-app-protect-dos/monitoring/access-log/index.md description: Learn about the F5 DoS for NGINX Request Log Mechanism. -------------------------------------------------------------------------------- ## Access Logs Access Log is NGINX’s request log mechanism. It is controlled by the following two directives.
### log_format This directive determines the format of the log messages using predefined variables. App Protect DoS will enrich this set of variables with several security log attributes that are available to be included in the `log_format`. If `log_format` is not specified then the built-in format `combined` is used but, because that format does not include the extended App Protect DoS variables, this directive must be used when the user wants to add App Protect DoS information to the log. ### access_log This directive determines the destination of the `access_log` and the name of the format according to the official [F5 NGINX documentation](https://docs.nginx.com). For example: `access_log /var/log/nginx/access.log log_dos`; (`log_dos` is predefined in the log_format directive). ## App Protect Variables for Access Log These are the variables added to Access Log. They are a subset of the Security log attributes. The Security log names are prefixed with `$app_protect_dos`. |Name| Meaning |Comment| |--- |------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------ | |`$app_protect_dos_outcome`| One of:
**Allow**: request was sent to origin server
**Redirect**: http redirection
**Challenge**: JS challenge
**Block**: blocked request || |`$app_protect_dos_outcome reason`| One of:
**Allow**: Request not mitigated, passed DoS flow successfully.
**Allowlist**: Request not mitigated because it is on the allowlist.
**Bypass**: Request not mitigated due to internal failure.
**Bad_Actor:**: Request mitigated as a bad actor.
**Signature**: Request mitigated as a matched DoS attack signature.
**Global_Rate**: Request mitigated as exceeding the calculated global request rate.
**Slow_Body**: Request mitigated due to being a slow request. |Combine `MITIGATED_BY_GLOBAL_RATE` with global rate value (in RPS) for example `Global_Rate, value=152,` | |`$app_protect_dos_tls_fp`| TLS Fingerprint - a value which identifies the sender |Applicable only in TLS (SSL) traffic| |`$app_protect_dos_policy_name`| The name of the policy that enforced the request || |`$app_protect_dos_vs_name`| The name of the protected object || |`$app_protect_dos_version`| The App Protect DoS version string:
major.minor.build format. |Does not include the F5 NGINX plus version (e.g. R21). The latter is available in `$version` variable.| **Note:** Many of the other Security log attributes that are not included here have exact or similar parallels among the NGINX variables also available for access log. For example, `$request` is parallel to the `request` security log attribute. See the full list of [NGINX variables](https://nginx.org/en/docs/http/ngx_http_log_module.html). ## Logging Rate Limit - mandatory configuration During a DoS attack, there is a large quantity of incoming requests which can flood the Access Log. The rate of the access log's entries can be limited in order to avoid this flood. NGINX logs all the requests during peacetime and logs up to 10 entries per second for each outcome reason during attack time. In worst case it can be 50 requests per second under attack. Two things should be configured in the `nginx conf` file: 1. Create a variable called `loggable` using NGINX's `set` directive and give it any value (string or numerical).
Note that the scope of the `set` directive is **server** or **location** block.
For example: **set $loggable '1'**; 2. Add the string **"if=$loggable"** to the **access_log** directive's argument. For example: access_log /var/log/nginx/access.log custom **if=$loggable**; ## Example ```nginx http { log_format security_dos 'request_time=$request_time client_ip=$remote_addr,' 'request="$request", status=$status,' 'dos_policy=$app_protect_dos_policy_name, dos_protected_object=app_protect_dos_vs_name' 'dos_action=$app_protect_dos_outcome, dos_action_reason=$app_protect_dos_outcome_reason'; server { location / { set $loggable 1; access_log /var/log/nginx/access.log security_dos if=$loggable;; ... } } } ``` -------------------------------------------------------------------------------- title: "F5 DoS for NGINX 2.4" url: https://docs.nginx.com/nginx-app-protect-dos/releases/about-2.4/index.md -------------------------------------------------------------------------------- Here you can find the release information for F5 DoS for NGINX v2.4. F5 DoS for NGINX provides behavioral protection against Denial of Service (DoS) for your web applications. ## Release 2.4 June 28, 2022 ### New Features - [Support for Debian 11](learn-about-deployment.md#debian-10--debian-11-installation) - Accelerated bad actors detection when the signatures mitigation is disabled. ### Supported Packages #### App Protect DoS ##### CentOS 7.4+ / RHEL 7.4+ / UBI7 - app-protect-dos-27+2.4.1-1.el7.ngx.el7.ngx.x86_64.rpm ##### RHEL 8 / UBI8 - app-protect-dos-27+2.4.1-1.el8.ngx.el8.ngx.x86_64.rpm ##### Debian 10 - app-protect-dos_27+2.4.1-1~buster_amd64.deb ##### Debian 11 - app-protect-dos_27+2.4.1-1~bullseye_amd64.deb ##### Ubuntu 18.04 - app-protect-dos_27+2.4.1-1~bionic_amd64.deb ##### Ubuntu 20.04 - app-protect-dos_27+2.4.1-1~focal_amd64.deb ##### Alpine 3.15 - app-protect-dos-27.2.4.1-r1.apk #### NGINX Plus - NGINX Plus R27 ### Resolved Issues - Honoring allow/deny directives for restricting access to the F5 DoS for NGINX Live Activity Monitoring API location. ### Important Notes - If F5 WAF for NGINX is installed, app protect should be disabled for the location of DoS Live Activity Monitoring API For example: ```shell location /api { app_protect_enable off; app_protect_dos_api; } ``` - Misconfiguration of `app_protect_dos_monitor` potentially can cause a false attack declaration. Port configuration should correspond to the port the server listens to. For example: ```shell server { listen 8080; location / { app_protect_dos_monitor "myservice.com:8080"; } } ``` - `proxy_request_buffering` off is not supported. - gRPC and HTTP/2 protection require active monitoring of the protected service. The directive `app_protect_dos_monitor` is mandatory for these use cases, otherwise, the attack will not be detected. - [TLS fingerprint](/nap-dos/directives-and-policy/learn-about-directives-and-policy.md#policy-directive-app_protect_dos_policy_file) feature is not used in CentOS 7.4 and RHEL 7 / UBI 7 due to the old OpenSSL version. The required OpenSSL version is 1.1.1 or higher. - Slow POST attack always mitigates with block action while other types of attacks can also be mitigated with redirection or JS challenges. - The recommended option of running NGINX Plus in a Docker Container is with the `daemon off` flag. It's mandatory for UBI 8. -------------------------------------------------------------------------------- title: "F5 DoS for NGINX 2.3" url: https://docs.nginx.com/nginx-app-protect-dos/releases/about-2.3/index.md -------------------------------------------------------------------------------- Here you can find the release information for F5 DoS for NGINX v2.3. F5 DoS for NGINX provides behavioral protection against Denial of Service (DoS) for your web applications. ## Release 2.3 May 9, 2022 ### New Features - [Support for Alpine 3.15 deployment](learn-about-deployment.md#alpine-315-installation) - [DoS Live Activity Monitoring](/nap-dos/monitoring/live-activity-monitoring.md) - [New Arbitrator FQDN/IP directive](learn-about-directives-and-policy.md#arbitrator-fqdn-directive-app_protect_dos_arb_fqdn) ### Supported Packages #### App Protect DoS ##### CentOS 7.4+ / RHEL 7.4+ / UBI7 - app-protect-dos-26+2.3.46-1.el7.ngx.el7.ngx.x86_64.rpm ##### RHEL 8 / UBI8 - app-protect-dos-26+2.3.46-1.el8.ngx.el8.ngx.x86_64.rpm ##### Debian 10 - app-protect-dos_26+2.3.46-1~buster_amd64.deb ##### Ubuntu 18.04 - app-protect-dos_26+2.3.46-1~bionic_amd64.deb ##### Ubuntu 20.04 - app-protect-dos_26+2.3.46-1~focal_amd64.deb ##### Alpine 3.15 - app-protect-dos-26.2.3.48-r1.apk #### NGINX Plus - NGINX Plus R26 ### Resolved Issues - Improved Security logger configuration parsing: Detects invalid JSON structure, generates an appropriate message if field/value is not valid, doesn't stop on the first failure. ### Important Notes - Misconfiguration of `app_protect_dos_monitor` potentially can cause a false attack declaration. Port configuration should correspond to the port the server listens to. For example: ```shell server { listen 8080; server_name myservice.com; location / { app_protect_dos_monitor "myservice.com:8080/"; } } ``` Please note that the above syntax for the Monitor directive is old but still supported. The new syntax for the Monitor directive is as stated below, where **protocol** and **timeout** arguments are optional and default. Refer to [Learn about Directives and Policy](/nginx-app-protect-dos/directives-and-policy/learn-about-directives-and-policy/#monitor-directive-app_protect_dos_monitor) for more details on Monitor directive new syntax. For example: `app_protect_dos_monitor uri=myservice.com:8080/ protocol=http1 timeout=5;`

- `proxy_request_buffering` off is not supported. - gRPC and HTTP/2 protection require active monitoring of the protected service. The directive `app_protect_dos_monitor` is mandatory for these use cases, otherwise, the attack will not be detected. - [TLS fingerprint](/nap-dos/directives-and-policy/learn-about-directives-and-policy.md#policy-directive-app_protect_dos_policy_file) feature is not used in CentOS 7.4 and RHEL 7/UBI 7 due to the old OpenSSL version. The required OpenSSL version is 1.1.1 or higher. - Slow POST attack always mitigates with block action while other types of attacks can also be mitigated with redirection or JS challenges. - The recommended option of running NGINX Plus in a Docker Container is with the `daemon off` flag. It's mandatory for UBI 8. -------------------------------------------------------------------------------- title: "F5 DoS for NGINX 2.2" url: https://docs.nginx.com/nginx-app-protect-dos/releases/about-2.2/index.md -------------------------------------------------------------------------------- Here you can find the release information for F5 DoS for NGINX v2.2. F5 DoS for NGINX provides behavioral protection against Denial of Service (DoS) for your web applications. ## Release 2.2 February 15, 2022 In this release, support for F5 DoS for NGINX is added to NGINX Plus R26. ### New Features - **Improve Signature Lifecycle** ### Supported Packages #### App Protect DoS ##### CentOS 7.4+ / RHEL 7.4+ / UBI7 - app-protect-dos-26+2.2.20-1.el7.ngx.el7.ngx.x86_64.rpm ##### RHEL 8 / UBI8 - app-protect-dos-26+2.2.20-1.el8.ngx.el8.ngx.x86_64.rpm ##### Debian 10 - app-protect-dos_26+2.2.20-1~buster_amd64.deb ##### Ubuntu 18.04 - app-protect-dos_26+2.2.20-1~bionic_amd64.deb ##### Ubuntu 20.04 - app-protect-dos_26+2.2.20-1~focal_amd64.deb #### NGINX Plus - NGINX Plus R26 ### Resolved Issues - Monitor requests are sent in new connections. It improves monitoring health capability and allows better detection of slow POST attacks. - Adaptive memory allocation by adminstall in order to support maximum cores. The amount of allocated memory for F5 DoS for NGINX is controlled by an argument of adminstall. The default value is 80 MB. For certain deployments, this amount of memory size is not enough. The default memory size is adaptive now. If the Virtual Machine (VM) has a big number of CPU cores, then we can also increase the amount of memory. For the case of up to 4 CPU cores, the allocated memory is 80MB, for more than 4 CPU cores, the allocated memory will be calculated as 80MB + CPUs * 2.5MB ### Important Notes - Misconfiguration of `app_protect_dos_monitor` can cause a false attack declaration. Port configuration should correspond to the port the server listens to. For example: ```shell server { listen 8080; server_name myservice.com; location / { app_protect_dos_monitor "myservice.com:8080/"; } } ``` - `proxy_request_buffering` off is not supported. - gRPC and HTTP/2 protection require active monitoring of the protected service. The directive `app_protect_dos_monitor` is mandatory for these use cases, otherwise, the attack will not be detected. - [TLS fingerprint](/nap-dos/directives-and-policy/learn-about-directives-and-policy.md#policy-directive-app_protect_dos_policy_file) feature is not used in CentOS 7.4 and RHEL 7 / UBI 7 due to the old OpenSSL version. The required OpenSSL version is 1.1.1 or higher. - Slow POST attack always mitigates with block action while other types of attacks can also be mitigated with redirection or JS challenges. - The recommended option of running NGINX Plus in a Docker Container is with the `daemon off` flag. It's mandatory for UBI 8. -------------------------------------------------------------------------------- title: "Operation Log" url: https://docs.nginx.com/nginx-app-protect-dos/monitoring/operation-log/index.md description: Learn about the F5 DoS for NGINX Operation Log. -------------------------------------------------------------------------------- ## Overview The operation logs consists of system operational and health events. The events are sent to the NGINX error log and are distinguished by the `APP_PROTECT_DOS` prefix followed by JSON body. The log level depends on the event: success is usually `notice` while failure is `error`. The timestamp is inherent in the error log. ## Events |Event Type|Level|Meaning| |--------- |-----|------ | |Configuration Error |error |There were errors in the [directives](/nap-dos/directives-and-policy/learn-about-directives-and-policy.md) in the `nginx.conf` file.
Configuration error event is produced when one of F5 DoS for NGINX directives is supplied with an incorrect data. An additional information will be added to the message, describing what was incorrect. NGINX will run with default values for this directive.
Please note that if the directive supplied with an incorrect number of arguments then NGINX will issue an error and NGINX will not run. It is a generic NGINX behavior.| ```json { "event": "configuration_error", "error_message": "unknown argument", "line_number": 58 } ``` |Event Type|Level|Meaning| |--------- |-----|------ | |Configuration Load Failure |error |There was an error in one of the configuration files: file not found, failed to parse.| ```json { "event": "configuration_load_failure", "error_message": "Failed to load Policy '/etc/app_protect_dos/BADOSDefaultPolicy.json' : Fail parse JSON Policy: malformed JSON string, neither tag, array, object, number, string or atom, at character offset 0 (before \"xxxx\\nhdjk\\n\\n555\\n\") \n.\n", "error_line_number": 58 } ``` |Event Type|Level|Meaning| |--------- |-----|------ | |Configuration Load Success | notice |The `APP_PROTECT_DOS` configuration process ended successfully: all policies, log configuration and global settings were loaded to F5 DoS for NGINX and all traffic will be handled by this configuration. The `error_message` contains warnings. This event is also generated on the initial configuration (when NGINX Plus starts).| ```json { "event": "configuration_load_success", "software_version": "x.x.x.x.x", "error_message": "warning if exists..." } ``` |Event Type|Level|Meaning| |--------- |-----|------ | |Shared Memory Failure |error |A worker attempted to connect to shared memory but the operation failed. One time error (per worker) - upon switch from `init` or `operational` mode to `failure`. The `mode` should be `failure`.| ```json { "event": "shared_memory_failure", "worker_pid": 4928, "mode": "failure", "mode_changed": true } ``` |Event Type|Level|Meaning| |--------- |-----|------ | |Shared Memory Connected |notice |A worker successfully connected to shared memory.
One time message (per worker) - upon switch from init or failure mode to `operational`.
The `mode` attribute should be `operational`, unless there is an ongoing problem.| ```json { "event": "shared_memory_connected", "worker_pid": 4928, "mode": "operational", "mode_changed": true } ``` |Event Type|Level|Meaning| |--------- |-----|------ | | eBPF Failure | error | A worker attempted to connect to eBPF maps but the operation failed. One time error (per worker) - upon switch from `init` or `operational` mode to `failure`. The mode should be `failure`.| ```json { "event": "ebpf_failure", "worker_pid": 4928, "mode": "failure", "mode_changed": true } ``` |Event Type|Level|Meaning| |--------- |-----|------ | | eBPF Connected | notice | A worker successfully connected to eBPF maps.
One time message (per worker) - upon switch from `init` or `failure` mode to `operational`.
The `mode` attribute should be `operational`, unless there is an ongoing problem. | ```json { "event": "ebpf_connected", "worker_pid": 4928, "mode": "operational", "mode_changed": true } ``` -------------------------------------------------------------------------------- title: "F5 DoS for NGINX 2.1" url: https://docs.nginx.com/nginx-app-protect-dos/releases/about-2.1/index.md -------------------------------------------------------------------------------- Here you can find the release information for F5 DoS for NGINX v2.1. F5 DoS for NGINX provides behavioral protection against Denial of Service (DoS) for your web applications. ## Release 2.1 December 29, 2021 ### New Features - [Support for RHEL (7.4.x and above) Virtual Machine (VM) deployment](learn-about-deployment.md#rhel-74-installation) - [Support for RHEL 8 and UBI 8](learn-about-deployment.md#rhel-8-installation) - [GRPC and HTTP/2 protection support for Centos (7.4.x and above) and RHEL (7.4.x and above)](/nap-dos/deployment-guide/learn-about-deployment.md) ### Supported Packages #### App Protect DoS ##### CentOS 7.4+ / RHEL 7.4+ / UBI7 - app-protect-dos-25+2.1.8-1.el7.ngx.el7.ngx.x86_64.rpm ##### RHEL 8 / UBI8 - app-protect-dos-25+2.1.8-1.el8.ngx.el8.ngx.x86_64.rpm ##### Debian 10 - app-protect-dos_25+2.1.8-1~buster_amd64.deb ##### Ubuntu 18.04 - app-protect-dos_25+2.1.8-1~bionic_amd64.deb ##### Ubuntu 20.04 - app-protect-dos_25+2.1.8-1~focal_amd64.deb #### NGINX Plus - NGINX Plus R25 ### Resolved Issues - The `app_protect_dos_name` directive is not inherited by the inner blocks, causing to have more VSs than expected. - Signature should not be created if good and bad actor use the same type of traffic. - When there's a clear anomaly on the User-Agent header signal, the signature doesn't include it. - HTTP Method signal is named incorrectly in signatures. ### Important Notes - `proxy_request_buffering` off is not supported. - gRPC and HTTP/2 protection require active monitoring of the protected service. The directive `app_protect_dos_monitor` is mandatory for these use cases, otherwise, the attack will not be detected. - [TLS fingerprint](/nap-dos/directives-and-policy/learn-about-directives-and-policy.md#policy-directive-app_protect_dos_policy_file) feature is not used in CentOS 7.4 and RHEL 7 / UBI 7 due to the old OpenSSL version. The required OpenSSL version is 1.1.1 or higher. - Slow POST attack always mitigates with block action while other types of attacks can also be mitigated with redirection or JS challenges. - The recommended option of running NGINX Plus in a Docker Container is with the `daemon off` flag. It's mandatory for UBI 8. -------------------------------------------------------------------------------- title: "Troubleshooting Guide" url: https://docs.nginx.com/nginx-app-protect-dos/troubleshooting/how-to-troubleshoot/index.md description: Learn about the F5 DoS for NGINX Troubleshooting Guide. -------------------------------------------------------------------------------- ## Overview This Troubleshooting Guide is intended to provide guidance to customers in the detection and correction of programming issues in F5 DoS for NGINX. It may also be useful to IT. ## Resolving Known Problems ### Configuration |Problem|Solution| |-------|--------| | NGINX is not running (ps -aux)

Reloading NGINX fails| Check the error log at `/var/log/nginx/error.log`.
Fix the problem and re-run NGINX.| | No original source IP in logs|1. XFF is not configured (or not configured correctly)
2. External Load Balancer doesn't forward XFF | | F5 DoS for NGINX functionality is not as expected| F5 DoS for NGINX has several logs which can be used for troubleshooting.
Usually, it is best to look for any warning or error messages within the logs.
Refer to [Logs Overview](/nap-dos/monitoring/types-of-logs.md)| | `Too many open files` error message | Increase number of file descriptors.
For example: `worker_rlimit_nofile 65535;` in the main context of `nginx.conf` file.
Refer to [worker_rlimit_nofile directive](https://nginx.org/en/docs/ngx_core_module.html#worker_rlimit_nofile) | | `setrlimit ... failed (Permission denied)` error message | Increase the limit using the following command as the root user:
`setsebool -P httpd_setrlimit 1;`
Refer to [Issue 4: Too many files are open Error](https://www.f5.com/company/blog/nginx/using-nginx-plus-with-selinux/) | | More protected objects than expected | The `app_protect_dos_enable` directive is inherited by all server and location blocks beneath it, each block will be a protected object.
Consider moving this directive from outer to inner block.
Refer to: [F5 DoS for NGINX - Directives and Policy](/nap-dos/directives-and-policy/learn-about-directives-and-policy.md) | | `No DOS protection for ngx_worker at idx X` warning message | There are more nginx processes than allowed.
Either decrease the number of nginx processes (ngx_processes directive in `nginx.conf` file) or increase the number of supported workers for F5 DoS for NGINX using the flag `--max-workers NUM` for `/usr/bin/adminstall`. | | `unknown directive 'app_protect_dos_xxx'` error message | App Protect DOS module is not loaded. Add this line to the main (global) context of nginx.conf:
`load_module "/etc/nginx/modules/ngx_http_app_protect_dos_module.so";` | | NGINX struggles handling a high rate of incoming connections | Linux machine should be tuned for optimal performance.
Refer to [Tuning NGINX for Performance](https://www.nginx.com/blog/tuning-nginx/) | | Error in `adminstall` process, such as `Failed to allocate` | Insufficient memory to allocate all the required resources.
Increase the `--memory` size or decrease the number of nginx workers (`--max_workers`) if not all of them are going to be in use.
Use the `--help` flag for more info. | ### ELK issues ELK issues are addressed directly in GitHub by posting the issue to Kibana dashboards for [F5 DoS for NGINX GitHub repo](https://github.com/f5devcentral/nap-dos-elk-dashboards). ### SELinux Configure SELinux to allow F5 DoS for NGINX. The configuration steps are found in the [SELinux configuration](/nap-dos/deployment-guide/learn-about-deployment.md#selinux-configuration) section of the deployment guide. If SELinux still denies access to something, it means that one of more security exceptions should be enabled. The following steps describe how to find the problematic exception and enable it. 1. Temporarily add the `httpd_t` domain to the permissive list (this will completely enable all the fields).
In this configuration SELinux will not deny anything related to NGINX as NGINX is labeled with the `httpd_t` context.
```shell semanage permissive -a httpd_t ``` 2. Repeat the scenario which made SELinux deny and see that it now works. 3. In permissive mode, security exceptions are logged to the default Linux audit log. Due to the previous step, the permitted exception will be logged.
The log can be found in `/var/log/audit/audit.log`. 4. The following command will parse the audit log and build a SELinux command that will permit all the exceptions found in the log: ```shell grep nginx /var/log/audit/audit.log | audit2allow -m nginx ``` 5. Compare the generated output to the nginx.te file mentioned in the deployment guide. Add all the missing commands to the nginx.te file and repeat the SELinux configuration mentioned in the deployment guide. 6. Delete the `httpd_t` domain from the permissive list: ```shell semanage permissive -d httpd_t ``` For more information about how to use NGINX Plus with SELinux - check our [blog](https://www.f5.com/company/blog/nginx/using-nginx-plus-with-selinux/) ### Send Logs to Support If there are any problems, collect the troubleshooting information in a tarball and send it to your customer support engineer. 1. Get package version: a. Get F5 DoS for NGINX version:
```shell /usr/bin/admd -v > package_versions.txt ``` b. Get packages version:
For CentOS/RHEL:
```shell rpm -qa nginx-plus* app-protect* >> package_versions.txt ``` For Debian/Ubuntu:
```shell apt list --installed | grep -E 'nginx-plus|app-protect' >> package_versions.txt ``` c. Get OS version:
```shell cat /etc/os-release > system_version.txt && uname -r >> system_version.txt && cat /proc/version >> system_version.txt ``` d. Get F5 DoS for NGINX shared memory dump:
```shell admd -c > napd_shmem.txt ``` e. Get Linux shared memory dump:
```shell ipcs -m > linux_shmem.txt ``` 2. Create a list of files for tarball:
a. Create a file using your favorite editor (i.e VI editor)
```shell vi logs.txt ``` b. Insert the following content into the file created above:
```shell package_versions.txt system_version.txt napd_shmem.txt linux_shmem.txt /var/log/adm/* /var/run/adm/* /var/log/nginx/* ``` c. Add the path of your NGINX configuration files including all references, for example:
```shell /etc/nginx/nginx.conf /etc/nginx/conf.d/* ``` d. Add all policies and log file configuration, for example:
```shell /etc/app_protect_dos/* ``` 3. Create the tarball: ```shell tar cvfz logs.tgz `cat logs.txt` ``` 4. Send `logs.tgz` to your customer support. -------------------------------------------------------------------------------- title: "F5 DoS for NGINX 2.0" url: https://docs.nginx.com/nginx-app-protect-dos/releases/about-2.0/index.md -------------------------------------------------------------------------------- Here you can find the release information for F5 DoS for NGINX v2.0. F5 DoS for NGINX provides behavioral protection against Denial of Service (DoS) for your web applications. ## Release 2.0 October 20, 2021 ### New Features #### [Protection of gRPC services against application layer Denial of Service attacks](/nap-dos/deployment-guide/learn-about-deployment.md) - **Message flood** Attacker supplies multiple gRPC requests that exceed the service capacity. - **Concurrent large requests** Attacker supplies a number of concurrent large requests that exceed the server capacity of concurrent requests. - **Slow gRPC POST** Attacker supplies a number of concurrent slow POST gRPC requests that exceed the server capacity of concurrent requests. - **HTTP/2 attack on gRPC service** Attacker runs typical DoS HTTP/2 attacks: HTTP flood and slow attacks on gRPC service. #### [Protection of HTTP/2 services against application layer Denial of Service attacks](/nap-dos/deployment-guide/learn-about-deployment.md) ### Supported Packages #### App Protect DoS ##### CentOS 7.4+ / UBI7 - app-protect-dos-25+2.0.1-1.el7.ngx.el7.ngx.x86_64.rpm ##### Debian 10 - app-protect-dos_25+2.0.1-1~buster_amd64.deb ##### Ubuntu 18.04 - app-protect-dos_25+2.0.1-1~bionic_amd64.deb ##### Ubuntu 20.04 - app-protect-dos_25+2.0.1-1~focal_amd64.deb #### NGINX Plus - NGINX Plus R25 ### Resolved Issues - Security log keeps working on removed Protected Objects. - Monitoring requests show up in the access log. - `app_protect_dos_name` longer than 32 characters creates a garbage name in the logs. - Created protected objects for the not configured contexts. - Wrong reporting of attack status with arbitrator. - Wrong `impact_rps` value in **Bad actor expired** log message. - Rate limit in Access Log should be optional. ### Important Notes - `proxy_request_buffering` off is not supported. - gRPC and HTTP/2 protection require active monitoring of the protected service. The directive `app_protect_dos_monitor` is mandatory for these use cases, otherwise, the attack will not be detected. - gRPC and HTTP/2 protection are available only on Debian 10, Ubuntu 18.04 and Ubuntu 20.04 platforms. For the rest of the platforms, F5 DoS for NGINX does not protect gRPC and HTTP/2 services. The traffic is bypassed. - [TLS fingerprint](/nap-dos/directives-and-policy/learn-about-directives-and-policy.md#policy-directive-app_protect_dos_policy_file) feature is not used in CentOS 7.4 due to the old OpenSSL version. The required OpenSSL version is 1.1.1 or higher. - Slow POST attack always mitigates with block action while other types of attacks can also be mitigated with redirection or JS challenges. - New optional configuration parameters of the directive `app_protect_dos_monitor` to support gRPC and HTTP/2 protocols. - Added new fields in Security Log: - `baseline_dps` (datagrams per second) instead of `baseline_tps`, `incoming_datagrams`
- `successful_responses` instead of `successful_transactions`
- `unsuccessful_requests` instead of `unsuccessful_requests_count`. - In the case of an upgrade from the previous `app-protect-dos` version, it's necessary to remove the old `nginx-plus` and install the new `app-protect-dos` that will install a correspondent version of `nginx-plus` as described in the [F5 DoS for NGINX Deployment Guide](/nap-dos/deployment-guide/learn-about-deployment.md). -------------------------------------------------------------------------------- title: "F5 DoS for NGINX Arbitrator 1.1.0" url: https://docs.nginx.com/nginx-app-protect-dos/releases/about-1.1.0/index.md -------------------------------------------------------------------------------- Here you can find the release information for F5 DoS for NGINX Arbitrator v1.1.0. ## Arbitrator Service Release 1.1.0 December 1, 2021 This release is focused on security and stability. ### New Features - Improve security by enabling the arbitrator to work as a non-root user. - Remove operating system dependencies to work as a native service utilizing golang. ### Resolved Issues - Special characters like a slash inside the protected object name prevented Arbitrator to save the state file. ### Important Notes - The current release upgrades Arbitrator service only. This change is agnostic to F5 DoS for NGINX functionalities. - `proxy_request_buffering` off is not supported. - gRPC and HTTP/2 protection require active monitoring of the protected service. The directive `app_protect_dos_monitor` is mandatory for these use cases, otherwise, the attack will not be detected. - gRPC and HTTP/2 protection are available only on Debian 10, Ubuntu 18.04, and Ubuntu 20.04 platforms. For the rest of the platforms, F5 DoS for NGINX does not protect gRPC and HTTP/2 services. The traffic is bypassed. - [TLS fingerprint](/nap-dos/directives-and-policy/learn-about-directives-and-policy.md#policy-directive-app_protect_dos_policy_file) feature is not used in CentOS 7.4 due to the old OpenSSL version. The required OpenSSL version is 1.1.1 or higher. - Slow POST attack always mitigates with block action while other types of attacks can also be mitigated with redirection or JS challenges. - New optional configuration parameters of the directive `app_protect_dos_monitor` to support gRPC and HTTP/2 protocols. - Added new fields in Security Log: - `baseline_dps` (datagrams per second) instead of `baseline_tps`, `incoming_datagrams`
- `successful_responses` instead of `successful_transactions`
- `unsuccessful_requests` instead of `unsuccessful_requests_count`. - In the case of an upgrade from the previous `app-protect-dos` version, it's necessary to remove the old `nginx-plus` and install the new `app-protect-dos` that will install a correspondent version of `nginx-plus` as described in the [F5 DoS for NGINX Deployment Guide](/nap-dos/deployment-guide/learn-about-deployment.md). -------------------------------------------------------------------------------- title: "NGINX App Protect 1.0" url: https://docs.nginx.com/nginx-app-protect-dos/releases/about-1.0/index.md -------------------------------------------------------------------------------- Here you can find the release information for F5 DoS for NGINX v1. F5 DoS for NGINX provides behavioral protection against Denial of Service (DoS) for your web applications. ## Release 1.0 July 6, 2021 ### New Features #### Protection against application layer Denial of Service attacks - GET and POST flood
- Slowloris, Slowread, Slowpost
- Distributed variations of attacks (see above)
- Challenge Collapsar (CC) attack/random URIs
#### False-positive management mechanisms - HTTP Redirection - Client-side validation - TLS fingerprinting #### Use Cases #### Application Types - Traditional HTML-based web applications - XML-based web services - REST APIs (JSON) #### Deployment Options - Kubernetes Per-pod proxy - Kubernetes Per-service proxy - API Gateway - Traditional edge proxy ### Supported Packages #### App Protect DoS ##### CentOS 7.4+ - app-protect-dos-24+1.69.6-1.el7.ngx.el7.ngx.x86_64.rpm ##### Debian 10 - app-protect-dos_24+1.69.6-1~buster_amd64.deb ##### Ubuntu 18.04 - app-protect-dos_24+1.69.6-1~bionic_amd64.deb ##### Ubuntu 20.04 - app-protect-dos_24+1.69.6-1~focal_amd64.deb #### NGINX Plus - NGINX Plus R24 ### Known Issues - `proxy_request_buffering off` is not supported. - F5 DoS for NGINX does not protect `grpc` and `http2` services. The traffic is bypassed. - [TLS fingerprint](/nap-dos/directives-and-policy/learn-about-directives-and-policy.md#policy-directive-app_protect_dos_policy_file) feature is not used in CentOS 7.4 due to the old OpenSSL version. The required OpenSSL version is 1.1.1 or higher. - Slow POST attack always mitigates with block action while other types of attacks can also be mitigated with redirection or JS challenges. -------------------------------------------------------------------------------- title: "F5 NGINX Gateway Fabric" url: https://docs.nginx.com/nginx-gateway-fabric/index.md -------------------------------------------------------------------------------- ## About NGINX Gateway Fabric provides an implementation of the [Gateway API](https://gateway-api.sigs.k8s.io/) using [NGINX](https://nginx.org/) as the data plane. The goal of the project is to implement the core Gateway APIs needed to configure an HTTP or TCP/UDP load balancer, reverse proxy, or API gateway for Kubernetes applications. ## Featured content ### [Get started](/nginx-gateway-fabric/get-started) Use kind to quickly deploy and test a NGINX Gateway Fabric cluster. ### [Deploy NGINX Gateway Fabric](/nginx-gateway-fabric/install) Choose how to deploy NGINX Gateway Fabric in production. ### [Gateway API compatibility](/nginx-gateway-fabric/overview/gateway-api-compatibility/) View how much of the Gateway API NGINX Gateway Fabric supports. ## Design NGINX Gateway Fabric separates the control plane and data plane into distinct deployments. The control plane interacts with the Kubernetes API, watching for Gateway API resources. When a new Gateway resource is provisioned, it dynamically creates and manages a corresponding NGINX data plane Deployment and Service. Each NGINX data plane pod consists of an NGINX container integrated with [NGINX Agent](https://github.com/nginx/agent). The control plane translates Gateway API resources into NGINX configurations and sends these configurations to the agent to ensure consistent traffic management. This design enables centralized management of multiple Gateways while ensuring that each NGINX instance stays aligned with the cluster’s current configuration. For more information, see the [Gateway architecture](/ngf/overview/gateway-architecture.md) topic. ## More information ### [Changelog](/nginx-gateway-fabric/changelog) Review the changes from the latest releases. ### [Technical specifications](/nginx-gateway-fabric/overview/technical-specifications) Check which versions of NGINX Gateway Fabric match the API. ### [Routing traffic to applications](/nginx-gateway-fabric/traffic-management/basic-routing) Create simple rules for directing network traffic with HTTPRoute resources. ### [Secure traffic using Let's Encrypt](/nginx-gateway-fabric/traffic-security/integrate-cert-manager) Implement HTTPS with Let's Encrypt to secure client-server communication. -------------------------------------------------------------------------------- title: "Add certificates for secure authentication" url: https://docs.nginx.com/nginx-gateway-fabric/install/secure-certificates/index.md -------------------------------------------------------------------------------- By default, NGINX Gateway Fabric installs self-signed certificates to secure the connection between the NGINX Gateway Fabric control plane and the NGINX data plane pods. These certificates are created by a `cert-generator` job when NGINX Gateway Fabric is first installed. However, because these certificates are self-signed and will expire after 3 years, we recommend a solution such as [cert-manager](https://cert-manager.io) to create and manage these certificates in a production environment. This guide will step through how to install and use `cert-manager` to secure this connection. **Note:** These steps should be completed before you install NGINX Gateway Fabric. --- ## Before you begin To complete this guide, you will need the following prerequisites: - Administrator access to a Kubernetes cluster. - [Helm](https://helm.sh) and [kubectl](https://kubernetes.io/docs/tasks/tools/#kubectl) must be installed locally. ## Install Gateway API CRDs ```shell kubectl kustomize "https://github.com/nginx/nginx-gateway-fabric/config/crd/gateway-api/standard?ref=v" | kubectl apply -f - ``` ## Install cert-manager Add the Helm repository: ```shell helm repo add jetstack https://charts.jetstack.io helm repo update ``` Install cert-manager: ```shell helm install \ cert-manager jetstack/cert-manager \ --namespace cert-manager \ --create-namespace \ --set config.apiVersion="controller.config.cert-manager.io/v1alpha1" \ --set config.kind="ControllerConfiguration" \ --set config.enableGatewayAPI=true \ --set crds.enabled=true ``` This also enables Gateway API features for cert-manager, which can be useful for [securing your workload traffic](/ngf/traffic-security/integrate-cert-manager.md). ## Create the CA issuer The first step is to create the CA (certificate authority) issuer. **Note:** This example uses a self-signed Issuer, which should not be used in production environments. For production environments, you should use a real [CA issuer](https://cert-manager.io/docs/configuration/ca/). Create the namespace: ```shell kubectl create namespace nginx-gateway ``` ```yaml kubectl apply -f - <-nginx-gateway-fabric..svc`. The default Helm release name used in our installation docs is `ngf`, and the default namespace is `nginx-gateway`, so the `dnsName` should be `ngf-nginx-gateway-fabric.nginx-gateway.svc`. #### Manifests The full service name is of the format: `..svc`. By default, the base service name is `nginx-gateway`, and the namespace is `nginx-gateway`, so the `dnsName` should be `nginx-gateway.nginx-gateway.svc`. ```yaml kubectl apply -f - <Packages:

gateway.nginx.org/v1alpha1

Package v1alpha1 contains API Schema definitions for the gateway.nginx.org API group.

Resource Types:

AuthenticationFilter

AuthenticationFilter configures request authentication and is referenced by HTTPRoute and GRPCRoute filters using ExtensionRef.

Field Description
apiVersion
string		 gateway.nginx.org/v1alpha1
kind
string 		AuthenticationFilter
metadata
Kubernetes meta/v1.ObjectMeta 		Refer to the Kubernetes API documentation for the fields of the metadata field.
spec
AuthenticationFilterSpec

Spec defines the desired state of the AuthenticationFilter.



basic
BasicAuth 		(Optional)

Basic configures HTTP Basic Authentication.
oidc
OIDCAuth 		(Optional)

OIDC configures OpenID Connect Authentication (NGINX Plus).
jwt
JWTAuth 		(Optional)

JWT configures JSON Web Token authentication (NGINX Plus).
type
AuthType

Type selects the authentication mechanism.
status
AuthenticationFilterStatus

Status defines the state of the AuthenticationFilter.

ClientSettingsPolicy

ClientSettingsPolicy is an Inherited Attached Policy. It provides a way to configure the behavior of the connection between the client and NGINX Gateway Fabric.

Field Description
apiVersion
string		 gateway.nginx.org/v1alpha1
kind
string 		ClientSettingsPolicy
metadata
Kubernetes meta/v1.ObjectMeta 		Refer to the Kubernetes API documentation for the fields of the metadata field.
spec
ClientSettingsPolicySpec

Spec defines the desired state of the ClientSettingsPolicy.



body
ClientBody 		(Optional)

Body defines the client request body settings.
keepAlive
ClientKeepAlive 		(Optional)

KeepAlive defines the keep-alive settings.
targetRef
sigs.k8s.io/gateway-api/apis/v1.LocalPolicyTargetReference

TargetRef identifies an API object to apply the policy to. Object must be in the same namespace as the policy. Support: Gateway, HTTPRoute, GRPCRoute.
status
sigs.k8s.io/gateway-api/apis/v1.PolicyStatus

Status defines the state of the ClientSettingsPolicy.

ExternalLoadBalancer

ExternalLoadBalancer configures an external load balancer that fronts a Gateway. It references a Gateway through TargetRefs. NGINX Gateway Fabric provisions the external load balancer integration for the Gateway’s data plane Service.

ExternalLoadBalancer maps one-to-one to a Gateway: a Gateway yields exactly one data plane Service, so it is fronted by exactly one ExternalLoadBalancer. When more than one ExternalLoadBalancer references the same Gateway, the oldest is accepted and the others are rejected with Accepted=False.

A resource configures exactly one external load balancer backend. The gatewayLink backend integrates F5 BIG-IP through F5 CIS.

Field Description
apiVersion
string		 gateway.nginx.org/v1alpha1
kind
string 		ExternalLoadBalancer
metadata
Kubernetes meta/v1.ObjectMeta 		Refer to the Kubernetes API documentation for the fields of the metadata field.
spec
ExternalLoadBalancerSpec

Spec defines the desired state of the ExternalLoadBalancer.



gatewayLink
GatewayLinkConfig 		(Optional)

GatewayLink configures F5 BIG-IP as the external load balancer using F5 Container Ingress Services. It is the first supported backend. Additional backend types may be added as sibling fields in the future.
targetRefs
[]sigs.k8s.io/gateway-api/apis/v1.LocalPolicyTargetReference

TargetRefs identifies the Gateways this external load balancer applies to. Each object must be in the same namespace as the ExternalLoadBalancer resource. Exactly one Gateway is supported for now. Support: Gateway.
status
ExternalLoadBalancerStatus

Status defines the state of the ExternalLoadBalancer.

NginxGateway

NginxGateway represents the dynamic configuration for an NGINX Gateway Fabric control plane.

Field Description
apiVersion
string		 gateway.nginx.org/v1alpha1
kind
string 		NginxGateway
metadata
Kubernetes meta/v1.ObjectMeta 		Refer to the Kubernetes API documentation for the fields of the metadata field.
spec
NginxGatewaySpec

NginxGatewaySpec defines the desired state of the NginxGateway.



logging
Logging 		(Optional)

Logging defines logging related settings for the control plane.
status
NginxGatewayStatus

NginxGatewayStatus defines the state of the NginxGateway.

PayloadProcessor

PayloadProcessor is an Inherited Attached Policy. It enables declarative processing of HTTP request and response payload content by attaching to a Gateway or HTTPRoute.

note: this CRD is based on the proposed “PayloadProcessor” policy from the Gateway API. Once the upstream API is finalized, migration to the official API will be considered.

Field Description
apiVersion
string		 gateway.nginx.org/v1alpha1
kind
string 		PayloadProcessor
metadata
Kubernetes meta/v1.ObjectMeta 		Refer to the Kubernetes API documentation for the fields of the metadata field.
spec
PayloadProcessorSpec

Spec defines the desired state of the PayloadProcessor.



targetRef
sigs.k8s.io/gateway-api/apis/v1.LocalPolicyTargetReference

TargetRef identifies the Gateway or HTTPRoute this policy applies to. Objects must be in the same namespace as the policy. Follows the standard policy attachment pattern (GEP-713).

Support: Gateway, HTTPRoute
processors
[]PayloadProcessorEntry

Processors is an ordered list of processing steps to be applied to the request and response payloads. It is currently limited to a single processor (MaxItems=1); the list form is reserved for future multi-processor pipelines.
status
sigs.k8s.io/gateway-api/apis/v1.PolicyStatus

Status defines the state of the PayloadProcessor.

ProxySettingsPolicy

ProxySettingsPolicy is an Inherited Attached Policy. It provides a way to configure the behavior of the connection between NGINX Gateway Fabric and the upstream applications (backends).

Field Description
apiVersion
string		 gateway.nginx.org/v1alpha1
kind
string 		ProxySettingsPolicy
metadata
Kubernetes meta/v1.ObjectMeta 		Refer to the Kubernetes API documentation for the fields of the metadata field.
spec
ProxySettingsPolicySpec

Spec defines the desired state of the ProxySettingsPolicy.



buffering
ProxyBuffering 		(Optional)

Buffering configures the buffering of responses from the proxied server.
timeout
ProxyTimeout 		(Optional)

Timeout configures timeouts for the connection to the proxied server.
targetRefs
[]sigs.k8s.io/gateway-api/apis/v1.LocalPolicyTargetReference

TargetRefs identifies the API object(s) to apply the policy to. Objects must be in the same namespace as the policy. Support: Gateway, HTTPRoute, GRPCRoute
status
sigs.k8s.io/gateway-api/apis/v1.PolicyStatus

Status defines the state of the ProxySettingsPolicy.

RateLimitPolicy

RateLimitPolicy is an Inherited Attached Policy. It provides a way to set local rate limiting rules in NGINX.

Field Description
apiVersion
string		 gateway.nginx.org/v1alpha1
kind
string 		RateLimitPolicy
metadata
Kubernetes meta/v1.ObjectMeta 		Refer to the Kubernetes API documentation for the fields of the metadata field.
spec
RateLimitPolicySpec

Spec defines the desired state of the RateLimitPolicy.



rateLimit
RateLimit 		(Optional)

RateLimit defines the Rate Limit settings.
targetRefs
[]sigs.k8s.io/gateway-api/apis/v1.LocalPolicyTargetReference

TargetRefs identifies API object(s) to apply the policy to. Objects must be in the same namespace as the policy.

Support: Gateway, HTTPRoute, GRPCRoute
status
sigs.k8s.io/gateway-api/apis/v1.PolicyStatus

Status defines the state of the RateLimitPolicy.

SnippetsFilter

SnippetsFilter is a filter that allows inserting NGINX configuration into the generated NGINX config for HTTPRoute and GRPCRoute resources.

Field Description
apiVersion
string		 gateway.nginx.org/v1alpha1
kind
string 		SnippetsFilter
metadata
Kubernetes meta/v1.ObjectMeta 		Refer to the Kubernetes API documentation for the fields of the metadata field.
spec
SnippetsFilterSpec

Spec defines the desired state of the SnippetsFilter.



snippets
[]Snippet

Snippets is a list of NGINX configuration snippets. There can only be one snippet per context. Allowed contexts: main, http, http.server, http.server.location.
status
SnippetsFilterStatus

Status defines the state of the SnippetsFilter.

SnippetsPolicy

SnippetsPolicy provides a way to inject NGINX snippets into the configuration on Gateway level.

Field Description
apiVersion
string		 gateway.nginx.org/v1alpha1
kind
string 		SnippetsPolicy
metadata
Kubernetes meta/v1.ObjectMeta 		Refer to the Kubernetes API documentation for the fields of the metadata field.
spec
SnippetsPolicySpec

Spec defines the desired state of the SnippetsPolicy.



targetRefs
[]sigs.k8s.io/gateway-api/apis/v1.LocalPolicyTargetReference

TargetRefs identifies API object(s) to apply the policy to.
snippets
[]Snippet 		(Optional)

Snippets is a list of snippets to be injected into the NGINX configuration.
status
sigs.k8s.io/gateway-api/apis/v1.PolicyStatus

Status defines the current state of the SnippetsPolicy.

UpstreamSettingsPolicy

UpstreamSettingsPolicy is a Direct Attached Policy. It provides a way to configure the behavior of the connection between NGINX and the upstream applications.

Field Description
apiVersion
string		 gateway.nginx.org/v1alpha1
kind
string 		UpstreamSettingsPolicy
metadata
Kubernetes meta/v1.ObjectMeta 		Refer to the Kubernetes API documentation for the fields of the metadata field.
spec
UpstreamSettingsPolicySpec

Spec defines the desired state of the UpstreamSettingsPolicy.



zoneSize
Size 		(Optional)

ZoneSize is the size of the shared memory zone used by the upstream. This memory zone is used to share the upstream configuration between nginx worker processes. The more servers that an upstream has, the larger memory zone is required. Default: OSS: 512k, Plus: 1m. Directive: https://nginx.org/en/docs/http/ngx_http_upstream_module.html#zone
keepAlive
UpstreamKeepAlive 		(Optional)

KeepAlive defines the keep-alive settings.
loadBalancingMethod
LoadBalancingType 		(Optional)

LoadBalancingMethod specifies the load balancing algorithm to be used for the upstream. If not specified, NGINX Gateway Fabric defaults to random two least_conn, which differs from the standard NGINX default round-robin.
hashMethodKey
HashMethodKey 		(Optional)

HashMethodKey defines the key used for hash-based load balancing methods. This field is required when LoadBalancingMethod is set to hash or hash consistent.
useClusterIP
bool 		(Optional)

UseClusterIP configures NGINX to route to the Service ClusterIP and port instead of individual Pod IPs. When enabled, NGINX will target a single upstream server corresponding to the Service’s ClusterIP, which is useful for service mesh compatibility and other Kubernetes controllers/operators that require traffic to traverse the Service VIP. This setting applies only when the target Service has a ClusterIP. For headless Services (ClusterIP: None) and ExternalName Services, normal endpoint resolution is used instead. This setting is also not applied to L4/stream upstreams. Defaults to false.
targetRefs
[]sigs.k8s.io/gateway-api/apis/v1.LocalPolicyTargetReference

TargetRefs identifies API object(s) to apply the policy to. Objects must be in the same namespace as the policy. Support: Service

TargetRefs must be distinct. The name field must be unique for all targetRef entries in the UpstreamSettingsPolicy.
status
sigs.k8s.io/gateway-api/apis/v1.PolicyStatus

Status defines the state of the UpstreamSettingsPolicy.

WAFPolicy

WAFPolicy is an Inherited Attached Policy. It provides a way to configure F5 WAF for NGINX for Gateways and Routes by referencing compiled WAF policy bundles. Bundles can be fetched directly from an HTTP/HTTPS URL (type: HTTP), from an NGINX Instance Manager instance (type: NIM), from an F5 NGINX One Console instance (type: N1C), or from a Policy Lifecycle Manager’s S3-compatible storage (type: PLM).

Field Description
apiVersion
string		 gateway.nginx.org/v1alpha1
kind
string 		WAFPolicy
metadata
Kubernetes meta/v1.ObjectMeta 		Refer to the Kubernetes API documentation for the fields of the metadata field.
spec
WAFPolicySpec

Spec defines the desired state of the WAFPolicy.



targetRefs
[]sigs.k8s.io/gateway-api/apis/v1.LocalPolicyTargetReference

TargetRefs identifies API object(s) to apply the policy to. Objects must be in the same namespace as the policy. All targets must be of the same Kind (all Gateways OR all HTTPRoutes OR all GRPCRoutes). Support: Gateway, HTTPRoute, GRPCRoute.
type
PolicySourceType

Type identifies the source type for the policy bundle. HTTP fetches directly from a URL; NIM uses the NGINX Instance Manager bundles API; N1C uses the F5 NGINX One Console security policies API; PLM references an APPolicy CRD managed by the Policy Lifecycle Manager.
policySource
PolicySource 		(Optional)

PolicySource holds all non-CRD bundle fetch configuration. Used for HTTP, NIM, and N1C policy types. Must not be set when type is PLM.
policyRef
PolicyRef 		(Optional)

PolicyRef holds all CRD-backed policy references. Used for the PLM policy type.
securityLogs
[]WAFSecurityLog 		(Optional)

SecurityLogs defines security logging configurations.
status
sigs.k8s.io/gateway-api/apis/v1.PolicyStatus

Status defines the state of the WAFPolicy.

APLogConfReference

(Appears on: LogRef)

APLogConfReference identifies an APLogConf resource managed by PLM.

Field Description
namespace
string 		(Optional)

Namespace is the namespace of the APLogConf resource. If not set, the namespace of the WAFPolicy is used. Cross-namespace references require a ReferenceGrant.
name
string

Name is the name of the APLogConf resource.

APPolicyReference

(Appears on: PolicyRef)

APPolicyReference identifies an APPolicy resource managed by PLM.

Field Description
namespace
string 		(Optional)

Namespace is the namespace of the APPolicy resource. If not set, the namespace of the WAFPolicy is used. Cross-namespace references require a ReferenceGrant.
name
string

Name is the name of the APPolicy resource.

AuthType (string alias)

(Appears on: AuthenticationFilterSpec)

AuthType defines the authentication mechanism.

Value Description

"Basic"

AuthTypeBasic is the HTTP Basic Authentication mechanism.

"JWT"

AuthTypeJWT is the JWT Authentication mechanism.

"OIDC"

AuthTypeOIDC is the OpenID Connect Authentication mechanism.

AuthenticationFilterConditionReason (string alias)

AuthenticationFilterConditionReason is a reason for an AuthenticationFilter condition type.

Value Description

"Accepted"

AuthenticationFilterConditionReasonAccepted is used with the Accepted condition type when the condition is true.

"Invalid"

AuthenticationFilterConditionReasonInvalid is used with the Accepted condition type when the filter is invalid.

AuthenticationFilterConditionType (string alias)

AuthenticationFilterConditionType is a type of condition associated with AuthenticationFilter.

Value Description

"Accepted"

AuthenticationFilterConditionTypeAccepted indicates that the AuthenticationFilter is accepted.

Possible reasons for this condition to be True: * Accepted

Possible reasons for this condition to be False: * Invalid.

AuthenticationFilterSpec

(Appears on: AuthenticationFilter)

AuthenticationFilterSpec defines the desired configuration.

Field Description
basic
BasicAuth 		(Optional)

Basic configures HTTP Basic Authentication.
oidc
OIDCAuth 		(Optional)

OIDC configures OpenID Connect Authentication (NGINX Plus).
jwt
JWTAuth 		(Optional)

JWT configures JSON Web Token authentication (NGINX Plus).
type
AuthType

Type selects the authentication mechanism.

AuthenticationFilterStatus

(Appears on: AuthenticationFilter)

AuthenticationFilterStatus defines the state of AuthenticationFilter.

Field Description
controllers
[]ControllerStatus

Controllers is a list of Gateway API controllers that processed the AuthenticationFilter and the status of the AuthenticationFilter with respect to each controller.

Authorization

(Appears on: JWTAuth, OIDCAuth)

Authorization specifies a set of required claim rules that a token’s claim must match to be authorized, given the require type defined.

Field Description
require
RequireType 		(Optional)

Require sets top level authorization requirement. When set to All, the requirements for all claims in a rule must be met. When set to Any, the requirements for any one claim in a rule must be met.
rules
[]Rule

Rules defines a list of claims and their specific authorization requirements.

BasicAuth

(Appears on: AuthenticationFilterSpec)

BasicAuth configures HTTP Basic Authentication.

Field Description
secretRef
LocalObjectReference

SecretRef references a Secret containing credentials in the same namespace.
realm
string

Realm used by NGINX auth_basic directive. https://nginx.org/en/docs/http/ngx_http_auth_basic_module.html#auth_basic Also configures “realm=”” in WWW-Authenticate header in error page location.

BundleAuth

(Appears on: LogSource, PolicySource)

BundleAuth configures authentication for bundle fetching.

Field Description
secretRef
LocalObjectReference

SecretRef references a Kubernetes Secret in the same namespace as the WAFPolicy. The Secret may contain: - “username” and “password” fields for HTTP Basic Authentication - “token” field for Bearer Token Authentication (NIM) or APIToken Authentication (N1C)

BundlePolling

(Appears on: LogSource, PolicySource)

BundlePolling configures automatic re-fetching of a bundle.

Field Description
interval
Kubernetes meta/v1.Duration 		(Optional)

Interval is the period between poll cycles. Defaults to 5m when polling is enabled but no interval is set.
enabled
bool 		(Optional)

Enabled activates periodic re-fetching of the bundle. When true, NGF fetches the bundle on each interval and deploys it only if its checksum differs from the last successfully fetched version.

BundleValidation

(Appears on: LogSource, PolicySource)

BundleValidation configures integrity verification for a bundle. Exactly one of verifyChecksum or expectedChecksum may be set.

Field Description
expectedChecksum
string 		(Optional)

ExpectedChecksum is the expected SHA256 checksum of the bundle. If set, the downloaded bundle must match this checksum or it will be rejected. For N1C sources, the checksum reported by the N1C API is verified automatically; set this field only if you want to enforce an additional, independently known value.
verifyChecksum
bool 		(Optional)

VerifyChecksum enables automatic checksum verification by fetching a companion checksum file at .sha256 and comparing it against the downloaded bundle. Only supported when the policy source type is HTTP (policySource.httpSource or logSource.url); setting this for NIM or N1C sources is rejected at admission. Note: for N1C sources, bundle integrity is always verified automatically using the checksum returned by the N1C compile API — this field is not needed. Mutually exclusive with expectedChecksum.

Claim

(Appears on: Rule)

Claim describes the exact name/value pair of claims that must be matched.

Field Description
proxySetHeader
string

ProxySetHeader sets both the name and variable for proxy_set_header Example: For claim name sub for JWT auth

proxy_set_header X-JWT-Claim-Sub $jwt_claim_sub;
name
string

Name is the name of the claim within the token.
match
ClaimMatchType

Match sets the match type for the claim.
values
[]string

Values are the values within the claim. When more than one value is set, the claim must match any of these values.

ClaimMatchType (string alias)

(Appears on: Claim)

ClaimMatchType defines how claim values are parsed.

Value Description

"Exact"

ClaimMatchTypeExact treats claim values as their exact value.

"Regex"

ClaimMatchTypeRegex treats claim values as a regex value.

ClientBody

(Appears on: ClientSettingsPolicySpec)

ClientBody contains the settings for the client request body.

Field Description
maxSize
Size 		(Optional)

MaxSize sets the maximum allowed size of the client request body. If the size in a request exceeds the configured value, the 413 (Request Entity Too Large) error is returned to the client. Setting size to 0 disables checking of client request body size. Default: https://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size.
bufferSize
Size 		(Optional)

BufferSize sets the buffer size for reading the client request body. If the request body is larger than the buffer, the whole body or only its part is written to a temporary file. Default: https://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_buffer_size.
timeout
Duration 		(Optional)

Timeout defines a timeout for reading client request body. The timeout is set only for a period between two successive read operations, not for the transmission of the whole request body. If a client does not transmit anything within this time, the request is terminated with the 408 (Request Time-out) error. Default: https://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_timeout.

ClientKeepAlive

(Appears on: ClientSettingsPolicySpec)

ClientKeepAlive defines the keep-alive settings for clients.

Field Description
requests
int32 		(Optional)

Requests sets the maximum number of requests that can be served through one keep-alive connection. After the maximum number of requests are made, the connection is closed. Closing connections periodically is necessary to free per-connection memory allocations. Therefore, using too high maximum number of requests is not recommended as it can lead to excessive memory usage. Default: https://nginx.org/en/docs/http/ngx_http_core_module.html#keepalive_requests.
time
Duration 		(Optional)

Time defines the maximum time during which requests can be processed through one keep-alive connection. After this time is reached, the connection is closed following the subsequent request processing. Default: https://nginx.org/en/docs/http/ngx_http_core_module.html#keepalive_time.
timeout
ClientKeepAliveTimeout 		(Optional)

Timeout defines the keep-alive timeouts for clients.
minTimeout
Duration 		(Optional)

MinTimeout defines the timeout for which the keep-alive client connection will not be closed on the server side for connection reuse or on graceful shutdown of worker processes. Default: https://nginx.org/en/docs/http/ngx_http_core_module.html#keepalive_min_timeout.

ClientKeepAliveTimeout

(Appears on: ClientKeepAlive)

ClientKeepAliveTimeout defines the timeouts related to keep-alive client connections. Default: https://nginx.org/en/docs/http/ngx_http_core_module.html#keepalive_timeout.

Field Description
server
Duration 		(Optional)

Server sets the timeout during which a keep-alive client connection will stay open on the server side. Setting this value to 0 disables keep-alive client connections.
header
Duration 		(Optional)

Header sets the timeout in the “Keep-Alive: timeout=time” response header field.

ClientSettingsPolicySpec

(Appears on: ClientSettingsPolicy)

ClientSettingsPolicySpec defines the desired state of ClientSettingsPolicy.

Field Description
body
ClientBody 		(Optional)

Body defines the client request body settings.
keepAlive
ClientKeepAlive 		(Optional)

KeepAlive defines the keep-alive settings.
targetRef
sigs.k8s.io/gateway-api/apis/v1.LocalPolicyTargetReference

TargetRef identifies an API object to apply the policy to. Object must be in the same namespace as the policy. Support: Gateway, HTTPRoute, GRPCRoute.

ControllerLogLevel (string alias)

(Appears on: Logging)

ControllerLogLevel type defines the logging level for the control plane.

Value Description

"debug"

ControllerLogLevelDebug is the debug level for control plane logging.

"error"

ControllerLogLevelError is the error level for control plane logging.

"info"

ControllerLogLevelInfo is the info level for control plane logging.

ControllerStatus

(Appears on: AuthenticationFilterStatus, ExternalLoadBalancerStatus, SnippetsFilterStatus)

Field Description
controllerName
sigs.k8s.io/gateway-api/apis/v1.GatewayController

ControllerName is a domain/path string that indicates the name of the controller that wrote this status. This corresponds with the controllerName field on GatewayClass.

Example: “example.net/gateway-controller”.

The format of this field is DOMAIN “/” PATH, where DOMAIN and PATH are valid Kubernetes names (https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names).

Controllers MUST populate this field when writing status. Controllers should ensure that entries to status populated with their ControllerName are cleaned up when they are no longer necessary.
conditions
[]Kubernetes meta/v1.Condition 		(Optional)

Conditions describe the status of the resource with respect to this controller.

DefaultLogProfile (string alias)

(Appears on: LogSource)

DefaultLogProfile identifies a built-in WAF log profile bundle.

Value Description

"log_all"

DefaultLogProfileAll logs all events.

"log_blocked"

DefaultLogProfileBlocked logs blocked events.

"log_default"

DefaultLogProfileDefault logs illegal events (equivalent to log_illegal).

"log_grpc_all"

DefaultLogProfileGRPCAll logs all gRPC events.

"log_grpc_blocked"

DefaultLogProfileGRPCBlocked logs blocked gRPC events.

"log_grpc_illegal"

DefaultLogProfileGRPCIllegal logs illegal gRPC events.

"log_illegal"

DefaultLogProfileIllegal logs illegal events.

Duration (string alias)

(Appears on: ClientBody, ClientKeepAlive, ClientKeepAliveTimeout, JWTAuth, OIDCSessionConfig, ProxyTimeout, UpstreamKeepAlive, DNSResolver, TelemetryExporter)

Duration is a string value representing a duration in time. Duration can be specified in milliseconds (ms), seconds (s), minutes (m), hours (h). A value without a suffix is seconds. Examples: 120s, 50ms, 5m, 1h.

ExtProcessConfig

(Appears on: PayloadProcessorEntry)

ExtProcessConfig defines the configuration for an ExtProcess processor that delegates to an external service.

Field Description
authTokenRef
LocalObjectReference

AuthTokenRef is a reference to a Secret containing an authentication token for the external service.
backendRef
sigs.k8s.io/gateway-api/apis/v1.BackendObjectReference

BackendRef is a reference to the external service that will process the payloads. The referenced backend must be a core Service and must specify a port.

ExternalLoadBalancerConditionReason (string alias)

ExternalLoadBalancerConditionReason is a reason for an ExternalLoadBalancer condition type.

Value Description

"Accepted"

ExternalLoadBalancerConditionReasonAccepted is used with the Accepted condition type when the condition is true.

"Conflicted"

ExternalLoadBalancerConditionReasonConflicted is used with the Accepted condition type when another ExternalLoadBalancer already references the same Gateway. A Gateway can be fronted by exactly one external load balancer, so the oldest is accepted and the others are Conflicted.

"Invalid"

ExternalLoadBalancerConditionReasonInvalid is used with the Accepted condition type when the ExternalLoadBalancer is invalid.

ExternalLoadBalancerConditionType (string alias)

ExternalLoadBalancerConditionType is a type of condition associated with ExternalLoadBalancer.

Value Description

"Accepted"

ExternalLoadBalancerConditionTypeAccepted indicates that the ExternalLoadBalancer is accepted.

Possible reasons for this condition to be True:

  • Accepted

Possible reasons for this condition to be False:

  • Invalid
  • Conflicted.

ExternalLoadBalancerSpec

(Appears on: ExternalLoadBalancer)

ExternalLoadBalancerSpec defines the desired state of ExternalLoadBalancer.

Field Description
gatewayLink
GatewayLinkConfig 		(Optional)

GatewayLink configures F5 BIG-IP as the external load balancer using F5 Container Ingress Services. It is the first supported backend. Additional backend types may be added as sibling fields in the future.
targetRefs
[]sigs.k8s.io/gateway-api/apis/v1.LocalPolicyTargetReference

TargetRefs identifies the Gateways this external load balancer applies to. Each object must be in the same namespace as the ExternalLoadBalancer resource. Exactly one Gateway is supported for now. Support: Gateway.

ExternalLoadBalancerStatus

(Appears on: ExternalLoadBalancer)

ExternalLoadBalancerStatus defines the state of ExternalLoadBalancer.

Field Description
controllers
[]ControllerStatus

Controllers is a list of Gateway API controllers that processed the ExternalLoadBalancer and the status of the ExternalLoadBalancer with respect to each controller.

GatewayLinkConfig

(Appears on: ExternalLoadBalancerSpec)

GatewayLinkConfig defines the configuration for integrating with F5 BIG-IP as the external load balancer for NGINX Gateway Fabric using F5 Container Ingress Services. IngressLink API Definition: https://github.com/F5Networks/k8s-bigip-ctlr/blob/master/docs/config_examples/customResourceDefinitions/customresourcedefinitions.yml

Field Description
virtualServerAddress
string 		(Optional)

VirtualServerAddress is the static IP address to configure on BIG-IP for the virtual server. This is mutually exclusive with IPAMLabel.
virtualServerName
string 		(Optional)

VirtualServerName is a custom name for the BIG-IP virtual server.
ipamLabel
string 		(Optional)

IPAMLabel is the label used by F5 IPAM Controller to allocate an IP address. The IPAM controller will assign an IP from the pool associated with this label. This is mutually exclusive with VirtualServerAddress.
host
string 		(Optional)

Host is the hostname for the BIG-IP virtual server.
partition
string 		(Optional)

Partition is the BIG-IP partition where resources will be created. The partition must already exist on BIG-IP and cannot be “Common”.
bigipRouteDomain
int32 		(Optional)

BigIPRouteDomain is the route domain ID for the BIG-IP virtual server.
tls
GatewayLinkTLS 		(Optional)

TLS defines the TLS configuration for the BIG-IP virtual server.
multiCluster
GatewayLinkMultiCluster 		(Optional)

MultiCluster defines the multi-cluster configuration for load balancing traffic across NGINX instances in multiple clusters.
serviceAddress
GatewayLinkServiceAddress 		(Optional)

ServiceAddress configures Layer 3 settings for the BIG-IP virtual server address.
additionalIngressLinkSpec
k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1.JSON 		(Optional)

AdditionalIngressLinkSpec is an escape hatch for IngressLink fields that are not yet modeled by GatewayLink. Its contents are merged verbatim into the generated IngressLink spec and are NOT validated by NGINX Gateway Fabric. Fields set here take lower precedence than the explicitly modeled GatewayLink fields above; NGINX Gateway Fabric always sets the IngressLink selector internally and it cannot be overridden through this field. Use with caution since contents bypass schema validation, defaulting, and CEL rules, and flow through to BIG-IP via F5 CIS.
iRules
[]string 		(Optional)

IRules is a list of BIG-IP iRules to apply to the virtual server. Each iRule must be specified using the full path format /partition/irule_name, for example “/Common/Proxy_Protocol_iRule”.
monitors
[]GatewayLinkMonitor 		(Optional)

Monitors is a list of BIG-IP health monitors to associate with the virtual server pool.

GatewayLinkMonitor

(Appears on: GatewayLinkConfig)

GatewayLinkMonitor defines a BIG-IP health monitor reference.

Field Description
name
string

Name is the full path of the health monitor on BIG-IP (e.g., “/Common/http”).
reference
string

Reference specifies the source of the monitor. Currently only “bigip” is supported.

GatewayLinkMultiCluster

(Appears on: GatewayLinkConfig)

GatewayLinkMultiCluster defines the multi-cluster configuration for GatewayLink. When configured, CIS load balances traffic across NGINX instances in multiple clusters. This is set only on the cluster that runs CIS. The other clusters run NGINX with a matching Gateway and Service but not CIS, so they do not set multiCluster. CIS reaches those clusters over a kubeconfig.

Field Description
localClusterName
string

LocalClusterName is the name of this cluster as configured in the CIS deployment via the –local-cluster-name flag. NGINX Gateway Fabric uses it as the cluster name for the local entry in the IngressLink’s multiClusterServices, which points at this cluster’s own Gateway Service. It must match the name CIS knows this cluster by, otherwise CIS cannot resolve the local service.
remoteClusters
[]GatewayLinkRemoteCluster

RemoteClusters is the list of remote clusters that also run NGINX Gateway Fabric.

GatewayLinkRemoteCluster

(Appears on: GatewayLinkMultiCluster)

GatewayLinkRemoteCluster defines a remote cluster for multi-cluster load balancing.

Field Description
clusterName
string

ClusterName is one of the names of the remote clusters as configured in the CIS deployment.
namespace
string 		(Optional)

Namespace is the namespace of the NGINX service in the remote cluster. If not specified, defaults to the local Gateway’s namespace.
service
string 		(Optional)

Service is the name of the NGINX service in the remote cluster. If not specified, defaults to the local Gateway’s service name.
weight
int32 		(Optional)

Weight is the load balancing weight for this cluster’s service.

GatewayLinkServiceAddress

(Appears on: GatewayLinkConfig)

GatewayLinkServiceAddress configures Layer 3 settings for the BIG-IP virtual server address.

Field Description
icmpEcho
ICMPEcho 		(Optional)

ICMPEcho controls whether the virtual server address responds to ICMP echo (ping).
trafficGroup
string 		(Optional)

TrafficGroup is the BIG-IP traffic group that owns the virtual server address, in the full path format, for example “/Common/traffic-group-test”.

GatewayLinkTLS

(Appears on: GatewayLinkConfig)

GatewayLinkTLS defines the TLS configuration for the BIG-IP virtual server.

Field Description
reference
TLSReferenceType 		(Optional)

Reference specifies the source of the SSL profiles. “bigip” means the profiles already exist on BIG-IP. “secret” means they come from Kubernetes secrets of type kubernetes.io/tls. If not specified, defaults to “bigip”.
clientSSLs
[]string 		(Optional)

ClientSSLs is a list of client SSL profiles that BIG-IP uses to terminate TLS from the client. When reference is “bigip”, each entry is the full path of a profile on BIG-IP in the form /partition/profile_name, for example /Common/clientssl. When reference is “secret”, each entry is the name of a Kubernetes secret of type kubernetes.io/tls that holds the certificate and key.
serverSSLs
[]string 		(Optional)

ServerSSLs is a list of server SSL profiles that BIG-IP uses to re-encrypt traffic to NGINX. When reference is “bigip”, each entry is the full path of a profile on BIG-IP in the form /partition/profile_name, for example /Common/serverssl. When reference is “secret”, each entry is the name of a Kubernetes secret of type kubernetes.io/tls that holds the certificate and key.

HTTPBundleSource

(Appears on: LogSource, PolicySource)

HTTPBundleSource configures direct bundle fetching from an HTTP/HTTPS URL.

Field Description
url
string

URL is the full URL of the compiled policy bundle (.tgz), e.g. https://storage.example.com/bundles/policy.tgz.

HashMethodKey (string alias)

(Appears on: UpstreamSettingsPolicySpec)

HashMethodKey defines the key used for hash-based load balancing methods. The key must be a valid NGINX variable name starting with ‘$’ followed by lowercase letters and underscores only. For a full list of NGINX variables, refer to: https://nginx.org/en/docs/http/ngx_http_upstream_module.html#variables

ICMPEcho (string alias)

(Appears on: GatewayLinkServiceAddress)

ICMPEcho controls whether the BIG-IP virtual server address responds to ICMP echo.

Value Description

"disable"

ICMPEchoDisable means the virtual server address never responds to ICMP echo.

"enable"

ICMPEchoEnable means the virtual server address always responds to ICMP echo.

"selective"

ICMPEchoSelective means BIG-IP responds to ICMP echo based on the state of the virtual server.

JWTAuth

(Appears on: AuthenticationFilterSpec)

JWTAuth configures JWT-based authentication (NGINX Plus).

Field Description
file
JWTFileKeySource 		(Optional)

File specifies local JWKS configuration. Required when Source == File.
keyCache
Duration 		(Optional)

KeyCache is the cache duration for keys. Configures auth_jwt_key_cache directive. https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_cache Example: “auth_jwt_key_cache 10m;”.
remote
JWTRemoteKeySource 		(Optional)

Remote specifies remote JWKS configuration. Required when Source == Remote.
authorization
Authorization 		(Optional)

Authorization defines the authorization (authz) specification. Enables configuration of token claim validation.
leeway
Duration 		(Optional)

Leeway is the acceptable clock skew for exp & nbf claims. If exp & nbf claims are not defined, this directive takes no effect. Configures auth_jwt_leeway directive. https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_leeway Example: “auth_jwt_leeway 60s”. Default: 0s.
realm
string

Realm used by NGINX auth_jwt directive https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt Configures “realm=”” in WWW-Authenticate header in error page location.
source
JWTKeySource

Source selects how JWT keys are provided: local file or remote JWKS.

JWTFileKeySource

(Appears on: JWTAuth)

JWTFileKeySource specifies local JWKS key configuration.

Field Description
secretRef
LocalObjectReference

SecretRef references a Secret containing the JWKS.

JWTKeySource (string alias)

(Appears on: JWTAuth)

JWTKeySource specifies the source of the keys used to verify JWT signatures.

Value Description

"File"

JWTKeySourceFile configures JWT to fetch JWKS from a local secret.

"Remote"

JWTKeySourceRemote configures JWT to fetch JWKS from a remote source.

JWTRemoteKeySource

(Appears on: JWTAuth)

JWTRemoteKeySource specifies remote JWKS configuration.

Field Description
uri
string

URI is the JWKS endpoint.
caCertificateRefs
[]LocalObjectReference 		(Optional)

CACertificateRefs references a list of secrets containing trusted CA certificates in PEM format used to verify the server certificate of the JWKS endpoint. The referenced secrets must contain an entry with the key “ca.crt”. Only one secret can be referenced currently. If not specified, the system CA bundle is used.

Directive: https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_trusted_certificate

LoadBalancingType (string alias)

(Appears on: UpstreamSettingsPolicySpec)

LoadBalancingType defines the supported load balancing methods.

Value Description

"hash"

LoadBalancingTypeHash enables generic hash-based load balancing, routing requests to upstream servers based on a hash of a specified key HashMethodKey field must be set when this method is selected. Example configuration: hash $binary_remote_addr;.

"hash consistent"

LoadBalancingTypeHashConsistent enables consistent hash-based load balancing, which minimizes the number of keys remapped when a server is added or removed. HashMethodKey field must be set when this method is selected. Example configuration: hash $binary_remote_addr consistent;.

"ip_hash"

LoadBalancingTypeIPHash enables IP hash-based load balancing, ensuring requests from the same client IP are routed to the same upstream server.

"least_conn"

LoadBalancingTypeLeastConnection enables least-connections load balancing, routing requests to the upstream server with the fewest active connections.

"least_time header"

LoadBalancingTypeLeastTimeHeader enables least-time load balancing, routing requests to the upstream server with the least time to receive the response header.

"least_time header inflight"

LoadBalancingTypeLeastTimeHeaderInflight enables least-time load balancing, routing requests to the upstream server with the least time to receive the response header, considering the incomplete requests.

"least_time last_byte"

LoadBalancingTypeLeastTimeLastByte enables least-time load balancing, routing requests to the upstream server with the least time to receive the full response.

"least_time last_byte inflight"

LoadBalancingTypeLeastTimeLastByteInflight enables least-time load balancing, routing requests to the upstream server with the least time to receive the full response, considering the incomplete requests.

"random"

LoadBalancingTypeRandom enables random load balancing, routing requests to upstream servers in a random manner.

"random two"

LoadBalancingTypeRandomTwo enables a variation of random load balancing that randomly selects two servers and forwards traffic to one of them. The default method is least_conn which passes a request to a server with the least number of active connections.

"random two least_conn"

LoadBalancingTypeRandomTwoLeastConnection enables a variation of least-connections balancing that randomly selects two servers and forwards traffic to the one with fewer active connections.

"random two least_time=header"

LoadBalancingTypeRandomTwoLeastTimeHeader enables a variation of least-time load balancing that randomly selects two servers and forwards traffic to the one with the least time to receive the response header.

"random two least_time=last_byte"

LoadBalancingTypeRandomTwoLeastTimeLastByte enables a variation of least-time load balancing that randomly selects two servers and forwards traffic to the one with the least time to receive the full response.

"round_robin"

LoadBalancingTypeRoundRobin enables round-robin load balancing, distributing requests evenly across all upstream servers.

LocalObjectReference

(Appears on: BasicAuth, BundleAuth, ExtProcessConfig, JWTFileKeySource, JWTRemoteKeySource, LogSource, OIDCAuth, PolicySource)

LocalObjectReference specifies a local Kubernetes object.

Field Description
name
string

Name is the name of the referenced object.

LocalRateLimit

(Appears on: RateLimit)

LocalRateLimit contains the local rate limit rules.

Field Description
rules
[]RateLimitRule 		(Optional)

Rules contains the list of rate limit rules.

LogRef

(Appears on: WAFSecurityLog)

LogRef holds all CRD-backed log profile references.

Field Description
apLogConfRef
APLogConfReference 		(Optional)

APLogConfRef references an APLogConf CRD compiled by PLM. Cross-namespace references require a ReferenceGrant.

LogSource

(Appears on: WAFSecurityLog)

LogSource holds all non-CRD configuration for fetching a WAF log profile bundle. Exactly one of DefaultProfile, HTTPSource, NIMSource, or N1CSource must be set.

Field Description
defaultProfile
DefaultLogProfile 		(Optional)

DefaultProfile selects one of the built-in WAF log profile bundles shipped with the WAF engine. Mutually exclusive with HTTPSource, NIMSource, and N1CSource.
httpSource
HTTPBundleSource 		(Optional)

HTTPSource configures direct bundle fetching from an HTTP/HTTPS URL. Mutually exclusive with DefaultProfile, NIMSource and N1CSource.
nimSource
NIMLogProfileBundleSource 		(Optional)

NIMSource configures bundle fetching from NGINX Instance Manager. Mutually exclusive with DefaultProfile, HTTPSource and N1CSource.
n1cSource
N1CLogProfileBundleSource 		(Optional)

N1CSource configures bundle fetching from F5 NGINX One Console. Mutually exclusive with DefaultProfile, HTTPSource, and NIMSource.
auth
BundleAuth 		(Optional)

Auth configures authentication credentials for fetching the log bundle. Only applicable when url is set.
tlsSecret
LocalObjectReference 		(Optional)

TLSSecretRef references a Secret containing a custom CA certificate (key: “ca.crt”). Only applicable when url is set.
validation
BundleValidation 		(Optional)

Validation configures integrity verification for the downloaded log bundle. Only applicable when url is set.
polling
BundlePolling 		(Optional)

Polling configures automatic periodic re-fetching of the log bundle. Only applicable when url is set.
timeout
Kubernetes meta/v1.Duration 		(Optional)

Timeout is the maximum duration for a single log bundle fetch attempt. Defaults to 30s when not set. Only applicable when url is set.
retryAttempts
int32

RetryAttempts is the maximum number of additional fetch attempts on transient failures (network errors, HTTP 5xx). Set to 0 to disable retries. Defaults to 3. Non-transient errors (HTTP 4xx, checksum mismatch) are never retried. Only applicable when url is set.
insecureSkipVerify
bool 		(Optional)

InsecureSkipVerify disables TLS certificate verification when fetching the bundle. Not recommended for production use.

Logging

(Appears on: NginxGatewaySpec)

Logging defines logging related settings for the control plane.

Field Description
level
ControllerLogLevel 		(Optional)

Level defines the logging level.

N1CBundleSource

(Appears on: PolicySource)

N1CBundleSource configures bundle fetching from F5 NGINX One Console (N1C). Exactly one of policyName or policyObjectID must be set.

Field Description
policyName
string 		(Optional)

PolicyName is the name of the security policy in N1C. Mutually exclusive with policyObjectID.
policyObjectID
string 		(Optional)

PolicyObjectID is the unique object identifier of the security policy in N1C (e.g. “pol_-IUuEUN7ST63oRC7AlQPLw”). Mutually exclusive with policyName.
policyVersionID
string 		(Optional)

PolicyVersionID pins a specific version of the policy bundle using its opaque version ID (e.g. “pv_UJ2gL5fOQ3Gnb3OVuVo1XA”). When omitted, the latest available version is used.
url
string

URL is the base URL of the F5 NGINX One Console instance, e.g. https://.volterra.us.
namespace
string

Namespace is the NGINX One Console namespace that owns the security policy.

N1CLogProfileBundleSource

(Appears on: LogSource)

N1CLogProfileBundleSource configures log profile bundle fetching from F5 NGINX One Console (N1C). Exactly one of profileName or profileObjectID must be set.

Field Description
profileName
string

ProfileName is the name of the log profile in N1C that corresponds to the log profile bundle.
profileObjectID
string

ProfileObjectID is the unique object identifier of the log profile in N1C (e.g. “lp_8s8uZxLpThWwEGF7LTn_rA”) that corresponds to the log profile bundle.
url
string

URL is the base URL of the F5 NGINX One Console instance, e.g. https://.volterra.us.
namespace
string

Namespace is the NGINX One Console namespace that owns the log profile.

NIMBundleSource

(Appears on: PolicySource)

NIMBundleSource configures bundle fetching from NGINX Instance Manager (NIM). Exactly one of policyName or policyUID must be set.

Field Description
policyName
string 		(Optional)

PolicyName is the name of the compiled policy bundle in NIM. Mutually exclusive with policyUID.
policyUID
string 		(Optional)

PolicyUID is the unique identifier of the compiled policy bundle in NIM. Mutually exclusive with policyName. Must be a valid UUID (e.g. “2bc1e3ac-7990-4ca4-910a-8634c444c804”).
url
string

URL is the base URL of the NGINX Instance Manager instance, e.g. https://nim.example.com.

NIMLogProfileBundleSource

(Appears on: LogSource)

NIMLogProfileBundleSource configures log profile bundle fetching from NGINX Instance Manager (NIM).

Field Description
profileName
string

ProfileName is the name of the compiled log profile bundle in NIM.
url
string

URL is the base URL of the NGINX Instance Manager instance, e.g. https://nim.example.com.

NginxContext (string alias)

(Appears on: Snippet)

NginxContext represents the NGINX configuration context.

Value Description

"http"

NginxContextHTTP is the http context of the NGINX configuration. https://nginx.org/en/docs/http/ngx_http_core_module.html#http

"http.server"

NginxContextHTTPServer is the server context of the NGINX configuration. https://nginx.org/en/docs/http/ngx_http_core_module.html#server

"http.server.location"

NginxContextHTTPServerLocation is the location context of the NGINX configuration. https://nginx.org/en/docs/http/ngx_http_core_module.html#location

"main"

NginxContextMain is the main context of the NGINX configuration.

NginxGatewayConditionReason (string alias)

NginxGatewayConditionReason defines the set of reasons that explain why a particular NginxGateway condition type has been raised.

Value Description

"Invalid"

NginxGatewayReasonInvalid is a reason that is used with the “Valid” condition when the condition is False.

"Valid"

NginxGatewayReasonValid is a reason that is used with the “Valid” condition when the condition is True.

NginxGatewayConditionType (string alias)

NginxGatewayConditionType is a type of condition associated with an NginxGateway. This type should be used with the NginxGatewayStatus.Conditions field.

Value Description

"Valid"

NginxGatewayConditionValid is a condition that is true when the NginxGateway configuration is syntactically and semantically valid.

NginxGatewaySpec

(Appears on: NginxGateway)

NginxGatewaySpec defines the desired state of the NginxGateway.

Field Description
logging
Logging 		(Optional)

Logging defines logging related settings for the control plane.

NginxGatewayStatus

(Appears on: NginxGateway)

NginxGatewayStatus defines the state of the NginxGateway.

Field Description
conditions
[]Kubernetes meta/v1.Condition 		(Optional)

OIDCAuth

(Appears on: AuthenticationFilterSpec)

OIDCAuth configures OpenID Connect Authentication. Only available for NGINX Plus users.

Field Description
extraAuthArgs
map[string]string 		(Optional)

ExtraAuthArgs sets additional query arguments for the authentication request URL. Arguments are appended with “&”. For example: “prompt=consent&audience=api”. Directive: https://nginx.org/en/docs/http/ngx_http_oidc_module.html#extra_auth_args
crlSecretRef
LocalObjectReference 		(Optional)

CRLSecretRef references a Secret containing a certificate revocation list in PEM format. The referenced Secret must contain an entry with the key “ca.crl”. This is used to verify that certificates presented by the OpenID Provider endpoints have not been revoked.
configURL
string 		(Optional)

ConfigURL sets a custom URL to retrieve the OpenID Provider metadata. Directive: https://nginx.org/en/docs/http/ngx_http_oidc_module.html#config_url NGINX Default: /.well-known/openid-configuration
pkce
bool 		(Optional)

PKCE enables Proof Key for Code Exchange (PKCE) for the authentication flow. If nil, NGINX automatically enables PKCE when the OpenID Provider requires it. Directive: https://nginx.org/en/docs/http/ngx_http_oidc_module.html#pkce
session
OIDCSessionConfig 		(Optional)

Session configures session management for OIDC authentication.
logout
OIDCLogoutConfig 		(Optional)

Logout defines the logout behavior for OIDC authentication.
redirectURI
string 		(Optional)

RedirectURI sets a custom redirect URI for the OIDC callback. If a path-only URI is specified, a callback location block is created to handle the redirect from the OIDC provider. If a full URI is specified, it points to an external callback handler; no location block is created. If not specified, defaults to /oidccallback_. Directive: https://nginx.org/en/docs/http/ngx_http_oidc_module.html#redirect_uri NGINX Default: /oidc_callback Example: /oidc_callback, https://cafe.example.com:8442/oidc_callback
authorization
Authorization 		(Optional)

Authorization defines the authorization (authz) specification. Enables configuration of token claim validation.
clientSecretRef
LocalObjectReference

ClientSecretRef references a Kubernetes secret which contains the OIDC client secret to be used in the Authentication Request: https://openid.net/specs/openid-connect-core-1_0.html#AuthRequest. The referenced Secret must contain an entry with the key “client-secret”. Directive: https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_secret
issuer
string

Issuer is the URL of the OpenID Provider. Must exactly match the “issuer” value from the provider’s .well-known/openid-configuration endpoint. Directive: https://nginx.org/en/docs/http/ngx_http_oidc_module.html#issuer Examples: - Keycloak: https://keycloak.example.com/realms/my-realm - Okta: https://dev-123456.okta.com/oauth2/default - Auth0: https://my-tenant.auth0.com/
clientID
string

ClientID is the client identifier registered with the OpenID Provider. Directive: https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_id
caCertificateRefs
[]LocalObjectReference 		(Optional)

CACertificateRefs references a list of secrets containing trusted CA certificates in PEM format used to verify the certificates of the OpenID Provider endpoints. The referenced secrets must contain an entry with the key “ca.crt”. Only one secret can be referenced currently. If not specified, the system CA bundle is used.

Directive: https://nginx.org/en/docs/http/ngx_http_oidc_module.html#ssl_trusted_certificate NGINX Default: system CA bundle

OIDCLogoutConfig

(Appears on: OIDCAuth)

OIDCLogoutConfig defines the logout behavior for OIDC authentication.

Field Description
uri
string 		(Optional)

URI defines the path for initiating session logout. Directive: https://nginx.org/en/docs/http/ngx_http_oidc_module.html#logout_uri Example: /logout
postLogoutURI
string 		(Optional)

PostLogoutURI defines the URI to redirect to after logout. Must match the configuration on the provider’s side. Directive: https://nginx.org/en/docs/http/ngx_http_oidc_module.html#post_logout_uri Example: /after_logout, https://example.com/after_logout
frontChannelLogoutURI
string 		(Optional)

FrontChannelLogoutURI defines the path for front-channel logout. The OpenID Provider should be configured to set “iss” and “sid” arguments. Directive: https://nginx.org/en/docs/http/ngx_http_oidc_module.html#frontchannel_logout_uri Example: /frontchannel_logout
tokenHint
bool 		(Optional)

TokenHint adds the id_token_hint argument to the provider’s Logout Endpoint. Some OpenID Providers require this. Directive: https://nginx.org/en/docs/http/ngx_http_oidc_module.html#logout_token_hint NGINX Default: false

OIDCSessionConfig

(Appears on: OIDCAuth)

OIDCSessionConfig configures session management for OIDC authentication.

Field Description
cookieName
string 		(Optional)

CookieName sets the name of the session cookie. Directive: https://nginx.org/en/docs/http/ngx_http_oidc_module.html#cookie_name NGINX Default: NGX_OIDC_SESSION
timeout
Duration 		(Optional)

Timeout sets the session timeout duration. Directive: https://nginx.org/en/docs/http/ngx_http_oidc_module.html#session_timeout NGINX Default: 8h

PayloadProcessorEntry

(Appears on: PayloadProcessorSpec)

PayloadProcessorEntry defines a single processing step in the pipeline.

Field Description
extProcess
ExtProcessConfig 		(Optional)

ExtProcess defines the configuration for an ExtProcess processor that delegates to an external service.
type
ProcessorType

Type specifies how the processor executes. ExtProcess calls an external service.

PayloadProcessorSpec

(Appears on: PayloadProcessor)

PayloadProcessorSpec defines the desired state of a PayloadProcessor.

Field Description
targetRef
sigs.k8s.io/gateway-api/apis/v1.LocalPolicyTargetReference

TargetRef identifies the Gateway or HTTPRoute this policy applies to. Objects must be in the same namespace as the policy. Follows the standard policy attachment pattern (GEP-713).

Support: Gateway, HTTPRoute
processors
[]PayloadProcessorEntry

Processors is an ordered list of processing steps to be applied to the request and response payloads. It is currently limited to a single processor (MaxItems=1); the list form is reserved for future multi-processor pipelines.

PolicyRef

(Appears on: WAFPolicySpec)

PolicyRef holds all CRD-backed policy references.

Field Description
apPolicyRef
APPolicyReference 		(Optional)

APPolicyRef references an APPolicy CRD compiled by PLM. Cross-namespace references require a ReferenceGrant.

PolicySource

(Appears on: WAFPolicySpec)

PolicySource holds all non-CRD configuration for fetching a WAF policy bundle.

Field Description
httpSource
HTTPBundleSource 		(Optional)

HTTPSource configures direct bundle fetching from an HTTP/HTTPS URL. Required when type is HTTP; must not be set for other types.
nimSource
NIMBundleSource 		(Optional)

NIMSource configures bundle fetching from NGINX Instance Manager. Required when type is NIM; must not be set for other types.
n1cSource
N1CBundleSource 		(Optional)

N1CSource configures bundle fetching from F5 NGINX One Console. Required when type is N1C; must not be set for other types.
auth
BundleAuth 		(Optional)

Auth configures authentication credentials for fetching the bundle.
tlsSecret
LocalObjectReference 		(Optional)

TLSSecretRef references a Secret containing a custom CA certificate (key: “ca.crt”) for verifying the bundle server’s TLS certificate.
validation
BundleValidation 		(Optional)

Validation configures integrity verification for the downloaded bundle.
polling
BundlePolling 		(Optional)

Polling configures automatic periodic re-fetching of the bundle.
timeout
Kubernetes meta/v1.Duration 		(Optional)

Timeout is the maximum duration for a single bundle fetch attempt. Defaults to 30s when not set.
retryAttempts
int32

RetryAttempts is the maximum number of additional fetch attempts on transient failures (network errors, HTTP 5xx). Set to 0 to disable retries. Defaults to 3. Non-transient errors (HTTP 4xx, checksum mismatch) are never retried.
insecureSkipVerify
bool 		(Optional)

InsecureSkipVerify disables TLS certificate verification when fetching the bundle. Not recommended for production use.

PolicySourceType (string alias)

(Appears on: WAFPolicySpec)

PolicySourceType identifies the source type for a WAF bundle.

Value Description

"HTTP"

PolicySourceTypeHTTP fetches a compiled .tgz bundle directly from an HTTP/HTTPS URL.

"N1C"

PolicySourceTypeN1C fetches a compiled bundle from the F5 NGINX One Console security policies API. Requires managedSource.n1cNamespace in addition to managedSource.policyName. Authentication uses the APIToken scheme: the “token” key from the referenced Secret is sent as “Authorization: APIToken ”.

"NIM"

PolicySourceTypeNIM fetches a compiled bundle from the NGINX Instance Manager security policies API.

"PLM"

PolicySourceTypePLM references an APPolicy CRD managed by the Policy Lifecycle Manager (PLM). Bundles are fetched from PLM’s S3-compatible storage (SeaweedFS). Cluster-wide S3 connection parameters are configured via CLI flags (–plm-storage-*).

ProcessorType (string alias)

(Appears on: PayloadProcessorEntry)

ProcessorType specifies how the processor executes. ExtProcess calls an external service.

Value Description

"ExtProcess"

ProcessorTypeExtProcess delegates processing to an external service.

ProxyBuffering

(Appears on: ProxySettingsPolicySpec)

ProxyBuffering contains the settings for proxy buffering.

Field Description
disable
bool 		(Optional)

Disable enables or disables buffering of responses from the proxied server. If Disable is true, buffering is disabled. If Disable is false, or if Disable is not set, buffering is enabled. Directive: https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffering
bufferSize
Size 		(Optional)

BufferSize sets the size of the buffer used for reading the first part of the response received from the proxied server. This part usually contains a small response header. Directive: https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffer_size
buffers
ProxyBuffers 		(Optional)

Buffers sets the number and size of buffers used for reading a response from the proxied server, for a single connection. Directive: https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffers
busyBuffersSize
Size 		(Optional)

BusyBuffersSize sets the total size of buffers that can be busy sending a response to the client, while the response is not yet fully read. Directive: https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_busy_buffers_size

ProxyBuffers

(Appears on: ProxyBuffering)

ProxyBuffers defines the number and size of the proxy buffers.

Field Description
size
Size

Size sets the size of each buffer.
number
int32

Number sets the number of buffers.

ProxySettingsPolicySpec

(Appears on: ProxySettingsPolicy)

ProxySettingsPolicySpec defines the desired state of the ProxySettingsPolicy.

Field Description
buffering
ProxyBuffering 		(Optional)

Buffering configures the buffering of responses from the proxied server.
timeout
ProxyTimeout 		(Optional)

Timeout configures timeouts for the connection to the proxied server.
targetRefs
[]sigs.k8s.io/gateway-api/apis/v1.LocalPolicyTargetReference

TargetRefs identifies the API object(s) to apply the policy to. Objects must be in the same namespace as the policy. Support: Gateway, HTTPRoute, GRPCRoute

ProxyTimeout

(Appears on: ProxySettingsPolicySpec)

ProxyTimeout defines timeout settings for the connection to the proxied server.

Field Description
connect
Duration 		(Optional)

Connect sets the timeout for establishing a connection with the proxied server. Directive: https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_connect_timeout
read
Duration 		(Optional)

Read sets the timeout for reading a response from the proxied server. Directive: https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_read_timeout
send
Duration 		(Optional)

Send sets the timeout for transmitting a request to the proxied server. Directive: https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_send_timeout

Rate (string alias)

(Appears on: RateLimitRule)

Rate is a string value representing a rate. Rate can be specified in r/s or r/m.

RateLimit

(Appears on: RateLimitPolicySpec)

RateLimit contains settings for Rate Limiting.

Field Description
local
LocalRateLimit 		(Optional)

Local defines the local rate limit rules for this policy.
dryRun
bool 		(Optional)

DryRun enables the dry run mode. In this mode, the rate limit is not actually applied, but the number of excessive requests is accounted as usual in the shared memory zone.

Directive: https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_dry_run
logLevel
RateLimitLogLevel 		(Optional)

LogLevel sets the desired logging level for cases when the server refuses to process requests due to rate exceeding, or delays request processing. Allowed values are info, notice, warn or error.

Directive: https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_log_level
rejectCode
int32 		(Optional)

RejectCode sets the status code to return in response to rejected requests. Must fall into the range 400-599.

Directive: https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_status

RateLimitLogLevel (string alias)

(Appears on: RateLimit)

RateLimitLogLevel defines the log level for cases when the server refuses to process requests due to rate exceeding, or delays request processing.

Value Description

"error"

RateLimitLogLevelError is the error level rate limit logs.

"info"

RateLimitLogLevelInfo is the info level rate limit logs.

"notice"

RateLimitLogLevelNotice is the notice level rate limit logs.

"warn"

RateLimitLogLevelWarn is the warn level rate limit logs.

RateLimitPolicySpec

(Appears on: RateLimitPolicy)

RateLimitPolicySpec defines the desired state of the RateLimitPolicy.

Field Description
rateLimit
RateLimit 		(Optional)

RateLimit defines the Rate Limit settings.
targetRefs
[]sigs.k8s.io/gateway-api/apis/v1.LocalPolicyTargetReference

TargetRefs identifies API object(s) to apply the policy to. Objects must be in the same namespace as the policy.

Support: Gateway, HTTPRoute, GRPCRoute

RateLimitRule

(Appears on: LocalRateLimit)

RateLimitRule contains settings for a RateLimit Rule.

Field Description
zoneSize
Size 		(Optional)

ZoneSize is the size of the shared memory zone.

Directive: https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_zone
delay
int32 		(Optional)

Delay specifies a limit at which excessive requests become delayed. Default value is zero, which means all excessive requests are delayed.

Directive: https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req
noDelay
bool 		(Optional)

NoDelay disables the delaying of excessive requests while requests are being limited. NoDelay cannot be true when Delay is also set.

Directive: https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req
burst
int32 		(Optional)

Burst sets the maximum burst size of requests. If the requests rate exceeds the rate configured for a zone, their processing is delayed such that requests are processed at a defined rate. Excessive requests are delayed until their number exceeds the maximum burst size in which case the request is terminated with an error.

Directive: https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req
rate
Rate

Rate represents the rate of requests permitted. The rate is specified in requests per second (r/s) or requests per minute (r/m).

Directive: https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_zone
key
string

Key represents the key to which the rate limit is applied. The key can contain text, variables, and their combination.

Directive: https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_zone

RequireType (string alias)

(Appears on: Authorization, Rule)

RequireType defines how JWT Claims are validated.

Value Description

"All"

RequireTypeAll authorizes claims that satisfy all requirements.

"Any"

RequireTypeAny authorizes claims that satisfy any requirement.

Rule

(Appears on: Authorization)

Rule defines a list of claims, and authorization rules for those claims.

Field Description
require
RequireType 		(Optional)

Require sets the authorization mode for a specific claim within a rule. When set to All, a token’s claim must match all values within that claim. When set to Any, a token’s claim must match at least one value with that claim.
claims
[]Claim

Claims defines a list of claims required by users.

SecurityLogDestination

(Appears on: WAFSecurityLog)

SecurityLogDestination defines the destination for security logs.

Field Description
file
SecurityLogFile 		(Optional)

File defines the file destination configuration. Only valid when type is “file”.
syslog
SecurityLogSyslog 		(Optional)

Syslog defines the syslog destination configuration. Only valid when type is “syslog”.
type
SecurityLogDestinationType

Type identifies the type of security log destination.

SecurityLogDestinationType (string alias)

(Appears on: SecurityLogDestination)

SecurityLogDestinationType defines the supported security log destination types.

Value Description

"file"

SecurityLogDestinationTypeFile writes logs to a specified file path.

"stderr"

SecurityLogDestinationTypeStderr outputs logs to container stderr.

"syslog"

SecurityLogDestinationTypeSyslog sends logs to a syslog server via TCP.

SecurityLogFile

(Appears on: SecurityLogDestination)

SecurityLogFile defines the file destination configuration for security logs.

Field Description
path
string

Path is the file path where security logs will be written. Must be accessible to the waf-enforcer container.

SecurityLogSyslog

(Appears on: SecurityLogDestination)

SecurityLogSyslog defines the syslog destination configuration for security logs.

Field Description
server
string

Server is the syslog server address in the format “host:port”.

Size (string alias)

(Appears on: ClientBody, ProxyBuffering, ProxyBuffers, RateLimitRule, UpstreamSettingsPolicySpec, CompressionBuffers, NginxProxySpec)

Size is a string value representing a size. Size can be specified in bytes, kilobytes (k), megabytes (m), or gigabytes (g). Examples: 1024, 8k, 1m.

Snippet

(Appears on: SnippetsFilterSpec, SnippetsPolicySpec)

Snippet represents an NGINX configuration snippet.

Field Description
context
NginxContext

Context is the NGINX context to insert the snippet into.
value
string

Value is the NGINX configuration snippet.

SnippetsFilterConditionReason (string alias)

SnippetsFilterConditionReason is a reason for a SnippetsFilter condition type.

Value Description

"Accepted"

SnippetsFilterConditionReasonAccepted is used with the Accepted condition type when the condition is true.

"Invalid"

SnippetsFilterConditionReasonInvalid is used with the Accepted condition type when SnippetsFilter is invalid.

SnippetsFilterConditionType (string alias)

SnippetsFilterConditionType is a type of condition associated with SnippetsFilter.

Value Description

"Accepted"

SnippetsFilterConditionTypeAccepted indicates that the SnippetsFilter is accepted.

Possible reasons for this condition to be True:

  • Accepted

Possible reasons for this condition to be False:

  • Invalid.

SnippetsFilterSpec

(Appears on: SnippetsFilter)

SnippetsFilterSpec defines the desired state of the SnippetsFilter.

Field Description
snippets
[]Snippet

Snippets is a list of NGINX configuration snippets. There can only be one snippet per context. Allowed contexts: main, http, http.server, http.server.location.

SnippetsFilterStatus

(Appears on: SnippetsFilter)

SnippetsFilterStatus defines the state of SnippetsFilter.

Field Description
controllers
[]ControllerStatus

Controllers is a list of Gateway API controllers that processed the SnippetsFilter and the status of the SnippetsFilter with respect to each controller.

SnippetsPolicySpec

(Appears on: SnippetsPolicy)

SnippetsPolicySpec defines the desired state of the SnippetsPolicy.

Field Description
targetRefs
[]sigs.k8s.io/gateway-api/apis/v1.LocalPolicyTargetReference

TargetRefs identifies API object(s) to apply the policy to.
snippets
[]Snippet 		(Optional)

Snippets is a list of snippets to be injected into the NGINX configuration.

SpanAttribute

(Appears on: Telemetry, Tracing)

SpanAttribute is a key value pair to be added to a tracing span.

Field Description
key
string

Key is the key for a span attribute. Format: must have all ‘“’ escaped and must not contain any ‘$’ or end with an unescaped ‘\’
value
string

Value is the value for a span attribute. Format: must have all ‘“’ escaped and must not contain any ‘$’ or end with an unescaped ‘\’

TLSReferenceType (string alias)

(Appears on: GatewayLinkTLS)

TLSReferenceType specifies where the BIG-IP SSL profiles come from.

Value Description

"bigip"

TLSReferenceBigIP means the SSL profiles already exist on BIG-IP.

"secret"

TLSReferenceSecret means the SSL profiles are sourced from Kubernetes secrets.

UpstreamKeepAlive

(Appears on: UpstreamSettingsPolicySpec)

UpstreamKeepAlive defines the keep-alive settings for upstreams.

Field Description
connections
int32 		(Optional)

Connections sets the maximum number of idle keep-alive connections to upstream servers that are preserved in the cache of each nginx worker process. When this number is exceeded, the least recently used connections are closed. The keepAlive directive for upstreams defaults to 32. To override this value, set the connections field. To disable the keepAlive directive, set connections to 0. Directive: https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive
requests
int32 		(Optional)

Requests sets the maximum number of requests that can be served through one keep-alive connection. After the maximum number of requests are made, the connection is closed. Directive: https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive_requests
time
Duration 		(Optional)

Time defines the maximum time during which requests can be processed through one keep-alive connection. After this time is reached, the connection is closed following the subsequent request processing. Directive: https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive_time
timeout
Duration 		(Optional)

Timeout defines the keep-alive timeout for upstreams. Directive: https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive_timeout

UpstreamSettingsPolicySpec

(Appears on: UpstreamSettingsPolicy)

UpstreamSettingsPolicySpec defines the desired state of the UpstreamSettingsPolicy.

Field Description
zoneSize
Size 		(Optional)

ZoneSize is the size of the shared memory zone used by the upstream. This memory zone is used to share the upstream configuration between nginx worker processes. The more servers that an upstream has, the larger memory zone is required. Default: OSS: 512k, Plus: 1m. Directive: https://nginx.org/en/docs/http/ngx_http_upstream_module.html#zone
keepAlive
UpstreamKeepAlive 		(Optional)

KeepAlive defines the keep-alive settings.
loadBalancingMethod
LoadBalancingType 		(Optional)

LoadBalancingMethod specifies the load balancing algorithm to be used for the upstream. If not specified, NGINX Gateway Fabric defaults to random two least_conn, which differs from the standard NGINX default round-robin.
hashMethodKey
HashMethodKey 		(Optional)

HashMethodKey defines the key used for hash-based load balancing methods. This field is required when LoadBalancingMethod is set to hash or hash consistent.
useClusterIP
bool 		(Optional)

UseClusterIP configures NGINX to route to the Service ClusterIP and port instead of individual Pod IPs. When enabled, NGINX will target a single upstream server corresponding to the Service’s ClusterIP, which is useful for service mesh compatibility and other Kubernetes controllers/operators that require traffic to traverse the Service VIP. This setting applies only when the target Service has a ClusterIP. For headless Services (ClusterIP: None) and ExternalName Services, normal endpoint resolution is used instead. This setting is also not applied to L4/stream upstreams. Defaults to false.
targetRefs
[]sigs.k8s.io/gateway-api/apis/v1.LocalPolicyTargetReference

TargetRefs identifies API object(s) to apply the policy to. Objects must be in the same namespace as the policy. Support: Service

TargetRefs must be distinct. The name field must be unique for all targetRef entries in the UpstreamSettingsPolicy.

WAFPolicySpec

(Appears on: WAFPolicy)

WAFPolicySpec defines the desired state of a WAFPolicy.

Field Description
targetRefs
[]sigs.k8s.io/gateway-api/apis/v1.LocalPolicyTargetReference

TargetRefs identifies API object(s) to apply the policy to. Objects must be in the same namespace as the policy. All targets must be of the same Kind (all Gateways OR all HTTPRoutes OR all GRPCRoutes). Support: Gateway, HTTPRoute, GRPCRoute.
type
PolicySourceType

Type identifies the source type for the policy bundle. HTTP fetches directly from a URL; NIM uses the NGINX Instance Manager bundles API; N1C uses the F5 NGINX One Console security policies API; PLM references an APPolicy CRD managed by the Policy Lifecycle Manager.
policySource
PolicySource 		(Optional)

PolicySource holds all non-CRD bundle fetch configuration. Used for HTTP, NIM, and N1C policy types. Must not be set when type is PLM.
policyRef
PolicyRef 		(Optional)

PolicyRef holds all CRD-backed policy references. Used for the PLM policy type.
securityLogs
[]WAFSecurityLog 		(Optional)

SecurityLogs defines security logging configurations.

WAFSecurityLog

(Appears on: WAFPolicySpec)

WAFSecurityLog defines security logging configuration for app_protect_security_log directives. Exactly one of logSource or logRef must be set.

Field Description
logSource
LogSource 		(Optional)

LogSource configures all non-CRD log profile bundle sources for this log entry. Used for defaultProfile, httpSource, nimSource, and n1cSource. Must not be set when logRef is used.
logRef
LogRef 		(Optional)

LogRef configures all CRD-backed log profile references for this log entry. Used for PLM-backed APLogConf references.
destination
SecurityLogDestination

Destination defines where security logs are sent.

gateway.nginx.org/v1alpha2

Package v1alpha2 contains API Schema definitions for the gateway.nginx.org API group.

Resource Types:

NginxProxy

NginxProxy is a configuration object that can be referenced from a GatewayClass parametersRef or a Gateway infrastructure.parametersRef. It provides a way to configure data plane settings. If referenced from a GatewayClass, the settings apply to all Gateways attached to the GatewayClass. If referenced from a Gateway, the settings apply to that Gateway alone. If both a Gateway and its GatewayClass reference an NginxProxy, the settings are merged. Settings specified on the Gateway NginxProxy override those set on the GatewayClass NginxProxy.

Field Description
apiVersion
string		 gateway.nginx.org/v1alpha2
kind
string 		NginxProxy
metadata
Kubernetes meta/v1.ObjectMeta 		Refer to the Kubernetes API documentation for the fields of the metadata field.
spec
NginxProxySpec

Spec defines the desired state of the NginxProxy.



ipFamily
IPFamilyType 		(Optional)

IPFamily specifies the IP family to be used by the NGINX. If not set, NGF inspects the default/kubernetes Service’s ipFamilies field at startup to obtain the IP family of the cluster and configure NGINX accordingly.
telemetry
Telemetry 		(Optional)

Telemetry specifies the OpenTelemetry configuration.
metrics
Metrics 		(Optional)

Metrics defines the configuration for Prometheus scraping metrics. Changing this value results in a re-roll of the NGINX deployment.
rewriteClientIP
RewriteClientIP 		(Optional)

RewriteClientIP defines configuration for rewriting the client IP to the original client’s IP.
logging
NginxLogging 		(Optional)

Logging defines logging related settings for NGINX.
nginxPlus
NginxPlus 		(Optional)

NginxPlus specifies NGINX Plus additional settings.
disableHTTP2
bool 		(Optional)

DisableHTTP2 defines if http2 should be disabled for all servers. If not specified, or set to false, http2 will be enabled for all servers.
useClusterIP
bool 		(Optional)

UseClusterIP configures NGINX to route to the Service ClusterIP and port instead of individual Pod IPs. When enabled, NGINX will target a single upstream server corresponding to the Service’s ClusterIP, which is useful for service mesh compatibility and other Kubernetes controllers/operators that require traffic to traverse the Service VIP. This setting applies only when the target Service has a ClusterIP. For headless Services (ClusterIP: None) and ExternalName Services, normal endpoint resolution is used instead. This setting is also not applied to L4/stream upstreams. A UseClusterIP value set in an UpstreamSettingsPolicy for a Service takes precedence over this setting. Defaults to false.
disableSNIHostValidation
bool 		(Optional)

DisableSNIHostValidation disables the validation that ensures the SNI hostname matches the Host header in HTTPS requests. When disabled, HTTPS connections can be reused for requests to different hostnames covered by the same certificate. This resolves HTTP/2 connection coalescing issues with wildcard certificates but introduces security risks as described in Gateway API GEP-3567. If not specified, defaults to false (validation enabled).
kubernetes
KubernetesSpec 		(Optional)

Kubernetes contains the configuration for the NGINX Deployment and Service Kubernetes objects.
workerConnections
int32 		(Optional)

WorkerConnections specifies the maximum number of simultaneous connections that can be opened by a worker process. Default is 1024.
workerProcesses
int32 		(Optional)

WorkerProcesses configures the number of NGINX worker processes. The value must be an integer between 1 and 1024. When unset, NGINX uses “auto” (one worker per CPU core).
workerRlimitNofile
int32 		(Optional)

WorkerRlimitNofile changes the limit on the maximum number of open files (RLIMIT_NOFILE) for worker processes. Used to raise the limit without restarting the main process. When unset, NGINX inherits the limit from the operating system. NGINX directive: https://nginx.org/en/docs/ngx_core_module.html#worker_rlimit_nofile
dnsResolver
DNSResolver 		(Optional)

DNSResolver specifies the DNS resolver configuration for external name resolution. This enables support for routing to ExternalName Services.
serverTokens
string 		(Optional)

ServerTokens configures whether NGINX emits its version in the “Server” response header and on error pages.

OSS NGINX accepts: - “on”: Shows nginx and version (e.g. “nginx/1.25.0”) - “off”: Shows nginx only (e.g. “nginx”) - “build”: Shows version and build name (e.g. “nginx/1.25.0 (build-name)”)

NGINX Plus additionally accepts: - “”: Suppress the “Server” response header entirely - : Set a custom header value and supports variables

See: https://nginx.org/en/docs/http/ngx_http_core_module.html#server_tokens NGINX directive: https://nginx.org/en/docs/http/ngx_http_core_module.html#server_tokens Default is “off”.
compression
Compression 		(Optional)

Compression defines the configuration for HTTP response compression. When set, NGINX compresses responses for clients that support it, reducing bandwidth usage.
waf
WAFSpec 		(Optional)

WAF configures NGINX App Protect WAF functionality.
zoneSize
Size 		(Optional)

ZoneSize is the size of the shared memory zone used by the upstream. This memory zone is used to share the upstream configuration between nginx worker processes. The more servers that an upstream has, the larger memory zone is required. Default: OSS: 512k, Plus: 1m. Directive: https://nginx.org/en/docs/http/ngx_http_upstream_module.html#zone
disableBaseHeaders
[]BaseHeaderName 		(Optional)

DisableBaseHeaders specifies which default X-* base headers should be omitted from being added to the base proxy_set_header directives in the NGINX configuration. This allows users to set these headers themselves without NGF overriding them.

Supported values are limited to X-* base headers and “”. A value of “” disables all X-* base headers.

ObservabilityPolicy

ObservabilityPolicy is a Direct Attached Policy. It provides a way to configure observability settings for the NGINX Gateway Fabric data plane. Used in conjunction with the NginxProxy CRD that is attached to the GatewayClass parametersRef.

Field Description
apiVersion
string		 gateway.nginx.org/v1alpha2
kind
string 		ObservabilityPolicy
metadata
Kubernetes meta/v1.ObjectMeta 		Refer to the Kubernetes API documentation for the fields of the metadata field.
spec
ObservabilityPolicySpec

Spec defines the desired state of the ObservabilityPolicy.



tracing
Tracing 		(Optional)

Tracing allows for enabling and configuring tracing.
targetRefs
[]sigs.k8s.io/gateway-api/apis/v1.LocalPolicyTargetReference

TargetRefs identifies the API object(s) to apply the policy to. Objects must be in the same namespace as the policy. Support: HTTPRoute, GRPCRoute.

TargetRefs must be distinct. This means that the multi-part key defined by kind and name must be unique across all targetRef entries in the ObservabilityPolicy.
status
sigs.k8s.io/gateway-api/apis/v1.PolicyStatus

Status defines the state of the ObservabilityPolicy.

AgentLogLevel (string alias)

(Appears on: NginxLogging)

AgentLevel defines the log level of the NGINX agent process.

Value Description

"debug"

AgentLogLevelDebug is the debug level NGINX agent logs.

"error"

AgentLogLevelError is the error level NGINX agent logs.

"fatal"

AgentLogLevelFatal is the fatal level NGINX agent logs.

"info"

AgentLogLevelInfo is the info level NGINX agent logs.

"panic"

AgentLogLevelPanic is the panic level NGINX agent logs.

AutoscalingSpec

(Appears on: DeploymentSpec)

AutoscalingSpec is the configuration for the Horizontal Pod Autoscaling.

Field Description
behavior
Kubernetes autoscaling/v2.HorizontalPodAutoscalerBehavior 		(Optional)

Behavior configures the scaling behavior of the target in both Up and Down directions (scaleUp and scaleDown fields respectively). If not set, the default HPAScalingRules for scale up and scale down are used.
targetCPUUtilizationPercentage
int32 		(Optional)

Target cpu utilization percentage of HPA.
targetMemoryUtilizationPercentage
int32 		(Optional)

Target memory utilization percentage of HPA.
minReplicas
int32 		(Optional)

Minimum number of replicas.
metrics
[]Kubernetes autoscaling/v2.MetricSpec 		(Optional)

Metrics configures additional metrics options.
maxReplicas
int32

Maximum number of replicas.
enable
bool

Enable or disable Horizontal Pod Autoscaler.

BaseHeaderName (string alias)

(Appears on: NginxProxySpec)

BaseHeaderName is the name of a base X-* header that can be disabled from being added to the base proxy_set_header directives in the NGINX configuration.

Value Description

"*"

AllXBaseHeaders disables all X-* base headers.

"X-Forwarded-For"

HeaderXForwardedFor is the X-Forwarded-For header.

"X-Forwarded-Host"

HeaderXForwardedHost is the X-Forwarded-Host header.

"X-Forwarded-Port"

HeaderXForwardedPort is the X-Forwarded-Port header.

"X-Forwarded-Proto"

HeaderXForwardedProto is the X-Forwarded-Proto header.

"X-Real-IP"

HeaderXRealIP is the X-Real-IP header.

Compression

(Appears on: NginxProxySpec)

Compression defines the configuration for HTTP response compression.

Field Description
gzip
GzipSettings 		(Optional)

Gzip defines gzip module-specific compression settings.
buffers
CompressionBuffers 		(Optional)

Buffers sets the number and size of buffers used to compress a response.

NGINX directive: https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_buffers
level
int32 		(Optional)

Level sets the compression level. Higher values provide better compression but use more CPU.

NGINX directive: https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_comp_level
minLength
int32 		(Optional)

MinLength sets the minimum length of a response that will be compressed. The length is determined from the “Content-Length” response header field.

NGINX directive: https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_min_length
type
CompressionType

Type specifies the compression algorithm to use. Currently only gzip is supported.
mimeTypes
[]string 		(Optional)

MimeTypes specifies the MIME types to compress in addition to “text/html”. “text/html” is always compressed when compression is enabled. Wildcards like “text/*” are not supported by NGINX. Example: [“application/json”, “text/css”, “application/javascript”]

NGINX directive: https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_types

CompressionBuffers

(Appears on: Compression)

CompressionBuffers defines the number and size of buffers used for compression.

Field Description
size
Size

Size sets the size of each buffer.
number
int32

Number sets the number of buffers.

CompressionType (string alias)

(Appears on: Compression)

CompressionType defines the type of compression algorithm.

Value Description

"gzip"

GzipCompressionType specifies gzip compression.

ContainerSpec

(Appears on: DaemonSetSpec, DeploymentSpec)

ContainerSpec defines container fields for the NGINX container.

Field Description
debug
bool 		(Optional)

Debug enables debugging for NGINX by using the nginx-debug binary.
image
Image 		(Optional)

Image is the NGINX image to use.
resources
Kubernetes core/v1.ResourceRequirements 		(Optional)

Resources describes the compute resource requirements.
lifecycle
Kubernetes core/v1.Lifecycle 		(Optional)

Lifecycle describes actions that the management system should take in response to container lifecycle events. For the PostStart and PreStop lifecycle handlers, management of the container blocks until the action is complete, unless the container process fails, in which case the handler is aborted.
readinessProbe
ReadinessProbeSpec 		(Optional)

ReadinessProbe defines the readiness probe for the NGINX container.
hostPorts
[]HostPort 		(Optional)

HostPorts are the list of ports to expose on the host.
volumeMounts
[]Kubernetes core/v1.VolumeMount 		(Optional)

VolumeMounts describe the mounting of Volumes within a container.

DNSResolver

(Appears on: NginxProxySpec)

DNSResolver specifies the DNS resolver configuration for NGINX. This enables dynamic DNS resolution for ExternalName Services. Corresponds to the NGINX resolver directive: https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver

Field Description
timeout
Duration 		(Optional)

Timeout specifies the timeout for name resolution.
cacheTTL
Duration 		(Optional)

CacheTTL specifies how long to cache DNS responses.
disableIPv6
bool 		(Optional)

DisableIPv6 disables IPv6 lookups. If not specified, or set to false, IPv6 lookups will be enabled.
addresses
[]DNSResolverAddress

Addresses specifies the list of DNS server addresses. Each address can be an IP address or hostname. Example: [{“type”: “IPAddress”, “value”: “8.8.8.8”}, {“type”: “Hostname”, “value”: “dns.google”}]

DNSResolverAddress

(Appears on: DNSResolver)

DNSResolverAddress specifies the address type and value for a DNS resolver address.

Field Description
type
DNSResolverAddressType

Type specifies the type of address.
value
string

Value specifies the address value. When Type is “IPAddress”, this must be a valid IPv4 or IPv6 address. When Type is “Hostname”, this must be a valid hostname.

DNSResolverAddressType (string alias)

(Appears on: DNSResolverAddress)

DNSResolverAddressType specifies the type of DNS resolver address.

Value Description

"Hostname"

DNSResolverHostnameType specifies that the address is a hostname.

"IPAddress"

DNSResolverIPAddressType specifies that the address is an IP address.

DaemonSetSpec

(Appears on: KubernetesSpec)

DaemonSet is the configuration for the NGINX DaemonSet.

Field Description
container
ContainerSpec 		(Optional)

Container defines container fields for the NGINX container.
wafContainers
WAFContainerSpec 		(Optional)

WAFContainers defines container specifications for NGINX App Protect WAF v5 containers. These containers are only deployed when WAF is enabled in the NginxProxy spec.
pod
PodSpec 		(Optional)

Pod defines Pod-specific fields.
patches
[]Patch 		(Optional)

Patches are custom patches to apply to the NGINX DaemonSet.

DeploymentSpec

(Appears on: KubernetesSpec)

Deployment is the configuration for the NGINX Deployment.

Field Description
replicas
int32 		(Optional)

Number of desired Pods.
autoscaling
AutoscalingSpec 		(Optional)

Autoscaling defines the configuration for Horizontal Pod Autoscaling.
podDisruptionBudget
PodDisruptionBudgetSpec 		(Optional)

PodDisruptionBudget is the configuration for limiting the number of concurrent disruptions of a pod. A PodDisruptionBudget is created when this field is set.
wafContainers
WAFContainerSpec 		(Optional)

WAFContainers defines container specifications for NGINX App Protect WAF v5 containers. These containers are only deployed when WAF is enabled in the NginxProxy spec.
pod
PodSpec 		(Optional)

Pod defines Pod-specific fields.
container
ContainerSpec 		(Optional)

Container defines container fields for the NGINX container.
patches
[]Patch 		(Optional)

Patches are custom patches to apply to the NGINX Deployment.

DisableTelemetryFeature (string alias)

(Appears on: Telemetry)

DisableTelemetryFeature is a telemetry feature that can be disabled.

Value Description

"DisableTracing"

DisableTracing disables the OpenTelemetry tracing feature.

ExternalTrafficPolicy (string alias)

(Appears on: ServiceSpec)

ExternalTrafficPolicy describes how nodes distribute service traffic they receive on one of the Service’s “externally-facing” addresses (NodePorts and LoadBalancer IPs).

Value Description

"Cluster"

ExternalTrafficPolicyCluster routes traffic to all endpoints.

"Local"

ExternalTrafficPolicyLocal preserves the source IP of the traffic by routing only to endpoints on the same node as the traffic was received on (dropping the traffic if there are no local endpoints).

GzipHTTPVersion (string alias)

(Appears on: GzipSettings)

GzipHTTPVersion defines the minimum HTTP version required for gzip compression.

Value Description

"1.0"

GzipHTTPVersion10 sets the minimum HTTP version to 1.0.

"1.1"

GzipHTTPVersion11 sets the minimum HTTP version to 1.1.

GzipProxiedType (string alias)

(Appears on: GzipSettings)

GzipProxiedType defines the conditions under which responses from proxied requests are compressed.

Value Description

"any"

GzipProxiedAny enables compression for all proxied requests.

"auth"

GzipProxiedAuth enables compression if a request header includes “Authorization”.

"expired"

GzipProxiedExpired enables compression if a response header includes the “Expires” field.

"no-cache"

GzipProxiedNoCache enables compression if a response header includes “Cache-Control” with the “no-cache” parameter.

"no_etag"

GzipProxiedNoETag enables compression if a response header does not include “ETag”.

"no_last_modified"

GzipProxiedNoLastModified enables compression if a response header does not include “Last-Modified”.

"no-store"

GzipProxiedNoStore enables compression if a response header includes “Cache-Control” with the “no-store” parameter.

"off"

GzipProxiedOff disables compression for all proxied requests.

"private"

GzipProxiedPrivate enables compression if a response header includes “Cache-Control” with the “private” parameter.

GzipSettings

(Appears on: Compression)

GzipSettings defines gzip module-specific compression settings.

Field Description
vary
bool 		(Optional)

Vary enables or disables inserting the “Vary: Accept-Encoding” response header when gzip compression is active.

NGINX directive: https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_vary
httpVersion
GzipHTTPVersion 		(Optional)

HTTPVersion sets the minimum HTTP version of a request required to compress a response.

NGINX directive: https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_http_version
disable
[]string 		(Optional)

Disable specifies regular expressions to match User-Agent headers of requests that should not be gzip-compressed.

NGINX directive: https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_disable
proxied
[]GzipProxiedType 		(Optional)

Proxied enables or disables gzip compression for proxied requests depending on the request and response. Accepted values are: “off”, “expired”, “no-cache”, “no-store”, “private”, “no_last_modified”, “no_etag”, “auth”, “any”. Multiple values can be specified.

NGINX directive: https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_proxied

HostPort

(Appears on: ContainerSpec)

HostPort exposes an nginx container port on the host.

Field Description
port
int32

Port to expose on the host.
containerPort
int32

ContainerPort is the port on the nginx container to map to the HostPort.

IPFamilyType (string alias)

(Appears on: NginxProxySpec)

IPFamilyType specifies the IP family to be used by NGINX.

Value Description

"dual"

Dual specifies that NGINX will use both IPv4 and IPv6.

"ipv4"

IPv4 specifies that NGINX will use only IPv4.

"ipv6"

IPv6 specifies that NGINX will use only IPv6.

Image

(Appears on: ContainerSpec, WAFContainerConfig)

Image is the NGINX image to use.

Field Description
repository
string 		(Optional)

Repository is the image path. Default is ghcr.io/nginx/nginx-gateway-fabric/nginx.
tag
string 		(Optional)

Tag is the image tag to use. Default matches the tag of the control plane.
pullPolicy
PullPolicy 		(Optional)

PullPolicy describes a policy for if/when to pull a container image.

KubernetesSpec

(Appears on: NginxProxySpec)

KubernetesSpec contains the configuration for the NGINX Deployment and Service Kubernetes objects.

Field Description
deployment
DeploymentSpec 		(Optional)

Deployment is the configuration for the NGINX Deployment. This is the default deployment option.
daemonSet
DaemonSetSpec 		(Optional)

DaemonSet is the configuration for the NGINX DaemonSet.
service
ServiceSpec 		(Optional)

Service is the configuration for the NGINX Service.

Metrics

(Appears on: NginxProxySpec)

Metrics defines the configuration for Prometheus scraping metrics.

Field Description
port
int32 		(Optional)

Port where the Prometheus metrics are exposed.
disable
bool 		(Optional)

Disable serving Prometheus metrics on the listen port.

NginxAccessLog

(Appears on: NginxLogging)

NginxAccessLog defines the configuration for an NGINX access log.

Field Description
disable
bool 		(Optional)

Disable turns off access logging when set to true.
format
string 		(Optional)

Format specifies the custom log format string. If not specified, NGINX default ‘combined’ format is used. For now only path /dev/stdout can be used. Single quotes and line breaks are not allowed because the format is rendered inside a single-quoted NGINX log_format directive. See https://nginx.org/en/docs/http/ngx_http_log_module.html#log_format
escape
NginxAccessLogEscapeType 		(Optional)

Escape specifies how to escape characters in variables for access log. Possible values are: default, json, none. If not specified, ‘default’ escaping is used. See https://nginx.org/en/docs/http/ngx_http_log_module.html#log_format

NginxAccessLogEscapeType (string alias)

(Appears on: NginxAccessLog)

NginxAccessLogEscapeType defines the escape setting for variables in access log format.

Value Description

"default"

NginxAccessLogEscapeDefault specifies that characters ‘\“’, ‘\’, and other characters with values less than 32 or above 126 are escaped as ‘\xXX’.

"json"

NginxAccessLogEscapeJSON specifies that all characters not allowed in JSON strings are escaped. Characters ‘\“’ and ‘\’ are escaped as ‘\”’ and ‘\’, characters with values less than 32 are escaped as ‘\n’, ‘\r’, ‘\t’, ‘\b’, ‘\f’, or ‘\u00XX’.

"none"

NginxAccessLogEscapeNone disables escaping of characters.

NginxErrorLogFormat (string alias)

(Appears on: NginxLogging)

NginxErrorLogFormat defines the output format for NGINX error logs.

Value Description

"default"

NginxErrorLogFormatDefault uses NGINX’s standard error log format.

"json"

NginxErrorLogFormatJSON enables JSON-formatted error logs. Requires NGINX Plus.

NginxErrorLogLevel (string alias)

(Appears on: NginxLogging)

NginxErrorLogLevel type defines the log level of error logs for NGINX.

Value Description

"alert"

NginxLogLevelAlert is the alert level for NGINX error logs.

"crit"

NginxLogLevelCrit is the crit level for NGINX error logs.

"debug"

NginxLogLevelDebug is the debug level for NGINX error logs.

"emerg"

NginxLogLevelEmerg is the emerg level for NGINX error logs.

"error"

NginxLogLevelError is the error level for NGINX error logs.

"info"

NginxLogLevelInfo is the info level for NGINX error logs.

"notice"

NginxLogLevelNotice is the notice level for NGINX error logs.

"warn"

NginxLogLevelWarn is the warn level for NGINX error logs.

NginxLogging

(Appears on: NginxProxySpec)

NginxLogging defines logging related settings for NGINX.

Field Description
errorLevel
NginxErrorLogLevel 		(Optional)

ErrorLevel defines the error log level. Possible log levels listed in order of increasing severity are debug, info, notice, warn, error, crit, alert, and emerg. Setting a certain log level will cause all messages of the specified and more severe log levels to be logged. For example, the log level ‘error’ will cause error, crit, alert, and emerg messages to be logged. https://nginx.org/en/docs/ngx_core_module.html#error_log
errorLogFormat
NginxErrorLogFormat 		(Optional)

ErrorLogFormat controls the output format of the NGINX error_log directive. Set to ‘json’ to enable JSON-formatted error logs for NGINX Plus only and cannot be combined with errorLevel: debug. When set to ‘json’, NGINX Gateway Fabric also emits a JSON-formatted access log if the user has not supplied a custom access log format. See https://nginx.org/en/docs/ngx_core_module.html#error_log
agentLevel
AgentLogLevel 		(Optional)

AgentLevel defines the log level of the NGINX agent process. Changing this value results in a re-roll of the NGINX deployment.
accessLog
NginxAccessLog 		(Optional)

AccessLog defines the access log settings, including format itself and disabling option. For now only path /dev/stdout can be used.

NginxPlus

(Appears on: NginxProxySpec)

NginxPlus specifies NGINX Plus additional settings. These will only be applied if NGINX Plus is being used.

Field Description
allowedAddresses
[]NginxPlusAllowAddress 		(Optional)

AllowedAddresses specifies IPAddresses or CIDR blocks to the allow list for accessing the NGINX Plus API.

NginxPlusAllowAddress

(Appears on: NginxPlus)

NginxPlusAllowAddress specifies the address type and value for an NginxPlus allow address.

Field Description
type
NginxPlusAllowAddressType

Type specifies the type of address.
value
string

Value specifies the address value.

NginxPlusAllowAddressType (string alias)

(Appears on: NginxPlusAllowAddress)

NginxPlusAllowAddressType specifies the type of address.

Value Description

"CIDR"

NginxPlusAllowCIDRAddressType specifies that the address is a CIDR block.

"IPAddress"

NginxPlusAllowIPAddressType specifies that the address is an IP address.

NginxProxySpec

(Appears on: NginxProxy)

NginxProxySpec defines the desired state of the NginxProxy.

Field Description
ipFamily
IPFamilyType 		(Optional)

IPFamily specifies the IP family to be used by the NGINX. If not set, NGF inspects the default/kubernetes Service’s ipFamilies field at startup to obtain the IP family of the cluster and configure NGINX accordingly.
telemetry
Telemetry 		(Optional)

Telemetry specifies the OpenTelemetry configuration.
metrics
Metrics 		(Optional)

Metrics defines the configuration for Prometheus scraping metrics. Changing this value results in a re-roll of the NGINX deployment.
rewriteClientIP
RewriteClientIP 		(Optional)

RewriteClientIP defines configuration for rewriting the client IP to the original client’s IP.
logging
NginxLogging 		(Optional)

Logging defines logging related settings for NGINX.
nginxPlus
NginxPlus 		(Optional)

NginxPlus specifies NGINX Plus additional settings.
disableHTTP2
bool 		(Optional)

DisableHTTP2 defines if http2 should be disabled for all servers. If not specified, or set to false, http2 will be enabled for all servers.
useClusterIP
bool 		(Optional)

UseClusterIP configures NGINX to route to the Service ClusterIP and port instead of individual Pod IPs. When enabled, NGINX will target a single upstream server corresponding to the Service’s ClusterIP, which is useful for service mesh compatibility and other Kubernetes controllers/operators that require traffic to traverse the Service VIP. This setting applies only when the target Service has a ClusterIP. For headless Services (ClusterIP: None) and ExternalName Services, normal endpoint resolution is used instead. This setting is also not applied to L4/stream upstreams. A UseClusterIP value set in an UpstreamSettingsPolicy for a Service takes precedence over this setting. Defaults to false.
disableSNIHostValidation
bool 		(Optional)

DisableSNIHostValidation disables the validation that ensures the SNI hostname matches the Host header in HTTPS requests. When disabled, HTTPS connections can be reused for requests to different hostnames covered by the same certificate. This resolves HTTP/2 connection coalescing issues with wildcard certificates but introduces security risks as described in Gateway API GEP-3567. If not specified, defaults to false (validation enabled).
kubernetes
KubernetesSpec 		(Optional)

Kubernetes contains the configuration for the NGINX Deployment and Service Kubernetes objects.
workerConnections
int32 		(Optional)

WorkerConnections specifies the maximum number of simultaneous connections that can be opened by a worker process. Default is 1024.
workerProcesses
int32 		(Optional)

WorkerProcesses configures the number of NGINX worker processes. The value must be an integer between 1 and 1024. When unset, NGINX uses “auto” (one worker per CPU core).
workerRlimitNofile
int32 		(Optional)

WorkerRlimitNofile changes the limit on the maximum number of open files (RLIMIT_NOFILE) for worker processes. Used to raise the limit without restarting the main process. When unset, NGINX inherits the limit from the operating system. NGINX directive: https://nginx.org/en/docs/ngx_core_module.html#worker_rlimit_nofile
dnsResolver
DNSResolver 		(Optional)

DNSResolver specifies the DNS resolver configuration for external name resolution. This enables support for routing to ExternalName Services.
serverTokens
string 		(Optional)

ServerTokens configures whether NGINX emits its version in the “Server” response header and on error pages.

OSS NGINX accepts: - “on”: Shows nginx and version (e.g. “nginx/1.25.0”) - “off”: Shows nginx only (e.g. “nginx”) - “build”: Shows version and build name (e.g. “nginx/1.25.0 (build-name)”)

NGINX Plus additionally accepts: - “”: Suppress the “Server” response header entirely - : Set a custom header value and supports variables

See: https://nginx.org/en/docs/http/ngx_http_core_module.html#server_tokens NGINX directive: https://nginx.org/en/docs/http/ngx_http_core_module.html#server_tokens Default is “off”.
compression
Compression 		(Optional)

Compression defines the configuration for HTTP response compression. When set, NGINX compresses responses for clients that support it, reducing bandwidth usage.
waf
WAFSpec 		(Optional)

WAF configures NGINX App Protect WAF functionality.
zoneSize
Size 		(Optional)

ZoneSize is the size of the shared memory zone used by the upstream. This memory zone is used to share the upstream configuration between nginx worker processes. The more servers that an upstream has, the larger memory zone is required. Default: OSS: 512k, Plus: 1m. Directive: https://nginx.org/en/docs/http/ngx_http_upstream_module.html#zone
disableBaseHeaders
[]BaseHeaderName 		(Optional)

DisableBaseHeaders specifies which default X-* base headers should be omitted from being added to the base proxy_set_header directives in the NGINX configuration. This allows users to set these headers themselves without NGF overriding them.

Supported values are limited to X-* base headers and “”. A value of “” disables all X-* base headers.

NodePort

(Appears on: ServiceSpec)

NodePort creates a port on each node on which the NGINX data plane service is exposed. The NodePort MUST map to a Gateway listener port, otherwise it will be ignored. If not specified, Kubernetes allocates a NodePort automatically if required. The default NodePort range enforced by Kubernetes is 30000-32767.

Field Description
port
int32

Port is the NodePort to expose.
listenerPort
int32

ListenerPort is the Gateway listener port that this NodePort maps to.

ObservabilityPolicySpec

(Appears on: ObservabilityPolicy)

ObservabilityPolicySpec defines the desired state of the ObservabilityPolicy.

Field Description
tracing
Tracing 		(Optional)

Tracing allows for enabling and configuring tracing.
targetRefs
[]sigs.k8s.io/gateway-api/apis/v1.LocalPolicyTargetReference

TargetRefs identifies the API object(s) to apply the policy to. Objects must be in the same namespace as the policy. Support: HTTPRoute, GRPCRoute.

TargetRefs must be distinct. This means that the multi-part key defined by kind and name must be unique across all targetRef entries in the ObservabilityPolicy.

Patch

(Appears on: DaemonSetSpec, DeploymentSpec, ServiceSpec)

Patch defines a patch to apply to a Kubernetes object.

Field Description
type
PatchType 		(Optional)

Type is the type of patch. Defaults to StrategicMerge.
value
k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1.JSON 		(Optional)

Value is the patch data as raw JSON. For StrategicMerge and Merge patches, this should be a JSON object. For JSONPatch patches, this should be a JSON array of patch operations.

PatchType (string alias)

(Appears on: Patch)

PatchType specifies the type of patch.

Value Description

"JSONPatch"

PatchTypeJSONPatch uses JSON patch (RFC 6902).

"Merge"

PatchTypeMerge uses merge patch (RFC 7386).

"StrategicMerge"

PatchTypeStrategicMerge uses strategic merge patch.

PodDisruptionBudgetSpec

(Appears on: DeploymentSpec)

PodDisruptionBudgetSpec is the configuration for PodDisruptionBudget, which limits the number of concurrent disruptions of a pod.

Field Description
minAvailable
k8s.io/apimachinery/pkg/util/intstr.IntOrString 		(Optional)

MinAvailable is the minimum number of pods that must be available after an eviction. Value can be an absolute number (e.g. 1) or a percentage of desired pods (e.g. 50%). Mutually exclusive with MaxUnavailable.
maxUnavailable
k8s.io/apimachinery/pkg/util/intstr.IntOrString 		(Optional)

MaxUnavailable is the maximum number of pods that can be unavailable after an eviction. Value can be an absolute number (e.g. 1) or a percentage of desired pods (e.g. 50%). Mutually exclusive with MinAvailable.
unhealthyPodEvictionPolicy
Kubernetes policy/v1.UnhealthyPodEvictionPolicyType 		(Optional)

UnhealthyPodEvictionPolicy defines when unhealthy pods should be considered for eviction. Valid values are IfHealthyBudget and AlwaysAllow. Defaults to IfHealthyBudget if not set.

PodSpec

(Appears on: DaemonSetSpec, DeploymentSpec)

PodSpec defines Pod-specific fields.

Field Description
terminationGracePeriodSeconds
int64 		(Optional)

TerminationGracePeriodSeconds is the optional duration in seconds the pod needs to terminate gracefully. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). If this value is nil, the default grace period will be used instead. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. Defaults to 30 seconds.
affinity
Kubernetes core/v1.Affinity 		(Optional)

Affinity is the pod’s scheduling constraints.
nodeSelector
map[string]string 		(Optional)

NodeSelector is a selector which must be true for the pod to fit on a node. Selector which must match a node’s labels for the pod to be scheduled on that node.
tolerations
[]Kubernetes core/v1.Toleration 		(Optional)

Tolerations allow the scheduler to schedule Pods with matching taints.
volumes
[]Kubernetes core/v1.Volume 		(Optional)

Volumes represents named volumes in a pod that may be accessed by any container in the pod.
topologySpreadConstraints
[]Kubernetes core/v1.TopologySpreadConstraint 		(Optional)

TopologySpreadConstraints describes how a group of Pods ought to spread across topology domains. Scheduler will schedule Pods in a way which abides by the constraints. All topologySpreadConstraints are ANDed.

PullPolicy (string alias)

(Appears on: Image)

PullPolicy describes a policy for if/when to pull a container image.

Value Description

"Always"

PullAlways means that kubelet always attempts to pull the latest image. Container will fail if the pull fails.

"IfNotPresent"

PullIfNotPresent means that kubelet pulls if the image isn’t present on disk. Container will fail if the image isn’t present and the pull fails.

"Never"

PullNever means that kubelet never pulls an image, but only uses a local image. Container will fail if the image isn’t present.

ReadinessProbeSpec

(Appears on: ContainerSpec)

ReadinessProbeSpec defines the configuration for the NGINX readiness probe.

Field Description
port
int32 		(Optional)

Port is the port on which the readiness endpoint is exposed. If not specified, the default port is 8081.
path
string 		(Optional)

Path is the path on which the readiness endpoint is exposed. If not specified, the default path is /readyz. Must start with a forward slash and contain only valid URL path characters.
initialDelaySeconds
int32 		(Optional)

InitialDelaySeconds is the number of seconds after the container has started before the readiness probe is initiated. If not specified, the default is 3 seconds.
expose
bool 		(Optional)

Expose toggles whether the endpoint should be exposed through the Gateway Service object. This allows an external LoadBalancer to perform healthchecks. Default is false.

RewriteClientIP

(Appears on: NginxProxySpec)

RewriteClientIP specifies the configuration for rewriting the client’s IP address.

Field Description
mode
RewriteClientIPModeType 		(Optional)

Mode defines how NGINX will rewrite the client’s IP address. There are two possible modes: - ProxyProtocol: NGINX will rewrite the client’s IP using the PROXY protocol header. - XForwardedFor: NGINX will rewrite the client’s IP using the X-Forwarded-For header. Sets NGINX directive real_ip_header: https://nginx.org/en/docs/http/ngx_http_realip_module.html#real_ip_header
setIPRecursively
bool 		(Optional)

SetIPRecursively configures whether recursive search is used when selecting the client’s address from the X-Forwarded-For header. It is used in conjunction with TrustedAddresses. If enabled, NGINX will recurse on the values in X-Forwarded-Header from the end of array to start of array and select the first untrusted IP. For example, if X-Forwarded-For is [11.11.11.11, 22.22.22.22, 55.55.55.1], and TrustedAddresses is set to 55.55.55.1⁄32, NGINX will rewrite the client IP to 22.22.22.22. If disabled, NGINX will select the IP at the end of the array. In the previous example, 55.55.55.1 would be selected. Sets NGINX directive real_ip_recursive: https://nginx.org/en/docs/http/ngx_http_realip_module.html#real_ip_recursive
trustedAddresses
[]RewriteClientIPAddress 		(Optional)

TrustedAddresses specifies the addresses that are trusted to send correct client IP information. If a request comes from a trusted address, NGINX will rewrite the client IP information, and forward it to the backend in the X-Forwarded-For* and X-Real-IP headers. If the request does not come from a trusted address, NGINX will not rewrite the client IP information. To trust all addresses (not recommended for production), set to 0.0.0.0/0. If no addresses are provided, NGINX will not rewrite the client IP information. Sets NGINX directive set_real_ip_from: https://nginx.org/en/docs/http/ngx_http_realip_module.html#set_real_ip_from This field is required if mode is set.

RewriteClientIPAddress

(Appears on: RewriteClientIP)

RewriteClientIPAddress specifies the address type and value for a RewriteClientIP address.

Field Description
type
RewriteClientIPAddressType

Type specifies the type of address.
value
string

Value specifies the address value.

RewriteClientIPAddressType (string alias)

(Appears on: RewriteClientIPAddress)

RewriteClientIPAddressType specifies the type of address.

Value Description

"CIDR"

RewriteClientIPCIDRAddressType specifies that the address is a CIDR block.

"Hostname"

RewriteClientIPHostnameAddressType specifies that the address is a Hostname.

"IPAddress"

RewriteClientIPIPAddressType specifies that the address is an IP address.

RewriteClientIPModeType (string alias)

(Appears on: RewriteClientIP)

RewriteClientIPModeType defines how NGINX Gateway Fabric will determine the client’s original IP address.

Value Description

"ProxyProtocol"

RewriteClientIPModeProxyProtocol configures NGINX to accept PROXY protocol and set the client’s IP address to the IP address in the PROXY protocol header. Sets the proxy_protocol parameter on the listen directive of all servers and sets real_ip_header to proxy_protocol: https://nginx.org/en/docs/http/ngx_http_realip_module.html#real_ip_header.

"XForwardedFor"

RewriteClientIPModeXForwardedFor configures NGINX to set the client’s IP address to the IP address in the X-Forwarded-For HTTP header. https://nginx.org/en/docs/http/ngx_http_realip_module.html#real_ip_header.

ServiceSpec

(Appears on: KubernetesSpec)

ServiceSpec is the configuration for the NGINX Service.

Field Description
type
ServiceType 		(Optional)

ServiceType describes ingress method for the Service.
externalTrafficPolicy
ExternalTrafficPolicy 		(Optional)

ExternalTrafficPolicy describes how nodes distribute service traffic they receive on one of the Service’s “externally-facing” addresses (NodePorts and LoadBalancer IPs).
loadBalancerIP
string 		(Optional)

LoadBalancerIP is a static IP address for the load balancer. Requires service type to be LoadBalancer.
loadBalancerClass
string 		(Optional)

LoadBalancerClass is the class of the load balancer implementation this Service belongs to. Requires service type to be LoadBalancer.
loadBalancerSourceRanges
[]string 		(Optional)

LoadBalancerSourceRanges are the IP ranges (CIDR) that are allowed to access the load balancer. Requires service type to be LoadBalancer.
nodePorts
[]NodePort 		(Optional)

NodePorts are the list of NodePorts to expose on the NGINX data plane service. Each NodePort MUST map to a Gateway listener port, otherwise it will be ignored. The default NodePort range enforced by Kubernetes is 30000-32767.
patches
[]Patch 		(Optional)

Patches are custom patches to apply to the NGINX Service.

ServiceType (string alias)

(Appears on: ServiceSpec)

ServiceType describes ingress method for the Service.

Value Description

"ClusterIP"

ServiceTypeClusterIP means a Service will only be accessible inside the cluster, via the cluster IP.

"LoadBalancer"

ServiceTypeLoadBalancer means a Service will be exposed via an external load balancer (if the cloud provider supports it), in addition to ‘NodePort’ type.

"NodePort"

ServiceTypeNodePort means a Service will be exposed on one port of every node, in addition to ‘ClusterIP’ type.

Telemetry

(Appears on: NginxProxySpec)

Telemetry specifies the OpenTelemetry configuration.

Field Description
disabledFeatures
[]DisableTelemetryFeature 		(Optional)

DisabledFeatures specifies OpenTelemetry features to be disabled.
exporter
TelemetryExporter 		(Optional)

Exporter specifies OpenTelemetry export parameters.
serviceName
string 		(Optional)

ServiceName is the “service.name” attribute of the OpenTelemetry resource. Default is ‘ngf:gateway-namespace:gateway-name’. If a value is provided by the user, then the default becomes a prefix to that value.
spanAttributes
[]SpanAttribute 		(Optional)

SpanAttributes are custom key/value attributes that are added to each span.

TelemetryExporter

(Appears on: Telemetry)

TelemetryExporter specifies OpenTelemetry export parameters.

Field Description
interval
Duration 		(Optional)

Interval is the maximum interval between two exports. Default: https://nginx.org/en/docs/ngx_otel_module.html#otel_exporter
batchSize
int32 		(Optional)

BatchSize is the maximum number of spans to be sent in one batch per worker. Default: https://nginx.org/en/docs/ngx_otel_module.html#otel_exporter
batchCount
int32 		(Optional)

BatchCount is the number of pending batches per worker, spans exceeding the limit are dropped. Default: https://nginx.org/en/docs/ngx_otel_module.html#otel_exporter
endpoint
string 		(Optional)

Endpoint is the address of OTLP/gRPC endpoint that will accept telemetry data. Format: alphanumeric hostname with optional http scheme and optional port.

TraceContext (string alias)

(Appears on: Tracing)

TraceContext specifies how to propagate traceparent/tracestate headers.

Value Description

"extract"

TraceContextExtract uses an existing trace context from the request, so that the identifiers of a trace and the parent span are inherited from the incoming request.

"ignore"

TraceContextIgnore skips context headers processing.

"inject"

TraceContextInject adds a new context to the request, overwriting existing headers, if any.

"propagate"

TraceContextPropagate updates the existing context (combines extract and inject).

TraceStrategy (string alias)

(Appears on: Tracing)

TraceStrategy defines the tracing strategy.

Value Description

"parent"

TraceStrategyParent enables tracing and only records spans if the parent span was sampled.

"ratio"

TraceStrategyRatio enables ratio-based tracing, defaulting to 100% sampling rate.

Tracing

(Appears on: ObservabilityPolicySpec)

Tracing allows for enabling and configuring OpenTelemetry tracing.

Field Description
strategy
TraceStrategy

Strategy defines if tracing is ratio-based or parent-based.
ratio
int32 		(Optional)

Ratio is the percentage of traffic that should be sampled. Integer from 0 to 100. By default, 100% of http requests are traced. Not applicable for parent-based tracing. If ratio is set to 0, tracing is disabled.
context
TraceContext 		(Optional)

Context specifies how to propagate traceparent/tracestate headers. Default: https://nginx.org/en/docs/ngx_otel_module.html#otel_trace_context
spanName
string 		(Optional)

SpanName defines the name of the Otel span. By default is the name of the location for a request. If specified, applies to all locations that are created for a route. Format: must have all ‘“’ escaped and must not contain any ‘$’ or end with an unescaped ‘\’ Examples of invalid names: some-$value, quoted-“value”-name, unescaped
spanAttributes
[]SpanAttribute 		(Optional)

SpanAttributes are custom key/value attributes that are added to each span.

WAFContainerConfig

(Appears on: WAFContainerSpec)

WAFContainerConfig defines the configuration for a single WAF container.

Field Description
image
Image 		(Optional)

Image is the container image to use for this WAF container.
resources
Kubernetes core/v1.ResourceRequirements 		(Optional)

Resources describes the compute resource requirements for this WAF container.
volumeMounts
[]Kubernetes core/v1.VolumeMount 		(Optional)

VolumeMounts describe the mounting of Volumes within the WAF container.

WAFContainerSpec

(Appears on: DaemonSetSpec, DeploymentSpec)

WAFContainerSpec defines the container specifications for NGINX App Protect WAF v5. NAP v5 requires two additional containers: waf-enforcer and waf-config-mgr.

Field Description
enforcer
WAFContainerConfig 		(Optional)

Enforcer defines the configuration for the WAF enforcer container. This container performs the actual WAF enforcement and policy application.
configManager
WAFContainerConfig 		(Optional)

ConfigManager defines the configuration for the WAF configuration manager container. This container manages policy configuration and communication with the enforcer.

WAFSpec

(Appears on: NginxProxySpec)

WAFSpec configures NGINX App Protect WAF.

Field Description
enable
bool 		(Optional)

Enable enables NGINX App Protect WAF functionality. When enabled, NGINX Gateway Fabric will deploy additional WAF containers (waf-enforcer and waf-config-mgr) alongside the main NGINX container. Default is false.
disableCookieSeed
bool 		(Optional)

DisableCookieSeed disables the app_protect_cookie_seed directive. By default, NGF sets this directive to a stable value derived from the Gateway UID, ensuring WAF session cookies are consistent across multiple NGINX replicas. Set this to true if you have pre-compiled the cookie seed into your WAF policy bundles via the compiler global settings, to avoid conflicting with the compiled-in value. Default is false.
bundleFailOpen
bool 		(Optional)

BundleFailOpen controls the behavior when a WAF policy bundle (policy or log profile) has not yet been successfully fetched. When set to true, NGINX configuration is pushed and traffic is served without WAF protection until the bundle becomes available. When false (the default), the configuration push is withheld until the bundle is fetched, maintaining a fail-closed posture.

Generated with gen-crd-api-reference-docs

-------------------------------------------------------------------------------- title: "Command-line reference guide" url: https://docs.nginx.com/nginx-gateway-fabric/reference/cli-help/index.md -------------------------------------------------------------------------------- ## Overview Learn about the commands available for the executable file of the NGINX Gateway Fabric container. --- ## Controller This command runs the NGINX Gateway Fabric control plane. *Usage*: ```shell gateway controller [flags] ``` --- ### Flags | Name | Type | Description | |-------------------------------------|----------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | _gateway-ctlr-name_ | _string_ | The name of the Gateway controller. The controller name must be in the form: `DOMAIN/PATH`. The controller's domain is `gateway.nginx.org`. | | _gatewayclass_ | _string_ | The name of the GatewayClass resource. Every NGINX Gateway Fabric must have a unique corresponding GatewayClass resource. | | _nginx-plus_ | _bool_ | Enable support for NGINX Plus. | | _gateway-api-experimental-features_ | _bool_ | Enable the experimental features of Gateway API which are supported by NGINX Gateway Fabric. Requires the Gateway APIs installed from the experimental channel. | | _config_ | _string_ | The name of the NginxGateway resource to be used for this controller's dynamic configuration. Lives in the same namespace as the controller. | | _service_ | _string_ | The name of the service that fronts this NGINX Gateway Fabric pod. Lives in the same namespace as the controller. | | _metrics-disable_ | _bool_ | Disable exposing metrics in the Prometheus format (Default: `false`). | | _metrics-port_ | _int_ | Sets the port where the Prometheus metrics are exposed. An integer between 1024 - 65535 (Default: `9113`) | | _metrics-secure-serving_ | _bool_ | Configures if the metrics endpoint should be secured using https. Note that this endpoint will be secured with a self-signed certificate (Default `false`). | | _health-disable_ | _bool_ | Disable running the health probe server (Default: `false`). | | _health-port_ | _int_ | Set the port where the health probe server is exposed. An integer between 1024 - 65535 (Default: `8081`). | | _leader-election-disable_ | _bool_ | Disable leader election, which is used to avoid multiple replicas of the NGINX Gateway Fabric reporting the status of the Gateway API resources. If disabled, all replicas of NGINX Gateway Fabric will update the statuses of the Gateway API resources (Default: `false`). | | _leader-election-lock-name_ | _string_ | The name of the leader election lock. A lease object with this name will be created in the same namespace as the controller (Default: `"nginx-gateway-leader-election-lock"`). | | _leader-election-lease-duration_ | _duration_ | The duration that non-leader candidates will wait to force acquire leadership. Must be parsable by [`time.ParseDuration`](https://pkg.go.dev/time#ParseDuration). Must be greater than the renew deadline. If not set, defaults to controller-runtime's built-in default (`15s`). | | _leader-election-renew-deadline_ | _duration_ | The duration that the acting leader will retry refreshing leadership before giving up. Must be parsable by [`time.ParseDuration`](https://pkg.go.dev/time#ParseDuration). Must be greater than the retry period and less than the lease duration. Increase this value if leader election is failing due to slow Kubernetes API server responses. If not set, defaults to controller-runtime's built-in default (`10s`). | | _leader-election-retry-period_ | _duration_ | The duration the leader election clients should wait between action tries. Must be parsable by [`time.ParseDuration`](https://pkg.go.dev/time#ParseDuration). Must be less than the renew deadline. If not set, defaults to controller-runtime's built-in default (`2s`). | | _product-telemetry-disable_ | _bool_ | Disable the collection of product telemetry (Default: `false`). | | _nginx-docker-secret_ | _list_ | The name of the NGINX docker registry Secret(s). Must exist in the same namespace that the NGINX Gateway Fabric control plane is running in (default namespace: nginx-gateway). | | _plm-storage-url_ | _string_ | The URL of the Policy Lifecycle Manager (PLM) storage service (HTTP or HTTPS). | | _plm-storage-credentials-secret_ | _string_ | The name of the Secret containing the PLM storage S3 secret access key (`seaweedfs_admin_secret`). Must exist in the same namespace as the NGINX Gateway Fabric control plane (default: `nginx-gateway`), unless prefixed with `/`. | | _plm-storage-ca-secret_ | _string_ | The name of the Secret containing the CA certificate (`ca.crt`) for verifying the PLM storage TLS server certificate. Must exist in the same namespace as the NGINX Gateway Fabric control plane (default: `nginx-gateway`), unless prefixed with `/`. | | _plm-storage-client-ssl-secret_ | _string_ | The name of the Secret containing the client certificate and key (`tls.crt`/`tls.key`) for mutual TLS with PLM storage. Must exist in the same namespace as the NGINX Gateway Fabric control plane (default: `nginx-gateway`), unless prefixed with `/`. | | _plm-storage-skip-verify_ | _bool_ | Disable TLS certificate verification for PLM storage connections. Use for testing only (Default: `false`). | | _usage-report-secret_ | _string_ | The name of the Secret containing the JWT for NGINX Plus usage reporting. Must exist in the same namespace that the NGINX Gateway Fabric control plane is running in (default namespace: nginx-gateway) | | _usage-report-endpoint_ | _string_ | The endpoint of the NGINX Plus usage reporting server. | | _usage-report-resolver_ | _string_ | The nameserver used to resolve the NGINX Plus usage reporting endpoint. Used with NGINX Instance Manager. | | _usage-report-skip-verify_ | _bool_ | Disable client verification of the NGINX Plus usage reporting server certificate. | | _usage-report-ca-secret_ | _string_ | The name of the Secret containing the NGINX Instance Manager CA certificate. Must exist in the same namespace that the NGINX Gateway Fabric control plane is running in (default namespace: nginx-gateway) | | _usage-report-client-ssl-secret_ | _string_ | The name of the Secret containing the client certificate and key for authenticating with NGINX Instance Manager. Must exist in the same namespace that the NGINX Gateway Fabric control plane is running in (default namespace: nginx-gateway) | | _usage-report-enforce-initial-report_ | _bool_ | Enables or disables the 180-day grace period for sending the initial usage report. | | _snippets_ | _bool_ | Enable Snippets feature through SnippetsFilter and SnippetsPolicy APIs. SnippetsFilters allow inserting NGINX configuration into the generated NGINX config for HTTPRoute and GRPCRoute resources. SnippetsPolicies allow inserting NGINX configuration into the generated NGINX config for Gateway resources. | | _nginx-scc_ | _string_ | The name of the SecurityContextConstraints to be used with the NGINX data plane Pods. Only applicable in OpenShift. | | _nginx-one-dataplane-key-secret_ | _string_ | The name of the secret which holds the dataplane key that is required to authenticate with the NGINX One Console. Secret must exist in the same namespace that the NGINX Gateway Fabric control plane is running in (default namespace: nginx-gateway). | | _nginx-one-telemetry-endpoint-host_ | _string_ | The endpoint host that the NGINX One Console telemetry metrics will be sent to. | | _nginx-one-telemetry-endpoint-port_ | _int_ | The endpoint port that the NGINX One Console telemetry metrics will be sent to. | | _nginx-one-tls-skip-verify_ | _bool_ | Skip TLS verification for NGINX One Console connections. | | _nim-telemetry-endpoint-host_ | _string_ | The endpoint host that the NGINX Instance Manager telemetry metrics will be sent to. | | _nim-telemetry-endpoint-port_ | _int_ | The endpoint port that the NGINX Instance Manager telemetry metrics will be sent to. | | _gateway-api-inference-extension_ | _bool_ | Enable Gateway API Inference Extension support. Allows for configuring InferencePools to route traffic to AI workloads. (Default: `false`) | | _external-load-balancer_ | _bool_ | Enable ExternalLoadBalancer support. Allows for fronting a Gateway with an external load balancer. Supported load balancers: F5 BIG-IP, through F5 Container Ingress Services. (Default: `false`) | | _agent-tls-secret_ | _string_ | The name of the base Secret containing TLS CA, certificate, and key for the NGINX Agent to securely communicate with the NGINX Gateway Fabric control plane. Must exist in the same namespace that the NGINX Gateway Fabric control plane is running in (default namespace: nginx-gateway). (Default `agent-tls`) | | _endpoint-picker-disable-tls_ | _bool_ | Disables TLS when connecting to the EndpointPicker. Set to true only for development/testing or when using a service mesh for encryption. (Default: `false`) | | _endpoint-picker-tls-skip-verify_ | _bool_ | Disables server certificate verification when connecting to the EndpointPicker, if TLS is enabled. REQUIRED: Must be true until Gateway API Inference Extension EndpointPicker supports mounting certificates. (Default `true`) | | _watch-namespaces_ | _list_ | Comma-separated list of namespaces to watch for resources. If not set, all namespaces are watched. The controller's own namespace is always included. | | _payload-processor_ | _bool_ | Enable the PayloadProcessor API. PayloadProcessors enable declarative, ordered processing of HTTP request and response payloads by attaching to a Gateway or HTTPRoute, and are used to implement features such as Guardrails for AI workloads. | | _cluster-domain_ | _string_ | The DNS domain of your Kubernetes cluster. | ## Sleep This command sleeps for specified duration, then exits. _Usage_: ```shell gateway sleep [flags] ``` | Name | Type | Description | | -------- | --------------- | ----------------------------------------------------------------------------------------------------------------------------- | | duration | `time.Duration` | Set the duration of sleep. Must be parsable by [`time.ParseDuration`](https://pkg.go.dev/time#ParseDuration). (default `30s`) | -------------------------------------------------------------------------------- title: "Configure tracing" url: https://docs.nginx.com/nginx-gateway-fabric/monitoring/tracing/index.md -------------------------------------------------------------------------------- This guide explains how to enable tracing on HTTPRoutes in NGINX Gateway Fabric using the OpenTelemetry Collector. Jaeger is used to process and collect the traces. ## Overview NGINX Gateway Fabric supports tracing using [OpenTelemetry](https://opentelemetry.io/). The official [NGINX OpenTelemetry Module](https://github.com/nginxinc/nginx-otel) instruments the NGINX data plane to export traces to a configured collector. Tracing data can be used with an OpenTelemetry Protocol (OTLP) exporter, such as the [OpenTelemetry Collector](https://github.com/open-telemetry/opentelemetry-collector). This collector can then export data to one or more upstream collectors like [Jaeger](https://www.jaegertracing.io/), [DataDog](https://docs.datadoghq.com/tracing/), and many others. This is called the [Agent model](https://opentelemetry.io/docs/collector/deployment/agent/). ## Install the collectors The first step is to install the collectors. NGINX Gateway Fabric will be configured to export to the OpenTelemetry Collector, which is configured to export to Jaeger. This model allows the visualization collector (Jaeger) to be swapped with something else, or to add more collectors without needing to reconfigure NGINX Gateway Fabric. It is also possible to configure NGINX Gateway Fabric to export directly to Jaeger. Create the namespace: ```shell kubectl create namespace tracing ``` Download the following files containing the configurations for the collectors: - [Download: otel-collector.yaml](/ngf/otel-collector.yaml) - [Download: jaeger.yaml](/ngf/jaeger.yaml) **Note:** These collectors are for demonstration purposes and are not tuned for production use. Then install them: ```shell kubectl apply -f otel-collector.yaml -f jaeger.yaml -n tracing ``` Ensure the Pods are running: ```shell kubectl -n tracing get pods ``` ```text NAME READY STATUS RESTARTS AGE jaeger-8469f69b86-bfpk9 1/1 Running 0 9s otel-collector-f786b7dfd-h2x9l 1/1 Running 0 9s ``` Once running, you can access the Jaeger dashboard by using port-forwarding in the background: ```shell kubectl port-forward -n tracing svc/jaeger 16686:16686 & ``` Visit [http://127.0.0.1:16686](http://127.0.0.1:16686) to view the dashboard. ## Enable tracing To enable tracing, you must configure two resources: - `NginxProxy`: This resource contains global settings relating to the NGINX data plane. It is created and managed by the [cluster operator](https://gateway-api.sigs.k8s.io/docs/concepts/roles-and-personas/), and is referenced in the `parametersRef` field of the GatewayClass. By default, an `NginxProxy` resource is created in the same namespace where NGINX Gateway Fabric is installed, attached to the GatewayClass. You can set configuration options in the `nginx` Helm value section, and the resource will be created and attached using the set values. When installed using the Helm chart, the NginxProxy resource is named `-proxy-config` and is created in the release Namespace. The `NginxProxy` resource contains configuration for the collector, and applies to all Gateways and routes under the GatewayClass. It does not enable tracing, but is a prerequisite to the next piece of configuration. **Note:** You can also override the tracing configuration for a particular Gateway by manually creating and attaching specific `NginxProxy` resources to target the different Gateways. This guide covers the global tracing configuration only. - `ObservabilityPolicy`: This resource is a [Direct PolicyAttachment](https://gateway-api.sigs.k8s.io/reference/policy-attachment/) that targets HTTPRoutes or GRPCRoutes. It is created by the [application developer](https://gateway-api.sigs.k8s.io/docs/concepts/roles-and-personas/) and enables tracing for a specific route or routes. It requires the `NginxProxy` resource to exist in order to complete the tracing configuration. For all the possible configuration options for these resources, see the [API reference](/ngf/reference/api.md). ### Install NGINX Gateway Fabric with global tracing configuration **Note:** Ensure that you [install the Gateway API resources](/ngf/install/helm.md#installing-the-gateway-api-resources). Referencing the previously deployed collector, create the following `values.yaml` file for installing NGINX Gateway Fabric: ```yaml cat < values.yaml nginx: config: telemetry: exporter: endpoint: otel-collector.tracing.svc:4317 spanAttributes: - key: cluster-attribute-key value: cluster-attribute-value EOT ``` The span attribute will be added to all tracing spans. To install: ```shell helm install ngf oci://ghcr.io/nginx/charts/nginx-gateway-fabric --create-namespace -n nginx-gateway -f values.yaml ``` You should see the following configuration: ```shell kubectl get nginxproxies.gateway.nginx.org ngf-proxy-config -n nginx-gateway -o yaml ``` ```yaml apiVersion: gateway.nginx.org/v1alpha2 kind: NginxProxy metadata: name: ngf-proxy-config spec: telemetry: exporter: endpoint: otel-collector.tracing.svc:4317 spanAttributes: - key: cluster-attribute-key value: cluster-attribute-value ``` ```shell kubectl get gatewayclasses.gateway.networking.k8s.io nginx -o yaml ``` ```yaml apiVersion: gateway.networking.k8s.io/v1 kind: GatewayClass metadata: name: nginx spec: controllerName: gateway.nginx.org/nginx-gateway-controller parametersRef: group: gateway.nginx.org kind: NginxProxy name: ngf-proxy-config status: conditions: - lastTransitionTime: "2024-05-22T15:18:35Z" message: GatewayClass is accepted observedGeneration: 1 reason: Accepted status: "True" type: Accepted - lastTransitionTime: "2024-05-22T15:18:35Z" message: Gateway API CRD versions are supported observedGeneration: 1 reason: SupportedVersion status: "True" type: SupportedVersion - lastTransitionTime: "2024-05-22T15:18:35Z" message: parametersRef resource is resolved observedGeneration: 1 reason: ResolvedRefs status: "True" type: ResolvedRefs ``` If you already have NGINX Gateway Fabric installed, then you can modify the `NginxProxy` resource to include the tracing configuration: ```shell kubectl edit nginxproxies.gateway.nginx.org ngf-proxy-config -n nginx-gateway ``` You can now create the application, route, and tracing policy. ### Create the application and route Create the basic **coffee** application: ```yaml kubectl apply -f - < ``` Check that traffic can flow to the application. **Note:** If you have a DNS record allocated for `cafe.example.com`, you can send the request directly to that hostname, without needing to resolve. ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee ``` You should receive a response from the coffee Pod. ```text Server address: 10.244.0.69:8080 Server name: coffee-6b8b6d6486-k5w5w URI: /coffee ``` You shouldn't see any information from the [Jaeger dashboard](http://127.0.0.1:16686) yet: you need to create the `ObservabilityPolicy`. ### Create the ObservabilityPolicy To enable tracing for the coffee HTTPRoute, create the following policy: ```yaml kubectl apply -f - <:`. ![image](/ngf/img/jaeger-trace-overview.png)

Select a trace to view the attributes. ![image](/ngf/img/jaeger-trace-attributes.png) The trace includes the attribute from the global NginxProxy resource as well as the attribute from the ObservabilityPolicy. ## See also - [Data plane configuration](/ngf/how-to/data-plane-configuration.md): learn how to dynamically update the NGINX Gateway Fabric global data plane configuration, including how to override the telemetry configuration for a particular Gateway. - [Custom policies](/ngf/overview/custom-policies.md): learn about how NGINX Gateway Fabric custom policies work. - [API reference](/ngf/reference/api.md): all configuration fields for the policies mentioned in this guide -------------------------------------------------------------------------------- title: "F5 WAF for NGINX overview" url: https://docs.nginx.com/nginx-gateway-fabric/waf-integration/overview/index.md -------------------------------------------------------------------------------- F5 NGINX Gateway Fabric integrates with F5 WAF for NGINX to provide web application firewall protection. WAF policies are compiled externally and deployed to the data plane using the `WAFPolicy` custom resource. **Note:** F5 WAF for NGINX requires NGINX Plus and a separate F5 WAF for NGINX subscription. Contact your F5 sales representative for licensing details. --- ## Architecture F5 WAF for NGINX uses a multi-container architecture. When WAF is enabled, each NGINX Pod is extended with two sidecar containers: - **waf-enforcer**: Enforces WAF policies on incoming traffic. - **waf-config-mgr**: Manages WAF configuration and distributes policy bundles to the enforcer. Shared ephemeral volumes connect these containers to the main NGINX container. ```mermaid graph LR CP[NGINX Gateway Fabric] -->|gRPC: config + policy bundle| Agent subgraph NginxPod["NGINX Pod"] Agent[NGINX Agent] -->|writes bundle| Vol[(Shared Volume)] NGINX[NGINX + WAF Module] --- Vol Enforcer[WAF Enforcer] --- Vol ConfigMgr[WAF Config Mgr] --- Vol end Client[Client] ==> NGINX ==> Backend[Backend Service] ``` --- ## Enable WAF on the NginxProxy WAF is enabled by setting `waf.enable: true` on an `NginxProxy` resource. This instructs NGINX Gateway Fabric to deploy the WAF sidecar containers alongside the NGINX Pod. You can enable WAF at two levels: - **All Gateways** -- Set WAF on the GatewayClass-level `NginxProxy` so that every Gateway managed by this NGINX Gateway Fabric instance gets WAF sidecars by default. A per-Gateway `NginxProxy` can override this (for example, to disable WAF on a specific Gateway). - **Per Gateway** -- Create an `NginxProxy` and reference it from the `spec.infrastructure.parametersRef` field on a Gateway. Only that Gateway gets WAF sidecars. For details on how GatewayClass and Gateway-level NginxProxy settings are merged, see [Data plane configuration](/ngf/how-to/data-plane-configuration.md). ### Enable WAF for all Gateways To enable WAF at install time, use the **NGINX Plus with WAF** tab in the [Helm install guide](/ngf/install/helm.md). This sets the WAF-enabled NGINX Plus image (`nginx-plus-f5waf`) and enables WAF on the GatewayClass-level `NginxProxy`, so every Gateway gets WAF sidecars by default. To disable WAF for a specific Gateway, create a per-Gateway `NginxProxy` with `waf.enable: false` and reference it from that Gateway. **Note:** For additional WAF-related NginxProxy settings (including `disableCookieSeed`, `bundleFailOpen`, and custom WAF container images), see [Configure WAF settings](/ngf/waf-integration/configuration.md). ### Enable WAF per Gateway If you installed with the standard NGINX Plus image and want WAF on a specific Gateway only, create a per-Gateway `NginxProxy`. You must also set the NGINX image to `nginx-plus-f5waf`, because the standard `nginx-plus` image inherited from the GatewayClass doesn't include the WAF module: ```yaml apiVersion: gateway.nginx.org/v1alpha2 kind: NginxProxy metadata: name: waf-enabled-proxy spec: waf: enable: true kubernetes: deployment: container: image: repository: private-registry.nginx.com/nginx-gateway-fabric/nginx-plus-f5waf ``` ```yaml apiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: name: gateway spec: gatewayClassName: nginx infrastructure: parametersRef: name: waf-enabled-proxy group: gateway.nginx.org kind: NginxProxy listeners: - name: http port: 80 protocol: HTTP ``` For the full list of available images, see [Supported container images](/ngf/overview/technical-specifications.md#supported-container-images). --- ## Policy lifecycle ### Bundles A WAF bundle is a compiled policy package produced by the [F5 WAF for NGINX compiler](/waf/configure/compiler.md). It contains the security policy, optional logging profile, [attack signatures](/waf/policies/attack-signatures.md), [threat campaign](/waf/policies/threat-campaigns.md) data, [bot signatures](/waf/policies/bot-signatures.md), and related metadata. The format lets the WAF engine load and enforce the policy at runtime. Pre-compiling policies into bundles results in faster, more reliable WAF startup: policies are resolved and validated at build time rather than on the running data plane. ### Compilation WAF policies must be compiled before they can be applied. Compilation takes a JSON policy definition (and optionally [global settings](/waf/configure/compiler.md) such as a cookie seed and [user-defined signatures](/waf/policies/user-signatures.md)) and produces a `.tgz` bundle. NGINX Gateway Fabric doesn't compile policies. Its role begins with fetching a compiled bundle and deploying it to the data plane. ### Source types Set the source type using the `spec.type` field on the `WAFPolicy` resource: | Type | Description | |--------|---------------------------------------------------------------------------------------------------| | `NIM` | NGINX Instance Manager -- fetched by policy name or UID using the NGINX Instance Manager API | | `N1C` | NGINX One Console -- fetched by policy name or object ID using the NGINX One Console API | | `HTTP` | Direct HTTP/HTTPS URL to a compiled bundle file | | `PLM` | Policy Lifecycle Management -- `APPolicy`/`APLogConf` CRDs, fetched from in-cluster storage | The `NIM`, `N1C`, and `HTTP` source types reference an externally compiled bundle through `policySource` (and `logSource` for log profiles). They detect updates by polling. The `PLM` source type is Kubernetes-native and event-driven: it references `APPolicy` and `APLogConf` custom resources through `policyRef` (and `logRef`), and doesn't require polling. See [PLM (Policy Lifecycle Management)](#plm-policy-lifecycle-management) below. For details on configuring each source type, see [Configure policy sources](/ngf/waf-integration/policy-sources.md). ### PLM (Policy Lifecycle Management) Policy Lifecycle Management (PLM) is a Kubernetes-native policy source. Instead of pointing NGINX Gateway Fabric at an externally compiled bundle, you define your WAF security posture as `APPolicy` and `APLogConf` custom resources in the cluster. The PLM controller watches these resources, compiles them automatically, and stores the resulting bundles in in-cluster S3-compatible storage. NGINX Gateway Fabric fetches the bundles from that storage and deploys them to the data plane. The following table summarizes how PLM differs from the HTTP, NGINX Instance Manager, and NGINX One Console source types: | Aspect | HTTP / NIM / N1C | PLM | |--------------------|-------------------------------------------------|---------------------------------------------------------| | Policy definition | Authored externally (file/Git, NIM, or N1C) | Authored in-cluster as `APPolicy`/`APLogConf` CRDs | | Compilation | External (compiler CLI/CI-CD, NIM, or N1C) | Automatic, by the PLM controller | | Bundle storage | HTTP server, NIM, or N1C | In-cluster S3-compatible storage | | `WAFPolicy` fields | `policySource` / `logSource` | `policyRef.apPolicyRef` / `logRef.apLogConfRef` | | Update detection | Polling (checksum or conditional GET) | Event-driven Kubernetes watch (no polling) | | Authentication | Per-`WAFPolicy` credentials Secret | Cluster-wide PLM storage credentials, set at install | | Network access | External egress to the policy source | Fully in-cluster | At runtime, the flow is: ```text Create APPolicy/APLogConf → PLM compiles and sets status.bundle.state: ready → NGINX Gateway Fabric detects the ready status via watch → fetches the bundle from in-cluster storage → deploys to the data plane ``` Changes to an `APPolicy` or `APLogConf` spec trigger recompilation and an automatic re-fetch. No polling is required, and no change to the `WAFPolicy` resource is needed. When a `WAFPolicy` references an `APPolicy` or `APLogConf` in a different namespace, create a [ReferenceGrant](https://gateway-api.sigs.k8s.io/reference/api-types/referencegrant/) in the target namespace to permit the reference. **note:** PLM requires the PLM system to be installed in the cluster and PLM storage to be configured on NGINX Gateway Fabric at install time. For a complete walkthrough, see [Get started with F5 WAF for NGINX using PLM](/ngf/waf-integration/get-started-plm.md). --- ## Policy attachment `WAFPolicy` uses **inherited policy attachment**, following the [Gateway API policy attachment model](https://gateway-api.sigs.k8s.io/reference/policy-attachment/): - A **Gateway-level** `WAFPolicy` protects all HTTPRoutes and GRPCRoutes attached to that Gateway automatically. New routes inherit protection without any additional configuration. - A **Route-level** `WAFPolicy` can be applied to a specific HTTPRoute or GRPCRoute to override the Gateway-level policy for that route. - More specific (route-level) policies take precedence over less specific (gateway-level) policies. The route-level policy completely replaces the gateway-level policy for that route. There's no merging. - Only one `WAFPolicy` may target a given resource at a given level. If two policies target the same Gateway or Route, the second is rejected with `Accepted=False` and reason `Conflicted`. ```text Gateway-level WAFPolicy → HTTPRoute (inherited automatically) Route-level WAFPolicy → Overrides Gateway-level for that route only ``` **Note:** GRPCRoutes are protected in the same way as HTTPRoutes. To target a GRPCRoute, set `kind: GRPCRoute` in the `targetRefs` field. Built-in gRPC log profiles (`log_grpc_all`, `log_grpc_blocked`, `log_grpc_illegal`) are available for gRPC-specific security logging. --- ## See also - [Get started with F5 WAF for NGINX](/ngf/waf-integration/get-started-http.md) - [Get started with F5 WAF for NGINX using PLM](/ngf/waf-integration/get-started-plm.md) - [Configure policy sources (NGINX Instance Manager, NGINX One Console, and HTTP)](/ngf/waf-integration/policy-sources.md) - [Configure WAF settings](/ngf/waf-integration/configuration.md) - [WAFPolicy and NginxProxy API reference](/ngf/reference/api.md) - [F5 WAF for NGINX documentation](/waf/) -------------------------------------------------------------------------------- title: "Gateway architecture" url: https://docs.nginx.com/nginx-gateway-fabric/overview/gateway-architecture/index.md -------------------------------------------------------------------------------- Learn about the architecture and design principles of NGINX Gateway Fabric: a Kubernetes Gateway API implementation which uses NGINX as the data plane. This document is intended for: - _Cluster Operators_ who want to understand how NGINX Gateway Fabric works in production, how it manages traffic, and how to troubleshoot failures. - _Application Developers_ who would like to use NGINX Gateway Fabric to expose and route traffic to their applications within Kubernetes. The reader needs to be familiar with core Kubernetes concepts, such as pods, deployments, services, and endpoints. For an understanding of how NGINX itself works, you can read the ["Inside NGINX: How We Designed for Performance & Scale"](https://blog.nginx.org/blog/inside-nginx-how-we-designed-for-performance-scale) blog post. If you are interested in contributing to the project or learning about its internal implementation details, please see the [Developer Architecture Guide](https://github.com/nginx/nginx-gateway-fabric/tree/main/docs/architecture). --- ## NGINX Gateway Fabric Deployment Model and Architectural Overview NGINX Gateway Fabric splits its architecture into two main parts to provide better security, flexibility, and reliability: ### Control Plane: Centralized Management The control plane operates as a Deployment, serving as a [Kubernetes controller](https://kubernetes.io/docs/concepts/architecture/controller/) built with the [controller-runtime](https://github.com/kubernetes-sigs/controller-runtime) library. It manages all aspects of resource provisioning and configuration for the NGINX data planes by watching Gateway API resources and other Kubernetes objects such as Services, Endpoints, and Secrets. Key functionalities include: - Dynamic provisioning: When a new Gateway resource is created, the control plane automatically provisions a dedicated NGINX Deployment and exposes it using a Service. - Configuration management: Kubernetes and Gateway API resources are translated into NGINX configurations, which are securely delivered to the data plane pods via a gRPC connection to the NGINX Agent. - Secure communication: By default, the gRPC connection uses self-signed certificates generated during installation. Integration with [cert-manager](https://cert-manager.io/) is also supported for optional certificate management. ### Data Plane: Autonomous Traffic Management Each NGINX data plane pod can be provisioned as an independent Deployment or DaemonSet containing an `nginx` container. This container runs both the `nginx` process and the [NGINX agent](https://github.com/nginx/agent), which is responsible for: - Applying configurations: The agent receives updates from the control plane and applies them to the NGINX instance. - Handling reloads: NGINX Agent handles configuration reconciliation and reloading NGINX, eliminating the need for shared volumes or Unix signals between the control plane and data plane pods. ### Gateway Resource Management Users can have multiple gateways running side-by-side in the same cluster. This supports flexible operation and isolation across Gateways: - Concurrent Gateways: Multiple Gateway objects can run simultaneously within a single installation. - 1:1 resource mapping: Each Gateway resource corresponds uniquely to a dedicated data plane deployment, ensuring clear delineation of ownership and operational segregation. --- ## High-level overview of NGINX Gateway Fabric in execution This figure depicts an example of NGINX Gateway Fabric exposing three web applications within a Kubernetes cluster to clients on the internet: ```mermaid {grid="wide"} graph LR %% Nodes and Relationships subgraph KubernetesCluster["Kubernetes Cluster"] subgraph NamespaceNGF["Namespace: nginx-gateway"] NGFControlPlanePod["NGINX Gateway Fabric Control Plane Pod"] NGFControlPlanePod --> KubernetesAPI["Kubernetes API"] end subgraph ApplicationsNamespaceA["Namespace: applications"] subgraph DataplaneComponentsA["Dataplane Components"] GatewayA["Gateway A
Listener: *.example.com"] subgraph NGINXDataPlanePodA["NGINX Data Plane Pod"] subgraph NGINXContainerA["NGINX Container"] NGINXProcessA["NGINX Process"] NGINXAgentA["NGINX Agent"] end end end subgraph HTTPRouteAAndApplications["HTTPRoutes and Applications"] HTTPRouteA["HTTPRoute A
Host: a.example.com"] HTTPRouteB["HTTPRoute B
Host: b.example.com"] ApplicationA["Application A
Pods: 2"] ApplicationB["Application B
Pods: 1"] end end subgraph ApplicationsNamespaceB["Namespace: applications-2"] subgraph DataplaneComponentsB["Dataplane Components"] GatewayB["Gateway B
Listener: *.other-example.com"] subgraph NGINXDataPlanePodB["NGINX Data Plane Pod"] subgraph NGINXContainerB["NGINX Container"] NGINXProcessB["NGINX Process"] NGINXAgentB["NGINX Agent"] end end end subgraph HTTPRouteBandApplications["HTTPRoutes and Applications"] HTTPRouteC["HTTPRoute C
Host: c.other-example.com"] ApplicationC["Application C
Pods: 1"] end end end subgraph UsersAndClients["Users and Clients"] UserOperator["Cluster Operator"] UserDevA["Application Developer A"] UserDevB["Application Developer B"] ClientA["Client A"] ClientB["Client B"] end subgraph SharedInfrastructure["Public Endpoint"] PublicEndpoint["TCP Load Balancer / NodePort"] end %% Updated Traffic Flow ClientA-->|a.example.com|PublicEndpoint ClientB-->|c.other-example.com|PublicEndpoint PublicEndpoint==>NGINXProcessA PublicEndpoint==>NGINXProcessB NGINXProcessA==>ApplicationA NGINXProcessA==>ApplicationB NGINXProcessB==>ApplicationC %% Kubernetes Configuration Flow HTTPRouteA-->GatewayA HTTPRouteB-->GatewayA HTTPRouteC-->GatewayB UserOperator-->KubernetesAPI NGFControlPlanePod--gRPC-->NGINXAgentA NGFControlPlanePod--gRPC-->NGINXAgentB NGINXAgentA-->NGINXProcessA NGINXAgentB-->NGINXProcessB UserDevA-->KubernetesAPI UserDevB-->KubernetesAPI %% Styling style UserOperator fill:#66CDAA,stroke:#333,stroke-width:2px style GatewayA fill:#66CDAA,stroke:#333,stroke-width:2px style GatewayB fill:#66CDAA,stroke:#333,stroke-width:2px style NGFControlPlanePod fill:#66CDAA,stroke:#333,stroke-width:2px style NGINXProcessA fill:#66CDAA,stroke:#333,stroke-width:2px style NGINXProcessB fill:#66CDAA,stroke:#333,stroke-width:2px style KubernetesAPI fill:#9370DB,stroke:#333,stroke-width:2px style HTTPRouteAAndApplications fill:#E0FFFF,stroke:#333,stroke-width:2px style HTTPRouteBandApplications fill:#E0FFFF,stroke:#333,stroke-width:2px style UserDevA fill:#FFA07A,stroke:#333,stroke-width:2px style HTTPRouteA fill:#FFA07A,stroke:#333,stroke-width:2px style HTTPRouteB fill:#FFA07A,stroke:#333,stroke-width:2px style ApplicationA fill:#FFA07A,stroke:#333,stroke-width:2px style ApplicationB fill:#FFA07A,stroke:#333,stroke-width:2px style ClientA fill:#FFA07A,stroke:#333,stroke-width:2px style UserDevB fill:#87CEEB,stroke:#333,stroke-width:2px style HTTPRouteC fill:#87CEEB,stroke:#333,stroke-width:2px style ApplicationC fill:#87CEEB,stroke:#333,stroke-width:2px style ClientB fill:#87CEEB,stroke:#333,stroke-width:2px style PublicEndpoint fill:#FFD700,stroke:#333,stroke-width:2px ``` **Note:** The figure does not show many of the necessary Kubernetes resources the Cluster Operators and Application Developers need to create, like deployment and services. The figure shows: | **Category** | **Description** | |-------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------| | **Namespaces** | - _Namespace: nginx-gateway_: Contains the NGINX Gateway Fabric Control Plane Pod, responsible for managing Gateway API configurations and provisioning NGINX Data Plane Pods.
- _Namespace: applications_: Contains Gateway A for `*.example.com`, handling Application A and Application B.
- _Namespace: applications-2_: Contains Gateway B for `*.other-example.com`, handling Application C. | | **Users** | - _Cluster Operator_: Sets up the NGINX Gateway Fabric Control Plane Pod and manages Gateway API resources by provisioning Gateways (A and B).
- _Developers A & B_: Developers deploy their applications and create HTTPRoutes associated with their Gateways. | | **Clients** | - _Client A_: Interacts with Application A through `a.example.com`.
- _Client B_: Interacts with Application C through `c.other-example.com`. | | **NGINX Gateway Fabric Control Plane Pod** | The control plane pod, deployed in the `nginx-gateway` namespace, communicates with the Kubernetes API to:
- Fetch Gateway API resources.
- Dynamically provision and configure NGINX Data Plane Pods.
- Deliver traffic routing and configuration updates to NGINX Agent instances over gRPC. | | **Gateways** | - _Gateway A_: Listens for requests under `*.example.com`. Routes:
   • _HTTPRoute A_: Routes requests to `a.example.com` into Application A.
   • _HTTPRoute B_: Routes requests to `b.example.com` into Application B.
- _Gateway B_: Listens for requests under `*.other-example.com`. Routes:
   • _HTTPRoute C_: Routes requests to `c.other-example.com` into Application C. | | **Applications** | - _Application A_: Deployed by Developer A (2 pods), routed by Gateway A via HTTPRoute A.
- _Application B_: Deployed by Developer A (1 pod), routed by Gateway A via HTTPRoute B.
- _Application C_: Deployed by Developer B (1 pod), routed by Gateway B via HTTPRoute C. | | **NGINX Data Plane Pods** | - _NGINX Data Plane Pod A_: Handles traffic routed from Gateway A:
   • _NGINX Process A_: Forwards requests to Application A and Application B as defined in Gateway A's HTTPRoutes.
   • _NGINX Agent A_: Receives configuration updates from the NGINX Gateway Fabric Control Plane Pod via gRPC.
- _NGINX Data Plane Pod B_: Manages traffic routed from Gateway B:
   • _NGINX Process B_: Forwards requests to Application C as defined in Gateway B’s HTTPRoute.
   • _NGINX Agent B_: Receives configuration updates via gRPC from the NGINX Gateway Fabric Control Plane Pod. | | **Traffic Flow** | - _Client A_:
   1. Sends requests to `a.example.com` via the Public Endpoint.
   2. Requests are routed by Gateway A and processed by NGINX Process A.
   3. Traffic is forwarded to Application A.
- _Client B_:
   1. Sends requests to `c.other-example.com` via the Public Endpoint.
   2. Requests are routed by Gateway B and processed by NGINX Process B.
   3. Traffic is forwarded to Application C. | | **Public Endpoint** | A shared entry point (TCP Load Balancer or NodePort) that exposes the NGINX Data Plane externally to forward client traffic into the cluster. | | **Kubernetes API** | Acts as the central hub for resource management:
- Fetches Gateway API resources for Gateway A and Gateway B.
- Facilitates NGINX configuration updates via the NGINX Gateway Fabric Control Plane Pod. | _Color Coding_ : - Cluster Operator resources (e.g., NGINX Gateway Fabric, NGINX Pods and Gateways) are marked in _green_. - Resources owned by _Application Developer A_ (e.g., HTTPRoute A, Application A) are marked in _orange_. - Resources owned by _Application Developer B_ (e.g., HTTPRoute B, Application C) are marked in _blue_. --- ## NGINX Gateway Fabric: Component Communication Workflow ```mermaid graph LR %% Main Components subgraph nginx-gateway["Namespace: nginx-gateway"] NGFPod[NGINX Gateway Fabric Control Plane Pod] end subgraph NGINXDataPlane["NGINX Data Plane Pod"] NGINXAgent[NGINX Agent] NGINXMaster[NGINX Master] NGINXWorker[NGINX Worker] ConfigFiles[Config Files] end F5Telemetry[F5 Telemetry Service] PrometheusMonitor[Prometheus] KubernetesAPI[Kubernetes API] Client[Client] BackendApplication[Backend Application] %% High-Level Configuration Flow KubernetesAPI -->|"(1) Updates Resources"| NGFPod NGFPod -->|"(2) Sends Configuration Metadata via gRPC"| NGINXAgent NGINXAgent -->|"(3) Validates & Writes Configuration"| ConfigFiles NGINXAgent -->|"(4) Signals NGINX Master to Reload"| NGINXMaster %% Prometheus Monitoring PrometheusMonitor -->|"(5) Fetches Metrics from NGINX"| NGINXWorker %% Telemetry Data NGFPod -->|"(6) Sends Telemetry Data"| F5Telemetry %% Client Traffic Flow Client -->|"(7) Sends Traffic"| NGINXWorker NGINXWorker -->|"(8) Routes Traffic"| BackendApplication %% Styling classDef important fill:#66CDAA,stroke:#333,stroke-width:2px; classDef metrics fill:#FFC0CB,stroke:#333,stroke-width:2px; classDef io fill:#FFD700,stroke:#333,stroke-width:2px; class KubernetesAPI,NGFPod important; class PrometheusMonitor,F5Telemetry metrics; class NGINXAgent,NGINXMaster,NGINXWorker,ConfigFiles io; class Client,BackendApplication important; ``` The following table describes the connections, preceeded by their types in parentheses. For brevity, the suffix "process" has been omitted from the process descriptions. | # | Component/Protocol | Description | | ---| ----------------------- | ------------------------------------------------------------------------------------------------------------ | | 1 | Kubernetes API (HTTPS) | _Kubernetes API → NGINX Gateway Fabric Control Plane Pod_: The NGINX Gateway Fabric Control Plane Pod (in the `nginx-gateway` namespace) watches the Kubernetes API for updates to Gateway API resources (e.g., Gateways, HTTPRoutes), fetching the latest configuration to manage routing and traffic control. | | 2 | gRPC | _NGINX Gateway Fabric Control Plane Pod → NGINX Agent_: The NGINX Gateway Fabric Control Plane Pod processes Gateway API resources, generates NGINX configuration settings, and securely delivers them to the NGINX Agent inside the NGINX Data Plane Pod via gRPC. | | 3 | File I/O | _NGINX Agent → Config Files_: The NGINX Agent (within the NGINX Data Plane Pod) validates the configuration metadata received from the Control Plane Pod and writes it to NGINX configuration files within the pod. These files store dynamic routing rules and traffic settings. | | 4 | Signal | _NGINX Agent → NGINX Master_: After writing the configuration files, the NGINX Agent signals the NGINX Master process to reload the configuration. This ensures the NGINX Data Plane Pod immediately applies the updated routes and settings. | | 5 | HTTP/HTTPS | _Prometheus → NGINX Worker_: Prometheus collects runtime metrics (e.g., traffic statistics, request rates, and active connections) from the NGINX Worker process, which is part of the NGINX Data Plane Pod. The `/metrics` endpoint exposes these metrics for monitoring and observability. | | 6 | HTTPS | _NGINX Gateway Fabric Control Plane Pod → F5 Telemetry Service_: The NGINX Gateway Fabric Control Plane Pod sends telemetry data (e.g., API requests handled, usage metrics, performance stats, error rates) to the external F5 Telemetry Service for centralized monitoring and diagnostics. | | 7 | HTTP/HTTPS | _Client → NGINX Worker_: Clients send incoming application traffic (e.g., HTTP/HTTPS requests) to the NGINX Worker process within the NGINX Data Plane Pod. These requests are typically routed through a shared Public Endpoint (e.g., LoadBalancer or NodePort) before reaching the NGINX Data Plane. | | 8 | HTTP/HTTPS | _NGINX Worker → Backend Application_: The NGINX Worker process forwards client traffic to the appropriate backend application services (e.g., Pods) as defined in the routing rules and configuration received from the Control Plane Pod. | --- ### Additional features and enhancements when using NGINX Plus NGINX Gateway Fabric supports both NGINX Open Source and NGINX Plus. While the previous diagram shows NGINX Open Source, using NGINX Plus provides additional capabilities, including: - The ability for administrators to connect to the NGINX Plus API on port 8765 (restricted to localhost by default). - Dynamic updates to upstream servers without requiring a full reload: - Changes to upstream servers, such as application scaling (e.g., adding or removing pods in Kubernetes), can be applied using the [NGINX Plus API](http://nginx.org/en/docs/http/ngx_http_api_module.html). - This reduces the frequency of configuration reloads, minimizing potential disruptions and improving system stability during updates. These features enable reduced downtime, improved performance during scaling events, and more fine-grained control over traffic management. --- ### Resilience and fault isolation This architecture separates the control plane and data plane, creating clear operational boundaries that improve resilience and fault isolation: #### Control plane resilience In the event of a control plane failure or downtime: - Existing data plane pods continue serving traffic using their last-valid cached configurations. - Updates to routes or Gateways are temporarily paused, but stable traffic delivery continues without degradation. - Recovery restores functionality, resynchronizing configuration updates seamlessly. #### Data plane resilience If a data plane pod encounters an outage or restarts: - Only routes tied to the specific linked Gateway object experience brief disruptions. - Configurations automatically resynchronize with the data plane upon pod restart, minimizing the scope of impact. - Other data plane pods remain unaffected and continue serving traffic normally. --- ## Pod readiness The control plane (`nginx-gateway`) and data plane (`nginx`) containers provide a readiness endpoint at `/readyz`. A [readiness probe](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#define-readiness-probes) periodically checks this endpoint during startup. The probe reports `200 OK` when: - The control plane is ready to configure the NGINX data planes. - The data plane is ready to handle traffic. This marks the pods ready ensuring traffic is routed to healthy pods, ensuring reliable startup and smooth operations. -------------------------------------------------------------------------------- title: "Use Manifests to install NGINX Gateway Fabric with NGINX Open Source" url: https://docs.nginx.com/nginx-gateway-fabric/install/manifests/open-source/index.md -------------------------------------------------------------------------------- This page describes how to use Manifests to install NGINX Gateway Fabric with NGINX Open Source. It explains how to install the Gateway API resources and add authentication certificates, then deploy NGINX Gateway Fabric and its custom resource definitions. By following these instructions, you will finish with a functional NGINX Gateway Fabric instance for your Kubernetes cluster. **Note:** To learn which Gateway API resources NGINX Gateway Fabric currently supports, view the [Gateway API Compatibility](/ngf/overview/gateway-api-compatibility.md) topic. To install an experimental NGINX Gateway Fabric version view the [Use Manifests to install NGINX Gateway Fabric (experimental) with NGINX Open Source](/ngf/install/manifests/open-source-experimental.md) topic. ## Before you begin To complete this guide, you will need the following pre-requisites: - [A supported Kubernetes version](/ngf/overview/technical-specifications.md) - A functional Kubernetes cluster ## Install the Gateway API resources **Note:** If you have already installed Gateway API resources in your cluster, ensure they are a version [supported by NGINX Gateway Fabric](/ngf/overview/technical-specifications.md) To install the Gateway API resources, use `kubectl kustomize`: ```shell kubectl kustomize "https://github.com/nginx/nginx-gateway-fabric/config/crd/gateway-api/standard?ref=v" | kubectl apply -f - ``` #### Example output ```text customresourcedefinition.apiextensions.k8s.io/gatewayclasses.gateway.networking.k8s.io created customresourcedefinition.apiextensions.k8s.io/gateways.gateway.networking.k8s.io created customresourcedefinition.apiextensions.k8s.io/grpcroutes.gateway.networking.k8s.io created customresourcedefinition.apiextensions.k8s.io/httproutes.gateway.networking.k8s.io created customresourcedefinition.apiextensions.k8s.io/referencegrants.gateway.networking.k8s.io created ``` You should also create the _nginx-gateway_ namespace, which is used by the Manifest files by default: ```shell kubectl create namespace nginx-gateway ``` ## Install cert-manager Install cert-manager onto the cluster using Helm with Gateway API features enabled. - Add the Helm repository. ```shell helm repo add jetstack https://charts.jetstack.io helm repo update ``` - Install cert-manager, and enable the GatewayAPI feature gate: ```shell helm install \ cert-manager jetstack/cert-manager \ --namespace cert-manager \ --create-namespace \ --set config.apiVersion="controller.config.cert-manager.io/v1alpha1" \ --set config.kind="ControllerConfiguration" \ --set config.enableGatewayAPI=true \ --set crds.enabled=true ``` ## Add certificates for secure authentication **Note:** These steps use a self-signed issuer, which should not be used in production environments. For production environments, you should use a real [CA issuer](https://cert-manager.io/docs/configuration/ca/). First, create a CA (certificate authority) issuer: ```yaml kubectl apply -f - <= v1.18.0, the default value changed from `Never` to `Always`. certificate.cert-manager.io/nginx-gateway-ca created issuer.cert-manager.io/nginx-gateway-issuer created ``` You will then need to create a server certificate for the NGINX Gateway Fabric control plane (server): **Note:** The default service name is _nginx-gateway_, and the namespace is _nginx-gateway_, so the `dnsNames` value should be `nginx-gateway.nginx-gateway.svc`. This value becomes the name of the NGINX Gateway Fabric control plane service. ```yaml {hl_lines=[13]} kubectl apply -f - <coffee] pod1[Pod
coffee] pod2[Pod
coffee] end svc --> pod1 & pod2 class pod1,pod2,svc appNode classDef appNode fill:#edbd8c,stroke:#D9822B ``` Using this architecture, the **coffee** application is not accessible outside the cluster. We want to expose this application on the hostname "cafe.example.com" so that clients outside the cluster can access it. Install NGINX Gateway Fabric and create two Gateway API resources: a [gateway](https://gateway-api.sigs.k8s.io/reference/spec/#gateway) and an [HTTPRoute](https://gateway-api.sigs.k8s.io/reference/spec/#httproute). Using these resources we will configure a simple routing rule to match all HTTP traffic with the hostname "cafe.example.com" and route it to the **coffee** service. --- ## Set up Create the **coffee** application in Kubernetes by copying and pasting the following block into your terminal: ```yaml kubectl apply -f - < 80/TCP 77s ``` --- ## Application architecture with NGINX Gateway Fabric To route traffic to the **coffee** application, we will create a Gateway and HTTPRoute. The following diagram shows the configuration we are creating in the next step: ```mermaid graph LR subgraph config [Namespace default] subgraph padding [" "] direction LR style config fill:#FFFFFF,stroke:#000000 subgraph gw[Gateway cafe] subgraph gwPadding [" "] gwContents[HTTP/80] end end subgraph hr[HTTPRoute coffee] subgraph hrPadding [" "] hrContents[cafe.example.com] subgraph describeMatchAll [Match all
traffic] subgraph describeMatchPadding [" "] matchAll[Host: *
Path: *] end end subgraph describeService [Group matching
pods within a Service] subgraph describePadding [" "] coffeeSvc[Service
coffee] end end end end end end gwContents --> hrContents --> matchAll --> coffeeSvc class padding,gwPadding,hrPadding,describeMatchAll,describeService,describePadding,describeMatchPadding noBorder class gw gateway class hr httpRoute class matchAll,hrContents,coffeeSvc appDevNode class gwContents clusterOppNode classDef noBorder stroke:none,fill:none,text-align:center classDef default fill:#FFFFFF,stroke:#000000 classDef gateway fill:#FFFFFF,stroke:#blue,stroke-dasharray: 3 3,text-align:center classDef httpRoute fill:#FFFFFF,stroke:#D9822B,stroke-dasharray: 3 3,text-align:center classDef appDevNode fill:#edbd8c,stroke:#D9822B classDef clusterOppNode fill:lightblue,stroke:darkblue ``` We need a Gateway to create an entry point for HTTP traffic coming into the cluster. The **cafe** Gateway we are going to create will open an entry point to the cluster on port 80 for HTTP traffic. To route HTTP traffic from the Gateway to the **coffee** service, we need to create an HTTPRoute named **coffee** and attach it to the Gateway. This HTTPRoute will have a single routing rule that routes all traffic to the hostname "cafe.example.com" from the Gateway to the **coffee** service. Once NGINX Gateway Fabric processes the **cafe** Gateway and **coffee** HTTPRoute, it will configure a data plane (NGINX) to route all HTTP requests sent to "cafe.example.com" to the pods that the **coffee** service targets: ```mermaid graph LR style cluster fill:#FFFFFF,stroke:#000000 clients[Clients] ngfSvc["Public IP Address for
cafe.example.com"] subgraph cluster [Kubernetes Cluster] subgraph appNs [Namespace
default] subgraph nsPadding [" "] nginxPod[Pod NGINX] coffeePod1[Pod coffee] coffeePod2[Pod coffee] end end end ngfSvc --> nginxPod nginxPod --> coffeePod1 & coffeePod2 clients --> ngfSvc class clusterPadding,nsPadding,clusterPadding2 noBorder class gwNS,appNs namespace class nginxPod nginxNode class coffeePod1,coffeePod2 coffeeNode class ngfSvc svc class clients clientNode classDef noBorder stroke:none,fill:none classDef default fill:#FFFFFF,stroke:#000000 classDef namespace fill:#FFFFFF,stroke:#036ffc,stroke-dasharray: 5 5,text-align:center classDef nginxNode fill:#b4e0ad,stroke:#2AA317 classDef svc fill:lightblue,stroke:darkblue classDef coffeeNode fill:#edbd8c,stroke:#D9822B classDef clientNode fill:#D3D3D3 ``` The **coffee** service is omitted from the diagram above because the NGINX Pod routes directly to the pods that the **coffee** service targets. **Note:** In the diagrams above, all resources that are the responsibility of the cluster operator are shown in blue. The orange resources are the responsibility of the application developers. See the [roles and personas](https://gateway-api.sigs.k8s.io/docs/concepts/roles-and-personas/#roles-and-personas_1) Gateway API document for more information on these roles. --- ## Create the Gateway API resources To create the **cafe** gateway, copy and paste the following into your terminal: ```yaml kubectl apply -f - < ``` **Note:** In a production environment, you should have a DNS record for the external IP address that is exposed, and it should refer to the hostname that the gateway will forward for. This Gateway is associated with NGINX Gateway Fabric through the **gatewayClassName** field. The default installation of NGINX Gateway Fabric creates a GatewayClass with the name **nginx**. NGINX Gateway Fabric will only configure Gateways with a **gatewayClassName** of **nginx** unless you change the name via the `--gatewayclass` [command-line flag](/ngf/reference/cli-help.md#controller). We specify a [listener](https://gateway-api.sigs.k8s.io/reference/spec/#listener) on the Gateway to open an entry point on the cluster. In this case, since the coffee application accepts HTTP requests, we create an HTTP listener, named **http**, that listens on port 80. By default, Gateways only allow routes (such as HTTPRoutes) to attach if they are in the same namespace as the Gateway. If you want to change this behavior, you can set the [**allowedRoutes**](https://gateway-api.sigs.k8s.io/reference/spec/#allowedroutes) field. Next you will create the HTTPRoute by copying and pasting the following into your terminal: ```yaml kubectl apply -f - < 404 Not Found

404 Not Found

nginx/1.25.2
``` --- ## Upgrade WebSocket connections NGINX Gateway Fabric can upgrade HTTP/1.1 requests to websocket connections when the appropriate headers are present and the backend supports websocket connections. The backend is responsible for completing the handshake by responding with `101 Switching Protocols`. To ensure this example works, your backend must support websocket connections. Send a `curl` request that includes the required websocket upgrade headers, described below: - `Connection: Upgrade` — requests a protocol change on the current connection. - `Upgrade: websocket` — indicates the desired protocol. - `Sec-WebSocket-Version: 13` — required websocket protocol version. - `Sec-WebSocket-Key` — a random, base64-encoded nonce used by the server to compute `Sec-WebSocket-Accept`. ```shell curl --http1.1 --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee -H "Upgrade: websocket" -H "Connection: upgrade" -H "Sec-WebSocket-Version: 13" -H "Sec-WebSocket-Key: $(openssl rand -base64 16)" ``` A websocket-capable backend will reply with: ```text < HTTP/1.1 101 Switching Protocols < Server: nginx < Connection: upgrade < Upgrade: websocket < Sec-WebSocket-Accept: On5gtTNGCqB4Emnh7Ck4T0b2gks= ``` Receiving a `101 Switching Protocols` response along with the `Upgrade`, `Connection`, and `Sec-WebSocket-Accept` headers confirms that the websocket connection upgrade completed successfully. --- ## Troubleshooting If you have any issues while testing the configuration, try the following to debug your configuration and setup: - Make sure you set the shell variables $GW_IP and $GW_PORT to the public IP and port of the NGINX Service. Refer to the [Installation](/ngf/install/) guides for more information. - Check the status of the gateway: ```shell kubectl describe gateway cafe ``` The gateway status should look similar to this: ```text Status: Addresses: Type: IPAddress Value: 10.244.0.85 Conditions: Last Transition Time: 2023-08-15T20:57:21Z Message: Gateway is accepted Observed Generation: 1 Reason: Accepted Status: True Type: Accepted Last Transition Time: 2023-08-15T20:57:21Z Message: Gateway is programmed Observed Generation: 1 Reason: Programmed Status: True Type: Programmed Listeners: Attached Routes: 1 Conditions: Last Transition Time: 2023-08-15T20:57:21Z Message: Listener is accepted Observed Generation: 1 Reason: Accepted Status: True Type: Accepted Last Transition Time: 2023-08-15T20:57:21Z Message: Listener is programmed Observed Generation: 1 Reason: Programmed Status: True Type: Programmed Last Transition Time: 2023-08-15T20:57:21Z Message: All references are resolved Observed Generation: 1 Reason: ResolvedRefs Status: True Type: ResolvedRefs Last Transition Time: 2023-08-15T20:57:21Z Message: No conflicts Observed Generation: 1 Reason: NoConflicts Status: False Type: Conflicted Name: http ``` Check that the conditions match and that the attached routes for the **http** listener equals 1. If it is 0, there may be an issue with the HTTPRoute. - Check the status of the HTTPRoute: ```shell kubectl describe httproute coffee ``` The HTTPRoute status should look similar to this: ```text Status: Parents: Conditions: Last Transition Time: 2023-08-15T20:57:21Z Message: The route is accepted Observed Generation: 1 Reason: Accepted Status: True Type: Accepted Last Transition Time: 2023-08-15T20:57:21Z Message: All references are resolved Observed Generation: 1 Reason: ResolvedRefs Status: True Type: ResolvedRefs Controller Name: gateway.nginx.org/nginx-gateway-controller Parent Ref: Group: gateway.networking.k8s.io Kind: Gateway Name: cafe Namespace: default ``` Check for any error messages in the conditions. - Check the generated nginx config: ```shell kubectl exec -it deployments/cafe-nginx -- nginx -T ``` The config should contain a server block with the server name "cafe.example.com" that listens on port 80. This server block should have a single location `/` that proxy passes to the coffee upstream: ```nginx configuration server { listen 80; server_name cafe.example.com; location / { ... proxy_pass http://default_coffee_80$request_uri; # the upstream is named default_coffee_80 ... } } ``` There should also be an upstream block with a name that matches the upstream in the **proxy_pass** directive. This upstream block should contain the pod IPs of the **coffee** pods: ```nginx configuration upstream default_coffee_80 { ... server 10.12.0.18:8080; # these should be the pod IPs of the coffee pods server 10.12.0.19:8080; ... } ``` **Note:** The entire configuration is not shown because it is subject to change. Ellipses indicate that there's configuration not shown. If your issue persists, [contact us](https://github.com/nginx/nginx-gateway-fabric#contacts). --- ## See also To learn more about the Gateway API and the resources we created in this guide, check out the following resources: - [Gateway API Overview](https://gateway-api.sigs.k8s.io/concepts/api-overview/) - [Deploying a simple Gateway](https://gateway-api.sigs.k8s.io/guides/getting-started/simple-gateway/) - [HTTP Routing](https://gateway-api.sigs.k8s.io/guides/http-routing/) -------------------------------------------------------------------------------- title: "Secure traffic using Let's Encrypt and cert-manager" url: https://docs.nginx.com/nginx-gateway-fabric/traffic-security/integrate-cert-manager/index.md -------------------------------------------------------------------------------- Learn how to issue and manage certificates using Let's Encrypt and cert-manager. ## Overview Securing client server communication is a crucial part of modern application architectures. One of the most important steps in this process is implementing HTTPS (HTTP over TLS/SSL) for all communications. This encrypts the data transmitted between the client and server, preventing eavesdropping and tampering. To do this, you need an SSL/TLS certificate from a trusted Certificate Authority (CA). However, issuing and managing certificates can be a complicated manual process. Luckily, there are many services and tools available to simplify and automate certificate issuance and management. Follow the steps in this guide to: - Configure HTTPS for your application using a [gateway](https://gateway-api.sigs.k8s.io/api-types/gateway/). - Use [Let’s Encrypt](https://letsencrypt.org) as the Certificate Authority (CA) issuing the TLS certificate. - Use [cert-manager](https://cert-manager.io) to automate the provisioning and management of the certificate. --- ## Before you begin You need: - Administrator access to a Kubernetes cluster. - [Helm](https://helm.sh) and [kubectl](https://kubernetes.io/docs/tasks/tools/#kubectl) must be installed locally. - [NGINX Gateway Fabric deployed](/ngf/install/) in the Kubernetes cluster. - A DNS-resolvable domain name is required. It must resolve to the public endpoint of the NGINX Gateway Fabric deployment, and this public endpoint must be an external IP address or alias accessible over the internet. The process here will depend on your DNS provider. This DNS name will need to be resolvable from the Let’s Encrypt servers, which may require that you wait for the record to propagate before it will work. --- ## Secure traffic using Let's Encrypt and cert-manager ![cert-manager ACME challenge and certificate management with Gateway API](ngf/img/cert-manager-gateway-workflow.png) The diagram above shows a simplified representation of the cert-manager ACME challenge and certificate issuance process using Gateway API. Please note that not all of the kubernetes objects created in this process are represented in this diagram. At a high level, the process looks like this: 1. We deploy cert-manager and create a ClusterIssuer which specifies Let’s Encrypt as our CA and gateway as our ACME HTTP01 challenge solver. 1. We create a gateway resource for our domain (cafe.example.com) and configure cert-manager integration using an annotation. 1. This starts the certificate issuance process – cert-manager contacts Let’s Encrypt to obtain a certificate, and Let’s Encrypt starts the ACME challenge. As part of this challenge, cert-manager creates a temporary HTTPRoute resource which directs the traffic through NGINX Gateway Fabric to verify we control the domain name in the certificate request. 1. Once the domain has been verified, the certificate is issued. Cert-manager stores the keypair in a Kubernetes secret that is referenced by the gateway resource. As a result, NGINX is configured to terminate HTTPS traffic from clients using this signed keypair. 1. We deploy our application and our HTTPRoute which defines our routing rules. The routing rules defined configure NGINX to direct requests to `https://cafe.example.com/coffee` to our coffee-app application, and to use the HTTPS listener defined in our gateway resource. 1. When the client connects to `https://cafe.example.com/coffee`, the request is routed to the coffee-app application and the communication is secured using the signed keypair contained in the cafe-secret secret. 1. The certificate will be automatically renewed when it is close to expiry, the secret will be updated using the new certificate, and NGINX Gateway Fabric will dynamically update the keypair on the filesystem used by NGINX for HTTPS termination once the secret is updated. --- ## Securing traffic ### Deploy cert-manager Install cert-manager onto the cluster using Helm with Gateway API features enabled. - Add the Helm repository. ```shell helm repo add jetstack https://charts.jetstack.io helm repo update ``` - Install cert-manager, and enable the GatewayAPI feature gate: ```shell helm install \ cert-manager jetstack/cert-manager \ --namespace cert-manager \ --create-namespace \ --set config.apiVersion="controller.config.cert-manager.io/v1alpha1" \ --set config.kind="ControllerConfiguration" \ --set config.enableGatewayAPI=true \ --set crds.enabled=true ``` --- ### Create a ClusterIssuer Next we need to create a [ClusterIssuer](https://cert-manager.io/docs/concepts/issuer/), a Kubernetes resource that represents the certificate authority (CA) that will generate the signed certificates by honoring certificate signing requests. We are using the ACME Issuer type, and Let's Encrypt as the CA server. In order for Let's Encypt to verify that we own the domain a certificate is being requested for, we must complete "challenges". This is to ensure clients are unable to request certificates for domains they do not own. We will configure the issuer to use a HTTP01 challenge, and our gateway resource that we will create in the next step as the solver. To read more about HTTP01 challenges, see the [cert-manager documentation](https://cert-manager.io/docs/configuration/acme/http01/). Use the following YAML definition to create the resource, but please note the `email` field must be updated to your own email address. ```yaml apiVersion: cert-manager.io/v1 kind: ClusterIssuer metadata: name: letsencrypt-prod spec: acme: # You must replace this email address with your own. # Let's Encrypt will use this to contact you about expiring # certificates, and issues related to your account. email: my-name@example.com server: https://acme-v02.api.letsencrypt.org/directory privateKeySecretRef: # Secret resource that will be used to store the account's private key. name: issuer-account-key # Add a single challenge solver, HTTP01 using NGINX Gateway Fabric solvers: - http01: gatewayHTTPRoute: parentRefs: # This is the name of the Gateway that will be created in the next step - name: gateway namespace: default kind: Gateway ``` --- ### Deploy our Gateway with the cert-manager annotation Next we need to deploy our gateway. You can use the YAML manifest below, updating the `spec.listeners[1].hostname` field to the required value for your environment. ```yaml apiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: name: gateway annotations: # This is the name of the ClusterIssuer created in the previous step cert-manager.io/cluster-issuer: letsencrypt-prod spec: gatewayClassName: nginx listeners: - name: http port: 80 protocol: HTTP - name: https # Important: The hostname needs to be set to your domain hostname: "cafe.example.com" port: 443 protocol: HTTPS tls: mode: Terminate certificateRefs: - kind: Secret name: cafe-secret ``` It's worth noting a couple of key details in this manifest: - The cert-manager annotation is present in the metadata – this enables the cert-manager integration, and tells cert-manager which ClusterIssuer configuration it should use for the certificates. - There are two listeners configured, an HTTP listener on port 80, and an HTTPS listener on port 443. - The HTTP listener on port 80 is required for the HTTP01 ACME challenge to work. This is because as part of the HTTP01 challenge, a temporary HTTPRoute will be created by cert-manager to solve the ACME challenge, and this HTTPRoute requires a listener on port 80. See the [HTTP01 Gateway API solver documentation](https://cert-manager.io/docs/configuration/acme/http01/#configuring-the-http-01-gateway-api-solver) for more information. - The HTTPS listener on port 443 is the listener we will use in our HTTPRoute in the next step. Cert-manager will create a certificate for this listener block. - The hostname needs to set to the required value. A new certificate will be issued from the `letsencrypt-prod` ClusterIssuer for the domain, e.g. "cafe.example.com", once the ACME challenge is successful. Once the certificate has been issued, cert-manager will create a certificate resource on the cluster and the `cafe-secret` Secret containing the signed keypair in the same Namespace as the gateway. We can verify the secret has been created successfully using `kubectl`. Note it will take a little bit of time for the challenge to complete and the secret to be created: ```shell kubectl get secret cafe-secret ``` ```text NAME TYPE DATA AGE cafe-secret kubernetes.io/tls 2 20s ``` --- - Multiple `certificateRefs` apply to the listener's `hostname` and are not automatically selected based on the hostnames of attached HTTPRoute resources. Use separate HTTPS listeners for different domains that require different certificates. ### Deploy our application and HTTPRoute Now we can create our coffee deployment and service, and configure the routing rules. You can use the following manifest to create the deployment and service: ```yaml apiVersion: apps/v1 kind: Deployment metadata: name: coffee spec: replicas: 1 selector: matchLabels: app: coffee template: metadata: labels: app: coffee spec: containers: - name: coffee image: nginxdemos/nginx-hello:plain-text ports: - containerPort: 8080 --- apiVersion: v1 kind: Service metadata: name: coffee spec: ports: - port: 80 targetPort: 8080 protocol: TCP name: http selector: app: coffee ``` Deploy our HTTPRoute to configure our routing rules for the coffee application. Note the `parentRefs` section in the spec refers to the listener configured in the previous step. ```yaml apiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: name: coffee spec: parentRefs: - name: gateway sectionName: https hostnames: # Update the hostname to match what is configured in the Gateway resource - "cafe.example.com" rules: - matches: - path: type: PathPrefix value: /coffee backendRefs: - name: coffee port: 80 ``` --- ## Testing To test everything has worked correctly, we can use curl to the navigate to our endpoint, for example, `https://cafe.example.com/coffee`. To verify using curl, we can use the `-v` option to increase verbosity and inspect the presented certificate. ```shell curl https://cafe.example.com/coffee -v ``` The output will look similar to this: ```text * Trying 54.195.47.105:443... * Connected to cafe.example.com (54.195.47.105) port 443 (#0) * ALPN: offers h2,http/1.1 * (304) (OUT), TLS handshake, Client hello (1): * CAfile: /etc/ssl/cert.pem * CApath: none * (304) (IN), TLS handshake, Server hello (2): * (304) (IN), TLS handshake, Unknown (8): * (304) (IN), TLS handshake, Certificate (11): * (304) (IN), TLS handshake, CERT verify (15): * (304) (IN), TLS handshake, Finished (20): * (304) (OUT), TLS handshake, Finished (20): * SSL connection using TLSv1.3 / AEAD-CHACHA20-POLY1305-SHA256 * ALPN: server accepted http/1.1 * Server certificate: * subject: CN=cafe.example.com * start date: Aug 11 08:22:11 2023 GMT * expire date: Nov 9 08:22:10 2023 GMT * subjectAltName: host "cafe.example.com" matched cert's "cafe.example.com" * issuer: C=US; O=Let's Encrypt; CN=R3 * SSL certificate verify ok. * using HTTP/1.1 > GET /coffee HTTP/1.1 > Host: cafe.example.com > User-Agent: curl/7.88.1 > Accept: */* > < HTTP/1.1 200 OK < Server: nginx/1.25.1 < Date: Fri, 11 Aug 2023 10:03:21 GMT < Content-Type: text/plain < Content-Length: 163 < Connection: keep-alive < Expires: Fri, 11 Aug 2023 10:03:20 GMT < Cache-Control: no-cache < Server address: 192.168.78.136:8080 Server name: coffee-9bf875848-xvkqv Date: 11/Aug/2023:10:03:21 +0000 URI: /coffee Request ID: e64c54a2ac253375ac085d48980f000a * Connection #0 to host cafe.example.com left intact ``` --- ## Troubleshooting - To troubleshoot any issues related to the cert-manager installation or issuer setup, see [the cert-manager troubleshooting guide](https://cert-manager.io/docs/troubleshooting/). - To troubleshoot the HTTP01 ACME challenge, please see the cert-manager [ACME troubleshooting guide](https://cert-manager.io/docs/troubleshooting/acme/). - Note that for the HTTP01 challenge to work using the gateway resource, HTTPS redirect must not be configured. - The temporary HTTPRoute created by cert-manager routes the traffic between cert-manager and the Let's Encrypt server through NGINX Gateway Fabric. If the challenge is not successful, it may be useful to inspect the NGINX logs to see the ACME challenge requests. You should see something like the following: ```shell kubectl logs -n <...> 52.208.162.19 - - [15/Aug/2023:13:18:12 +0000] "GET /.well-known/acme-challenge/bXQn27Lenax2AJKmOOS523T-MWOKeFhL0bvrouNkUc4 HTTP/1.1" 200 87 "-" "cert-manager-challenges/v1.12.0 (linux/amd64) cert-manager/bd192c4f76dd883f9ee908035b894ffb49002384" 52.208.162.19 - - [15/Aug/2023:13:18:14 +0000] "GET /.well-known/acme-challenge/bXQn27Lenax2AJKmOOS523T-MWOKeFhL0bvrouNkUc4 HTTP/1.1" 200 87 "-" "cert-manager-challenges/v1.12.0 (linux/amd64) cert-manager/bd192c4f76dd883f9ee908035b894ffb49002384" 52.208.162.19 - - [15/Aug/2023:13:18:16 +0000] "GET /.well-known/acme-challenge/bXQn27Lenax2AJKmOOS523T-MWOKeFhL0bvrouNkUc4 HTTP/1.1" 200 87 "-" "cert-manager-challenges/v1.12.0 (linux/amd64) cert-manager/bd192c4f76dd883f9ee908035b894ffb49002384" 52.208.162.19 - - [15/Aug/2023:13:18:18 +0000] "GET /.well-known/acme-challenge/bXQn27Lenax2AJKmOOS523T-MWOKeFhL0bvrouNkUc4 HTTP/1.1" 200 87 "-" "cert-manager-challenges/v1.12.0 (linux/amd64) cert-manager/bd192c4f76dd883f9ee908035b894ffb49002384" 52.208.162.19 - - [15/Aug/2023:13:18:20 +0000] "GET /.well-known/acme-challenge/bXQn27Lenax2AJKmOOS523T-MWOKeFhL0bvrouNkUc4 HTTP/1.1" 200 87 "-" "cert-manager-challenges/v1.12.0 (linux/amd64) cert-manager/bd192c4f76dd883f9ee908035b894ffb49002384" 3.128.204.81 - - [15/Aug/2023:13:18:22 +0000] "GET /.well-known/acme-challenge/bXQn27Lenax2AJKmOOS523T-MWOKeFhL0bvrouNkUc4 HTTP/1.1" 200 87 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)" 23.178.112.204 - - [15/Aug/2023:13:18:22 +0000] "GET /.well-known/acme-challenge/bXQn27Lenax2AJKmOOS523T-MWOKeFhL0bvrouNkUc4 HTTP/1.1" 200 87 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)" 35.166.192.222 - - [15/Aug/2023:13:18:22 +0000] "GET /.well-known/acme-challenge/bXQn27Lenax2AJKmOOS523T-MWOKeFhL0bvrouNkUc4 HTTP/1.1" 200 87 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)" <...> ``` ## HTTPS Traffic without SNI (Server Name Indication) Some frontend load balancers strip out SNI information before the traffic reaches the NGINX gateway. In order for NGINX to still process and forward this traffic properly, you must define your HTTPS Listener without a hostname. This instructs NGINX Gateway Fabric to configure a default HTTPS virtual server to handle non-SNI traffic. The TLS configuration on this Listener will be used to verify and terminate TLS for this traffic, before the Host header is then used to forward to the proper virtual server to handle the request. You can attach your HTTPRoutes to this empty Listener. By default, NGINX Gateway Fabric verifies that the Listener hostname matches both the SNI and Host header on an incoming client request. This does not require the SNI and Host header to be the same. This is to avoid misdirected requests, and returns a 421 response code. If you run into issues and want to disable this SNI/Host verification, you can update the [NginxProxy CRD](/ngf/how-to/data-plane-configuration.md) with the following field in the spec: ```yaml spec: disableSNIHostValidation: true ``` --- ## See also - [Gateway docs](https://gateway-api.sigs.k8s.io) - [Cert-manager gateway usage](https://cert-manager.io/docs/usage/gateway/) - [Cert-manager ACME](https://cert-manager.io/docs/configuration/acme/) - [Let’s Encrypt](https://letsencrypt.org) - [NGINX HTTPS docs](https://docs.nginx.com/nginx/admin-guide/security-controls/terminating-ssl-http/) -------------------------------------------------------------------------------- title: "Use F5 BIG-IP as an external load balancer" url: https://docs.nginx.com/nginx-gateway-fabric/external-loadbalancers/big-ip/quickstart/index.md description: Configure an ExternalLoadBalancer so F5 BIG-IP acts as the external load balancer for an NGINX Gateway Fabric Gateway in a single cluster, preserving client IP addresses with the PROXY protocol. -------------------------------------------------------------------------------- This guide describes how to use an F5 BIG-IP system as the external load balancer for an NGINX Gateway Fabric Gateway. ## Overview GatewayLink integrates NGINX Gateway Fabric with F5 BIG-IP Container Ingress Services to configure an F5 BIG-IP system as the external load balancer for a Gateway. You describe the desired BIG-IP configuration through the `ExternalLoadBalancer` custom resource. In this guide, the F5 IPAM Controller allocates the address that BIG-IP listens on, and an iRule preserves the original client address by forwarding it to NGINX using the PROXY protocol. ### How configuration reaches BIG-IP NGINX Gateway Fabric watches `ExternalLoadBalancer` resources. For each one, it creates an `IngressLink` resource, the custom resource F5 Container Ingress Services uses to describe a Gateway that BIG-IP fronts. The IngressLink carries the settings from the ExternalLoadBalancer spec, along with a label selector that matches the Gateway's data plane Service. F5 Container Ingress Services watches IngressLink resources. It resolves the selector to the data plane Service, reads its node addresses and NodePorts, and compiles them into an AS3 declaration. It posts that declaration to the AS3 endpoint on BIG-IP, which creates the virtual server and its pool. F5 Container Ingress Services reposts the declaration whenever the endpoints or the IngressLink change, so BIG-IP stays current as Pods come and go. ```mermaid flowchart LR A[ExternalLoadBalancer
NGINX Gateway Fabric] --> B[IngressLink
F5 Container Ingress Services] B --> C[AS3 declaration
POST to BIG-IP] C --> D[BIG-IP
virtual server, pool] ``` ## Before you begin You need: - A Kubernetes cluster. - An F5 BIG-IP system running version or later, and an account on it with administrator privileges. - Network access from the cluster to the BIG-IP system, and from BIG-IP to the cluster node addresses. - Python 3.14 or later. This guide installs the AS3 extension, the F5 IPAM Controller, F5 Container Ingress Services, and NGINX Gateway Fabric. The shell commands in this guide read the following environment variables, so set them once in the shell you work from and the commands can be copied as they appear: ```shell export BIGIP_ADDRESS="192.0.2.10:443" export BIGIP_USERNAME="admin" export BIGIP_PASSWORD="" export IPAM_ADDRESS_RANGE="192.0.2.100-192.0.2.110" ``` - `BIGIP_ADDRESS` is the BIG-IP management address, including the port. BIG-IP listens on 443 by default. - `BIGIP_USERNAME` and `BIGIP_PASSWORD` are your BIG-IP credentials. - `IPAM_ADDRESS_RANGE` is a free address range on the BIG-IP subnet, which the F5 IPAM Controller allocates from. You choose this range in [Install the F5 IPAM Controller](#install-the-f5-ipam-controller). Two more variables are set later, once their values exist: - `ALLOCATED_ADDRESS` is the virtual server address the F5 IPAM Controller allocates, read from the `IngressLink` status in [Verify the configuration](#verify-the-configuration). - `NGINX_POD_NAME` is the name of an NGINX Pod, used when reading its logs. ## Prepare BIG-IP In this section you install the AS3 extension and create the two BIG-IP objects this guide depends on: a partition for F5 Container Ingress Services to own, and an iRule that adds a PROXY protocol header. ### AS3 extension F5 Container Ingress Services configures BIG-IP by posting AS3 declarations, so AS3 must be installed before anything else. Follow [Downloading and installing the BIG-IP AS3 package](https://clouddocs.f5.com/products/extensions/f5-appsvcs-extension/latest/userguide/installation.html) in the F5 documentation, then return here. ### Partition Create a partition named `k8s` for F5 Container Ingress Services to own: ```shell curl -sku "$BIGIP_USERNAME:$BIGIP_PASSWORD" -X POST "https://$BIGIP_ADDRESS/mgmt/tm/auth/partition" \ -H "Content-Type: application/json" -d '{"name":"k8s"}' ``` The response describes the new partition: ```json { "name": "k8s", "fullPath": "k8s", "defaultRouteDomain": 0 } ``` F5 Container Ingress Services manages the full contents of its partition. The partition cannot be `Common`, because Container Ingress Services must not modify shared configuration. ### TCP iRule This guide uses a TCP iRule named `Proxy_Protocol_iRule`: ```text when SERVER_CONNECTED { TCP::respond "PROXY TCP[IP::version] [IP::client_addr] [clientside {IP::local_addr}] [TCP::client_port] [clientside {TCP::local_port}]\r\n" } ``` The iRule runs on the `SERVER_CONNECTED` event, which fires when BIG-IP opens a connection to NGINX, before any application data is sent. It writes a single PROXY protocol header onto that connection. The header carries the original client address, so NGINX can report it instead of the BIG-IP self-IP address. To create the iRule: ```shell curl -sku "$BIGIP_USERNAME:$BIGIP_PASSWORD" -X POST "https://$BIGIP_ADDRESS/mgmt/tm/ltm/rule" \ -H "Content-Type: application/json" -d '{ "name": "Proxy_Protocol_iRule", "apiAnonymous": "when SERVER_CONNECTED {\n TCP::respond \"PROXY TCP[IP::version] [IP::client_addr] [clientside {IP::local_addr}] [TCP::client_port] [clientside {TCP::local_port}]\\r\\n\"\n}" }' ``` The response describes the new iRule: ```json { "name": "Proxy_Protocol_iRule", "fullPath": "/Common/Proxy_Protocol_iRule", "apiAnonymous": "when SERVER_CONNECTED { ... }" } ``` ## Install F5 Container Ingress Services ### Install the F5 IPAM Controller The F5 IPAM Controller allocates the virtual server address from a range you define, so you do not have to pick and track an address by hand. Allocation is a handoff between the two controllers through a shared `IPAM` resource. F5 Container Ingress Services creates that resource on startup when it is installed with `--ipam=true`. When an `IngressLink` names an IPAM label, Container Ingress Services adds an entry to the resource `spec` requesting an address under that label. The F5 IPAM Controller watches the same resource, takes an address from the range configured for that label, and records the assignment in the resource `status`. Container Ingress Services reads the address from the status and uses it as the virtual server address in the AS3 declaration. Install the F5 IPAM Controller before Container Ingress Services, so it is watching by the time the first request is made. Install the `IPAM` custom resource definition: ```yaml kubectl apply -f - < ``` F5 Container Ingress Services writes this status after posting the AS3 declaration. A status of `OK` means BIG-IP accepted the declaration, and `vsAddress` is the address the F5 IPAM Controller allocated. Store that address for the remaining commands: ```shell export ALLOCATED_ADDRESS=$(kubectl get ingresslink gateway-nginx -o jsonpath='{.status.vsAddress}') ``` Send a request through BIG-IP: ```shell curl -H "Host: cafe.example.com" http://$ALLOCATED_ADDRESS/coffee ``` The request returns `200 OK` with a response body from the backend application. ```text Server address: 10.42.0.43:8080 Server name: coffee-7b9578cff9-t7r7v Date: 05/Aug/2026:14:33:40 +0000 URI: /coffee Request ID: a2ae0944885fdf99bb5f86038aeae84f ``` Confirm NGINX sees the original client address: ```shell export NGINX_POD_NAME=$(kubectl get pods -l app.kubernetes.io/name=gateway-nginx -o jsonpath='{.items[0].metadata.name}') kubectl logs $NGINX_POD_NAME -c nginx | grep coffee ``` The access log records the address of the machine you sent the request from. ## Troubleshooting ### No IngressLink is created Confirm the `--external-load-balancer` flag is set on the control plane deployment. Helm ignores values a chart does not define, so a chart without external load balancer support renders a deployment without the flag: ```shell kubectl get deploy -n nginx-gateway ngf-nginx-gateway-fabric \ -o jsonpath='{.spec.template.spec.containers[?(@.name=="nginx-gateway")].args}' ``` ### The IngressLink has no status F5 Container Ingress Services writes this status, so an empty status means it has not processed the resource. Wait up to two minutes for reconciliation, then check its logs: ```shell kubectl logs -n kube-system deploy/f5-cis-f5-bigip-ctlr ``` ### No address is allocated Confirm F5 Container Ingress Services was deployed with `args.ipam=true`, then check the F5 IPAM Controller logs: ```shell kubectl logs -n kube-system -l app=f5-ipam-controller --tail=20 ``` A label that does not match a configured pool is reported directly: ```text [PROV] IPAM LABEL: gatewaylink Not Found ``` Set `ipamLabel` on the `ExternalLoadBalancer` to a pool name from the `args.ip_range` map used when installing the F5 IPAM Controller. ### The AS3 declaration is rejected Read the BIG-IP response in the F5 Container Ingress Services logs, which usually names the problem: ```shell kubectl logs -n kube-system deploy/f5-cis-f5-bigip-ctlr | grep -E "AS3\]\[POST\]|response:" ``` ### F5 Container Ingress Services reports that AS3 is not installed The Pod is in `CrashLoopBackOff` and its logs contain `[ERROR] AS3 RPM is not installed on BIGIP`. F5 Container Ingress Services infers this from a 404 on the AS3 endpoint, so it also appears when AS3 is installed but not serving. See [Troubleshooting](https://clouddocs.f5.com/products/extensions/f5-appsvcs-extension/latest/userguide/troubleshooting.html) in the F5 documentation. After restoring AS3, delete the Pod so it retries without waiting out its backoff: ```shell kubectl delete pod -n kube-system -l app=f5-cis-f5-bigip-ctlr ``` ### A pool is empty Confirm the type of the Gateway's Service matches the F5 Container Ingress Services `pool_member_type`, and that the Gateway has a listener on the port the pool was built for. A missing or invalid `certificateRefs` Secret leaves an HTTPS listener unprogrammed, so the Service never exposes port 443: ```shell kubectl get svc gateway-nginx -o jsonpath='{.spec.type}{"\n"}{.spec.ports}' kubectl describe gateways.gateway.networking.k8s.io gateway ``` ### NGINX logs show an internal address as the client The client address travels inside the PROXY protocol header. NGINX reads it only when both the connection address and the address inside the header are trusted, so an internal address in the log means the header never arrived or was discarded. Confirm the iRule is attached. Creating an iRule on BIG-IP does not attach it to anything: ```shell curl -sku "$BIGIP_USERNAME:$BIGIP_PASSWORD" "https://$BIGIP_ADDRESS/mgmt/tm/ltm/virtual" \ | python3 -c 'import sys,json for v in json.load(sys.stdin)["items"]: print(v["fullPath"], "->", v.get("rules", "no rules"))' ``` If the iRule is attached, confirm the trusted addresses: ```shell kubectl exec $NGINX_POD_NAME -c nginx -- grep set_real_ip_from /etc/nginx/conf.d/http.conf ``` Set `trustedAddresses` on the `NginxProxy` resource to the subnet of the IP address which the BIG-IP system uses to send traffic to NGINX. ### A configured field has no effect Kubernetes discards fields that are not in the installed custom resource definition schema without reporting an error, so both controllers report success while the field never arrives. Check where the field stops: ```shell export FIELD_NAME="ipamLabel" kubectl get crd ingresslinks.cis.f5.com -o yaml | grep -A5 "$FIELD_NAME" kubectl logs -n nginx-gateway deploy/ngf-nginx-gateway-fabric | grep "unknown field" kubectl get ingresslink gateway-nginx -o jsonpath='{.spec}' | python3 -m json.tool ``` An `unknown field` message means the installed custom resource definition is older than the NGINX Gateway Fabric release. Install a matching version. ## Remove the configuration Delete the `ExternalLoadBalancer` so F5 Container Ingress Services deletes the objects it created on BIG-IP: ```shell kubectl delete externalloadbalancer gateway-elb ``` Confirm the virtual servers are gone: ```shell curl -sku "$BIGIP_USERNAME:$BIGIP_PASSWORD" "https://$BIGIP_ADDRESS/mgmt/tm/ltm/virtual" | python3 -m json.tool | grep fullPath ``` ## References - [Distribute traffic across clusters with F5 BIG-IP](/ngf/external-loadbalancers/BIG-IP/multicluster.md): terminate TLS at BIG-IP and spread traffic across two clusters, with health monitors and iRules. - [F5 IngressLink documentation](https://clouddocs.f5.com/containers/latest/userguide/ingresslink/): the F5 Container Ingress Services resource that NGINX Gateway Fabric generates. - [F5 Application Services 3 Extension reference](https://clouddocs.f5.com/products/extensions/f5-appsvcs-extension/latest/refguide/schema-reference.html): the declaration format F5 Container Ingress Services posts to BIG-IP. - [NGINX Gateway Fabric](https://github.com/nginx/nginx-gateway-fabric): the NGINX Gateway Fabric source, including the `ExternalLoadBalancer` custom resource definitions. - [F5 Container Ingress Services](https://github.com/F5Networks/k8s-bigip-ctlr): the F5 Container Ingress Services source and custom resource definitions. - [F5 IPAM Controller](https://github.com/F5Networks/f5-ipam-controller): allocates virtual server addresses. - [F5 Container Ingress Services configuration parameters](https://clouddocs.f5.com/containers/latest/userguide/config-parameters.html): the full list of deployment options. - [PROXY protocol specification](https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt): the header format the iRule generates. -------------------------------------------------------------------------------- title: "Get started with F5 WAF for NGINX (PLM)" url: https://docs.nginx.com/nginx-gateway-fabric/waf-integration/get-started-plm/index.md -------------------------------------------------------------------------------- Use this tutorial to set up end-to-end traffic protection with F5 WAF for NGINX using Policy Lifecycle Management (PLM). By the end, you'll have: - Deployed the PLM infrastructure (Policy Controller and SeaweedFS storage) - Connected NGINX Gateway Fabric to PLM storage - Defined a WAF policy using `APPolicy` and `APLogConf` custom resources - Attached a `WAFPolicy` to a Gateway and configured HTTPRoutes - Validated policy compilation and verified that attacks are blocked PLM is one of four WAF policy source types. With PLM, you define your security posture as `APPolicy` and `APLogConf` custom resources instead of compiling and hosting bundles yourself. For a comparison with the other source types, see [PLM (Policy Lifecycle Management)](/ngf/waf-integration/overview.md#plm-policy-lifecycle-management). ## Before you begin Before you start, make sure you have: - `kubectl` access to a Kubernetes cluster. - A valid F5 WAF for NGINX subscription. F5 WAF for NGINX is a separate add-on to NGINX Plus and isn't included with the NGINX Plus license. - Your F5 WAF for NGINX JWT from MyF5. To get it: 1. Log in to [MyF5](https://my.f5.com/manage/s/). 2. Go to **My Products & Plans** > **Subscriptions** to see your active subscriptions. 3. Find your NGINX products or services subscription and select the **Subscription ID** for details. 4. Download the **JSON Web Token (JWT)** from the subscription page. **Note:** The Connectivity Stack for Kubernetes JWT does not work with NGINX Plus reporting. Use a regular NGINX Plus instance JWT. - Your private registry credentials Secret for `private-registry.nginx.com`. You'll reference this Secret when you install NGINX Gateway Fabric. ### PLM prerequisites The following requirements apply to the PLM backend you'll install in this tutorial: - A Kubernetes cluster with a default StorageClass that supports dynamic provisioning. The PLM object store relies on PersistentVolumeClaims. Without a default StorageClass, the SeaweedFS pods stay `Pending`. - Helm 3.x installed. The PLM backend installs as a Helm chart. - An F5 WAF for NGINX JWT from MyF5, used to pull images from `private-registry.nginx.com`. - Optionally, `nginx-repo.crt` and `nginx-repo.key` from MyF5, needed only for authenticated signature updates from `pkgs.nginx.com`. PLM ships with an embedded SeaweedFS S3-compatible object store. The bundled SeaweedFS operator deploys and manages it. You don't need to provide an external S3 bucket. The PLM controller writes compiled policy bundles to this store; the data plane reads from it. By default, the deployment creates one master pod, one filer pod, and three volume pods, each backed by its own PVC. The chart generates credentials for the store and saves them in the `-f5-waf-seaweedfs-auth` Secret. By default, communication between PLM and the object store uses unencrypted HTTP. To enable TLS, see the PLM chart values (`helm show values nginx-stable/f5-waf-policy-controller`). ### Example values This tutorial uses the following example values. You can use different values — if you do, replace them consistently throughout. | Example value | What it represents | |---|---| | `plm-system` | Namespace for the PLM backend components | | `plm` | Helm release name for the PLM installation | | `` | F5 WAF for NGINX Policy Controller chart and image version | | `security` | Namespace for `APPolicy` and `APLogConf` resources | | `default` | Namespace for the Gateway and `WAFPolicy` | | `cafe.example.com` | Example hostname for HTTPRoutes | ## Deploy PLM infrastructure The Policy Lifecycle Manager (PLM) backend runs as a Kubernetes operator. It watches WAF custom resources and compiles WAF policies into bundles. The Policy Controller delegates compilation to a separate compiler service over gRPC. The resulting bundles are stored in an embedded SeaweedFS S3-compatible object store. F5 WAF for NGINX is installed using a separate Helm chart from your NGINX data plane. The steps in this section install only the F5 WAF for NGINX PLM components and do not affect your existing NGINX installation. ### Create the registry pull secret Create a namespace for the PLM components, store your JWT in a Kubernetes Secret, then create the registry pull secret for the private F5 container registry. 1. Create the namespace and store your JWT. The following commands assume your JWT file is named `license.jwt`: ```shell kubectl create namespace plm-system kubectl create secret generic jwt-reg-secret \ --namespace plm-system \ --from-file=license.jwt ``` 2. Retrieve the JWT from the Secret and create the registry pull secret: ```shell JWT=$(kubectl get secret jwt-reg-secret \ --namespace plm-system \ -o jsonpath='{.data.license\.jwt}' | base64 -d) kubectl create secret docker-registry regcred \ --namespace plm-system \ --docker-server=private-registry.nginx.com \ --docker-username="$JWT" \ --docker-password=none \ --dry-run=client --output yaml | kubectl apply -f - ``` ### Install the Policy Controller Create a values file for the Helm installation. The `securityUpdatesRepo.cert` and `securityUpdatesRepo.key` fields are optional. They are only required if your signature repository needs certificate-based authentication. The Policy Controller starts successfully with these fields left empty. If your signature repository requires them, replace `` and `` with the base64-encoded contents of your `nginx-repo.crt` and `nginx-repo.key` files. To encode them, run: ```shell base64 --wrap=0 < nginx-repo.crt base64 --wrap=0 < nginx-repo.key ``` Create `/tmp/plm-values.yaml`: ```yaml imagePullSecrets: - name: regcred securityUpdatesRepo: cert: "" # optional: only needed for authenticated signature repository access key: "" # optional: only needed for authenticated signature repository access policyController: image: tag: "" compiler: image: tag: "" seaweedfsOperatorConfig: seaweedfs: image: tag: "" seaweedfs-operator: image: tag: "" pullSecrets: regcred ``` #### Enable TLS for PLM storage (optional) By default, communication between PLM components and the SeaweedFS object store uses unencrypted HTTP. To enable TLS, add a `certificates` block to `/tmp/plm-values.yaml`: ```yaml seaweedfsOperatorConfig: seaweedfs: certificates: enabled: true ``` **Note:** The PLM chart does not generate certificates. You must create the five Secrets listed in the commands below before running `helm upgrade --install`. If any Secret is missing, the SeaweedFS pods will fail to mount their certificates and will not start. **Note:** If you're enabling TLS on an existing installation, the storage backend restarts and objects written before the switch can become orphaned. See the [APPolicy shows `invalid` with `unexpected EOF` after enabling TLS](#troubleshoot-the-deployment) entry in the troubleshooting section. A fresh installation with TLS enabled from the start doesn't have this issue. Create the Secrets from your CA and certificate files before installing. The chart expects Secret names in the form `-f5-waf-seaweedfs-` — for the `plm` release name used in this tutorial, those are: ```shell kubectl create secret generic plm-f5-waf-seaweedfs-ca-cert \ --namespace plm-system \ --from-file=tls.crt= \ --from-file=ca.crt= kubectl create secret tls plm-f5-waf-seaweedfs-master-cert \ --namespace plm-system \ --cert= \ --key= kubectl create secret tls plm-f5-waf-seaweedfs-volume-cert \ --namespace plm-system \ --cert= \ --key= kubectl create secret tls plm-f5-waf-seaweedfs-filer-cert \ --namespace plm-system \ --cert= \ --key= kubectl create secret tls plm-f5-waf-seaweedfs-client-cert \ --namespace plm-system \ --cert= \ --key= ``` The CA Secret requires both `tls.crt` and `ca.crt` keys, both pointing to the same CA certificate file. The PLM chart mounts the CA using `tls.crt` into the Policy Controller, compiler, and SeaweedFS pods. The data plane's S3 client reads `ca.crt` from the same Secret when verifying the storage endpoint. The four component Secrets use `kubectl create secret tls`, which produces `tls.crt` and `tls.key` — no `ca.crt` key is needed for them. Replace each `` placeholder with the path to the corresponding certificate and key file from your PKI. The CA must sign all component certificates. If you don't have an existing PKI, generate a CA and sign the five component certificates before proceeding. #### Install the chart Add the NGINX Helm repository and install the chart: ```shell helm repo add nginx-stable https://helm.nginx.com/stable helm repo update nginx-stable helm upgrade --install plm nginx-stable/f5-waf-policy-controller \ --version \ --namespace plm-system \ --values /tmp/plm-values.yaml ``` To see all available configuration options for the PLM chart, run: ```shell helm show values nginx-stable/f5-waf-policy-controller --version ``` ### Verify the deployment Wait for all PLM components to become ready. The Policy Controller's init container waits for both the compiler service and the SeaweedFS S3 endpoint to be available before it starts, so the controller pod will show `Init:0/1` until SeaweedFS is ready. Wait for the SeaweedFS storage backend: ```shell kubectl rollout status deployment/plm-seaweedfs-operator \ --namespace plm-system --timeout=120s ``` The SeaweedFS operator creates the SeaweedFS pods after it reconciles the SeaweedFS custom resource, so there is a window where the operator deployment is ready but no SeaweedFS pods exist yet. Poll until the pods appear and are ready: ```shell end=$((SECONDS + 300)) until kubectl wait pods \ --selector app.kubernetes.io/name=seaweedfs \ --for=condition=Ready \ --namespace plm-system \ --timeout=10s 2>/dev/null; do if [ $SECONDS -ge $end ]; then echo "Timed out waiting for SeaweedFS pods" exit 1 fi sleep 5 done ``` Wait for the Policy Controller: ```shell kubectl rollout status deployment/plm-f5-waf-policy-controller \ --namespace plm-system --timeout=180s ``` Confirm all pods are running: ```shell kubectl get pods --namespace plm-system ``` Example output: ```text NAME READY STATUS RESTARTS plm-f5-waf-compiler-service-xxxxx 1/1 Running 0 plm-f5-waf-policy-controller-xxxxx 1/1 Running 0 plm-seaweedfs-operator-xxxxx 1/1 Running 0 plm-f5-waf-seaweed-master-0 1/1 Running 0 plm-f5-waf-seaweed-filer-0 1/1 Running 0 plm-f5-waf-seaweed-volume-0 1/1 Running 0 plm-f5-waf-seaweed-volume-1 1/1 Running 0 plm-f5-waf-seaweed-volume-2 1/1 Running 0 ``` Confirm the CRDs are present: ```shell kubectl get crd | grep appprotect.f5.com ``` Expected output: ```text aplogconfs.appprotect.f5.com appolicies.appprotect.f5.com apsignatures.appprotect.f5.com apusersigs.appprotect.f5.com ``` All eight pods running and all four CRDs present confirms the PLM backend is ready. ### Update the CRDs **Note:** Skip this step on a fresh install — Helm installs the CRDs automatically. Only follow these steps when upgrading an existing PLM installation. When upgrading PLM, apply the CRDs manually before running `helm upgrade`: ```shell kubectl apply -f https://raw.githubusercontent.com/nginx/waf-policy-controller//manifests/1-deploy-crds.yaml ``` ### Troubleshoot the deployment These are the most common failures during PLM installation, roughly in order of likelihood. #### Pods stuck in `ImagePullBackOff` The JWT is wrong, expired, or contains a line break. Check the events log: ```shell kubectl get events --namespace plm-system --field-selector reason=Failed ``` Use the full JWT string as the registry username. Use the literal string `none` as the password. #### Policy Controller stuck in `Init:0/1` The `Init:0/1` state is expected during startup. The init container waits for the compiler service and the S3 endpoint before it starts. If the pod stays in `Init:0/1` for more than a few minutes, check that the SeaweedFS pods are `Running`: ```shell kubectl get pods --namespace plm-system --selector app.kubernetes.io/name=seaweedfs ``` The most common cause is PVCs stuck in `Pending` because the cluster has no default StorageClass. #### SeaweedFS pods `Pending` SeaweedFS pods stay `Pending` when the cluster has no default StorageClass or insufficient capacity. Check the PVCs and available storage classes: ```shell kubectl get pvc --namespace plm-system kubectl get storageclass ``` #### `APPolicy` shows `invalid` with `unexpected EOF` after enabling TLS Enabling TLS on an existing installation restarts the storage backend. Objects written before TLS was enabled can become orphaned. Check the filer log: ```shell kubectl logs --namespace plm-system plm-f5-waf-seaweed-filer-0 | grep "not found" ``` If the output contains `volume N not found`, orphaned objects exist. Delete the affected `APPolicy` resource and reapply it. The Policy Controller regenerates the bundle. #### Helm install fails on a ClusterRole If the error references `seaweed-editor-role` or `seaweed-viewer-role`, another PLM installation already exists in the cluster. Only one PLM installation is supported per cluster. Remove the existing release before installing. #### Check the Policy Controller logs Use the Policy Controller logs to diagnose any policy-related failure: ```shell kubectl logs --namespace plm-system deploy/plm-f5-waf-policy-controller -c policy-controller ``` **Note:** The `-c policy-controller` flag is required because the pod has more than one container. The containers are distroless, so `kubectl exec` isn't available for interactive debugging. ## Connect NGINX Gateway Fabric to PLM storage NGINX Gateway Fabric fetches compiled bundles from in-cluster PLM storage. You set up storage access once, cluster-wide, at install time. This configuration applies to every `WAFPolicy` that uses `type: PLM`. Create a `values.yaml` file that enables WAF and sets the PLM storage connection details under `nginxGateway.plmStorage`. #### HTTPS (secure) ```yaml nginxGateway: plmStorage: url: "https://plm-f5-waf-seaweed-filer.plm-system.svc.cluster.local:9333" credentialsSecretName: "plm-system/plm-f5-waf-seaweedfs-auth" # contains the seaweedfs_admin_secret field tls: caSecretName: "plm-ca-secret" # Secret with ca.crt for verifying the storage service clientSSLSecretName: "plm-client-secret" # Secret with tls.crt/tls.key for mutual TLS insecureSkipVerify: false # use only for testing ``` **Note:** Always use HTTPS with TLS verification (`caSecretName`) in production. Add `clientSSLSecretName` for mutual TLS in high-security environments, and never set `insecureSkipVerify: true`. **Note:** `credentialsSecretName` and `caSecretName` must reference Secrets in the NGINX Gateway Fabric control plane namespace, unless you prefix them with `/`. #### HTTP ```yaml nginxGateway: plmStorage: url: "http://plm-f5-waf-seaweed-filer.plm-system.svc.cluster.local:8333" credentialsSecretName: "plm-system/plm-f5-waf-seaweedfs-auth" # contains the seaweedfs_admin_secret field ``` Install NGINX Gateway Fabric by following [the installation guide](/ngf/install/helm.md) and using the **NGINX Plus with WAF** tab. Apply this `values.yaml` file in your install or upgrade command by specifying `--values values.yaml`. The PLM installation creates the credentials Secret automatically, containing the S3 secret access key in the `seaweedfs_admin_secret` field (access key ID `admin` by default): ```yaml apiVersion: v1 kind: Secret metadata: name: plm-storage-credentials namespace: nginx-gateway type: Opaque data: seaweedfs_admin_secret: ``` NGINX Gateway Fabric reloads the PLM credentials and TLS Secrets when they change, so you can rotate credentials without restarting the pod. If you install NGINX Gateway Fabric using Kubernetes manifests, use the equivalent `plm-storage-*` flags documented in the [command-line reference](/ngf/reference/cli-help.md). ## Deploy the sample application Deploy the `customers` and `orders` sample applications. The `customers` app returns a response containing fake sensitive data (credit card number and SSN), which you'll use later to demonstrate data guard masking: ```yaml kubectl apply -f - < This section is typically owned by the security team. If you're not on the security team, share this section with them before continuing. Create the `security` namespace. The security team's `APPolicy` and `APLogConf` resources live here. In NGINX Gateway Fabric, the `ReferenceGrant` that permits cross-namespace WAFPolicy references also lives in this namespace. ```shell kubectl create namespace security ``` ## Configure security logging (optional) **Note:** Omit the `securityLogs` field when you create the `WAFPolicy` in [Deploy the Gateway and attach WAFPolicy](#deploy-the-gateway-and-attach-wafpolicy). This section is typically owned by the security team. If you're not on the security team, share this section with them before continuing. PLM security logging profiles are defined as `APLogConf` custom resources. Define a log profile that logs illegal requests: ```yaml kubectl apply -f - < The security team usually owns this section. They define the policy in the `security` namespace, separate from the Gateway namespace, so they can manage security resources independently from routing configuration. If you're not on the security team, share this section with them. You'll need the `APPolicy` name and namespace before continuing. The `APPolicy` resource defines the security policy. The PLM controller watches the resource, compiles the policy, and writes `status.bundle` with `state: ready` when the bundle is available. #### Inline Create an `APPolicy` resource with an inline policy that blocks all attack signatures: ```yaml kubectl apply -f - <`, ``, ``, ``, ``, and `` with your values: ```shell kubectl apply -f - <<'EOF' apiVersion: appprotect.f5.com/v1 kind: APPolicy metadata: name: namespace: spec: policy: $ref: externalReferenceDetails: repositoryDetails: repository: https://github.com//.git ref: "" EOF ``` **Note:** Pin `ref` to a tag or commit SHA rather than a branch name in production environments. Check that the bundle compiled successfully: ```shell kubectl get appolicy \ --namespace \ --output jsonpath='State: {.status.bundle.state}{"\n"}Bundle: {.status.bundle.location}{"\n"}Compiler: {.status.bundle.compilerVersion}{"\n"}' ``` The output shows `State: ready` when compilation succeeds. #### Private repository For private repositories, create a Kubernetes Secret with your personal access token (PAT): ```shell kubectl create secret generic git-token-secret \ --namespace \ --from-literal=token= ``` Then reference the Secret in the `APPolicy` resource: ```shell kubectl apply -f - <<'EOF' apiVersion: appprotect.f5.com/v1 kind: APPolicy metadata: name: namespace: spec: policy: $ref: externalReferenceDetails: repositoryDetails: repository: https://github.com//.git ref: "" authentication: token: git-token-secret EOF ``` #### Update a Git-referenced policy The Policy Controller doesn't poll the Git repository for changes. It fetches the policy file when the `APPolicy` spec changes. To pick up a new version of the policy, push your changes to the repository. Then update `ref` in the `APPolicy` resource to the new tag or commit SHA and reapply the resource. Reapplying an unchanged `APPolicy` doesn't trigger a fetch. Changing an annotation doesn't trigger a fetch either. If you need to re-fetch the same `ref` (for example, after force-updating a tag), delete the `APPolicy` resource and recreate it. #### Precompiled bundle The precompiled-bundle method lets you reference a `.tgz` policy bundle stored in an artifact registry (for example, Artifactory or Nexus). The Policy Controller imports the bundle and stores it in the SeaweedFS object store without recompiling the bundle. Use this method when: - Your security team compiles and publishes bundles through an external pipeline. - You want to separate policy compilation from cluster operations. Create an `APPolicy` resource that references your bundle. Replace ``, ``, and `` with your values: ```shell kubectl apply -f - <<'EOF' apiVersion: appprotect.f5.com/v1 kind: APPolicy metadata: name: namespace: security spec: policy: $ref: "https:///.tgz" EOF ``` **Note:** The Policy Controller must reach the artifact registry over HTTPS. If the registry uses a private certificate authority (CA), mount the CA certificate into the Policy Controller pod and set the `SSL_CERT_FILE` environment variable to its path. `SSL_CERT_FILE` replaces the system trust store entirely. It doesn't append to the system trust store. If SeaweedFS TLS is also turned on, combine both CAs into a single file and reference that file. If the `APPolicy` status shows `x509: certificate signed by unknown authority`, the Policy Controller doesn't trust the artifact registry CA. Check the status for the full error: ```shell kubectl describe appolicy --namespace security ``` #### Confirm the policy is ready The Policy Controller processes the bundle and updates the `APPolicy` status. Check the `bundle.state` field: ```shell kubectl get appolicy \ --namespace security \ --output jsonpath='State: {.status.bundle.state}{"\n"}Bundle: {.status.bundle.location}{"\n"}isCompiled: {.status.processing.isCompiled}{"\n"}' ``` When the bundle is ready, the output looks like this: ```text State: ready Bundle: s3://plm-system/bundles/_imported_.tgz isCompiled: false ``` `isCompiled: false` confirms the bundle was imported without recompilation. `bundle.state` can be one of: | State | Meaning | |-------|---------| | `pending` | The Policy Controller hasn't yet processed the resource. | | `processing` | The Policy Controller is importing or storing the bundle. | | `ready` | The bundle is stored and ready to use. The Policy Controller has populated `bundle.location`. | | `invalid` | The Policy Controller couldn't import the bundle. Check the status for error detail. | #### Update a precompiled bundle The Policy Controller doesn't poll the artifact registry for changes. To pick up a new version of a bundle, update the `$ref` URL in your `APPolicy` resource and reapply the resource. Changing an annotation doesn't trigger a new download. If you need to re-fetch the same URL, delete the `APPolicy` resource and recreate it. Replace ``, ``, and `` with your values: ```shell kubectl apply -f - <<'EOF' apiVersion: appprotect.f5.com/v1 kind: APPolicy metadata: name: namespace: security spec: policy: $ref: "https:///.tgz" EOF ``` The `APPolicy` and `APLogConf` are in the `security` namespace, but the `WAFPolicy` you create next targets a Gateway in the `default` namespace. To permit the cross-namespace reference, create a `ReferenceGrant` in the `security` namespace: ```yaml kubectl apply -f - < ``` **Verify normal traffic flows.** Send a request to the `customers` route — the response contains the fake sensitive data from the `customers` backend: **Note:** If you have a DNS record for `cafe.example.com`, you can send the request directly to that hostname without `--resolve`. ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/customers ``` ```text Customer List: Name: John Doe Credit Card: 4111-1111-1111-1111 SSN: 123-45-6789 ``` The sensitive data passes through because the gateway-level `attack-signatures` policy only inspects inbound requests for attack patterns — it doesn't mask outbound response data. **Verify attacks are blocked.** Send a request with a cross-site scripting (XSS) payload: ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP "http://cafe.example.com:$GW_PORT/customers?x=" ``` The WAF detects the attack signature and rejects the request: ```text Request Rejected ... ``` **Verify the `orders` route is also protected.** Because the policy targets the Gateway, all attached routes inherit protection: ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP "http://cafe.example.com:$GW_PORT/orders?x=" ``` ```text Request Rejected ... ``` **Note:** The exact blocking response depends on your WAF policy configuration. Check the security log for a corresponding blocked event using `kubectl logs -c waf-enforcer`. ## Apply a route-level override (optional) The `customers` route returns sensitive data (credit card numbers and SSNs) in the response body. The gateway-level policy blocks inbound attacks but doesn't inspect outbound responses. This is a common pattern for SecOps and app team collaboration: the security team defines a stricter policy for a specific service, and the platform engineer or app developer attaches it as a route-level override. The override applies only to the `customers` route — other routes continue using the gateway-level policy. **Note:** Only one `WAFPolicy` can target a given resource at a given level. If a second `WAFPolicy` targets the same Gateway or route, it is rejected with `Accepted=False` and reason `Conflicted`. See [Policy attachment](/ngf/waf-integration/overview.md#policy-attachment). To protect sensitive data in responses, define a **data guard** `APPolicy` and apply it as a route-level override on the `customers` route: ```yaml kubectl apply -f - <for
cafe.example.com"] subgraph cluster [Kubernetes Cluster] subgraph appNs [Namespace
default] subgraph nsPadding [" "] nginxPod[Pod
NGINX] coffeeV1Pod[Pod
coffee v1] coffeeV2Pod[Pod
coffee v2] teaPod[Pod
tea] teaPostPod[Pod
tea-post] end end end ngfSvc --> nginxPod nginxPod --/coffee--> coffeeV1Pod nginxPod --/coffee
header: version=v2
OR
/coffee?TEST=v2--> coffeeV2Pod nginxPod --GET /tea--> teaPod nginxPod --POST /tea--> teaPostPod users --> ngfSvc class clusterPadding,nsPadding,clusterPadding2 noBorder class gwNS,appNs namespace class ngfSvc,nginxPod nginxNode class coffeeV1Pod,coffeeV2Pod coffeeNode class teaPod,teaPostPod teaNode classDef noBorder stroke:none,fill:none classDef default fill:#FFFFFF,stroke:#000000 classDef namespace fill:#FFFFFF,stroke:#036ffc,stroke-dasharray: 5 5,text-align:center classDef nginxNode fill:#b4e0ad,stroke:#2AA317 classDef coffeeNode fill:#edbd8c,stroke:#D9822B classDef teaNode fill:#ff8f6a,stroke:#e5805f ``` The goal is to create a set of rules that will result in client requests being sent to specific backends based on the request attributes. In this diagram, we have two versions of the `coffee` service. Traffic for v1 needs to be directed to the old application, while traffic for v2 needs to be directed towards the new application. We also have two `tea` services, one that handles GET operations and one that handles POST operations. Both the `tea` and `coffee` applications share the same Gateway. ## Before you begin - [Install](/ngf/install/) NGINX Gateway Fabric. ## Coffee applications ### Deploy the Coffee applications Begin by deploying the `coffee-v1`, `coffee-v2` and `coffee-v3` applications: ```shell kubectl apply -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v/examples/advanced-routing/coffee.yaml ``` ### Deploy the Gateway API Resources for the Coffee applications The [gateway](https://gateway-api.sigs.k8s.io/api-types/gateway/) resource is typically deployed by the [cluster operator](https://gateway-api.sigs.k8s.io/docs/concepts/roles-and-personas/#roles-and-personas_1). To deploy the gateway: ```yaml kubectl apply -f - < ``` **Note:** In a production environment, you should have a DNS record for the external IP address that is exposed, and it should refer to the hostname that the gateway will forward for. The [HTTPRoute](https://gateway-api.sigs.k8s.io/api-types/httproute/) is typically deployed by the [application developer](https://gateway-api.sigs.k8s.io/docs/concepts/roles-and-personas/#roles-and-personas_1). To deploy the `coffee` HTTPRoute: ```yaml kubectl apply -f - < remote-kubeconfig.yaml </` form. F5 Container Ingress Services reads the list of external clusters and their kubeconfig Secrets from it, so without this value it has no way to reach cluster B. - `args.pool_member_type` must match the type of the Gateway's Service. Use `nodeport` with `NodePort`, or `cluster` with `ClusterIP`. - `args.log-as3-response=true` logs the BIG-IP response to each declaration, which is useful for troubleshooting. Confirm F5 Container Ingress Services reached BIG-IP and accepted the mode: ```shell kubectl logs -n kube-system deploy/f5-cis-f5-bigip-ctlr | grep -E "authn/login|multi-cluster-mode" ``` The log shows a successful login and the configured multi-cluster mode: ```text [DEBUG] [BIGIP] postConfig request: POST https://192.0.2.10:443/mgmt/shared/authn/login 200 OK [DEBUG] Multi-cluster-mode: standalone, local cluster name: local ``` ## Set up for both clusters Apply the following resources to **both** clusters. Use the same Gateway name and the same listeners in each, so the data plane Services carry matching labels and expose the same ports. F5 Container Ingress Services builds one virtual server per Service port and pools every cluster behind that virtual server. ### Install the custom resource definitions Install the F5 Container Ingress Services custom resource definitions in **both** clusters, including cluster B, which does not run F5 Container Ingress Services: ```shell kubectl apply -f https://raw.githubusercontent.com/F5Networks/k8s-bigip-ctlr/v/docs/config_examples/customResourceDefinitions/customresourcedefinitions.yml ``` Confirm the `IngressLink` custom resource definition is installed: ```shell kubectl get crd ingresslinks.cis.f5.com ``` ```text NAME CREATED AT ingresslinks.cis.f5.com 2026-08-05T01:40:54Z ``` With external load balancer support enabled, NGINX Gateway Fabric watches `IngressLink` resources on startup in every cluster it runs in. A cluster without the custom resource definition leaves the control plane unable to start, and its Pod restarts continuously: ```text no matches for kind "IngressLink" in version "cis.f5.com/v1" failed to start control loop: failed to wait for provisioner-IngressLink caches to sync ``` ### Install NGINX Gateway Fabric [Install](/ngf/install/) NGINX Gateway Fabric with external load balancer support enabled. Using Helm, set the `nginxGateway.externalLoadBalancer.enable=true` value. Using Kubernetes manifests, add the `--external-load-balancer` flag to the `nginx-gateway` container arguments. ### Create a Gateway Create an `NginxProxy` resource named `gatewaylink-proxy`, which exposes the readiness probe: ```yaml kubectl apply -f - <", v.get("rules", "no rules"))' ``` If the iRule is attached, confirm the trusted addresses: ```shell kubectl exec $NGINX_POD_NAME -c nginx -- grep set_real_ip_from /etc/nginx/conf.d/http.conf ``` Set `trustedAddresses` on the `NginxProxy` resource to the subnet of the IP address which the BIG-IP system uses to send traffic to NGINX. ### A configured field has no effect Kubernetes discards fields that are not in the installed custom resource definition schema without reporting an error, so both controllers report success while the field never arrives. Check where the field stops: ```shell export FIELD_NAME="ipamLabel" kubectl get crd ingresslinks.cis.f5.com -o yaml | grep -A5 "$FIELD_NAME" kubectl logs -n nginx-gateway deploy/ngf-nginx-gateway-fabric | grep "unknown field" kubectl get ingresslink gateway-nginx -o jsonpath='{.spec}' | python3 -m json.tool ``` An `unknown field` message means the installed custom resource definition is older than the NGINX Gateway Fabric release. Install a matching version. ### The control plane restarts continuously in cluster B The NGINX Gateway Fabric Pod reports `CrashLoopBackOff`, and its logs end with a cache sync failure: ```text no matches for kind "IngressLink" in version "cis.f5.com/v1" failed to start control loop: failed to wait for provisioner-IngressLink caches to sync ``` The F5 Container Ingress Services custom resource definitions are missing from that cluster. With external load balancer support enabled, NGINX Gateway Fabric watches `IngressLink` resources on startup, whether or not Container Ingress Services runs there. - Install the custom resource definitions in the affected cluster: ```shell kubectl apply -f https://raw.githubusercontent.com/F5Networks/k8s-bigip-ctlr/v/docs/config_examples/customResourceDefinitions/customresourcedefinitions.yml ``` - Delete the Pod so it restarts immediately rather than waiting out its backoff: ```shell kubectl delete pod -n nginx-gateway -l app.kubernetes.io/name=nginx-gateway-fabric ``` ### The remote cluster has no pool Only `_local` pools exist on BIG-IP, and traffic never reaches cluster B. F5 Container Ingress Services could not load the cluster B kubeconfig, so it has no endpoints to pool. Start with its log, which names the cause directly: ```shell kubectl logs -n kube-system deploy/f5-cis-f5-bigip-ctlr | grep -i "MultiCluster" ``` - Confirm the Secret exists under the name and namespace the `extended-spec-config` ConfigMap refers to. A Secret created under a different name is reported as missing: ```text error occurred while fetching Secret: remote-kubeconfig for the cluster: remote, Error: secrets "remote-kubeconfig" not found ``` - Confirm the token is still valid. A token issued for a ServiceAccount that has since been deleted and recreated is rejected: ```text the server has asked for the client to provide credentials ``` Regenerate the kubeconfig on cluster B and recreate the Secret. - Confirm the Secret holding the cluster B kubeconfig parses. A kubeconfig with broken indentation is stored without complaint and fails only when F5 Container Ingress Services loads it: ```shell kubectl get secret remote-kubeconfig -n kube-system -o jsonpath='{.data.kubeconfig}' | base64 -d > /tmp/check.yaml KUBECONFIG=/tmp/check.yaml kubectl get nodes ``` The command lists the cluster B nodes. An error such as `mapping values are not allowed in this context` means the file is malformed, so regenerate it and recreate the Secret. - Confirm the `clusterName` in the extended spec ConfigMap matches the `clusterName` under `remoteClusters` in the `ExternalLoadBalancer`. - Restart F5 Container Ingress Services after replacing the Secret, because it reads the kubeconfig at startup: ```shell kubectl rollout restart deploy/f5-cis-f5-bigip-ctlr -n kube-system ``` ### The remote pool member is down The `_remote` pool exists but its member reports `offline`, so all traffic goes to the local cluster. - Confirm the cluster B data plane Service exposes the same ports as cluster A. A missing HTTPS listener leaves nothing listening on the 443 NodePort: ```shell kubectl get svc gateway-nginx -o jsonpath='{range .spec.ports[*]}{.name} {.port}:{.nodePort}{"\n"}{end}' ``` - Confirm the `nginx-tls` Secret exists in cluster B. Without it the HTTPS listener is not programmed and NGINX never listens on 443. - Restart the data plane after creating a certificate. NGINX does not load a certificate created after the Pod started, and the Gateway reports every condition as healthy while the listener is missing from the configuration: ```shell kubectl rollout restart deploy/gateway-nginx -n default ``` ### Requests fail with a connection reset A request through BIG-IP fails with `Recv failure: Connection reset by peer`. NGINX Gateway Fabric enables HTTP/2 by default. BIG-IP SSL profiles do not negotiate HTTP/2 unless configured to, so BIG-IP sends HTTP/1.1 into a connection NGINX set up for HTTP/2. - Set `disableHTTP2: true` on the `NginxProxy` resource, or use a BIG-IP SSL profile with HTTP/2 enabled. Confirm the setting reached the data plane rather than trusting the resource: ```shell kubectl exec $NGINX_POD_NAME -c nginx -- grep "listen 443" /etc/nginx/conf.d/http.conf ``` The absence of an `http2` token on the `listen` line means HTTP/2 is off. ## Remove the configuration Delete the `ExternalLoadBalancer` so F5 Container Ingress Services deletes the objects it created on BIG-IP: ```shell kubectl delete externalloadbalancer gateway-elb ``` Confirm the virtual servers are gone: ```shell curl -sku "$BIGIP_USERNAME:$BIGIP_PASSWORD" "https://$BIGIP_ADDRESS/mgmt/tm/ltm/virtual" | python3 -m json.tool | grep fullPath ``` ## References - [F5 IngressLink documentation](https://clouddocs.f5.com/containers/latest/userguide/ingresslink/): the F5 Container Ingress Services resource that NGINX Gateway Fabric generates. - [F5 Application Services 3 Extension reference](https://clouddocs.f5.com/products/extensions/f5-appsvcs-extension/latest/refguide/schema-reference.html): the declaration format F5 Container Ingress Services posts to BIG-IP. - [F5 Container Ingress Services](https://github.com/F5Networks/k8s-bigip-ctlr): the F5 Container Ingress Services source and custom resource definitions. - [F5 IPAM Controller](https://github.com/F5Networks/f5-ipam-controller): allocates virtual server addresses when using the `ipamLabel` field instead of a fixed address. - [F5 BIG-IP iControl REST API](https://clouddocs.f5.com/api/icontrol-rest/): the API used by the `curl` commands in this guide. - [BIG-IP Virtual Edition on Amazon Web Services](https://clouddocs.f5.com/cloud/public/v1/aws_index.html) - [BIG-IP Virtual Edition on Microsoft Azure](https://clouddocs.f5.com/cloud/public/v1/azure_index.html) - [BIG-IP Virtual Edition on Google Cloud Platform](https://clouddocs.f5.com/cloud/public/v1/google_index.html) - [F5 Container Ingress Services multi-cluster guide](https://clouddocs.f5.com/containers/latest/userguide/multicluster/): multi-cluster deployment topologies. -------------------------------------------------------------------------------- title: "Gateway API compatibility" url: https://docs.nginx.com/nginx-gateway-fabric/overview/gateway-api-compatibility/index.md -------------------------------------------------------------------------------- This page describes which Gateway API resources NGINX Gateway Fabric supports and to which level. For information about NGINX Gateway Fabric and Gateway API version compatibility, view the [Technical specifications](/ngf/overview/technical-specifications.md) topic. ## Summary **Note:** You can view the [Milestone Roadmap](https://github.com/orgs/nginx/projects/10/views/5) in the NGINX Gateway Fabric GitHub project to see what API resources will be supported in upcoming releases. | Resource | Core Support Level | Extended Support Level | Implementation-Specific Support Level | API Version | API Release Channel | |---------------------------------------|---------------------|------------------------|---------------------------------------|-------------|---------------------| | [GatewayClass](/ngf/overview/gateway-api-compatibility.md#gatewayclass) | Supported | Not supported | Supported | v1 | Standard | | [Gateway](/ngf/overview/gateway-api-compatibility.md#gateway) | Supported | Partially supported | Not supported | v1 | Standard | | [HTTPRoute](/ngf/overview/gateway-api-compatibility.md#httproute) | Supported | Partially supported | Not supported | v1 | Standard | | [GRPCRoute](/ngf/overview/gateway-api-compatibility.md#grpcroute) | Supported | Partially supported | Not supported | v1 | Standard | | [ReferenceGrant](/ngf/overview/gateway-api-compatibility.md#referencegrant) | Supported | N/A | Not supported | v1 | Standard | | [TLSRoute](/ngf/overview/gateway-api-compatibility.md#tlsroute) | Supported | Not supported | Not supported | v1 | Standard | | [TCPRoute](/ngf/overview/gateway-api-compatibility.md#tcproute) | Supported | Supported | Not supported | v1 | Standard | | [UDPRoute](/ngf/overview/gateway-api-compatibility.md#udproute) | Supported | Supported | Not supported | v1 | Standard | | [BackendTLSPolicy](/ngf/overview/gateway-api-compatibility.md#backendtlspolicy) | Partially supported | Supported | Partially supported | v1 | Standard | | [Custom policies](/ngf/overview/gateway-api-compatibility.md#custom-policies) | N/A | N/A | Supported | N/A | N/A | ## Terminology Gateway API features has three [support levels](https://gateway-api.sigs.k8s.io/concepts/conformance/#2-support-levels): Core, Extended and Implementation-specific. We use the following terms to describe the support status for each level and resource field: - _Supported_. The resource or field is fully supported. - _Partially supported_. The resource or field is supported partially, with limitations. It will become fully supported in future releases. - _Not supported_. The resource or field is not yet supported. It will become partially or fully supported in future releases. **Note:** It's possible that NGINX Gateway Fabric will never support some resources or fields of the Gateway API. They will be documented on a case by case basis. Please note that while we make every effort to reflect the support status of experimental fields in our code and documentation, there may be instances where this is not explicitly indicated. Support for such fields is provided on a best-effort basis. ## Resources Each resource below includes the support status of their corresponding fields. For a description of each field, visit the [Gateway API documentation](https://gateway-api.sigs.k8s.io/references/spec/). ### GatewayClass | Resource | Core Support Level | Extended Support Level | Implementation-Specific Support Level | API Version | API Release Channel | |--------------|--------------------|------------------------|---------------------------------------|-------------|---------------------| | GatewayClass | Supported | Not supported | Supported | v1 | Standard | NGINX Gateway Fabric supports a single GatewayClass resource configured with the `--gatewayclass` flag of the [controller](/ngf/reference/cli-help.md#controller) command. **Fields**: - `spec` - `controllerName` - supported. - `parametersRef` - NginxProxy resource supported. - `description` - supported. - `status` - `conditions` - supported (Condition/Status/Reason): - `Accepted/True/Accepted` - `Accepted/True/InvalidParameters` - `Accepted/False/UnsupportedVersion` - `Accepted/False/GatewayClassConflict` - `SupportedVersion/True/SupportedVersion` - `SupportedVersion/False/UnsupportedVersion` - `ResolvedRefs/True/ResolvedRefs` - `ResolvedRefs/False/ParametersRefNotFound` - `ResolvedRefs/False/ParametersRefInvalid` - `supportedFeatures` - supported. ### Gateway | Resource | Core Support Level | Extended Support Level | Implementation-Specific Support Level | API Version | API Release Channel | |----------|--------------------|------------------------|---------------------------------------|-------------|---------------------| | Gateway | Supported | Partially supported | Not supported | v1 | Standard | NGINX Gateway Fabric supports multiple Gateway resources. The Gateway resources must reference NGINX Gateway Fabric's corresponding GatewayClass. See the [controller](/ngf/reference/cli-help.md#controller) command for more information. **Fields**: - `spec` - `gatewayClassName`: Supported. - `infrastructure`: Supported. - `parametersRef`: NginxProxy resource supported. - `labels`: Supported. - `annotations`: Supported. - `listeners` - `name`: Supported. - `hostname`: Supported. - `port`: Supported. - `protocol`: Supported. - `tls` - `mode`: Supported. - `certificateRefs` - The TLS certificate and key must be stored in a Secret resource of type `kubernetes.io/tls`. - `options`: The options `nginx.org/ssl-protocols`, `nginx.org/ssl-ciphers`, `nginx.org/ssl-prefer-server-ciphers`, `nginx.org/ssl-session-cache`, `nginx.org/ssl-session-timeout` and `nginx.org/ssl-ecdh-curve` are supported. See [ngx_http_ssl_module](https://nginx.org/en/docs/http/ngx_http_ssl_module.html) for more information. - `allowedRoutes`: Supported. - `addresses`: Valid IPAddresses will be added to the `externalIP` field in the related Services fronting NGINX. Users should ensure that the IP Family of the address matches the IP Family set in the NginxProxy resource (default is dual, meaning both IPv4 and IPv6), otherwise there may be networking issues. - `type`: Partially supported. Allowed value: `IPAddress`. - `value`: Partially supported. Dynamic address allocation when value is unspecified is not supported. - `TLS`: - `frontend`: - `default`: - `caCertificateRefs`: Supported. - `perPort`: - `caCertificateRefs`: Supported. - `backend`: - `clientCertificateRef`: Supported. - `allowedListeners`: Supported. - `defaultScope`: Not supported. - `status` - `addresses`: Partially supported (LoadBalancer and ClusterIP). - `attachedListenerSets`: Supported. - `conditions`: Supported (Condition/Status/Reason): - `Accepted/True/Accepted` - `Accepted/True/ListenersNotValid` - `Accepted/True/InvalidParameters` - `Accepted/True/UnsupportedField` - `Accepted/False/ListenersNotValid` - `Accepted/False/Invalid` - `Accepted/False/UnsupportedValue` - `Accepted/False/UnsupportedAddress` - `Programmed/True/Programmed` - `Programmed/False/Invalid` - `Programmed/False/UnsupportedValue` - `Programmed/False/AddressNotUsable` - `Programmed/False/AddressNotAssigned` - `ResolvedRefs/True/ResolvedRefs` - `ResolvedRefs/False/ParametersRefNotFound` - `ResolvedRefs/False/ParametersRefInvalid` - `ResolvedRefs/False/InvalidClientCertificateRef` - `ResolvedRefs/False/RefNotPermitted` - `InsecureFrontendValidationMode/True/ConfigurationChanged` - `listeners` - `name`: Supported. - `supportedKinds`: Supported. - `attachedRoutes`: Supported. - `conditions`: Supported (Condition/Status/Reason): - `Accepted/True/Accepted` - `Accepted/False/UnsupportedProtocol` - `Accepted/False/InvalidCertificateRef` - `Accepted/False/NoValidCACertificate` - `Accepted/False/ProtocolConflict` - `Accepted/False/HostnameConflict` - `Accepted/False/UnsupportedValue` - `Accepted/False/RefNotPermitted` - `Programmed/True/Programmed` - `Programmed/False/Invalid` - `ResolvedRefs/True/ResolvedRefs` - `ResolvedRefs/False/InvalidCertificateRef` - `ResolvedRefs/False/InvalidCACertificateRef` - `ResolvedRefs/False/InvalidRouteKinds` - `ResolvedRefs/False/RefNotPermitted` - `Conflicted/True/ProtocolConflict` - `Conflicted/True/HostnameConflict` - `Conflicted/False/NoConflicts` - `OverlappingTLSConfig/True/OverlappingHostnames` ### HTTPRoute | Resource | Core Support Level | Extended Support Level | Implementation-Specific Support Level | API Version | API Release Channel | |-----------|--------------------|------------------------|---------------------------------------|-------------|---------------------| | HTTPRoute | Supported | Partially supported | Not supported | v1 | Standard | **Fields**: - `spec` - `parentRefs`: Supported. - `hostnames`: Supported. - `rules` - `matches` - `path`: Partially supported. Only `PathPrefix` and `Exact` types. - `headers`: Supported. - `queryParams`: Supported. - `method`: Supported. - `filters` - `type`: Supported. - `requestRedirect`: Supported. If multiple filters are configured, NGINX Gateway Fabric will choose the first and ignore the rest. Incompatible with `urlRewrite`. - `requestHeaderModifier`: Supported. If multiple filters are configured, NGINX Gateway Fabric will choose the first and ignore the rest. - `urlRewrite`: Supported. If multiple filters are configured, NGINX Gateway Fabric will choose the first and ignore the rest. Incompatible with `requestRedirect`. - `responseHeaderModifier`: Supported. If multiple filters are configured, NGINX Gateway Fabric will choose the first and ignore the rest. - `requestMirror`: Supported. Multiple mirrors can be specified. Percent and fraction-based mirroring are supported. - `cors`: Supported. If multiple filters are configured, NGINX Gateway Fabric will choose the first and ignore the rest. - `extensionRef`: Supported for SnippetsFilters and AuthenticationFilters. - `externalAuth`: Supported. If multiple filters are configured, NGINX Gateway Fabric uses the first and ignores the rest. - `backendRefs`: Partially supported. Backend ref `filters` are not supported. - `name`: Not supported. - `timeouts`: Not supported. - `retry`: Not supported. - `sessionPersistence`: Supported (NGINX Plus). - `status` - `parents` - `parentRef`: Supported. - `controllerName`: Supported. - `conditions`: Partially supported. Supported (Condition/Status/Reason): - `Accepted/True/Accepted` - `Accepted/True/UnsupportedField` - `Accepted/False/NoMatchingListenerHostname` - `Accepted/False/NoMatchingParent` - `Accepted/False/NotAllowedByListeners` - `Accepted/False/UnsupportedValue` - `Accepted/False/InvalidListener` - `Accepted/False/HostnameConflict` - `Accepted/False/MultipleRoutesOnListener` - `Accepted/False/InvalidGateway` - `ResolvedRefs/True/ResolvedRefs` - `ResolvedRefs/False/InvalidKind` - `ResolvedRefs/False/RefNotPermitted` - `ResolvedRefs/False/BackendNotFound` - `ResolvedRefs/False/UnsupportedValue` - `ResolvedRefs/False/InvalidIPFamily` - `ResolvedRefs/False/UnsupportedProtocol` - `ResolvedRefs/False/InvalidFilter` - `ResolvedRefs/False/InvalidInferencePool` - `PartiallyInvalid/True/UnsupportedValue` **Note:** If `name`, `timeouts`, or `retry` are defined for a HTTPRoute rule, they will be ignored and rule still will be created. ### GRPCRoute | Resource | Core Support Level | Extended Support Level | Implementation-Specific Support Level | API Version | API Release Channel | |-----------|--------------------|------------------------|---------------------------------------|-------------|---------------------| | GRPCRoute | Supported | Partially supported | Not supported | v1 | Standard | **Fields**: - `spec` - `parentRefs`: Supported. - `hostnames`: Supported. - `rules` - `matches` - `method`: Partially supported. Only `Exact` type with both `method.service` and `method.method` specified. - `headers`: Supported - `filters` - `type`: Supported. - `requestHeaderModifier`: Supported. If multiple filters are configured, NGINX Gateway Fabric will choose the first and ignore the rest. - `responseHeaderModifier`: Supported. If multiple filters are configured, NGINX Gateway Fabric will choose the first and ignore the rest. - `requestMirror`: Supported. Multiple mirrors can be specified. - `extensionRef`: Supported for SnippetsFilters. - `backendRefs`: Partially supported. Backend ref `filters` are not supported. - `name`: Not supported. - `sessionPersistence`: Supported (NGINX Plus). - `status` - `parents` - `parentRef`: Supported. - `controllerName`: Supported. - `conditions`: Partially supported. Supported (Condition/Status/Reason): - `Accepted/True/Accepted` - `Accepted/True/UnsupportedField` - `Accepted/False/NoMatchingListenerHostname` - `Accepted/False/NoMatchingParent` - `Accepted/False/NotAllowedByListeners` - `Accepted/False/UnsupportedValue` - `Accepted/False/InvalidListener` - `Accepted/False/HostnameConflict` - `Accepted/False/MultipleRoutesOnListener` - `Accepted/False/InvalidGateway` - `Accepted/False/UnsupportedConfiguration` - `ResolvedRefs/True/ResolvedRefs` - `ResolvedRefs/False/InvalidKind` - `ResolvedRefs/False/RefNotPermitted` - `ResolvedRefs/False/BackendNotFound` - `ResolvedRefs/False/UnsupportedValue` - `ResolvedRefs/False/InvalidIPFamily` - `ResolvedRefs/False/UnsupportedProtocol` - `ResolvedRefs/False/InvalidFilter` - `PartiallyInvalid/True/UnsupportedValue` **Note:** If `name` is defined for a GRPCRoute rule, it will be ignored and rule still will be created. ### ReferenceGrant | Resource | Core Support Level | Extended Support Level | Implementation-Specific Support Level | API Version | API Release Channel | |----------------|--------------------|------------------------|---------------------------------------|-------------|---------------------| | ReferenceGrant | Supported | N/A | Not supported | v1 | Standard | Fields: - `spec` - `to` - `group` - supported. - `kind` - supported. - `name`- supported. - `from` - `group` - supported. - `kind` - supported. - `namespace`- supported. ### TLSRoute | Resource | Core Support Level | Extended Support Level | Implementation-Specific Support Level | API Version | API Release Channel | |----------|--------------------|------------------------|---------------------------------------|-------------|---------------------| | TLSRoute | Supported | Not supported | Not supported | v1 | Standard | **Fields**: - `spec` - `parentRefs`: Supported. - `hostnames`: Supported. - `rules` - `backendRefs`: Partially supported. Only one backend ref allowed. - `weight`: Not supported. - `status` - `parents` - `parentRef`: Supported. - `controllerName`: Supported. - `conditions`: Supported (Condition/Status/Reason): - `Accepted/True/Accepted` - `Accepted/False/NoMatchingListenerHostname` - `Accepted/False/NoMatchingParent` - `Accepted/False/NotAllowedByListeners` - `Accepted/False/UnsupportedValue` - `Accepted/False/InvalidListener` - `Accepted/False/HostnameConflict` - `ResolvedRefs/True/ResolvedRefs` - `ResolvedRefs/False/InvalidKind` - `ResolvedRefs/False/RefNotPermitted` - `ResolvedRefs/False/BackendNotFound` - `ResolvedRefs/False/UnsupportedValue` - `PartiallyInvalid/True/UnsupportedValue` ### TCPRoute | Resource | Core Support Level | Extended Support Level | Implementation-Specific Support Level | API Version | API Release Channel | |----------|--------------------|------------------------|---------------------------------------|-------------|---------------------| | TCPRoute | Supported | Supported | Not supported | v1 | Standard | ### UDPRoute | Resource | Core Support Level | Extended Support Level | Implementation-Specific Support Level | API Version | API Release Channel | |----------|--------------------|------------------------|---------------------------------------|-------------|---------------------| | UDPRoute | Supported | Supported | Not supported | v1 | Standard | ### BackendTLSPolicy | Resource | Core Support Level | Extended Support Level | Implementation-Specific Support Level | API Version | API Release Channel | |------------------|---------------------|------------------------|---------------------------------------|-------------|---------------------| | BackendTLSPolicy | Supported | Supported | Partially Supported | v1 | Standard | Fields: - `spec` - `targetRefs` - `group`: Supported. - `kind`: Supports `Service`. - `name`: Supported. - `validation` - `caCertificateRefs`: Supports single reference to a `ConfigMap` or `Secret`, with the CA certificate in a key named `ca.crt`. - `name`: Supported. - `group`: Supported. - `kind`: Supports `ConfigMap` and `Secret`. - `hostname`: Supported. - `wellKnownCertificates`: Supports `System`. This will set the CA certificate to the Alpine system root CA path `/etc/ssl/cert.pem`. NB: This option will only work if the NGINX image used is Alpine based. The NGF NGINX images are Alpine based by default. - `subjectAltNames`: Not supported. - `options`: Not supported. - `status` - `ancestors` - `ancestorRef`: Supported. - `controllerName`: Supported. - `conditions`: Partially supported. Supported (Condition/Status/Reason): - `Accepted/True/PolicyReasonAccepted` - `Accepted/False/PolicyReasonInvalid` - `Accepted/False/NoValidCACertificate` - `ResolvedRefs/True/ResolvedRefs` - `ResolvedRefs/False/InvalidCACertificateRef` - `ResolvedRefs/False/InvalidKind` **Note:** If multiple `backendRefs` are defined for a HTTPRoute rule, all the referenced Services *must* have matching BackendTLSPolicy configuration. BackendTLSPolicy configuration is considered to be matching if 1. CACertRefs reference the same ConfigMap, or 2. WellKnownCACerts are the same, and 3. Hostname is the same. ### ListenerSet | Resource | Core Support Level | Extended Support Level | Implementation-Specific Support Level | API Version | API Release Channel | |-------------|--------------------|------------------------|---------------------------------------|-------------|---------------------| | ListenerSet | Supported | Supported | Not supported | v1 | Standard | **Fields**: - `spec` - `parentRef`: Supported. - `listeners` - `name`: Supported. - `hostname`: Supported. - `port`: Supported. - `protocol`: Supported. - `tls` - `mode`: Supported. - `certificateRefs` - The TLS certificate and key must be stored in a Secret resource of type `kubernetes.io/tls`. - `options`: The options `nginx.org/ssl-protocols`, `nginx.org/ssl-ciphers`, `nginx.org/ssl-prefer-server-ciphers`, `nginx.org/ssl-session-cache`, `nginx.org/ssl-session-timeout` and `nginx.org/ssl-ecdh-curve` are supported. See [ngx_http_ssl_module](https://nginx.org/en/docs/http/ngx_http_ssl_module.html) for more information. - `allowedRoutes`: Supported. - `status` - `conditions`: Supported (Condition/Status/Reason): - `Accepted/True/Accepted` - `Accepted/True/ListenersNotValid` - `Accepted/False/NotAllowed` - `Accepted/False/ParentNotAllowed` - `Accepted/False/ListenersNotValid` - `Programmed/True/Programmed` - `Programmed/False/NotAllowed` - `Programmed/False/ParentNotAllowed` - `Programmed/False/ListenersNotValid` - `listeners` - `name`: Supported. - `supportedKinds`: Supported. - `attachedRoutes`: Supported. - `conditions`: Supported (Condition/Status/Reason): - `Accepted/True/Accepted` - `Accepted/False/UnsupportedProtocol` - `Accepted/False/InvalidCertificateRef` - `Accepted/False/ProtocolConflict` - `Accepted/False/HostnameConflict` - `Accepted/False/UnsupportedValue` - `Accepted/False/RefNotPermitted` - `Programmed/True/Programmed` - `Programmed/False/Invalid` - `ResolvedRefs/True/ResolvedRefs` - `ResolvedRefs/False/InvalidCertificateRef` - `ResolvedRefs/False/InvalidRouteKinds` - `ResolvedRefs/False/RefNotPermitted` - `Conflicted/True/ProtocolConflict` - `Conflicted/True/HostnameConflict` - `Conflicted/False/NoConflicts` - `OverlappingTLSConfig/True/OverlappingHostnames` ### Custom Policies | Resource | Core Support Level | Extended Support Level | Implementation-Specific Support Level | API Version | API Release Channel | |-----------------|--------------------|------------------------|---------------------------------------|-------------|---------------------| | Custom policies | N/A | N/A | Supported | N/A | N/A | Custom policies are NGINX Gateway Fabric-specific CRDs (Custom Resource Definitions) that support features such as tracing, and client connection settings. These important data-plane features are not part of the Gateway API specifications. While these CRDs are not part of the Gateway API, the mechanism to attach them to Gateway API resources is part of the Gateway API. See the [Policy Attachment documentation](https://gateway-api.sigs.k8s.io/references/policy-attachment/). See the [custom policies](/ngf/overview/custom-policies.md) document for more information. -------------------------------------------------------------------------------- title: "Get started" url: https://docs.nginx.com/nginx-gateway-fabric/get-started/index.md -------------------------------------------------------------------------------- **Note:** This document is for trying out NGINX Gateway Fabric, and not intended for a production environment. For standard deployments, you should read the [Install NGINX Gateway Fabric](/ngf/install/) section. This is a guide for getting started with NGINX Gateway Fabric. It explains how to: - Set up a [kind (Kubernetes in Docker)](https://kind.sigs.k8s.io/) cluster - Install [NGINX Gateway Fabric](https://github.com/nginx/nginx-gateway-fabric) with [NGINX](https://nginx.org/) - Test NGINX Gateway Fabric with an example application By following the steps in order, you will finish with a functional NGINX Gateway Fabric cluster. ## Before you begin To complete this guide, you need the following prerequisites installed: - [Go 1.16](https://go.dev/dl/) or newer, which is used by kind - [Docker](https://docs.docker.com/get-started/get-docker/), for creating and managing containers - [kind](https://kind.sigs.k8s.io/#installation-and-usage), which allows for running a local Kubernetes cluster using Docker - [kubectl](https://kubernetes.io/docs/tasks/tools/), which provides a command line interface (CLI) for interacting with Kubernetes clusters - [Helm 3.0](https://helm.sh/docs/intro/install/) or newer to install NGINX Gateway Fabric - [curl](https://curl.se/), to test the example application ## Set up a kind cluster Create the file _cluster-config.yaml_ with the following contents, noting the highlighted lines: ```yaml {linenos=true, hl_lines=[6, 9]} apiVersion: kind.x-k8s.io/v1alpha4 kind: Cluster nodes: - role: control-plane extraPortMappings: - containerPort: 31437 hostPort: 8080 protocol: TCP - containerPort: 30478 hostPort: 8443 protocol: TCP ``` **Note:** The _containerPort_ value is used to later configure a _NodePort_. Run the following command: ```shell kind create cluster --config cluster-config.yaml ``` #### Example output ```text Creating cluster "kind" ... ✓ Ensuring node image (kindest/node:v1.31.0) 🖼 ✓ Preparing nodes 📦 ✓ Writing configuration 📜 ✓ Starting control-plane 🕹️ ✓ Installing CNI 🔌 ✓ Installing StorageClass 💾 Set kubectl context to "kind-kind" You can now use your cluster with: kubectl cluster-info --context kind-kind Thanks for using kind! 😊 ``` **Note:** If you have cloned [the NGINX Gateway Fabric repository](https://github.com/nginx/nginx-gateway-fabric/tree/main), you can also create a kind cluster from the root folder with the following _make_ command: ```shell make create-kind-cluster ``` ## Install NGINX Gateway Fabric ### Add Gateway API resources Use `kubectl` to add the API resources for NGINX Gateway Fabric with the following command: ```shell kubectl kustomize "https://github.com/nginx/nginx-gateway-fabric/config/crd/gateway-api/standard?ref=v" | kubectl apply -f - ``` #### Example output ```text customresourcedefinition.apiextensions.k8s.io/gatewayclasses.gateway.networking.k8s.io created customresourcedefinition.apiextensions.k8s.io/gateways.gateway.networking.k8s.io created customresourcedefinition.apiextensions.k8s.io/grpcroutes.gateway.networking.k8s.io created customresourcedefinition.apiextensions.k8s.io/httproutes.gateway.networking.k8s.io created customresourcedefinition.apiextensions.k8s.io/referencegrants.gateway.networking.k8s.io created ``` ### Install the Helm chart Use `helm` to install NGINX Gateway Fabric, specifying the NodePort configuration that will be set on the NGINX Service when it is provisioned: ```shell helm install ngf oci://ghcr.io/nginx/charts/nginx-gateway-fabric --create-namespace -n nginx-gateway --set nginx.service.type=NodePort --set-json 'nginx.service.nodePorts=[{"port":31437,"listenerPort":80}, {"port":30478,"listenerPort":8443}]' ``` **Note:** The port value should equal the _containerPort_ value from _cluster-config.yaml_ [when you created the kind cluster](#set-up-a-kind-cluster). The _listenerPort_ value will match the port that we expose in the Gateway listener. #### Example output ```text NAME: ngf LAST DEPLOYED: Tue Apr 29 14:45:14 2025 NAMESPACE: nginx-gateway STATUS: deployed REVISION: 1 TEST SUITE: None ``` ## Create an example application In the previous section, you deployed NGINX Gateway Fabric to a local cluster. This section shows you how to deploy a simple web application to test that NGINX Gateway Fabric works. **Note:** The YAML code in the following sections can be found in the [cafe-example folder](https://github.com/nginx/nginx-gateway-fabric/tree/main/examples/cafe-example) of the GitHub repository. ### Create the application resources Run the following command to create the file _cafe.yaml_, which is then used to deploy the *coffee* application to your cluster: ```yaml cat < cafe.yaml apiVersion: apps/v1 kind: Deployment metadata: name: coffee spec: replicas: 1 selector: matchLabels: app: coffee template: metadata: labels: app: coffee spec: containers: - name: coffee image: nginxdemos/nginx-hello:plain-text ports: - containerPort: 8080 --- apiVersion: v1 kind: Service metadata: name: coffee spec: ports: - port: 80 targetPort: 8080 protocol: TCP name: http selector: app: coffee --- apiVersion: apps/v1 kind: Deployment metadata: name: tea spec: replicas: 1 selector: matchLabels: app: tea template: metadata: labels: app: tea spec: containers: - name: tea image: nginxdemos/nginx-hello:plain-text ports: - containerPort: 8080 --- apiVersion: v1 kind: Service metadata: name: tea spec: ports: - port: 80 targetPort: 8080 protocol: TCP name: http selector: app: tea EOF kubectl apply -f cafe.yaml ``` #### Example output ```text deployment.apps/coffee created service/coffee created deployment.apps/tea created service/tea created ``` Verify that the new pods are in the `default` namespace: ```shell kubectl get pods ``` #### Example output ```text NAME READY STATUS RESTARTS AGE coffee-676c9f8944-k2bmd 1/1 Running 0 9s tea-6fbfdcb95d-9lhbj 1/1 Running 0 9s ``` ### Create Gateway and HTTPRoute resources Run the following command to create the file _gateway.yaml_, which is then used to deploy a Gateway to your cluster: ```yaml cat < gateway.yaml apiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: name: gateway spec: gatewayClassName: nginx listeners: - name: http port: 80 protocol: HTTP hostname: "*.example.com" EOF kubectl apply -f gateway.yaml ``` Verify that the NGINX deployment has been provisioned: ```shell kubectl get pods ``` #### Example output ```text NAME READY STATUS RESTARTS AGE coffee-676c9f8944-k2bmd 1/1 Running 0 31s gateway-nginx-66b5d78f8f-4fmtb 1/1 Running 0 13s tea-6fbfdcb95d-9lhbj 1/1 Running 0 31s ``` Run the following command to create the file _cafe-routes.yaml_. It is then used to deploy two *HTTPRoute* resources in your cluster: one each for _/coffee_ and _/tea_. ```yaml cat < cafe-routes.yaml apiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: name: coffee spec: parentRefs: - name: gateway sectionName: http hostnames: - "cafe.example.com" rules: - matches: - path: type: PathPrefix value: /coffee backendRefs: - name: coffee port: 80 --- apiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: name: tea spec: parentRefs: - name: gateway sectionName: http hostnames: - "cafe.example.com" rules: - matches: - path: type: Exact value: /tea backendRefs: - name: tea port: 80 EOF kubectl apply -f cafe-routes.yaml ``` ### Verify the configuration You can check that all of the expected services are available using `kubectl get`: ```shell kubectl get services ``` #### Example output ```text NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE coffee ClusterIP 10.96.206.93 80/TCP 2m2s gateway-nginx NodePort 10.96.157.168 80:31437/TCP 104s kubernetes ClusterIP 10.96.0.1 443/TCP 142m tea ClusterIP 10.96.43.183 80/TCP 2m2s ``` You can also use `kubectl describe` on the new resources to check their status: ```shell kubectl describe httproutes ``` #### Example output ```text Name: coffee Namespace: default Labels: Annotations: API Version: gateway.networking.k8s.io/v1 Kind: HTTPRoute Metadata: Creation Timestamp: 2025-04-29T19:06:31Z Generation: 1 Resource Version: 12285 UID: c8055a74-b4c6-442f-b3fb-350fb88b2a7c Spec: Hostnames: cafe.example.com Parent Refs: Group: gateway.networking.k8s.io Kind: Gateway Name: gateway Section Name: http Rules: Backend Refs: Group: Kind: Service Name: coffee Port: 80 Weight: 1 Matches: Path: Type: PathPrefix Value: /coffee Status: Parents: Conditions: Last Transition Time: 2025-04-29T19:06:31Z Message: The route is accepted Observed Generation: 1 Reason: Accepted Status: True Type: Accepted Last Transition Time: 2025-04-29T19:06:31Z Message: All references are resolved Observed Generation: 1 Reason: ResolvedRefs Status: True Type: ResolvedRefs Controller Name: gateway.nginx.org/nginx-gateway-controller Parent Ref: Group: gateway.networking.k8s.io Kind: Gateway Name: gateway Namespace: default Section Name: http Events: Name: tea Namespace: default Labels: Annotations: API Version: gateway.networking.k8s.io/v1 Kind: HTTPRoute Metadata: Creation Timestamp: 2025-04-29T19:06:31Z Generation: 1 Resource Version: 12284 UID: 55aa0ab5-9b1c-4028-9bb5-4903f05bb998 Spec: Hostnames: cafe.example.com Parent Refs: Group: gateway.networking.k8s.io Kind: Gateway Name: gateway Section Name: http Rules: Backend Refs: Group: Kind: Service Name: tea Port: 80 Weight: 1 Matches: Path: Type: Exact Value: /tea Status: Parents: Conditions: Last Transition Time: 2025-04-29T19:06:31Z Message: The route is accepted Observed Generation: 1 Reason: Accepted Status: True Type: Accepted Last Transition Time: 2025-04-29T19:06:31Z Message: All references are resolved Observed Generation: 1 Reason: ResolvedRefs Status: True Type: ResolvedRefs Controller Name: gateway.nginx.org/nginx-gateway-controller Parent Ref: Group: gateway.networking.k8s.io Kind: Gateway Name: gateway Namespace: default Section Name: http Events: ``` ```shell kubectl describe gateways ``` #### Example output ```text Name: gateway Namespace: default Labels: Annotations: API Version: gateway.networking.k8s.io/v1 Kind: Gateway Metadata: Creation Timestamp: 2025-04-29T19:05:01Z Generation: 1 Resource Version: 12286 UID: 0baa6e15-55e0-405a-9e7c-de22472fc3ad Spec: Gateway Class Name: nginx Listeners: Allowed Routes: Namespaces: From: Same Hostname: *.example.com Name: http Port: 80 Protocol: HTTP Status: Addresses: Type: IPAddress Value: 10.96.157.168 Conditions: Last Transition Time: 2025-04-29T19:06:31Z Message: Gateway is accepted Observed Generation: 1 Reason: Accepted Status: True Type: Accepted Last Transition Time: 2025-04-29T19:06:31Z Message: Gateway is programmed Observed Generation: 1 Reason: Programmed Status: True Type: Programmed Listeners: Attached Routes: 2 Conditions: Last Transition Time: 2025-04-29T19:06:31Z Message: Listener is accepted Observed Generation: 1 Reason: Accepted Status: True Type: Accepted Last Transition Time: 2025-04-29T19:06:31Z Message: Listener is programmed Observed Generation: 1 Reason: Programmed Status: True Type: Programmed Last Transition Time: 2025-04-29T19:06:31Z Message: All references are resolved Observed Generation: 1 Reason: ResolvedRefs Status: True Type: ResolvedRefs Last Transition Time: 2025-04-29T19:06:31Z Message: No conflicts Observed Generation: 1 Reason: NoConflicts Status: False Type: Conflicted Name: http Supported Kinds: Group: gateway.networking.k8s.io Kind: HTTPRoute Group: gateway.networking.k8s.io Kind: GRPCRoute Events: ``` ## Test NGINX Gateway Fabric By configuring the cluster with the port `31437`, there is implicit port forwarding from your local machine to NodePort, allowing for direct communication to the NGINX Gateway Fabric service. You can use `curl` to test the new services by targeting the hostname (_cafe.example.com_) with the _/coffee_ and _/tea_ paths: ```shell curl --resolve cafe.example.com:8080:127.0.0.1 http://cafe.example.com:8080/coffee ``` #### Example output ```text Server address: 10.244.0.16:8080 Server name: coffee-676c9f8944-k2bmd Date: 29/Apr/2025:19:08:21 +0000 URI: /coffee Request ID: f34e138922171977a79b1b0d0395b97e ``` ```shell curl --resolve cafe.example.com:8080:127.0.0.1 http://cafe.example.com:8080/tea ``` #### Example output ```text Server address: 10.244.0.17:8080 Server name: tea-6fbfdcb95d-9lhbj Date: 29/Apr/2025:19:08:31 +0000 URI: /tea Request ID: 1b5c8f3a4532ea7d7510cf14ffeb27af ``` ## Next steps - [Install NGINX Gateway Fabric](/ngf/install/), for additional ways to install NGINX Gateway Fabric - [Traffic management](/ngf/traffic-management/), for more in-depth traffic management configuration - [How-to guides](/ngf/how-to/), for configuring your cluster -------------------------------------------------------------------------------- title: "Get started with F5 WAF for NGINX (HTTP)" url: https://docs.nginx.com/nginx-gateway-fabric/waf-integration/get-started-http/index.md -------------------------------------------------------------------------------- This guide walks through the complete flow of protecting traffic with F5 WAF for NGINX: deploy a sample application, compile a WAF policy, apply it to a Gateway, and verify that attacks are blocked. For an overview of WAF concepts and architecture, see [F5 WAF for NGINX overview](/ngf/waf-integration/overview.md). --- ## Before you begin - [Install](/ngf/install/helm.md) NGINX Gateway Fabric using the **NGINX Plus with WAF** tab. This sets the WAF-enabled NGINX Plus image and enables WAF globally. - Have a valid F5 WAF for NGINX subscription. F5 WAF for NGINX is a separate add-on to NGINX Plus and is not included with the NGINX Plus license. - Have NGINX Gateway Fabric configured with an `imagePullSecret` for the NGINX private container registry (`private-registry.nginx.com`), either through Helm values or deployment manifests. When a Gateway is deployed, NGINX Gateway Fabric automatically creates the registry secret in the Gateway's namespace with the naming convention `-nginx-. The bundle server Deployment in this guide references the same secret for pulling the F5 WAF compiler image, be sure to update the secret name to match your environment. --- ## Deploy the sample application Deploy the `customers` and `tea` sample applications. The `customers` app is configured to return a response containing fake sensitive data (credit card number and SSN), which is used later to demonstrate data guard masking: ```yaml kubectl apply -f - <.svc.cluster.local/attack-signatures-blocking.tgz`. --- ## Verify WAF protection ### Verify the WAF containers are running Verify that the NGINX Pod has all three containers running: ```shell kubectl get pods -l app.kubernetes.io/name=gateway-nginx ``` Each NGINX Pod should show `3/3` in the `READY` column, indicating the main NGINX container, `waf-enforcer`, and `waf-config-mgr` are all running: ```text NAME READY STATUS RESTARTS AGE gateway-nginx-7f9b8d6c4d-xxxxx 3/3 Running 0 2m ``` If a container is not starting, check its logs: ```shell kubectl logs -c nginx kubectl logs -c waf-enforcer kubectl logs -c waf-config-mgr ``` ### Check WAFPolicy status Verify the WAFPolicy has been accepted and programmed: ```shell kubectl describe wafpolicy gateway-base-protection ``` Look for three conditions in the output: ```text Status: Conditions: Message: The Policy is accepted Observed Generation: 1 Reason: Accepted Status: True Type: Accepted Message: All references are resolved Observed Generation: 1 Reason: ResolvedRefs Status: True Type: ResolvedRefs Message: Policy is programmed in the data plane Observed Generation: 1 Reason: Programmed Status: True Type: Programmed ``` If any condition is `False`, the message field describes the problem. See [Troubleshoot WAFPolicy status](/ngf/waf-integration/troubleshooting.md) for guidance. ### Test WAF protection Confirm the Gateway was assigned an IP address and reports a `Programmed=True` status with `kubectl describe`: ```shell kubectl describe gateways.gateway.networking.k8s.io gateway ``` ```text Addresses: Type: IPAddress Value: 192.0.2.1 ``` Save the public IP address and port(s) of the Gateway into shell variables: ```text GW_IP=XXX.YYY.ZZZ.III GW_PORT= ``` **Verify normal traffic flows.** Send a request to the `customers` route — the response contains the fake sensitive data from the `customers` backend: **Note:** If you have a DNS record allocated for `cafe.example.com`, you can send the request directly to that hostname, without needing to resolve. ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/customers ``` ```text Customer List: Name: John Doe Credit Card: 4111-1111-1111-1111 SSN: 123-45-6789 ``` The sensitive data passes through because the gateway-level `attack-signatures-blocking` policy only inspects inbound requests for attack patterns — it does not mask outbound response data. **Verify attacks are blocked.** Send a request with a cross-site scripting (XSS) payload: ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP "http://cafe.example.com:$GW_PORT/customers?x=" ``` The WAF detects the attack signature and rejects the request: ```text Request Rejected ... ``` **Verify the `tea` route is also protected.** Since the policy targets the Gateway, all attached routes inherit protection: ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP "http://cafe.example.com:$GW_PORT/tea?x=" ``` ```text Request Rejected ... ``` **Note:** The exact blocking response depends on your WAF policy configuration. Check the security log (stderr in this example) for a corresponding blocked event using `kubectl logs -c waf-enforcer`. --- ## Apply a route-level override In the previous step, you saw that the `customers` route returns sensitive data (credit card numbers and SSNs) in the response body. The gateway-level `attack-signatures-blocking` policy blocks inbound attacks, but does not inspect outbound responses. To protect sensitive data in responses, apply a **data guard** policy as a route-level override on the `customers` route. This policy masks credit card numbers and Social Security numbers in response bodies. The `dataguard-blocking` bundle was already compiled by the bundle server init container at startup — no additional compilation is needed. ### Apply the route-level WAFPolicy ```yaml kubectl apply -f - < **Subscriptions** to see your active subscriptions. 3. Find your NGINX products or services subscription and select the **Subscription ID** for details. 4. Download the **JSON Web Token (JWT)** from the subscription page. **Note:** The Connectivity Stack for Kubernetes JWT does not work with NGINX Plus reporting. Use a regular NGINX Plus instance JWT. ### Create the Docker Registry Secret **Note:** If you would rather pull the NGINX Plus image and push to a private registry, you can skip this specific step and instead follow [this step](/ngf/install/nginx-plus.md#pull-an-image-for-local-use). If the `nginx-gateway` namespace does not yet exist, create it: ```shell kubectl create namespace nginx-gateway ``` Create a Kubernetes `docker-registry` secret type using the contents of the JWT as the username and `none` for password (as the password is not used). The name of the docker server is `private-registry.nginx.com`. ```shell kubectl create secret docker-registry nginx-plus-registry-secret --docker-server=private-registry.nginx.com --docker-username= --docker-password=none -n nginx-gateway ``` It is important that the `--docker-username=` contains the contents of the token and is not pointing to the token itself. When you copy the contents of the JWT, ensure there are no additional characters such as extra whitespaces. This can invalidate the token, causing 401 errors when trying to authenticate to the registry. ### Create the NGINX Plus Secret Place the JWT in a file called `license.jwt`. Create a Kubernetes Secret using the contents of the JWT file. ```shell kubectl create secret generic nplus-license --from-file license.jwt -n nginx-gateway ``` You can now delete the `license.jwt` file. If you need to update the JWT at any time, update the `license.jwt` field in the Secret using `kubectl edit` and apply the changes. **Note:** For more information on why this is needed and additional configuration options, including how to report to NGINX Instance Manager instead, see the [NGINX Plus Image and JWT Requirement](/ngf/install/nginx-plus.md) document. ## Deploy NGINX Gateway Fabric ### Installing the Gateway API resources **Note:** The [Gateway API resources](https://github.com/kubernetes-sigs/gateway-api) from the standard channel must be installed before deploying NGINX Gateway Fabric. If they are already installed in your cluster, please ensure they are the correct version as supported by the NGINX Gateway Fabric - [see the Technical Specifications](https://github.com/nginx/nginx-gateway-fabric/blob/v/README.md#technical-specifications). To install the Gateway API resources, run the following: ```shell kubectl kustomize "https://github.com/nginx/nginx-gateway-fabric/config/crd/gateway-api/standard?ref=v" | kubectl apply -f - ``` **Note:** If you plan to use the `edge` version of NGINX Gateway Fabric, you can replace the version in `ref` with `main`, for example `ref=main`. Alternatively, you can install the Gateway API resources from the experimental channel. Installing Gateway API resources from the experimental channel includes everything in the standard release channel plus additional experimental resources and fields. NGINX Gateway Fabric currently supports a subset of the additional features provided by the experimental channel. To install from the experimental channel, run the following: ```shell kubectl kustomize "https://github.com/nginx/nginx-gateway-fabric/config/crd/gateway-api/experimental?ref=v" | kubectl apply -f - ``` **Note:** To learn more about what Gateway API resources NGINX Gateway Fabric currently supports, visit our [Gateway API Compatibility](/ngf/overview/gateway-api-compatibility.md) document. ### Install from the OCI registry The following steps install NGINX Gateway Fabric directly from the OCI helm registry. If you prefer, you can [install from sources](#install-from-sources) instead. #### NGINX To install the latest stable release of NGINX Gateway Fabric in the **nginx-gateway** namespace, run the following command: ```shell helm install ngf oci://ghcr.io/nginx/charts/nginx-gateway-fabric --create-namespace -n nginx-gateway ``` #### NGINX Plus **Note:** If applicable, replace the F5 Container registry `private-registry.nginx.com` with your internal registry for your NGINX Plus image, and replace `nginx-plus-registry-secret` with your Secret name containing the registry credentials. If your NGINX Plus JWT Secret has a different name than the default `nplus-license`, then define that name using the `nginx.usage.secretName` flag. To install the latest stable release of NGINX Gateway Fabric in the **nginx-gateway** namespace, run the following command: ```shell helm install ngf oci://ghcr.io/nginx/charts/nginx-gateway-fabric --set nginx.image.repository=private-registry.nginx.com/nginx-gateway-fabric/nginx-plus --set nginx.plus=true --set nginx.imagePullSecret=nginx-plus-registry-secret -n nginx-gateway ``` #### NGINX Plus with WAF **Note:** If applicable, replace the F5 Container registry `private-registry.nginx.com` with your internal registry, and replace `nginx-plus-registry-secret` with your Secret name containing the registry credentials. If your NGINX Plus JWT Secret has a different name than the default `nplus-license`, then define that name using the `nginx.usage.secretName` flag. To install the latest stable release of NGINX Gateway Fabric with F5 WAF for NGINX enabled globally across all Gateways, run the following command: ```shell helm install ngf oci://ghcr.io/nginx/charts/nginx-gateway-fabric --set nginx.image.repository=private-registry.nginx.com/nginx-gateway-fabric/nginx-plus-f5waf --set nginx.plus=true --set nginx.config.waf.enable=true --set nginx.imagePullSecret=nginx-plus-registry-secret --create-namespace -n nginx-gateway ``` `ngf` is the name of the release, and can be changed to any name you want. This name is added as a prefix to the Deployment name. If you want the latest version from the **main** branch, add `--version 0.0.0-edge` to your install command. To wait for the Deployment to be ready, you can either add the `--wait` flag to the `helm install` command, or run the following after installing: ```shell kubectl wait --timeout=5m -n nginx-gateway deployment/ngf-nginx-gateway-fabric --for=condition=Available ``` ### Install from sources {#install-from-sources} If you prefer to install directly from sources, instead of through the OCI helm registry, use the following steps. ```shell helm pull oci://ghcr.io/nginx/charts/nginx-gateway-fabric --untar cd nginx-gateway-fabric ``` For the latest version from the **main** branch, add _--version 0.0.0-edge_ to your pull command. #### NGINX To install the chart into the **nginx-gateway** namespace, run the following command: ```shell helm install ngf . --create-namespace -n nginx-gateway ``` #### NGINX Plus **Note:** If applicable, replace the F5 Container registry `private-registry.nginx.com` with your internal registry for your NGINX Plus image, and replace `nginx-plus-registry-secret` with your Secret name containing the registry credentials. If your NGINX Plus JWT Secret has a different name than the default `nplus-license`, then define that name using the `nginx.usage.secretName` flag. To install the chart into the **nginx-gateway** namespace, run the following command: ```shell helm install ngf . --set nginx.image.repository=private-registry.nginx.com/nginx-gateway-fabric/nginx-plus --set nginx.plus=true --set nginx.imagePullSecret=nginx-plus-registry-secret -n nginx-gateway ``` #### NGINX Plus with WAF **Note:** If applicable, replace the F5 Container registry `private-registry.nginx.com` with your internal registry, and replace `nginx-plus-registry-secret` with your Secret name containing the registry credentials. If your NGINX Plus JWT Secret has a different name than the default `nplus-license`, then define that name using the `nginx.usage.secretName` flag. To install the chart into the **nginx-gateway** namespace with F5 WAF for NGINX enabled globally across all Gateways, run the following command: ```shell helm install ngf . --set nginx.image.repository=private-registry.nginx.com/nginx-gateway-fabric/nginx-plus-f5waf --set nginx.plus=true --set nginx.config.waf.enable=true --set nginx.imagePullSecret=nginx-plus-registry-secret -n nginx-gateway ``` `ngf` is the name of the release, and can be changed to any name you want. This name is added as a prefix to the Deployment name. To wait for the Deployment to be ready, you can either add the `--wait` flag to the `helm install` command, or run the following after installing: ```shell kubectl wait --timeout=5m -n nginx-gateway deployment/ngf-nginx-gateway-fabric --for=condition=Available ``` ### Custom installation options #### Service type By default, the NGINX Gateway Fabric control plane deploys a LoadBalancer Service when a Gateway is created. To use a NodePort Service instead: ```shell helm install ngf oci://ghcr.io/nginx/charts/nginx-gateway-fabric --create-namespace -n nginx-gateway --set nginx.service.type=NodePort ``` #### Experimental features We support a subset of the additional features provided by the Gateway API experimental channel. To enable the experimental features of Gateway API which are supported by NGINX Gateway Fabric: ```shell helm install ngf oci://ghcr.io/nginx/charts/nginx-gateway-fabric --create-namespace -n nginx-gateway --set nginxGateway.gwAPIExperimentalFeatures.enable=true ``` **Note:** Requires the Gateway APIs installed from the experimental channel. #### Watch subset of namespaces By default, the NGINX Gateway Fabric controller watches resources in all namespaces. To restrict the list of namespaces, you can specify `--set "nginxGateway.watchNamespaces={x,y,z}"` with your namespace names when installing. #### Examples You can find several examples of configuration options of the `values.yaml` file in the [helm examples](https://github.com/nginx/nginx-gateway-fabric/tree/v/examples/helm) directory. ### Access NGINX Gateway Fabric When NGINX Gateway Fabric is installed, it provisions a ClusterIP Service used only for internal communication between the control plane and data planes. To deploy NGINX itself and get a LoadBalancer Service, you should follow the [Deploy a Gateway for data plane instances](/ngf/install/deploy-data-plane.md) instructions. ## Uninstall NGINX Gateway Fabric Follow these steps to uninstall NGINX Gateway Fabric and Gateway API from your Kubernetes cluster: 1. **Uninstall NGINX Gateway Fabric:** - To uninstall NGINX Gateway Fabric, run: ```shell helm uninstall ngf -n nginx-gateway ``` If needed, replace `ngf` with your chosen release name. 2. **Remove namespace and CRDs:** - To remove the **nginx-gateway** namespace and its custom resource definitions (CRDs), run: ```shell kubectl delete ns nginx-gateway kubectl delete -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v/deploy/crds.yaml ``` 3. **Remove the Gateway API resources:** - **Note:** This step will remove all corresponding custom resources in your entire cluster, across every namespace. Ensure you don't have any custom resources you need to keep, and confirm that there are no other Gateway API implementations active in your cluster. To uninstall the Gateway API resources, run the following: ```shell kubectl kustomize "https://github.com/nginx/nginx-gateway-fabric/config/crd/gateway-api/standard?ref=v" | kubectl delete -f - ``` #### Example output ```text customresourcedefinition.apiextensions.k8s.io "backendtlspolicies.gateway.networking.k8s.io" deleted customresourcedefinition.apiextensions.k8s.io "gatewayclasses.gateway.networking.k8s.io" deleted customresourcedefinition.apiextensions.k8s.io "gateways.gateway.networking.k8s.io" deleted customresourcedefinition.apiextensions.k8s.io "grpcroutes.gateway.networking.k8s.io" deleted customresourcedefinition.apiextensions.k8s.io "httproutes.gateway.networking.k8s.io" deleted customresourcedefinition.apiextensions.k8s.io "referencegrants.gateway.networking.k8s.io" deleted ``` If you installed the Gateway APIs from the experimental channel, run the following instead: ```shell kubectl kustomize "https://github.com/nginx/nginx-gateway-fabric/config/crd/gateway-api/experimental?ref=v" | kubectl delete -f - ``` ## Next steps - [Deploy a Gateway for data plane instances](/ngf/install/deploy-data-plane.md) - [Routing traffic to applications](/ngf/traffic-management/basic-routing.md) For a full list of the Helm Chart configuration parameters, read [the NGINX Gateway Fabric Helm Chart](https://github.com/nginx/nginx-gateway-fabric/blob/v/charts/nginx-gateway-fabric/README.md#configuration). -------------------------------------------------------------------------------- title: "Monitoring with Prometheus and Grafana" url: https://docs.nginx.com/nginx-gateway-fabric/monitoring/prometheus/index.md -------------------------------------------------------------------------------- This document describes how to monitor NGINX Gateway Fabric using Prometheus and Grafana. It explains installation and configuration, as well as what metrics are available. ## Overview NGINX Gateway Fabric metrics are displayed in [Prometheus](https://prometheus.io/) format. These metrics are served through a metrics server orchestrated by the controller-runtime package on HTTP port `9113`. When installed, Prometheus automatically scrapes this port and collects metrics. [Grafana](https://grafana.com/) can be used for rich visualization of these metrics. **Note:** Metrics are served over HTTP by default. Enabling HTTPS will secure the metrics endpoint with a self-signed certificate. When using HTTPS, adjust the Prometheus Pod scrape settings by adding the `insecure_skip_verify` flag to handle the self-signed certificate. For further details, refer to the [Prometheus documentation](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#tls_config). ## Installing Prometheus and Grafana **Note:** These installations are for demonstration purposes and have not been tuned for a production environment. ### Prometheus ```shell helm repo add prometheus-community https://prometheus-community.github.io/helm-charts helm repo update helm install prometheus prometheus-community/prometheus -n monitoring --create-namespace --set server.global.scrape_interval=15s ``` Once running, you can access the Prometheus dashboard by using port-forwarding in the background: ```shell kubectl port-forward -n monitoring svc/prometheus-server 9090:80 & ``` Visit [http://127.0.0.1:9090](http://127.0.0.1:9090) to view the dashboard. ### Grafana ```shell helm repo add grafana-community https://grafana-community.github.io/helm-charts helm repo update helm install grafana grafana-community/grafana -n monitoring --create-namespace ``` Once running, you can access the Grafana dashboard by using port-forwarding in the background: ```shell kubectl port-forward -n monitoring svc/grafana 3000:80 & ``` Visit [http://127.0.0.1:3000](http://127.0.0.1:3000) to view the Grafana UI. The username for login is `admin`. The password can be acquired by running: ```shell kubectl get secret -n monitoring grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo ``` #### Configuring Grafana In the Grafana UI menu, go to `Connections` then `Data sources`. Add your Prometheus service (`http://prometheus-server.monitoring.svc`) as a data source. Download the following sample dashboard and Import as a new Dashboard in the Grafana UI. - [Download: ngf-grafana-dashboard.json](ngf/grafana-dashboard.json) ## Available metrics in NGINX Gateway Fabric NGINX Gateway Fabric provides a variety of metrics for monitoring and analyzing performance. These metrics are categorized as follows: ### NGINX/NGINX Plus metrics NGINX metrics include NGINX-specific data such as the total number of accepted client connections. These metrics are collected through NGINX Agent and are reported by each NGINX Pod. NGINX Gateway Fabric currently supports a subset of all metrics available through NGINX OSS and Plus. Listed below are the supported metrics along with a small accompanying description. Metrics provided by NGINX Open Source include: - `nginx_http_connection_count_connections`: The current number of connections. - `nginx_http_connections_total`: The total number of connections, since NGINX was last started or reloaded. - `nginx_http_request_count_requests`: The total number of client requests received, since the last collection interval. - `nginx_http_requests_total`: The total number of client requests received, since NGINX was last started or reloaded. In addition to the previous metrics provided by NGINX Open Source, NGINX Plus includes: - `nginx_config_reloads_total`: The total number of NGINX config reloads. - `nginx_http_response_count_responses`: The total number of HTTP responses sent to clients since the last collection interval, grouped by status code range. - `nginx_http_response_status_responses_total`: The total number of responses since NGINX was last started or reloaded, grouped by status code range. - `nginx_http_request_discarded_requests_total`: The total number of requests completed without sending a response. - `nginx_http_request_processing_count_requests`: The number of client requests that are currently being processed. - `nginx_http_request_byte_io_bytes_total`: The total number of HTTP byte IO. - `nginx_http_upstream_keepalive_count_connections`: The current number of idle keepalive connections per HTTP upstream. - `nginx_http_upstream_peer_connection_count_connections`: The average number of active connections per HTTP upstream peer. - `nginx_http_upstream_peer_byte_io_bytes_total`: The total number of byte IO per HTTP upstream peer. - `nginx_http_upstream_peer_count_peers`: The current count of peers on the HTTP upstream grouped by state. - `nginx_http_upstream_peer_fails_attempts_total`: The total number of unsuccessful attempts to communicate with the HTTP upstream peer. - `nginx_http_upstream_peer_header_time_milliseconds`: The average time to get the response header from the HTTP upstream peer. - `nginx_http_upstream_peer_health_checks_requests_total`: The total number of health check requests made to a HTTP upstream peer. - `nginx_http_upstream_peer_requests_total`: The total number of client requests forwarded to the HTTP upstream peer. - `nginx_http_upstream_peer_response_time_milliseconds`: The average time to get the full response from the HTTP upstream peer. - `nginx_http_upstream_peer_responses_total`: The total number of responses obtained from the HTTP upstream peer grouped by status range. - `nginx_http_upstream_peer_state_is_deployed`: Current state of an upstream peer in deployment. - `nginx_http_upstream_peer_unavailables_requests_total`: Number of times the server became unavailable for client requests (“unavail”). - `nginx_http_upstream_queue_limit_requests`: The maximum number of requests that can be in the queue at the same time. - `nginx_http_upstream_queue_overflows_responses_total`: The total number of requests rejected due to the queue overflow. - `nginx_http_upstream_queue_usage_requests`: The current number of requests in the queue. - `nginx_http_upstream_zombie_count_is_deployed`: The current number of upstream peers removed from the group but still processing active client requests. - `nginx_slab_page_free_pages`: The current number of free memory pages. - `nginx_slab_page_usage_pages`: The current number of used memory pages. - `nginx_slab_slot_allocations_total`: The number of attempts to allocate memory of specified size. - `nginx_slab_slot_free_slots`: The current number of free memory slots. - `nginx_slab_slot_usage_slots`: The current number of used memory slots. - `nginx_ssl_certificate_verify_failures_certificates_total`: The total number of SSL certificate verification failures. - `nginx_ssl_handshakes_total`: The total number of SSL handshakes. ### NGINX Gateway Fabric metrics Metrics specific to NGINX Gateway Fabric include: - `event_batch_processing_milliseconds`: Time in milliseconds to process batches of Kubernetes events. All these metrics are under the `nginx_gateway_fabric` namespace and include a `class` label set to the GatewayClass of NGINX Gateway Fabric. For example, `nginx_gateway_fabric_event_batch_processing_milliseconds_sum{class="nginx"}`. ### Controller-runtime metrics Provided by the [controller-runtime](https://github.com/kubernetes-sigs/controller-runtime) library, these metrics include: - General resource usage like CPU and memory. - Go runtime metrics such as the number of Go routines, garbage collection duration, and Go version. - Controller-specific metrics, including reconciliation errors per controller, length of the reconcile queue, and reconciliation latency. ## Change the default metrics configuration You can configure monitoring metrics for NGINX Gateway Fabric using Helm or Manifests. ### Using Helm If you're setting up NGINX Gateway Fabric with Helm, you can adjust the `metrics.*` parameters to fit your needs. For detailed options and instructions, see the [Helm README](https://github.com/nginx/nginx-gateway-fabric/blob/v/charts/nginx-gateway-fabric/README.md). ### Using Kubernetes manifests For setups using Kubernetes manifests, change the metrics configuration by editing the NGINX Gateway Fabric manifest that you want to deploy. You can find some examples in the [deploy](https://github.com/nginx/nginx-gateway-fabric/tree/v/deploy) directory. #### Disabling metrics If you need to disable metrics: 1. Set the `-metrics-disable` [command-line argument](/ngf/reference/cli-help.md) to `true` in the NGINX Gateway Fabric Pod's configuration. Remove any other `-metrics-*` arguments. 2. In the Pod template for NGINX Gateway Fabric, delete the metrics port entry from the container ports list: ```yaml - name: metrics containerPort: 9113 ``` 3. Also, remove the following annotations from the NGINX Gateway Fabric Pod template: ```yaml annotations: prometheus.io/scrape: "true" prometheus.io/port: "9113" ``` #### Changing the default port To change the default port for metrics: 1. Update the `-metrics-port` [command-line argument](/ngf/reference/cli-help.md) in the NGINX Gateway Fabric Pod's configuration to your chosen port number. 2. In the Pod template, change the metrics port entry to reflect the new port: ```yaml - name: metrics containerPort: ``` 3. Modify the `prometheus.io/port` annotation in the Pod template to match the new port: ```yaml annotations: <...> prometheus.io/port: "" <...> ``` #### Enabling HTTPS for metrics For enhanced security with HTTPS: 1. Enable HTTPS security by setting the `-metrics-secure-serving` [command-line argument](/ngf/reference/cli-help.md) to `true` in the NGINX Gateway Fabric Pod's configuration. 2. Add an HTTPS scheme annotation to the Pod template: ```yaml annotations: <...> prometheus.io/scheme: "https" <...> ``` -------------------------------------------------------------------------------- title: "Use Manifests to install NGINX Gateway Fabric with NGINX Plus" url: https://docs.nginx.com/nginx-gateway-fabric/install/manifests/plus/index.md -------------------------------------------------------------------------------- This page describes how to use Manifests to install NGINX Gateway Fabric with NGINX Plus. It explains how to install the Gateway API resources and add certificates for secure authentication, then deploy NGINX Gateway Fabric and its custom resource definitions. By following these instructions, you will finish with a functional NGINX Gateway Fabric instance for your Kubernetes cluster. **Note:** To learn which Gateway API resources NGINX Gateway Fabric currently supports, view the [Gateway API Compatibility](/ngf/overview/gateway-api-compatibility.md) topic. To install an experimental NGINX Gateway Fabric version view the [Use Manifests to install NGINX Gateway Fabric with NGINX Plus](/ngf/install/manifests/plus-experimental.md) topic. ## Before you begin To complete this guide, you will need the following pre-requisites: - An active NGINX Plus subscription (Purchased or trial) - [A supported Kubernetes version](/ngf/overview/technical-specifications.md) - A functional Kubernetes cluster ## Download your JSON web token 1. Log in to [MyF5](https://my.f5.com/manage/s/). 2. Go to **My Products & Plans** > **Subscriptions** to see your active subscriptions. 3. Find your NGINX products or services subscription and select the **Subscription ID** for details. 4. Download the **JSON Web Token (JWT)** from the subscription page. **Note:** The Connectivity Stack for Kubernetes JWT does not work with NGINX Plus reporting. Use a regular NGINX Plus instance JWT. ## Create license and registry secrets First, create the _nginx-gateway_ namespace, which is used by the Manifest files by default: ```shell kubectl create namespace nginx-gateway ``` **Note:** The commands in the rest of this document should be run in the same directory as your **license.jwt** file. JWTs are sensitive information and should be stored securely. Delete them after use to prevent unauthorized access. Once you have obtained your license JWT, create a Kubernetes secret using `kubectl create`: ```shell kubectl create -n nginx-gateway secret generic nplus-license --from-file license.jwt ``` Then create another Kubernetes secret to allow interactions with the F5 registry: ```shell kubectl create -n nginx-gateway secret docker-registry nginx-plus-registry-secret \ --docker-server=private-registry.nginx.com \ --docker-username=$(cat license.jwt) \ --docker-password=none ``` You can verify the creation of the secrets using `kubectl get`: ```shell kubectl get -n nginx-gateway secrets ``` #### Example output ```text NAME TYPE DATA AGE nginx-plus-registry-secret kubernetes.io/dockerconfigjson 1 8s nplus-license Opaque 1 21s ``` ## Install the Gateway API resources **Note:** If you have already installed Gateway API resources in your cluster, ensure they are a version [supported by NGINX Gateway Fabric](/ngf/overview/technical-specifications.md) To install the Gateway API resources, use `kubectl kustomize`: ```shell kubectl kustomize "https://github.com/nginx/nginx-gateway-fabric/config/crd/gateway-api/standard?ref=v" | kubectl apply -f - ``` #### Example output ```text customresourcedefinition.apiextensions.k8s.io/gatewayclasses.gateway.networking.k8s.io created customresourcedefinition.apiextensions.k8s.io/gateways.gateway.networking.k8s.io created customresourcedefinition.apiextensions.k8s.io/grpcroutes.gateway.networking.k8s.io created customresourcedefinition.apiextensions.k8s.io/httproutes.gateway.networking.k8s.io created customresourcedefinition.apiextensions.k8s.io/referencegrants.gateway.networking.k8s.io created ``` ## Install cert-manager Install cert-manager onto the cluster using Helm with Gateway API features enabled. - Add the Helm repository. ```shell helm repo add jetstack https://charts.jetstack.io helm repo update ``` - Install cert-manager, and enable the GatewayAPI feature gate: ```shell helm install \ cert-manager jetstack/cert-manager \ --namespace cert-manager \ --create-namespace \ --set config.apiVersion="controller.config.cert-manager.io/v1alpha1" \ --set config.kind="ControllerConfiguration" \ --set config.enableGatewayAPI=true \ --set crds.enabled=true ``` ## Add certificates for secure authentication **Note:** These steps use a self-signed issuer, which should not be used in production environments. For production environments, you should use a real [CA issuer](https://cert-manager.io/docs/configuration/ca/). First, create a CA (certificate authority) issuer: ```yaml kubectl apply -f - <= v1.18.0, the default value changed from `Never` to `Always`. certificate.cert-manager.io/nginx-gateway-ca created issuer.cert-manager.io/nginx-gateway-issuer created ``` You will then need to create a server certificate for the NGINX Gateway Fabric control plane (server): **Note:** The default service name is _nginx-gateway_, and the namespace is _nginx-gateway_, so the `dnsNames` value should be `nginx-gateway.nginx-gateway.svc`. This value becomes the name of the NGINX Gateway Fabric control plane service. ```yaml {hl_lines=[13]} kubectl apply -f - <>gw: Send request and present cert for validation gw->>client: Present certificate from Secret: cafe-secret client->>client: Validate Gateway's certificate gw->>gw: Validate client certificate with CA Secret gw->>app: Request to backend app-->>gw: Response gw-->>client: Response ``` ## Before you begin Before starting, you will need: - Administrator access to a Kubernetes cluster. - [Helm](https://helm.sh/) and [kubectl](https://kubernetes.io/docs/tasks/tools/#kubectl) must be installed locally. - [NGINX Gateway Fabric deployed](/ngf/install/) in the Kubernetes cluster. ## Install cert-manager Frontend TLS requires CA certificates for client certificate validation. This example uses [cert-manager](https://cert-manager.io/) to issue these certificates. Add the Helm repository: ```shell helm repo add jetstack https://charts.jetstack.io helm repo update ``` Install cert-manager: ```shell helm install \ cert-manager jetstack/cert-manager \ --namespace cert-manager \ --create-namespace \ --set config.apiVersion="controller.config.cert-manager.io/v1alpha1" \ --set config.kind="ControllerConfiguration" \ --set config.enableGatewayAPI=true \ --set crds.enabled=true ``` ## Create CA certificates and issuers Create a CA issuer to generate our certificates. **Note:** This example uses a `selfSigned` Issuer, which should not be used in production environments. For production environments, use a real [CA issuer](https://cert-manager.io/docs/configuration/ca/). Next, we create the following resources: 1. A self-signed issuer. 2. A CA certificate named `default-validation-ca-secret` for our **default** frontend TLS validation. 3. A CA certificate named `per-port-validation-ca-secret` for our **perPort** frontend TLS validation. 4. A CA certificate named `cafe-secret`. The HTTPS listeners on the Gateway reference this certificate and present it to the client during the TLS handshake. This is required for mutual TLS. **Note:** For the Gateway's certificate, replace `cafe.example.com` with the correct hostname for your environment ```yaml kubectl apply -f - < GW_PER_PORT= ``` ## Create HTTPRoutes Copy the YAML code below into your terminal to create HTTPRoutes to route traffic to the backend applications: ```yaml kubectl apply -f - < -- Status: Parents: Conditions: Last Transition Time: 2026-05-06T06:57:53Z Message: The Route is accepted Observed Generation: 1 Reason: Accepted Status: True Type: Accepted Last Transition Time: 2026-05-06T06:57:53Z Message: All references are resolved Observed Generation: 1 Reason: ResolvedRefs Status: True Type: ResolvedRefs Controller Name: gateway.nginx.org/nginx-gateway-controller Parent Ref: Group: gateway.networking.k8s.io Kind: Gateway Name: gateway Namespace: default Section Name: https Events: ``` ## Setup configuration test To send requests to the Gateway, you must provide a valid certificate and key signed by a valid Certificate Authority (CA). Copy the following block into your terminal to create two Certificate resources. This will create two `Secret` resources with TLS certs and keys signed by the CAs created earlier in this example. **Note:** Replace `cafe.example.com` with the correct hostname for your environment ```yaml kubectl apply -f - <` with the actual name of the pod). ```shell kubectl port-forward 8765:8765 -n ``` Afterwards, use a browser to access [http://127.0.0.1:8765/dashboard.html](http://127.0.0.1:8765/dashboard.html) to view the dashboard. The dashboard will look like this: ![image](/ngf/img/nginx-plus-dashboard.png) **Note:** The [API](https://nginx.org/en/docs/http/ngx_http_api_module.html) used by the dashboard for metrics is also accessible using the `/api` path. ### Configure dashboard access through NginxProxy To access the NGINX Plus dashboard from sources than the default `127.0.0.1`, you can use the NginxProxy resource to allow access to other IP Addresses or CIDR blocks. The following example configuration allows access to the NGINX Plus dashboard from the IP Addresses `192.0.2.8` and `192.0.2.0` and the CIDR block `198.51.100.0/24`: ```yaml apiVersion: gateway.nginx.org/v1alpha1 kind: NginxProxy metadata: name: ngf-proxy-config spec: nginxPlus: allowedAddresses: - type: IPAddress value: 192.0.2.8 - type: IPAddress value: 192.0.2.0 - type: CIDR value: 198.51.100.0/24 ``` For more information on configuring the NginxProxy resource, visit the [data plane configuration](/ngf/how-to/data-plane-configuration.md) document. -------------------------------------------------------------------------------- title: "Advanced features with NGINX Plus" url: https://docs.nginx.com/nginx-gateway-fabric/overview/nginx-plus/index.md -------------------------------------------------------------------------------- NGINX Gateway Fabric can use NGINX Open Source or NGINX Plus as its data plane. [NGINX Plus](https://www.f5.com/products/nginx/nginx-plus) is the closed source, commercial version of NGINX. Using NGINX Plus as the data plane offers additional benefits compared to the open source version. ## Benefits of NGINX Plus - **Robust metrics**: A plethora of [additional Prometheus metrics](/ngf/monitoring/prometheus.md) are available. - **Live activity monitoring**: The [NGINX Plus dashboard](/ngf/monitoring/dashboard.md) shows real-time metrics and information about your server infrastructure. - **Dynamic upstream configuration**: NGINX Plus can dynamically reconfigure upstream servers when applications in Kubernetes scale up and down, preventing the need for an NGINX reload. - **Session persistence**: NGINX Plus provides support for cookie-based session persistence, allowing client requests to be consistently routed to the same upstream pod. - **Load balancing methods**: NGINX Plus provides additional latency-aware load balancing methods `random two least_time=header` and `random two least_time=last_byte` that route traffic to the server with the least average response time among two randomly selected servers. - **JWT and OIDC Authentication**: [JSON Web Token (JWT)](/ngf/traffic-security/jwt-authentication.md) and [OpenID Connect (OIDC)](/ngf/traffic-security/oidc-authentication.md) authentication support for access control and auth delegation. - **Web Application Firewall**: NGINX Plus enables integration with [F5 WAF for NGINX](/ngf/waf-integration/overview.md) (separate add-on subscription), providing enterprise-grade protection against web exploits, injection attacks, and other OWASP Top 10 threats. WAF policies are applied at the Gateway or Route level using the `WAFPolicy` custom resource. - **Support**: With an NGINX Plus license, you can take advantage of full [support](https://my.f5.com/manage/s/article/K000140156/) from NGINX, Inc. -------------------------------------------------------------------------------- title: "Configure HTTPS termination" url: https://docs.nginx.com/nginx-gateway-fabric/traffic-management/https-termination/index.md -------------------------------------------------------------------------------- Learn how to terminate HTTPS traffic using NGINX Gateway Fabric. ## Overview In this guide, we will show how to configure HTTPS termination for your application, using an [HTTPRoute](https://gateway-api.sigs.k8s.io/reference/api-types/httproute/) redirect filter, secret, and [ReferenceGrant](https://gateway-api.sigs.k8s.io/reference/api-types/referencegrant/). **Note:** To validate client certificates using mutual TLS (mTLS), see [Securing frontend client traffic using mutual TLS](/ngf/traffic-security/client-validation.md). --- ## Before you begin - [Install](/ngf/install/) NGINX Gateway Fabric. ## Set up Create the **coffee** application in Kubernetes by copying and pasting the following block into your terminal: ```yaml kubectl apply -f - < 80/TCP 40s ``` ## Configure HTTPS termination and routing For HTTPS, the deployment requires a certificate and a private key stored in a Secret. The Secret lives in a separate namespace, so a ReferenceGrant is required to access it. cert-manager issues the certificate from a local self-signed CA and automatically creates the `cafe-secret` Secret. To create the **certificate** namespace, copy and paste the following into your terminal: ```yaml kubectl apply -f - < GW_HTTPS_PORT= ``` **Note:** In a production environment, you should have a DNS record for the external IP address that is exposed, and it should refer to the hostname that the gateway will forward for. To create the httproute resources, copy and paste the following into your terminal: ```yaml kubectl apply -f - <" password: "" EOF ``` To use a Bearer Token instead, create the Secret with a `token` key: ```yaml kubectl apply -f - < EOF ``` ### Create a gateway-level WAFPolicy The following `WAFPolicy` targets the Gateway and protects all attached routes: ```yaml kubectl apply -f - < EOF ``` ### Create a WAFPolicy The following example uses `policyObjectID` to reference the policy directly. You can use `policyName` instead if you prefer to reference by name: ```yaml kubectl apply -f - <.console.ves.volterra.io namespace: default policyObjectID: pol_12345_WTHGmDEX9qnbVjQ policyVersionID: pv_Tm__12345oWmJgwxiKlHAg auth: secretRef: name: n1c-credentials securityLogs: - destination: type: syslog syslog: server: syslog-svc.default.svc.cluster.local:514 logSource: n1cSource: url: https://.console.ves.volterra.io namespace: default profileObjectID: "lp_8s8uZxLpThWwEGF7LTn_rA" auth: secretRef: name: n1c-credentials EOF ``` Replace `` with your NGINX One Console tenant hostname. The `namespace` field refers to the NGINX One Console namespace where the policy resides. **Note:** To pin a specific policy version, set `policyVersionID`. A pinned version always resolves to the same compiled bundle. Turn off polling to avoid unnecessary network requests. If you use only `policyName` or `policyObjectID` without a version pin, NGINX Gateway Fabric fetches the latest compiled bundle on each reconciliation or poll cycle. --- ## HTTP/HTTPS server Use this option when you compile F5 WAF policies using the F5 WAF compiler CLI or a CI/CD pipeline and host the resulting bundle on an HTTP/HTTPS server. For details on using the compiler, see [Build and use the compiler tool](/waf/configure/compiler.md). For a complete walkthrough including policy compilation and a bundle server deployment, see [Get started with F5 WAF for NGINX](/ngf/waf-integration/get-started-http.md). In production environments, host compiled bundles on an HTTPS server with authentication. See [Configure WAF settings](/ngf/waf-integration/configuration.md) for details on adding credentials, custom CA certificates, and checksum verification to your `policySource`. --- ## Policy Lifecycle Management (PLM) Use this option when you manage F5 WAF policies as Kubernetes resources with Policy Lifecycle Management (PLM). With PLM, you define your security posture as `APPolicy` and `APLogConf` custom resources instead of `policySource`/`logSource`. The PLM controller compiles these resources automatically and stores the resulting bundles in in-cluster storage. NGINX Gateway Fabric fetches those bundles and deploys them to the data plane. Because PLM is event-driven, you don't need a per-`WAFPolicy` credentials Secret or polling configuration. For a comparison of PLM with the other source types, see [PLM](/ngf/waf-integration/overview.md#plm-policy-lifecycle-management). For a complete walkthrough, including PLM storage setup, defining `APPolicy`/`APLogConf` resources, and applying a `WAFPolicy`, see [Get started with F5 WAF for NGINX using PLM](/ngf/waf-integration/get-started-plm.md). --- ## Policy deployment visibility NGINX Instance Manager and NGINX One Console don't show which F5 WAF policies are deployed to NGINX Gateway Fabric. Neither console shows which compiled bundle version NGINX Gateway Fabric has fetched. This is intentional. NGINX Gateway Fabric pulls compiled bundles from the management plane and deploys them directly in Kubernetes using native manifests, not through NGINX Instance Manager or NGINX One Console. This design lets you create and compile policies, then make them available to NGINX Gateway Fabric through the API, without a console-managed deployment step. F5 plans to add policy association visibility for NGINX Instance Manager and NGINX One Console in a future release. In the meantime, use `kubectl describe wafpolicy ` to check deployment status. --- ## Security event monitoring ### NGINX Instance Manager #### Connect NGINX Gateway Fabric to F5 NGINX Instance Manager Configure NGINX Gateway Fabric to connect to NGINX Instance Manager before continuing. Follow [Connect NGINX Gateway Fabric to NGINX Instance Manager](/nim/connect-kubernetes/connect-ngf.md). #### Export security logs to F5 NGINX Instance Manager The Security Monitoring dashboard in NGINX Instance Manager doesn't show which F5 WAF policies are deployed to NGINX Gateway Fabric data planes. You can still export F5 WAF security events to NGINX Instance Manager. Exporting these events gives your security operations team visibility into blocked attacks, violations, and traffic patterns directly in the dashboard. **Note:** This integration requires NGINX Instance Manager 2.23 or later. To export these events, configure a `securityLogs` entry on the `WAFPolicy` resource. The entry sends events to the syslog listener that NGINX Agent runs inside the NGINX pod. NGINX Agent's built-in OpenTelemetry collector transforms the events and exports them to NGINX Instance Manager. ```yaml kubectl apply -f - <.console.ves.volterra.io namespace: default policyName: "ngfExample" auth: secretRef: name: n1c-credentials securityLogs: - destination: type: syslog syslog: server: localhost:1514 logSource: n1cSource: url: https://.console.ves.volterra.io namespace: default profileName: "secops_dashboard" auth: secretRef: name: n1c-credentials EOF ``` The `localhost:1514` syslog destination points to NGINX Agent's OpenTelemetry collector, which runs as a sidecar in the NGINX pod. NGINX Agent forwards the security events to NGINX One Console, where they appear in the security monitoring dashboard. **Note:** The `profileName: "secops_dashboard"` log profile must exist in your NGINX One Console namespace. Events don't appear correctly in the NGINX One Console security dashboard without this profile. --- ## See also - [F5 WAF for NGINX overview](/ngf/waf-integration/overview.md) - [Get started with F5 WAF for NGINX](/ngf/waf-integration/get-started-http.md) - [Get started with F5 WAF for NGINX using PLM](/ngf/waf-integration/get-started-plm.md) - [Configure WAF settings](/ngf/waf-integration/configuration.md) - [Troubleshoot WAFPolicy status](/ngf/waf-integration/troubleshooting.md) - [Technical specifications](/ngf/overview/technical-specifications.md) - [WAFPolicy and NginxProxy API reference](/ngf/reference/api.md) -------------------------------------------------------------------------------- title: "Install NGINX Gateway Fabric with NGINX Plus" url: https://docs.nginx.com/nginx-gateway-fabric/install/nginx-plus/index.md -------------------------------------------------------------------------------- ## Overview NGINX Gateway Fabric with NGINX Plus requires a valid JSON Web Token (JWT) to download the container image from the F5 registry. From version 1.5.0, this JWT token is also required to run NGINX Plus. This requirement is part of F5’s broader licensing program and aligns with industry best practices. The JWT will streamline subscription renewals and usage reporting, helping you manage your NGINX Plus subscription more efficiently. The [telemetry](#telemetry) data we collect helps us improve our products and services to better meet your needs. The JWT is required for validating your subscription and reporting telemetry data. For environments connected to the internet, telemetry is automatically sent to F5’s licensing endpoint. In offline environments, telemetry is routed through [NGINX Instance Manager](/nim/). Usage is reported every hour and on startup whenever NGINX is reloaded. **Note:** The following Secrets should be created in the same namespace as the NGINX Gateway Fabric control plane (default: nginx-gateway). The control plane will copy these Secrets into any namespaces where NGINX gets deployed. If you need to update the Secrets, update the originals that you created in the control plane namespace, and the control plane will propagate those updates to all duplicated Secrets. ## Set up the JWT The JWT needs to be configured before deploying NGINX Gateway Fabric. The JWT will be stored in two Kubernetes Secrets: one for downloading the NGINX Plus container image, and the other for running NGINX Plus. **Note:** For security, follow these practices with JSON Web Tokens (JWTs), passwords, and shell history: 1. **JWTs:** JWTs are sensitive information. Store them securely. Delete them after use to prevent unauthorized access. 1. **Shell history:** Commands that include JWTs or passwords are recorded in the history of your shell, in plain text. Clear your shell history after running such commands. For example, if you use bash, you can delete commands in your `~/.bash_history` file. Alternatively, you can run the `history -c` command to erase your shell history. Follow these practices to help ensure the security of your system and data. ### Download the JWT from MyF5 1. Log in to [MyF5](https://my.f5.com/manage/s/). 2. Go to **My Products & Plans** > **Subscriptions** to see your active subscriptions. 3. Find your NGINX products or services subscription and select the **Subscription ID** for details. 4. Download the **JSON Web Token (JWT)** from the subscription page. **Note:** The Connectivity Stack for Kubernetes JWT does not work with NGINX Plus reporting. Use a regular NGINX Plus instance JWT. ### Docker Registry Secret **Note:** If you would rather pull the NGINX Plus image and push to a private registry, you can skip this specific step and instead follow [this step](/ngf/install/nginx-plus.md#pull-an-image-for-local-use). If the `nginx-gateway` namespace does not yet exist, create it: ```shell kubectl create namespace nginx-gateway ``` Create a Kubernetes `docker-registry` secret type using the contents of the JWT as the username and `none` for password (as the password is not used). The name of the docker server is `private-registry.nginx.com`. ```shell kubectl create secret docker-registry nginx-plus-registry-secret --docker-server=private-registry.nginx.com --docker-username= --docker-password=none -n nginx-gateway ``` It is important that the `--docker-username=` contains the contents of the token and is not pointing to the token itself. When you copy the contents of the JWT, ensure there are no additional characters such as extra whitespaces. This can invalidate the token, causing 401 errors when trying to authenticate to the registry. Provide the name of this Secret when installing NGINX Gateway Fabric: #### Helm Specify the Secret name using the `nginx.imagePullSecret` or `nginx.imagePullSecrets` helm value. #### Manifests Specify the Secret name in the `nginx-docker-secret` command-line argument of the `nginx-gateway` container. ### NGINX Plus Secret Place the JWT in a file called `license.jwt`. Create a Kubernetes Secret using the contents of the JWT file. ```shell kubectl create secret generic nplus-license --from-file license.jwt -n nginx-gateway ``` You can now delete the `license.jwt` file. If you need to update the JWT at any time, update the `license.jwt` field in the Secret using `kubectl edit` and apply the changes. If using a name other than the default `nplus-license`, provide the name of this Secret when installing NGINX Gateway Fabric: #### Helm Specify the Secret name using the `nginx.usage.secretName` helm value. #### Manifests Specify the Secret name in the `--usage-report-secret` command-line flag on the `nginx-gateway` container. **Note:** If you are reporting to the default licensing endpoint, then you can now proceed with [installing NGINX Gateway Fabric](/ngf/install/). Otherwise, follow the steps below to configure reporting to NGINX Instance Manager. ### Reporting to NGINX Instance Manager {#nim} If you are deploying NGINX Gateway Fabric in an environment where you need to report to NGINX Instance Manager instead of the default licensing endpoint, a few extra steps may be required. First, you must specify the endpoint of your NGINX Instance Manager: #### Helm Specify the endpoint using the `nginx.usage.endpoint` helm value. #### Manifests Specify the endpoint in the `--usage-report-endpoint` command-line flag on the `nginx-gateway` container. #### CA and Client certificate/key {#nim-cert} To configure a CA cert and/or client certificate and key, a few extra steps are needed. First, you need to create two Secrets in the `nginx-gateway` namespace. The CA must live under the key `ca.crt`: ```shell kubectl -n nginx-gateway create secret generic nim-ca --from-file ca.crt ``` The client cert and key must be added to a TLS Secret: ```shell kubectl -n nginx-gateway create secret tls nim-client --cert /path/to/cert --key /path/to/key ``` #### Helm Specify the CA Secret name using the `nginx.usage.caSecretName` helm value. Specify the client Secret name using the `nginx.usage.clientSSLSecretName` helm value. #### Manifests Specify the CA Secret name in the `--usage-report-ca-secret` command-line flag on the `nginx-gateway` container. Specify the client Secret name in the `--usage-report-client-ssl-secret` command-line flag on the `nginx-gateway` container.
**Note:** Once these Secrets are created and configuration options are set, you can now [install NGINX Gateway Fabric](/ngf/install/). ## Installation flags to configure usage reporting {#flags} When installing NGINX Gateway Fabric, the following flags can be specified to configure usage reporting to fit your needs: If using Helm, the `nginx.usage` values should be set as necessary: - `secretName` should be the name of the JWT Secret you created. By default this field is set to `nplus-license`. This field is required. - `endpoint` is the endpoint to send the telemetry data to. This is optional, and by default is `product.connect.nginx.com`. - `resolver` is the nameserver used to resolve the NGINX Plus usage reporting endpoint. This is optional and used with NGINX Instance Manager. - `skipVerify` disables client verification of the NGINX Plus usage reporting server certificate. - `caSecretName` is the name of the Secret containing the NGINX Instance Manager CA certificate. Must exist in the same namespace that the NGINX Gateway Fabric control plane is running in (default namespace: nginx-gateway). - `clientSSLSecretName` is the name of the Secret containing the client certificate and key for authenticating with NGINX Instance Manager. Must exist in the same namespace that the NGINX Gateway Fabric control plane is running in (default namespace: nginx-gateway). If using manifests, the following command-line options should be set as necessary on the `nginx-gateway` container: - `--usage-report-secret` should be the name of the JWT Secret you created. Must exist in the same namespace that the NGINX Gateway Fabric control plane is running in (default namespace: nginx-gateway). By default this field is set to `nplus-license`. - `--usage-report-endpoint` is the endpoint to send the telemetry data to. This is optional, and by default is `product.connect.nginx.com`. - `--usage-report-resolver` is the nameserver used to resolve the NGINX Plus usage reporting endpoint. This is optional and used with NGINX Instance Manager. - `--usage-report-skip-verify` disables client verification of the NGINX Plus usage reporting server certificate. - `--usage-report-ca-secret` is the name of the Secret containing the NGINX Instance Manager CA certificate. Must exist in the same namespace that the NGINX Gateway Fabric control plane is running in (default namespace: nginx-gateway). - `--usage-report-client-ssl-secret` is the name of the Secret containing the client certificate and key for authenticating with NGINX Instance Manager. Must exist in the same namespace that the NGINX Gateway Fabric control plane is running in (default namespace: nginx-gateway). ## What’s reported and how it’s protected {#telemetry} NGINX Plus reports the following data every hour by default: - **NGINX version and status**: The version of NGINX Plus running on the instance. - **Instance UUID**: A unique identifier for each NGINX Plus instance. - **Traffic data**: - **Bytes received from and sent to clients**: HTTP and stream traffic volume between clients and NGINX Plus. - **Bytes received from and sent to upstreams**: HTTP and stream traffic volume between NGINX Plus and upstream servers. - **Client connections**: The number of accepted client connections (HTTP and stream traffic). - **Requests handled**: The total number of HTTP requests processed. - **NGINX uptime**: The number of reloads and worker connections during uptime. - **Usage report timestamps**: Start and end times for each usage report. - **Kubernetes node details**: Information about Kubernetes nodes. ### Security and privacy of reported data All communication between your NGINX Plus instances, NGINX Instance Manager, and F5’s licensing endpoint (`product.connect.nginx.com`) is protected using **SSL/TLS** encryption. Only **operational metrics** are reported — no **personally identifiable information (PII)** or **sensitive customer data** is transmitted. ## Pull an image for local use To pull an image for local use, use this command: ```shell docker login private-registry.nginx.com --username= --password=none ``` Replace the contents of `` with the contents of the JWT token itself. You can then pull the image: ```shell docker pull private-registry.nginx.com/nginx-gateway-fabric/nginx-plus: ``` Once you have successfully pulled the image, you can tag it as needed, then push it to a different container registry. For a complete list of available NGINX Plus images, including UBI-based and WAF variants, see [Supported container images](/ngf/overview/technical-specifications.md#supported-container-images). ## Alternative installation options There are alternative ways to get an NGINX Plus image for NGINX Gateway Fabric: - [Build the Gateway Fabric image](/ngf/install/build-image.md) describes how to use the source code with an NGINX Plus subscription certificate and key to build an image. ## Next steps - [Deploy a Gateway for data plane instances](/ngf/install/deploy-data-plane.md) - [Routing traffic to applications](/ngf/traffic-management/basic-routing.md) -------------------------------------------------------------------------------- title: "Use Manifests to install NGINX Gateway Fabric (experimental) with NGINX Open Source" url: https://docs.nginx.com/nginx-gateway-fabric/install/manifests/open-source-experimental/index.md -------------------------------------------------------------------------------- This page describes how to use Manifests to install NGINX Gateway Fabric (experimental) with NGINX Open Source. It explains how to install the Gateway API resources and add authentication certificates, then deploy NGINX Gateway Fabric and its custom resource definitions. Using experimental NGINX Gateway Fabric versions allows to test API resources from upcoming releases as outlined by the [Milestone Roadmap](https://github.com/orgs/nginx/projects/10/views/5). By following these instructions, you will finish with a functional NGINX Gateway Fabric instance for your Kubernetes cluster. **Note:** To learn which Gateway API resources NGINX Gateway Fabric currently supports, view the [Gateway API Compatibility](/ngf/overview/gateway-api-compatibility.md) topic. ## Before you begin To complete this guide, you will need the following pre-requisites: - [A supported Kubernetes version](/ngf/overview/technical-specifications.md) - A functional Kubernetes cluster ## Install the Gateway API resources **Note:** If you have already installed Gateway API resources in your cluster, ensure they are a version [supported by NGINX Gateway Fabric](/ngf/overview/technical-specifications.md) To install API resources from the experimental channel, use `kubectl kustomize`: ```shell kubectl kustomize "https://github.com/nginx/nginx-gateway-fabric/config/crd/gateway-api/experimental?ref=v" | kubectl apply --server-side -f - ``` #### Example output ```text customresourcedefinition.apiextensions.k8s.io/backendtlspolicies.gateway.networking.k8s.io serverside-applied customresourcedefinition.apiextensions.k8s.io/gatewayclasses.gateway.networking.k8s.io serverside-applied customresourcedefinition.apiextensions.k8s.io/gateways.gateway.networking.k8s.io serverside-applied customresourcedefinition.apiextensions.k8s.io/grpcroutes.gateway.networking.k8s.io serverside-applied customresourcedefinition.apiextensions.k8s.io/httproutes.gateway.networking.k8s.io serverside-applied customresourcedefinition.apiextensions.k8s.io/referencegrants.gateway.networking.k8s.io serverside-applied customresourcedefinition.apiextensions.k8s.io/tcproutes.gateway.networking.k8s.io serverside-applied customresourcedefinition.apiextensions.k8s.io/tlsroutes.gateway.networking.k8s.io serverside-applied customresourcedefinition.apiextensions.k8s.io/udproutes.gateway.networking.k8s.io serverside-applied customresourcedefinition.apiextensions.k8s.io/xbackendtrafficpolicies.gateway.networking.x-k8s.io serverside-applied customresourcedefinition.apiextensions.k8s.io/xlistenersets.gateway.networking.x-k8s.io serverside-applied ``` You should also create the _nginx-gateway_ namespace, which is used by the Manifest files by default: ```shell kubectl create namespace nginx-gateway ``` ## Install cert-manager Install cert-manager onto the cluster using Helm with Gateway API features enabled. - Add the Helm repository. ```shell helm repo add jetstack https://charts.jetstack.io helm repo update ``` - Install cert-manager, and enable the GatewayAPI feature gate: ```shell helm install \ cert-manager jetstack/cert-manager \ --namespace cert-manager \ --create-namespace \ --set config.apiVersion="controller.config.cert-manager.io/v1alpha1" \ --set config.kind="ControllerConfiguration" \ --set config.enableGatewayAPI=true \ --set crds.enabled=true ``` ## Add certificates for secure authentication **Note:** These steps use a self-signed issuer, which should not be used in production environments. For production environments, you should use a real [CA issuer](https://cert-manager.io/docs/configuration/ca/). First, create a CA (certificate authority) issuer: ```yaml kubectl apply -f - <= v1.18.0, the default value changed from `Never` to `Always`. certificate.cert-manager.io/nginx-gateway-ca created issuer.cert-manager.io/nginx-gateway-issuer created ``` You will then need to create a server certificate for the NGINX Gateway Fabric control plane (server): **Note:** The default service name is _nginx-gateway_, and the namespace is _nginx-gateway_, so the `dnsNames` value should be `nginx-gateway.nginx-gateway.svc`. This value becomes the name of the NGINX Gateway Fabric control plane service. ```yaml {hl_lines=[13]} kubectl apply -f - <>gw: Request gw->>app: HTTPS request gw->>app: start TLS handshake app->>gw: request client certificate gw->>app: present client certificate from Secret: gateway-presents-this-cert-for-validation app->>app: validate client certificate using ca.crt in Secret: app-tls-secret app->>gw: present backend certificate gw->>gw: validate backend certificate using BackendTLSPolicy and ca.crt in Secret: backend-cert app->>gw: complete TLS handshake app-->>gw: HTTPS response gw-->>client: Response ``` ## Before you begin - [Install](/ngf/install/) NGINX Gateway Fabric. ## Set up Install cert-manager onto the cluster using Helm with Gateway API features enabled. - Add the Helm repository. ```shell helm repo add jetstack https://charts.jetstack.io helm repo update ``` - Install cert-manager, and enable the GatewayAPI feature gate: ```shell helm install \ cert-manager jetstack/cert-manager \ --namespace cert-manager \ --create-namespace \ --set config.apiVersion="controller.config.cert-manager.io/v1alpha1" \ --set config.kind="ControllerConfiguration" \ --set config.enableGatewayAPI=true \ --set crds.enabled=true ``` Create a self-signed `ClusterIssuer`, a CA `Certificate`, and a CA-backed `ClusterIssuer`. cert-manager uses the resulting `local-ca-issuer` to sign certificates in any namespace: ```yaml kubectl apply -f - < 8443/TCP 9s ``` ## Configure routing rules First, create the Gateway resource with an HTTP listener: ```yaml kubectl apply -f - < ``` **Note:** In a production environment, you should have a DNS record for the external IP address that is exposed, and it should refer to the hostname that the gateway will forward for. --- ## Send traffic without mutual TLS configured Using the external IP address and port for the NGINX Service, send traffic to the secure-app application. To show what happens before we configure backend TLS and have the Gateway present its client certificate for verification, send a request now and observe how the connection to the application fails with a bad request error. **Note:** If you have a DNS record allocated for `secure-app.example.com`, you can send the request directly to that hostname, without needing to resolve. ```shell curl --resolve secure-app.example.com:$GW_PORT:$GW_IP http://secure-app.example.com:$GW_PORT/ ``` ```text 400 The plain HTTP request was sent to HTTPS port

400 Bad Request

The plain HTTP request was sent to HTTPS port
nginx/1.29.2
``` We can see a status 400 Bad Request message from NGINX. --- ## Configure TLS for Gateway and Backend applications To create a Secret named `gateway-presents-this-cert-for-validation` signed by the local CA that Gateway presents to verify its identity, copy and paste the following command: ```yaml kubectl apply -f - < Annotations: API Version: gateway.networking.k8s.io/v1 Kind: BackendTLSPolicy Metadata: Creation Timestamp: 2025-11-13T23:28:36Z Generation: 1 Resource Version: 1288 UID: d7e3f026-afe3-44d1-aed5-c168e954b52f Spec: Target Refs: Group: Kind: Service Name: secure-app Validation: Ca Certificate Refs: Group: Kind: Secret Name: backend-cert Hostname: secure-app.example.com Status: Ancestors: Ancestor Ref: Group: gateway.networking.k8s.io Kind: Gateway Name: gateway Namespace: default Conditions: Last Transition Time: 2025-11-13T23:28:37Z Message: All CACertificateRefs are resolved Observed Generation: 1 Reason: ResolvedRefs Status: True Type: ResolvedRefs Last Transition Time: 2025-11-13T23:28:37Z Message: The Policy is accepted Observed Generation: 1 Reason: Accepted Status: True Type: Accepted Controller Name: gateway.nginx.org/nginx-gateway-controller Events: ``` --- ## Send traffic with backend TLS configuration Now send traffic again: ```shell curl --resolve secure-app.example.com:$GW_PORT:$GW_IP http://secure-app.example.com:$GW_PORT/ ``` ```text hello from pod secure-app ``` To verify that the backend validated the gateway’s client certificate, inspect the logs of the `secure-app` pod and check the reported client subject: ```shell POD_NAME=$(kubectl get pod -l app=secure-app -o jsonpath='{.items[0].metadata.name}') kubectl logs "$POD_NAME" ``` ```text 10.244.0.145 ssl_client_verify=SUCCESS ssl_client_subject=CN=gateway ``` --- ## See also To learn more about configuring backend TLS termination using the Gateway API, see the following resources: - [Backend TLS Policy](https://gateway-api.sigs.k8s.io/api-types/backendtlspolicy/) - [Backend TLS Policy GEP](https://gateway-api.sigs.k8s.io/geps/gep-1897/) - [Gateway Backend TLS](https://gateway-api.sigs.k8s.io/reference/spec/#gatewaybackendtls) -------------------------------------------------------------------------------- title: "Technical specifications" url: https://docs.nginx.com/nginx-gateway-fabric/overview/technical-specifications/index.md description: NGINX Gateway Fabric technical specifications. -------------------------------------------------------------------------------- This page describes the technical specifications for NGINX Gateway Fabric. The information included covers version compatibility between NGINX Gateway Fabric and the Gateway API, as well as other NGINX products. ## NGINX Gateway Fabric versions The following table lists the software versions NGINX Gateway Fabric supports. Only the latest patch release for each minor version is shown. | NGINX Gateway Fabric | Gateway API | Kubernetes | NGINX OSS | NGINX Plus | NGINX Agent | F5 WAF for NGINX | |----------------------|-------------|------------|-----------|------------|-------------|------------------| | Edge | 1.6.1 | 1.32+ | 1.31.6 | R37.1 | v3.12.0 | 5.15.0 | | 2.7.2 | 1.6.1 | 1.32+ | 1.31.6 | R37.1 | v3.12.0 | 5.15.0 | | 2.6.8 | 1.5.1 | 1.31+ | 1.31.3 | R37.0 | v3.11.2 | 5.13.2 | | 2.5.1 | 1.5.1 | 1.31+ | 1.29.7 | R36 | v3.8.0 | --- | | 2.4.2 | 1.4.1 | 1.25+ | 1.29.5 | R36 | v3.7.1 | --- | | 2.3.0 | 1.4.1 | 1.25+ | 1.29.3 | R36 | v3.6.0 | --- | | 2.2.2 | 1.3.0 | 1.25+ | 1.29.2 | R35 | v3.6.0 | --- | | 2.1.4 | 1.3.0 | 1.25+ | 1.29.1 | R35 | v3.3.1 | --- | | 2.0.2 | 1.3.0 | 1.25+ | 1.28.0 | R34 | v3.0.1 | --- | | 1.6.2 | 1.2.1 | 1.25+ | 1.27.4 | R33 | --- | --- | | 1.5.1 | 1.2.0 | 1.25+ | 1.27.2 | R33 | --- | --- | | 1.4.0 | 1.1.0 | 1.25+ | 1.27.1 | R32 | --- | --- | | 1.3.0 | 1.1.0 | 1.25+ | 1.27.0 | R32 | --- | --- | | 1.2.0 | 1.0.0 | 1.23+ | 1.25.4 | R31 | --- | --- | **Note:** For version requirements to export F5 WAF for NGINX security events to NGINX Instance Manager, see [Export security logs to F5 NGINX Instance Manager](/ngf/waf-integration/policy-sources.md#export-security-logs-to-f5-nginx-instance-manager). ### OpenShift Compatibility The following table lists the OpenShift versions and Operator versions compatible with NGINX Gateway Fabric. | NGINX Gateway Fabric | Operator | Preferred Gateway API | Compatible Gateway API | OCP with Preferred GWAPI | Supported OCP Versions | |----------------------|----------|-----------------------|------------------------|--------------------------|------------------------| | 2.7.x | v1.5.x | v1.6.x | v1.2.1-v1.6.x | --- | 4.19 - 4.22 | | 2.6.x | v1.4.x | v1.5.x | v1.2.1-v1.5.x | --- | 4.19 - 4.21 | | 2.5.x | v1.3.x | v1.5.x | v1.2.1-v1.5.x | --- | 4.19 - 4.21 | | 2.4.x | v1.2.x | v1.4.x | v1.2.1-v1.4.x | 4.20 & 4.21 | 4.19 - 4.21 | | 2.2.x | v1.0.x | v1.3.0 | v1.2.1 | --- | 4.19 | NGINX Gateway Fabric is conformant with the Gateway API version installed on supported OCP versions. The "OCP with Preferred GWAPI" column shows which OCP versions ship with the preferred Gateway API version. On OCP versions with an older Gateway API installed, NGF remains fully conformant with that installed version, but features from newer Gateway API versions that NGF supports will be unavailable. ## Supported container images NGINX Gateway Fabric provides container images for the control plane and the NGINX data plane. All images are available for `amd64` and `arm64` architectures unless otherwise noted. ### Control plane images The control plane image contains the NGINX Gateway Fabric binary. | Name | Base image | Image | Architectures | |-----------------|------------------------|--------------------------------------------------------------|----------------| | Default image | `scratch` | `ghcr.io/nginx/nginx-gateway-fabric:` | amd64
arm64 | | UBI-based image | `redhat/ubi10-minimal` | `ghcr.io/nginx/nginx-gateway-fabric:-ubi` | amd64
arm64 | ### Data plane images with NGINX | Name | Base image | Image | Architectures | |-----------------|-----------------------|--------------------------------------------------------------------|----------------| | Default image | `alpine:3.24` | `ghcr.io/nginx/nginx-gateway-fabric/nginx:` | amd64
arm64 | | UBI-based image | `redhat/ubi10-minimal` | `ghcr.io/nginx/nginx-gateway-fabric/nginx:-ubi` | amd64
arm64 | ### Data plane images with NGINX Plus NGINX Plus images are available through the F5 Container registry `private-registry.nginx.com`. For setup instructions and authentication details, see [Install NGINX Gateway Fabric with NGINX Plus](/ngf/install/nginx-plus.md). | Name | Base image | Image | Architectures | |---------------------------------------|-----------------------|--------------------------------------------------------------------------------------------|----------------| | Default image | `alpine:3.24` | `private-registry.nginx.com/nginx-gateway-fabric/nginx-plus:` | amd64
arm64 | | UBI-based image | `redhat/ubi10-minimal` | `private-registry.nginx.com/nginx-gateway-fabric/nginx-plus:-ubi` | amd64
arm64 | | Default image with F5 WAF for NGINX | `alpine:3.24` | `private-registry.nginx.com/nginx-gateway-fabric/nginx-plus-f5waf:` | amd64 | | UBI-based image with F5 WAF for NGINX | `redhat/ubi10-minimal` | `private-registry.nginx.com/nginx-gateway-fabric/nginx-plus-f5waf:-ubi` | amd64 | ### WAF sidecar images When F5 WAF for NGINX is enabled, two additional sidecar containers are deployed alongside the NGINX container. These images are available from the F5 Container registry. | Name | Image | Architectures | |--------------------|---------------------------------------------------------------------------------|-------| | WAF Enforcer | `private-registry.nginx.com/nap/waf-enforcer:` | amd64 | | WAF Config Manager | `private-registry.nginx.com/nap/waf-config-mgr:` | amd64 | For more information on WAF integration, see [F5 WAF for NGINX overview](/ngf/waf-integration/overview.md). ### Custom images You can build custom NGINX Gateway Fabric images from source. For instructions, see [Build NGINX Gateway Fabric](/ngf/install/build-image.md). --- ## Gateway API compatibility The following tables summarizes which Gateway API resources NGINX Gateway Fabric supports and to which level. You can read more information by viewing the [Gateway API compatibility](/ngf/overview/gateway-api-compatibility.md) topic, or by selecting the resource name to go directly to the full details. **Note:** You can view the [Milestone Roadmap](https://github.com/orgs/nginx/projects/10/views/5) in the NGINX Gateway Fabric GitHub project to see what API resources will be supported in upcoming releases. | Resource | Core Support Level | Extended Support Level | Implementation-Specific Support Level | API Version | API Release Channel | |---------------------------------------|---------------------|------------------------|---------------------------------------|-------------|---------------------| | [GatewayClass](/ngf/overview/gateway-api-compatibility.md#gatewayclass) | Supported | Not supported | Supported | v1 | Standard | | [Gateway](/ngf/overview/gateway-api-compatibility.md#gateway) | Supported | Partially supported | Not supported | v1 | Standard | | [HTTPRoute](/ngf/overview/gateway-api-compatibility.md#httproute) | Supported | Partially supported | Not supported | v1 | Standard | | [GRPCRoute](/ngf/overview/gateway-api-compatibility.md#grpcroute) | Supported | Partially supported | Not supported | v1 | Standard | | [ReferenceGrant](/ngf/overview/gateway-api-compatibility.md#referencegrant) | Supported | N/A | Not supported | v1 | Standard | | [TLSRoute](/ngf/overview/gateway-api-compatibility.md#tlsroute) | Supported | Not supported | Not supported | v1 | Standard | | [TCPRoute](/ngf/overview/gateway-api-compatibility.md#tcproute) | Supported | Supported | Not supported | v1 | Standard | | [UDPRoute](/ngf/overview/gateway-api-compatibility.md#udproute) | Supported | Supported | Not supported | v1 | Standard | | [BackendTLSPolicy](/ngf/overview/gateway-api-compatibility.md#backendtlspolicy) | Partially supported | Supported | Partially supported | v1 | Standard | | [Custom policies](/ngf/overview/gateway-api-compatibility.md#custom-policies) | N/A | N/A | Supported | N/A | N/A | -------------------------------------------------------------------------------- title: "Configure basic authentication" url: https://docs.nginx.com/nginx-gateway-fabric/traffic-security/basic-authentication/index.md -------------------------------------------------------------------------------- This page describes how to configure basic authentication in NGINX Gateway Fabric using the AuthenticationFilter custom resource definition (CRD). Authentication can be used to secure applications and APIs, ensuring only trusted and authorized users have access. By following these instructions, you will create two sample application endpoints. One will include basic authentication and the other will not, allowing you to review how each behaves. ## Before you begin - [Install](/ngf/install/) NGINX Gateway Fabric. ## Setup In this part of the document, we will set up several resources in your cluster to demonstrate usage of the AuthenticationFilter CRD. ## Deploy sample applications To deploy the `coffee` and `tea` applications, run the following YAML with `kubectl apply`: ```yaml kubectl apply -f - < ``` ## Create a user credentials secret and AuthenticationFilter Deploy a secret with user credentials, and the AuthenticationFilter by running the following YAML with `kubectl apply`: ```yaml kubectl apply -f - < ``` ## Deploy a HTTPRoute referencing the AuthenticationFilter Deploy a HTTPRoute resource which references the AuthenticationFilter using the `ExtensionRef` filter type. In this example, the filter is applied to the `/coffee` path: run the following YAML with `kubectl apply` ```yaml kubectl apply -f - < ``` ## Verify Basic Authentication **Note:** Your clients should be able to resolve the domain name "cafe.example.com" to the public IP of the NGINX Service. This guide simulates it using curl's `--resolve` option. Accessing `/coffee` with valid credentials: ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee -u user1:password1 ``` Response: ```text Server address: 10.244.0.7:8080 Server name: coffee-654ddf664b-nhhvr Date: 06/Jan/2026:15:20:15 +0000 URI: /coffee Request ID: 13a925b2514b62c45ea4a79800248d5c ``` Accessing `/coffee` without credentials: ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee ``` Response: ```text 401 Authorization Required

401 Authorization Required

nginx
``` Accessing `/coffee` with incorrect credentials: ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee -u user1:wrong ``` Response: ```text 401 Authorization Required

401 Authorization Required

nginx
``` Accessing `/tea` Since tea has no AuthenticationFilter attached, responses are processed normally: ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/tea ``` Response: ```text Server address: 10.244.0.10:8080 Server name: tea-75bc9f4b6d-ms2n8 Date: 06/Jan/2026:15:36:26 +0000 URI: /tea Request ID: c7eb0509303de1c160cb7e7d2ac1d99f ``` ## Troubleshooting - Ensure the HTTPRoute is Accepted and references the correct AuthenticationFilter name and group. - Confirm the secret key is named `auth`. - Ensure the secret referenced by the AuthenticationFilter is in the same namespace. ## Further reading - [Example deployment files for AuthenticationFilter](https://github.com/nginx/nginx-gateway-fabric/tree/main/examples/basic-authentication) - [NGINX HTTP Basic Auth Module](https://nginx.org/en/docs/http/ngx_http_auth_basic_module.html) -------------------------------------------------------------------------------- title: "Configure WAF settings" url: https://docs.nginx.com/nginx-gateway-fabric/waf-integration/configuration/index.md -------------------------------------------------------------------------------- This page covers operational configuration for F5 WAF for NGINX in NGINX Gateway Fabric: security logging, automatic policy updates, TLS and authentication, bundle integrity verification, cookie seed management, fetch failure handling, and WAF container settings. --- ## Configure security logging The `securityLogs` field on a `WAFPolicy` supports multiple log destinations. Each entry generates an `app_protect_security_log` directive in the NGINX configuration. ### Log destinations | Destination type | Description | |------------------|-----------------------------------------------------| | `stderr` | Write to standard error (appears in container logs) | | `file` | Write to a file path inside the NGINX container | | `syslog` | Send to a syslog server | ### Log source types For the `NIM`, `N1C`, and `HTTP` source types, each log entry must specify a `logSource` using one of: | Field | Description | |------------------|--------------------------------------| | `defaultProfile` | A built-in WAF log profile name | | `httpSource` | URL to a compiled log profile bundle | | `nimSource` | NGINX Instance Manager log profile configuration | | `n1cSource` | NGINX One Console log profile configuration | **Built-in log profiles:** `log_default`, `log_all`, `log_blocked`, `log_illegal`, `log_grpc_all`, `log_grpc_blocked`, `log_grpc_illegal` For the `PLM` source type, use `logRef.apLogConfRef` instead of `logSource` to reference an `APLogConf` custom resource by name and namespace. See [PLM (Policy Lifecycle Management)](/ngf/waf-integration/overview.md#plm-policy-lifecycle-management) for the full field comparison. ### Example: multiple log destinations ```yaml securityLogs: - destination: type: stderr logSource: defaultProfile: log_all - destination: type: file file: path: "/var/log/app_protect/security.log" logSource: httpSource: url: https://bundles.example.com/waf/custom-log-profile.tgz auth: secretRef: name: bundle-credentials validation: verifyChecksum: true - destination: type: syslog syslog: server: syslog-svc.default.svc.cluster.local:514 logSource: defaultProfile: log_blocked ``` --- ## Configure automatic policy updates (polling) Polling enables NGINX Gateway Fabric to detect and deploy updated policy bundles without modifying the `WAFPolicy` resource. This is useful when the same URL or policy name always resolves to the latest compiled bundle — for example, in a CI/CD workflow that overwrites the bundle file in place. Enable polling on a `policySource` or `logSource`: ```yaml policySource: nimSource: url: https://nim.example.com policyName: ngfBlocking auth: secretRef: name: nim-credentials polling: enabled: true interval: 10m # optional; defaults to 5m ``` **How polling works by source type:** - **NGINX Instance Manager and NGINX One Console**: NGINX Gateway Fabric first fetches only the bundle checksum or metadata. The full bundle is downloaded only if the checksum has changed, avoiding unnecessary traffic. - **HTTP**: NGINX Gateway Fabric sends a conditional GET using the stored `ETag` or `Last-Modified` header from the previous fetch. A `304 Not Modified` response skips the download entirely. **When not to enable polling:** If you pin a specific version — `policyUID` for NGINX Instance Manager, `policyVersionID` for NGINX One Console, or a version-specific URL for HTTP — the source always returns the same bundle. Every poll will detect "unchanged" and trigger no action. In that case, disable polling to avoid unnecessary network requests. **On poll failure:** If a poll attempt fails, the existing deployed bundle remains active. WAF protection is not interrupted. The error is recorded in the `WAFPolicy` status with reason `StaleBundleWarning`. --- ## Configure the WAF cookie seed When WAF is enabled, NGINX Gateway Fabric automatically sets the `app_protect_cookie_seed` NGINX directive to a stable value derived from the Gateway's UID. This ensures that WAF session cookies issued by one NGINX replica can be validated by any other replica in the same deployment. Without this, each replica generates its own random seed at startup, which causes cross-replica cookie validation failures. If you have pre-compiled the cookie seed into your WAF policy bundles using the [compiler global settings](/waf/configure/compiler.md), disable the automatic cookie seed in the NginxProxy CRD to avoid conflicting with the compiled-in value: ```yaml apiVersion: gateway.nginx.org/v1alpha2 kind: NginxProxy metadata: name: waf-enabled-proxy spec: waf: enable: true disableCookieSeed: true ``` **Note:** Only set `disableCookieSeed: true` if your compiled bundles already contain a cookie seed. If neither the bundle nor NGINX Gateway Fabric provides a seed, each replica generates its own random value, which breaks WAF session cookie validation in multi-replica deployments. --- ## Configure TLS and authentication ### Custom CA certificate If your policy server uses a self-signed certificate or an internal CA, provide the CA certificate in a Secret: ```yaml apiVersion: v1 kind: Secret metadata: name: custom-ca-secret type: Opaque data: ca.crt: ``` Reference it on the `policySource`: ```yaml policySource: httpSource: url: https://internal-server.example.com/policy.tgz tlsSecret: name: custom-ca-secret ``` The CA certificate is appended to the system CA pool. This is supported for all source types. **Note:** Do not set `insecureSkipVerify: true` in production environments. This disables TLS certificate verification and should be used only for local testing. ### Authentication methods NGINX Gateway Fabric infers the authentication method from the keys present in the Secret referenced by `auth.secretRef`: | Secret keys | Authentication method | |-------------------------|-----------------------------------------------------------------------| | `username` + `password` | HTTP Basic Auth | | `token` | Bearer Token (NGINX Instance Manager) or APIToken (NGINX One Console) | | (none) | No authentication header | Secrets must be in the same namespace as the `WAFPolicy`. --- ## Configure bundle integrity verification ### HTTP source Set `validation.verifyChecksum: true` to have NGINX Gateway Fabric fetch a companion `.sha256` file and compare its SHA-256 digest against the downloaded bundle. Any mismatch prevents the bundle from being deployed and sets `Programmed=False` with reason `IntegrityError`. Generate the companion file: ```shell sha256sum compiled-policy.tgz > compiled-policy.tgz.sha256 ``` Upload both files to the same location. NGINX Gateway Fabric appends `.sha256` to the bundle URL automatically. ### Known-checksum enforcement For any source type, you can pin an expected SHA-256 checksum to reject bundles that do not match: ```yaml policySource: nimSource: url: https://nim.example.com policyName: ngfBlocking validation: expectedChecksum: "abc1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcd" ``` The `expectedChecksum` must be a 64-character hexadecimal SHA-256 digest. **Note:** `verifyChecksum` and `expectedChecksum` are mutually exclusive. You can use one or the other on the same policy source, but not both. --- ## Policy fetch failure handling When a WAF policy bundle cannot be fetched — for example, due to a network error, authentication failure, or the bundle not yet being compiled — the behavior of NGINX Gateway Fabric depends on whether this is the **first** fetch or a **subsequent** update. ### First-time fetch failure When the bundle has never been successfully fetched, you can control whether NGINX Gateway Fabric withholds the configuration push or allows traffic to flow unprotected while waiting for the bundle. This is configured using `waf.bundleFailOpen` on the `NginxProxy` resource: **Fail-closed (default, `bundleFailOpen: false`):** NGINX Gateway Fabric withholds the NGINX configuration push entirely until the bundle is available. No configuration changes — including unrelated route additions — are applied to the data plane while any pending bundle exists for the Gateway. The `app_protect_policy_file` directive is not emitted. The WAFPolicy status reflects the withheld push with `Programmed=False`. This is the safe default: the operator must resolve the bundle fetch failure before traffic is served. **Fail-open (`bundleFailOpen: true`):** NGINX Gateway Fabric pushes the NGINX configuration normally, but omits the pending WAFPolicy from the generated config — no `app_protect_policy_file` directive is emitted, so NGINX loads successfully without WAF protection. Traffic flows unprotected until the bundle becomes available, at which point NGINX Gateway Fabric includes the policy in the next configuration push. The WAFPolicy status continues to show `Programmed=False` while the bundle is pending, so operators are aware that WAF protection is not yet active. To enable fail-open, set `waf.bundleFailOpen: true` on the `NginxProxy`: ```yaml apiVersion: gateway.nginx.org/v1alpha2 kind: NginxProxy metadata: name: waf-enabled-proxy spec: waf: enable: true bundleFailOpen: true ``` **Note:** `bundleFailOpen: true` means traffic is served without WAF protection if the initial bundle fetch fails. Use this only when availability is more critical than security posture during startup. Monitor the WAFPolicy status to confirm when the bundle becomes active. **Note:** The first-time fetch rules apply whenever NGINX Gateway Fabric starts without an already-fetched bundle. This includes upgrades, control plane pod restarts, and new Gateway deployments, because policy bundles are not persisted across pod restarts. With the default fail-closed setting, configuration pushes are withheld until all bundles have been re-fetched after each restart. Set `bundleFailOpen: true` if you need traffic to flow immediately after a restart and can accept a brief window without WAF protection. In both cases, the WAFPolicy status condition is set to `Programmed=False` with reason `Pending` until the bundle is successfully fetched. ### Subsequent fetch failures If a bundle fetch fails after a policy has already been successfully deployed — for example, during a polling cycle or after a WAFPolicy update — the **existing deployed bundle remains active**. WAF protection is not interrupted. The failure is reflected in the WAFPolicy status with `Programmed=True` and reason `StaleBundleWarning`, indicating that the previously fetched bundle is still in use: ```text - Type: Programmed Status: True Reason: StaleBundleWarning Message: policy bundle fetch failed; using previously fetched bundle: connection timeout ``` NGINX Gateway Fabric retries on the next reconciliation or poll cycle. No manual intervention is required unless the error persists. --- ## Configure WAF containers When WAF is enabled, NGINX Gateway Fabric deploys two sidecar containers — `waf-enforcer` and `waf-config-mgr` — alongside the main NGINX container. These settings are configured under `spec.kubernetes.deployment.wafContainers` (or `spec.kubernetes.daemonSet.wafContainers` for DaemonSet mode) in the NginxProxy resource. This follows the same infrastructure configuration pattern described in [Configure infrastructure-related settings](/ngf/how-to/data-plane-configuration.md#configure-infrastructure-related-settings). For the full list of configurable fields, see the `NginxProxy` spec in the [API reference](/ngf/reference/api.md). Each container (`enforcer` and `configManager`) supports the following fields: - **`image`**: Override the default image repository, tag, and pull policy. If not specified, NGINX Gateway Fabric uses the defaults from the F5 Container registry. For the default images, see [Supported container images](/ngf/overview/technical-specifications.md#supported-container-images). - **`resources`**: Set CPU and memory requests and limits. - **`volumeMounts`**: Add extra volume mounts. NGINX Gateway Fabric automatically configures the shared volumes required for communication between the NGINX, `waf-enforcer`, and `waf-config-mgr` containers. Additional mounts are appended to these defaults. The following example uses custom images from a private registry and sets resource requirements for both containers: ```yaml apiVersion: gateway.nginx.org/v1alpha2 kind: NginxProxy metadata: name: waf-enabled-proxy spec: waf: enable: true kubernetes: deployment: wafContainers: enforcer: image: repository: registry.example.com/nap/waf-enforcer tag: "" resources: requests: cpu: 100m memory: 128Mi limits: cpu: "1" memory: 1Gi configManager: image: repository: registry.example.com/nap/waf-config-mgr tag: "" resources: requests: cpu: 50m memory: 64Mi limits: cpu: 500m memory: 256Mi ``` When installing with Helm, set the equivalent values under `nginx.wafContainers`: ```yaml # values.yaml nginx: config: waf: enable: true wafContainers: enforcer: image: repository: registry.example.com/nap/waf-enforcer tag: "" resources: requests: cpu: 100m memory: 128Mi configManager: image: repository: registry.example.com/nap/waf-config-mgr tag: "" resources: requests: cpu: 50m memory: 64Mi ``` **Note:** Image pull Secrets for private registries must be configured at install time using the `nginx.imagePullSecret` or `nginx.imagePullSecrets` Helm values (or the `--nginx-docker-secret` flag for manifest installs). The control plane copies these Secrets into any namespace where NGINX is deployed. For details, see [Install NGINX Gateway Fabric with NGINX Plus](/ngf/install/nginx-plus.md). --- ## See also - [F5 WAF for NGINX overview](/ngf/waf-integration/overview.md) - [Configure policy sources (NGINX Instance Manager and NGINX One Console)](/ngf/waf-integration/policy-sources.md) - [Configure infrastructure-related settings](/ngf/how-to/data-plane-configuration.md#configure-infrastructure-related-settings) - [Troubleshoot WAFPolicy status](/ngf/waf-integration/troubleshooting.md) - [Supported container images](/ngf/overview/technical-specifications.md#supported-container-images) - [WAFPolicy and NginxProxy API reference](/ngf/reference/api.md) - [Build and use the compiler tool](/waf/configure/compiler.md) -------------------------------------------------------------------------------- title: "Control plane configuration" url: https://docs.nginx.com/nginx-gateway-fabric/how-to/control-plane-configuration/index.md -------------------------------------------------------------------------------- Learn how to dynamically update the NGINX Gateway Fabric control plane configuration. ## Overview NGINX Gateway Fabric can dynamically update the control plane configuration without restarting. The control plane configuration is stored in the NginxGateway custom resource, created during the installation of NGINX Gateway Fabric. NginxGateway is deployed in the same namespace as the controller (Default: `nginx-gateway`). The resource's default name is based on your [installation method](/ngf/install/): - Helm: `-config` - Manifests: `nginx-gateway-config` The control plane only watches this single instance of the custom resource. If the resource is invalid to the OpenAPI schema, the Kubernetes API server will reject the changes. If the resource is deleted or deemed invalid by NGINX Gateway Fabric, a warning event is created in the `nginx-gateway` namespace, and the default values will be used by the control plane for its configuration. Additionally, the control plane updates the status of the resource (if it exists) to reflect whether it is valid or not. **For a full list of configuration options that can be set, see the `NginxGateway spec` in the [API reference](/ngf/reference/api.md).** --- ## Viewing and Updating the Configuration **Note:** For the following examples, the name `ngf-config` should be updated to the name of the resource created for your installation. To view the current configuration and its status: ```shell kubectl -n nginx-gateway describe nginxgateways ngf-config ``` ```text ... Status: Conditions: Last Transition Time: 2024-08-13T19:22:14Z Message: NginxGateway is valid Observed Generation: 1 Reason: Valid Status: True Type: Valid ``` To update the configuration: ```shell kubectl -n nginx-gateway edit nginxgateways ngf-config ``` This will open the configuration in your default editor. You can then update and save the configuration, which is applied automatically to the control plane. -------------------------------------------------------------------------------- title: "HTTP redirects and rewrites" url: https://docs.nginx.com/nginx-gateway-fabric/traffic-management/redirects-and-rewrites/index.md -------------------------------------------------------------------------------- Learn how to redirect or rewrite your HTTP traffic using NGINX Gateway Fabric. ## Overview [HTTPRoute](https://gateway-api.sigs.k8s.io/api-types/httproute/) filters can be used to configure HTTP redirects or rewrites. Redirects return HTTP 3XX responses to a client, instructing it to retrieve a different resource. Rewrites modify components of a client request (such as hostname and/or path) before proxying it upstream. In this guide, we will set up the coffee application to demonstrate path URL rewriting, and the tea and soda applications to showcase path-based request redirection. For an introduction to exposing your application, we recommend that you follow the [basic guide](/ngf/traffic-management/basic-routing.md) first. To see an example of a redirect using scheme and port, see the [HTTPS Termination](/ngf/traffic-management/https-termination.md) guide. ## Before you begin - [Install](/ngf/install/) NGINX Gateway Fabric. ## HTTP rewrites and redirects examples We will configure a common gateway for the `URLRewrite` and `RequestRedirect` filter examples mentioned below. ### Deploy the Gateway resource for the applications The [Gateway](https://gateway-api.sigs.k8s.io/api-types/gateway/) resource is typically deployed by the [Cluster Operator](https://gateway-api.sigs.k8s.io/docs/concepts/roles-and-personas/#roles-and-personas_1). This Gateway defines a single listener on port 80. Since no hostname is specified, this listener matches on all hostnames. To deploy the Gateway: ```yaml kubectl apply -f - < ``` **Note:** In a production environment, you should have a DNS record for the external IP address that is exposed, and it should refer to the hostname that the gateway will forward for. ## URLRewrite example This examples demonstrates how to rewrite the traffic URL for a simple coffee application. An HTTPRoute resource is used to define two `URLRewrite` filters that will rewrite requests. You can verify the server responds with the rewritten URL. ### Deploy the coffee application Create the **coffee** application in Kubernetes by copying and pasting the following block into your terminal: ```yaml kubectl apply -f - < 80/TCP 40s ``` ### Configure a path rewrite The following HTTPRoute defines two filters that will rewrite requests such as the following: - `http://cafe.example.com/coffee` to `http://cafe.example.com/beans` - `http://cafe.example.com/coffee/flavors` to `http://cafe.example.com/beans` - `http://cafe.example.com/latte/prices` to `http://cafe.example.com/prices` To create the httproute resource, copy and paste the following into your terminal: ```yaml kubectl apply -f - < 80/TCP 89m service/tea ClusterIP 10.96.151.194 80/TCP 120m ``` ### Configure a path redirect In this section, we'll define two HTTPRoutes for the tea and soda applications to demonstrate different types of request redirection using the `RequestRedirect` filter. 1. The `tea-redirect` route uses the `ReplacePrefixMatch` type for path modification. This configuration matches the prefix of the original path and updates it to a new path, preserving the rest of the original URL structure. It will redirect request as follows: - `http://cafe.example.com/tea` to `http://cafe.example.com/organic` - `http://cafe.example.com/tea/origin` to `http://cafe.example.com/organic/origin` 2. The `soda-redirect` route uses the `ReplaceFullPath` type for path modification. This configuration updates the entire original path to the new location, effectively overwriting it. It will redirect request as follows: - `http://cafe.example.com/soda` to `http://cafe.example.com/flavors` - `http://cafe.example.com/soda/pepsi` to `http://cafe.example.com/flavors` To create the httproute resource, copy and paste the following into your terminal: ```yaml kubectl apply -f - < **Subscriptions** to see your active subscriptions. 3. Find your NGINX products or services subscription and select the **Subscription ID** for details. 4. Download the **JSON Web Token (JWT)** from the subscription page. **Note:** The Connectivity Stack for Kubernetes JWT does not work with NGINX Plus reporting. Use a regular NGINX Plus instance JWT. ## Create license and registry secrets First, create the _nginx-gateway_ namespace, which is used by the Manifest files by default: ```shell kubectl create namespace nginx-gateway ``` **Note:** The commands in the rest of this document should be run in the same directory as your **license.jwt** file. JWTs are sensitive information and should be stored securely. Delete them after use to prevent unauthorized access. Once you have obtained your license JWT, create a Kubernetes secret using `kubectl create`: ```shell kubectl create -n nginx-gateway secret generic nplus-license --from-file license.jwt ``` Then create another Kubernetes secret to allow interactions with the F5 registry: ```shell kubectl create -n nginx-gateway secret docker-registry nginx-plus-registry-secret \ --docker-server=private-registry.nginx.com \ --docker-username=$(cat license.jwt) \ --docker-password=none ``` You can verify the creation of the secrets using `kubectl get`: ```shell kubectl get -n nginx-gateway secrets ``` #### Example output ```text NAME TYPE DATA AGE nginx-plus-registry-secret kubernetes.io/dockerconfigjson 1 8s nplus-license Opaque 1 21s ``` ## Install the Gateway API resources **Note:** If you have already installed Gateway API resources in your cluster, ensure they are a version [supported by NGINX Gateway Fabric](/ngf/overview/technical-specifications.md) To install API resources from the experimental channel, use `kubectl kustomize`: ```shell kubectl kustomize "https://github.com/nginx/nginx-gateway-fabric/config/crd/gateway-api/experimental?ref=v" | kubectl apply --server-side -f - ``` #### Example output ```text customresourcedefinition.apiextensions.k8s.io/backendtlspolicies.gateway.networking.k8s.io serverside-applied customresourcedefinition.apiextensions.k8s.io/gatewayclasses.gateway.networking.k8s.io serverside-applied customresourcedefinition.apiextensions.k8s.io/gateways.gateway.networking.k8s.io serverside-applied customresourcedefinition.apiextensions.k8s.io/grpcroutes.gateway.networking.k8s.io serverside-applied customresourcedefinition.apiextensions.k8s.io/httproutes.gateway.networking.k8s.io serverside-applied customresourcedefinition.apiextensions.k8s.io/referencegrants.gateway.networking.k8s.io serverside-applied customresourcedefinition.apiextensions.k8s.io/tcproutes.gateway.networking.k8s.io serverside-applied customresourcedefinition.apiextensions.k8s.io/tlsroutes.gateway.networking.k8s.io serverside-applied customresourcedefinition.apiextensions.k8s.io/udproutes.gateway.networking.k8s.io serverside-applied customresourcedefinition.apiextensions.k8s.io/xbackendtrafficpolicies.gateway.networking.x-k8s.io serverside-applied customresourcedefinition.apiextensions.k8s.io/xlistenersets.gateway.networking.x-k8s.io serverside-applied ``` ## Install cert-manager Install cert-manager onto the cluster using Helm with Gateway API features enabled. - Add the Helm repository. ```shell helm repo add jetstack https://charts.jetstack.io helm repo update ``` - Install cert-manager, and enable the GatewayAPI feature gate: ```shell helm install \ cert-manager jetstack/cert-manager \ --namespace cert-manager \ --create-namespace \ --set config.apiVersion="controller.config.cert-manager.io/v1alpha1" \ --set config.kind="ControllerConfiguration" \ --set config.enableGatewayAPI=true \ --set crds.enabled=true ``` ## Add certificates for secure authentication **Note:** These steps use a self-signed issuer, which should not be used in production environments. For production environments, you should use a real [CA issuer](https://cert-manager.io/docs/configuration/ca/). First, create a CA (certificate authority) issuer: ```yaml kubectl apply -f - <= v1.18.0, the default value changed from `Never` to `Always`. certificate.cert-manager.io/nginx-gateway-ca created issuer.cert-manager.io/nginx-gateway-issuer created ``` You will then need to create a server certificate for the NGINX Gateway Fabric control plane (server): **Note:** The default service name is _nginx-gateway_, and the namespace is _nginx-gateway_, so the `dnsNames` value should be `nginx-gateway.nginx-gateway.svc`. This value becomes the name of the NGINX Gateway Fabric control plane service. ```yaml {hl_lines=[13]} kubectl apply -f - <_` ```yaml kubectl apply -f - < 53/UDP,53/TCP 10d ``` NGINX Gateway Fabric creates an `NginxProxy` resource during installation. Edit it to add the `dnsResolver` field: ```bash kubectl edit nginxproxies.gateway.nginx.org -n nginx-gateway ngf-proxy-config ``` ```yaml spec: dnsResolver: addresses: - type: IPAddress value: 10.96.0.10 ``` ### Create the Keycloak Secret This Secret holds the client secret and the CA certificate NGINX uses to verify Keycloak's TLS certificate on outbound connections. The CA certificate is extracted from the `keycloak-tls-cert` Secret that cert-manager created. ```shell kubectl create secret generic keycloak-secret \ --from-literal=client-secret=$CLIENT_SECRET \ --from-file=ca.crt=<(kubectl get secret keycloak-tls-cert -o jsonpath='{.data.ca\.crt}' | base64 -d) ``` ### Create the AuthenticationFilter The `AuthenticationFilter` defines how NGINX communicates with the IdP. The only required fields are `issuer`, `clientID`, and `clientSecretRef`. Everything else is optional. ```yaml kubectl apply -f - < ``` ### Deploy the HTTPRoutes Create an HTTPRoute with two rules. The `/coffee` rule attaches the `AuthenticationFilter`, whereas `/tea` is publicly accessible with no authentication required. ```yaml kubectl apply -f - < ``` --- ## Verify OIDC authentication For local testing, add the following entry to your `/etc/hosts` file so your browser can resolve `cafe.example.com` to the Gateway's public IP: ```text cafe.example.com ``` The steps below use a browser for OIDC since the flow involves redirects and cookies that curl cannot handle end-to-end. ### Accessing the protected `/coffee` route Open `https://cafe.example.com:$GW_PORT/coffee` in a browser. Because the route has an `AuthenticationFilter`, NGINX will: 1. Detect there is no valid session cookie. 1. Redirect your browser to the IdP's login page. 1. Log in with username `testuser` and password `testpassword`. 1. After you log in, redirect you back to NGINX with an authorization code. 1. Exchange the code for tokens in the background, set a session cookie, and forward you to the `coffee` backend. You will see a response from the `coffee` application only after a successful login. ### Accessing the unprotected `/tea` route Since `/tea` requires no authentication, you can access it directly with curl: ```shell curl -k --resolve cafe.example.com:$GW_PORT:$GW_IP https://cafe.example.com:$GW_PORT/tea ``` ```text Server address: 10.244.0.10:8080 Server name: tea-75bc9f4b6d-ms2n8 Date: 17/Mar/2026:10:01:00 +0000 URI: /tea Request ID: c7eb0509303de1c160cb7e7d2ac1d99f ``` The `tea` backend responds immediately with no authentication challenge because no `AuthenticationFilter` is attached to that rule. ## Optional configuration ### Session management By default, NGINX issues a session cookie named `NGX_OIDC_SESSION` with an 8-hour timeout that resets on each request to a protected resource. Use `session.cookieName` and `session.timeout` to override these values. ```yaml spec: type: OIDC oidc: session: cookieName: my-app-session timeout: 30m ``` ### Logout Use `logout.uri` to set the path a user visits to log out. When a request hits that path, NGINX clears the session and redirects to the IdP's logout endpoint. If `logout.postLogoutURI` is not set, NGINX returns a `200 OK` with the body "You have been logged out.". It can be set to a path to redirect the user there after logout. The path must be matched by an existing HTTPRoute rule. Set it to a full URL to redirect the user to an external page. Use `logout.frontChannelLogoutURI` if your IdP uses front-channel logout, where the IdP sends a logout request to a browser-visible URL to clear the NGINX session. The IdP must send `iss` and `sid` as query parameters. Set `logout.tokenHint` to `true` if your IdP requires the original ID token to be passed in the logout request. ```yaml spec: type: OIDC oidc: logout: uri: /logout postLogoutURI: /after_logout frontChannelLogoutURI: /frontchannel_logout tokenHint: true ``` ### PKCE PKCE (Proof Key for Code Exchange) prevents authorization code interception attacks. NGINX enables it automatically when the IdP requires the `S256` code challenge method. Set `pkce` explicitly if you need to force it on or off. ```yaml spec: type: OIDC oidc: pkce: true ``` ### Extra authentication arguments Use `extraAuthArgs` to append additional query parameters to the authorization request sent to the IdP. For example, `prompt: "login"` forces the IdP to show the login page on every request, and `max_age` sets the maximum time in seconds since the user last authenticated before re-authentication is required. ```yaml spec: type: OIDC oidc: extraAuthArgs: prompt: "login" max_age: "3600" ``` ### Custom redirect URI By default, NGINX Gateway Fabric registers the OIDC callback at `/oidc_callback__`. Use `redirectURI` to set a different path. If you provide a path-only value, NGINX creates a location block to handle the callback. If you provide a full URL, it is treated as an external handler and no location block is created. Register the same value in your IdP as an allowed redirect URI. ```yaml spec: type: OIDC oidc: redirectURI: /my_callback ``` ### Custom IdP metadata URL By default, NGINX fetches IdP metadata from `/.well-known/openid-configuration`. Use `configURL` if your IdP exposes metadata at a different path. ```yaml spec: type: OIDC oidc: configURL: "https://keycloak.example.com/realms/my-realm/.well-known/openid-configuration" ``` ### Certificate Revocation List (CRL) A CRL is a list of certificate serial numbers that a CA has revoked before expiry. When NGINX connects to the IdP over TLS, it checks the IdP's certificate against the CRL and rejects the connection if the certificate has been revoked. You are responsible for keeping the CRL up to date. A stale CRL may not catch recently revoked certificates. The `crlSecretRef` and `caCertificateRefs` fields are separate so you can rotate the CRL independently, though both keys can live in the same Secret. ```yaml spec: type: OIDC oidc: crlSecretRef: name: oidc-crl ``` --- ## Troubleshooting ### AuthenticationFilter is not accepted - Confirm the filter's `type` is `OIDC` and the `oidc` block is present. - Check that the Secrets referenced by `clientSecretRef`, `caCertificateRefs`, and `crlSecretRef` exist in the same namespace as the filter and contain the expected keys (`client-secret`, `ca.crt`, `ca.crl`). ### HTTPRoute is not accepted or reports `ResolvedRefs=False` - Verify the `extensionRef` in the HTTPRoute matches the `AuthenticationFilter` name and namespace exactly. - Confirm the route's `parentRefs` points to an HTTPS listener. OIDC filters are rejected on non-HTTPS listeners. ### Browser is stuck in a redirect loop - Confirm the `redirectURI` registered in the IdP exactly matches the path NGINX is using (default: `/oidc_callback__`). - Ensure the Gateway's TLS certificate is valid for the hostname the browser is using. ## Further reading - [NGINX OIDC module reference](https://nginx.org/en/docs/http/ngx_http_oidc_module.html) - [How OpenID Connect works](https://openid.net/developers/how-connect-works/) - [Single Sign-On with OpenID Connect and Identity Providers](https://docs.nginx.com/nginx/admin-guide/security-controls/configuring-oidc) -------------------------------------------------------------------------------- title: "Data plane configuration" url: https://docs.nginx.com/nginx-gateway-fabric/how-to/data-plane-configuration/index.md -------------------------------------------------------------------------------- Learn how to dynamically update the NGINX Gateway Fabric global data plane configuration. ## Overview NGINX Gateway Fabric can dynamically update the global data plane configuration without restarting. The data plane configuration contains configuration for NGINX that is not available using the standard Gateway API resources. This includes options such as configuring an OpenTelemetry collector, disabling HTTP/2, changing the IP family, and setting the NGINX error log level. It also contains options for [updating the NGINX Deployment and Service config](#configure-infrastructure-related-settings), either through native fields or patches. The data plane configuration is stored in the `NginxProxy` custom resource, which is a namespace-scoped resource that can be attached to a GatewayClass or Gateway. When attached to a GatewayClass, the fields in the NginxProxy affect all Gateways that belong to the GatewayClass. When attached to a Gateway, the fields in the NginxProxy only affect the Gateway. If a GatewayClass and its Gateway both specify an NginxProxy, the GatewayClass NginxProxy provides defaults that can be overridden by the Gateway NginxProxy. See the [Merging Semantics](#merging-semantics) section for more detail. **For a full list of configuration options that can be set, see the `NginxProxy spec` in the [API reference](/ngf/reference/api.md).** --- ## Merging Semantics NginxProxy resources are merged when a GatewayClass and a Gateway reference different NginxProxy resources. For fields that are bools, integers, and strings: - If a field on the Gateway's NginxProxy is unspecified (`nil`), the Gateway __inherits__ the value of the field in the GatewayClass's NginxProxy. - If a field on the Gateway's NginxProxy is specified, its value __overrides__ the value of the field in the GatewayClass's NginxProxy. For array fields: - If the array on the Gateway's NginxProxy is unspecified (`nil`), the Gateway __inherits__ the entire array in the GatewayClass's NginxProxy. - If the array on the Gateway's NginxProxy is empty, it __overrides__ the entire array in the GatewayClass's NginxProxy, effectively unsetting the field. - If the array on the Gateway's NginxProxy is specified and not empty, it __overrides__ the entire array in the GatewayClass's NginxProxy. ### Merging Examples This section contains examples of how NginxProxy resources are merged when they are attached to both a Gateway and its GatewayClass. #### Disable HTTP/2 for a Gateway A GatewayClass references the following NginxProxy which explicitly allows HTTP/2 traffic and sets the IPFamily to ipv4: ```yaml apiVersion: gateway.nginx.org/v1alpha2 kind: NginxProxy metadata: name: gateway-class-enable-http2 namespace: default spec: ipFamily: "ipv4" disableHTTP: false ``` To disable HTTP/2 traffic for a particular Gateway, reference the following NginxProxy in the Gateway's spec: ```yaml apiVersion: gateway.nginx.org/v1alpha2 kind: NginxProxy metadata: name: gateway-disable-http namespace: default spec: disableHTTP: true ``` These NginxProxy resources are merged and the following settings are applied to the Gateway: ```yaml ipFamily: "ipv4" disableHTTP: true ``` #### Change Telemetry configuration for a Gateway A GatewayClass references the following NginxProxy which configures telemetry: ```yaml apiVersion: gateway.nginx.org/v1alpha2 kind: NginxProxy metadata: name: gateway-class-telemetry namespace: default spec: telemetry: exporter: endpoint: "my.telemetry.collector:9000" interval: "60s" batchSize: 20 serviceName: "my-company" spanAttributes: - key: "company-key" value: "company-value" ``` To change the telemetry configuration for a particular Gateway, reference the following NginxProxy in the Gateway's spec: ```yaml apiVersion: gateway.nginx.org/v1alpha2 kind: NginxProxy metadata: name: gateway-telemetry-service-name namespace: default spec: telemetry: exporter: batchSize: 50 batchCount: 5 serviceName: "my-app" spanAttributes: - key: "app-key" value: "app-value" ``` These NginxProxy resources are merged and the following settings are applied to the Gateway: ```yaml telemetry: exporter: endpoint: "my.telemetry.collector:9000" interval: "60s" batchSize: 50 batchCount: 5 serviceName: "my-app" spanAttributes: - key: "app-key" value: "app-value" ``` #### Disable Tracing for a Gateway A GatewayClass references the following NginxProxy which configures telemetry: ```yaml apiVersion: gateway.nginx.org/v1alpha2 kind: NginxProxy metadata: name: gateway-class-telemetry namespace: default spec: telemetry: exporter: endpoint: "my.telemetry.collector:9000" interval: "60s" serviceName: "my-company" ``` To disable tracing for a particular Gateway, reference the following NginxProxy in the Gateway's spec: ```yaml apiVersion: gateway.nginx.org/v1alpha2 kind: NginxProxy metadata: name: gateway-disable-tracing namespace: default spec: telemetry: disabledFeatures: - DisableTracing ``` These NginxProxy resources are merged and the following settings are applied to the Gateway: ```yaml telemetry: exporter: endpoint: "my.telemetry.collector:9000" interval: "60s" serviceName: "my-app" disabledFeatures: - DisableTracing ``` --- ## Configuring the GatewayClass NginxProxy on install By default, an `NginxProxy` resource is created in the same namespace where NGINX Gateway Fabric is installed, attached to the GatewayClass. You can set configuration options in the `nginx` Helm value section, and the resource will be created and attached using the set values. You can also [manually create and attach](#manually-creating-nginxProxies) specific `NginxProxy` resources to target different Gateways. When installed using the Helm chart, the NginxProxy resource is named `-proxy-config` and is created in the release Namespace. **For a full list of configuration options that can be set, see the `NginxProxy spec` in the [API reference](/ngf/reference/api.md).** **Note:** Some global configuration also requires an [associated policy](/ngf/overview/custom-policies.md) to fully enable a feature (such as [tracing](/ngf/monitoring/tracing.md), for example). --- ## Manually Creating NginxProxies The following command creates a basic `NginxProxy` configuration in the `default` namespace that sets the IP family to `ipv4` instead of the default value of `dual`: ```yaml kubectl apply -f - < ``` This will open your default editor, allowing you to add the following to the `spec`: ```yaml infrastructure: parametersRef: group: gateway.nginx.org kind: NginxProxy name: ngf-proxy-config ``` **Note:** The `NginxProxy` resource must reside in the same namespace as the Gateway it is attached to. After updating, you can check the status of the Gateway to see if the configuration is valid: ```shell kubectl describe gateway ``` ```text ... Status: Conditions: ... Message: parametersRef resource is resolved Observed Generation: 1 Reason: ResolvedRefs Status: True Type: ResolvedRefs ``` If everything is valid, the `ResolvedRefs` condition should be `True`. Otherwise, you will see an `InvalidParameters` condition in the status. --- ## Configure the data plane log level You can use the `NginxProxy` resource to dynamically configure the log level. The following command creates a basic `NginxProxy` configuration that sets the log level to `warn` instead of the default value of `info`: ```yaml kubectl apply -f - < ``` **Note:** In a production environment, you should have a DNS record for the external IP address that is exposed, and it should refer to the hostname that the gateway will forward for. ## RequestHeaderModifier example This examples demonstrates how to configure traffic routing for a simple echo server. A HTTPRoute resource is used to route traffic to the headers application, using the `RequestHeaderModifier` filter to modify headers in the request. You can then verify that the server responds with the modified request headers. ### Deploy the Headers application Begin by deploying the example application `headers`. It is a simple application that returns the request headers which will be modified later. ```shell kubectl apply -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v/examples/http-request-header-filter/headers.yaml ``` This will create the headers Service and a Deployment with one Pod. Run the following command to verify the resources were created: ```shell kubectl get pods,svc ``` ```text pod/headers-545698447b-z52kj 1/1 Running 0 23s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/headers ClusterIP 10.96.26.161 80/TCP 23s ``` ### Configure the HTTPRoute with RequestHeaderModifier filter Create a HTTPRoute that exposes the header application outside the cluster using the listener created in the previous section. Use the following command: ```yaml kubectl apply -f - < 80/TCP 95s ``` ### Configure the basic HTTPRoute Create a HTTPRoute that exposes the header application outside the cluster using the listener created in the previous section. You can do this with the following command: ```yaml kubectl apply -f - <` to inspect status conditions. This page documents all condition types, reasons, and common troubleshooting steps. --- ## Condition reference ### Accepted | Status | Reason | Meaning | |---------|--------------------|-----------------------------------------------------------------------------| | `True` | `Accepted` | Policy is valid and targets a known resource | | `False` | `Invalid` | Policy spec fails validation (for example, wrong source field for the type) | | `False` | `TargetNotFound` | The targeted Gateway or Route does not exist | | `False` | `Conflicted` | Another WAFPolicy already targets this resource at the same level | | `False` | `NginxProxyNotSet` | WAF is not enabled in the referenced NginxProxy | ### ResolvedRefs | Status | Reason | Meaning | |---------|--------------------|--------------------------------------------------------------------------| | `True` | `ResolvedRefs` | All referenced Secrets, `APPolicy`, and `APLogConf` resources resolved successfully | | `False` | `InvalidRef` | A referenced Secret was not found or is missing expected keys; or a referenced `APPolicy`/`APLogConf` doesn't exist | | `False` | `RefNotPermitted` | A referenced `APPolicy` or `APLogConf` is in a different namespace and no `ReferenceGrant` permits the reference | ### Programmed | Status | Reason | Meaning | |---------|----------------------|----------------------------------------------------------------------------------| | `True` | `Programmed` | Bundle fetched and deployed to the data plane | | `True` | `BundleUpdated` | A poll cycle detected a changed bundle and deployed it | | `True` | `StaleBundleWarning` | A poll cycle failed; previously deployed bundle remains active | | `False` | `FetchError` | Bundle could not be fetched (network error, HTTP error, auth failure, timeout) | | `False` | `IntegrityError` | Bundle checksum verification failed | | `False` | `Pending` | Bundle has never been fetched; configuration withheld or WAF omitted (fail-open) | --- ## Common issues ### `FetchError` with HTTP 403 The credentials Secret is either missing, contains the wrong keys, or the credentials are invalid. Verify the Secret exists in the same namespace as the `WAFPolicy` and that the keys match the authentication method (`username`/`password` for Basic Auth, `token` for Bearer/APIToken). ### `FetchError` with HTTP 404 on NGINX Instance Manager or NGINX One Console The referenced policy was not found or has not been compiled yet. For NGINX Instance Manager, verify that compilation succeeded in the NGINX Instance Manager console before creating the `WAFPolicy`. For NGINX One Console, NGINX Gateway Fabric triggers compilation if no bundle exists, and a 404 after initial setup may indicate the policy was deleted in NGINX One Console. ### `Pending` The bundle has never been successfully fetched. If `bundleFailOpen` is `false` (the default), the NGINX configuration push is withheld for this Gateway. If `bundleFailOpen` is `true`, traffic flows without WAF protection. Check the `Programmed` condition message for the last fetch error. Common causes include network connectivity issues, incorrect URLs, or authentication failures. Verify the policy source URL and credentials Secret. ### `IntegrityError` The downloaded bundle does not match the expected checksum. For HTTP source, ensure the `.sha256` file matches the bundle file. For `expectedChecksum`, verify the digest matches the bundle you intend to deploy. ### Policy not applied to a route The route does not show a `gateway.nginx.org/WAFPolicyAffected` condition. Verify that: - The `WAFPolicy` `targetRefs` field matches the Gateway or Route name and namespace. - The Gateway has `waf.enable: true` in its referenced `NginxProxy`. - The `WAFPolicy` `Accepted` condition is `True`. ### WAF sidecars not starting Verify that the `waf-enforcer` and `waf-config-mgr` container images are accessible from your cluster, and that any required `imagePullSecrets` are configured in the `NginxProxy` Kubernetes spec. ### Duplicate policy name error If two `WAFPolicy` resources in the same Gateway reference different compiled bundles that were compiled under the **same policy name**, the WAF engine rejects the configuration with an error like: ```text "error_message": "Duplicate policy name found: " ``` This occurs because the WAF engine uses the logical policy name embedded inside the compiled bundle — not the Kubernetes resource name or bundle filename — to identify policies. When the same logical name appears more than once in a single NGINX configuration, the configuration test fails and the update is rolled back. **How to identify the problem:** Check the NGINX Gateway Fabric controller logs for a configuration error containing `Duplicate policy name found`: ```shell kubectl logs -n nginx-gateway deploy/nginx-gateway -c nginx-gateway | grep "Duplicate policy name" ``` **Resolution:** Each `WAFPolicy` attached to a Gateway must reference a compiled bundle with a unique logical policy name. This is the `name` field set inside the policy definition JSON at compile time, not the `WAFPolicy` resource name or the bundle filename. To resolve the conflict, choose one of the following approaches: - **Recompile with a distinct name**: Update the policy definition to use a unique `name` field for each policy, then recompile and republish the bundle. - **Pin a single version per Gateway**: If the intent is to apply the same policy everywhere, use a single gateway-level `WAFPolicy` instead of multiple route-level policies referencing different versions of the same named policy. - **Check for overlapping WAFPolicies**: Run `kubectl get wafpolicies -A` and confirm that no two policies targeting the same Gateway reference bundles compiled from definitions with the same logical policy name. --- ### Security events aren't reaching NGINX Instance Manager F5 WAF for NGINX generates security events, but they don't appear in the NGINX Instance Manager Security Monitoring dashboard, even though the `WAFPolicy` resource shows `Programmed`. **How to identify the problem:** Check whether the event reached NGINX Agent inside the pod: ```shell kubectl exec -n -c nginx -- \ tail -100 /var/log/nginx-agent/opentelemetry-collector-agent.log ``` If the event isn't in this log, F5 WAF for NGINX isn't reaching NGINX Agent. If the event is in the log but not in NGINX Instance Manager, the export from NGINX Agent is failing. **Resolution:** - **Event missing from the NGINX Agent log:** Confirm the `WAFPolicy` `securityLogs.destination.syslog.server` field is set to exactly `localhost:1514`. Any other value prevents the event from reaching NGINX Agent, which listens on `127.0.0.1:1514` inside the `nginx` container. - **Event in the log but export fails:** Check the log for `Unauthenticated` errors. A JWT authentication failure between NGINX Agent and NGINX Instance Manager causes this error. This is the same NGINX Plus subscription JWT used to create the NGINX Plus Secret in [Connect NGINX Gateway Fabric to NGINX Instance Manager](/nim/connect-kubernetes/connect-ngf.md). If the JWT has expired or was revoked, download a new one from [MyF5](https://my.f5.com/manage/s/) and repeat the steps to recreate the Secret. NGINX Gateway Fabric doesn't pick up a rotated Secret automatically. Restart the Gateway pod after recreating it. - **Export succeeds but NGINX Instance Manager shows nothing:** Confirm NGINX Instance Manager's embedded OpenTelemetry collector is running and reachable on port `4317`. See [Troubleshooting](/nim/security-monitoring/troubleshooting.md) for the NGINX Instance Manager–side checks. --- ## See also - [F5 WAF for NGINX overview](/ngf/waf-integration/overview.md) - [Configure WAF settings](/ngf/waf-integration/configuration.md) - [WAFPolicy and NginxProxy API reference](/ngf/reference/api.md) -------------------------------------------------------------------------------- title: "Use Manifests to uninstall NGINX Gateway Fabric" url: https://docs.nginx.com/nginx-gateway-fabric/install/manifests/uninstall/index.md -------------------------------------------------------------------------------- This page describes how to uninstall NGINX Gateway Fabric in a Manifest-based deployment, its custom resource definitions, and its Gateway API resources. By following these instrucions, you will remove NGINX Gateway Fabric and all Gateway resources from a Kubernetes cluster. ## Remove NGINX Gateway Fabric To remove NGINX Gateway Fabric run: ```shell kubectl delete namespace nginx-gateway kubectl delete clusterrole nginx-gateway kubectl delete clusterrolebinding nginx-gateway ``` ## Delete the custom resource definitions To remove the NGINX Gateway Fabric custom resource definitions, run: ```shell kubectl delete -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v/deploy/crds.yaml ``` #### Example output ```text customresourcedefinition.apiextensions.k8s.io "authenticationfilters.gateway.nginx.org" deleted customresourcedefinition.apiextensions.k8s.io "clientsettingspolicies.gateway.nginx.org" deleted customresourcedefinition.apiextensions.k8s.io "nginxgateways.gateway.nginx.org" deleted customresourcedefinition.apiextensions.k8s.io "nginxproxies.gateway.nginx.org" deleted customresourcedefinition.apiextensions.k8s.io "observabilitypolicies.gateway.nginx.org" deleted customresourcedefinition.apiextensions.k8s.io "proxysettingspolicies.gateway.nginx.org" deleted customresourcedefinition.apiextensions.k8s.io "ratelimitpolicies.gateway.nginx.org" deleted customresourcedefinition.apiextensions.k8s.io "snippetsfilters.gateway.nginx.org" deleted customresourcedefinition.apiextensions.k8s.io "snippetspolicies.gateway.nginx.org" deleted customresourcedefinition.apiextensions.k8s.io "upstreamsettingspolicies.gateway.nginx.org" deleted ``` ## Remove the Gateway API resources To uninstall the Gateway API resources, run the command based on your deployment type: #### Default ```shell kubectl kustomize "https://github.com/nginx/nginx-gateway-fabric/config/crd/gateway-api/standard?ref=v" | kubectl delete -f - ``` #### Experimental ```shell kubectl kustomize "https://github.com/nginx/nginx-gateway-fabric/config/crd/gateway-api/experimental?ref=v" | kubectl delete -f - ``` #### Example output ```text customresourcedefinition.apiextensions.k8s.io "backendtlspolicies.gateway.networking.k8s.io" deleted customresourcedefinition.apiextensions.k8s.io "gatewayclasses.gateway.networking.k8s.io" deleted customresourcedefinition.apiextensions.k8s.io "gateways.gateway.networking.k8s.io" deleted customresourcedefinition.apiextensions.k8s.io "grpcroutes.gateway.networking.k8s.io" deleted customresourcedefinition.apiextensions.k8s.io "httproutes.gateway.networking.k8s.io" deleted customresourcedefinition.apiextensions.k8s.io "referencegrants.gateway.networking.k8s.io" deleted ``` -------------------------------------------------------------------------------- title: "Configure external authentication" url: https://docs.nginx.com/nginx-gateway-fabric/traffic-security/external-authentication/index.md -------------------------------------------------------------------------------- This guide describes how to configure external authentication in NGINX Gateway Fabric using the `ExternalAuth` filter on an HTTPRoute. External authentication delegates the authorization decision for each request to an external service. NGINX issues a subrequest to that service before proxying the original request, and forwards the request only if the service responds with a 2xx status. Following these instructions to create two sample applications and compare the behavior of each: - `coffee` endpoint: Protected by an `ExternalAuth` filter. - `tea` endpoint: Exposed without any external authentication filter. ## Overview The `ExternalAuth` filter is declared in the `filters` list of an HTTPRoute rule. When NGINX processes a request that matches the rule, it first sends a subrequest to the backend referenced by the filter. Based on the status returned by that backend, NGINX either forwards the original request to the route's `backendRefs` or returns the error status to the client. Each route rule supports only one `ExternalAuth` filter. If your authentication flow requires multiple checks, consolidate them into a single authentication service that performs all the necessary validations. The filter translates to NGINX's [ngx_http_auth_request_module](https://nginx.org/en/docs/http/ngx_http_auth_request_module.html) directives: - [`auth_request`](https://nginx.org/en/docs/http/ngx_http_auth_request_module.html#auth_request) — Sends a subrequest to the specified URI and grants or denies access based on the response status. - [`auth_request_set`](https://nginx.org/en/docs/http/ngx_http_auth_request_module.html#auth_request_set) — Captures a value from the authentication response and stores it in a variable for use in the main request. ## Note on Gateway API Experimental Features **important:** ExternalAuth is a Gateway API resource from the experimental release channel. To use Gateway API experimental resources, the Gateway API resources from the experimental channel must be installed before deploying NGINX Gateway Fabric. Additionally, NGINX Gateway Fabric must have experimental features enabled. **Note:** As noted in the [Gateway API documentation](https://gateway-api.sigs.k8s.io/guides/#install-experimental-channel), future releases of the Gateway API can include breaking changes to experimental resources and fields. To install the Gateway API resources from the experimental channel, run the following: ```shell kubectl kustomize "https://github.com/nginx/nginx-gateway-fabric/config/crd/gateway-api/experimental?ref=v" | kubectl apply -f - ``` **Note:** If you plan to use the `edge` version of NGINX Gateway Fabric, you can replace the version in `ref` with `main`, for example `ref=main`. To enable experimental features on NGINX Gateway Fabric: Using Helm: Set `nginxGateway.gwAPIExperimentalFeatures.enable` to true. An example can be found in the [Installation with Helm](/ngf/install/helm.md#custom-installation-options) guide. Using Kubernetes manifests: Add the `--gateway-api-experimental-features` command-line flag to the deployment manifest args. An example can be found in the [Installation with Kubernetes manifests](/ngf/install/manifests.md#3-deploy-nginx-gateway-fabric) guide. ## Before you begin - [Install](/ngf/install/) NGINX Gateway Fabric with experimental features enabled. ## Deploy sample applications Run the following `kubectl apply` command to create the `coffee` and `tea` deployments and services: ```yaml kubectl apply -f - < ``` ## Deploy the external authentication server This sample authentication service is an NGINX deployment that checks the `X-Api-Key` request header. If the header value is `my-custom-secret`, the server responds with `200 OK`; otherwise it responds with `401 Unauthorized`. ```yaml kubectl apply -f - <<'EOF' apiVersion: v1 kind: ConfigMap metadata: name: ext-auth-config data: default.conf: | server { listen 8080; location / { if ($http_x_api_key != "my-custom-secret") { return 401 "unauthorized"; } return 200 "ok"; } } --- apiVersion: apps/v1 kind: Deployment metadata: name: ext-auth-server spec: replicas: 1 selector: matchLabels: app: ext-auth-server template: metadata: labels: app: ext-auth-server spec: containers: - name: nginx image: nginx:latest ports: - containerPort: 8080 volumeMounts: - name: config mountPath: /etc/nginx/conf.d volumes: - name: config configMap: name: ext-auth-config --- apiVersion: v1 kind: Service metadata: name: ext-auth-server spec: ports: - port: 80 targetPort: 8080 protocol: TCP name: http selector: app: ext-auth-server EOF ``` ## Configure routing with the ExternalAuth filter Run the following `kubectl apply` command to create an HTTPRoute for `coffee` and `tea` applications. The `coffee` route uses an `ExternalAuth` filter to require authentication, while the `tea` route is exposed without one: ```yaml kubectl apply -f - < 401 Authorization Required

401 Authorization Required

nginx
``` Access `/coffee` with a valid API key: ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee -H "X-Api-Key: my-custom-secret" ``` ```text Server address: 10.244.0.151:8080 Server name: coffee-654ddf664b-l9ml5 Date: 16/Apr/2026:20:14:28 +0000 URI: /coffee Request ID: 217931bc5fe27254d1821cec91e1f2d8 ``` The `X-Api-Key` header is listed in `allowedHeaders` so that it reaches the authentication server, which responds `200 OK`. NGINX then proxies the request to the `coffee` backend. Access `/tea`, which has no `ExternalAuth` filter and responds normally: ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/tea ``` ```text Server address: 10.244.0.149:8080 Server name: tea-75bc9f4b6d-q5wg5 Date: 16/Apr/2026:20:14:41 +0000 URI: /tea Request ID: d27f6ef4edc2f1e09bb455824ac67a07 ``` ### Exceed the body size limit Because `forwardBody.maxSize: 1024` is applied as `client_max_body_size` on the `/coffee` location, any client request with a body larger than 1024 bytes is rejected with `413 Request Entity Too Large` before the authorization subrequest runs. Send a 1100-byte body to demonstrate this: ```shell BODY=$(head -c 1100 /dev/zero | tr '\0' 'x') curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee -X POST -H "X-Api-Key: my-custom-secret" -d "$BODY" ``` ```text 413 Request Entity Too Large

413 Request Entity Too Large

nginx
``` `client_max_body_size` can also be set on a route through a [ClientSettingsPolicy](/ngf/traffic-management/client-settings.md) via its `body.maxSize` field. If a ClientSettingsPolicy with `body.maxSize` is attached to the same HTTPRoute as an `ExternalAuth` filter that sets `forwardBody.maxSize`, the HTTPRoute is marked invalid with reason `InvalidFilter`. ## Troubleshooting - If the HTTPRoute is not accepted, run `kubectl describe httproute coffee` and check the `Status` conditions for validation errors. - If every request returns `401`, confirm that the authentication server is reachable from the NGINX pod and that the `backendRef` name, namespace, and port are correct. - If a required request header cannot reach the authentication server, confirm it is listed in `http.allowedHeaders`. - If a response header from the authentication server cannot reach the backend, confirm it is listed in `http.allowedResponseHeaders`. - If a request is rejected with `413 Request Entity Too Large`, raise `forwardBody.maxSize` to accommodate the client body. - If the HTTPRoute reports `ResolvedRefs: False` with an `InvalidFilter` reason mentioning `body.maxSize`, remove either the `ExternalAuth` filter's `forwardBody.maxSize` or the ClientSettingsPolicy's `body.maxSize` as they both cannot be set on the same route. ## Further reading - [NGINX HTTP auth request module](https://nginx.org/en/docs/http/ngx_http_auth_request_module.html) - [Gateway API HTTPExternalAuthFilter specification](https://gateway-api.sigs.k8s.io/reference/spec/#httpexternalauthfilter) -------------------------------------------------------------------------------- title: "Configure JSON Web Token (JWT) authentication" url: https://docs.nginx.com/nginx-gateway-fabric/traffic-security/jwt-authentication/index.md -------------------------------------------------------------------------------- This guide describes how to configure JSON Web Token (JWT) authentication in NGINX Gateway Fabric using the AuthenticationFilter custom resource definition (CRD). JWT authentication secures applications and APIs by validating JSON Web Tokens in incoming requests. Only requests with valid JWTs are allowed access. By following these instructions, you will create two sample application endpoints: one with JWT authentication and one without, so you can see how each behaves. **Note:** JWT authentication requires NGINX Plus. ## Overview JWT authentication in NGINX Gateway Fabric validates JSON Web Tokens using JSON Web Key Sets (JWKS). The JWKS contains the public keys used to verify JWT signatures. When a request arrives with a JWT in the `Authorization` header, NGINX Plus validates the token against the configured JWKS before forwarding the request to your application. NGINX Gateway Fabric supports two JWKS source types, set using the `source` field on the `AuthenticationFilter`: - **File** — JWKS is stored locally in a Kubernetes Secret. Use this when you manage your own keys or want to avoid external dependencies. - **Remote** — NGINX Plus fetches JWKS from an HTTPS endpoint at runtime. Use this when your identity provider (for example, Keycloak or Auth0) exposes a JWKS URI. ## Before you begin - [Install](/ngf/install/) NGINX Gateway Fabric with NGINX Plus. ## Common setup The following steps are required for both file-based and remote JWT authentication. ### Deploy sample applications To deploy the `coffee` and `tea` applications, run the following YAML with `kubectl apply`: ```yaml kubectl apply -f - < ``` --- ## File-based JWT authentication Use file-based JWT authentication when your JWKS is stored in a Kubernetes Secret. NGINX Plus loads the key material directly from the Secret at startup and after each reload. ### Generate a JWKS For testing purposes, the following example shows a simple JWKS with a single RSA key. In production, use properly generated keys from your identity provider or key management system. ```json { "keys": [ { "kty": "RSA", "kid": "test-key", "use": "sig", "n": "0vx7agoebGcQSuuPiLJXZptN9nndrQmbXEps2aiAFbWhM78LhWx4cbbfAAtVT86zwu1RK7aPFFxuhDR1L6tSoc_BJECPebWKRXjBZCiFV4n3oknjhMstn64tZ_2W-5JsGY4Hc5n9yBXArwl93lqt7_RN5w6Cf0h4QyQ5v-65YGjQR0_FDW2QvzqY368QQMicAtaSqzs8KJZgnYb9c7d0zgdAZHzu6qMQvRL5hajrn1n91CbOpbISD08qNLyrdkt-bFTWhAI4vMQFh6WeZu0fM4lFd2NcRwr3XPksINHaQ-G_xBniIqbw0Ls1jF44-csFCur-kEgU8awapJzKnqDKgw", "e": "AQAB" } ] } ``` **Note:** This example JWKS is for demonstration only. In production, use keys from your identity provider or key management system. ### Create a JWKS Secret and AuthenticationFilter Deploy a Secret containing your JWKS and the AuthenticationFilter by running these `kubectl` commands: ``` kubectl create secret generic jwks-secret --from-file=auth=jwks.json ``` ```yaml kubectl apply -f - < ``` ### Deploy an HTTPRoute referencing the AuthenticationFilter Deploy an HTTPRoute that references the AuthenticationFilter using the `ExtensionRef` filter type. In this example, the filter is applied to the `/coffee` path only. Run the following YAML with `kubectl apply`: ```yaml kubectl apply -f - < ``` ### Verify file-based JWT authentication **Note:** Your clients should be able to resolve "cafe.example.com" to the public IP of the NGINX Service. This guide simulates that using curl's `--resolve` option. To test the authentication, you need a JWT signed with the private key that corresponds to the public key in your JWKS. You can use [jwt.io](https://jwt.io) or other JWT tools to generate one. Store it in a shell variable: ```shell JWT_TOKEN="" ``` Access `/coffee` with a valid JWT: ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee -H "Authorization: Bearer $JWT_TOKEN" ``` ```text Server address: 10.244.0.7:8080 Server name: coffee-654ddf664b-nhhvr Date: 10/Mar/2026:15:20:15 +0000 URI: /coffee Request ID: 13a925b2514b62c45ea4a79800248d5c ``` Access `/coffee` without a JWT: ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee ``` ```text 401 Authorization Required

401 Authorization Required

nginx
``` Access `/coffee` with an invalid JWT: ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee -H "Authorization: Bearer invalid.jwt.token" ``` ```text 401 Authorization Required

401 Authorization Required

nginx
``` Access `/tea`, which has no AuthenticationFilter and responds normally: ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/tea ``` ```text Server address: 10.244.0.10:8080 Server name: tea-75bc9f4b6d-ms2n8 Date: 10/Mar/2026:15:36:26 +0000 URI: /tea Request ID: c7eb0509303de1c160cb7e7d2ac1d99f ``` --- ## Remote JWT authentication Use remote JWT authentication when your identity provider (IdP) exposes a JWKS endpoint. NGINX Plus fetches the JWKS from the URI at runtime using an internal subrequest, so keys are always up to date without requiring a Secret or NGINX reload. ### Install cert-manager Install cert-manager onto the cluster using Helm with Gateway API features enabled. - Add the Helm repository. ```shell helm repo add jetstack https://charts.jetstack.io helm repo update ``` - Install cert-manager, and enable the GatewayAPI feature gate: ```shell helm install \ cert-manager jetstack/cert-manager \ --namespace cert-manager \ --create-namespace \ --set config.apiVersion="controller.config.cert-manager.io/v1alpha1" \ --set config.kind="ControllerConfiguration" \ --set config.enableGatewayAPI=true \ --set crds.enabled=true ``` ### Generate certificates The following steps use `cert-manager` to issue a local Certificate Authority (CA) and sign certificates for both Keycloak and NGINX. `cert-manager` creates the required Kubernetes Secrets directly so no manual secret creation is needed for TLS. Create a self-signed `ClusterIssuer` to bootstrap the CA, then issue the CA certificate and create a second `ClusterIssuer` backed by it: ```yaml kubectl apply -f - < ``` ### Deploy an HTTPRoute referencing the remote AuthenticationFilter Deploy an HTTPRoute that applies the remote AuthenticationFilter to `/coffee`. Run the following YAML with `kubectl apply`: ```yaml kubectl apply -f - < ``` ### Obtain a JWT Expose Keycloak with port-forward: ```shell kubectl port-forward svc/keycloak 8443:8443 ``` Set your `JWT_TOKEN` environment variable by calling the `tokens` endpoint in keycloak: ```shell export JWT_TOKEN=$(curl -s -k -X POST https://localhost:8443/realms/nginx-gateway/protocol/openid-connect/token \ -H "Content-Type: application/x-www-form-urlencoded" \ -d "username=testuser" \ -d "password=testpassword" \ -d "grant_type=password" \ -d "client_id=cafe-app" | jq -r '.access_token') ``` ### Verify remote JWT authentication **Note:** Your clients should be able to resolve "cafe.example.com" to the public IP of the NGINX Service. This guide simulates that using curl's `--resolve` option. Access `/coffee` with a valid JWT: ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee -H "Authorization: Bearer $JWT_TOKEN" ``` ```text Server address: 10.244.0.7:8080 Server name: coffee-654ddf664b-nhhvr Date: 10/Mar/2026:15:20:15 +0000 URI: /coffee Request ID: 13a925b2514b62c45ea4a79800248d5c ``` Access `/coffee` without a JWT: ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee ``` ```text 401 Authorization Required

401 Authorization Required

nginx
``` Access `/tea`, which has no AuthenticationFilter and responds normally: ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/tea ``` ```text Server address: 10.244.0.10:8080 Server name: tea-75bc9f4b6d-ms2n8 Date: 10/Mar/2026:15:36:26 +0000 URI: /tea Request ID: c7eb0509303de1c160cb7e7d2ac1d99f ``` --- ## Troubleshooting - Ensure NGINX Gateway Fabric is deployed with NGINX Plus. JWT authentication is not supported in the open source version. - Ensure the HTTPRoute is accepted and references the correct AuthenticationFilter name and group. - For file-based JWT: confirm the Secret key is named `auth` and contains valid JWKS JSON. The Secret must be in the same namespace as the AuthenticationFilter. - For remote JWT: confirm the `uri` uses the `https://` scheme and the endpoint is reachable from the NGINX Plus pod. - For remote JWT with a custom CA: confirm the Secret key is named `ca.crt` and contains a valid PEM certificate. The Secret must be in the same namespace as the AuthenticationFilter. - Verify your JWT includes the `kid` (key ID) claim that matches one of the keys in your JWKS. - Check that the JWT is not expired by verifying the `exp` claim. - Ensure the JWT signature algorithm (typically RS256) matches the key type in your JWKS. ## Further reading - [AuthenticationFilter API reference](/ngf/reference/api.md#gateway.nginx.org/v1alpha1.AuthenticationFilter) - [NGINX JWT Authentication Module](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html) - [JWT.io - JWT debugger and generator](https://jwt.io) - [RFC 7519 - JSON Web Token (JWT)](https://datatracker.ietf.org/doc/html/rfc7519) - [RFC 7517 - JSON Web Key (JWK)](https://datatracker.ietf.org/doc/html/rfc7517) -------------------------------------------------------------------------------- title: "Configure TLS routing with TLSRoute" url: https://docs.nginx.com/nginx-gateway-fabric/traffic-management/tlsroute/index.md -------------------------------------------------------------------------------- Learn how to configure TLS routing with [TLSRoute](https://gateway-api.sigs.k8s.io/reference/spec/#tlsroute) using NGINX Gateway Fabric. ## Overview TLSRoute supports two TLS modes: - **Passthrough**: The Gateway reads the SNI and forwards encrypted TCP traffic to the backend. The backend holds and terminates TLS with its own certificate. Use this mode when the backend needs its own certificate, or when you can't expose the private key to the gateway. - **Terminate**: The Gateway holds the certificate, terminates TLS, and forwards plain TCP to the backend. Use this mode when the backend shouldn't handle TLS, or when it serves a non-HTTP TCP protocol. **note:** You can add an HTTPS listener on the same port that terminates TLS connections, as long as the hostname doesn't overlap with the TLS listener hostname. ## Before you begin - [Install](/ngf/install/) NGINX Gateway Fabric. Set up cert-manager and a local CA for both examples: Install cert-manager onto the cluster using Helm with Gateway API features enabled. - Add the Helm repository. ```shell helm repo add jetstack https://charts.jetstack.io helm repo update ``` - Install cert-manager, and enable the GatewayAPI feature gate: ```shell helm install \ cert-manager jetstack/cert-manager \ --namespace cert-manager \ --create-namespace \ --set config.apiVersion="controller.config.cert-manager.io/v1alpha1" \ --set config.kind="ControllerConfiguration" \ --set config.enableGatewayAPI=true \ --set crds.enabled=true ``` Create a self-signed `ClusterIssuer`, a CA `Certificate`, and a CA-backed `ClusterIssuer`. cert-manager uses the resulting `local-ca-issuer` to sign certificates in any namespace: ```yaml kubectl apply -f - < 8443/TCP 12s ``` Create a Gateway with a TLS listener in passthrough mode. Copy and paste this into your terminal: ```yaml kubectl apply -f - < ``` **Note:** In a production environment, you should have a DNS record for the external IP address that is exposed, and it should refer to the hostname that the Gateway will forward for. Create a TLSRoute that attaches to the Gateway and routes requests to `app.example.com` to the `secure-app` Service: ```yaml kubectl apply -f - < GET / HTTP/1.1 > Host: app.example.com:8443 > User-Agent: curl/8.7.1 > Accept: */* > * Request completely sent off < HTTP/1.1 200 OK < Server: nginx/1.29.1 < Date: Wed, 06 May 2026 21:25:18 GMT < Content-Type: text/plain < Content-Length: 42 < Connection: keep-alive < hello from pod secure-app-59bbd475b-phgsv ``` Note that the server certificate used to terminate the TLS connection has the subject common name of `app.example.com`. This is the server certificate that the `secure-app` is configured with and shows that the TLS connection was terminated by the `secure-app`, not NGINX Gateway Fabric. ## TLS terminate In terminate mode, NGINX Gateway Fabric holds the TLS certificate, terminates the TLS connection, and forwards plain TCP traffic to the backend. The backend doesn't need a certificate or TLS configuration. Use TLS terminate mode when: - Your backend serves a non-HTTP TCP protocol, such as a database or custom binary protocol. - You want to centralize certificate management at the gateway rather than on each backend. **note:** If your backend serves HTTP traffic and you need HTTP-level routing — such as path matching or header manipulation — use an HTTPS listener with an HTTPRoute instead. See [Configure HTTPS termination](/ngf/traffic-management/https-termination.md). ### Set up Create a `Certificate` for `app.example.com`. cert-manager creates the `gateway-tls-secret` Secret, which the Gateway uses to terminate TLS: ```yaml kubectl apply -f - < 80/TCP 10s ``` Create a Gateway with a TLS listener in terminate mode. Copy and paste this into your terminal: ```yaml kubectl apply -f - < ``` **note:** In a production environment, you should have a DNS record for the external IP address that is exposed, and it should refer to the hostname that the Gateway will forward for. Create a TLSRoute that attaches to the Gateway and routes requests to `app.example.com` to the `app` Service: ```yaml kubectl apply -f - < ``` From the information obtained with `kubectl describe` you can see the default settings for the provisioned NGINX Deployment and Service. Under `Spec.Kubernetes` you can see a few things: - The NGINX container image settings - How many NGINX Deployment replicas are specified - The type of Service and external traffic policy **Note:** Depending on installation configuration, the default NginxProxy settings may be slightly different from what is shown in the example. For more information on NginxProxy and its configurable fields, see the [API reference](/ngf/reference/api.md). Modify the NginxProxy resource to change the type of Service. Use `kubectl edit` to modify the default NginxProxy and insert the following under `spec.kubernetes.service`: ```yaml type: NodePort ``` After saving the changes, use `kubectl get` on the service, and you should see the service type has changed to `NodePort`. ```shell kubectl get service cafe-nginx ``` ```text NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE cafe-nginx NodePort 10.96.172.204 80:32615/TCP 3h5m ``` ### Set annotations and labels on provisioned resources While the majority of configuration will happen on the NginxProxy resource, that is not always the case. Uniquely, if you want to set any annotations or labels on the NGINX Deployment or Service, you need to set those annotations on the Gateway which provisioned them. You can use `kubectl edit` on the Gateway and add the following to the `spec`: ```yaml infrastructure: annotations: annotationKey: annotationValue labels: labelKey: labelValue ``` After saving the changes, check the Service and NGINX deployment with `kubectl describe`. ```shell kubectl describe deployment cafe ``` ```text Name: cafe-nginx Namespace: default CreationTimestamp: Mon, 05 May 2025 16:49:33 -0700 ... Pod Template: Labels: app.kubernetes.io/instance=ngf app.kubernetes.io/managed-by=ngf-nginx app.kubernetes.io/name=cafe-nginx gateway.networking.k8s.io/gateway-name=cafe labelKey=labelValue Annotations: annotationKey: annotationValue prometheus.io/port: 9113 prometheus.io/scrape: true ... ``` ```shell kubectl describe service cafe-nginx ``` ```text Name: cafe-nginx Namespace: default Labels: app.kubernetes.io/instance=ngf app.kubernetes.io/managed-by=ngf-nginx app.kubernetes.io/name=cafe-nginx gateway.networking.k8s.io/gateway-name=cafe labelKey=labelValue Annotations: annotationKey: annotationValue ``` ## See also For more guides on routing traffic to applications and more information on Data Plane configuration, check out the following resources: - [Routing traffic to applications](/ngf/traffic-management/basic-routing.md) - [Application routes using HTTP matching conditions](/ngf/traffic-management/advanced-routing.md) - [Data plane configuration](/ngf/how-to/data-plane-configuration.md) - [API reference](/ngf/reference/api.md) -------------------------------------------------------------------------------- title: "Product telemetry" url: https://docs.nginx.com/nginx-gateway-fabric/overview/product-telemetry/index.md -------------------------------------------------------------------------------- Learn why, what and how NGINX Gateway Fabric collects telemetry. ## Overview NGINX Gateway Fabric collects telemetry by default, which allows its developers to understand what features are most popular with its user base. This data is used to triage development work, prioritizing features and functionality that will positively impact the most people. Telemetry data is collected once every 24 hours and sent to a service managed by F5 over HTTPS. Personally identifiable information (PII) is **not** collected. The list of data points that are collected can be seen below. **If you would prefer to not have data collected, you can [opt-out](#opt-out) when installing NGINX Gateway Fabric.** ## Collected data - **Kubernetes:** - **Platform:** the Kubernetes platform that NGINX Gateway Fabric is running on - **Version:** the Kubernetes version that NGINX Gateway Fabric is running on. - **Architecture:** the architecture that the Kubernetes environment is running on. - **Cluster UID:** the UID of the `kube-system` Namespace in the cluster that NGINX Gateway Fabric is running in. - **Cluster Node Count:** the number of Nodes in the cluster. - **Version:** the version of the NGINX Gateway Fabric Deployment. - **Deployment UID:** the UID of the NGINX Gateway Fabric Deployment. - **Image Build Source:** whether the image was built by GitHub or locally (values are `gha`, `local`, or `unknown`). The source repository of the images is **not** collected. - **Build OS:** the base operating system the image was built on (values are currently `alpine` or `ubi`). - **Deployment Flags:** a list of NGINX Gateway Fabric Deployment flags that are specified by a user. The actual values of non-boolean flags are **not** collected; we only record that they are either `true` or `false` for boolean flags and `default` or `user-defined` for the rest. - **Count of Resources:** the total count of resources related to NGINX Gateway Fabric. This includes supported Gateway API CRDs (such as `Gateways` and `HTTPRoutes`) and NGINX Gateway Fabric specific CRDs (such as `ClientSettingsPolicies` and `WAFPolicies`). The data within these resources is **not** collected. - **Snippets Info:** a list of directive-context strings from applied `SnippetFilters` and `SnippetsPolicies`, and a total count per strings. The actual value of any NGINX directive is **not** collected. - **Control Plane Pod Count:** the count of NGINX Gateway Fabric Pods. - **Data Plane Pod Count:** the count of NGINX data plane Pods. - **NGINX One Console Connection Info:** indicates whether the connection to the NGINX One Console is enabled. This data is used to identify the following information: - The flavors of Kubernetes environments that are most popular among our users. - The number of unique NGINX Gateway Fabric installations. - The scale of NGINX Gateway Fabric Deployments. - The scale of Gateway API resources. - The used features of NGINX Gateway Fabric. - The cluster is connected to NGINX One Console. Our goal is to publicly discuss data trends to drive roadmap discussions in our [Community Meeting](https://github.com/nginx/nginx-gateway-fabric/discussions/1472). ## Opt out You can disable product telemetry when installing NGINX Gateway Fabric using an option dependent on your installation method: ### Helm Set the `nginxGateway.productTelemetry.enable=false` flag either in the `values.yaml` file or when installing: ```shell helm install ... --set nginxGateway.productTelemetry.enable=false ``` ### Manifests Add the `--product-telemetry-disable` flag to the `nginx-gateway` container in your Deployment manifest. -------------------------------------------------------------------------------- title: "Troubleshooting" url: https://docs.nginx.com/nginx-gateway-fabric/troubleshooting/index.md -------------------------------------------------------------------------------- This topic describes possible issues when using NGINX Gateway Fabric and general troubleshooting techniques. When possible, suggested workarounds are provided. ### General troubleshooting When investigating a problem or requesting help, there are important data points that can be collected to help understand what issues may exist. #### Resource status To check the status of a resource, use `kubectl describe`. This example checks the status of the `coffee` HTTPRoute, which has an error: ```shell kubectl describe httproutes.gateway.networking.k8s.io coffee -n nginx-gateway ``` ```text ... Status: Parents: Conditions: Last Transition Time: 2024-05-31T17:20:51Z Message: The route is accepted Observed Generation: 4 Reason: Accepted Status: True Type: Accepted Last Transition Time: 2024-05-31T17:20:51Z Message: spec.rules[0].backendRefs[0].name: Not found: "bad-backend" Observed Generation: 4 Reason: BackendNotFound Status: False Type: ResolvedRefs Controller Name: gateway.nginx.org/nginx-gateway-controller Parent Ref: Group: gateway.networking.k8s.io Kind: Gateway Name: gateway Namespace: default Section Name: http ``` If a resource has errors relating to its configuration or relationship to other resources, they can likely be read in the status. The `ObservedGeneration` in the status should match the `ObservedGeneration` of the resource. Otherwise, this could mean that the resource hasn't been processed yet or that the status failed to update. If no `Status` is written on the resource, further debug by checking if the referenced resources exist and belong to NGINX Gateway Fabric. #### Events Events created by NGINX Gateway Fabric or other Kubernetes components could indicate system or configuration issues. To see events: ```shell kubectl get events -n nginx-gateway ``` For example, a warning event when the NginxGateway configuration CRD is deleted: ```text kubectl -n nginx-gateway get event LAST SEEN TYPE REASON OBJECT MESSAGE 5s Warning ResourceDeleted nginxgateway/ngf-config NginxGateway configuration was deleted; using defaults ``` #### Get shell access to NGINX container Getting shell access to containers allows developers and operators to view the environment of a running container, see its logs or diagnose any problems. To get shell access to the NGINX container, use `kubectl exec`: ```shell kubectl exec -it -n -- /bin/sh ``` #### Logs Logs from the NGINX Gateway Fabric control plane and data plane can contain information that isn't available to status or events. These can include errors in processing or passing traffic. **Note:** You can see logs for a crashed or killed container by adding the `-p` flag to the `kubectl logs` commands below. 1. Container Logs To see logs for the control plane container: ```shell kubectl -n nginx-gateway logs -c nginx-gateway ``` To see logs for the data plane container: ```shell kubectl -n logs -c nginx ``` 1. Error Logs For the _nginx-gateway_ container, you can `grep` the logs for the word `error`: ```shell kubectl -n nginx-gateway logs -c nginx-gateway | grep error ``` For the _nginx_ container you can `grep` for various [error](https://nginx.org/en/docs/ngx_core_module.html#error_log) logs. For example, to search for all logs logged at the `emerg` level: ```shell kubectl -n logs -c nginx | grep emerg ``` For example, if a variable is too long, NGINX may display such an error message: ```text kubectl logs -n dev-env gateway-nginx-bb8598998-jwk2m -c nginx | grep emerg 2024/06/13 20:04:17 [emerg] 27#27: too long parameter, probably missing terminating """ character in /etc/nginx/conf.d/http.conf:78 ``` 1. Access Logs NGINX access logs record all requests processed by the NGINX server. These logs provide detailed information about each request, which can be useful for troubleshooting and analyzing web traffic. Access logs can be viewed with the above method of using `kubectl logs`, or by viewing the access log file directly. To do that, get shell access to your NGINX container using these [steps](#get-shell-access-to-nginx-container). The access logs are located in the file `/var/log/nginx/access.log` in the NGINX container. 1. Modify Log Levels To modify log levels for the control plane in NGINX Gateway Fabric, edit the `NginxGateway` configuration. This can be done either before or after deploying NGINX Gateway Fabric. Refer to this [guide](https://docs.nginx.com/nginx-gateway-fabric/how-to/control-plane-configuration/) to do so. To check error logs, modify the log level to `error` to view error logs. Similarly, change the log level to `debug` and `grep` for the word `debug` to view debug logs. #### Understanding the generated NGINX configuration Understanding the NGINX configuration is key for fixing issues because it shows how NGINX handles requests. This helps tweak settings to make sure NGINX behaves the way you want it to for your application. To see your current configuration, you can open a shell in the _nginx_ container by following these [steps](#get-shell-access-to-nginx-container) and run `nginx -T`. To understand the usage of NGINX directives in the configuration file, consult this list of [NGINX directives](https://nginx.org/en/docs/dirindex.html). In this section, we will see how the configuration gets updated as we configure different Services, Deployments and HTTPRoutes with NGINX Gateway Fabric. In the configuration file, you'll often find several server blocks, each assigned to specific ports and server names. NGINX selects the appropriate server for a request and evaluates the URI against the location directives within that block. When only a Gateway resource is defined, but no Services or HTTPRoutes are configured, NGINX generates a basic configuration. This includes a default server listening on the ports specified in the Gateway listeners, handling all incoming requests. Additionally, there are blocks to manage errors with status codes 500 or 503. This is a default `server` block listening on port 80: ```text server { listen 80 default_server; default_type text/html; return 404; } ``` Once an HTTPRoute with path matches and rules are defined, nginx.conf is updated accordingly to determine which location block will manage incoming requests. To demonstrate how `nginx.conf` is changed, create some resources: 1. A Gateway with single listener with the hostname `*.example.com` on port 80. 2. A simple `coffee` application. 3. An HTTPRoute that exposes the `coffee` application outside the cluster using the listener created in step 1. The path and rule matches create different location blocks in `nginx.conf` to route requests as needed. For example, this `coffee` route matches requests with path `/coffee` and type `prefix`. Examine how the `nginx.conf` is modified: ```yaml apiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: name: coffee spec: parentRefs: - name: gateway sectionName: http hostnames: - "cafe.example.com" rules: - matches: - path: type: PathPrefix value: /coffee backendRefs: - name: coffee port: 80 ``` The modified `nginx.conf`: ```shell server { listen 80 default_server; default_type text/html; return 404; } server { listen 80; server_name cafe.example.com; location /coffee/ { proxy_set_header Host "$gw_api_compliant_host"; proxy_set_header X-Forwarded-For "$proxy_add_x_forwarded_for"; proxy_set_header Upgrade "$http_upgrade"; proxy_set_header Connection "$connection_upgrade"; proxy_http_version 1.1; proxy_pass http://default_coffee_80$request_uri; } location = /coffee { proxy_set_header Host "$gw_api_compliant_host"; proxy_set_header X-Forwarded-For "$proxy_add_x_forwarded_for"; proxy_set_header Upgrade "$http_upgrade"; proxy_set_header Connection "$connection_upgrade"; proxy_http_version 1.1; proxy_pass http://default_coffee_80$request_uri; } location / { return 404 ""; } } upstream default_coffee_80 { random two least_conn; zone default_coffee_80 512k; server 10.244.0.13:8080; } ``` Key information to note is: 1. A new `server` block is created with the hostname of the HTTPRoute. When a request is sent to this hostname, it will be handled by this `server` block. 1. Within the `server` block, three new `location` blocks are added for _coffee_, each with distinct prefix and exact paths. Requests directed to the _coffee_ application with a path prefix `/coffee/hello` will be managed by the first location block, while those with an exact path `/coffee` will be handled by the second location block. Any other requests not recognized by the server block for this hostname will default to the third location block, returning a 404 Not Found status. 1. Each `location` block has headers and directives that configure the NGINX proxy to forward requests to the `/coffee` path correctly, preserving important client information and ensuring compatibility with the upstream server. 1. The `upstream` block in the given NGINX configuration defines a group of backend servers and configures how NGINX should load balance requests among them. Review the behavior when a curl request is sent to the `coffee` application: Matches location /coffee/ block ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee/hello Handling connection for 8080 Server address: 10.244.0.13:8080 Server name: coffee-56b44d4c55-hwpkp Date: 13/Jun/2024:22:51:52 +0000 URI: /coffee/hello Request ID: 21fc2baad77337065e7cf2cd57e04383 ``` Matches location = /coffee block ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee Handling connection for 8080 Server address: 10.244.0.13:8080 Server name: coffee-56b44d4c55-hwpkp Date: 13/Jun/2024:22:51:40 +0000 URI: /coffee Request ID: 4d8d719e95063303e290ad74ecd7339f ``` Matches location / block ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/ Handling connection for 8080 404 Not Found

404 Not Found

nginx/1.25.4
``` **Note:** The configuration may change in future releases. This configuration is valid for version 1.3. #### Metrics for troubleshooting Metrics can be useful to identify performance bottlenecks and pinpoint areas of high resource consumption within NGINX Gateway Fabric. To set up metrics collection, refer to the [Prometheus Metrics guide](/ngf/monitoring/prometheus.md). The metrics dashboard will help you understand problems with the way NGINX Gateway Fabric is set up or potential issues that could show up with time. #### Access the NGINX Plus Dashboard If you have NGINX Gateway Fabric installed with NGINX Plus, you can access the NGINX Plus dashboard at `http://localhost:8080/dashboard.html`. Verify that the port number (for example, `8080`) matches the port number you have port-forwarded to your NGINX Gateway Fabric Pod. For further details, see the [dashboard guide](monitoring/dashboard.md) ### Common errors | Problem Area | Symptom | Troubleshooting Method | Common Cause | |------------------------------|----------------------------------------|---------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------| | Startup | NGINX Gateway Fabric fails to start. | Check logs for _nginx_ and _nginx-gateway_ containers. | Readiness probe failed. | | Resources not configured | Status missing on resources. | Check referenced resources. | Referenced resources do not belong to NGINX Gateway Fabric. | | NGINX errors | Reload failures on NGINX | Fix permissions for control plane. | Security context not configured. | | NGINX Plus errors | Failure to start; traffic interruptions | Set up the [NGINX Plus JWT](/ngf/install/nginx-plus.md) | License is not configured or has expired. | | Client Settings | Request entity too large error | Adjust client settings. Refer to [Client Settings Policy](/ngf/traffic-management/client-settings.md) | Payload is greater than the [`client_max_body_size`](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size) value.| #### NGINX fails to reload NGINX reload errors can occur for various reasons, including syntax errors in configuration files, permission issues, and more. To determine if NGINX has failed to reload, check logs for your _nginx-gateway_ and _nginx_ containers. You will see the following error in the _nginx-gateway_ logs: `failed to reload NGINX:`, followed by the reason for the failure. Similarly, error logs in _nginx_ container start with `emerg`. For example, `2024/06/12 14:25:11 [emerg] 12345#0: open() "/var/run/nginx.pid" failed (13: Permission denied)` shows a critical error, such as a permission problem preventing NGINX from accessing necessary files. To debug why your reload has failed, start with verifying the syntax of your configuration files by opening a shell in the NGINX container following these [steps](#get-shell-access-to-nginx-container) and running `nginx -T`. If there are errors in your configuration file, the reload will fail and specify the reason for it. #### NGINX Gateway Fabric Pod is not running or ready To understand why the NGINX Gateway Fabric Pod has not started running or is not ready, check the state of the Pod to get detailed information about the current status and events happening in the Pod. To do this, use `kubectl describe`: ```shell kubectl describe pod -n nginx-gateway ``` The Pod description includes details about the image name, tags, current status, and environment variables. Verify that these details match your setup and cross-check with the events to ensure everything is functioning as expected. For example, the Pod below has the nginx-gateway container that is running and the events reflect the same. ```text Containers: nginx-gateway: Container ID: containerd://492f380d5919ae2cdca0e009e7a7d5bf4092f8e1910f52d8951d58b73f125646 Image: nginx-gateway-fabric:latest Image ID: sha256:c034f1e5bde0490b1f2441e0e9b0bcfce5f2e259bb6210c55d4d67f808a74ecb Ports: 8443/TCP, 9113/TCP, 8081/TCP Host Ports: 0/TCP, 0/TCP, 0/TCP SeccompProfile: RuntimeDefault Args: controller --gateway-ctlr-name=gateway.nginx.org/nginx-gateway-controller --gatewayclass=nginx --config=my-release-config --service=my-release-nginx-gateway-fabric --agent-tls-secret=agent-tls --metrics-port=9113 --health-port=8081 --leader-election-lock-name=my-release-nginx-gateway-fabric-leader-election State: Running Started: Thu, 24 Apr 2025 10:57:16 -0700 Ready: True Restart Count: 0 Readiness: http-get http://:health/readyz delay=3s timeout=1s period=1s #success=1 #failure=3 Environment: POD_NAMESPACE: nginx-gateway (v1:metadata.namespace) POD_NAME: my-release-nginx-gateway-fabric-b99bd5cdd-qzp5q (v1:metadata.name) POD_UID: (v1:metadata.uid) INSTANCE_NAME: (v1:metadata.labels['app.kubernetes.io/instance']) IMAGE_NAME: nginx-gateway-fabric:latest Mounts: /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-5dg45 (ro) /var/run/secrets/ngf from nginx-agent-tls (rw) Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 22s default-scheduler Successfully assigned nginx-gateway/my-release-nginx-gateway-fabric-b99bd5cdd-qzp5q to kind-control-plane Normal Pulled 20s kubelet Container image "nginx-gateway-fabric:latest" already present on machine Normal Created 20s kubelet Created container: nginx-gateway Normal Started 20s kubelet Started container nginx-gateway ``` #### NGINX Pod is not running or ready To understand why the NGINX Pod has not started running or is not ready, check the state of the Pod to get detailed information about the current status and events happening in the Pod. To do this, use `kubectl describe`: ```shell kubectl describe pod -n ``` The Pod description includes details about the image name, tags, current status, and environment variables. Verify that these details match your setup and cross-check with the events to ensure everything is functioning as expected. For example, the Pod below has the nginx container that is running and the events reflect the same. ```text Containers: nginx: Container ID: containerd://0dd33fd358ba3b369de315be15b197e369342aba7aa8d3ea12e4455823fa90ce Image: nginx-gateway-fabric/nginx:latest Image ID: sha256:e5cb19bab49cbde6222df607a0946e1e00c1af767263b79ae36e4c69f8547f20 Ports: 80/TCP, 9113/TCP Host Ports: 0/TCP, 0/TCP SeccompProfile: RuntimeDefault State: Running Started: Thu, 24 Apr 2025 10:57:36 -0700 Ready: True Restart Count: 0 Environment: Mounts: /etc/nginx-agent from nginx-agent (rw) /etc/nginx/conf.d from nginx-conf (rw) /etc/nginx/includes from nginx-includes (rw) /etc/nginx/main-includes from nginx-main-includes (rw) /etc/nginx/secrets from nginx-secrets (rw) /etc/nginx/stream-conf.d from nginx-stream-conf (rw) /var/cache/nginx from nginx-cache (rw) /var/lib/nginx-agent from nginx-agent-lib (rw) /var/log/nginx-agent from nginx-agent-log (rw) /var/run/nginx from nginx-run (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-f9kph (ro) /var/run/secrets/ngf from nginx-agent-tls (rw) /var/run/secrets/ngf/serviceaccount from token (rw) Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 2m57s default-scheduler Successfully assigned default/gateway-nginx-85f7f6d7d-fx7q2 to kind-control-plane Normal Pulled 2m54s kubelet Container image "nginx-gateway-fabric:latest" already present on machine Normal Created 2m54s kubelet Created container: init Normal Started 2m54s kubelet Started container init Normal Pulled 2m53s kubelet Container image "nginx-gateway-fabric/nginx:latest" already present on machine Normal Created 2m53s kubelet Created container: nginx Normal Started 2m53s kubelet Started container nginx ``` #### NGINX Plus failure to start or traffic interruptions Beginning with NGINX Gateway Fabric 1.5.0, NGINX Plus requires a valid JSON Web Token (JWT) to run. If this is not set up properly, or your JWT token has expired, you may see errors in the NGINX logs that look like the following: ```text nginx: [error] invalid license token ``` ```text nginx: [emerg] License file is required. Download JWT license from MyF5 and configure its location... ``` ```text nginx: [emerg] license expired ``` These errors could prevent NGINX Plus from starting or prevent traffic from flowing. To fix these issues, see the [NGINX Plus JWT](/ngf/install/nginx-plus.md) guide. #### 413 Request Entity Too Large If you receive the following error: ```text 413 Request Entity Too Large

413 Request Entity Too Large

nginx/1.25.5
``` Or view the following error message in the NGINX logs: ```text 2024/05/30 21:48:22 [error] 138#138: *43 client intended to send too large body: 112 bytes, client: 127.0.0.1, server: cafe.example.com, request: "POST /coffee HTTP/1.1", host: "cafe.example.com:8080" ``` The request body exceeds the [client_max_body_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size). To **resolve** this, you can configure the `client_max_body_size` using the `ClientSettingsPolicy` API. Read the [Client Settings Policy](/ngf/traffic-management/client-settings.md) documentation for more information. #### IP Family Mismatch Errors If you `describe` your HTTPRoute and see the following error: ```text Conditions: Last Transition Time: 2024-07-14T23:36:37Z Message: The route is accepted Observed Generation: 1 Reason: Accepted Status: True Type: Accepted Last Transition Time: 2024-07-14T23:36:37Z Message: Service configured with IPv4 family but NginxProxy is configured with IPv6 Observed Generation: 1 Reason: InvalidServiceIPFamily Status: False Type: ResolvedRefs Controller Name: gateway.nginx.org/nginx-gateway-controller ``` The Service associated with your HTTPRoute is configured with a IP Family different than the one specified in the NginxProxy configuration. To **resolve** this, you can do one of the following: - Update the NginxProxy configuration with the proper [`ipFamily`](/ngf/reference/api.md) field. You can edit the NginxProxy configuration using `kubectl edit`. For example: ```shell kubectl edit -n nginx-gateway nginxproxies.gateway.nginx.org ngf-proxy-config ``` - When installing NGINX Gateway Fabric, change the IPFamily by modifying the field `nginx.config.ipFamily` in the `values.yaml` or add the `--set nginx.config.ipFamily=` flag to the `helm install` command. The supported IPFamilies are `ipv4`, `ipv6` and `dual` (default). - Adjust the IPFamily of your Service to match that of the NginxProxy configuration. #### Policy cannot be applied to target If you `describe` your Policy and see the following error: ```text Conditions: Last Transition Time: 2024-08-20T14:48:53Z Message: Policy cannot be applied to target "default/route1" since another Route "default/route2" shares a hostname:port/path combination with this target Observed Generation: 3 Reason: TargetConflict Status: False Type: Accepted ``` This means you are attempting to attach a Policy to a Route that has an overlapping hostname:port/path combination with another Route. To work around this, you can do one of the following: - Combine the Route rules for the overlapping path into a single Route. - If the Policy allows it, specify both Routes in the `targetRefs` list. #### Broken Header error If you check your _nginx_ container logs and see the following error: ```text 2024/07/25 00:50:45 [error] 211#211: *22 broken header: "GET /coffee HTTP/1.1" while reading PROXY protocol, client: 127.0.0.1, server: 0.0.0.0:80 ``` It indicates that `proxy_protocol` is enabled for the gateway listeners, but the request sent to the application endpoint does not contain proxy information. To **resolve** this, you can do one of the following: - Unassign the field [`rewriteClientIP.mode`](/ngf/reference/api.md) in the NginxProxy configuration. - Send valid proxy information with requests being handled by your application. #### Custom Resource Definition is invalid due to being too large If you apply the NGINX Gateway Fabric CRDs and see the following error: ```text The CustomResourceDefinition "nginxproxies.gateway.nginx.org" is invalid: metadata.annotations: Too long: may not be more than 262144 bytes ``` This is a known issue with Kubernetes, more specifically the client-side-apply annotation limit with kubectl, and our recommended workaround is to use kubectl's server-side apply when installing the NGINX Gateway Fabric CRDs. To do this, you can use the `--server-side` flag when applying the CRDs: ```shell kubectl apply --server-side -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v/deploy/crds.yaml ``` ### See also You can view the [Kubernetes Troubleshooting Guide](https://kubernetes.io/docs/tasks/debug/debug-application/) for more debugging guidance. -------------------------------------------------------------------------------- title: "Upgrade applications without downtime" url: https://docs.nginx.com/nginx-gateway-fabric/how-to/upgrade-apps-without-downtime/index.md -------------------------------------------------------------------------------- Learn how to use NGINX Gateway Fabric to upgrade applications without downtime. --- ## Overview **Note:** See the [Architecture document](/ngf/overview/gateway-architecture.md) to learn more about NGINX Gateway Fabric architecture. NGINX Gateway Fabric allows upgrading applications without downtime. To understand the upgrade methods, you need to be familiar with the NGINX features that help prevent application downtime: Graceful configuration reloads and upstream server updates. --- ### Graceful configuration reloads If a relevant gateway API or built-in Kubernetes resource is changed, NGINX Gateway Fabric will update NGINX by regenerating the NGINX configuration. NGINX Gateway Fabric then sends a reload signal to the master NGINX process to apply the new configuration. We call such an operation a "reload", during which client requests are not dropped - which defines it as a graceful reload. This process is further explained in the [NGINX configuration documentation](https://nginx.org/en/docs/control.html?#reconfiguration). --- ### Upstream server updates Endpoints frequently change during application upgrades: Kubernetes creates pods for the new version of an application and removes the old ones, creating and removing the respective endpoints as well. NGINX Gateway Fabric detects changes to endpoints by watching their corresponding [EndpointSlices](https://kubernetes.io/docs/concepts/services-networking/endpoint-slices/). In an NGINX configuration, a service is represented as an [upstream](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#upstream), and an endpoint as an [upstream server](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#server). Adding and removing endpoints are two of the most common cases: - If an endpoint is added, NGINX Gateway Fabric adds an upstream server to NGINX that corresponds to the endpoint, then reloads NGINX. Next, NGINX will start proxying traffic to that endpoint. - If an endpoint is removed, NGINX Gateway Fabric removes the corresponding upstream server from NGINX. After a reload, NGINX will stop proxying traffic to that server. However, it will finish proxying any pending requests to that server before switching to another endpoint. As long as you have more than one endpoint ready, clients won't experience downtime during upgrades. **Note:** It is good practice to configure a [Readiness probe](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/) in the deployment so that a pod can report when it is ready to receive traffic. Note that NGINX Gateway Fabric will not add any endpoint to NGINX that is not ready. --- ## Prerequisites - You have deployed your application as a [deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) - The pods of the deployment belong to a [service](https://kubernetes.io/docs/concepts/services-networking/service/) so that Kubernetes creates an [endpoint](https://kubernetes.io/docs/reference/kubernetes-api/service-resources/endpoints-v1/) for each pod. - You have exposed the application to the clients via an [HTTPRoute](https://gateway-api.sigs.k8s.io/api-types/httproute/) resource that references that service. For example, an application can be exposed using a routing rule like below: ```yaml - matches: - path: type: PathPrefix value: / backendRefs: - name: my-app port: 80 ``` **Note:** See the [Cafe example](https://github.com/nginx/nginx-gateway-fabric/tree/v/examples/cafe-example) for a basic example. The upgrade methods in the next sections cover: - Rolling deployment upgrades - Blue-green deployments - Canary releases --- ## Rolling deployment upgrade To start a [rolling deployment upgrade](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#rolling-update-deployment), you update the deployment to use the new version tag of the application. As a result, Kubernetes terminates the pods with the old version and create new ones. By default, Kubernetes also ensures that some number of pods always stay available during the upgrade. This upgrade will add new upstream servers to NGINX and remove the old ones. As long as the number of pods (ready endpoints) during an upgrade does not reach zero, NGINX will be able to proxy traffic, and therefore prevent any downtime. This method does not require you to update the **HTTPRoute**. --- ## Blue-green deployments With this method, you deploy a new version of the application (blue version) as a separate deployment, while the old version (green) keeps running and handling client traffic. Next, you switch the traffic from the green version to the blue. If the blue works as expected, you terminate the green. Otherwise, you switch the traffic back to the green. There are two ways to switch the traffic: - Update the service selector to select the pods of the blue version instead of the green. As a result, NGINX Gateway Fabric removes the green upstream servers from NGINX and adds the blue ones. With this approach, it is not necessary to update the **HTTPRoute**. - Create a separate service for the blue version and update the backend reference in the **HTTPRoute** to reference this service, which leads to the same result as with the previous option. --- ## Canary releases Canary releases involve gradually introducing a new version of your application to a subset of nodes in a controlled manner, splitting the traffic between the old are new (canary) release. This allows for monitoring and testing the new release's performance and reliability before full deployment, helping to identify and address issues without impacting the entire user base. To support canary releases, you can implement an approach with two deployments behind the same service (see [Canary deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#canary-deployment) in the Kubernetes documentation). However, this approach lacks precision for defining the traffic split between the old and the canary version. You can greatly influence it by controlling the number of pods (for example, four pods of the old version and one pod of the canary). However, note that NGINX Gateway Fabric uses [`random two least_conn`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#random) load balancing method, which doesn't guarantee an exact split based on the number of pods (80/20 in the given example). A more flexible and precise way to implement canary releases is to configure a traffic split in an **HTTPRoute**. In this case, you create a separate deployment for the new version with a separate service. For example, for the rule below, NGINX will proxy 95% of the traffic to the old version endpoints and only 5% to the new ones. ```yaml - matches: - path: type: PathPrefix value: / backendRefs: - name: my-app-old port: 80 weight: 95 - name: my-app-new port: 80 weight: 5 ``` **Note:** Every request coming from the same client won't necessarily be sent to the same backend. NGINX will independently split each request among the backend references. By updating the rule you can further increase the share of traffic the new version gets and finally completely switch to the new version: ```yaml - matches: - path: type: PathPrefix value: / backendRefs: - name: my-app-old port: 80 weight: 0 - name: my-app-new port: 80 weight: 1 ``` See the [Traffic splitting example](https://github.com/nginx/nginx-gateway-fabric/tree/v/examples/traffic-splitting) from our repository. -------------------------------------------------------------------------------- title: "Configure JSON Web Token (JWT) claim validation" url: https://docs.nginx.com/nginx-gateway-fabric/traffic-security/jwt-claim-validation/index.md description: How to configure JSON Web Token (JWT) claim validation in F5 NGINX Gateway Fabric using the `AuthenticationFilter` custom resource definition (CRD). -------------------------------------------------------------------------------- This guide describes how to configure JWT claim validation in F5 NGINX Gateway Fabric using the AuthenticationFilter custom resource definition (CRD). JWT claim validation adds authorization after [JWT authentication](/ngf/traffic-security/jwt-authentication.md) and [OIDC authentication](/ngf/traffic-security/oidc-authentication.md). Authentication checks whether a token is valid and signed correctly. Claim validation checks claims in the token payload. You can require a specific issuer, audience, or custom claim before users access your application. By following these instructions, you will configure an AuthenticationFilter with claim validation rules and verify that only tokens containing the expected claims are allowed through. For demonstration purposes, in this document you will deploy and configure a `type: JWT` AuthenticationFilter using `mode: File`. **Note:** JWT claim validation requires NGINX Plus. ## Overview JWT claim validation is configured through the `authorization` field on the AuthenticationFilter spec. It uses a two-level `require` model that controls how rules and claims are evaluated. **Top-level require** The top-level `require` field (`authorization.require`) controls how **rules** relate to each other: - **Any** (default) — A request is authorized if **any one** of the rules are satisfied. - **All** — A request is authorized only if **every** rule is satisfied. **Per-rule require** Each rule has its own `require` field (`rules[].require`) that controls how the **claims within that rule** relate to each other: - **Any** (default) — The rule is satisfied if **any one** of its claims matches. - **All** — The rule is satisfied only if **every** claim in the rule matches. **How the two levels work together** Consider an AuthenticationFilter with `authorization.require: Any` and two rules, each with `require: All`: - Rule 0 requires **all** of: `iss=issuer-1` **and** `aud=api` - Rule 1 requires **all** of: `iss=issuer-2` **and** `aud=admin` Since the top-level require is `Any`, a request is authorized if the token satisfies **either** rule 0 **or** rule 1. A token that only partially matches both rules would be rejected as it does not fully satisfy either rule. ## Before you begin - [Install](/ngf/install/) NGINX Gateway Fabric with NGINX Plus. ## Set up In this part of the document, we will set up several resources in your cluster to demonstrate the authorization field of the AuthenticationFilter CRD. ### Deploy sample applications To deploy the `coffee` and `tea` applications, run the following YAML with `kubectl apply`: ```yaml kubectl apply -f - < GW_PORT= ``` ### Generate a JWKS and create a Secret For testing purposes, the following example shows a simple JWKS with a single RSA key. In production, use properly generated keys from your identity provider or key management system. Save the JWKS to a file called `auth` and create a Secret: ```shell cat < auth { "keys": [ { "kty": "RSA", "kid": "test-key", "use": "sig", "n": "0vx7agoebGcQSuuPiLJXZptN9nndrQmbXEps2aiAFbWhM78LhWx4cbbfAAtVT86zwu1RK7aPFFxuhDR1L6tSoc_BJECPebWKRXjBZCiFV4n3oknjhMstn64tZ_2W-5JsGY4Hc5n9yBXArwl93lqt7_RN5w6Cf0h4QyQ5v-65YGjQR0_FDW2QvzqY368QQMicAtaSqzs8KJZgnYb9c7d0zgdAZHzu6qMQvRL5hajrn1n91CbOpbISD08qNLyrdkt-bFTWhAI4vMQFh6WeZu0fM4lFd2NcRwr3XPksINHaQ-G_xBniIqbw0Ls1jF44-csFCur-kEgU8awapJzKnqDKgw", "e": "AQAB" } ] } EOF kubectl create secret generic jwks-secret --from-file=auth ``` **Note:** This example JWKS is for demonstration only. In production, use keys from your identity provider or key management system. --- ## Configure JWT claim validation This example creates an AuthenticationFilter with two rules that enforce different issuer and audience combinations. The top-level `require` is set to `Any`, and each rule's `require` is set to `All`. This means a request is authorized if its JWT satisfies **all** claims in rule 0 **or** all claims in rule 1. ### Create the AuthenticationFilter Deploy the AuthenticationFilter with claim validation rules by running the following YAML with `kubectl apply`: ```yaml kubectl apply -f - < ``` ### Understanding the configuration - **`authorization.require: Any`** — The request passes if **any** rule is satisfied. - **Rule 0** (`require: All`) — The JWT must contain **both** `iss` equal to `https://issuer-1.example.com` **and** `aud` equal to `api`. - **Rule 1** (`require: All`) — The JWT must contain **both** `iss` equal to `https://issuer-2.example.com` **and** `aud` equal to `admin`. The following table summarizes which tokens are authorized: | Token claims | Rule 0 | Rule 1 | Result | |---|---|---|---| | `iss=issuer-1`, `aud=api` | ✅ All matched | ❌ | ✅ Authorized | | `iss=issuer-2`, `aud=admin` | ❌ | ✅ All matched | ✅ Authorized | | `iss=issuer-1`, `aud=admin` | ❌ Partial | ❌ Partial | ❌ Rejected | | `iss=issuer-2`, `aud=api` | ❌ Partial | ❌ Partial | ❌ Rejected | ### Deploy an HTTPRoute referencing the AuthenticationFilter Deploy an HTTPRoute that applies the AuthenticationFilter to the `/coffee` path. The `/tea` path has no authentication and responds normally. Run the following YAML with `kubectl apply`: ```yaml kubectl apply -f - < ``` ### Verify JWT claim validation **Note:** Your clients should be able to resolve "cafe.example.com" to the public IP of the NGINX Service. This guide simulates that using curl's `--resolve` option. To test claim validation, you need JWTs signed with the private key corresponding to the public key in your JWKS. You can use [jwt.io](https://jwt.io) or other JWT tools to generate tokens with different claim payloads. Store each token in a shell variable. **Token matching rule 0** (`iss=https://issuer-1.example.com`, `aud=api`): ```shell JWT_RULE0="" ``` **Token matching rule 1** (`iss=https://issuer-2.example.com`, `aud=admin`): ```shell JWT_RULE1="" ``` **Token matching neither rule** (`iss=https://issuer-1.example.com`, `aud=admin`): ```shell JWT_NEITHER="" ``` Access `/coffee` with a token matching rule 0 ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee -H "Authorization: Bearer $JWT_RULE0" ``` ```text Server address: 192.0.2.7:8080 Server name: coffee-654ddf664b-nhhvr Date: 10/Mar/2026:15:20:15 +0000 URI: /coffee Request ID: 13a925b2514b62c45ea4a79800248d5c ``` The request succeeds as the token satisfies all claims in rule 0. Access `/coffee` with a token matching rule 1 ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee -H "Authorization: Bearer $JWT_RULE1" ``` ```text Server address: 192.0.2.7:8080 Server name: coffee-654ddf664b-nhhvr Date: 02/Sep/2026:15:21:30 +0000 URI: /coffee Request ID: 7b2e4a1c9f0d3e5a8c6b4d2f1a0e9c8b ``` The request succeeds as the token satisfies all claims in rule 1. Access `/coffee` with a token matching neither rule ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee -H "Authorization: Bearer $JWT_NEITHER" ``` ```text 401 Authorization Required

401 Authorization Required

nginx
``` The request is rejected. Although the token has valid claims, it only partially matches each rule (`iss` from rule 0 and `aud` from rule 1). Since each rule requires **all** claims to match and the token does not fully satisfy either rule, authorization fails. Access `/coffee` without a token ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee ``` ```text 401 Authorization Required

401 Authorization Required

nginx
``` Without a JWT, the request fails authentication before claim validation is evaluated. Access `/tea` without authentication ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/tea ``` ```text Server address: 10.244.0.10:8080 Server name: tea-75bc9f4b6d-ms2n8 Date: 03/Sep/2026:15:36:26 +0000 URI: /tea Request ID: c7eb0509303de1c160cb7e7d2ac1d99f ``` The `/tea` path has no AuthenticationFilter attached and responds normally. ## Additional configuration ### Nested claims JWT claims can be nested at multiple levels. Use the slash (`/`) separator to identify the required claim value. In this example, we set up the rule to access the values of the `roles` claim, which is nested under `realm_access` Example JSON payload ```json { "realm_access": { "roles": ["reader", "admin"] } } ``` ```yaml authorization: require: Any rules: - require: All claims: - name: "realm_access/roles" values: - "reader" - "admin" ``` ### Claim match type By default, claim values are evaluated against their exact value. This can also be set to `Regex` allowing for a more complex and expressive matching configuration. This example allows an `email` claim that contains `@example.com`. This would match on `foo@example.com`, `user@example.com` etc... ```yaml authorization: require: Any rules: - require: Any claims: - name: email match: Regex values: - ".*@example\\.com" ``` ### Header forwarding By defining the `proxySetHeader` for a specific claim, you can forward the value of a matched claim as a request header to the upstream application. This example shows defining a header called `X-Aud` on the `aud` claim. ```yaml authorization: require: Any rules: - require: All claims: - name: "aud" proxySetHeader: X-Aud values: - "api" ``` --- ## Troubleshooting - Ensure NGINX Gateway Fabric is deployed with NGINX Plus. JWT claim validation is not supported in the open source version. - Ensure the AuthenticationFilter is accepted by checking its status with `kubectl describe`. - Ensure the HTTPRoute references the correct AuthenticationFilter name and group. - Confirm the Secret key is named `auth` and contains valid JWKS JSON. The Secret must be in the same namespace as the AuthenticationFilter. - Verify your JWT includes the `kid` (key ID) claim that matches one of the keys in your JWKS. - Check that the JWT is not expired by verifying the `exp` claim. - Ensure the JWT signature algorithm (typically RS256) matches the key type in your JWKS. - If claim validation rejects a token you expect to pass, decode the token at [jwt.io](https://jwt.io) and verify that the claim names and values exactly match what is configured in the AuthenticationFilter. ## Further reading - [AuthenticationFilter API reference](/ngf/reference/api.md#gateway.nginx.org/v1alpha1.AuthenticationFilter) - [Configure JWT authentication](/ngf/traffic-security/jwt-authentication.md) - [Configure OIDC authentication](/ngf/traffic-security/oidc-authentication.md) - [NGINX JWT Authentication Module](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html) - [NGINX auth_jwt_require directive](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_require) - [RFC 7519 - JSON Web Token (JWT)](https://datatracker.ietf.org/doc/html/rfc7519) -------------------------------------------------------------------------------- title: "Configure Cross-Origin Request Sharing (CORS)" url: https://docs.nginx.com/nginx-gateway-fabric/traffic-security/cors/index.md -------------------------------------------------------------------------------- This document describes how to configure the HTTPCORSFilter in F5 NGINX Gateway Fabric to handle Cross-Origin Resource Sharing (CORS) for your applications. CORS is a security feature that allows or denies web applications running at one domain to make requests for resources from a different domain. The HTTPCORSFilter in the gateway API provides a standard way to configure CORS policies. ## Before you begin - [Install](/ngf/install/) NGINX Gateway Fabric. ## Deploy sample application To deploy the `coffee` application, run the following YAML with `kubectl apply`: ```yaml kubectl apply -f - < ``` ## Deploy a HTTPRoute with the HTTPCORSFilter In this example, the filter is applied to the `/coffee` path. Run the following command to apply the route: ```yaml kubectl apply -f - < ``` ## Verify CORS pre-flight check **Note:** Your clients should be able to resolve the domain name "cafe.example.com" to the public IP of the NGINX Service. This guide simulates it using curl's `--resolve` option. Send a preflight request using curl: ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee -H "Origin: https://example.com" -X OPTIONS -v ``` Response: ```text > OPTIONS /coffee HTTP/1.1 > Host: cafe.example.com:8080 > User-Agent: curl/8.7.1 > Accept: */* > Origin: https://example.com > * Request completely sent off < HTTP/1.1 200 OK < Server: nginx < Date: Wed, 18 Feb 2026 11:49:23 GMT < Content-Type: application/octet-stream < Content-Length: 0 < Connection: keep-alive < Access-Control-Allow-Origin: https://example.com < Access-Control-Allow-Methods: GET, POST < Access-Control-Allow-Headers: Keep-Alive, Content-Type, User-Agent, Authorization < Access-Control-Expose-Headers: Content-Security-Policy < Access-Control-Allow-Credentials: true < Access-Control-Max-Age: 10 < * Connection #0 to host cafe.example.com left intact ``` ## Further reading - [Example deployment files for HTTPCORSFilter](https://github.com/nginx/nginx-gateway-fabric/tree/main/examples/cors-filter) - [Gateway API Specification](https://gateway-api.sigs.k8s.io/reference/spec/#httpcorsfilter) - [Gateway API CORS](https://gateway-api.sigs.k8s.io/guides/http-cors/) -------------------------------------------------------------------------------- title: "Configure Request Mirroring" url: https://docs.nginx.com/nginx-gateway-fabric/traffic-management/mirror/index.md -------------------------------------------------------------------------------- Learn how to mirror your HTTP or gRPC traffic using NGINX Gateway Fabric. ## Overview [HTTPRoute](https://gateway-api.sigs.k8s.io/api-types/httproute/) and [GRPCRoute](https://gateway-api.sigs.k8s.io/api-types/grpcroute/) filters can be used to configure request mirroring. Mirroring copies a request to another backend. In this guide, we will set up two applications, **coffee** and **tea**, and mirror requests between them. All requests sent to the **coffee** application will also be sent to the **tea** application automatically. ## Before you begin - [Install](/ngf/install/) NGINX Gateway Fabric. ## Set up Create the **coffee** and **tea** applications in Kubernetes by copying and pasting the following block into your terminal: ```yaml kubectl apply -f - < 80/TCP 3s service/tea ClusterIP 10.96.185.235 80/TCP 3s ``` --- ## Configure request mirroring First, create the **cafe** Gateway resource: ```yaml kubectl apply -f - < ``` **Note:** In a production environment, you should have a DNS record for the external IP address that is exposed, and it should refer to the hostname that the gateway will forward for. Now create an HTTPRoute that defines a RequestMirror filter that copies all requests sent to `/coffee` to be sent to the **coffee** backend and mirrored to the **tea** backend. Use the following command: ```yaml kubectl apply -f - <` when running `helm install` or `helm upgrade`. To upgrade the release with Helm, you can use the OCI registry, or download the chart and upgrade from the source. If needed, replace `ngf` with your chosen release name. #### Upgrade from the OCI registry To avoid downtime when upgrading from v2.0.x to v2.1, run the following commands. Be sure to include your previous installation flags and values if necessary. This will not affect user traffic, as the NGINX data plane deployment won't be removed as part of this process. ```shell helm uninstall ngf -n nginx-gateway helm install ngf oci://ghcr.io/nginx/charts/nginx-gateway-fabric -n nginx-gateway ``` Otherwise, for all other version upgrades: ```shell helm upgrade ngf oci://ghcr.io/nginx/charts/nginx-gateway-fabric -n nginx-gateway ``` #### Upgrade from sources ```shell helm pull oci://ghcr.io/nginx/charts/nginx-gateway-fabric --untar cd nginx-gateway-fabric ``` For the latest version from the **main** branch, add _--version 0.0.0-edge_ to your pull command. To avoid downtime when upgrading from v2.0.x to v2.1, run the following. Be sure to include your previous installation flags and values if necessary. This will not affect user traffic, as the NGINX data plane deployment won't be removed as part of this process. ```shell helm uninstall ngf -n nginx-gateway helm install ngf . -n nginx-gateway ``` Otherwise, for all other version upgrades: ```shell helm upgrade ngf . -n nginx-gateway ``` #### Manifests Select the deployment manifest that matches your current deployment from options available in the [Deploy NGINX Gateway Fabric](/ngf/install/manifests.md#deploy-nginx-gateway-fabric-1) section. To avoid downtime when upgrading from v2.0.x to v2.1, delete the previous NGINX Gateway Fabric control plane deployment in the `nginx-gateway` namespace, using `kubectl delete deployment`. Then `kubectl apply` the updated manifest file. This will not affect user traffic, as the NGINX data plane deployment won't be removed as part of this process. ## Upgrade from v1.x to v2.x This section provides step-by-step instructions for upgrading NGINX Gateway Fabric from version 1.x to 2.x, highlighting key architectural changes, expected downtime, and important considerations for CRDs. To upgrade NGINX Gateway Fabric from version 1.x to the new architecture in version 2.x, you must uninstall the existing NGINX Gateway Fabric CRDs and deployment, and perform a fresh installation. This will cause brief downtime during the upgrade process. **Note:** You do not need to uninstall the Gateway API CRDs during the upgrade. These resources are compatible with the new NGINX Gateway Fabric version. ### Uninstall NGINX Gateway Fabric v1.x To remove the previous version 1.x of NGINX Gateway Fabric, follow these steps: First, run the following command to uninstall NGINX Gateway Fabric from the `nginx-gateway` namespace, and update `ngf` to your release name if it is different: ```shell helm uninstall ngf -n nginx-gateway ``` Afterwards, remove CRDs associated with NGINX Gateway Fabric version 1.x with the following command: ```shell kubectl delete -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v1.6.2/deploy/crds.yaml ``` ### Install NGINX Gateway Fabric 2.x **Note:** Before installing 2.x, we recommend following [Add certificates for secure authentication](/ngf/install/secure-certificates.md). By default, NGINX Gateway Fabric installs self-signed certificates, which may be unsuitable for a production environment. #### Helm Use the following `helm install` command to install the latest stable NGINX Gateway Fabric release in the `nginx-gateway` namespace. It will also install the CRDs required for the deployment: ```shell helm install ngf oci://ghcr.io/nginx/charts/nginx-gateway-fabric --create-namespace -n nginx-gateway ``` For customization options during the Helm installation process, view the [Install NGINX Gateway Fabric with Helm](/ngf/install/helm.md) topic. #### Manifests Apply the new CRDs with the following command: ```shell kubectl apply --server-side -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v/deploy/crds.yaml ``` Next, install the latest stable release of NGINX Gateway Fabric in the `nginx-gateway` namespace with the following command: ```shell kubectl apply -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v/deploy/default/deploy.yaml ``` For customization options during the Manifest installation process, view the [Install NGINX Gateway Fabric with Manifests](/ngf/install/manifests.md) topic. ### Architecture changes With this release, NGINX Gateway Fabric adopts a new architecture that separates the control plane and data plane into independent deployments. This separation improves scalability, security, and operational clarity. The control plane is a Kubernetes controller that watches Gateway API and Kubernetes resources (e.g., Services, Endpoints, Secrets) and dynamically provisions NGINX data plane deployments for each Gateway. NGINX configurations are generated by the control plane and securely delivered to the data planes via gRPC, using the NGINX Agent. TLS is enabled by default, with optional integration with `cert-manager`. Each data plane pod runs NGINX alongside the Agent, which applies config updates and handles reloads without shared volumes or signals. This design ensures dynamic, per-Gateway traffic management and operational isolation. New fields have been added to the `NginxProxy` resource to configure infrastructure-related settings for data plane deployments. The `NginxProxy` resource is now a namespaced-scoped resource, instead of a cluster-scoped resource, and can be modified at either the Gateway or GatewayClass level. These new fields provide the flexibility to customize deployment and service configurations. For detailed instructions on how to modify these settings, refer to the [Configure infrastructure-related settings](/ngf/how-to/data-plane-configuration.md#configure-infrastructure-related-settings) guide. ### Key links for the version 2.x update - To read more on [modifying data plane configuration](/ngf/how-to/data-plane-configuration.md). - To learn more about [deploying a Gateway for data plane instances](/ngf/install/deploy-data-plane.md). - To add secure [authentication to control plane and data planes](/ngf/install/secure-certificates.md). - To read more about [architecture changes](/ngf/overview/gateway-architecture.md). - For detailed [API reference](/ngf/reference/api.md). ## Access NGINX Gateway Fabric 1.x documentation The documentation website is intended for the latest version of NGINX Gateway Fabric. To review documentation prior to 2.x, check out the desired release branch (such as _release-1.6_): ```shell git clone git@github.com:nginx/nginx-gateway-fabric.git git checkout release-1.6 ``` To review the documentation in a local webserver, run _make watch_ in the _/site_ folder: ```shell cd site make watch ``` ```text Hugo is available and has a version greater than 133. Proceeding with build. hugo --bind 0.0.0.0 -p 1313 server --disableFastRender Watching for changes in /home//nginx-gateway-fabric/site/{content,layouts,static} Watching for config changes in /home//nginx-gateway-fabric/site/config/_default, /home//nginx-gateway-fabric/site/config/development, /home//nginx-gateway-fabric/site/go.mod Start building sites … hugo v0.135.0-f30603c47f5205e30ef83c70419f57d7eb7175ab linux/amd64 BuildDate=2024-09-27T13:17:08Z VendorInfo=gohugoio | EN -------------------+------ Pages | 72 Paginator pages | 0 Non-page files | 0 Static files | 176 Processed images | 0 Aliases | 9 Cleaned | 0 Built in 213 ms Environment: "development" Serving pages from disk Web Server is available ``` You can then follow [this localhost link](http://localhost:1313/nginx-gateway-fabric/) for 1.x NGINX Gateway Fabric documentation. ## Upgrade from NGINX Open Source to NGINX Plus **Note:** Ensure that you [Set up the JWT](/ngf/install/nginx-plus.md#set-up-the-jwt) before upgrading. These instructions only apply to Helm. To upgrade from NGINX Open Source to NGINX Plus, update the Helm command to include the necessary values for Plus: **Note:** If applicable: - Replace the F5 Container registry `private-registry.nginx.com` with your internal registry for your NGINX Plus image - Replace `nginx-plus-registry-secret` with your Secret name containing the registry credentials - Replace `ngf` with your chosen release name. ```shell helm upgrade ngf oci://ghcr.io/nginx/charts/nginx-gateway-fabric --set nginx.image.repository=private-registry.nginx.com/nginx-gateway-fabric/nginx-plus --set nginx.plus=true --set nginx.imagePullSecret=nginx-plus-registry-secret -n nginx-gateway ``` -------------------------------------------------------------------------------- title: "Gateway API Inference Extension" url: https://docs.nginx.com/nginx-gateway-fabric/how-to/gateway-api-inference-extension/index.md -------------------------------------------------------------------------------- Learn how to use NGINX Gateway Fabric with the Gateway API Inference Extension to optimize traffic routing to self-hosting Generative AI Models on Kubernetes. ## Overview The [Gateway API Inference Extension](https://gateway-api-inference-extension.sigs.k8s.io/) is an official Kubernetes project that aims to provide optimized load-balancing for self-hosted Generative AI Models on Kubernetes. The project's goal is to improve and standardize routing to inference workloads across the ecosystem. Coupled with the provided Endpoint Picker Service, NGINX Gateway Fabric becomes an [Inference Gateway](https://gateway-api-inference-extension.sigs.k8s.io/#concepts-and-definitions), with additional AI specific traffic management features such as model-aware routing, serving priority for models, model rollouts, and more. ## Set up Install the Gateway API Inference Extension CRDs: ```shell kubectl kustomize "https://github.com/nginx/nginx-gateway-fabric/config/crd/inference-extension/?ref=v" | kubectl apply -f - ``` To enable the Gateway API Inference Extension, [install](/ngf/install/) NGINX Gateway Fabric with these modifications: - Using Helm: set the `nginxGateway.gwAPIInferenceExtension.enable=true` Helm value. - Using Kubernetes manifests: set the `--gateway-api-inference-extension` flag in the nginx-gateway container argument, update the ClusterRole RBAC to add the `inferencepools`: ```yaml - apiGroups: - inference.networking.k8s.io resources: - inferencepools verbs: - get - list - watch - apiGroups: - inference.networking.k8s.io resources: - inferencepools/status verbs: - update ``` See this [example manifest](https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/main/deploy/inference/deploy.yaml) for clarification. ## Deploy a sample model server The [vLLM simulator](https://github.com/llm-d/llm-d-inference-sim/tree/main) model server does not use GPUs and is ideal for test/development environments. To deploy the vLLM simulator, run the following command: ```shell kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api-inference-extension/refs/tags/v/config/manifests/vllm/sim-deployment.yaml ``` ## Deploy the InferencePool and Endpoint Picker Extension The InferencePool is a Gateway API Inference Extension resource that represents a set of Inference-focused Pods. With InferencePool, you can configure a routing extension as well as inference-specific routing optimizations. For more information on this resource, refer to the Gateway API Inference Extension [InferencePool documentation](https://gateway-api-inference-extension.sigs.k8s.io/api-types/inferencepool/). Install an InferencePool named `vllm-qwen3-32b` that selects from endpoints with label `app: vllm-qwen3-32b` and listening on port 8000. The Helm install command automatically installs the Endpoint Picker Extension and InferencePool. NGINX will query the Endpoint Picker Extension to determine the appropriate pod endpoint to route traffic to. These pods are selected from a pool of ready pods designated by the assigned InferencePool's Selector field. For more information on the [Endpoint Picker](https://github.com/llm-d/llm-d-router/blob/main/pkg/epp/README.md). **Note:** The Endpoint Picker Extension is a third-party application written and provided by the Gateway API Inference Extension project. Communication between NGINX and the Endpoint Picker uses TLS with certificate verification disabled by default. NGINX Gateway Fabric is not responsible for any threats or risks associated with using this third-party Endpoint Picker Extension application. ```shell export IGW_CHART_VERSION=v helm install vllm-qwen3-32b \ --dependency-update \ --set inferencePool.modelServers.matchLabels.app=vllm-qwen3-32b \ --version $IGW_CHART_VERSION \ --set inferenceExtension.resources.requests.memory=4Gi \ oci://registry.k8s.io/gateway-api-inference-extension/charts/inferencepool ``` **Note:** For test environments, lower the CPU and memory requests and limits to reduce resource use: ```shell export IGW_CHART_VERSION=v helm install vllm-qwen3-32b \ --dependency-update \ --set inferencePool.modelServers.matchLabels.app=vllm-qwen3-32b \ --version $IGW_CHART_VERSION \ --set inferenceExtension.resources.requests.cpu=100m \ --set inferenceExtension.resources.requests.memory=512Mi \ --set inferenceExtension.resources.limits.memory=2Gi \ oci://registry.k8s.io/gateway-api-inference-extension/charts/inferencepool ``` Confirm that the Endpoint Picker was deployed and is running: ```shell kubectl describe deployment vllm-qwen3-32b-epp ``` ## Deploy an Inference Gateway ```yaml kubectl apply -f - < ``` ## Deploy an HTTPRoute ```yaml kubectl apply -f - < > gateway-api-resources.yaml ``` This tool will analyze your Ingress resources from the input file and output the equivalent Gateway API resources to a file named `gateway-api-resources.yaml`. Review the generated Gateway API resources in the output file and apply them to your cluster: ```shell kubectl apply -f gateway-api-resources.yaml ``` For detailed information about NGINX-specific features and conversion options, see the [NGINX provider documentation](https://github.com/kubernetes-sigs/ingress2gateway/blob/main/pkg/i2gw/providers/nginx/README.md). -------------------------------------------------------------------------------- title: "Snippets" url: https://docs.nginx.com/nginx-gateway-fabric/traffic-management/snippets/index.md -------------------------------------------------------------------------------- **Note:** This guide previously showed how to use `SnippetsFilters` to configure Rate Limiting in NGINX. However, first-class support for Rate Limiting is now supported through the `RateLimitPolicy` API. This guide has been changed to provide an example of how to use `limit_conn` to limit the number of connections to a location and `limit_except` to limit the allowed HTTP methods. For a guide on how to configure Rate Limiting, see our guide on the [RateLimitPolicy API](). This topic introduces Snippets, how to implement them using the `SnippetsFilter` and `SnippetsPolicy` APIs, and provides an example of how to use them to limit the number of connections to a location using the [limit_conn](https://nginx.org/en/docs/http/ngx_http_limit_conn_module.html#limit_conn) NGINX directive and limit the allowed HTTP methods inside a location using the [limit_except](https://nginx.org/en/docs/http/ngx_http_core_module.html#limit_except) NGINX directive. ## Overview Snippets allow users to insert NGINX configuration into different contexts of the NGINX configurations that NGINX Gateway Fabric generates. Snippets should only be used by advanced NGINX users who need more control over the generated NGINX configuration, and only in cases where Gateway API resources or NGINX extension policies don't apply. Users can configure Snippets through either the `SnippetsFilter` or `SnippetsPolicy` APIs. `SnippetsFilter` can be an [HTTPRouteFilter](https://gateway-api.sigs.k8s.io/reference/spec/#gateway.networking.k8s.io/v1.HTTPRouteFilter) or [GRPCRouteFilter](https://gateway-api.sigs.k8s.io/reference/spec/#gateway.networking.k8s.io/v1.GRPCRouteFilter), that can be defined in an HTTPRoute/GRPCRoute rule and is intended to modify NGINX configuration specifically for that Route rule. `SnippetsFilter` is an `extensionRef` type filter, and must be in the same namespace as the Route it is referenced on. `SnippetsPolicy` is a [Direct PolicyAttachment](https://gateway-api.sigs.k8s.io/reference/policy-attachment/) that can only be applied to a Gateway(s) in the same namespace as the `SnippetsPolicy`. `SnippetsPolicy` affects the Gateway and all Routes attached to it, and is meant to contrast the route-focused `SnippetsFilter`. ## Disadvantages of Snippets **Note:** We recommend managing NGINX configuration through Gateway API resources, [first-class policies](/ngf/overview/custom-policies.md), and other existing [NGINX extensions](/ngf/how-to/data-plane-configuration.md) before using Snippets. Snippets are configured using the `SnippetsFilter` or `SnippetsPolicy` API, but are disabled by default due to their complexity and security implications. Snippets have the following disadvantages: - _Complexity_. Snippets require you to: - Understand NGINX configuration primitives to implement correct NGINX configuration. - Understand how NGINX Gateway Fabric generates NGINX configuration so that a Snippet doesn’t interfere with the other features in the configuration. - _Decreased robustness_. An incorrect Snippet can invalidate NGINX configuration, causing reload failures. Until the snippet is fixed, it will prevent any new configuration updates, including updates for the other Gateway resources. - _Security implications_. Snippets give access to NGINX configuration primitives, which are not validated by NGINX Gateway Fabric. For example, a Snippet can configure NGINX to serve the TLS certificates and keys used for TLS termination for Gateway resources. ## Best practices for Snippets When using both `SnippetsPolicies` and `SnippetsFilters`, be sure to double check that the Snippets applied do not conflict and cause invalid NGINX configuration. NGINX Gateway Fabric does not validate NGINX configuration in a Snippet. If the Snippet creates invalid NGINX configuration, NGINX Gateway Fabric will rollback to the latest valid configuration with a related log message outputted. ### SnippetsFilters There are endless ways to use `SnippetsFilters` to modify NGINX configuration, and equal ways to generate invalid or undesired NGINX configuration. We have outlined a few best practices to keep in mind when using `SnippetsFilters` to keep NGINX Gateway Fabric functioning correctly: 1. Using the [Roles and Personas](https://gateway-api.sigs.k8s.io/docs/concepts/roles-and-personas/) defined in the Gateway API, `SnippetsFilter` access should be limited to Cluster operators. Application developers should not be able to create, modify, or delete `SnippetsFilters` as they affect other applications. `SnippetsFilter` creates a natural split of responsibilities between the Cluster operator and the Application developer: the Cluster operator creates a `SnippetsFilter`; the Application developer references the `SnippetsFilter` in an HTTPRoute/GRPCRoute to enable it. 1. In a `SnippetsFilter`, only one Snippet per NGINX context is allowed, however multiple `SnippetsFilters` can be referenced in the same routing rule. As such, `SnippetsFilters` should not conflict with each other. If `SnippetsFilters` do conflict, they should not be referenced on the same routing rule. 1. `SnippetsFilters` that define Snippets targeting NGINX contexts `main`, `http`, or `http.server`, can potentially affect more than the routing rule they are referenced by. Proceed with caution and verify the behavior of the NGINX configuration before creating those `SnippetsFilters` in a production scenario. ### SnippetsPolicies `SnippetsPolicies` share many of the same best practices as `SnippetsFilters` but with a couple of adjustments: 1. `SnippetsPolicy` access should be limited to Cluster operators. Additionally, since `SnippetsPolicies` use the `targetRefs` field to specify which resources to target, this means there is no need for an Application developer to reference it on a Route like the `SnippetsFilter` does. ## Setup To enable the `SnippetsFilter` and `SnippetsPolicy` APIs, [install](/ngf/install/) NGINX Gateway Fabric with these modifications: - Using Helm: set the `nginxGateway.snippets.enable=true` Helm value. - Using Kubernetes manifests: set the `--snippets` flag in the nginx-gateway container argument, add `snippetsfilters` and `snippetspolicies` to the RBAC rules with verbs `list` and `watch`, and add `snippetsfilters/status` and `snippetspolicies/status` to the RBAC rules with verb `update`. See this [example manifest](https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/main/deploy/snippets/deploy.yaml) for clarification. Create the coffee and tea example applications: ```shell kubectl apply -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v/examples/snippets/app.yaml ``` Create a Gateway: ```shell kubectl apply -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v/examples/snippets/gateway.yaml ``` After creating the Gateway resource, NGINX Gateway Fabric will provision an NGINX Pod and Service fronting it to route traffic. Verify the gateway is created: ```shell kubectl describe gateways.gateway.networking.k8s.io gateway ``` Verify the status is `Accepted`: ```text Status: Addresses: Type: IPAddress Value: 10.96.36.219 Conditions: Last Transition Time: 2026-01-09T05:40:37Z Message: The Gateway is accepted Observed Generation: 1 Reason: Accepted Status: True Type: Accepted Last Transition Time: 2026-01-09T05:40:37Z Message: The Gateway is programmed Observed Generation: 1 Reason: Programmed Status: True Type: Programmed ``` Save the public IP address and port(s) of the Gateway into shell variables: ```text GW_IP=XXX.YYY.ZZZ.III GW_PORT= ``` **Note:** In a production environment, you should have a DNS record for the external IP address that is exposed, and it should refer to the hostname that the gateway will forward for. Create HTTPRoutes for the coffee and tea applications: ```shell kubectl apply -f - < ``` ### Configure coffee HTTPRoute to reference SnippetsFilter To use the `limit-except-sf` `SnippetsFilter`, update the coffee HTTPRoute to reference it: ```yaml kubectl apply -f - < 403 Forbidden

403 Forbidden

nginx
``` Since the `SnippetsFilter` was only referenced on the coffee HTTPRoute and the context was `http.server.location`, the tea HTTPRoute should not be affected. To verify, send a `POST` request to tea: ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/tea -POST ``` ```text Server address: 10.244.0.36:8080 Server name: tea-75bc9f4b6d-9msq7 Date: 21/Jan/2026:21:09:52 +0000 URI: /tea Request ID: 1fa64c2795e476534d2d8d6759a85c4c ``` ## Create a SnippetsPolicy Create a `SnippetsPolicy` named `limit-conn-sp` by adding the following `SnippetsPolicy`: ```yaml kubectl apply -f - < ``` Verify that the Gateway has been configured correctly: ```shell kubectl describe gateway gateway ``` You should see the following condition: ```text Status: Conditions: Last Transition Time: 2026-01-17T00:50:58Z Message: The SnippetsPolicy is applied to the resource Observed Generation: 1 Reason: PolicyAffected Status: True Type: SnippetsPolicyAffected ``` ### Send multiple requests in parallel The `SnippetsPolicy` targets the Gateway, meaning all attached Routes should be affected. This means that both the coffee and tea applications should be affected and have their maximum concurrent connections set to one. To verify the `SnippetsPolicy` is working, we will send multiple curl requests in parallel. Send requests to the coffee application: ```shell curl --parallel --parallel-immediate \ --resolve cafe.example.com:8080:127.0.0.1 "http://cafe.example.com:8080/coffee" \ --resolve cafe.example.com:8080:127.0.0.1 "http://cafe.example.com:8080/coffee" \ --resolve cafe.example.com:8080:127.0.0.1 "http://cafe.example.com:8080/coffee" ``` You should see that only a single request had a successful response while the others had `503 Service Temporarily Unavailable` status codes returned: ```text ... 503 Service Temporarily Unavailable

503 Service Temporarily Unavailable

nginx
Server address: 10.244.0.35:8080 Server name: coffee-654ddf664b-kmzzs Date: 21/Jan/2026:21:19:55 +0000 URI: /coffee Request ID: 1ffcc0fae174ea95da472c6984466ae5 ``` Next, verify the same behavior in the tea application: Send requests to the tea application: ```shell curl --parallel --parallel-immediate \ --resolve cafe.example.com:8080:127.0.0.1 "http://cafe.example.com:8080/tea" \ --resolve cafe.example.com:8080:127.0.0.1 "http://cafe.example.com:8080/tea" \ --resolve cafe.example.com:8080:127.0.0.1 "http://cafe.example.com:8080/tea" ``` The results should be the same: ```text ... 503 Service Temporarily Unavailable

503 Service Temporarily Unavailable

nginx
Server address: 10.244.0.36:8080 Server name: tea-75bc9f4b6d-9msq7 Date: 21/Jan/2026:21:23:07 +0000 URI: /tea Request ID: dfb42090e331b6ae31ed894b9b35d38d ``` This shows the `SnippetsPolicy` affecting all of the Gateway, in contrast to the `SnippetsFilter` that only affects the coffee HTTPRoute. ## Conclusion Snippets are a powerful tool to modifying NGINX configuration unavailable in first-class policies. This example showed a few small simple Snippets, but Snippets can contain any valid NGINX configuration, allowing users to fully customize configuration generated by NGINX Gateway Fabric and shape their NGINX configuration to fit their needs. However, as Snippets grow in complexity, because NGINX Gateway Fabric does not provide validation, the risk of generating invalid configuration due to conflicts in existing configuration grows and debugging can be challenging. This is one reason to try to keep Snippet configuration minimal, and to transition to using newly supported first-class polices and APIs if they provide the same NGINX configuration. ## Troubleshooting If a `SnippetsFilter` or `SnippetsPolicy` is applied with a Snippet which includes an invalid NGINX configuration, NGINX will continue to operate with the last valid configuration and an event with the error will be outputted. No new configuration will be applied until the invalid Snippet is fixed. An example of an error from the NGINX Gateway Fabric `nginx-gateway` container logs: ```text {"level":"error","ts":"2026-01-21T21:30:15Z","logger":"eventHandler","msg":"Failed to update NGINX configuration","error":"msg: Config apply failed, rolling back config; error: failed to parse config invalid number of arguments in \"limit_conn\" directive in /etc/nginx/includes/SnippetsPolicy_location_default-limit-conn-sp.conf:3","stacktrace":"github.com/nginx/nginx-gateway-fabric/v2/internal/controller.(*eventHandlerImpl).waitForStatusUpdates\n\tgithub.com/nginx/nginx-gateway-fabric/v2/internal/controller/handler.go:298"} ``` An example of an error from the NGINX Pod's `nginx` container logs: ```text time=2026-01-21T21:30:15.263Z level=ERROR msg="errors found during config apply, sending error status, rolling back config" err="failed to parse config invalid number of arguments in \"limit_conn\" directive in /etc/nginx/includes/SnippetsPolicy_location_default-limit-conn-sp.conf:3" correlation_id=8334bef4-44b3-4d13-93a9-a5fe84e7f351 server_type=command ``` If a Gateway is affected by a `SnippetsFilter` or `SnippetsPolicy` that creates invalid NGINX configuration, it may also contain information in its conditions describing the error: ```text Conditions: Last Transition Time: 2026-01-18T21:58:47Z Message: The Gateway is accepted Observed Generation: 1 Reason: Accepted Status: True Type: Accepted Last Transition Time: 2026-01-18T21:58:47Z Message: The Gateway is not programmed due to a failure to reload nginx with the configuration: msg: Config apply failed, rolling back config; error: failed to parse config invalid number of arguments in "limit_conn" directive in /etc/nginx/includes/SnippetsPolicy_location_default-limit-conn-sp.conf:3 Observed Generation: 1 Reason: Invalid Status: False Type: Programmed ... Listeners: Attached Routes: 2 Conditions: ... Last Transition Time: 2026-01-18T21:58:47Z Message: The Listener is not programmed due to a failure to reload nginx with the configuration: msg: Config apply failed, rolling back config; error: failed to parse config invalid number of arguments in "limit_conn" directive in /etc/nginx/includes/SnippetsPolicy_location_default-limit-conn-sp.conf:3 Observed Generation: 1 Reason: Invalid Status: False Type: Programmed ``` If a Route references a `SnippetsFilter` which cannot be resolved, the route will return a 500 HTTP error response on all requests. The Route conditions will contain information describing the error: ```text Conditions: Last Transition Time: 2026-01-18T22:01:13Z Message: The Route is accepted Observed Generation: 3 Reason: Accepted Status: True Type: Accepted Last Transition Time: 2026-01-18T22:01:13Z Message: Spec.rules[0].filters[0].extensionRef: Not found: {"group":"gateway.nginx.org","kind":"SnippetsFilter","name":"limit-except-sf"} Observed Generation: 3 Reason: InvalidFilter Status: False Type: ResolvedRefs ``` **Note:** If you run into situations where an NGINX directive fails to be applied and the troubleshooting information here isn't sufficient, please create an issue in the [NGINX Gateway Fabric Github repository](https://github.com/nginx/nginx-gateway-fabric). ## See also - [API reference](/ngf/reference/api.md): all configuration fields for the `SnippetsFilter` and `SnippetsPolicy` APIs. -------------------------------------------------------------------------------- title: "Support" url: https://docs.nginx.com/nginx-gateway-fabric/support/index.md -------------------------------------------------------------------------------- F5 NGINX Gateway Fabric adheres to the support policy detailed in the following knowledge base article: [K000140156](https://my.f5.com/manage/s/article/K000140156). After opening a support ticket, F5 staff will request additional information to better understand the problem. ## Kubernetes support plugin The [nginx-supportpkg-for-k8s](https://github.com/nginx/nginx-supportpkg-for-k8s) plugin collects the information needed by F5 Technical Support to assist with troubleshooting your issue. The plugin uses [krew](https://krew.sigs.k8s.io), the plugin manager for the Kubernetes [kubectl](https://kubernetes.io/docs/reference/kubectl/) command-line tool. The plugin may collect some or all of the following global and namespace-specific information: - Kubernetes version and information about Nodes and Custom Resources - Kubernetes metrics - Helm deployments - List of Pods, Events, ConfigMaps, Services, Deployments, Daemonsets, StatefulSets, ReplicaSets, and Leases - Pod log output - `nginx -T` output from NGINX-related Pods This plugin **does not** collect secrets or coredumps. Visit the [project’s GitHub repository](https://github.com/nginx/nginx-supportpkg-for-k8s) for further details. ## Support channels - If you experience issues with NGINX Gateway Fabric, please [open an issue](https://github.com/nginx/nginx-gateway-fabric/issues/new?assignees=&labels=&projects=&template=bug_report.md&title=) in GitHub. - If you have any suggestions or enhancement requests, please [open an idea](https://github.com/nginx/nginx-gateway-fabric/discussions/categories/ideas) on GitHub discussions. - You can also get help through the [NGINX Community Forum](https://community.nginx.org/). - If you need dedicated support for NGINX Gateway Fabric, or you would like to leverage our [advanced NGINX Plus features](https://docs.nginx.com/nginx-gateway-fabric/overview/nginx-plus/), you can contact [F5 Sales](https://www.f5.com/products/get-f5). -------------------------------------------------------------------------------- title: "Client Settings Policy API" url: https://docs.nginx.com/nginx-gateway-fabric/traffic-management/client-settings/index.md -------------------------------------------------------------------------------- Learn how to use the `ClientSettingsPolicy` API. ## Overview The `ClientSettingsPolicy` API allows Cluster Operators and Application Developers to configure the connection behavior between the client and NGINX. The settings in `ClientSettingsPolicy` correspond to the following NGINX directives: - [`client_max_body_size`](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size) - [`client_body_timeout`](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_timeout) - [`client_body_buffer_size`](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_buffer_size) - [`keepalive_requests`](https://nginx.org/en/docs/http/ngx_http_core_module.html#keepalive_requests) - [`keepalive_time`](https://nginx.org/en/docs/http/ngx_http_core_module.html#keepalive_time) - [`keepalive_timeout`](https://nginx.org/en/docs/http/ngx_http_core_module.html#keepalive_timeout) - [`keepalive_min_timeout`](https://nginx.org/en/docs/http/ngx_http_core_module.html#keepalive_min_timeout) `ClientSettingsPolicy` is an [Inherited PolicyAttachment](https://gateway-api.sigs.k8s.io/reference/policy-attachment/) that can be applied to a Gateway, HTTPRoute, or GRPCRoute in the same namespace as the `ClientSettingsPolicy`. When applied to a Gateway, the settings specified in the `ClientSettingsPolicy` affect all HTTPRoutes and GRPCRoutes attached to the Gateway. This allows Cluster Operators to set defaults for all applications using the Gateway. When applied to an HTTPRoute or GRPCRoute, the settings in the `ClientSettingsPolicy` affect only the route they are applied to. This allows Application Developers to set values for their applications based on their application's behavior or requirements. Settings applied to an HTTPRoute or GRPCRoute take precedence over settings applied to a Gateway. See the [custom policies](/ngf/overview/custom-policies.md) document for more information on policies. This guide will show you how to use the `ClientSettingsPolicy` API to configure the client max body size for your applications. For all the possible configuration options for `ClientSettingsPolicy`, see the [API reference](/ngf/reference/api.md). ## Before you begin - [Install](/ngf/install/) NGINX Gateway Fabric. Create the coffee and tea example applications: ```yaml kubectl apply -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v/examples/client-settings-policy/app.yaml ``` Create a Gateway: ```yaml kubectl apply -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v/examples/client-settings-policy/gateway.yaml ``` After creating the Gateway resource, NGINX Gateway Fabric will provision an NGINX Pod and Service fronting it to route traffic. Create HTTPRoutes for the coffee and tea applications: ```yaml kubectl apply -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v/examples/client-settings-policy/httproutes.yaml ``` Verify the gateway is created: ```shell kubectl describe gateways.gateway.networking.k8s.io gateway ``` Verify the status is `Accepted`: ```text Status: Conditions: Last Transition Time: 2026-01-09T05:40:37Z Message: The Gateway is accepted Observed Generation: 1 Reason: Accepted Status: True Type: Accepted Last Transition Time: 2026-01-09T05:40:37Z Message: The Gateway is programmed Observed Generation: 1 Reason: Programmed Status: True Type: Programmed ``` Save the public IP address and port(s) of the Gateway into shell variables: ```text GW_IP=XXX.YYY.ZZZ.III GW_PORT= ``` **Note:** In a production environment, you should have a DNS record for the external IP address that is exposed, and it should refer to the hostname that the gateway will forward for. Test the configuration: You can send traffic to the coffee and tea applications using the external IP address and port for the NGINX Service. Send a request to coffee: ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee ``` This request should receive a response from the coffee Pod: ```text Server address: 10.244.0.9:8080 Server name: coffee-76c7c85bbd-cf8nz ``` Send a request to tea: ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/tea ``` This request should receive a response from the tea Pod: ```text Server address: 10.244.0.9:8080 Server name: tea-76c7c85bbd-cf8nz ``` ## Configure client max body size ### Set a default client max body size for the Gateway To set a default client max body size for the Gateway created during setup, add the following `ClientSettingsPolicy`: ```yaml kubectl apply -f - < ``` Next, test that the policy is configured by sending a POST request to the coffee and tea applications exceeding the client's max body size of 50 bytes. ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee -X POST --data "this payload is greater than fifty bytes by four bytes" ``` You should receive the following error: ```text 413 Request Entity Too Large

413 Request Entity Too Large

nginx/1.25.5
``` Try again with a payload that's less than the 50 byte limit: ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee -X POST --data "this payload is under fifty bytes" ``` This time, you should receive a response from coffee: ```text Server address: 10.244.0.6:8080 Server name: coffee-56b44d4c55-7ldjc ``` You can repeat this test with the tea application to confirm that the policy affects both HTTPRoutes. ### Set a different client max body size for a route To set a different client max body size for a particular route, you can create another `ClientSettingsPolicy` that targets the route: ```yaml kubectl apply -f - < ``` Notice that the Ancestor Ref in the status is the tea HTTPRoute instead of the Gateway. Next, test that the policy is configured by sending a POST request to the tea application with a request body size greater than 50 bytes. ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/tea -X POST --data "this payload is greater than fifty bytes but less than seventy five" ``` You should receive a response from tea: ```text Server address: 10.244.0.7:8080 Server name: tea-596697966f-bf6tw ``` However, since the coffee app is still affected by the `ClientSettingsPolicy` attached to the Gateway, the same request to coffee should fail: ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee -X POST --data "this payload is greater than fifty bytes but less than seventy five" ``` ```text 413 Request Entity Too Large

413 Request Entity Too Large

nginx/1.25.5
``` To configure a `ClientSettingsPolicy` for a GRPCRoute, you can specify the GRPCRoute in the `spec.targetRef`: ```yaml kubectl apply -f - <" EOF ``` ## Configure the Guardrails backend Service The Guardrails backend can live outside or inside the cluster. NGINX Gateway Fabric picks the URL scheme from the referenced Service's type: | Backend location | Service type | Resolved URL | | ---------------- | ------------ | ------------ | | External | `ExternalName` | `https://:` | | In-cluster | `ClusterIP` (or any non-`ExternalName`) | `http://..svc.cluster.local:` | **note:** The `cluster.local` suffix in the in-cluster URL is the cluster's DNS domain. If your cluster uses a different domain, configure it with the `--cluster-domain` flag or `clusterDomain` Helm value when deploying NGINX Gateway Fabric (default: `cluster.local`). #### External To configure a Guardrails backend Service which is external, create an `ExternalName` Service pointing at your hosted Guardrails API: ```yaml kubectl apply -f - < ports: - name: https port: 443 protocol: TCP EOF ``` #### In-cluster For an in-cluster backend, your AI Guardrail backend pods will most likely have an existing Service which you can point the PayloadProcessor backendRef to, otherwise create a Service configured to expose your guardrail backends: ```yaml kubectl apply -f - <`. ## Deploy NGINX Gateway Fabric [Install](/ngf/install/) NGINX Gateway Fabric with the `PayloadProcessor` policy enabled: - Using Helm: set the `nginxGateway.payloadProcessor.enable=true` Helm value. - Using Kubernetes manifests: set the `--payload-processor` flag in the nginx-gateway container argument, and update the ClusterRole RBAC to add `payloadprocessors`: ```yaml - apiGroups: - gateway.nginx.org resources: - payloadprocessors verbs: - get - list - watch - apiGroups: - gateway.nginx.org resources: - payloadprocessors/status verbs: - update ``` ## Create a Gateway ```yaml kubectl apply -f - < ``` ## Create an HTTPRoute If you are using your own LLM, change the `backendRefs.name` and `backendRefs.port` to match the LLM's Service. ```yaml kubectl apply -f - <:$GW_PORT:$GW_IP http://:$GW_PORT/v1/completions \ -H "Content-Type: application/json" \ -d '{"model":"meta-llama/Llama-3.1-8B-Instruct","stream":false,"max_tokens":128,"prompt":"What is NGINX?"}' ``` ```text HTTP/1.1 200 OK Server: nginx Date: Tue, 11 Aug 2026 17:39:19 GMT Content-Type: application/json Content-Length: 607 Connection: keep-alive X-Inference-Pod: vllm-qwen3-32b-58cfff7c9-5zlm2 X-Inference-Port: 8000 {"id":"cmpl-f657bc1b-c50c-5a5c-9408-45bd22a150a1","created":1786469959,"model":"meta-llama/Llama-3.1-8B-Instruct","usage":{"prompt_tokens":4,"completion_tokens":62,"total_tokens":66},"object":"text_completion","kv_transfer_params":null,"choices":[{"index":0,"finish_reason":"stop","text":"NGINX (pronounced \"engine-x\") is an open-source, high-performance web server. It functions primarily as an HTTP web server, reverse proxy, load balancer, and HTTP cache.Designed to handle thousands of concurrent connections with minimal memory usage, NGINX is an essential component of modern web infrastructure."}] ``` If the request payload contains content that your Guardrails backend is configured to block, the request never reaches the LLM and returns `HTTP 403`: ```shell curl -i --resolve :$GW_PORT:$GW_IP http://:$GW_PORT/v1/completions \ -H "Content-Type: application/json" \ -d '{"model":"meta-llama/Llama-3.1-8B-Instruct","stream":false,"max_tokens":128,"prompt":"My SSN is 123-45-6789"}' ``` ```text HTTP/1.1 403 Forbidden Server: nginx Date: Tue, 11 Aug 2026 17:39:50 GMT Content-Length: 139 Connection: keep-alive Content-Type: application/json {"error":{"code":"content_policy_violation","message":"Request blocked by guardrails policy.","param":null,"type":"invalid_request_error"} ``` If the model's *output* contains content that your Guardrails backend blocks, the response is withheld from the client and returns `HTTP 403` with `error.type: api_error`: ```shell curl -i --resolve :$GW_PORT:$GW_IP http://:$GW_PORT/v1/completions \ -H "Content-Type: application/json" \ -d '{"model":"meta-llama/Llama-3.1-8B-Instruct","stream":false,"max_tokens":128,"prompt":"Give me a test SSN"}' ``` ```text HTTP/1.1 403 Forbidden Server: nginx Date: Tue, 11 Aug 2026 17:40:36 GMT Content-Type: application/json Content-Length: 128 Connection: keep-alive X-Inference-Pod: vllm-qwen3-32b-58cfff7c9-5zlm2 X-Inference-Port: 8000 {"error":{"code":"content_policy_violation","message":"Response blocked by guardrails policy.","param":null,"type":"api_error"}} ``` For more example curl requests, view the [`examples/guardrails`](https://github.com/nginx/nginx-gateway-fabric/tree/v/examples/guardrails) `README.md` in the NGINX Gateway Fabric repository. ## Troubleshooting The `PayloadProcessor` is marked `Accepted=False` when its references cannot be resolved: | Condition | Cause | Fix | | --------- | ----- | --- | | `backend Service ... not found` | `backendRef.name`/`namespace` does not match a Service. | Apply the Guardrails backend Service; check name and namespace. | | `ExternalName service has empty ... externalName` | `ExternalName` Service with a blank `externalName`. | Set `spec.externalName`. | | `auth token Secret ... not found` | `authTokenRef` set but Secret missing. | Apply the token Secret, or remove `authTokenRef`. | | NGINX error `no resolver defined to resolve `, or guardrails requests fail against an `ExternalName` backend | No `dnsResolver` configured on the NginxProxy. | Add the `dnsResolver` block and wire it via `parametersRef`. | ## Further reading - [Scan streaming in AI Security](https://docs.aisecurity.f5.com/api-docs/scan-request-streaming.html) - [Installation](/ngf/install/): install NGINX Gateway Fabric with the `PayloadProcessor` policy enabled. - [Custom policies](/ngf/overview/custom-policies.md): learn how inherited policies attach to Gateway API resources. - [`examples/guardrails`](https://github.com/nginx/nginx-gateway-fabric/tree/v/examples/guardrails): for more information on the example used in this guide. -------------------------------------------------------------------------------- title: "Upstream Settings Policy API" url: https://docs.nginx.com/nginx-gateway-fabric/traffic-management/upstream-settings/index.md -------------------------------------------------------------------------------- Learn how to use the `UpstreamSettingsPolicy` API. ## Overview The `UpstreamSettingsPolicy` API allows Application Developers to configure the behavior of a connection between NGINX and the upstream applications. The settings in `UpstreamSettingsPolicy` correspond to the following NGINX directives: - [`zone`]() - [`keepalive`]() - [`keepalive_requests`]() - [`keepalive_time`]() - [`keepalive_timeout`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive_timeout) - [`random`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#random) - [`least_conn`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#least_conn) - [`least_time`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#least_time) - [`upstream`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#upstream) - [`ip_hash`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#ip_hash) - [`hash`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#hash) - [`variables`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#variables) `UpstreamSettingsPolicy` is a [Direct Policy Attachment](https://gateway-api.sigs.k8s.io/reference/policy-attachment/) that can be applied to one or more services in the same namespace as the policy. `UpstreamSettingsPolicies` can only be applied to HTTP or gRPC services, in other words, services that are referenced by an HTTPRoute or GRPCRoute. See the [custom policies](/ngf/overview/custom-policies.md) document for more information on policies. This guide will show you how to use the `UpstreamSettingsPolicy` API to configure the load balancing method, upstream zone size and keepalives for your applications. For all the possible configuration options for `UpstreamSettingsPolicy`, see the [API reference](/ngf/reference/api.md). --- ## Before you begin - [Install](/ngf/install/) NGINX Gateway Fabric. ## Setup Create the `coffee` and `tea` example applications: ```yaml kubectl apply -f - < 80/TCP 23h service/tea ClusterIP 10.244.0.15 80/TCP 23h NAME READY STATUS RESTARTS AGE pod/coffee-676c9f8944-n9g6n 1/1 Running 0 23h pod/tea-6fbfdcb95d-cf84d 1/1 Running 0 23h ``` Create a Gateway: ```yaml kubectl apply -f - < ``` **Note:** In a production environment, you should have a DNS record for the external IP address that is exposed, and it should refer to the hostname that the gateway will forward for. Create HTTPRoutes for the `coffee` and `tea` applications: ```yaml kubectl apply -f - < ``` Next, verify that the policy has been applied to the `coffee` and `tea` upstreams by inspecting the NGINX configuration: ```shell kubectl exec -it deployments/gateway-nginx -- nginx -T ``` You should see the `zone` directive in the `coffee` and `tea` upstreams both specify the size `1m`: ```text upstream default_coffee_80 { random two least_conn; zone default_coffee_80 1m; server 10.244.0.14:8080; } upstream default_tea_80 { random two least_conn; zone default_tea_80 1m; server 10.244.0.15:8080; } ``` ## Enable keepalive connections By default, the `keepalive` directive is omitted, which results in the default NGINX `keepalive` value being used. You can override this value or disable `keepAlive` entirely by configuring an UpstreamSettingsPolicy. To disable keepalive, set the connections field to 0. The following example creates an `UpstreamSettingsPolicy` that configures keepalive connections for the `coffee` Service with a value of 24: ```yaml kubectl apply -f - < ``` Next, verify that the policy has been applied to the `coffee` upstreams, by inspecting the NGINX configuration: ```shell kubectl exec -it deployments/gateway-nginx -- nginx -T ``` You should see that the `coffee` upstream has the `keepalive` directive set to 24: ```text upstream default_coffee_80 { random two least_conn; zone default_coffee_80 1m; server 10.244.0.14:8080; keepalive 24; } ``` To disable the `keepalive` directive, lets create an `UpstreamSettingsPolicy` targeting the `tea` service with value 0: ```yaml kubectl apply -f - < ``` **note:** This setting applies only when the target Service has a ClusterIP. For headless Services (ClusterIP: None) and ExternalName Services, normal endpoint resolution is used instead. Additionally, this setting is also not applied to L4/stream upstreams. View the IP address of the `coffee` Service and verify it matches the IP address in the `coffee` upstream: ```shell kubectl get service coffee ``` ```text NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE coffee ClusterIP 10.96.23.26 80/TCP 16m ``` ```shell kubectl exec -it deployments/gateway-nginx -- nginx -T ``` ```text upstream default_coffee_80 { random two least_conn; zone default_coffee_80 1m; server 10.96.23.26:80; keepalive 32; } ``` --- ## Route upstream traffic to the Service ClusterIP By default, NGINX Gateway Fabric resolves each backend Service to its individual Pod IPs and uses those as the upstream servers. Setting `useClusterIP` to `true` in an `UpstreamSettingsPolicy` configures NGINX to route to the Service's ClusterIP and port instead, so the upstream contains a single server (the Service VIP). This is useful for service mesh compatibility and for controllers or operators that require traffic to traverse the Service VIP. You can also enable this globally for all Services through the `useClusterIP` field of the `NginxProxy` resource. When both are configured for the same Service, the `UpstreamSettingsPolicy` value takes precedence. See [Data plane configuration](/ngf/how-to/data-plane-configuration.md) for the global setting. **note:** Because the upstream contains only the Service VIP as a single server, you lose NGINX's load balancing across the backend Pods. Traffic is instead load balanced by the Kubernetes Service (kube-proxy), so the load balancing and keepalive settings of an `UpstreamSettingsPolicy` no longer apply to that Service. `useClusterIP` applies only when the target Service has a ClusterIP; headless (`ClusterIP: None`) and ExternalName Services fall back to the default Pod IP resolution. To route to the ClusterIP of the `coffee` service, create the following `UpstreamSettingsPolicy`: ```yaml kubectl apply -f - < ``` Find the ClusterIP of the `coffee` service: ```shell kubectl get service coffee ``` Next, verify that the `coffee` upstream targets that ClusterIP by inspecting the NGINX configuration: ```shell kubectl exec -it deployments/gateway-nginx -- nginx -T ``` You should see a single `server` in the `coffee` upstream set to the Service ClusterIP and port (`10.244.0.14` is the ClusterIP in this example): ```nginx upstream default_coffee_80 { random two least_conn; zone default_coffee_80 512k; server 10.244.0.14:80; keepalive 16; } ``` --- ## Further reading - [Custom policies](/ngf/overview/custom-policies.md): learn about how NGINX Gateway Fabric custom policies work. - [API reference](/ngf/reference/api.md): all configuration fields for the `UpstreamSettingsPolicy` API. -------------------------------------------------------------------------------- title: "Session Persistence" url: https://docs.nginx.com/nginx-gateway-fabric/traffic-management/session-persistence/index.md -------------------------------------------------------------------------------- Learn how to configure session persistence using NGINX Gateway Fabric. ## Overview In this guide, you’ll learn how to configure session persistence for your application. Session persistence ensures that multiple requests from the same client are consistently routed to the same backend Pod. This is useful when your application maintains in-memory state (for example, shopping carts or user sessions). NGINX Gateway Fabric supports configuring session persistence via `UpstreamSettingsPolicy` resource or directly on `HTTPRoute` and `GRPCRoute` resources. For NGINX OSS users, using the `ip_hash` load-balancing method provides basic session affinity by routing requests from the same client IP to the same backend Pod. For NGINX Plus users, cookie-based session persistence can be configured using the `sessionPersistence` field in a Route. In this guide, you will deploy three applications: - An application configured with `ip_hash` load-balancing method. - An application configured with cookie–based session persistence (if you have access to NGINX Plus). - A regular application with default load-balancing. These applications will showcase the benefits of session persistence for stateful workloads. The NGINX directives discussed in this guide are: - [`ip_hash`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#ip_hash) - [`sticky cookie`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#sticky) ## Note **Note:** Cookie-based `SessionPersistence` is only available for [NGINX Plus](/ngf/install/nginx-plus.md) users, with alternatives provided for NGINX OSS users. Session Persistence is a Gateway API field from the experimental release channel and is subject to change. ## Before you begin [Install](/ngf/install/nginx-plus.md) NGINX Gateway Fabric with **NGINX Plus** and experimental features enabled if you want to use cookie-based `sessionPersistence`. If you plan to use the `ip_hash` load-balancing method for session affinity instead, installing NGINX Gateway Fabric with **NGINX OSS** is sufficient. To use Gateway API experimental resources, the Gateway API resources from the experimental channel must be installed before deploying NGINX Gateway Fabric. Additionally, NGINX Gateway Fabric must have experimental features enabled. **Note:** As noted in the [Gateway API documentation](https://gateway-api.sigs.k8s.io/guides/#install-experimental-channel), future releases of the Gateway API can include breaking changes to experimental resources and fields. To install the Gateway API resources from the experimental channel, run the following: ```shell kubectl kustomize "https://github.com/nginx/nginx-gateway-fabric/config/crd/gateway-api/experimental?ref=v" | kubectl apply -f - ``` **Note:** If you plan to use the `edge` version of NGINX Gateway Fabric, you can replace the version in `ref` with `main`, for example `ref=main`. To enable experimental features on NGINX Gateway Fabric: Using Helm: Set `nginxGateway.gwAPIExperimentalFeatures.enable` to true. An example can be found in the [Installation with Helm](/ngf/install/helm.md#custom-installation-options) guide. Using Kubernetes manifests: Add the `--gateway-api-experimental-features` command-line flag to the deployment manifest args. An example can be found in the [Installation with Kubernetes manifests](/ngf/install/manifests.md#3-deploy-nginx-gateway-fabric) guide. ## Setup Create the `coffee`, `tea` and `latte` applications: ```yaml kubectl apply -f - < pod/coffee-5b9c74f9d9-7gfwn 1/1 Running 0 3h19m 10.244.0.94 kind-control-plane pod/latte-d5f64f67f-9t2j5 1/1 Running 0 3h19m 10.244.0.96 kind-control-plane pod/latte-d5f64f67f-drwc6 1/1 Running 0 3h19m 10.244.0.98 kind-control-plane pod/tea-859766c68c-cnb8n 1/1 Running 0 3h19m 10.244.0.93 kind-control-plane pod/tea-859766c68c-kttkb 1/1 Running 0 3h19m 10.244.0.97 kind-control-plane NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR service/coffee ClusterIP 10.96.169.1 80/TCP 3h19m app=coffee service/latte ClusterIP 10.96.42.39 80/TCP 3h19m app=latte service/tea ClusterIP 10.96.81.103 80/TCP 3h19m app=tea ``` Create a Gateway: ```yaml kubectl apply -f - < ``` **Note:** In a production environment, you should have a DNS record for the external IP address that is exposed, and it should refer to the hostname that the gateway will forward for. ## Session Persistence Methods ## Choosing the right session persistence method for your environment The choice between `ip_hash` and cookie-based session persistence depends on your use case. The `ip_hash` load-balancing method provides basic IP-based affinity: as long as NGINX sees the real client IP and not many users share that IP, requests from the same client will usually go to the same upstream Pod. However, there are important limitations: - **Shared IPs:** When there is a load balancer or proxy in front of NGINX that does not preserve the real client IP, or when many users appear to come from a single IP address (for example, corporate NAT or VPNs), `ip_hash` operates on the shared IP rather than on individual users. This means many different users behind the same IP are all routed to the same upstream Pod, so you do not get true per-user stickiness. - **Changing IPs:** If a user’s apparent IP changes over time (for example, when a load balancer or NAT pool uses multiple egress addresses), that user can be rehashed to a different upstream Pod and lose stickiness. Cookie-based session persistence with `sticky cookie` provides stronger, per-user stickiness. NGINX issues a session cookie, and all subsequent requests that present that cookie are routed to the same upstream Pod, regardless of changes in client IP or intermediate proxies. This is generally preferable for stateful, user-centric applications, while `ip_hash` can be a simpler option in NGINX OSS deployments where NGINX sees distinct, stable client IP addresses. ### Session Persistence with NGINX OSS In this section, you’ll configure a basic `coffee` HTTPRoute that routes traffic to the `coffee` Service. You’ll then attach an `UpstreamSettingsPolicy` to change the load-balancing method for that upstream to showcase session affinity behavior. NGINX hashes the client IP to select an upstream server, so requests from the same IP are routed to the same upstream as long as it is available. Session affinity quality with `ip_hash` depends on NGINX seeing the real client IP. In environments with external load balancers or proxies, operators must ensure appropriate `real_ip_header/set_real_ip_from` configuration so that `$remote_addr` reflects the end-user address otherwise, stickiness will be determined by the address of the front-end proxy rather than the actual client. To create an HTTPRoute for the `coffee` service, copy and paste the following into your terminal: ```yaml kubectl apply -f - < 81/TCP 3m1s service/tea ClusterIP 10.96.16.136 82/TCP 3m1s ``` Create a Gateway with two TCP listeners: ```yaml kubectl apply -f - < GW_PORT_2= ``` **Note:** In a production environment, you should have a DNS record for the external IP address that is exposed, and it should refer to the hostname that the gateway will forward for. Create TCPRoutes for routing to `coffee` and `tea` applications: ```yaml kubectl apply -f - < 53/UDP 13m ``` Create a Gateway with UDP listener: ```yaml kubectl apply -f - < ``` **Note:** In a production environment, you should have a DNS record for the external IP address that is exposed, and it should refer to the hostname that the gateway will forward for. Create UDPRoute for routing to `coredns` application: ```yaml kubectl apply -f - <> DiG 9.10.6 <<>> ${GW_IP} 53 cafe.example.com ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 13260 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ;; QUESTION SECTION: ;localhost. IN A ;; ANSWER SECTION: localhost. 600 IN A 127.0.0.1 ;; Query time: 136 msec ;; SERVER: 192.168.72.180#53(192.168.72.180) ;; WHEN: Thu Jan 08 21:44:46 MST 2026 ;; MSG SIZE rcvd: 54 ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 32036 ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ;; QUESTION SECTION: ;53. IN A ;; Query time: 141 msec ;; SERVER: 192.168.72.180#53(192.168.72.180) ;; WHEN: Thu Jan 08 21:44:47 MST 2026 ;; MSG SIZE rcvd: 31 ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 16210 ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ;; QUESTION SECTION: ;cafe.example.com. IN A ;; AUTHORITY SECTION: example.com. 900 IN SOA elliott.ns.cloudflare.com. dns.cloudflare.com. 2393120882 10000 2400 604800 1800 ;; Query time: 159 msec ;; SERVER: 192.168.72.180#53(192.168.72.180) ;; WHEN: Thu Jan 08 21:44:47 MST 2026 ;; MSG SIZE rcvd: 107 ``` ## Further Readings - [UDPRoute](https://gateway-api.sigs.k8s.io/reference/spec/#udproute) -------------------------------------------------------------------------------- title: "Proxy Settings Policy API" url: https://docs.nginx.com/nginx-gateway-fabric/traffic-management/proxy-settings/index.md -------------------------------------------------------------------------------- Learn how to use the `ProxySettingsPolicy` API. ## Overview The `ProxySettingsPolicy` API allows Cluster Operators and Application Developers to configure the connection behavior between NGINX Gateway Fabric and upstream applications (backends). The settings in `ProxySettingsPolicy` correspond to the following NGINX directives: - [`proxy_buffering`](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffering) - [`proxy_buffer_size`](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffer_size) - [`proxy_buffers`](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffers) - [`proxy_busy_buffers_size`](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_busy_buffers_size) - [`proxy_connect_timeout`](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_connect_timeout) - [`proxy_read_timeout`](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_read_timeout) - [`proxy_send_timeout`](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_send_timeout) `ProxySettingsPolicy` is an [Inherited PolicyAttachment](https://gateway-api.sigs.k8s.io/reference/policy-attachment/) that can be applied to a Gateway, HTTPRoute, or GRPCRoute in the same namespace as the `ProxySettingsPolicy`. When applied to a Gateway, the settings specified in the `ProxySettingsPolicy` affect all HTTPRoutes and GRPCRoutes attached to the Gateway. This allows Cluster Operators to set defaults for all applications using the Gateway. When applied to an HTTPRoute or GRPCRoute, the settings in the `ProxySettingsPolicy` affect only the route they are applied to. This allows Application Developers to set values for their applications based on their application's behavior or requirements. Settings applied to an HTTPRoute or GRPCRoute take precedence over settings applied to a Gateway. See the [custom policies](/ngf/overview/custom-policies.md) document for more information on policies. This guide will show you how to use the `ProxySettingsPolicy` API to configure proxy buffering for your applications. For all the possible configuration options for `ProxySettingsPolicy`, see the [API reference](/ngf/reference/api.md). ## Before you begin - [Install](/ngf/install/) NGINX Gateway Fabric. Create the coffee and tea example applications: ```shell kubectl apply -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v/examples/proxy-settings-policy/app.yaml ``` The coffee application is designed to generate large responses (10KB headers and 5MB body) to demonstrate buffering requirements. The tea application returns standard responses. Create a Gateway: ```shell kubectl apply -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v/examples/proxy-settings-policy/gateway.yaml ``` Create HTTPRoutes for the coffee and tea applications: ```shell kubectl apply -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v/examples/proxy-settings-policy/httproutes.yaml ``` After creating the Gateway resource, NGINX Gateway Fabric will provision an NGINX Pod and Service fronting it to route traffic. Verify the gateway is created: ```shell kubectl describe gateways.gateway.networking.k8s.io gateway ``` Verify the status is `Accepted`: ```text Status: Addresses: Type: IPAddress Value: 10.96.36.219 Conditions: Last Transition Time: 2026-01-09T05:40:37Z Message: The Gateway is accepted Observed Generation: 1 Reason: Accepted Status: True Type: Accepted Last Transition Time: 2026-01-09T05:40:37Z Message: The Gateway is programmed Observed Generation: 1 Reason: Programmed Status: True Type: Programmed ``` Save the public IP address and port(s) of the Gateway into shell variables: ```text GW_IP=XXX.YYY.ZZZ.III GW_PORT= ``` **Note:** In a production environment, you should have a DNS record for the external IP address that is exposed, and it should refer to the hostname that the gateway will forward for. Test the configuration: You can send traffic to the coffee and tea applications using the external IP address and port for the NGINX Service. Send a request to tea: ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/tea ``` This request should receive a response from the tea Pod: ```text Server address: 10.244.0.9:8080 Server name: tea-76c7c85bbd-cf8nz ``` Now send a request to coffee: ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee ``` This request will fail with a 502 Bad Gateway error: ```text 502 Bad Gateway

502 Bad Gateway

nginx
``` This error occurs because the coffee application generates a 10KB response header, which exceeds NGINX's default `proxy_buffer_size` (typically 4KB-8KB). You can verify this by checking the NGINX data plane logs: ```shell kubectl logs ``` Replace `` with the name of your NGINX Gateway Fabric data plane Pod (in the same namespace as your Gateway). You should see an error similar to: ```text [error] upstream sent too big header while reading response header from upstream, client: 127.0.0.1, server: cafe.example.com, request: "GET /coffee HTTP/1.1", upstream: "http://10.244.0.7:8080/coffee", host: "cafe.example.com:8080" ``` This demonstrates why proper proxy buffering configuration is essential for applications that generate large response headers or bodies. ## Configure proxy buffering ### Set default proxy buffering for the Gateway To set default proxy buffering settings for the Gateway created during setup, add the following `ProxySettingsPolicy`: ```shell kubectl apply -f - < ``` You can also verify that the policy was applied to the Gateway by checking the Gateway's status: ```shell kubectl describe gateway gateway ``` Look for the `ProxySettingsPolicyAffected` condition in the Gateway status: ```text Status: Conditions: Last Transition Time: 2026-01-08T10:03:29Z Message: The ProxySettingsPolicy is applied to the resource Observed Generation: 1 Reason: PolicyAffected Status: True Type: ProxySettingsPolicyAffected ``` This condition indicates that a `ProxySettingsPolicy` has been successfully applied to the Gateway. Test the configuration by sending requests to both applications: ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/tea ``` The tea application should respond normally with the configured buffering settings. ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee ``` The coffee application will still fail with a 502 Bad Gateway error because the Gateway-level policy's `bufferSize: "4k"` is not large enough to handle the coffee app's 10KB response headers. We'll fix this in the next section by applying a route-specific policy. ### Set different proxy buffering for a route To set different proxy buffering settings for a particular route, you can create another `ProxySettingsPolicy` that targets the route: ```shell kubectl apply -f - <10KB) bufferSize: "16k" # Configure more and larger buffers to handle 5MB response body buffers: number: 16 size: "64k" # Set busy buffers size to allow more data to be sent to client # while still receiving from upstream busyBuffersSize: "128k" EOF ``` This `ProxySettingsPolicy` targets the coffee HTTPRoute we created in the setup by specifying it in the `targetRefs` field. It sets larger buffering values specifically for the coffee application: - `bufferSize: "16k"`: Increases the buffer size to 16KB to accommodate the coffee app's 10KB response headers - `buffers.number: 16` and `buffers.size: "64k"`: Allocates 16 buffers of 64KB each (1MB total) to efficiently handle the 5MB response body - `busyBuffersSize: "128k"`: Allows more data to be sent to the client while still receiving from the upstream Since this policy is applied to the coffee HTTPRoute, it will only affect the coffee HTTPRoute. The `ProxySettingsPolicy` we created in the previous step will continue to affect all other routes attached to the Gateway, including the tea route. Verify that the `ProxySettingsPolicy` is Accepted: ```shell kubectl describe proxysettingspolicies.gateway.nginx.org coffee-proxy-settings ``` You should see the following status: ```text Status: Ancestors: Ancestor Ref: Group: gateway.networking.k8s.io Kind: HTTPRoute Name: coffee Namespace: default Conditions: Last Transition Time: 2026-01-08T10:03:29Z Message: Policy is accepted Observed Generation: 1 Reason: Accepted Status: True Type: Accepted Controller Name: gateway.nginx.org/nginx-gateway-controller Events: ``` Notice that the Ancestor Ref in the status is the coffee HTTPRoute instead of the Gateway. You can also verify that the policy was applied to the HTTPRoute by checking the route's status: ```shell kubectl describe httproute coffee ``` Look for the `ProxySettingsPolicyAffected` condition in the HTTPRoute status: ```text Status: Parents: Conditions: <...> Last Transition Time: 2026-01-08T10:03:29Z Message: The ProxySettingsPolicy is applied to the resource Observed Generation: 1 Reason: PolicyAffected Status: True Type: ProxySettingsPolicyAffected Controller Name: gateway.nginx.org/nginx-gateway-controller Parent Ref: Group: gateway.networking.k8s.io Kind: Gateway Name: gateway Namespace: default Section Name: http ``` Test that the policy is configured by sending requests to both applications: ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee ``` The coffee application should now successfully return the large response with the increased buffer settings. The request will complete successfully, and you'll receive the 5MB response body. ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/tea ``` The tea application continues to use the Gateway-level buffering settings since no route-specific policy is applied to it. To configure a `ProxySettingsPolicy` for a GRPCRoute, you can specify the GRPCRoute in the `spec.targetRefs`: ```shell kubectl apply -f - <(ms|s|m|h)` — for example, `5s`, `500ms`, `2m`. A value with no unit is interpreted as seconds. They are fully independent: there is no required ordering between them. ### Set proxy timeouts for the Gateway To set default timeout values for all routes attached to a Gateway: ```shell kubectl apply -f - < `bufferSize` 2. **Must be less than total buffers minus one buffer**: `busyBuffersSize` < (`buffers.number` × `buffers.size`) - `buffers.size` For example, with `buffers: {number: 8, size: "4k"}` (32KB total), valid values for `busyBuffersSize` are between `bufferSize` (exclusive) and 28KB (exclusive). **Validation limitation:** NGINX Gateway Fabric validates these constraints only when all fields are set in the same policy. If you set `busyBuffersSize` in one policy and `buffers` in another (via inheritance), you are responsible for ensuring the merged configuration satisfies NGINX's requirements. If the constraints are violated, NGINX will fail to reload and log an error. ## See also - [Custom policies](/ngf/overview/custom-policies.md): learn about how NGINX Gateway Fabric custom policies work. - [API reference](/ngf/reference/api.md): all configuration fields for the `ProxySettingsPolicy` API. -------------------------------------------------------------------------------- title: "ListenerSet API" url: https://docs.nginx.com/nginx-gateway-fabric/traffic-management/listener-sets/index.md -------------------------------------------------------------------------------- Learn how to use the `ListenerSet` API to specify additional listeners for a Gateway ## Overview A `ListenerSet` is a Gateway API type for specifying additional listeners for a Gateway. It decouples network listener configurations—such as ports, hostnames, and TLS termination—from the central Gateway resource and provides a mechanism to merge multiple listeners into a single Gateway. This enables multiple application developer teams who manage their own Services and Routes to configure their own listeners on a Gateway without needing to modify the Gateway itself (which may be owned by a different team). Additionally, by using ListenerSets, users are able to scale beyond the 64-listener limit of a single Gateway resource. ## Before you begin - [Install](/ngf/install/) NGINX Gateway Fabric. ## Set up Create the `coffee` application in Kubernetes by copying and pasting the following block into your terminal: ```yaml kubectl apply -f - < 80/TCP 7s ``` Create a Gateway: ```yaml kubectl apply -f - < ``` **Note:** In a production environment, you should have a DNS record for the external IP address that is exposed, and it should refer to the hostname that the gateway will forward for. ## Configure a ListenerSet For those that are familiar with the Gateway resource, ListenerSet `spec.listeners` is a direct copy of the Gateway's `spec.listeners`. For more information, view the [API Reference](https://gateway-api.sigs.k8s.io/reference/spec/#listenerset). Create a ListenerSet: ```yaml kubectl apply -f - < ``` **Note:** In a production environment, you should have a DNS record for the external IP address that is exposed, and it should refer to the hostname that the gateway will forward for. Test the configuration: You can send traffic to the coffee and tea applications using the external IP address and port for the NGINX Service. Send a request to coffee: ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee ``` This request should receive a response from the coffee Pod: ```text Server address: 10.244.0.22:8080 Server name: coffee-654ddf664b-6mwtb ``` Send a request to tea: ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/tea ``` This request should receive a response from the tea Pod: ```text Server address: 10.244.0.23:8080 Server name: tea-75bc9f4b6d-g9t84 ``` Before we enable rate limiting, try sending multiple requests to coffee: ```shell for i in `seq 1 10`; do curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee; done ``` You should see all successful responses in quick succession as we have not configured any rate limiting rules yet. ## Configure Rate Limiting ### Set Rate Limiting on an HTTPRoute To set rate limit settings for the coffee HTTPRoute created during setup, add the following `RateLimitPolicy`: ```shell kubectl apply -f - < ``` You can also verify that the policy was applied to the HTTPRoute by checking the HTTPRoute's status: ```shell kubectl describe httproute coffee ``` Look for the `RateLimitPolicyAffected` condition in the HTTPRoute status: ```text Status: Conditions: Last Transition Time: 2026-01-15T22:17:46Z Message: The RateLimitPolicy is applied to the resource Observed Generation: 1 Reason: PolicyAffected Status: True Type: RateLimitPolicyAffected ``` This condition indicates that a `RateLimitPolicy` has been successfully applied to the HTTPRoute. Test the configuration by sending a request to the coffee application: ```shell curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee ``` This request should receive a response from the coffee Pod: ```text Server address: 10.244.0.22:8080 Server name: coffee-654ddf664b-6mwtb ``` When processing a single request, the rate limiting configuration has no noticeable effect. Try to exceed the set rate limit with a script that sends multiple requests. ```shell for i in `seq 1 10`; do curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee; done ``` You should see all successful responses from the coffee Pod, but they should be spaced apart roughly one second each as expected through the rate limiting configuration. Using the same script on the tea application, we can see there are no rate limit settings applied to the tea application: ```shell for i in `seq 1 10`; do curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/tea; done ``` ### Set Rate Limiting on a GRPCRoute RateLimitPolicy can also target GRPCRoutes. To do so, re-use the policy created for the coffee HTTPRoute and add an additional targetRef: ```shell kubectl apply -f - < ``` You can also verify that the policy was applied to the GRPCRoute by checking the GRPCRoute's status: ```shell kubectl describe grpcroute grpc-route ``` Look for the `RateLimitPolicyAffected` condition in the GRPCRoute status: ```text Status: Conditions: Last Transition Time: 2026-01-15T22:19:35Z Message: The RateLimitPolicy is applied to the resource Observed Generation: 1 Reason: PolicyAffected Status: True Type: RateLimitPolicyAffected ``` This condition indicates that a `RateLimitPolicy` has been successfully applied to the GRPCRoute. #### Send gRPC traffic To access the application and test the `RateLimitPolicy` has been applied to the GRPCRoute, we will use [grpcurl](https://github.com/fullstorydev/grpcurl?tab=readme-ov-file#installation]). To test our application, we will need to create a separate `.proto` source file since we are running things locally. Create a new file named `grpc.proto` with these contents: ```proto syntax = "proto3"; option go_package = "google.golang.org/grpc/examples/helloworld/helloworld"; option java_multiple_files = true; option java_package = "io.grpc.examples.helloworld"; option java_outer_classname = "HelloWorldProto"; package helloworld; // The greeting service definition. service Greeter { // Sends a greeting rpc SayHello (HelloRequest) returns (HelloReply) {} } // The request message containing the user's name. message HelloRequest { string name = 1; } // The response message containing the greetings message HelloReply { string message = 1; } ``` In the same directory, test our gRPC application through grpcurl: ```shell grpcurl -plaintext -proto grpc.proto -authority grpc.example.com -d '{"name": "exact"}' ${GW_IP}:${GW_PORT} helloworld.Greeter/SayHello ``` The request should receive a response from the gRPC backend: ```text { "message": "Hello exact" } ``` When processing a single request, the rate limiting configuration has no noticeable effect. Try to exceed the set rate limit with a script that sends multiple requests. ```shell for i in `seq 1 10`; do grpcurl -plaintext -proto grpc.proto -authority grpc.example.com -d '{"name": "exact"}' ${GW_IP}:${GW_PORT} helloworld.Greeter/SayHello; done ``` You should see all successful responses from the grpc-backend, but they should be spaced apart roughly one second each as expected through the rate limiting configuration. ### Set Rate Limiting on a Gateway Before setting rate limiting on the Gateway, verify that the tea application isn't affected by any rate limit configuration. ```shell for i in `seq 1 10`; do curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/tea; done ``` You should see that these requests have successful responses with no delays. To set rate limit settings for the Gateway we created in the setup, add the following `RateLimitPolicy`: ```shell kubectl apply -f - < ``` You can also verify that the policy was applied to the Gateway by checking the Gateway's status: ```shell kubectl describe gateway gateway ``` Look for the `RateLimitPolicyAffected` condition in the Gateway status: ```text Status: Conditions: Last Transition Time: 2026-01-15T22:23:38Z Message: The RateLimitPolicy is applied to the resource Observed Generation: 1 Reason: PolicyAffected Status: True Type: RateLimitPolicyAffected ``` This condition indicates that a `RateLimitPolicy` has been successfully applied to the Gateway. Verify that the tea application is affected by the rate limit configuration: ```shell for i in `seq 1 10`; do curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/tea; done ``` Even though there is no `RateLimitPolicy` specifically targeting the tea HTTPRoute, since the HTTPRoute is attached to the Gateway, it is affected by the policy. There should be a mix of successful responses alongside `503` error codes (default error code). This is expected because of the omission of the `burst` field in this `RateLimitPolicy`, causing the excessive requests to have an error code returned as the response instead of being delayed. For more information on underlying NGINX configuration of the `limit_req_module`, see the official [NGINX documentation](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html). The `RateLimitPolicy` attached to the Gateway will affect all Routes, meaning the coffee HTTPRoute and grpc-route GRPCRoute are also affected by this policy. In NGINX, a request will pass through all `limit_req` directives that apply to it, if any of the rates are exceeded, that request will be delayed and subject to the rules of that `limit_req`. In this case, since the `RateLimitPolicy` attached to the Routes has the same `key` and has a lower rate of `1r/s`, requests sent to the coffee and grpc-backend under the rate of `10r/s` will still get delayed by the `1r/s` rate. To test this, send five requests to the coffee application: ```shell for i in `seq 1 5`; do curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee; done ``` The requests will each be delayed at a rate of one request per second. ### View and Configure Rate Limit Logs For each request that is rate limited by NGINX, a log message is associated with it. Send a few requests to both tea and coffee applications: ```shell for i in `seq 1 5`; do curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee; done ``` ```shell for i in `seq 1 5`; do curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/tea; done ``` View the NGINX logs: ```shell kubectl logs ``` ```text 127.0.0.1 - - [15/Jan/2026:20:44:57 +0000] "GET /coffee HTTP/1.1" 200 161 "-" "curl/8.7.1" 2026/01/15 20:44:57 [warn] 5788#5788: *2927 delaying request, excess: 0.943, by zone "default_rl_route-rate-limit_rule0", client: 127.0.0.1, server: cafe.example.com, request: "GET /coffee HTTP/1.1", host: "cafe.example.com:8080" 127.0.0.1 - - [15/Jan/2026:20:44:58 +0000] "GET /coffee HTTP/1.1" 200 161 "-" "curl/8.7.1" 2026/01/15 20:44:58 [info] 5788#5788: *2925 client 127.0.0.1 closed keepalive connection 2026/01/15 20:44:58 [warn] 5788#5788: *2930 delaying request, excess: 0.885, by zone "default_rl_route-rate-limit_rule0", client: 127.0.0.1, server: cafe.example.com, request: "GET /coffee HTTP/1.1", host: "cafe.example.com:8080" 127.0.0.1 - - [15/Jan/2026:20:44:59 +0000] "GET /coffee HTTP/1.1" 200 161 "-" "curl/8.7.1" ... 127.0.0.1 - - [15/Jan/2026:20:45:05 +0000] "GET /tea HTTP/1.1" 200 155 "-" "curl/8.7.1" 127.0.0.1 - - [15/Jan/2026:20:45:05 +0000] "GET /tea HTTP/1.1" 503 190 "-" "curl/8.7.1" 2026/01/15 20:45:05 [error] 5788#5788: *2938 limiting requests, excess: 0.500 by zone "default_rl_gateway-rate-limit_rule0", client: 127.0.0.1, server: cafe.example.com, request: "GET /tea HTTP/1.1", host: "cafe.example.com:8080" 127.0.0.1 - - [15/Jan/2026:20:45:05 +0000] "GET /tea HTTP/1.1" 200 155 "-" "curl/8.7.1" 127.0.0.1 - - [15/Jan/2026:20:45:05 +0000] "GET /tea HTTP/1.1" 503 190 "-" "curl/8.7.1" 2026/01/15 20:45:05 [error] 5788#5788: *2941 limiting requests, excess: 0.610 by zone "default_rl_gateway-rate-limit_rule0", client: 127.0.0.1, server: cafe.example.com, request: "GET /tea HTTP/1.1", host: "cafe.example.com:8080" ... ``` You should be able to see NGINX logs at the default `error` log level showing NGINX limiting requests. Since the `coffee` application has `burst=3`, the requests are delayed and are all met with the `200` status code, while the `tea` application doesn't have `burst`, meaning all excessive requests are met with a `503`. Both the error log level and status code are adjustable in the `RateLimitPolicy`: ```shell kubectl apply -f - < ``` ```text 127.0.0.1 - - [21/Jan/2026:18:55:39 +0000] "GET /tea HTTP/1.1" 200 155 "-" "curl/8.7.1" 127.0.0.1 - - [21/Jan/2026:18:55:39 +0000] "GET /tea HTTP/1.1" 429 162 "-" "curl/8.7.1" 2026/01/21 18:55:39 [warn] 1290#1290: *595 limiting requests, excess: 0.580 by zone "default_rl_gateway-rate-limit_rule0", client: 127.0.0.1, server: cafe.example.com, request: "GET /tea HTTP/1.1", host: "cafe.example.com:8080" 127.0.0.1 - - [21/Jan/2026:18:55:39 +0000] "GET /tea HTTP/1.1" 200 155 "-" "curl/8.7.1" 127.0.0.1 - - [21/Jan/2026:18:55:39 +0000] "GET /tea HTTP/1.1" 429 162 "-" "curl/8.7.1" 2026/01/21 18:55:39 [warn] 1292#1292: *598 limiting requests, excess: 0.470 by zone "default_rl_gateway-rate-limit_rule0", client: 127.0.0.1, server: cafe.example.com, request: "GET /tea HTTP/1.1", host: "cafe.example.com:8080" 2026/01/21 18:55:39 [warn] 1293#1293: *599 limiting requests, excess: 0.020 by zone "default_rl_gateway-rate-limit_rule0", client: 127.0.0.1, server: cafe.example.com, request: "GET /tea HTTP/1.1", host: "cafe.example.com:8080" 127.0.0.1 - - [21/Jan/2026:18:55:39 +0000] "GET /tea HTTP/1.1" 429 162 "-" "curl/8.7.1" ``` ## Important Notes ### RateLimitPolicies set on a Gateway and Route Since Routes are affected by all `RateLimitPolicies` on a Gateway, there is no way for a `RateLimitPolicy` attached to a Route to overwrite/negate a rule set by one on a Gateway. ### Conflicts When multiple `RateLimitPolicies` select the same targetRef and specify any of dryRun, logLevel, or rejectCode, only one policy will be applied. The controller selects the policy with the highest priority (based on time created, if created at the same time, ties are calculated on alphabetical order sorting of the policy name) and rejected policies will have the `Accepted` Condition set to false with the reason `Conflicted`. ### Dry Run The [limit_req_dry_run](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_dry_run) NGINX directive can be enabled by setting `spec.rateLimit.dryRun` to `true`. In this mode, rate limit is not applied, but the number of excessive requests is accounted as usual in the shared memory zone. ### NGINX Worker process not reloading after modifying key of existing limit_req_zone In NGINX, there is a measure of protection against modifying the `key` of an existing `limit_req_zone`, if done so, this error message will show up in the NGINX logs and the worker processes won't be reloaded, meaning the new configuration won't be in effect: ```text 2026/01/21 23:45:45 [emerg] 32645#32645: limit_req (a zone name) uses the (new key name) key while previously it used the (old key name) key ``` This will occur in the `RateLimitPolicy` if the `key` for a rule is modified after creation of the policy. This won't occur upon first creation of the `RateLimitPolicy`, or when adding new rules, only when modifying an existing rule. A workaround is to either delete and re-deploy the `RateLimitPolicy` or to modify the `zoneSize` field alongside the `key`. Modifying the `zoneSize` will result in the creation of a new zone and a reloading of NGINX processes. You can verify the worker processes have been reloaded by checking for these NGINX logs: ```text ... 2026/01/22 19:17:24 [notice] 12#12: start worker process 85 2026/01/22 19:17:24 [notice] 12#12: start worker process 86 2026/01/22 19:17:24 [notice] 12#12: start worker process 87 2026/01/22 19:17:24 [notice] 48#48: gracefully shutting down 2026/01/22 19:17:24 [notice] 49#49: gracefully shutting down ... time=2026-01-22T19:17:27.978Z level=INFO msg="NGINX workers have been reloaded" correlation_id=e0c6953c-cb1d-45fd-826f-0bf66a192700 server_type=command time=2026-01-22T19:17:27.978Z level=INFO msg="NGINX reloaded" process_id=12 correlation_id=e0c6953c-cb1d-45fd-826f-0bf66a192700 server_type=command ``` ## See also - [NGINX limit_req_module](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html): for more information on the underlying NGINX directives. - [Custom policies](/ngf/overview/custom-policies.md): learn about how NGINX Gateway Fabric custom policies work. - [API reference](/ngf/reference/api.md): all configuration fields for the `RateLimitPolicy` API. -------------------------------------------------------------------------------- title: "Configure upstream HTTP/2 through Service appProtocol" url: https://docs.nginx.com/nginx-gateway-fabric/traffic-management/upstream-http2-appprotocol/index.md description: Configure NGINX Gateway Fabric to use HTTP/2 for upstream connections by setting appProtocol to kubernetes.io/h2c on a Kubernetes Service port. -------------------------------------------------------------------------------- Learn how to configure NGINX Gateway Fabric to use HTTP/2 when proxying requests to upstream services using the Service port's `appProtocol` field. ## Overview The appProtocol field on a Kubernetes Service port provides a way to specify an application protocol. Controllers such as NGINX Gateway Fabric may use this field to enable protocol-specific functionality for supported protocols. For more information, view the official [Kubernetes Service Documentation](https://kubernetes.io/docs/concepts/services-networking/service/#application-protocol). When a Kubernetes Service port has `appProtocol` set to `kubernetes.io/h2c`, NGINX Gateway Fabric configures the corresponding NGINX location to use HTTP/2 for upstream connections by setting the [`proxy_http_version`](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_http_version) directive to `2` in the NGINX configuration. ## Important Notes - `kubernetes.io/h2c` is supported on HTTPRoutes and GRPCRoutes. It isn't supported on TLSRoutes. - When NGINX Gateway Fabric detects `kubernetes.io/h2c`, it also omits the `proxy_set_header Upgrade` and `proxy_set_header Connection` directives from the location block, because those headers are HTTP/1.1-specific and aren't used in HTTP/2 connections. ### Other supported appProtocols In addition to the `kubernetes.io/h2c` `appProtocol`, NGINX Gateway Fabric recognizes `kubernetes.io/ws` and `kubernetes.io/wss` as described in [RFC 6455](https://www.rfc-editor.org/info/rfc6455/). These `appProtocols` reference WebSocket over cleartext and WebSocket over TLS respectively and are supported natively by our default NGINX configuration. `kubernetes.io/ws` is only supported on HTTPRoutes and `kubernetes.io/wss` is only supported on TLSRoutes or on HTTPRoutes with an associated BackendTLSPolicy. If an `appProtocol` on a Service port is referenced by an unsupported Route type, that backendRef will be considered invalid and status will be written to the Route. NGINX Gateway Fabric is conformant to the information in [GEP-1911](https://gateway-api.sigs.k8s.io/geps/gep-1911/), which should be referenced for more detailed information. ## Troubleshooting - For NGINX to set `proxy_http_version 2` for a location, all valid backend references in the routing rule must have `appProtocol: kubernetes.io/h2c` set on their Service ports. If any valid backend doesn't use `kubernetes.io/h2c`, NGINX falls back to the default HTTP/1.1. ## See also - [Backend Protocol](https://gateway-api.sigs.k8s.io/guides/user-guides/backend-protocol/): the Gateway API guide for the `appProtocol` field. - [GEP-1911](https://gateway-api.sigs.k8s.io/geps/gep-1911/): for more details on how NGINX Gateway Fabric interacts with `appProtocols`. - [proxy_http_version NGINX directive](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_http_version). -------------------------------------------------------------------------------- title: "F5 NGINX Ingress Controller" url: https://docs.nginx.com/nginx-ingress-controller/index.md -------------------------------------------------------------------------------- ## About NGINX Ingress Controller is an [Ingress Controller](/glossary/#k8s-ingress-controller) implementation for NGINX that can load balance Websocket, gRPC, TCP and UDP applications. It supports standard [Ingress](/glossary/#k8s-ingress-controller) features such as content-based routing and TLS/SSL termination. Several NGINX and NGINX Plus features are available as extensions to Ingress resources through [Annotations](/nic/configuration/ingress-resources/advanced-configuration-with-annotations) and the [ConfigMap](/nic/configuration/global-configuration/configmap-resource) resource. ## Featured content ### [Install NGINX Ingress Controller with Helm](/nginx-ingress-controller/install/helm) Use Helm to deploy and configure a NGINX Ingress Controller cluster ### [Migrate from Ingress-NGINX Controller](/nginx-ingress-controller/install/migrate-ingress-nginx) Replace an Ingress-NGINX cluster with NGINX Ingress Controller ### [Changelog](/nginx-ingress-controller/changelog) Review the changes from the latest NGINX Ingress Controller releases -------------------------------------------------------------------------------- title: "About" url: https://docs.nginx.com/nginx-ingress-controller/lts/overview/about/index.md -------------------------------------------------------------------------------- This document describes the F5 NGINX Ingress Controller LTS, an Ingress Controller implementation for NGINX Plus. The LTS version of NGINX Ingress Controller is based on the latest stable release of NGINX. It is *feature-frozen* at the time of release — no new features are added after release. Only critical bug fixes and security patches are delivered as patch revisions. It is ideal for users who prioritize stability and long-term support over access to the latest features. This version is supported for 36 months from the date of release, with security patches provided throughout the support period. For users who require access to the latest features and improvements, we recommend using the latest mainline release of NGINX Ingress Controller. NGINX Ingress Controller LTS is an [Ingress Controller](/glossary/#k8s-ingress-controller) implementation for [NGINX Plus](https://www.f5.com/products/nginx/nginx-plus) that can load balance Websocket, gRPC, TCP and UDP applications. NGINX Ingress Controller LTS gives you a way to manage NGINX through the [Kubernetes](https://kubernetes.io/) API, and is built to handle the continuous change that happens in Kubernetes environments. It supports standard [Ingress](/glossary/#k8s-ingress-controller) features such as content-based routing and TLS/SSL termination. NGINX Plus features are available as extensions to Ingress resources through [Annotations](/nic/lts/configuration/ingress-resources/advanced-configuration-with-annotations) and the [ConfigMap](/nic/lts/configuration/global-configuration/configmap-resource) resource. NGINX Ingress Controller LTS supports the [VirtualServer and VirtualServerRoute resources](/nic/lts/configuration/virtualserver-and-virtualserverroute-resources) as alternatives to Ingress, enabling traffic splitting and advanced content-based routing. It also supports TCP, UDP and TLS Passthrough load balancing using [TransportServer resources](/nic/lts/configuration/transportserver-resource). To learn more about NGINX Ingress Controller LTS, please read the [The design of NGINX Ingress Controller LTS](/nic/lts/overview/design.md) and [Extensibility with NGINX Plus](/nic/lts/overview/nginx-plus.md) topics. -------------------------------------------------------------------------------- title: "About" url: https://docs.nginx.com/nginx-ingress-controller/overview/about/index.md -------------------------------------------------------------------------------- This document describes the F5 NGINX Ingress Controller, an Ingress Controller implementation for NGINX and NGINX Plus. NGINX Ingress Controller is an [Ingress Controller](/glossary/#k8s-ingress-controller) implementation for [NGINX](https://nginx.org) and [NGINX Plus](https://www.f5.com/products/nginx/nginx-plus) that can load balance Websocket, gRPC, TCP and UDP applications. NGINX Ingress Controller gives you a way to manage NGINX through the [Kubernetes](https://kubernetes.io/) API, and is built to handle the continuous change that happens in Kubernetes environments. It supports standard [Ingress](/glossary/#k8s-ingress-controller) features such as content-based routing and TLS/SSL termination. Several NGINX and NGINX Plus features are available as extensions to Ingress resources through [Annotations](/nic/configuration/ingress-resources/advanced-configuration-with-annotations) and the [ConfigMap](/nic/configuration/global-configuration/configmap-resource) resource. NGINX Ingress Controller supports the [VirtualServer and VirtualServerRoute resources](/nic/configuration/virtualserver-and-virtualserverroute-resources) as alternatives to Ingress, enabling traffic splitting and advanced content-based routing. It also supports TCP, UDP and TLS Passthrough load balancing using [TransportServer resources](/nic/configuration/transportserver-resource). To learn more about NGINX Ingress Controller, please read the [The design of NGINX Ingress Controller](/nic/overview/design.md) and [Extensibility with NGINX Plus](/nic/overview/nginx-plus.md) topics. -------------------------------------------------------------------------------- title: "Basic configuration" url: https://docs.nginx.com/nginx-ingress-controller/configuration/ingress-resources/basic-configuration/index.md -------------------------------------------------------------------------------- This document shows a basic Ingress resource definition for F5 NGINX Ingress Controller. It load balances requests for two services as part of a single application. Here is a breakdown of what this Ingress resource definition means: - The `metadata.name` field defines the name of the resource `cafe‑ingress`. - The `spec.tls` field sets up SSL/TLS termination: - The `hosts` field applies the certificate and key to the `cafe.example.com` host. - The `secretName` references a secret resource by its name, `cafe‑secret`. The secret must belong to the same namespace as the Ingress, of the type ``kubernetes.io/tls`` and contain keys named ``tls.crt`` and ``tls.key`` that hold the certificate and private key as described [here](https://kubernetes.io/docs/concepts/services-networking/ingress/#tls>). If the secret doesn't exist or is invalid, NGINX will break any attempt to establish a TLS connection to the hosts to which the secret is applied. - The `spec.rules` field defines a host with the domain name `cafe.example.com`. - The `paths` field defines two path‑based rules: - The rule with the path `/tea` instructs NGINX to distribute the requests with the `/tea` URI among the pods of the *tea* service, which is deployed with the name `tea‑svc` in the cluster. - The rule with the path `/coffee` instructs NGINX to distribute the requests with the `/coffee` URI among the pods of the *coffee* service, which is deployed with the name `coffee‑svc` in the cluster. - Both rules instruct NGINX to distribute the requests to `port 80` of the corresponding service (the `servicePort` field). To learn more about the Ingress resource, view [the official Kubernetes documentation for Ingress resources](https://kubernetes.io/docs/concepts/services-networking/ingress/). **Note:** For complete instructions on deploying Ingress and Secret resources in the cluster, see the [complete example](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/complete-example) in the GitHub repository. ## New features available in Kubernetes 1.18 Starting from Kubernetes 1.18, you can use the following new features: - The host field supports wildcard domain names, such as `*.example.com`. - The path supports different matching rules with the new field `pathType`, which takes the following values: `Prefix` for prefix-based matching, `Exact` for exact matching and `ImplementationSpecific`, which is the default type and is the same as `Prefix`. For example: ```yaml {hl_lines=[2, 7, 14]} - path: /tea pathType: Prefix backend: serviceName: tea-svc servicePort: 80 - path: /tea/green pathType: Exact backend: service: name: tea-svc port: number: 80 - path: /coffee pathType: ImplementationSpecific backend: service: name: coffee-svc port: number: 80 ``` - The `ingressClassName` field is now supported: ```yaml {hl_lines=[6]} apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: cafe-ingress spec: ingressClassName: nginx tls: - hosts: - cafe.example.com secretName: cafe-secret rules: - host: cafe.example.com . . . ``` When using this field you need to create the `IngressClass` resource with the corresponding `name`. View the [Create common resources](/nic/install/manifests.md#create-common-resources) section of the Installation with Manifests topic for more information. ## Restrictions NGINX Ingress Controller imposes the following restrictions on Ingress resources: - The `host` field is required by default. To allow Ingress resources to omit the `host` field, use the [`-allow-empty-ingress-host`](/nic/configuration/global-configuration/command-line-arguments.md#cmdoption-allow-empty-ingress-host) command-line argument. - When `-allow-empty-ingress-host` is enabled, `spec.tls` is not supported on Ingress resources without a `host`. To configure TLS, use the [`-default-server-tls-secret`](/nic/configuration/global-configuration/command-line-arguments.md#cmdoption-default-server-tls-secret) command-line argument instead. - The `host` value needs to be unique among all Ingress and VirtualServer resources unless the Ingress resource is a [mergeable minion](/nic/configuration/ingress-resources/cross-namespace-configuration.md). View the [Host and Listener collisions](/nic/configuration/host-and-listener-collisions.md) topic for more information. - The `path` field in `spec.rules[].http.paths[]` is required for `Exact` and `Prefix` `pathTypes`. - The ImplementationSpecific `pathType` is treated as equivalent to `Prefix` `pathType`, with the exception that when this `pathType` is configured, the `path` field in `spec.rules[].http.paths[]` is not mandatory. `path` defaults to `/` if not set but the `pathType` is set to ImplementationSpecific. ## Advanced configuration NGINX Ingress Controller generates NGINX configuration by executing a template file that contains the configuration options. These options are set with the Ingress resource and NGINX Ingress Controller's ConfigMap. The Ingress resource only allows you to use basic NGINX features: host and path-based routing and TLS termination. For advanced configuration, you have two options: - [Annotations](/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md) can be used to rewrite request URIs or inserting additional response headers. - [Snippets](/nic/configuration/ingress-resources/advanced-configuration-with-snippets) can be used to insert raw NGINX configuration, changing generated files. Additionally, it is possible to customize the template, described in the [Custom templates](/nic/configuration/global-configuration/custom-templates.md) topic. -------------------------------------------------------------------------------- title: "Basic configuration" url: https://docs.nginx.com/nginx-ingress-controller/lts/configuration/ingress-resources/basic-configuration/index.md -------------------------------------------------------------------------------- This document shows a basic Ingress resource definition for F5 NGINX Ingress Controller LTS. It load balances requests for two services as part of a single application. Here is a breakdown of what this Ingress resource definition means: - The `metadata.name` field defines the name of the resource `cafe‑ingress`. - The `spec.tls` field sets up SSL/TLS termination: - The `hosts` field applies the certificate and key to the `cafe.example.com` host. - The `secretName` references a secret resource by its name, `cafe‑secret`. The secret must belong to the same namespace as the Ingress, of the type ``kubernetes.io/tls`` and contain keys named ``tls.crt`` and ``tls.key`` that hold the certificate and private key as described [here](https://kubernetes.io/docs/concepts/services-networking/ingress/#tls>). If the secret doesn't exist or is invalid, NGINX will break any attempt to establish a TLS connection to the hosts to which the secret is applied. - The `spec.rules` field defines a host with the domain name `cafe.example.com`. - The `paths` field defines two path‑based rules: - The rule with the path `/tea` instructs NGINX to distribute the requests with the `/tea` URI among the pods of the *tea* service, which is deployed with the name `tea‑svc` in the cluster. - The rule with the path `/coffee` instructs NGINX to distribute the requests with the `/coffee` URI among the pods of the *coffee* service, which is deployed with the name `coffee‑svc` in the cluster. - Both rules instruct NGINX to distribute the requests to `port 80` of the corresponding service (the `servicePort` field). To learn more about the Ingress resource, view [the official Kubernetes documentation for Ingress resources](https://kubernetes.io/docs/concepts/services-networking/ingress/). **note:** For complete instructions on deploying Ingress and Secret resources in the cluster, see the [complete example](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/complete-example) in the GitHub repository. ## New features available in Kubernetes 1.18 Starting from Kubernetes 1.18, you can use the following new features: - The host field supports wildcard domain names, such as `*.example.com`. - The path supports different matching rules with the new field `pathType`, which takes the following values: `Prefix` for prefix-based matching, `Exact` for exact matching and `ImplementationSpecific`, which is the default type and is the same as `Prefix`. For example: ```yaml {hl_lines=[2, 7, 14]} - path: /tea pathType: Prefix backend: serviceName: tea-svc servicePort: 80 - path: /tea/green pathType: Exact backend: service: name: tea-svc port: number: 80 - path: /coffee pathType: ImplementationSpecific backend: service: name: coffee-svc port: number: 80 ``` - The `ingressClassName` field is now supported: ```yaml {hl_lines=[6]} apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: cafe-ingress spec: ingressClassName: nginx tls: - hosts: - cafe.example.com secretName: cafe-secret rules: - host: cafe.example.com . . . ``` When using this field you need to create the `IngressClass` resource with the corresponding `name`. View the [Create common resources](/nic/lts/install/manifests.md#create-common-resources) section of the Installation with Manifests topic for more information. ## Restrictions NGINX Ingress Controller LTS imposes the following restrictions on Ingress resources: - When defining an Ingress resource, the `host` field is required. - The `host` value needs to be unique among all Ingress and VirtualServer resources unless the Ingress resource is a [mergeable minion](/nic/lts/configuration/ingress-resources/cross-namespace-configuration.md). View the [Host and Listener collisions](/nic/lts/configuration/host-and-listener-collisions.md) topic for more information. - The `path` field in `spec.rules[].http.paths[]` is required for `Exact` and `Prefix` `pathTypes`. - The ImplementationSpecific `pathType` is treated as equivalent to `Prefix` `pathType`, with the exception that when this `pathType` is configured, the `path` field in `spec.rules[].http.paths[]` is not mandatory. `path` defaults to `/` if not set but the `pathType` is set to ImplementationSpecific. ## Advanced configuration NGINX Ingress Controller LTS generates NGINX configuration by executing a template file that contains the configuration options. These options are set with the Ingress resource and NGINX Ingress Controller LTS's ConfigMap. The Ingress resource only allows you to use basic NGINX features: host and path-based routing and TLS termination. For advanced configuration, you have two options: - [Annotations](/nic/lts/configuration/ingress-resources/advanced-configuration-with-annotations.md) can be used to rewrite request URIs or inserting additional response headers. - [Snippets](/nic/lts/configuration/ingress-resources/advanced-configuration-with-snippets) can be used to insert raw NGINX configuration, changing generated files. Additionally, it is possible to customize the template, described in the [Custom templates](/nic/lts/configuration/global-configuration/custom-templates.md) topic. -------------------------------------------------------------------------------- title: "Before you begin" url: https://docs.nginx.com/nginx-ingress-controller/configuration/policy-resource/before-you-begin/index.md -------------------------------------------------------------------------------- Use a Policy resource to define reusable configuration, such as access control, CORS, egress mTLS, and WAF. Attach that configuration to [VirtualServer and VirtualServerRoute resources](/nic/configuration/virtualserver-and-virtualserverroute-resources.md) or [Ingress resources](/nic/configuration/ingress-resources). A Policy resource is a [Custom Resource](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/). A Policy resource works together with [VirtualServer and VirtualServerRoute resources](/nic/configuration/virtualserver-and-virtualserverroute-resources.md) and [Ingress resources](/nic/configuration/ingress-resources). Create these resources separately before you attach a policy to them. ## What's next Learn about the [policy specification](/nic/configuration/policy-resource/policy-specification.md) and the supported policy types. -------------------------------------------------------------------------------- title: "Before you begin" url: https://docs.nginx.com/nginx-ingress-controller/lts/configuration/policy-resource/before-you-begin/index.md -------------------------------------------------------------------------------- Use a Policy resource to configure features like access control and rate limiting. Add this configuration to your [VirtualServer, VirtualServerRoute resources](/nic/lts/configuration/virtualserver-and-virtualserverroute-resources.md) and [Ingress resources](/nic/lts/configuration/ingress-resources). A Policy resource is a [Custom Resource](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/). This section is reference documentation for the Policy resource. For an example Policy for access control, see the [GitHub repository](https://github.com/nginx/kubernetes-ingress/blob/v/examples/custom-resources/access-control). A Policy resource works together with [VirtualServer, VirtualServerRoute resources](/nic/lts/configuration/virtualserver-and-virtualserverroute-resources.md) and [Ingress resources](/nic/lts/configuration/ingress-resources). Create these resources separately. ## What's next Learn about the [policy specification](/nic/lts/configuration/policy-resource/policy-specification.md). -------------------------------------------------------------------------------- title: "Build NGINX Ingress Controller with F5 DoS for NGINX" url: https://docs.nginx.com/nginx-ingress-controller/integrations/app-protect-dos/installation/index.md -------------------------------------------------------------------------------- This document explains how to build an image for F5 NGINX Ingress Controller with F5 DoS for NGINX from source code. **Note:** If you'd rather not build your own NGINX Ingress Controller image, see the [pre-built image options](#pre-built-images) at the end of this guide. ## Before you start - To use F5 DoS for NGINX with NGINX Ingress Controller, you must have NGINX Plus. ## Prepare the environment {#prepare-environment} Get your system ready for building and pushing the NGINX Ingress Controller image with F5 DoS for NGINX. 1. Sign in to your private registry. Replace `` with the path to your own private registry. ```shell docker login ``` 2. Clone the NGINX Ingress Controller GitHub repository. Replace `` with the version of NGINX Ingress Controller you want. ```shell git clone https://github.com/nginx/kubernetes-ingress.git --branch cd kubernetes-ingress ``` For instance if you want to clone version v, the commands to run would be: ```shell git clone https://github.com/nginx/kubernetes-ingress.git --branch v cd kubernetes-ingress/deployments ``` ## Build the image {#build-docker-image} Follow these steps to build the NGINX Controller Image with F5 DoS for NGINX. 1. Place your NGINX Plus license files (_nginx-repo.crt_ and _nginx-repo.key_) in the project's root folder. To verify they're in place, run: ```shell ls nginx-repo.* ``` You should see: ```shell nginx-repo.crt nginx-repo.key ``` 2. Build the image. Replace `` with your chosen build option and `` with your private registry's path. Refer to the [Makefile targets](#makefile-targets) table below for the list of build options. ```shell make PREFIX=/nginx-plus-ingress TARGET=download ``` For example, to build a Debian-based image with NGINX Plus and F5 DoS for NGINX, run: ```shell make debian-image-dos-plus PREFIX=/nginx-plus-ingress TARGET=download ``` **What to expect**: The image is built and tagged with a version number, which is derived from the `VERSION` variable in the [_Makefile_](/nic/install/build.md#makefile-details). This version number is used for tracking and deployment purposes. **Note:** In the event a patch version of NGINX Plus is released, make sure to rebuild your image to get the latest version. If your system is caching the Docker layers and not updating the packages, add `DOCKER_BUILD_OPTIONS="--pull --no-cache"` to the make command. ### Makefile targets | Makefile Target | Description | Compatible Systems | |---------------------------|-------------------------------------------------------------------|---------------------| | **debian-image-dos-plus** | Builds a Debian-based image with NGINX Plus and the [F5 DoS for NGINX](/nginx-app-protect-dos/) module. | Debian | | **debian-image-nap-dos-plus** | Builds a Debian-based image with NGINX Plus, [F5 DoS for NGINX](/nginx-app-protect-dos/), and [F5 WAF for NGINX](/nginx-app-protect/). | Debian | | **ubi-image-dos-plus** | Builds a UBI-based image with NGINX Plus and the [F5 DoS for NGINX](/nginx-app-protect-dos/) module. | OpenShift | | **ubi-image-nap-dos-plus** | Builds a UBI-based image with NGINX Plus, [F5 DoS for NGINX](/nginx-app-protect-dos/), and [F5 WAF for NGINX](/nginx-app-protect/). | OpenShift | **Note:** For the complete list of _Makefile_ targets and customizable variables, see the [Build NGINX Ingress Controller](/nic/install/build.md#makefile-details) topic. ## Push the image to your private registry Once you've successfully built the NGINX Ingress Controller image with F5 DoS for NGINX, the next step is to upload it to your private Docker registry. This makes the image available for deployment to your Kubernetes cluster. To upload the image, run the following command. If you're using a custom tag, add `TAG=your-tag` to the end of the command. Replace `` with your private registry's path. ```shell make push PREFIX=/nginx-plus-ingress ``` ## Set up role-based access control (RBAC) {#set-up-rbac} **Note:** To complete these steps you need admin access to your cluster. Refer to your Kubernetes platform's documentation to set up admin access. For Google Kubernetes Engine (GKE), you can refer to their [Role-Based Access Control guide](https://cloud.google.com/kubernetes-engine/docs/how-to/role-based-access-control). 1. Create a namespace and a service account: ```shell kubectl apply -f deployments/common/ns-and-sa.yaml ``` 2. Create a cluster role and binding for the service account: ```shell kubectl apply -f deployments/rbac/rbac.yaml ```
If you're planning to use F5 WAF for NGINX or F5 DoS for NGINX, additional roles and bindings are needed. 1. (F5 WAF for NGINX only) Create the *App Protect* role and binding: ```shell kubectl apply -f deployments/rbac/ap-rbac.yaml ``` 2. (F5 DoS for NGINX only) Create the *App Protect DoS* role and binding: ```shell kubectl apply -f deployments/rbac/apdos-rbac.yaml ``` ## Create common resources {#create-common-resources} In this section, you'll create resources that most NGINX Ingress Controller installations require: 1. (Optional) Create a secret for the default NGINX server's TLS certificate and key. Complete this step only if you're using the [default server TLS secret](/nic/configuration/global-configuration/command-line-arguments#cmdoption-default-server-tls-secret.md) command-line argument. If you're not, feel free to skip this step. By default, the server returns a _404 Not Found_ page for all requests when no ingress rules are set up. Although we provide a self-signed certificate and key for testing purposes, we recommend using your own certificate. ```shell make secrets kubectl apply -f examples/shared-examples/default-server-secret/default-server-secret.yaml ``` 2. Create a ConfigMap to customize your NGINX settings: ```shell kubectl apply -f deployments/common/nginx-config.yaml ``` 3. Create an `IngressClass` resource. NGINX Ingress Controller won't start without an `IngressClass` resource. ```shell kubectl apply -f deployments/common/ingress-class.yaml ``` If you want to make this NGINX Ingress Controller instance your cluster's default, uncomment the `ingressclass.kubernetes.io/is-default-class` annotation. This action will auto-assign `IngressClass` to new ingresses that don't specify an `ingressClassName`. ## Create custom resources {#create-custom-resources} To make sure your NGINX Ingress Controller pods reach the `Ready` state, you'll need to create custom resource definitions (CRDs) for various components. Alternatively, you can disable this requirement by setting the `-enable-custom-resources` command-line argument to `false`. There are two ways you can install the custom resource definitions: 1. Using a URL to apply a single CRD yaml file, which we recommend. 1. Applying your local copy of the CRD yaml files, which requires you to clone the repository. The core custom CRDs are the following: - [VirtualServer and VirtualServerRoute](/nic/configuration/virtualserver-and-virtualserverroute-resources.md) - [TransportServer](/nic/configuration/transportserver-resource.md) - [Policy](/nic/configuration/policy-resource/before-you-begin.md) - [GlobalConfiguration](/nic/configuration/global-configuration/globalconfiguration-resource.md) #### Install CRDs from single YAML ```shell kubectl apply -f https://raw.githubusercontent.com/nginx/kubernetes-ingress/v/deploy/crds.yaml ``` #### Install CRDs after cloning the repo **Note:** Read the steps outlined in [Upgrade from 3.x to 4.x](/nic/install/upgrade.md#upgrade-from-3x-to-4x) before running the CRD upgrade and perform the steps if applicable. ```shell kubectl apply -f config/crd/bases/k8s.nginx.org_virtualservers.yaml kubectl apply -f config/crd/bases/k8s.nginx.org_virtualserverroutes.yaml kubectl apply -f config/crd/bases/k8s.nginx.org_transportservers.yaml kubectl apply -f config/crd/bases/k8s.nginx.org_policies.yaml kubectl apply -f config/crd/bases/k8s.nginx.org_globalconfigurations.yaml ``` ## Create App Protect DoS custom resources #### Install CRDs from single YAML This single YAML file creates CRDs for the following resources: - `APDosPolicy` - `APDosLogConf` - `DosProtectedResource` ```shell kubectl apply -f https://raw.githubusercontent.com/nginx/kubernetes-ingress/v/deploy/crds-nap-dos.yaml ``` #### Install CRDs after cloning the repo These YAML files create CRDs for the following resources: - `APDosPolicy` - `APDosLogConf` - `DosProtectedResource` ```shell kubectl apply -f config/crd/bases/appprotectdos.f5.com_apdoslogconfs.yaml kubectl apply -f config/crd/bases/appprotectdos.f5.com_apdospolicy.yaml kubectl apply -f config/crd/bases/appprotectdos.f5.com_dosprotectedresources.yaml ``` ## Deploy NGINX Ingress Controller {#deploy-ingress-controller} You have two options for deploying NGINX Ingress Controller: - **Deployment**. Choose this method for the flexibility to dynamically change the number of NGINX Ingress Controller replicas. - **DaemonSet**. Choose this method if you want NGINX Ingress Controller to run on all nodes or a subset of nodes. - **StatefulSet**. Choose this method when you need stable, persistent storage and ordered deployment/scaling for your NGINX Ingress Controller pods. Before you start, update the [command-line arguments](/nic/configuration/global-configuration/command-line-arguments.md) for the NGINX Ingress Controller container in the relevant manifest file to meet your specific requirements. ### Using a Deployment For additional context on managing containers using Kubernetes Deployments, refer to the official Kubernetes [Deployments](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) documentation. When you deploy NGINX Ingress Controller as a Deployment, Kubernetes automatically sets up a single NGINX Ingress Controller pod. - For NGINX, run: ```shell kubectl apply -f deployments/deployment/nginx-ingress.yaml ``` - For NGINX Plus, run: ```shell kubectl apply -f deployments/deployment/nginx-plus-ingress.yaml ``` Update the `nginx-plus-ingress.yaml` file to include your chosen image from the F5 Container registry or your custom container image. ### Using a DaemonSet For additional context on managing containers using Kubernetes DaemonSets, refer to the official Kubernetes [DaemonSets](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) documentation. When you deploy NGINX Ingress Controller as a DaemonSet, Kubernetes creates an Ingress Controller pod on every node in the cluster. - For NGINX, run: ```shell kubectl apply -f deployments/daemon-set/nginx-ingress.yaml ``` - For NGINX Plus, run: ```shell kubectl apply -f deployments/daemon-set/nginx-plus-ingress.yaml ``` Update the `nginx-plus-ingress.yaml` file to include your chosen image from the F5 Container registry or your custom container image. ### Using a StatefulSet For additional context on managing containers using Kubernetes StatefulSets, refer to the official Kubernetes [StatefulSets](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/) documentation. When you deploy NGINX Ingress Controller as a StatefulSet, Kubernetes creates pods with stable network identities and persistent storage. - For NGINX, run: ```shell kubectl apply -f deployments/stateful-set/nginx-ingress.yaml ``` - For NGINX Plus, run: ```shell kubectl apply -f deployments/stateful-set/nginx-plus-ingress.yaml ``` Update the `nginx-plus-ingress.yaml` file to include your chosen image from the F5 Container registry or your custom container image. **Note:** StatefulSets include persistent volume claims for nginx cache storage via `volumeClaimTemplates`. You may need to configure a StorageClass in your cluster or modify the volumeClaimTemplates section in the manifest to match your storage requirements. Other volumes (like those needed for App Protect modules) are configured in the regular `volumes` section, not in volumeClaimTemplates. ## Install the App Protect DoS Arbitrator **Note:** If you install multiple NGINX Ingress Controllers in the same namespace, they will need to share the same Arbitrator because there can only be one Arbitrator in a single namespace. ### Helm Chart The App Protect DoS Arbitrator can be installed using the [F5 DoS for NGINX Helm Chart](https://github.com/nginxinc/nap-dos-arbitrator-helm-chart). If you have the NGINX Helm Repository already added, you can install the App Protect DoS Arbitrator by running the following command: ```shell helm install my-release-dos nginx-stable/nginx-appprotect-dos-arbitrator ``` ### YAML Manifests Alternatively, you can install the App Protect DoS Arbitrator using the YAML manifests provided in the NGINX Ingress Controller repo. 1. Create the namespace and service account: ```shell kubectl apply -f common/ns-and-sa.yaml ``` 2. Deploy the F5 WAF for NGINX Arbitrator as a Deployment and service: ```shell kubectl apply -f deployment/appprotect-dos-arb.yaml kubectl apply -f service/appprotect-dos-arb-svc.yaml ``` ## Enable F5 DoS for NGINX module To enable the F5 DoS for NGINX Module: - Add the `enable-app-protect-dos` [command-line argument](/nic/configuration/global-configuration/command-line-arguments.md#cmdoption-enable-app-protect-dos) to your Deployment, DaemonSet, or StatefulSet file. ## Confirm NGINX Ingress Controller is running To confirm the NGINX Ingress Controller pods are operational, run: ```shell kubectl get pods --namespace=nginx-ingress ``` For more information, see the [Configuration guide](/nic/integrations/app-protect-dos/configuration.md),the [NGINX Ingress Controller with App Protect DoS example for VirtualServer](https://github.com/nginx/kubernetes-ingress/tree/v/examples/custom-resources/app-protect-dos) and the [NGINX Ingress Controller with App Protect DoS example for Ingress](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/app-protect-dos). ## Alternatives to building your own image {#pre-built-images} If you prefer not to build your own NGINX Ingress Controller image, you can use pre-built images. Here are your options: - Download the image using your NGINX Ingress Controller subscription certificate and key. View the [Download NGINX Ingress Controller from the F5 Registry](/nic/install/images/registry-download.md) topic. - The [Add an NGINX Ingress Controller image to your cluster](/nic/install/images/add-image-to-cluster.md) topic describes how to use your subscription JWT token to get the image. -------------------------------------------------------------------------------- title: "Build NGINX Ingress Controller with F5 WAF for NGINX" url: https://docs.nginx.com/nginx-ingress-controller/integrations/app-protect-waf-v5/installation/index.md -------------------------------------------------------------------------------- This document explains how to build a F5 NGINX Ingress Controller image with F5 WAF for NGINX v5 from source code. **Note:** If you'd rather not build your own NGINX Ingress Controller image, see the [pre-built image options](#pre-built-images) at the end of this guide. ## Before you begin - To use F5 WAF for NGINX with NGINX Ingress Controller, you must have NGINX Plus. NGINX Ingress Controller supports the following versions of [F5 WAF for NGINX](https://docs.nginx.com/waf/): | NIC Version | NAP-WAF Version | Config Manager | Enforcer | | ------------------- | --------------- | -------------- | -------- | | | 37.1+ | | | | 5.5.4 | 37.0+5.690.0 | 5.14.0 | 5.14.0 | | 5.4.3 | 37.0+5.635.1 | 5.13.1 | 5.13.1 | | 5.3.4 | 35+5.527.0 | 5.11.2 | 5.11.2 | | 5.2.1 | 35+5.527.0 | 5.9.0 | 5.9.0 | | 5.1.1 | 35+5.498 | 5.8.0 | 5.8.0 | | 5.0.0 | 34+5.342 | 5.6.0 | 5.6.0 | | 4.0.1 | 33+5.264 | 5.5.0 | 5.5.0 | | 3.7.2 | 32+5.1 | 5.3.0 | 5.3.0 | | 3.6.2 | 32+5.48 | 5.2.0 | 5.2.0 | ## Prepare the environment Get your system ready for building and pushing the NGINX Ingress Controller image with F5 WAF for NGINX v5. 1. Sign in to your private registry. Replace `` with the path to your own private registry. ```shell docker login ``` 1. Pull the WAF Config Manager image: ```shell docker pull private-registry.nginx.com/nap/waf-config-mgr: ``` 1. Pull the WAF Enforcer Docker image ```shell docker pull private-registry.nginx.com/nap/waf-enforcer: ``` 1. Pull the WAF IP Intelligence image (if you plan to use the WAF IP Intelligence module with your WAF v5 policies): ```shell docker pull private-registry.nginx.com/nap/waf-ip-intelligence: ``` 1. Clone the NGINX Ingress Controller repository: ```console git clone https://github.com/nginx/kubernetes-ingress.git --branch v cd kubernetes-ingress ``` ## Build the image Follow these steps to build the NGINX Controller Image with F5 WAF for NGINX v5. 1. Place your NGINX Plus license files (_nginx-repo.crt_ and _nginx-repo.key_) in the project's root folder. To verify they're in place, run: ```shell ls nginx-repo.* ``` You should see: ```shell nginx-repo.crt nginx-repo.key ``` 2. Build the image. Replace `` with your chosen build option and `` with your private registry's path. Refer to the [Makefile targets](#makefile-targets) table below for the list of build options. ```shell make PREFIX=/nginx-plus-ingress TARGET=download ``` For example, to build a Debian-based image with NGINX Plus and F5 WAF for NGINX v5, run: ```shell make debian-image-nap-v5-plus PREFIX=/nginx-plus-ingress TARGET=download ``` **What to expect**: The image is built and tagged with a version number, which is derived from the `VERSION` variable in the [_Makefile_](/nic/install/build.md#makefile-details). This version number is used for tracking and deployment purposes. **Note:** In the event a patch of NGINX Plus is released, make sure to rebuild your image to get the latest version. If your system is caching the Docker layers and not updating the packages, add `DOCKER_BUILD_OPTIONS="--pull --no-cache"` to the make command. ### Makefile targets {#makefile-targets} Create Docker image for NGINX Ingress Controller (Alpine with NGINX Plus, F5 WAF for NGINX v5 and FIPS) | Makefile Target | Description | Compatible Systems | |---------------------------|-------------------------------------------------------------------|---------------------| | **alpine-image-nap-v5-plus-fips** | Builds a Alpine-based image with NGINX Plus and the [F5 WAF for NGINX v5](/nginx-app-protect-waf/v5/) module with FIPS. | Alpine | | **debian-image-nap-v5-plus** | Builds a Debian-based image with NGINX Plus and the [F5 WAF for NGINX v5](/nginx-app-protect-waf/v5/) module. | Debian | | **ubi-image-nap-v5-plus** | Builds a UBI-based image with NGINX Plus and the [F5 WAF for NGINX v5](/nginx-app-protect-waf/v5/) module. | OpenShift | | **ubi-image-nap-dos-v5-plus** | Builds a UBI-based image with NGINX Plus, [F5 WAF for NGINX v5](/nginx-app-protect-waf/v5/), and [F5 DoS for NGINX](/nginx-app-protect-dos/). | OpenShift | **Note:** For the complete list of _Makefile_ targets and customizable variables, see the [Build NGINX Ingress Controller](/nic/install/build.md#makefile-details) guide. If you intend to use [external references](/nginx-app-protect-waf/v5/configuration-guide/configuration/#external-references) in F5 WAF for NGINX policies, you may want to provide a custom CA certificate to authenticate with the hosting server. To do so, place the `*.crt` file in the build folder and uncomment the lines following this comment: `#Uncomment the lines below if you want to install a custom CA certificate` **Note:** External references are deprecated in NGINX Ingress Controller and will not be supported in future releases. ## Push the images to your private registry Once you've successfully pulled the WAF v5 manager and enforcer images and built the NGINX Ingress Controller image with F5 WAF for NGINX v5, the next step is to upload them to your private Docker registry. This makes the image available for deployment to your Kubernetes cluster. To upload the image, run the following command. If you're using a custom tag, add `TAG=your-tag` to the end of the command. Replace `` with your private registry's path. ```shell make push PREFIX=/nginx-plus-ingress ``` To upload the WAF config manager and enforcer images run the following commands: ```shell docker push /waf-config-mgr: ``` ```shell docker push /waf-enforcer: ``` To make sure your NGINX Ingress Controller pods reach the `Ready` state, you'll need to create custom resource definitions (CRDs) for various components. Alternatively, you can disable this requirement by setting the `-enable-custom-resources` command-line argument to `false`. There are two ways you can install the custom resource definitions: 1. Using a URL to apply a single CRD yaml file, which we recommend. 1. Applying your local copy of the CRD yaml files, which requires you to clone the repository. The core custom CRDs are the following: - [VirtualServer and VirtualServerRoute](/nic/configuration/virtualserver-and-virtualserverroute-resources.md) - [TransportServer](/nic/configuration/transportserver-resource.md) - [Policy](/nic/configuration/policy-resource/before-you-begin.md) - [GlobalConfiguration](/nic/configuration/global-configuration/globalconfiguration-resource.md) #### Install CRDs from single YAML ```shell kubectl apply -f https://raw.githubusercontent.com/nginx/kubernetes-ingress/v/deploy/crds.yaml ``` #### Install CRDs after cloning the repo **Note:** Read the steps outlined in [Upgrade from 3.x to 4.x](/nic/install/upgrade.md#upgrade-from-3x-to-4x) before running the CRD upgrade and perform the steps if applicable. ```shell kubectl apply -f config/crd/bases/k8s.nginx.org_virtualservers.yaml kubectl apply -f config/crd/bases/k8s.nginx.org_virtualserverroutes.yaml kubectl apply -f config/crd/bases/k8s.nginx.org_transportservers.yaml kubectl apply -f config/crd/bases/k8s.nginx.org_policies.yaml kubectl apply -f config/crd/bases/k8s.nginx.org_globalconfigurations.yaml ``` ## Deploy NGINX Ingress Controller {#deploy-ingress-controller} **important:** NGINX Ingress Controller with the AppProtect WAF v5 module works only with policy bundles. You need to modify the Deployment, DaemonSet, or StatefulSet file to include volumes, volume mounts and two WAF 5 docker images: `waf-config-mgr` and `waf-enforcer`. NGINX Ingress Controller **requires** the volume mount path to be `/etc/app_protect/bundles`. #### With Helm Below are examples of a `PersistentVolume` and `PersistentVolumeClaim` that you can reference in your Helm values: ```yaml ... volumes: - name: persistentVolumeClaim: claimName: ... ``` Add volume mounts to the `containers` section: ```yaml ... volumeMounts: - name: mountPath: /etc/app_protect/bundles ... ``` ### Enabling WAF v5 Start by setting `controller.appprotect.enable` to `true` in your Helm values. This will the standard F5 WAF for NGINX features. Afterwords, set `controller.approtect.v5` to `true`. This ensures that both the `waf-enforcer` and `waf-config-mgr` containers are deployed alongside the NGINX Ingress Controller containers. These two additional containers are required when using F5 WAF for NGINX v5. Your Helm values should look something like this: ```yaml controller: ... ## Support for F5 WAF for NGINX appprotect: ## Enable the F5 WAF for NGINX module in the Ingress Controller. enable: true ## Enables F5 WAF for NGINX v5. v5: true ``` ### Configuring volumes Whether you have created a new `PersistentVolume` and `PersistentVolumeClaim`, or you are referencing an existing `PersistentVolumeClaim`, update the `app-protect-bundles` volume to reference your `PersistentVolumeClaim`. Example helm values: ```yaml ... controller: ... appprotect: ... volumes: - name: app-protect-bundles persistentVolumeClaim: claimName: ... ``` **Note:** By default, `emptyDir` mounts are used. Bundles that are added to these kind of volume mounts will **NOT** persist across pod restarts. Example default volumes: ```yaml ... controller: ... appprotect: ... volumes: - name: app-protect-bundles emptyDir: {} ... ``` ### Configuring `readOnlyRootFilesystem` Create required volumes: ```yaml volumes: - name: nginx-etc emptyDir: {} - name: nginx-cache # do not set this value in statefulset if volumeclaimtemplate is set emptyDir: {} # do not set this value in statefulset if volumeclaimtemplate is set - name: nginx-lib emptyDir: {} - name: nginx-log emptyDir: {} - name: app-protect-bd-config emptyDir: {} - name: app-protect-config emptyDir: {} - name: app-protect-bundles emptyDir: {} ``` Set `controller.securityContext.readOnlyRootFilesystem` to `true`. Example Helm values: ```yaml controller: ... securityContext: readOnlyRootFilesystem: true ... ``` Set `controller.appprotect.enforcer.securityContext.readOnlyRootFilesystem` to `true`. Example Helm values: ```yaml controller: ... appprotect: ... enforcer: securityContext: readOnlyRootFilesystem: true ... ``` Set `controller.appprotect.configManager.securityContext.readOnlyRootFilesystem` to `true`. Example Helm values: ```yaml controller: ... appprotect: ... configManager: securityContext: readOnlyRootFilesystem: true ... ``` #### With Manifest You have two options for deploying NGINX Ingress Controller: - **Deployment**. Choose this method for the flexibility to dynamically change the number of NGINX Ingress Controller replicas. - **DaemonSet**. Choose this method if you want NGINX Ingress Controller to run on all nodes or a subset of nodes. - **StatefulSet**. Choose this method when you need stable, persistent storage and ordered deployment/scaling for your NGINX Ingress Controller pods. ### Set up role-based access control (RBAC) {#set-up-rbac} **Note:** To complete these steps you need admin access to your cluster. Refer to your Kubernetes platform's documentation to set up admin access. For Google Kubernetes Engine (GKE), you can refer to their [Role-Based Access Control guide](https://cloud.google.com/kubernetes-engine/docs/how-to/role-based-access-control). 1. Create a namespace and a service account: ```shell kubectl apply -f deployments/common/ns-and-sa.yaml ``` 2. Create a cluster role and binding for the service account: ```shell kubectl apply -f deployments/rbac/rbac.yaml ``` ### Volumes and VolumeMounts Add a `volumes` section to deployment template spec: ```yaml ... volumes: - name: persistentVolumeClaim: claimName: ... ``` Add volume mounts to the `containers` section: ```yaml ... volumeMounts: - name: mountPath: /etc/app_protect/bundles ... ``` ### WAF Config Manager and WAF Enforcer Add `waf-config-mgr` image to the `containers` section: ```yaml ... - name: waf-config-mgr image: private-registry.nginx.com/nap/waf-config-mgr: imagePullPolicy: IfNotPresent securityContext: allowPrivilegeEscalation: false capabilities: drop: - all volumeMounts: - name: app-protect-bd-config mountPath: /opt/app_protect/bd_config - name: app-protect-config mountPath: /opt/app_protect/config - name: app-protect-bundles mountPath: /etc/app_protect/bundles ... ``` Add `waf-enforcer` image to the `containers` section: ```yaml ... - name: waf-enforcer image: private-registry.nginx.com/nap/waf-enforcer: imagePullPolicy: IfNotPresent env: - name: ENFORCER_PORT value: "50000" - name: ENFORCER_CONFIG_TIMEOUT value: "0" volumeMounts: - name: app-protect-bd-config mountPath: /opt/app_protect/bd_config ... ``` ### Update NIC container in deployment, daemonset, or statefulset Add `volumeMounts` as below: ```yaml ... - image: : imagePullPolicy: IfNotPresent name: nginx-plus-ingress volumeMounts: - name: app-protect-bd-config mountPath: /opt/app_protect/bd_config - name: app-protect-config mountPath: /opt/app_protect/config - name: app-protect-bundles mountPath: /etc/app_protect/bundles ... ``` ### Configure `readOnlyRootFilesystem` Add `readOnlyRootFilesystem` to the NIC container and set valut to `true` as below: ```yaml ... - image: : imagePullPolicy: IfNotPresent name: nginx-plus-ingress ... securityContext: allowPrivilegeEscalation: false capabilities: add: - NET_BIND_SERVICE drop: - ALL readOnlyRootFilesystem: true runAsNonRoot: true runAsUser: 101 readOnlyRootFilesystem: true ... volumeMounts: - mountPath: /etc/nginx name: nginx-etc - mountPath: /var/cache/nginx name: nginx-cache - mountPath: /var/lib/nginx name: nginx-lib - mountPath: /var/log/nginx name: nginx-log - mountPath: /opt/app_protect/bd_config name: app-protect-bd-config - mountPath: /opt/app_protect/config name: app-protect-config - mountPath: /etc/app_protect/bundles name: app-protect-bundles ... ``` Add `readOnlyRootFilesystem` to the `waf-config-mgr` container and set value to `true` as below: ```yaml ... - name: waf-config-mgr image: private-registry.nginx.com/nap/waf-config-mgr: imagePullPolicy: IfNotPresent ... securityContext: readOnlyRootFilesystem: true ... ... ``` Add `readOnlyRootFilesystem` to the `waf-enforcer` container and set value to `true` as below: ```yaml ... - name: waf-enforcer image: private-registry.nginx.com/nap/waf-enforcer: imagePullPolicy: IfNotPresent ... securityContext: readOnlyRootFilesystem: true ... ... ``` **Note:** **StatefulSet Volume Configuration**: When using StatefulSet deployments, the `nginx-cache` volume is automatically provided via `volumeClaimTemplates` for persistent storage. F5 WAF for NGINX v5 volumes (like app-protect-config, app-protect-bundles) are still configured as regular volumes in the `volumes` section. Use `emptyDir` for temporary data or PersistentVolumeClaims if you need persistence for App Protect configurations across pod restarts. ### Using a Deployment For additional context on managing containers using Kubernetes Deployments, refer to the official Kubernetes [Deployments](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) documentation. When you deploy NGINX Ingress Controller as a Deployment, Kubernetes automatically sets up a single NGINX Ingress Controller pod. - For NGINX, run: ```shell kubectl apply -f deployments/deployment/nginx-ingress.yaml ``` - For NGINX Plus, run: ```shell kubectl apply -f deployments/deployment/nginx-plus-ingress.yaml ``` Update the `nginx-plus-ingress.yaml` file to include your chosen image from the F5 Container registry or your custom container image. ### Using a DaemonSet For additional context on managing containers using Kubernetes DaemonSets, refer to the official Kubernetes [DaemonSets](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) documentation. When you deploy NGINX Ingress Controller as a DaemonSet, Kubernetes creates an Ingress Controller pod on every node in the cluster. - For NGINX, run: ```shell kubectl apply -f deployments/daemon-set/nginx-ingress.yaml ``` - For NGINX Plus, run: ```shell kubectl apply -f deployments/daemon-set/nginx-plus-ingress.yaml ``` Update the `nginx-plus-ingress.yaml` file to include your chosen image from the F5 Container registry or your custom container image. ### Using a StatefulSet For additional context on managing containers using Kubernetes StatefulSets, refer to the official Kubernetes [StatefulSets](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/) documentation. When you deploy NGINX Ingress Controller as a StatefulSet, Kubernetes creates pods with stable network identities and persistent storage. - For NGINX, run: ```shell kubectl apply -f deployments/stateful-set/nginx-ingress.yaml ``` - For NGINX Plus, run: ```shell kubectl apply -f deployments/stateful-set/nginx-plus-ingress.yaml ``` Update the `nginx-plus-ingress.yaml` file to include your chosen image from the F5 Container registry or your custom container image. **Note:** StatefulSets include persistent volume claims for nginx cache storage via `volumeClaimTemplates`. You may need to configure a StorageClass in your cluster or modify the volumeClaimTemplates section in the manifest to match your storage requirements. Other volumes (like those needed for App Protect modules) are configured in the regular `volumes` section, not in volumeClaimTemplates. ### Enable F5 WAF for NGINX module To enable the F5 DoS for NGINX Module: - Add the `enable-app-protect` [command-line argument](/nic/configuration/global-configuration/command-line-arguments.md#cmdoption-enable-app-protect) to your Deployment, DaemonSet, or StatefulSet file. ## Confirm NGINX Ingress Controller is running To confirm the NGINX Ingress Controller pods are operational, run: ```shell kubectl get pods --namespace=nginx-ingress ``` For more information, see the [Configuration guide](/nic/integrations/app-protect-waf-v5/configuration.md) and the NGINX Ingress Controller with App Protect version 5 example resources on GitHub [for VirtualServer resources](https://github.com/nginx/kubernetes-ingress/tree/v/examples/custom-resources/app-protect-waf-v5). ## Alternatives to building your own image {#pre-built-images} If you prefer not to build your own NGINX Ingress Controller image, you can use pre-built images. Here are your options: - Download the image using your NGINX Ingress Controller subscription certificate and key. View the [Download NGINX Ingress Controller from the F5 Registry](/nic/install/images/registry-download.md) topic. - The [Add an NGINX Ingress Controller image to your cluster](/nic/install/images/add-image-to-cluster.md) topic describes how to use your subscription JWT token to get the image. -------------------------------------------------------------------------------- title: "Build NGINX Ingress Controller with F5 WAF for NGINX" url: https://docs.nginx.com/nginx-ingress-controller/integrations/app-protect-waf/installation/index.md -------------------------------------------------------------------------------- This document explains how to build a F5 NGINX Ingress Controller image with F5 WAF for NGINX from source code. **Note:** If you'd rather not build your own NGINX Ingress Controller image, see the [pre-built image options](#pre-built-images) at the end of this guide. ## Before you start - To use F5 WAF for NGINX with NGINX Ingress Controller, you must have NGINX Plus. - F5 WAF for NGINX package based installation (previously F5 WAF for NGINX v4) is not supported when `readOnlyRootFilesystem` is enabled. ## Prepare the environment Get your system ready for building and pushing the NGINX Ingress Controller image with F5 WAF for NGINX. 1. Sign in to your private registry. Replace `` with the path to your own private registry. ```shell docker login ``` 1. Clone the NGINX Ingress Controller repository: ```shell git clone https://github.com/nginx/kubernetes-ingress.git --branch v cd kubernetes-ingress ``` ## Build the image Follow these steps to build the NGINX Controller Image with F5 WAF for NGINX. 1. Place your NGINX Plus license files (_nginx-repo.crt_ and _nginx-repo.key_) in the project's root folder. To verify they're in place, run: ```shell ls nginx-repo.* ``` You should see: ```shell nginx-repo.crt nginx-repo.key ``` 2. Build the image. Replace `` with your chosen build option and `` with your private registry's path. Refer to the [Makefile targets](#makefile-targets) table below for the list of build options. ```shell make PREFIX=/nginx-plus-ingress TARGET=download ``` For example, to build a Debian-based image with NGINX Plus and F5 DoS for NGINX, run: ```shell make debian-image-dos-plus PREFIX=/nginx-plus-ingress TARGET=download ``` **What to expect**: The image is built and tagged with a version number, which is derived from the `VERSION` variable in the [_Makefile_](/nic/install/build.md#makefile-details). This version number is used for tracking and deployment purposes. **Note:** In the event a patch of NGINX Plus is released, make sure to rebuild your image to get the latest version. If your system is caching the Docker layers and not updating the packages, add `DOCKER_BUILD_OPTIONS="--pull --no-cache"` to the make command. ### Makefile targets {#makefile-targets} | Makefile Target | Description | Compatible Systems | |---------------------------|-------------------------------------------------------------------|---------------------| | **debian-image-nap-plus** | Builds a Debian-based image with NGINX Plus and the [F5 WAF for NGINX](/nginx-app-protect-waf/) module. | Debian | | **debian-image-nap-dos-plus** | Builds a Debian-based image with NGINX Plus, [F5 WAF for NGINX](/nginx-app-protect-waf/), and [F5 DoS for NGINX](/nginx-app-protect-dos/) | Debian | | **ubi-image-nap-plus** | Builds a UBI-based image with NGINX Plus and the [F5 WAF for NGINX](/nginx-app-protect-waf/) module. | OpenShift | | **ubi-image-nap-dos-plus** | Builds a UBNI-based image with NGINX Plus, [F5 WAF for NGINX](/nginx-app-protect-waf/), and [F5 DoS for NGINX](/nginx-app-protect-dos/). | OpenShift | **Note:** For the complete list of _Makefile_ targets and customizable variables, see the [Build NGINX Ingress Controller](/nic/install/build.md#makefile-details) topic. ## Push the image to your private registry Once you've successfully built the NGINX Ingress Controller image with F5 WAF for NGINX, the next step is to upload it to your private Docker registry. This makes the image available for deployment to your Kubernetes cluster. To upload the image, run the following command. If you're using a custom tag, add `TAG=your-tag` to the end of the command. Replace `` with your private registry's path. ```shell make push PREFIX=/nginx-plus-ingress ``` ## Set up role-based access control (RBAC) {#set-up-rbac} **Note:** To complete these steps you need admin access to your cluster. Refer to your Kubernetes platform's documentation to set up admin access. For Google Kubernetes Engine (GKE), you can refer to their [Role-Based Access Control guide](https://cloud.google.com/kubernetes-engine/docs/how-to/role-based-access-control). 1. Create a namespace and a service account: ```shell kubectl apply -f deployments/common/ns-and-sa.yaml ``` 2. Create a cluster role and binding for the service account: ```shell kubectl apply -f deployments/rbac/rbac.yaml ```
If you're planning to use F5 WAF for NGINX or F5 DoS for NGINX, additional roles and bindings are needed. 1. (F5 WAF for NGINX only) Create the *App Protect* role and binding: ```shell kubectl apply -f deployments/rbac/ap-rbac.yaml ``` 2. (F5 DoS for NGINX only) Create the *App Protect DoS* role and binding: ```shell kubectl apply -f deployments/rbac/apdos-rbac.yaml ``` ## Create common resources {#create-common-resources} In this section, you'll create resources that most NGINX Ingress Controller installations require: 1. (Optional) Create a secret for the default NGINX server's TLS certificate and key. Complete this step only if you're using the [default server TLS secret](/nic/configuration/global-configuration/command-line-arguments#cmdoption-default-server-tls-secret.md) command-line argument. If you're not, feel free to skip this step. By default, the server returns a _404 Not Found_ page for all requests when no ingress rules are set up. Although we provide a self-signed certificate and key for testing purposes, we recommend using your own certificate. ```shell make secrets kubectl apply -f examples/shared-examples/default-server-secret/default-server-secret.yaml ``` 2. Create a ConfigMap to customize your NGINX settings: ```shell kubectl apply -f deployments/common/nginx-config.yaml ``` 3. Create an `IngressClass` resource. NGINX Ingress Controller won't start without an `IngressClass` resource. ```shell kubectl apply -f deployments/common/ingress-class.yaml ``` If you want to make this NGINX Ingress Controller instance your cluster's default, uncomment the `ingressclass.kubernetes.io/is-default-class` annotation. This action will auto-assign `IngressClass` to new ingresses that don't specify an `ingressClassName`. ## Create core custom resources {#create-custom-resources} To make sure your NGINX Ingress Controller pods reach the `Ready` state, you'll need to create custom resource definitions (CRDs) for various components. Alternatively, you can disable this requirement by setting the `-enable-custom-resources` command-line argument to `false`. There are two ways you can install the custom resource definitions: 1. Using a URL to apply a single CRD yaml file, which we recommend. 1. Applying your local copy of the CRD yaml files, which requires you to clone the repository. The core custom CRDs are the following: - [VirtualServer and VirtualServerRoute](/nic/configuration/virtualserver-and-virtualserverroute-resources.md) - [TransportServer](/nic/configuration/transportserver-resource.md) - [Policy](/nic/configuration/policy-resource/before-you-begin.md) - [GlobalConfiguration](/nic/configuration/global-configuration/globalconfiguration-resource.md) #### Install CRDs from single YAML ```shell kubectl apply -f https://raw.githubusercontent.com/nginx/kubernetes-ingress/v/deploy/crds.yaml ``` #### Install CRDs after cloning the repo **Note:** Read the steps outlined in [Upgrade from 3.x to 4.x](/nic/install/upgrade.md#upgrade-from-3x-to-4x) before running the CRD upgrade and perform the steps if applicable. ```shell kubectl apply -f config/crd/bases/k8s.nginx.org_virtualservers.yaml kubectl apply -f config/crd/bases/k8s.nginx.org_virtualserverroutes.yaml kubectl apply -f config/crd/bases/k8s.nginx.org_transportservers.yaml kubectl apply -f config/crd/bases/k8s.nginx.org_policies.yaml kubectl apply -f config/crd/bases/k8s.nginx.org_globalconfigurations.yaml ``` ## Create F5 WAF for NGINX custom resources **Note:** If you're using NGINX Ingress Controller with the F5 WAF for NGINX module and policy bundles, you can skip this section. You will need to create and configure [Persistent Volume and Persistent Volume Claim](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) in your Kubernetes cluster. #### Install CRDs from single YAML This single YAML file creates CRDs for the following resources: - `APPolicy` - `APLogConf` - `APUserSig` ```shell kubectl apply -f https://raw.githubusercontent.com/nginx/kubernetes-ingress/v/deploy/crds-nap-waf.yaml ``` #### Install CRDs after cloning the repo **Note:** If you are installing the CRDs this way, ensure you have first cloned the repository. These YAML files create CRDs for the following resources: - `APPolicy` - `APLogConf` - `APUserSig` ```shell kubectl apply -f config/crd/bases/appprotect.f5.com_appolicies.yaml kubectl apply -f config/crd/bases/appprotect.f5.com_aplogconfs.yaml kubectl apply -f config/crd/bases/appprotect.f5.com_apusersigs.yaml ``` ## Deploy NGINX Ingress Controller {#deploy-ingress-controller} You have two options for deploying NGINX Ingress Controller: - **Deployment**. Choose this method for the flexibility to dynamically change the number of NGINX Ingress Controller replicas. - **DaemonSet**. Choose this method if you want NGINX Ingress Controller to run on all nodes or a subset of nodes. - **StatefulSet**. Choose this method when you need stable, persistent storage and ordered deployment/scaling for your NGINX Ingress Controller pods. Before you start, update the [command-line arguments](/nic/configuration/global-configuration/command-line-arguments.md) for the NGINX Ingress Controller container in the relevant manifest file to meet your specific requirements. **Note:** If you're using NGINX Ingress Controller with the AppProtect WAF module and policy bundles, you will need to modify the Deployment, DaemonSet, or StatefulSet file to include volumes and volume mounts. NGINX Ingress Controller **requires** the volume mount path to be `/etc/nginx/waf/bundles`. Add a `volumes` section to deployment template spec: ```yaml ... volumes: - name: persistentVolumeClaim: claimName: ... ``` Add volume mounts to the `containers` section: ```yaml ... volumeMounts: - name: mountPath: /etc/nginx/waf/bundles ... ``` **Note:** **StatefulSet Volume Configuration**: When using StatefulSet deployments, the `nginx-cache` volume is automatically provided via `volumeClaimTemplates` for persistent storage. F5 WAF for NGINX v5 volumes (like app-protect-config, app-protect-bundles) are still configured as regular volumes in the `volumes` section. Use `emptyDir` for temporary data or PersistentVolumeClaims if you need persistence for App Protect configurations across pod restarts. ### Using a Deployment For additional context on managing containers using Kubernetes Deployments, refer to the official Kubernetes [Deployments](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) documentation. When you deploy NGINX Ingress Controller as a Deployment, Kubernetes automatically sets up a single NGINX Ingress Controller pod. - For NGINX, run: ```shell kubectl apply -f deployments/deployment/nginx-ingress.yaml ``` - For NGINX Plus, run: ```shell kubectl apply -f deployments/deployment/nginx-plus-ingress.yaml ``` Update the `nginx-plus-ingress.yaml` file to include your chosen image from the F5 Container registry or your custom container image. ### Using a DaemonSet For additional context on managing containers using Kubernetes DaemonSets, refer to the official Kubernetes [DaemonSets](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) documentation. When you deploy NGINX Ingress Controller as a DaemonSet, Kubernetes creates an Ingress Controller pod on every node in the cluster. - For NGINX, run: ```shell kubectl apply -f deployments/daemon-set/nginx-ingress.yaml ``` - For NGINX Plus, run: ```shell kubectl apply -f deployments/daemon-set/nginx-plus-ingress.yaml ``` Update the `nginx-plus-ingress.yaml` file to include your chosen image from the F5 Container registry or your custom container image. ### Using a StatefulSet For additional context on managing containers using Kubernetes StatefulSets, refer to the official Kubernetes [StatefulSets](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/) documentation. When you deploy NGINX Ingress Controller as a StatefulSet, Kubernetes creates pods with stable network identities and persistent storage. - For NGINX, run: ```shell kubectl apply -f deployments/stateful-set/nginx-ingress.yaml ``` - For NGINX Plus, run: ```shell kubectl apply -f deployments/stateful-set/nginx-plus-ingress.yaml ``` Update the `nginx-plus-ingress.yaml` file to include your chosen image from the F5 Container registry or your custom container image. **Note:** StatefulSets include persistent volume claims for nginx cache storage via `volumeClaimTemplates`. You may need to configure a StorageClass in your cluster or modify the volumeClaimTemplates section in the manifest to match your storage requirements. Other volumes (like those needed for App Protect modules) are configured in the regular `volumes` section, not in volumeClaimTemplates. ## Enable F5 WAF for NGINX module (requires NGINX Plus) To enable the F5 WAF for NGINX Module: - Add the `--enable-app-protect` [command-line argument](/nic/configuration/global-configuration/command-line-arguments.md#cmdoption-enable-app-protect) to your Deployment, DaemonSet, or StatefulSet file. Additional WAF modules can be enabled by adding the appropriate command-line arguments: - To enable F5 WAF DoS Protection, add the `--enable-app-protect-dos` [command-line argument](/nic/configuration/global-configuration/command-line-arguments.md#cmdoption-enable-app-protect-dos). - To enable F5 WAF IP Intelligence, add the `--enable-app-protect-ip-intelligence` [command-line argument](/nic/configuration/global-configuration/command-line-arguments.md#cmdoption-enable-app-protect-ip-intelligence). ## Confirm NGINX Ingress Controller is running To confirm the NGINX Ingress Controller pods are operational, run: ```shell kubectl get pods --namespace=nginx-ingress ``` For more information, see the [Configuration guide](/nic/integrations/app-protect-waf/configuration.md) and the NGINX Ingress Controller with App Protect example resources on GitHub [for VirtualServer resources](https://github.com/nginx/kubernetes-ingress/tree/v/examples/custom-resources/app-protect-waf) and [for Ingress resources](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/app-protect-waf). ## Alternatives to building your own image {#pre-built-images} If you prefer not to build your own NGINX Ingress Controller image, you can use pre-built images. Here are your options: - Download the image using your NGINX Ingress Controller subscription certificate and key. View the [Download NGINX Ingress Controller from the F5 Registry](/nic/install/images/registry-download.md) topic. - The [Add an NGINX Ingress Controller image to your cluster](/nic/install/images/add-image-to-cluster.md) topic describes how to use your subscription JWT token to get the image. -------------------------------------------------------------------------------- title: "Command-line arguments" url: https://docs.nginx.com/nginx-ingress-controller/configuration/global-configuration/command-line-arguments/index.md -------------------------------------------------------------------------------- F5 NGINX Ingress Controller supports several command-line arguments, which are set based on installation method: - If you're using *Kubernetes Manifests* to install NGINX Ingress Controller, modify the Manifests to set the command-line arguments. View the [Installation with Manifests](/nic/install/manifests.md) topic for more information. - If you're using *Helm* to install NGINX Ingress Controller, modify the parameters of the Helm chart to set the command-line arguments. View the [Installation with Helm](/nic/install/helm.md) topic for more information. ### -enable-snippets Enable custom NGINX configuration snippets in Ingress, VirtualServer, VirtualServerRoute and TransportServer resources. Default `false`. ### -default-server-tls-secret `` Secret with a TLS certificate and key for TLS termination of the default server. - If not set, certificate and key in the file `/etc/nginx/secrets/default` are used. - If `/etc/nginx/secrets/default` doesn't exist, NGINX Ingress Controller will configure NGINX to reject TLS connections to the default server. - If a secret is set, but NGINX Ingress Controller is not able to fetch it from Kubernetes API, or it is not set and NGINX Ingress Controller fails to read the file "/etc/nginx/secrets/default", NGINX Ingress Controller will fail to start. Format: `/` When `-allow-empty-ingress-host` is enabled, this secret also provides TLS for Ingress resources without a `host`. ### -wildcard-tls-secret `` A Secret with a TLS certificate and key for TLS termination of every Ingress/VirtualServer host for which TLS termination is enabled but the Secret is not specified. - If the argument is not set, for such Ingress/VirtualServer hosts NGINX will break any attempt to establish a TLS connection - If the argument is set, but NGINX Ingress Controller is not able to fetch the Secret from Kubernetes API, NGINX Ingress Controller will fail to start. Format: `/` ### -enable-custom-resources Enables custom resources. Default `true`. ### -enable-oidc Enables OIDC policies. Default `false`. ### -enable-leader-election Enables Leader election to avoid multiple replicas of the controller reporting the status of Ingress, VirtualServer and VirtualServerRoute resources -- only one replica will report status. Default `true`. See [-report-ingress-status](#cmdoption-report-ingress-status) flag. ### -enable-tls-passthrough Enable TLS Passthrough on port 443. Requires [-enable-custom-resources](#cmdoption-enable-custom-resources). ### -tls-passthrough-port `` Set the port for TLS Passthrough. Format: `[1024 - 65535]` (default `443`) Requires [-enable-custom-resources](#cmdoption-enable-custom-resources). ### -enable-cert-manager Enable x509 automated certificate management for VirtualServer resources using cert-manager (cert-manager.io). Requires [-enable-custom-resources](#cmdoption-enable-custom-resources). ### -enable-config-safety **Note:** This is an experimental feature. Behavior and configuration may change in future releases. When this feature is turned on, users can experience delayed pod startup times as resources are validated prior to being written. Delays are directly proportional to the number and complexity of resources. The `-enable-config-safety` flag enhances the stability and reliability of the NGINX Ingress Controller. When turned on, this feature ensures the validation of new configurations before reloading nginx. If a configuration is deemed invalid, the new config for the associated resource is rejected and the old working config is restored. By Default it is set to `false`. ### -enable-external-dns Enable integration with ExternalDNS for configuring public DNS entries for VirtualServer resources using [ExternalDNS](https://github.com/kubernetes-sigs/external-dns). Requires [-enable-custom-resources](#cmdoption-enable-custom-resources). ### -external-service `` Specifies the name of the service with the type LoadBalancer through which the NGINX Ingress Controller pods are exposed externally. The external address of the service is used when reporting the status of Ingress, VirtualServer and VirtualServerRoute resources. For Ingress resources only: Requires [-report-ingress-status](#cmdoption-report-ingress-status). ### -ingresslink `` Specifies the name of the IngressLink resource, which exposes the NGINX Ingress Controller pods via a BIG-IP system. The IP of the BIG-IP system is used when reporting the status of Ingress, VirtualServer and VirtualServerRoute resources. For Ingress resources only: Requires [-report-ingress-status](#cmdoption-report-ingress-status). ### -global-configuration `` A GlobalConfiguration resource for global configuration of NGINX Ingress Controller. Format: `/` Requires [-enable-custom-resources](#cmdoption-enable-custom-resources). ### -health-status Adds a location "/nginx-health" to the default server. The location responds with the 200 status code for any request. Useful for external health-checking of NGINX Ingress Controller. ### -health-status-uri `` Sets the URI of health status location in the default server. Requires [-health-status](#cmdoption-health-status). (default `/nginx-health`) ### -ingress-class `` The `-ingress-class` argument refers to the name of the resource `kind: IngressClass`. An IngressClass resource with a name equal to the class must be deployed. Otherwise, NGINX Ingress Controller will fail to start. NGINX Ingress Controller will only process Ingress resources that belong to its class (Whose `ingressClassName` value matches the value of `-ingress-class`), skipping the ones without it. It will also process all the VirtualServer/VirtualServerRoute/TransportServer resources that do not have the `ingressClassName` field. Default `nginx`. ### -ingress-template-path `` Path to the ingress NGINX configuration template for an ingress resource. Default for NGINX is `nginx.ingress.tmpl`; default for NGINX Plus is `nginx-plus.ingress.tmpl`. ### -allow-empty-ingress-host {#cmdoption-allow-empty-ingress-host} Allows Ingress resources to omit the `host` field. Default `false`. If multiple Ingress resources without a `host` conflict, NGINX Ingress Controller resolves the collision using the [winner selection algorithm](/nic/configuration/host-and-listener-collisions.md). To use multiple Ingress resources without a `host` across namespaces without conflict, use [mergeable Ingress](/nic/configuration/ingress-resources/cross-namespace-configuration.md). To configure TLS, use the command-line argument [`-default-server-tls-secret`](#cmdoption-default-server-tls-secret). To configure listener ports, use the command-line arguments [`-default-http-listener-port`](#cmdoption-default-http-listener-port) or [`-default-https-listener-port`](#cmdoption-default-https-listener-port). ### -leader-election-lock-name `` Specifies the name of the ConfigMap, within the same namespace as the controller, used as the lock for leader election. Requires [-enable-leader-election](#cmdoption-enable-leader-election). ### -log_backtrace_at `` When logging hits line `file:N`, emit a stack trace. ### -main-template-path `` Path to the main NGINX configuration template. - Default for NGINX is `nginx.tmpl`. - Default for NGINX Plus is `nginx-plus.tmpl`. ### -nginx-configmaps `` A ConfigMap resource for customizing NGINX configuration. If a ConfigMap is set, but NGINX Ingress Controller is not able to fetch it from Kubernetes API, NGINX Ingress Controller will fail to start. Format: `/` ### -mgmt-configmap `` The Management ConfigMap resource is used for customizing the NGINX mgmt block. If using NGINX Plus, a Management ConfigMap must be set. If NGINX Ingress Controller is not able to fetch it from Kubernetes API, NGINX Ingress Controller will fail to start. Format: `/` ### -nginx-debug Enable debugging for NGINX. Uses the nginx-debug binary. Requires 'error-log-level: debug' in the ConfigMap. ### -nginx-plus Enable support for NGINX Plus. ### -nginx-reload-timeout `` Timeout in milliseconds which NGINX Ingress Controller will wait for a successful NGINX reload after a change or at the initial start. Default is 60000. ### -nginx-status Enable the NGINX stub_status, or the NGINX Plus API. Default `true`. ### -nginx-status-allow-cidrs `` Add IP/CIDR blocks to the allow list for NGINX stub_status or the NGINX Plus API. Separate multiple IP/CIDR by commas. (default `127.0.0.1,::1`) ### -nginx-status-port `` Set the port where the NGINX stub_status or the NGINX Plus API is exposed. Format: `[1024 - 65535]` (default `8080`) ### -proxy `` **Note:** This argument is intended for testing purposes only. Use a proxy server to connect to Kubernetes API started with `kubectl proxy`. NGINX Ingress Controller does not start NGINX and does not write any generated NGINX configuration files to disk. ### -report-ingress-status Updates the address field in the status of Ingress resources. Requires the [-external-service](#cmdoption-external-service) or [-ingresslink](#cmdoption-ingresslink) flag, or the `external-status-address` key in the ConfigMap. ### -transportserver-template-path `` Path to the TransportServer NGINX configuration template for a TransportServer resource. - Default for NGINX is `nginx.transportserver.tmpl`. - Default for NGINX Plus is `nginx-plus.transportserver.tmpl`. ### -log-level `` Log level for Ingress Controller logs. Allowed values: fatal, error, warn, info, debug, trace. - Default is `info`. ### -log-format `` Log format for Ingress Controller logs. Allowed values: glog, json, text. - Default is `glog`. ### -version Print the version, git-commit hash and build date and exit. ### -virtualserver-template-path `` Path to the VirtualServer NGINX configuration template for a VirtualServer resource. - Default for NGINX is `nginx.virtualserver.tmpl`. - Default for NGINX Plus is `nginx-plus.virtualserver.tmpl`. ### -vmodule `` A comma-separated list of pattern=N settings for file-filtered logging. ### -watch-namespace `` Comma separated list of namespaces NGINX Ingress Controller should watch for resources. By default NGINX Ingress Controller watches all namespaces. Mutually exclusive with "watch-namespace-label". ### -watch-namespace-label `` Configures NGINX Ingress Controller to watch only those namespaces with label foo=bar. By default NGINX Ingress Controller watches all namespaces. Mutually exclusive with "watch-namespace". ### -watch-secret-namespace `` Comma separated list of namespaces NGINX Ingress Controller should watch for secrets. If this arg is not configured, NGINX Ingress Controller watches the same namespaces for all resources, see "watch-namespace" and "watch-namespace-label". All namespaces included with this argument must be part of either `-watch-namespace` or `-watch-namespace-label`. ### -enable-prometheus-metrics Enables exposing NGINX or NGINX Plus metrics in the Prometheus format. ### -prometheus-metrics-listen-port `` Sets the port where the Prometheus metrics are exposed. Format: `[1024 - 65535]` (default `9113`) ### -prometheus-tls-secret `` A Secret with a TLS certificate and key for TLS termination of the Prometheus metrics endpoint. - If the argument is not set, the Prometheus endpoint will not use a TLS connection. - If the argument is set, but NGINX Ingress Controller is not able to fetch the Secret from Kubernetes API, NGINX Ingress Controller will fail to start. ### -enable-service-insight Exposes the Service Insight endpoint for Ingress Controller. ### -service-insight-listen-port `` Sets the port where the Service Insight is exposed. Format: `[1024 - 65535]` (default `9114`) ### -service-insight-tls-secret `` A Secret with a TLS certificate and key for TLS termination of the Service Insight endpoint. - If the argument is not set, the Service Insight endpoint will not use a TLS connection. - If the argument is set, but NGINX Ingress Controller is not able to fetch the Secret from Kubernetes API, NGINX Ingress Controller will fail to start. Format: `/` ### -enable-latency-metrics Enable collection of latency metrics for upstreams. Requires [-enable-prometheus-metrics](#cmdoption-enable-prometheus-metrics). ### -enable-app-protect Enables support for App Protect. Requires [-nginx-plus](#cmdoption-nginx-plus). - If the argument is set, but `nginx-plus` is set to false, NGINX Ingress Controller will fail to start. ### -app-protect-log-level `` Sets log level for App Protect. Allowed values: fatal, error, warn, info, debug, trace. Requires [-nginx-plus](#cmdoption-nginx-plus) and [-enable-app-protect](#cmdoption-enable-app-protect). - If the argument is set, but `nginx-plus` and `enable-app-protect` are set to false, NGINX Ingress Controller will fail to start. ### -enable-app-protect-dos Enables support for App Protect DoS. Requires [-nginx-plus](#cmdoption-nginx-plus). - If the argument is set, but `nginx-plus` is set to false, NGINX Ingress Controller will fail to start. ### -app-protect-dos-debug Enable debugging for App Protect DoS. Requires [-nginx-plus](#cmdoption-nginx-plus) and [-enable-app-protect-dos](#cmdoption-enable-app-protect-dos). - If the argument is set, but `nginx-plus` and `enable-app-protect-dos` are set to false, NGINX Ingress Controller will fail to start. ### -app-protect-dos-max-daemons Max number of ADMD instances. Default `1`. Requires [-nginx-plus](#cmdoption-nginx-plus) and [-enable-app-protect-dos](#cmdoption-enable-app-protect-dos). - If the argument is set, but `nginx-plus` and `enable-app-protect-dos` are set to false, NGINX Ingress Controller will fail to start. ### -app-protect-dos-max-workers Max number of nginx processes to support. Default `Number of CPU cores in the machine`. Requires [-nginx-plus](#cmdoption-nginx-plus) and [-enable-app-protect-dos](#cmdoption-enable-app-protect-dos). - If the argument is set, but `nginx-plus` and `enable-app-protect-dos` are set to false, NGINX Ingress Controller will fail to start. ### -app-protect-dos-memory RAM memory size to consume in MB Default `50% of free RAM in the container or 80MB, the smaller`. Requires [-nginx-plus](#cmdoption-nginx-plus) and [-enable-app-protect-dos](#cmdoption-enable-app-protect-dos). - If the argument is set, but `nginx-plus` and `enable-app-protect-dos` are set to false, NGINX Ingress Controller will fail to start. ### -enable-app-protect-ip-intelligence Enables App Protect IP Intelligence. Defaults to `false`. *Only applies to WAF single-container deployments*. Requires [-nginx-plus](#cmdoption-nginx-plus) and [-enable-app-protect](#cmdoption-enable-app-protect). ### -plm-storage-url SeaweedFS S3 endpoint from which NGINX Ingress Controller fetches the policy and logconf bundle. Leave empty to turn off PLM support. ### -plm-storage-credentials-secret A Secret containing the SeaweedFS admin secret in the `seaweedfs_admin_secret` key. Format: `/` ### -plm-storage-ca-secret An optional Secret containing `ca.crt` for SeaweedFS TLS verification. Format: `/` ### -plm-storage-client-ssl-secret An optional Secret containing `tls.crt` and `tls.key` for SeaweedFS mTLS. Format: `/` ### -plm-storage-insecure-skip-verify Turns off SeaweedFS TLS verification. For development and testing only. ### -ready-status Enables the readiness endpoint `/nginx-ready`. The endpoint returns a success code when NGINX has loaded all the config after the startup. Default `true`. ### -ready-status-port The HTTP port for the readiness endpoint. Format: `[1024 - 65535]` (default `8081`) ### -disable-ipv6 Disable IPV6 listeners explicitly for nodes that do not support the IPV6 stack. Default `false`. ### -default-http-listener-port Sets the port for the HTTP `default_server` listener. Default `80`. When [-allow-empty-ingress-host](#cmdoption-allow-empty-ingress-host) is enabled, also configures the HTTP listener port for Ingress resources without a `host`. ### -default-https-listener-port Sets the port for the HTTPS `default_server` listener. Default `443`. When [-allow-empty-ingress-host](#cmdoption-allow-empty-ingress-host) is enabled, also configures the HTTPS listener port for Ingress resources without a `host`. ### -ssl-dynamic-reload Used to activate or deactivate lazy loading for SSL Certificates. The default value is `true`. ### -weight-changes-dynamic-reload Enables the ability to change the weight distribution of two-way split clients without reloading NGINX. Requires [-nginx-plus](#cmdoption-nginx-plus). Using this feature may require increasing `map_hash_bucket_size`, `map_hash_max_size`, `variable_hash_bucket_size`, and `variable_hash_max_size` in the ConfigMap based on the number of two-way splits. The default value is `false`. - If the argument is set, but `nginx-plus` is set to false, NGINX Ingress Controller will ignore the flag. ### -enable-directive-autoadjust Automatically adjusts NGINX buffer directives to prevent configuration errors. The default value is `false`. When enabled, the controller automatically adjusts `proxy_buffers`, `proxy_buffer_size`, and `proxy_busy_buffers_size` to ensure they work together properly and NGINX can start successfully. More explanation about this feature can be found in the guide [here](/nic/configuration/proxy-buffers-configuration.md). ### -enable-telemetry-reporting Enable gathering and reporting of software telemetry. The default value is `true`. ### -agent Enable NGINX Agent which can used with `-enable-app-protect` to send events to Security Monitoring. The default value is `false`. ### -agent-instance-group Specify the instance group name to use for the NGINX Ingress Controller deployment when using `-agent`. -------------------------------------------------------------------------------- title: "Command-line arguments" url: https://docs.nginx.com/nginx-ingress-controller/lts/configuration/global-configuration/command-line-arguments/index.md -------------------------------------------------------------------------------- F5 NGINX Ingress Controller LTS supports several command-line arguments, which are set based on installation method: - If you're using *Kubernetes Manifests* to install NGINX Ingress Controller LTS, modify the Manifests to set the command-line arguments. View the [Installation with Manifests](/nic/lts/install/manifests.md) topic for more information. - If you're using *Helm* to install NGINX Ingress Controller LTS, modify the parameters of the Helm chart to set the command-line arguments. View the [Installation with Helm](/nic/lts/install/helm.md) topic for more information. ### -enable-snippets Enable custom NGINX configuration snippets in Ingress, VirtualServer, VirtualServerRoute and TransportServer resources. Default `false`. ### -default-server-tls-secret `` Secret with a TLS certificate and key for TLS termination of the default server. - If not set, certificate and key in the file `/etc/nginx/secrets/default` are used. - If `/etc/nginx/secrets/default` doesn't exist, NGINX Ingress Controller LTS will configure NGINX to reject TLS connections to the default server. - If a secret is set, but NGINX Ingress Controller LTS is not able to fetch it from Kubernetes API, or it is not set and NGINX Ingress Controller LTS fails to read the file "/etc/nginx/secrets/default", NGINX Ingress Controller LTS will fail to start. Format: `/` ### -wildcard-tls-secret `` A Secret with a TLS certificate and key for TLS termination of every Ingress/VirtualServer host for which TLS termination is enabled but the Secret is not specified. - If the argument is not set, for such Ingress/VirtualServer hosts NGINX will break any attempt to establish a TLS connection - If the argument is set, but NGINX Ingress Controller LTS is not able to fetch the Secret from Kubernetes API, NGINX Ingress Controller LTS will fail to start. Format: `/` ### -enable-custom-resources Enables custom resources. Default `true`. ### -enable-oidc Enables OIDC policies. Default `false`. ### -enable-leader-election Enables Leader election to avoid multiple replicas of the controller reporting the status of Ingress, VirtualServer and VirtualServerRoute resources -- only one replica will report status. Default `true`. See [-report-ingress-status](#cmdoption-report-ingress-status) flag. ### -enable-tls-passthrough Enable TLS Passthrough on port 443. Requires [-enable-custom-resources](#cmdoption-enable-custom-resources). ### -tls-passthrough-port `` Set the port for TLS Passthrough. Format: `[1024 - 65535]` (default `443`) Requires [-enable-custom-resources](#cmdoption-enable-custom-resources). ### -enable-cert-manager Enable x509 automated certificate management for VirtualServer resources using cert-manager (cert-manager.io). Requires [-enable-custom-resources](#cmdoption-enable-custom-resources). ### -enable-external-dns Enable integration with ExternalDNS for configuring public DNS entries for VirtualServer resources using [ExternalDNS](https://github.com/kubernetes-sigs/external-dns). Requires [-enable-custom-resources](#cmdoption-enable-custom-resources). ### -external-service `` Specifies the name of the service with the type LoadBalancer through which the NGINX Ingress Controller LTS pods are exposed externally. The external address of the service is used when reporting the status of Ingress, VirtualServer and VirtualServerRoute resources. For Ingress resources only: Requires [-report-ingress-status](#cmdoption-report-ingress-status). ### -ingresslink `` Specifies the name of the IngressLink resource, which exposes the NGINX Ingress Controller LTS pods via a BIG-IP system. The IP of the BIG-IP system is used when reporting the status of Ingress, VirtualServer and VirtualServerRoute resources. For Ingress resources only: Requires [-report-ingress-status](#cmdoption-report-ingress-status). ### -global-configuration `` A GlobalConfiguration resource for global configuration of NGINX Ingress Controller LTS. Format: `/` Requires [-enable-custom-resources](#cmdoption-enable-custom-resources). ### -health-status Adds a location "/nginx-health" to the default server. The location responds with the 200 status code for any request. Useful for external health-checking of NGINX Ingress Controller LTS. ### -health-status-uri `` Sets the URI of health status location in the default server. Requires [-health-status](#cmdoption-health-status). (default `/nginx-health`) ### -ingress-class `` The `-ingress-class` argument refers to the name of the resource `kind: IngressClass`. An IngressClass resource with a name equal to the class must be deployed. Otherwise, NGINX Ingress Controller LTS will fail to start. NGINX Ingress Controller LTS will only process Ingress resources that belong to its class (Whose `ingressClassName` value matches the value of `-ingress-class`), skipping the ones without it. It will also process all the VirtualServer/VirtualServerRoute/TransportServer resources that do not have the `ingressClassName` field. Default `nginx`. ### -ingress-template-path `` Path to the ingress NGINX configuration template for an ingress resource. Default: `nginx-plus.ingress.tmpl`. ### -leader-election-lock-name `` Specifies the name of the ConfigMap, within the same namespace as the controller, used as the lock for leader election. Requires [-enable-leader-election](#cmdoption-enable-leader-election). ### -log_backtrace_at `` When logging hits line `file:N`, emit a stack trace. ### -main-template-path `` Path to the main NGINX configuration template. - Default: `nginx-plus.tmpl`. ### -nginx-configmaps `` A ConfigMap resource for customizing NGINX configuration. If a ConfigMap is set, but NGINX Ingress Controller LTS is not able to fetch it from Kubernetes API, NGINX Ingress Controller LTS will fail to start. Format: `/` ### -mgmt-configmap `` The Management ConfigMap resource is used for customizing the NGINX mgmt block. If using NGINX Plus, a Management ConfigMap must be set. If NGINX Ingress Controller LTS is not able to fetch it from Kubernetes API, NGINX Ingress Controller LTS will fail to start. Format: `/` ### -nginx-debug Enable debugging for NGINX. Uses the nginx-debug binary. Requires 'error-log-level: debug' in the ConfigMap. ### -nginx-plus Enable support for NGINX Plus. ### -nginx-reload-timeout `` Timeout in milliseconds which NGINX Ingress Controller LTS will wait for a successful NGINX reload after a change or at the initial start. Default is 60000. ### -nginx-status Enable the NGINX Plus API. Default `true`. ### -nginx-status-allow-cidrs `` Add IP/CIDR blocks to the allow list for the NGINX Plus API. Separate multiple IP/CIDR by commas. (default `127.0.0.1,::1`) ### -nginx-status-port `` Set the port where the NGINX Plus API is exposed. Format: `[1024 - 65535]` (default `8080`) ### -proxy `` **warning:** This argument is intended for testing purposes only. Use a proxy server to connect to Kubernetes API started with `kubectl proxy`. NGINX Ingress Controller LTS does not start NGINX and does not write any generated NGINX configuration files to disk. ### -report-ingress-status Updates the address field in the status of Ingress resources. Requires the [-external-service](#cmdoption-external-service) or [-ingresslink](#cmdoption-ingresslink) flag, or the `external-status-address` key in the ConfigMap. ### -transportserver-template-path `` Path to the TransportServer NGINX configuration template for a TransportServer resource. - Default: `nginx-plus.transportserver.tmpl`. ### -log-level `` Log level for Ingress Controller logs. Allowed values: fatal, error, warn, info, debug, trace. - Default is `info`. ### -log-format `` Log format for Ingress Controller logs. Allowed values: glog, json, text. - Default is `glog`. ### -version Print the version, git-commit hash and build date and exit. ### -virtualserver-template-path `` Path to the VirtualServer NGINX configuration template for a VirtualServer resource. - Default: `nginx-plus.virtualserver.tmpl`. ### -vmodule `` A comma-separated list of pattern=N settings for file-filtered logging. ### -watch-namespace `` Comma separated list of namespaces NGINX Ingress Controller LTS should watch for resources. By default NGINX Ingress Controller LTS watches all namespaces. Mutually exclusive with "watch-namespace-label". ### -watch-namespace-label `` Configures NGINX Ingress Controller LTS to watch only those namespaces with label foo=bar. By default NGINX Ingress Controller LTS watches all namespaces. Mutually exclusive with "watch-namespace". ### -watch-secret-namespace `` Comma separated list of namespaces NGINX Ingress Controller LTS should watch for secrets. If this arg is not configured, NGINX Ingress Controller LTS watches the same namespaces for all resources, see "watch-namespace" and "watch-namespace-label". All namespaces included with this argument must be part of either `-watch-namespace` or `-watch-namespace-label`. ### -enable-prometheus-metrics Enables exposing NGINX Plus metrics in the Prometheus format. ### -prometheus-metrics-listen-port `` Sets the port where the Prometheus metrics are exposed. Format: `[1024 - 65535]` (default `9113`) ### -prometheus-tls-secret `` A Secret with a TLS certificate and key for TLS termination of the Prometheus metrics endpoint. - If the argument is not set, the Prometheus endpoint will not use a TLS connection. - If the argument is set, but NGINX Ingress Controller LTS is not able to fetch the Secret from Kubernetes API, NGINX Ingress Controller LTS will fail to start. ### -enable-service-insight Exposes the Service Insight endpoint for Ingress Controller. ### -service-insight-listen-port `` Sets the port where the Service Insight is exposed. Format: `[1024 - 65535]` (default `9114`) ### -service-insight-tls-secret `` A Secret with a TLS certificate and key for TLS termination of the Service Insight endpoint. - If the argument is not set, the Service Insight endpoint will not use a TLS connection. - If the argument is set, but NGINX Ingress Controller LTS is not able to fetch the Secret from Kubernetes API, NGINX Ingress Controller LTS will fail to start. Format: `/` ### -spire-agent-address `` Specifies the address of a running Spire agent. **For use with NGINX Service Mesh only**. - If the argument is set, but NGINX Ingress Controller LTS is unable to connect to the Spire Agent, NGINX Ingress Controller LTS will fail to start. ### -enable-internal-routes Enable support for internal routes with NGINX Service Mesh. **For use with NGINX Service Mesh only**. Requires [-spire-agent-address](#cmdoption-spire-agent-address). - If the argument is set, but `spire-agent-address` is not provided, NGINX Ingress Controller LTS will fail to start. ### -enable-latency-metrics Enable collection of latency metrics for upstreams. Requires [-enable-prometheus-metrics](#cmdoption-enable-prometheus-metrics). ### -ready-status Enables the readiness endpoint `/nginx-ready`. The endpoint returns a success code when NGINX has loaded all the config after the startup. Default `true`. ### -ready-status-port The HTTP port for the readiness endpoint. Format: `[1024 - 65535]` (default `8081`) ### -disable-ipv6 Disable IPV6 listeners explicitly for nodes that do not support the IPV6 stack. Default `false`. ### -default-http-listener-port Sets the port for the HTTP `default_server` listener. Default `80`. ### -default-https-listener-port Sets the port for the HTTPS `default_server` listener. Default `443`. ### -ssl-dynamic-reload Used to activate or deactivate lazy loading for SSL Certificates. The default value is `true`. ### -weight-changes-dynamic-reload Enables the ability to change the weight distribution of two-way split clients without reloading NGINX. Requires [-nginx-plus](#cmdoption-nginx-plus). Using this feature may require increasing `map_hash_bucket_size`, `map_hash_max_size`, `variable_hash_bucket_size`, and `variable_hash_max_size` in the ConfigMap based on the number of two-way splits. The default value is `false`. - If the argument is set, but `nginx-plus` is set to false, NGINX Ingress Controller LTS will ignore the flag. ### -enable-directive-autoadjust Automatically adjusts NGINX buffer directives to prevent configuration errors. The default value is `false`. When enabled, the controller automatically adjusts `proxy_buffers`, `proxy_buffer_size`, and `proxy_busy_buffers_size` to ensure they work together properly and NGINX can start successfully. More explanation about this feature can be found in the guide [here](/nic/lts/configuration/proxy-buffers-configuration.md). ### -enable-telemetry-reporting Enable gathering and reporting of software telemetry. The default value is `true`. -------------------------------------------------------------------------------- title: "Commercial support" url: https://docs.nginx.com/nginx-ingress-controller/lts/troubleshooting/troubleshoot-support/index.md -------------------------------------------------------------------------------- F5 NGINX Ingress Controller LTS adheres to the support policy detailed in the following knowledge base article: [K000140156](https://my.f5.com/manage/s/article/K000140156). After opening a support ticket, F5 staff will request additional information to better understand the problem. The [nginx-supportpkg-for-k8s](https://github.com/nginxinc/nginx-supportpkg-for-k8s) plugin collects the information needed by F5 Technical Support to assist with troubleshooting your issue. When used, the plugin will generate a tarball of the collected information which can be shared with the support channels. The plugin uses [krew](https://krew.sigs.k8s.io), the plugin manager for the Kubernetes [kubectl](https://kubernetes.io/docs/reference/kubectl/) command-line tool. The plugin may collect some or all of the following global and namespace-specific information: * K8s version, nodes information, and Custom Resources (kubectl describe output) * Pods' logs * List of Pods, events, ConfigMaps, Services, Deployments, Daemonsets, StatefulSets, ReplicaSets, and Leases * K8s metrics * Helm deployments * `nginx -T` output from NGINX-related pods This plugin **does not** collect secrets or coredumps. Visit the [project’s GitHub repository](https://github.com/nginxinc/nginx-supportpkg-for-k8s) for further details. ## Support channels - If you experience issues with NGINX Ingress Controller LTS, please [open an issue](https://github.com/nginx/kubernetes-ingress/issues/new?assignees=&labels=bug%2Cneeds+triage&projects=&template=BUG-REPORT.yml&title=%5BBug%5D%3A+) in GitHub. - If you have any enhancement requests, please [open a feature request](https://github.com/nginx/kubernetes-ingress/issues/new?assignees=&labels=proposal&projects=&template=feature_request.md&title=) in GitHub. - If you have any ideas or suggestions to discuss, please [open an idea discussion](https://github.com/nginx/kubernetes-ingress/discussions/categories/ideas) in GitHub. -------------------------------------------------------------------------------- title: "Commercial support" url: https://docs.nginx.com/nginx-ingress-controller/troubleshooting/troubleshoot-support/index.md -------------------------------------------------------------------------------- F5 NGINX Ingress Controller adheres to the support policy detailed in the following knowledge base article: [K000140156](https://my.f5.com/manage/s/article/K000140156). After opening a support ticket, F5 staff will request additional information to better understand the problem. The [nginx-supportpkg-for-k8s](https://github.com/nginxinc/nginx-supportpkg-for-k8s) plugin collects the information needed by F5 Technical Support to assist with troubleshooting your issue. When used, the plugin will generate a tarball of the collected information which can be shared with the support channels. The plugin uses [krew](https://krew.sigs.k8s.io), the plugin manager for the Kubernetes [kubectl](https://kubernetes.io/docs/reference/kubectl/) command-line tool. The plugin may collect some or all of the following global and namespace-specific information: * K8s version, nodes information, and Custom Resources (kubectl describe output) * Pods' logs * List of Pods, events, ConfigMaps, Services, Deployments, Daemonsets, StatefulSets, ReplicaSets, and Leases * K8s metrics * Helm deployments * `nginx -T` output from NGINX-related pods This plugin **does not** collect secrets or coredumps. Visit the [project’s GitHub repository](https://github.com/nginxinc/nginx-supportpkg-for-k8s) for further details. ## Support channels - If you experience issues with NGINX Ingress Controller, please [open an issue](https://github.com/nginx/kubernetes-ingress/issues/new?assignees=&labels=bug%2Cneeds+triage&projects=&template=BUG-REPORT.yml&title=%5BBug%5D%3A+) in GitHub. - If you have any enhancement requests, please [open a feature request](https://github.com/nginx/kubernetes-ingress/issues/new?assignees=&labels=proposal&projects=&template=feature_request.md&title=) in GitHub. - If you have any ideas or suggestions to discuss, please [open an idea discussion](https://github.com/nginx/kubernetes-ingress/discussions/categories/ideas) in GitHub. -------------------------------------------------------------------------------- title: "Download NGINX Ingress Controller from the F5 Registry" url: https://docs.nginx.com/nginx-ingress-controller/install/images/registry-download/index.md -------------------------------------------------------------------------------- This page describes how to download an F5 NGINX Plus Ingress Controller image from the official F5 Docker registry. The F5 Registry images include versions with F5 WAF for NGINX and F5 DoS for NGINX. ## Before you begin To follow these steps, you will need the following pre-requisites: - [Docker v18.09 or higher](https://docs.docker.com/engine/release-notes/18.09/) You can also get the NGINX Ingress Controller image using the following alternate methods: - [Add an NGINX Ingress Controller image to your cluster](/nic/install/images/add-image-to-cluster.md) - [Build NGINX Ingress Controller](/nic/install/build.md) - For NGINX Open Source, you can pull [an image from DockerHub](https://hub.docker.com/r/nginx/nginx-ingress/) ### Download your subscription credential files In order to obtain a container image, you will need the JSON Web Token file or SSL certificate and private key files provided with your NGINX Plus subscription. These files grant access to the package repository from which the script will download the NGINX Plus package: #### JSON Web Token #### SSL ### Set up Docker for the F5 Container Registry This step describes how to use Docker to communicate with the F5 Container Registry located at `private-registry.nginx.com`. **Note:** The steps provided are for Linux. For Mac or Windows, see the [Docker for Mac](https://docs.docker.com/docker-for-mac/#add-client-certificates) or [Docker for Windows](https://docs.docker.com/docker-for-windows/#how-do-i-add-client-certificates) documentation. For more details on Docker Engine security, you can refer to the [Docker Engine Security documentation](https://docs.docker.com/engine/security/). #### JSON Web Token Open the JSON Web Token file previously downloaded from [MyF5](https://my.f5.com) customer portal (for example, `nginx-repo-12345abc.jwt`) and copy its contents. Log in to the Docker registry using the contents of the JSON Web Token file: ```shell docker login private-registry.nginx.com --username= --password=none ``` #### SSL Create a directory and copy your certificate and key to this directory: ```shell mkdir -p /etc/docker/certs.d/private-registry.nginx.com cp /etc/docker/certs.d/private-registry.nginx.com/client.cert cp /etc/docker/certs.d/private-registry.nginx.com/client.key ``` ## Pull the image Identify which image you need using the [Technical specifications](/nic/technical-specifications.md#images-with-nginx-plus) topic. Next, pull the image from `private-registry.nginx.com`. Replace `` with the specific version you need, for example, ``. - For NGINX Plus Ingress Controller, run: ```shell docker pull private-registry.nginx.com/nginx-ic/nginx-plus-ingress: ``` - For NGINX Plus Ingress Controller with F5 WAF for NGINX, run: ```shell docker pull private-registry.nginx.com/nginx-ic-nap/nginx-plus-ingress: ``` - For NGINX Plus Ingress Controller with F5 WAF for NGINX and NGINX Agent 3 (required for NGINX One Console, available starting with NGINX Ingress Controller 5.5.0), run: ```shell docker pull private-registry.nginx.com/nginx-ic-nap/nginx-plus-ingress:-agent ``` - For NGINX Plus Ingress Controller with F5 WAF for NGINX v5, run: ```shell docker pull private-registry.nginx.com/nginx-ic-nap-v5/nginx-plus-ingress: ``` ```shell docker pull private-registry.nginx.com/nap/waf-config-mgr: ``` ```shell docker pull private-registry.nginx.com/nap/waf-enforcer: ``` - For NGINX Plus Ingress Controller with F5 WAF for NGINX v5 and NGINX Agent 3 (required for NGINX One Console, available starting with NGINX Ingress Controller 5.5.0), run: ```shell docker pull private-registry.nginx.com/nginx-ic-nap-v5/nginx-plus-ingress:-agent ``` ```shell docker pull private-registry.nginx.com/nap/waf-config-mgr: ``` ```shell docker pull private-registry.nginx.com/nap/waf-enforcer: ``` - For NGINX Plus Ingress Controller with F5 DoS for NGINX, run: ```shell docker pull private-registry.nginx.com/nginx-ic-dos/nginx-plus-ingress: ``` - For NGINX Plus Ingress Controller with F5 WAF for NGINX and F5 DoS for NGINX, run: ```shell docker pull private-registry.nginx.com/nginx-ic-nap-dos/nginx-plus-ingress: ``` You can use the Docker registry API to list the available image tags by running the following commands. Replace `` with the location of your client key and `` with the location of your client certificate. The `jq` command was used in these examples to make the JSON output easier to read. ```shell curl https://private-registry.nginx.com/v2/nginx-ic/nginx-plus-ingress/tags/list --key --cert ``` ```json { "name": "nginx-ic/nginx-plus-ingress", "tags": [ "-alpine", "-alpine-fips", "-ubi", "" ] } ``` ```shell curl https://private-registry.nginx.com/v2/nginx-ic-nap/nginx-plus-ingress/tags/list --key --cert ``` ```json { "name": "nginx-ic-nap/nginx-plus-ingress", "tags": [ "-alpine-fips", "-alpine-fips-agent", "-ubi", "-ubi-agent", "-ubi8", "-ubi8-agent", "", "-agent" ] } ``` ```shell curl https://private-registry.nginx.com/v2/nginx-ic-nap-v5/nginx-plus-ingress/tags/list --key --cert ``` ```json { "name": "nginx-ic-nap-v5/nginx-plus-ingress", "tags": [ "-alpine-fips", "-alpine-fips-agent", "-ubi", "-ubi-agent", "-ubi8", "-ubi8-agent", "", "-agent" ] } ``` ```shell curl https://private-registry.nginx.com/v2/nginx-ic-dos/nginx-plus-ingress/tags/list --key --cert ``` ```json { "name": "nginx-ic-dos/nginx-plus-ingress", "tags": [ "-ubi", "" ] } ``` ## Push to your private registry After pulling the image, tag it and upload it to your private registry. 1. Log in to your private registry: ```shell docker login ``` 1. Tag and push the image. Replace `` with your registry's path and `` with the version you're using, for example ``: - For NGINX Plus Ingress Controller, run: ```shell docker tag private-registry.nginx.com/nginx-ic/nginx-plus-ingress: /nginx-ic/nginx-plus-ingress: docker push /nginx-ic/nginx-plus-ingress: ``` - For NGINX Plus Ingress Controller with F5 WAF for NGINX, run: ```shell docker tag private-registry.nginx.com/nginx-ic-nap/nginx-plus-ingress: /nginx-ic-nap/nginx-plus-ingress: docker push /nginx-ic-nap/nginx-plus-ingress: ``` - For NGINX Plus Ingress Controller with F5 WAF for NGINX and NGINX Agent 3, run: ```shell docker tag private-registry.nginx.com/nginx-ic-nap/nginx-plus-ingress:-agent /nginx-ic-nap/nginx-plus-ingress:-agent docker push /nginx-ic-nap/nginx-plus-ingress:-agent ``` - For NGINX Plus Ingress Controller with F5 WAF for NGINX v5, run: ```shell docker tag private-registry.nginx.com/nginx-ic-nap-v5/nginx-plus-ingress: /nginx-ic-nap-v5/nginx-plus-ingress: docker push /nginx-ic-nap-v5/nginx-plus-ingress: ``` ```shell docker tag private-registry.nginx.com/nap/waf-config-mgr: /nap/waf-config-mgr: docker push /nap/waf-config-mgr: ``` ```shell docker tag private-registry.nginx.com/nap/waf-enforcer: /nap/waf-enforcer: docker push /nap/waf-enforcer: ``` - For NGINX Plus Ingress Controller with F5 WAF for NGINX v5 and NGINX Agent 3, run: ```shell docker tag private-registry.nginx.com/nginx-ic-nap-v5/nginx-plus-ingress:-agent /nginx-ic-nap-v5/nginx-plus-ingress:-agent docker push /nginx-ic-nap-v5/nginx-plus-ingress:-agent ``` ```shell docker tag private-registry.nginx.com/nap/waf-config-mgr: /nap/waf-config-mgr: docker push /nap/waf-config-mgr: ``` ```shell docker tag private-registry.nginx.com/nap/waf-enforcer: /nap/waf-enforcer: docker push /nap/waf-enforcer: ``` - For NGINX Plus Ingress Controller with F5 DoS for NGINX, run: ```shell docker tag private-registry.nginx.com/nginx-ic-dos/nginx-plus-ingress: /nginx-ic-dos/nginx-plus-ingress: docker push /nginx-ic-dos/nginx-plus-ingress: ``` ## Troubleshooting If you encounter issues while following this guide, here are some possible solutions: - **Certificate errors** - **Likely Cause**: Incorrect certificate or key location, or using an NGINX Plus certificate. - **Solution**: Verify you have the correct NGINX Ingress Controller certificate and key. Place them in the correct directory and ensure the certificate has a *.cert* extension. - **Docker version compatibility** - **Likely Cause**: Outdated Docker version. - **Solution**: Make sure you're running [Docker v18.09 or higher](https://docs.docker.com/engine/release-notes/18.09/). Upgrade if necessary. - **Can't pull the image** - **Likely Cause**: Mismatched image name or tag. - **Solution**: Double-check the image name and tag matches the [Technical specifications](/nic/technical-specifications.md#images-with-nginx-plus) document. - **Failed to push to private registry** - **Likely Cause**: Not logged into your private registry or incorrect image tagging. - **Solution**: Verify login status and correct image tagging before pushing. Consult the [Docker documentation](https://docs.docker.com/docker-hub/repos/) for more details. -------------------------------------------------------------------------------- title: "Download NGINX Ingress Controller LTS from the F5 Registry" url: https://docs.nginx.com/nginx-ingress-controller/lts/install/images/registry-download/index.md -------------------------------------------------------------------------------- This page describes how to download an F5 NGINX Plus Ingress Controller LTS image from the official F5 Docker registry. ## Before you begin To follow these steps, you will need the following pre-requisites: - [Docker v18.09 or higher](https://docs.docker.com/engine/release-notes/18.09/) You can also get the NGINX Ingress Controller LTS image using the following alternate methods: - [Add an NGINX Ingress Controller LTS image to your cluster](/nic/lts/install/images/add-image-to-cluster.md) ### Download your subscription credential files In order to obtain a container image, you will need the JSON Web Token file or SSL certificate and private key files provided with your NGINX Plus subscription. These files grant access to the package repository from which the script will download the NGINX Plus package: #### JSON Web Token #### SSL ### Set up Docker for the F5 Container Registry This step describes how to use Docker to communicate with the F5 Container Registry located at `private-registry.nginx.com`. **Note:** The steps provided are for Linux. For Mac or Windows, see the [Docker for Mac](https://docs.docker.com/docker-for-mac/#add-client-certificates) or [Docker for Windows](https://docs.docker.com/docker-for-windows/#how-do-i-add-client-certificates) documentation. For more details on Docker Engine security, you can refer to the [Docker Engine Security documentation](https://docs.docker.com/engine/security/). #### JSON Web Token Open the JSON Web Token file previously downloaded from [MyF5](https://my.f5.com) customer portal (for example, `nginx-repo-12345abc.jwt`) and copy its contents. Log in to the Docker registry using the contents of the JSON Web Token file: ```shell docker login private-registry.nginx.com --username= --password=none ``` #### SSL Create a directory and copy your certificate and key to this directory: ```shell mkdir -p /etc/docker/certs.d/private-registry.nginx.com cp /etc/docker/certs.d/private-registry.nginx.com/client.cert cp /etc/docker/certs.d/private-registry.nginx.com/client.key ``` ## Pull the image Identify which image you need using the [Technical specifications](/nic/lts/technical-specifications.md#images-with-nginx-plus) topic. Next, pull the image from `private-registry.nginx.com`. Replace `` with the specific version you need, for example, ``. - For NGINX Plus Ingress Controller, run: ```shell docker pull private-registry.nginx.com/nginx-ic/lts/nginx-plus-ingress: ``` You can use the Docker registry API to list the available image tags by running the following commands. Replace `` with the location of your client key and `` with the location of your client certificate. The `jq` command was used in these examples to make the JSON output easier to read. ```shell curl https://private-registry.nginx.com/v2/nginx-ic/lts/nginx-plus-ingress/tags/list --key --cert ``` ```json { "name": "nginx-ic/lts/nginx-plus-ingress", "tags": [ "" ] } ``` ## Push to your private registry After pulling the image, tag it and upload it to your private registry. 1. Log in to your private registry: ```shell docker login ``` 1. Tag and push the image. Replace `` with your registry's path and `` with the version you're using, for example ``: - For NGINX Ingress Controller LTS, run: ```shell docker tag private-registry.nginx.com/nginx-ic/lts/nginx-plus-ingress: /nginx-ic/lts/nginx-plus-ingress: docker push /nginx-ic/lts/nginx-plus-ingress: ``` ## Troubleshooting If you encounter issues while following this guide, here are some possible solutions: - **Certificate errors** - **Likely Cause**: Incorrect certificate or key location, or using an NGINX Plus certificate. - **Solution**: Verify you have the correct NGINX Ingress Controller LTS certificate and key. Place them in the correct directory and ensure the certificate has a *.cert* extension. - **Docker version compatibility** - **Likely Cause**: Outdated Docker version. - **Solution**: Make sure you're running [Docker v18.09 or higher](https://docs.docker.com/engine/release-notes/18.09/). Upgrade if necessary. - **Can't pull the image** - **Likely Cause**: Mismatched image name or tag. - **Solution**: Double-check the image name and tag matches the [Technical specifications](/nic/lts/technical-specifications.md#images-with-nginx-plus) document. - **Failed to push to private registry** - **Likely Cause**: Not logged into your private registry or incorrect image tagging. - **Solution**: Verify login status and correct image tagging before pushing. Consult the [Docker documentation](https://docs.docker.com/docker-hub/repos/) for more details. -------------------------------------------------------------------------------- title: "Logs available from NGINX Ingress Controller" url: https://docs.nginx.com/nginx-ingress-controller/logging-and-monitoring/logging/index.md -------------------------------------------------------------------------------- This document gives an overview of logging provided by F5 NGINX Ingress Controller. NGINX Ingress Controller exposes the logs of the Ingress Controller process (The process that generates NGINX configuration and reloads NGINX to apply it) and NGINX access and error logs. All logs are sent to the standard output and error of the NGINX Ingress Controller process. To view the logs, you can execute the `kubectl logs` command for an Ingress Controller pod. For example: ```shell kubectl logs -n nginx-ingress ``` ## NGINX Ingress Controller Process Logs The NGINX Ingress Controller process logs are configured through the `-log-level` command-line argument of the NGINX Ingress Controller, which sets the log level. The default value is `info`. Other options include: `trace`, `debug`, `info`, `warning`, `error` and `fatal`. The value `debug` is useful for troubleshooting: you will be able to see how NGINX Ingress Controller gets updates from the Kubernetes API, generates NGINX configuration and reloads NGINX. Read more about NGINX Ingress Controller [command-line arguments](/nic/configuration/global-configuration/command-line-arguments.md). ### Resource attributes on process log lines In a multi-tenant cluster, these attributes let you filter, route, and triage the NGINX Ingress Controller process log lines by namespace or resource, and trace a log line back to the object that produced it, without manual investigation. To make this possible, NGINX Ingress Controller stamps its process log lines with the identity of the Kubernetes resource it is processing. Three attributes carry this identity: - `resource_namespace`: the resource's namespace. - `resource_kind`: the resource's kind, such as `Ingress`, `VirtualServer`, `VirtualServerRoute`, `TransportServer`, or `Policy`. - `resource_name`: the resource's name. These attributes appear only on NGINX Ingress Controller process log lines emitted at any log level while it processes a resource. They do not appear in the NGINX access or error logs. One, two, or all three attributes can appear, depending on the code path. A line about a namespace-scoped operation may carry only `resource_namespace` and `resource_kind`, while a line about a specific object carries all three. How the attributes render depends on the [`-log-format`](/nic/configuration/global-configuration/command-line-arguments.md#cmdoption-log-format) command-line argument. The `glog` format renders them as space-separated `key=value` pairs, placed after the `file:line]` bracket and before the message. The `json` format emits them as native JSON fields, and the `text` format emits them as `key=value` fields. The following example shows an NGINX Ingress Controller log line in the default `glog` format. It is illustrative, not runnable: ```text W20260806 14:07:23.267011 1 controller.go:3057] resource_namespace=log-test-1 resource_kind=VirtualServer resource_name=webapp1 Error trying to get the secret log-test-1/tls-secret for VirtualServer webapp1: secret doesn't exist or of an unsupported type ``` ## NGINX Logs NGINX includes two logs: - *Access log*, where NGINX writes information about client requests in the access log right after the request is processed. The access log is configured via the [logging-related](/nic/configuration/global-configuration/configmap-resource.md#logging) ConfigMap keys: - `log-format` for HTTP and HTTPS traffic. - `stream-log-format` for TCP, UDP, and TLS Passthrough traffic. Additionally, you can disable access logging with the `access-log-off` ConfigMap key. - *Error log*, where NGINX writes information about encountered issues of different severity levels. It is configured via the `error-log-level` [ConfigMap key](/nic/configuration/global-configuration.md#configmap-resource#logging). To enable debug logging, set the level to `debug` and also set the `-nginx-debug` [command-line argument](/nic/configuration/global-configuration.md#command-line-arguments), so that NGINX is started with the debug binary `nginx-debug`. Read more about [NGINX logs](/nginx/admin-guide/monitoring/logging.md) from NGINX Admin guide. -------------------------------------------------------------------------------- title: "Logs available from NGINX Ingress Controller LTS" url: https://docs.nginx.com/nginx-ingress-controller/lts/logging-and-monitoring/logging/index.md -------------------------------------------------------------------------------- This document gives an overview of logging provided by F5 NGINX Ingress Controller LTS. NGINX Ingress Controller LTS exposes the logs of the Ingress Controller process (The process that generates NGINX configuration and reloads NGINX to apply it) and NGINX access and error logs. All logs are sent to the standard output and error of the NGINX Ingress Controller LTS process. To view the logs, you can execute the `kubectl logs` command for an Ingress Controller pod. For example: ```shell kubectl logs -n nginx-ingress ``` ## NGINX Ingress Controller LTS Process Logs The NGINX Ingress Controller LTS process logs are configured through the `-log-level` command-line argument of the NGINX Ingress Controller LTS, which sets the log level. The default value is `info`. Other options include: `trace`, `debug`, `info`, `warning`, `error` and `fatal`. The value `debug` is useful for troubleshooting: you will be able to see how NGINX Ingress Controller LTS gets updates from the Kubernetes API, generates NGINX configuration and reloads NGINX. Read more about NGINX Ingress Controller LTS [command-line arguments](/nic/lts/configuration/global-configuration/command-line-arguments.md). ## NGINX Logs NGINX includes two logs: - *Access log*, where NGINX writes information about client requests in the access log right after the request is processed. The access log is configured via the [logging-related](/nic/lts/configuration/global-configuration/configmap-resource.md#logging) ConfigMap keys: - `log-format` for HTTP and HTTPS traffic. - `stream-log-format` for TCP, UDP, and TLS Passthrough traffic. Additionally, you can disable access logging with the `access-log-off` ConfigMap key. - *Error log*, where NGINX writes information about encountered issues of different severity levels. It is configured via the `error-log-level` [ConfigMap key](/nic/lts/configuration/global-configuration.md#configmap-resource#logging). To enable debug logging, set the level to `debug` and also set the `-nginx-debug` [command-line argument](/nic/lts/configuration/global-configuration.md#command-line-arguments), so that NGINX is started with the debug binary `nginx-debug`. Read more about [NGINX logs](/nginx/admin-guide/monitoring/logging.md) from NGINX Admin guide. -------------------------------------------------------------------------------- title: "Use Helm to Install NGINX Ingress Controller with NGINX Open Source" url: https://docs.nginx.com/nginx-ingress-controller/install/helm/open-source/index.md -------------------------------------------------------------------------------- This page describes how to use Helm to install F5 NGINX Ingress Controller with NGINX Open Source. It explains the requirements for NGINX Ingress Controller, how to obtain and install the Helm chart, and what custom resource definitions (CRDs) are installed during the process. By following these instructions, you will finish with a functional NGINX Ingress Controller instance for your Kubernetes cluster. ## Before you begin - A [supported Kubernetes version](/nic/technical-specifications.md#supported-kubernetes-versions) - A functional Kubernetes cluster - [Helm 3.19+.](https://helm.sh/docs/intro/install) Throughout this page, you will see placeholder values indicated with angular brackets, such as **\**. Replace them accordingly for your installation. **Note:** The `edge` version **is not intended for production use**. It is intended for testing and development purposes only. If you'd like to test the latest changes in NGINX Ingress Controller before a new release, you can install the `edge` version, which is built from the `main` branch of the [NGINX Ingress Controller repository](https://github.com/nginx/kubernetes-ingress). You can install the `edge` version by specifying the `--version` flag with the value `0.0.0-edge`: ```shell helm install oci://ghcr.io/nginx/charts/nginx-ingress --version 0.0.0-edge ``` ## Install the Helm chart You have two options for installing the Helm chart: directly from the OCI registry, or using the source. ### OCI Registry To install NGINX Ingress Controller using the OCI registry, run this command with your release name: ```shell helm install oci://ghcr.io/nginx/charts/nginx-ingress --version ``` #### Example output ```text Pulled: ghcr.io/nginx/charts/nginx-ingress: Digest: sha256:bb452d593c31b6be39f459f9604882e170227429821bac01e7ddd7da16d91ba1 NAME: h4-oss LAST DEPLOYED: Fri Nov 28 11:53:57 2025 NAMESPACE: default STATUS: deployed REVISION: 1 DESCRIPTION: Install complete TEST SUITE: None NOTES: NGINX Ingress Controller has been installed. For release notes for this version please see: https://docs.nginx.com/nginx-ingress-controller/releases/ Installation and upgrade instructions: https://docs.nginx.com/nginx-ingress-controller/installation/installing-nic/installation-with-helm/ ``` ### From source To install NGINX Ingress Controller from source, first pull the chart by running this command: ```shell helm pull oci://ghcr.io/nginx/charts/nginx-ingress --untar --version ``` #### Example output ```text Pulled: ghcr.io/nginx/charts/nginx-ingress: Digest: sha256:bb452d593c31b6be39f459f9604882e170227429821bac01e7ddd7da16d91ba1 ``` Then use the `cd` command to change your working directory to _nginx-ingress_: ```shell cd nginx-ingress ``` Finally, install the chart with your release name with `helm install`: ```shell helm install . ``` #### Example output ```text NAME: h4-oss-source LAST DEPLOYED: Fri Nov 28 12:06:07 2025 NAMESPACE: default STATUS: deployed REVISION: 1 DESCRIPTION: Install complete TEST SUITE: None NOTES: NGINX Ingress Controller has been installed. For release notes for this version please see: https://docs.nginx.com/nginx-ingress-controller/releases/ Installation and upgrade instructions: https://docs.nginx.com/nginx-ingress-controller/installation/installing-nic/installation-with-helm/ ``` ## Verify the deployment To verify that NGINX Ingress Controller has been installed correctly, you can review `ingressclasses` with `kubectl get`: ```shell kubectl get ingressclasses ``` #### Example output ```text NAME CONTROLLER PARAMETERS AGE nginx nginx.org/ingress-controller 10m ``` ## Custom Resource Definitions When installing the chart, Helm will install the required CRDs. Without them, NGINX Ingress Controller pods will not become _Ready_. If you do not use the custom resources that require those CRDs, add the parameter `--skip-crds` in your `helm install` command. The following chart parameters should be set to `false`: - `controller.enableCustomResources` - `controller.appprotect.enable` - `controller.appprotectdos.enable` ## Next steps - [NGINX Ingress Controller Helm chart parameters](/nic/install/helm/parameters.md) - [Security recommendations](/nic/configuration/security.md) - [Basic configuration](/nic/configuration/ingress-resources/basic-configuration.md) - [Extensibility with NGINX Plus](/nic/overview/nginx-plus.md) -------------------------------------------------------------------------------- title: "Add an NGINX Ingress Controller image to your cluster" url: https://docs.nginx.com/nginx-ingress-controller/install/images/add-image-to-cluster/index.md -------------------------------------------------------------------------------- This document describes how to add an F5 NGINX Plus Ingress Controller image from the F5 Docker registry into your Kubernetes cluster using a JWT token. ## Before you begin To follow these steps, you will need the following pre-requisite: - [Create a license Secret](/nic/install/license-secret.md) You can also get the NGINX Ingress Controller image using the following alternate methods: - [Download NGINX Ingress Controller from the F5 Registry](/nic/install/images/registry-download.md) - [Build NGINX Ingress Controller](/nic/install/build.md) - For NGINX Open Source, you can pull the [nginx/nginx-ingress image](https://hub.docker.com/r/nginx/nginx-ingress/) from DockerHub ## Helm deployments If you are using Helm for deployment, there are two main methods: using a _chart_ or _source_. ### Add the image from chart The following command installs NGINX Ingress Controller with a Helm chart, passing required arguments using the `set` parameter. ```shell helm install my-release -n nginx-ingress oci://ghcr.io/nginx/charts/nginx-ingress --version --set controller.image.repository=private-registry.nginx.com/nginx-ic/nginx-plus-ingress --set controller.image.tag= --set controller.nginxplus=true --set controller.serviceAccount.imagePullSecretName=regcred ``` You can also use the certificate and key from the MyF5 portal and the Docker registry API to list the available image tags for the repositories, for example: ```shell curl https://private-registry.nginx.com/v2/nginx-ic/nginx-plus-ingress/tags/list --key --cert ``` ```json { "name": "nginx-ic/nginx-plus-ingress", "tags": [ "-alpine", "-alpine-fips", "-ubi", "" ] } ``` ```shell curl https://private-registry.nginx.com/v2/nginx-ic-nap/nginx-plus-ingress/tags/list --key --cert ``` ```json { "name": "nginx-ic-nap/nginx-plus-ingress", "tags": [ "-alpine-fips", "-ubi", "" ] } ``` ```shell curl https://private-registry.nginx.com/v2/nginx-ic-dos/nginx-plus-ingress/tags/list --key --cert ``` ```json { "name": "nginx-ic-dos/nginx-plus-ingress", "tags": [ "-ubi", "" ] } ``` The `jq` command was used in these examples to make the JSON output easier to read. ### Add the image from source The [Installation with Helm](/nic/install/helm.md#install-the-helm-chart-from-source) documentation has a section describing how to use sources: these are the unique steps for Docker secrets using JWT tokens. 1. Clone the NGINX [`kubernetes-ingress` repository](https://github.com/nginx/kubernetes-ingress). 1. Navigate to the `charts/nginx-ingress` folder of your local clone. 1. Open the `values.yaml` file in an editor. You must change a few lines NGINX Ingress Controller with NGINX Plus to be deployed. 1. Change the `nginxplus` argument to `true`. 1. Change the `repository` argument to the NGINX Ingress Controller image you intend to use. 1. Add an argument to `imagePullSecretName` or `imagePullSecretsNames` to allow Docker to pull the image from the private registry. The following code block shows snippets of the parameters you will need to change, and an example of their contents: ```yaml ## Deploys the Ingress Controller for NGINX Plus nginxplus: true ## Truncated fields ## ... ## ... image: ## The image repository for the desired NGINX Ingress Controller image repository: private-registry.nginx.com/nginx-ic/nginx-plus-ingress ## The version tag tag: serviceAccount: ## The annotations of the service account of the Ingress Controller pods. annotations: {} ## Truncated fields ## ... ## ... ## The name of the secret containing docker registry credentials. ## Secret must exist in the same namespace as the helm release. ## Note that also imagePullSecretsNames can be used here if multiple secrets need to be set. imagePullSecretName: regcred ``` With the modified `values.yaml` file, you can now use Helm to install NGINX Ingress Controller, for example: ```shell helm install nicdev01 -n nginx-ingress --create-namespace -f values.yaml . ``` The above command will install NGINX Ingress Controller in the `nginx-ingress` namespace. If the namespace does not exist, `--create-namespace` will create it. Using `-f values.yaml` tells Helm to use the `values.yaml` file that you modified earlier with the settings you want to apply for your NGINX Ingress Controller deployment. ## Manifest deployment The page ["Installation with Manifests"](/nic/install/manifests.md) explains how to install NGINX Ingress Controller using manifests. The following snippet is an example of a deployment: ```yaml spec: serviceAccountName: nginx-ingress imagePullSecrets: - name: regcred automountServiceAccountToken: true securityContext: seccompProfile: type: RuntimeDefault containers: - image: private-registry.nginx.com/nginx-ic/nginx-plus-ingress: imagePullPolicy: IfNotPresent name: nginx-plus-ingress ``` The `imagePullSecrets` and `containers.image` lines represent the Kubernetes secret, as well as the registry and version of NGINX Ingress Controller we are going to deploy. ## Download an image for local use If you need to download an image for local use (Such as to push to a different container registry), use this command: ```shell docker login private-registry.nginx.com --username= --password=none ``` Replace the contents of `` with the contents of the JWT token itself. Once you have successfully pulled the image, you can then tag it as needed. **Note:** For security, follow these practices with JSON Web Tokens (JWTs), passwords, and shell history: 1. **JWTs:** JWTs are sensitive information. Store them securely. Delete them after use to prevent unauthorized access. 1. **Shell history:** Commands that include JWTs or passwords are recorded in the history of your shell, in plain text. Clear your shell history after running such commands. For example, if you use bash, you can delete commands in your `~/.bash_history` file. Alternatively, you can run the `history -c` command to erase your shell history. Follow these practices to help ensure the security of your system and data. -------------------------------------------------------------------------------- title: "Add an NGINX Ingress Controller LTS image to your cluster" url: https://docs.nginx.com/nginx-ingress-controller/lts/install/images/add-image-to-cluster/index.md -------------------------------------------------------------------------------- This document describes how to add an F5 NGINX Plus Ingress Controller image from the F5 Docker registry into your Kubernetes cluster using a JWT token. ## Before you begin To follow these steps, you will need the following pre-requisite: - [Create a license Secret](/nic/lts/install/license-secret.md) You can also get the NGINX Ingress Controller LTS image using the following alternate methods: - [Download NGINX Ingress Controller LTS from the F5 Registry](/nic/lts/install/images/registry-download.md) ## Helm deployments If you are using Helm for deployment, there are two main methods: using a _chart_ or _source_. ### Add the image from chart The following command installs NGINX Ingress Controller LTS with a Helm chart, passing required arguments using the `set` parameter. ```shell helm install my-release -n nginx-ingress oci://ghcr.io/nginx/charts/nginx-ingress-lts --version --set controller.image.repository=private-registry.nginx.com/nginx-ic/lts/nginx-plus-ingress --set controller.image.tag= --set controller.nginxplus=true --set controller.serviceAccount.imagePullSecretName=regcred ``` You can also use the certificate and key from the MyF5 portal and the Docker registry API to list the available image tags for the repositories, for example: ```shell curl https://private-registry.nginx.com/v2/nginx-ic/lts/nginx-plus-ingress/tags/list --key --cert ``` ```json { "name": "nginx-ic/lts/nginx-plus-ingress", "tags": [ "" ] } ``` The `jq` command was used in these examples to make the JSON output easier to read. ### Add the image from source The [Installation with Helm](/nic/lts/install/helm/#install-the-helm-chart-from-source) documentation has a section describing how to use sources: these are the unique steps for Docker secrets using JWT tokens. 1. Clone the NGINX [`kubernetes-ingress` repository](https://github.com/nginx/kubernetes-ingress). 1. Navigate to the `charts/nginx-ingress` folder of your local clone. 1. Open the `values.yaml` file in an editor. You must change a few lines NGINX Ingress Controller LTS with NGINX Plus to be deployed. 1. Change the `nginxplus` argument to `true`. 1. Change the `repository` argument to the NGINX Ingress Controller LTS image you intend to use. 1. Add an argument to `imagePullSecretName` or `imagePullSecretsNames` to allow Docker to pull the image from the private registry. The following code block shows snippets of the parameters you will need to change, and an example of their contents: ```yaml ## Deploys the Ingress Controller for NGINX Plus nginxplus: true ## Truncated fields ## ... ## ... image: ## The image repository for the desired NGINX Ingress Controller LTS image repository: private-registry.nginx.com/nginx-ic/lts/nginx-plus-ingress ## The version tag tag: serviceAccount: ## The annotations of the service account of the Ingress Controller pods. annotations: {} ## Truncated fields ## ... ## ... ## The name of the secret containing docker registry credentials. ## Secret must exist in the same namespace as the helm release. ## Note that also imagePullSecretsNames can be used here if multiple secrets need to be set. imagePullSecretName: regcred ``` With the modified `values.yaml` file, you can now use Helm to install NGINX Ingress Controller LTS, for example: ```shell helm install nicdev01 -n nginx-ingress --create-namespace -f values.yaml . ``` The above command will install NGINX Ingress Controller LTS in the `nginx-ingress` namespace. If the namespace does not exist, `--create-namespace` will create it. Using `-f values.yaml` tells Helm to use the `values.yaml` file that you modified earlier with the settings you want to apply for your NGINX Ingress Controller LTS deployment. ## Manifest deployment The page ["Installation with Manifests"](/nic/lts/install/manifests.md) explains how to install NGINX Ingress Controller LTS using manifests. The following snippet is an example of a deployment: ```yaml spec: serviceAccountName: nginx-ingress imagePullSecrets: - name: regcred automountServiceAccountToken: true securityContext: seccompProfile: type: RuntimeDefault containers: - image: private-registry.nginx.com/nginx-ic/lts/nginx-plus-ingress: imagePullPolicy: IfNotPresent name: nginx-plus-ingress ``` The `imagePullSecrets` and `containers.image` lines represent the Kubernetes secret, as well as the registry and version of NGINX Ingress Controller LTS we are going to deploy. ## Download an image for local use If you need to download an image for local use (Such as to push to a different container registry), use this command: ```shell docker login private-registry.nginx.com --username= --password=none ``` Replace the contents of `` with the contents of the JWT token itself. Once you have successfully pulled the image, you can then tag it as needed. **Note:** For security, follow these practices with JSON Web Tokens (JWTs), passwords, and shell history: 1. **JWTs:** JWTs are sensitive information. Store them securely. Delete them after use to prevent unauthorized access. 1. **Shell history:** Commands that include JWTs or passwords are recorded in the history of your shell, in plain text. Clear your shell history after running such commands. For example, if you use bash, you can delete commands in your `~/.bash_history` file. Alternatively, you can run the `history -c` command to erase your shell history. Follow these practices to help ensure the security of your system and data. -------------------------------------------------------------------------------- title: "Advanced configuration with Annotations" url: https://docs.nginx.com/nginx-ingress-controller/configuration/ingress-resources/advanced-configuration-with-annotations/index.md -------------------------------------------------------------------------------- This topic explains how to enable advanced features in F5 NGINX Ingress Controller with Annotations. The Ingress resource can use basic NGINX features such as host or path-based routing and TLS termination. Advanced features like rewriting the request URI or inserting additional response headers can be enabled with Annotations. Outside of advanced features, Annotations are necessary for customizing NGINX behavior such as setting the value of connection timeouts. Customization is also available through the [ConfigMap](/nic/configuration/global-configuration/configmap-resource.md) resources: Annotations take priority. ## Using Annotations This example uses Annotations to customize the configuration for an Ingress resource: ```yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: cafe-ingress-with-annotations annotations: nginx.org/proxy-connect-timeout: "30s" nginx.org/proxy-read-timeout: "20s" nginx.org/client-max-body-size: "4m" nginx.org/server-snippets: | location / { return 302 /coffee; } spec: rules: - host: cafe.example.com http: paths: - path: /tea pathType: Prefix backend: service: name: tea-svc port: number: 80 - path: /coffee pathType: Prefix backend: service: name: coffee-svc port: number: 80 ``` ## Validation NGINX Ingress Controller validates the annotations of Ingress resources. If an Ingress is invalid, NGINX Ingress Controller will reject it: the Ingress will continue to exist in the cluster, but NGINX Ingress Controller will ignore it. You can check if NGINX Ingress Controller successfully applied the configuration for an Ingress resource. For the example `cafe-ingress-with-annotations` Ingress, you can run: ```shell kubectl describe ing cafe-ingress-with-annotations ``` ```text ... Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal AddedOrUpdated 3s nginx-ingress-controller Configuration for default/cafe-ingress-with-annotations was added or updated ``` The events section includes a Normal event with the AddedOrUpdated reason that informs us that the configuration was successfully applied. If you create an invalid Ingress, NGINX Ingress Controller will reject it and emit a Rejected event. For example, if you create an Ingress `cafe-ingress-with-annotations`, with an annotation `nginx.org/redirect-to-https` set to `yes please` instead of `true`, you will get: ```shell kubectl describe ing cafe-ingress-with-annotations ``` ```text Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning Rejected 13s nginx-ingress-controller annotations.nginx.org/redirect-to-https: Invalid value: "yes please": must be a boolean ``` Note how the events section includes a Warning event with the Rejected reason. **Note:** If you make an existing Ingress invalid, NGINX Ingress Controller will reject it and remove the corresponding configuration from NGINX. The `nginx.com/jwt-token` Ingress annotation has limited validation. ## Summary of Annotations The table below summarizes the available annotations. **Note:** Annotations that start with `nginx.com` are only supported with NGINX Plus. For session persistence, use `nginx.org/sticky-cookie-services`, which works with both NGINX and NGINX Plus. ### General customization |Annotation | ConfigMap Key | Description | Default | Example | | ---| ---| ---| ---| --- | | *nginx.org/proxy-connect-timeout* | *proxy-connect-timeout* | Sets the value of the [proxy_connect_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_connect_timeout) and [grpc_connect_timeout](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_connect_timeout) directive. | *60s* | | | *nginx.org/proxy-read-timeout* | *proxy-read-timeout* | Sets the value of the [proxy_read_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_read_timeout) and [grpc_read_timeout](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_read_timeout) directive. | *60s* | | | *nginx.org/proxy-send-timeout* | *proxy-send-timeout* | Sets the value of the [proxy_send_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_send_timeout) and [grpc_send_timeout](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_send_timeout) directive. | *60s* | | | *nginx.org/proxy-next-upstream* | N/A | Sets the value of the [proxy_next_upstream](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream) directive. | *error timeout* | *off* | | *nginx.org/proxy-next-upstream-timeout* | N/A | Sets the value of the [proxy_next_upstream_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream_timeout) directive. | *0s* | *3m* | | *nginx.org/proxy-next-upstream-tries* | N/A | Sets the value of the [proxy_next_upstream_tries](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream_tries) directive. | *0* | *5* | | *nginx.org/client-max-body-size* | *client-max-body-size* | Sets the value of the [client_max_body_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size) directive. | *1m* | | | *nginx.org/client-body-buffer-size* | *client-body-buffer-size* | Sets the value of the [client_body_buffer_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_buffer_size) directive. | N/A | | | *nginx.org/proxy-buffering* | *proxy-buffering* | Enables or disables [buffering of responses](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffering) from the proxied server. | *True* | | | *nginx.org/proxy-buffers* | *proxy-buffers* | Sets the value of the [proxy_buffers](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffers) directive. | Depends on the platform. | | | *nginx.org/proxy-buffer-size* | *proxy-buffer-size* | Sets the value of the [proxy_buffer_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffer_size) and [grpc_buffer_size](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_buffer_size) directives. | Depends on the platform. | | | *nginx.org/proxy-busy-buffers-size* | *proxy-busy-buffers-size* | Sets the value of the [proxy_busy_buffers_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_busy_buffers_size) directive. | Depends on the platform. | | | *nginx.org/proxy-max-temp-file-size* | *proxy-max-temp-file-size* | Sets the value of the [proxy_max_temp_file_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_max_temp_file_size) directive. | *1024m* | | | *nginx.org/server-tokens* | *server-tokens* | Enables or disables the [server_tokens](https://nginx.org/en/docs/http/ngx_http_core_module.html#server_tokens) directive. Additionally, with the NGINX Plus, you can specify a custom string value, including the empty string value, which disables the emission of the “Server” field. | *True* | | | *nginx.org/path-regex* | N/A | Enables regular expression modifiers for Ingress path parameter. This translates to the NGINX [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) directive. You can specify one of these values: "case_sensitive", "case_insensitive", or "exact". The annotation is applied to the entire Ingress resource and its paths. While using Master and Minion Ingresses i.e. Mergeable Ingresses, this annotation can be specified on Minion types. The `path-regex` annotation specified on Master is ignored, and has no effect on paths defined on Minions. | N/A | [path-regex](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/path-regex) | | *nginx.org/policies* | N/A | Applies one or more [Policy resources](/nic/configuration/policy-resource/before-you-begin.md) to an Ingress. Specify a comma-separated list of policy names. | N/A | *webapp-policy* | | *nginx.com/policies* | N/A | Applies one or more [Policy resources](/nic/configuration/policy-resource/before-you-begin.md) to an Ingress enabling NGINX Plus use-cases. Specify a comma-separated list of policy names. | N/A | *waf-policy* | ### URI and Header Manipulation |Annotation | ConfigMap Key | Description | Default | Example | | ---| ---| ---| ---| --- | | *nginx.org/proxy-hide-headers* | *proxy-hide-headers* | Sets the value of one or more [proxy_hide_header](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_hide_header) directives. Example: `"nginx.org/proxy-hide-headers": "header-a,header-b"` | N/A | | | *nginx.org/proxy-pass-headers* | *proxy-pass-headers* | Sets the value of one or more [proxy_pass_header](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass_header) directives. Example: `"nginx.org/proxy-pass-headers": "header-a,header-b"` | N/A | | | *nginx.org/rewrites* | N/A | Configures URI rewriting using [proxy_pass](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass) directive. | N/A | [rewrites](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/rewrites) | | *nginx.org/rewrite-target* | N/A | Configures URI rewriting using the [rewrite](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#rewrite) directive. The annotation value specifies the target path that requests should be rewritten to. Supports regex capture groups (`$1`, `$2`, etc.) when used with `nginx.org/path-regex`. Mutually exclusive with `nginx.org/rewrites`. | N/A | [rewrite-target](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/rewrite-target) | |*nginx.org/proxy-set-headers* | N/A | Enables customization of proxy headers and values using the [proxy_set_header](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_header) directive. Example: `"nginx.org/proxy-set-headers": "header-a: valueA,header-b: valueB,header-c: valueC"`. `Host` is not supported; use `nginx.org/upstream-vhost`. | N/A | [Proxy Set Headers](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/proxy-set-headers). | | *nginx.org/upstream-vhost* | N/A | Sets the `Host` header sent to the upstream to a fixed hostname, using the [proxy_set_header](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_header) directive (or [grpc_set_header](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_set_header) for gRPC backends). When not set, the upstream receives the Host header from the original request. Only a literal hostname is accepted; NGINX variables are not supported. | N/A | *example.internal* [upstream-vhost](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/upstream-vhost) | | *nginx.org/add-header* | *add-header* | Adds one or more response headers with the [add_header](https://nginx.org/en/docs/http/ngx_http_headers_module.html#add_header) directive. Use the format `Header-Name: value[:always]` and separate entries with commas. | N/A | *X-Frame-Options: DENY: always, X-Content-Type-Options: nosniff* | | *nginx.org/add-header-inherit* | *add-header-inherit* | Controls how [add_header_inherit](https://nginx.org/en/docs/http/ngx_http_headers_module.html#add_header_inherit) applies inherited response headers. Allowed values are `on`, `off`, and `merge`. | N/A | *merge* | | *nginx.org/proxy-redirect-from* | N/A | The `redirect` parameter for [proxy_redirect](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_redirect). Accepts `off`, `default`, a URL string, or a regex prefixed with `~` (case-sensitive) or `~*` (case-insensitive). Requires `nginx.org/proxy-redirect-to` when value is a URL or regex. | N/A | *http://redirect-backend-svc/v1/* or *off* | | *nginx.org/proxy-redirect-to* | N/A | The `replacement` parameter for [proxy_redirect](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_redirect). Required when `nginx.org/proxy-redirect-from` is a URL or regex; must not be set without `nginx.org/proxy-redirect-from`. | N/A | *http://cafe.example.com/coffee/* | ### Auth and SSL/TLS |Annotation | ConfigMap Key | Description | Default | Example | | ---| ---| ---| ---| --- | | *nginx.org/app-root* | N/A | Configures the application root path that the controller redirects requests for / to. Returns 302 redirect that will take precedence over other redirects. | N/A | `/` redirects to `/coffee` | | *nginx.org/redirect-to-https* | *redirect-to-https* | Sets a redirect rule based on the value of the `http_x_forwarded_proto` header on the server block to force incoming traffic to be over HTTPS. Useful when terminating SSL in a load balancer in front of NGINX Ingress Controller — see [115](https://github.com/nginx/kubernetes-ingress/issues/115). The redirect code can be configured with the `nginx.org/http-redirect-code` annotation or the `http-redirect-code` ConfigMap key. | *False* | | | *nginx.org/ssl-redirect* | *ssl-redirect* | Sets a redirect rule for all incoming HTTP traffic to force incoming traffic over HTTPS when TLS is configured. The redirect code can be configured with the `nginx.org/http-redirect-code` annotation or the `http-redirect-code` ConfigMap key. | *True* | | | *nginx.org/http-redirect-code* | *http-redirect-code* | Sets the HTTP redirect code for HTTPS redirects. Supported codes: 301, 302, 307, 308. | *301* | *307* | | *nginx.org/hsts* | *hsts* | Enables [HTTP Strict Transport Security (HSTS)](https://www.nginx.com/blog/http-strict-transport-security-hsts-and-nginx/)\ : the HSTS header is added to the responses from backends. The `preload` directive is included in the header. | *False* | | | *nginx.org/hsts-max-age* | *hsts-max-age* | Sets the value of the `max-age` directive of the HSTS header. | *2592000* (1 month) | | | *nginx.org/hsts-include-subdomains* | *hsts-include-subdomains* | Adds the `includeSubDomains` directive to the HSTS header. | *False* | | | *nginx.org/hsts-behind-proxy* | *hsts-behind-proxy* | Enables HSTS based on the value of the `http_x_forwarded_proto` request header. Should only be used when TLS termination is configured in a load balancer (proxy) in front of NGINX Ingress Controller. Note: to control redirection from HTTP to HTTPS configure the `nginx.org/redirect-to-https` annotation. | *False* | | | *nginx.org/basic-auth-secret* | N/A | Specifies a Secret resource with a user list for HTTP Basic authentication. | N/A | | | *nginx.org/basic-auth-realm* | N/A | Specifies a realm. | N/A | | | *nginx.org/ssl-ciphers* | *ssl-ciphers* | Sets the value of the [ssl_ciphers](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_ciphers) directive. | N/A | *HIGH:!aNULL:!MD5* | | *nginx.org/ssl-prefer-server-ciphers* | *ssl-prefer-server-ciphers* | Enables or disables the [ssl_prefer_server_ciphers](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_prefer_server_ciphers) directive. | *False* | | | *nginx.com/jwt-key* | N/A | Specifies a Secret resource with keys for validating JSON Web Tokens (JWTs). | N/A | [Support for JSON Web Tokens (JWTs)](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/jwt). | | *nginx.com/jwt-realm* | N/A | Specifies a realm. | N/A | [Support for JSON Web Tokens (JWTs)](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/jwt). | | *nginx.com/jwt-token* | N/A | Specifies a variable that contains a JSON Web Token. | By default, a JWT is expected in the `Authorization` header as a Bearer Token. | [Support for JSON Web Tokens (JWTs)](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/jwt). | | *nginx.com/jwt-login-url* | N/A | Specifies a URL to which a client is redirected in case of an invalid or missing JWT. | N/A | [Support for JSON Web Tokens (JWTs)](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/jwt). | ### Listeners | Annotation | ConfigMap Key | Description | Default | | ---| ---| ---| ---| | *nginx.org/listen-ports* | N/A | Configures HTTP ports that NGINX will listen on. Not supported on Ingress resources without a `host`; use the `-default-http-listener-port` command-line argument instead. | *[80]* | | *nginx.org/listen-ports-ssl* | N/A | Configures HTTPS ports that NGINX will listen on. Not supported on Ingress resources without a `host`; use the `-default-https-listener-port` command-line argument instead. | *[443]* | ### Backend services (Upstreams) |Annotation | ConfigMap Key | Description | Default | Example | | ---| ---| ---| ---| --- | | *nginx.org/lb-method* | *lb-method* | Sets the [load balancing method](/nginx/admin-guide/load-balancer/http-load-balancer.md#choosing-a-load-balancing-method). To use the round-robin method, specify `"round_robin"`. | `"random two least_conn"` | | | *nginx.org/ssl-services* | N/A | Enables HTTPS or gRPC over SSL when connecting to the endpoints of services. | N/A | [ssl-services](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/ssl-services) | | *nginx.org/grpc-services* | N/A | Enables gRPC for services. Note: requires HTTP/2 (see `http2` ConfigMap key); only works for Ingresses with TLS termination enabled. | N/A | [grpc-services](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/grpc-services) | | *nginx.org/websocket-services* | N/A | Enables WebSocket for services. | N/A | [websocket](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/websocket) | | *nginx.org/max-fails* | *max-fails* | Sets the value of the [max_fails](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#max_fails) parameter of the `server` directive. | *1* | | | *nginx.org/max-conns* | N\A | Sets the value of the [max_conns](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#max_conns) parameter of the `server` directive. | *0* | | | *nginx.org/upstream-zone-size* | *upstream-zone-size* | Sets the size of the shared memory [zone](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#zone) for upstreams. For NGINX, the special value 0 disables the shared memory zones. For NGINX Plus, shared memory zones are required and cannot be disabled. The special value 0 will be ignored. | *256K* | | | *nginx.org/fail-timeout* | *fail-timeout* | Sets the value of the [fail_timeout](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#fail_timeout) parameter of the `server` directive. | *10s* | | | *nginx.org/sticky-cookie-services* | N/A | Configures session persistence. This is the preferred annotation for session persistence configuration and works with both NGINX (since version 1.29.6) and NGINX Plus. If both `nginx.org/sticky-cookie-services` and `nginx.com/sticky-cookie-services` are set, the `nginx.org/` annotation takes precedence. *Requires NGINX >= v1.29.6*. | N/A | [session-persistence](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/session-persistence) | | *nginx.com/sticky-cookie-services* | N/A | Configures session persistence. Maintained for backward compatibility with NGINX Plus. Use `nginx.org/sticky-cookie-services` instead. | N/A | [session-persistence](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/session-persistence) | | *nginx.org/keepalive* | *keepalive* | Sets the value of the [keepalive](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive) directive. Note that `proxy_set_header Connection "";` is added to the generated configuration when the value > 0. | *0* | | | *nginx.com/health-checks* | N/A | Enables active health checks. | *False* | [health-checks](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/health-checks) | | *nginx.com/health-checks-mandatory* | N/A | Configures active health checks as mandatory. | *False* | [health-checks](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/health-checks) | | *nginx.com/health-checks-mandatory-queue* | N/A | When active health checks are mandatory, creates a queue where incoming requests are temporarily stored while NGINX Plus is checking the health of the endpoints after a configuration reload. | *0* | [health-checks](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/health-checks) | | *nginx.com/slow-start* | N/A | Sets the upstream server [slow-start period](/nginx/admin-guide/load-balancer/http-load-balancer.md#server-slow-start). By default, slow-start is activated after a server becomes [available](/nginx/admin-guide/load-balancer/http-health-check.md#passive-health-checks) or [healthy](/nginx/admin-guide/load-balancer/http-health-check.md#active-health-checks). To enable slow-start for newly-added servers, configure [mandatory active health checks](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/health-checks). | *"0s"* | | | *nginx.org/use-cluster-ip* | N/A | Enables using the Cluster IP and port of the service instead of the default behavior of using the IP and port of the pods. When this field is enabled, the fields that configure NGINX behavior related to multiple upstream servers (like `lb-method` and `next-upstream`) will have no effect, as NGINX Ingress Controller will configure NGINX with only one upstream server that will match the service Cluster IP. | *False* | | | *nginx.org/custom-http-errors* | N/A | Enables intercepting of backend responses using [proxy_intercept_errors](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_intercept_errors) and [error_page](https://nginx.org/en/docs/http/ngx_http_core_module.html#error_page) directives with matching status codes and routes the intercepted request to the Ingress's `spec.defaultBackend` so a user-provided Service can render a custom error body, while preserving the original upstream status code. | N/A | [custom-http-errors](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/custom-http-errors) | ### Rate limiting |Annotation | ConfigMap Key | Description | Default | Example | | ---| ---| ---| ---| --- | | *nginx.org/limit-req-rate* | N/A | Enables request-rate-limiting for this ingress by creating a [limit_req_zone](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_zone) and matching [limit_req](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req) for each location. All servers/locations of one ingress share the same zone. Must have unit r/s or r/m. | N/A | 200r/s | | *nginx.org/limit-req-key* | N/A | The key to which the rate limit is applied. Can contain text, variables, or a combination of them. Variables must be surrounded by ${}. | ${binary_remote_addr} | ${binary_remote_addr} | | *nginx.org/limit-req-zone-size* | N/A | Configures the size of the created [limit_req_zone](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_zone). | 10m | 20m | | *nginx.org/limit-req-delay* | N/A | Configures the delay-parameter of the [limit_req](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req) directive. | 0 | 100 | | *nginx.org/limit-req-no-delay* | N/A | Configures the nodelay-parameter of the [limit_req](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req) directive. | false | true | | *nginx.org/limit-req-burst* | N/A | Configures the burst-parameter of the [limit_req](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req) directive. | N/A | 100 | | *nginx.org/limit-req-dry-run* | N/A | Enables the dry run mode. In this mode, the rate limit is not actually applied, but the number of excessive requests is accounted as usual in the shared memory zone. | false | true | | *nginx.org/limit-req-log-level* | N/A | Sets the desired logging level for cases when the server refuses to process requests due to rate exceeding, or delays request processing. Allowed values are info, notice, warn or error. | error | info | | *nginx.org/limit-req-reject-code* | N/A | Sets the status code to return in response to rejected requests. Must fall into the range 400..599. | 429 | 503 | | *nginx.org/limit-req-scale* | N/A | Enables a constant rate-limit by dividing the configured rate by the number of nginx-ingress pods currently serving traffic. This adjustment ensures that the rate-limit remains consistent, even as the number of nginx-pods fluctuates due to autoscaling. Note: This will not work properly if requests from a client are not evenly distributed accross all ingress pods (sticky sessions, long lived TCP-Connections with many requests etc.). In such cases using [zone-sync](/nic/configuration/global-configuration/configmap-resource.md#zone-sync) instead would give better results. Enabling `zone-sync` will suppress this setting. | false | true | ### Snippets and custom templates | Annotation | ConfigMap Key | Description | Default | | ---| ---| ---| ---| | *nginx.org/location-snippets* | *location-snippets* | Sets a custom snippet in location context. | N/A | | *nginx.org/server-snippets* | *server-snippets* | Sets a custom snippet in server context. | N/A | ### F5 WAF for NGINX {#app-protect} **Note:** The App Protect annotations only work if the F5 WAF for NGINX module is [installed](/nic/integrations/app-protect-waf/installation.md). |Annotation | ConfigMap Key | Description | Default | Example | | ---| ---| ---| ---| --- | | *appprotect.f5.com/app-protect-policy* | N/A | The name of the App Protect Policy for the Ingress Resource. Format is `namespace/name`. If no namespace is specified, the same namespace of the Ingress Resource is used. If not specified but `appprotect.f5.com/app-protect-enable` is true, a default policy id applied. If the referenced policy resource does not exist, or policy is invalid, this annotation will be ignored, and the default policy will be applied. | N/A | [app-protect-waf](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/app-protect-waf) | | *appprotect.f5.com/app-protect-enable* | N/A | Enable App Protect for the Ingress Resource. | *False* | [app-protect-waf](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/app-protect-waf) | | *appprotect.f5.com/app-protect-security-log-enable* | N/A | Enable the [security log](/nginx-app-protect/troubleshooting/#app-protect-logging-overview) for App Protect. | *False* | [app-protect-waf](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/app-protect-waf) | | *appprotect.f5.com/app-protect-security-log* | N/A | The App Protect log configuration for the Ingress Resource. Format is `namespace/name`. If no namespace is specified, the same namespace as the Ingress Resource is used. If not specified the default is used which is: filter: `illegal`, format: `default`. Multiple configurations can be specified in a comma separated list. Both log configurations and destinations list (see below) must be of equal length. Configs and destinations are paired by the list indices. | N/A | [app-protect-waf](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/app-protect-waf) | | *appprotect.f5.com/app-protect-security-log-destination* | N/A | The destination of the security log. For more information check the [DESTINATION argument](/nginx-app-protect/troubleshooting/#app-protect-logging-overview). Multiple destinations can be specified in a comma-separated list. Both log configurations and destinations list (see above) must be of equal length. Configs and destinations are paired by the list indices. | *syslog:server=localhost:514* | [app-protect-waf](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/app-protect-waf) | ### App Protect DoS **Note:** The App Protect DoS annotations only work if the App Protect DoS module is [installed](/nic/integrations/app-protect-dos/installation.md). |Annotation | ConfigMap Key | Description | Default | Example | | ---| ---| ---| ---| --- | | *appprotectdos.f5.com/app-protect-dos-resource* | N/A | Enable App Protect DoS for the Ingress Resource by specifying a [DosProtectedResource](/nic/integrations/app-protect-dos/dos-protected.md). | N/A | [app-protect-dos](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/app-protect-dos) | -------------------------------------------------------------------------------- title: "Advanced configuration with Annotations" url: https://docs.nginx.com/nginx-ingress-controller/lts/configuration/ingress-resources/advanced-configuration-with-annotations/index.md -------------------------------------------------------------------------------- This topic explains how to enable advanced features in F5 NGINX Ingress Controller LTS with Annotations. The Ingress resource can use basic NGINX features such as host or path-based routing and TLS termination. Advanced features like rewriting the request URI or inserting additional response headers can be enabled with Annotations. Outside of advanced features, Annotations are necessary for customizing NGINX behavior such as setting the value of connection timeouts. Customization is also available through the [ConfigMap](/nic/lts/configuration/global-configuration/configmap-resource.md) resources: Annotations take priority. ## Using Annotations This example uses Annotations to customize the configuration for an Ingress resource: ```yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: cafe-ingress-with-annotations annotations: nginx.org/proxy-connect-timeout: "30s" nginx.org/proxy-read-timeout: "20s" nginx.org/client-max-body-size: "4m" nginx.org/server-snippets: | location / { return 302 /coffee; } spec: rules: - host: cafe.example.com http: paths: - path: /tea pathType: Prefix backend: service: name: tea-svc port: number: 80 - path: /coffee pathType: Prefix backend: service: name: coffee-svc port: number: 80 ``` ## Validation NGINX Ingress Controller LTS validates the annotations of Ingress resources. If an Ingress is invalid, NGINX Ingress Controller LTS will reject it: the Ingress will continue to exist in the cluster, but NGINX Ingress Controller LTS will ignore it. You can check if NGINX Ingress Controller LTS successfully applied the configuration for an Ingress resource. For the example `cafe-ingress-with-annotations` Ingress, you can run: ```shell kubectl describe ing cafe-ingress-with-annotations ``` ```text ... Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal AddedOrUpdated 3s nginx-ingress-controller Configuration for default/cafe-ingress-with-annotations was added or updated ``` The events section includes a Normal event with the AddedOrUpdated reason that informs us that the configuration was successfully applied. If you create an invalid Ingress, NGINX Ingress Controller LTS will reject it and emit a Rejected event. For example, if you create an Ingress `cafe-ingress-with-annotations`, with an annotation `nginx.org/redirect-to-https` set to `yes please` instead of `true`, you will get: ```shell kubectl describe ing cafe-ingress-with-annotations ``` ```text Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning Rejected 13s nginx-ingress-controller annotations.nginx.org/redirect-to-https: Invalid value: "yes please": must be a boolean ``` Note how the events section includes a Warning event with the Rejected reason. **note:** If you make an existing Ingress invalid, NGINX Ingress Controller LTS will reject it and remove the corresponding configuration from NGINX. The `nginx.com/jwt-token` Ingress annotation has limited validation. ## Summary of Annotations The table below summarizes the available annotations. ### General customization |Annotation | ConfigMap Key | Description | Default | Example | | ---| ---| ---| ---| --- | | *nginx.org/proxy-connect-timeout* | *proxy-connect-timeout* | Sets the value of the [proxy_connect_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_connect_timeout) and [grpc_connect_timeout](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_connect_timeout) directive. | *60s* | | | *nginx.org/proxy-read-timeout* | *proxy-read-timeout* | Sets the value of the [proxy_read_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_read_timeout) and [grpc_read_timeout](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_read_timeout) directive. | *60s* | | | *nginx.org/proxy-send-timeout* | *proxy-send-timeout* | Sets the value of the [proxy_send_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_send_timeout) and [grpc_send_timeout](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_send_timeout) directive. | *60s* | | | *nginx.org/proxy-next-upstream* | N/A | Sets the value of the [proxy_next_upstream](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream) directive. | *error timeout* | *off* | | *nginx.org/proxy-next-upstream-timeout* | N/A | Sets the value of the [proxy_next_upstream_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream_timeout) directive. | *0s* | *3m* | | *nginx.org/proxy-next-upstream-tries* | N/A | Sets the value of the [proxy_next_upstream_tries](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream_tries) directive. | *0* | *5* | | *nginx.org/client-max-body-size* | *client-max-body-size* | Sets the value of the [client_max_body_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size) directive. | *1m* | | | *nginx.org/client-body-buffer-size* | *client-body-buffer-size* | Sets the value of the [client_body_buffer_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_buffer_size) directive. | N/A | | | *nginx.org/proxy-buffering* | *proxy-buffering* | Enables or disables [buffering of responses](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffering) from the proxied server. | *True* | | | *nginx.org/proxy-buffers* | *proxy-buffers* | Sets the value of the [proxy_buffers](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffers) directive. | Depends on the platform. | | | *nginx.org/proxy-buffer-size* | *proxy-buffer-size* | Sets the value of the [proxy_buffer_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffer_size) and [grpc_buffer_size](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_buffer_size) directives. | Depends on the platform. | | | *nginx.org/proxy-busy-buffers-size* | *proxy-busy-buffers-size* | Sets the value of the [proxy_busy_buffers_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_busy_buffers_size) directive. | Depends on the platform. | | | *nginx.org/proxy-max-temp-file-size* | *proxy-max-temp-file-size* | Sets the value of the [proxy_max_temp_file_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_max_temp_file_size) directive. | *1024m* | | | *nginx.org/server-tokens* | *server-tokens* | Enables or disables the [server_tokens](https://nginx.org/en/docs/http/ngx_http_core_module.html#server_tokens) directive. You can also specify a custom string value, including the empty string value, which disables the emission of the “Server” field. | *True* | | | *nginx.org/path-regex* | N/A | Enables regular expression modifiers for Ingress path parameter. This translates to the NGINX [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) directive. You can specify one of these values: "case_sensitive", "case_insensitive", or "exact". The annotation is applied to the entire Ingress resource and its paths. While using Master and Minion Ingresses i.e. Mergeable Ingresses, this annotation can be specified on Minion types. The `path-regex` annotation specified on Master is ignored, and has no effect on paths defined on Minions. | N/A | [path-regex](https://github.com/nginx/kubernetes-ingress/tree/release-2026-lts/examples/ingress-resources/path-regex) | | *nginx.org/policies* | N/A | Applies one or more [Policy resources](/nic/lts/configuration/policy-resource/before-you-begin.md) to an Ingress. Specify a comma-separated list of policy names. | N/A | *webapp-policy* | ### Request URI/Header Manipulation |Annotation | ConfigMap Key | Description | Default | Example | | ---| ---| ---| ---| --- | | *nginx.org/proxy-hide-headers* | *proxy-hide-headers* | Sets the value of one or more [proxy_hide_header](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_hide_header) directives. Example: `"nginx.org/proxy-hide-headers": "header-a,header-b"` | N/A | | | *nginx.org/proxy-pass-headers* | *proxy-pass-headers* | Sets the value of one or more [proxy_pass_header](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass_header) directives. Example: `"nginx.org/proxy-pass-headers": "header-a,header-b"` | N/A | | | *nginx.org/rewrites* | N/A | Configures URI rewriting using [proxy_pass](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass) directive. | N/A | [rewrites](https://github.com/nginx/kubernetes-ingress/tree/release-2026-lts/examples/ingress-resources/rewrites) | | *nginx.org/rewrite-target* | N/A | Configures URI rewriting using the [rewrite](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#rewrite) directive. The annotation value specifies the target path that requests should be rewritten to. Supports regex capture groups (`$1`, `$2`, etc.) when used with `nginx.org/path-regex`. Mutually exclusive with `nginx.org/rewrites`. | N/A | [rewrite-target](https://github.com/nginx/kubernetes-ingress/tree/release-2026-lts/examples/ingress-resources/rewrite-target) | |*nginx.org/proxy-set-headers* | N/A | Enables customization of proxy headers and values using the [proxy_set_header](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_header) directive. Example: `"nginx.org/proxy-set-headers": "header-a: valueA,header-b: valueB,header-c: valueC"` | N/A | [Proxy Set Headers](https://github.com/nginx/kubernetes-ingress/tree/release-2026-lts/examples/ingress-resources/proxy-set-headers). | ### Auth and SSL/TLS |Annotation | ConfigMap Key | Description | Default | Example | | ---| ---| ---| ---| --- | | *nginx.org/app-root* | N/A | Configures the application root path that the controller redirects requests for / to. Returns 302 redirect that will take precedence over other redirects. | N/A | `/` redirects to `/coffee` | | *nginx.org/redirect-to-https* | *redirect-to-https* | Sets a redirect rule based on the value of the `http_x_forwarded_proto` header on the server block to force incoming traffic to be over HTTPS. Useful when terminating SSL in a load balancer in front of NGINX Ingress Controller LTS — see [115](https://github.com/nginx/kubernetes-ingress/issues/115). The redirect code can be configured with the `nginx.org/http-redirect-code` annotation or the `http-redirect-code` ConfigMap key. | *False* | | | *nginx.org/ssl-redirect* | *ssl-redirect* | Sets a redirect rule for all incoming HTTP traffic to force incoming traffic over HTTPS when TLS is configured. The redirect code can be configured with the `nginx.org/http-redirect-code` annotation or the `http-redirect-code` ConfigMap key. | *True* | | | *nginx.org/http-redirect-code* | *http-redirect-code* | Sets the HTTP redirect code for HTTPS redirects. Supported codes: 301, 302, 307, 308. | *301* | *307* | | *nginx.org/hsts* | *hsts* | Enables [HTTP Strict Transport Security (HSTS)](https://www.nginx.com/blog/http-strict-transport-security-hsts-and-nginx/)\ : the HSTS header is added to the responses from backends. The `preload` directive is included in the header. | *False* | | | *nginx.org/hsts-max-age* | *hsts-max-age* | Sets the value of the `max-age` directive of the HSTS header. | *2592000* (1 month) | | | *nginx.org/hsts-include-subdomains* | *hsts-include-subdomains* | Adds the `includeSubDomains` directive to the HSTS header. | *False* | | | *nginx.org/hsts-behind-proxy* | *hsts-behind-proxy* | Enables HSTS based on the value of the `http_x_forwarded_proto` request header. Should only be used when TLS termination is configured in a load balancer (proxy) in front of NGINX Ingress Controller LTS. Note: to control redirection from HTTP to HTTPS configure the `nginx.org/redirect-to-https` annotation. | *False* | | | *nginx.org/basic-auth-secret* | N/A | Specifies a Secret resource with a user list for HTTP Basic authentication. | N/A | | | *nginx.org/basic-auth-realm* | N/A | Specifies a realm. | N/A | | | *nginx.org/ssl-ciphers* | *ssl-ciphers* | Sets the value of the [ssl_ciphers](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_ciphers) directive. | N/A | *HIGH:!aNULL:!MD5* | | *nginx.org/ssl-prefer-server-ciphers* | *ssl-prefer-server-ciphers* | Enables or disables the [ssl_prefer_server_ciphers](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_prefer_server_ciphers) directive. | *False* | | | *nginx.com/jwt-key* | N/A | Specifies a Secret resource with keys for validating JSON Web Tokens (JWTs). | N/A | [Support for JSON Web Tokens (JWTs)](https://github.com/nginx/kubernetes-ingress/tree/release-2026-lts/examples/ingress-resources/jwt). | | *nginx.com/jwt-realm* | N/A | Specifies a realm. | N/A | [Support for JSON Web Tokens (JWTs)](https://github.com/nginx/kubernetes-ingress/tree/release-2026-lts/examples/ingress-resources/jwt). | | *nginx.com/jwt-token* | N/A | Specifies a variable that contains a JSON Web Token. | By default, a JWT is expected in the `Authorization` header as a Bearer Token. | [Support for JSON Web Tokens (JWTs)](https://github.com/nginx/kubernetes-ingress/tree/release-2026-lts/examples/ingress-resources/jwt). | | *nginx.com/jwt-login-url* | N/A | Specifies a URL to which a client is redirected in case of an invalid or missing JWT. | N/A | [Support for JSON Web Tokens (JWTs)](https://github.com/nginx/kubernetes-ingress/tree/release-2026-lts/examples/ingress-resources/jwt). | ### Listeners | Annotation | ConfigMap Key | Description | Default | | ---| ---| ---| ---| | *nginx.org/listen-ports* | N/A | Configures HTTP ports that NGINX will listen on. | *[80]* | | *nginx.org/listen-ports-ssl* | N/A | Configures HTTPS ports that NGINX will listen on. | *[443]* | ### Backend services (Upstreams) |Annotation | ConfigMap Key | Description | Default | Example | | ---| ---| ---| ---| --- | | *nginx.org/lb-method* | *lb-method* | Sets the [load balancing method](/nginx/admin-guide/load-balancer/http-load-balancer.md#choosing-a-load-balancing-method). To use the round-robin method, specify `"round_robin"`. | `"random two least_conn"` | | | *nginx.org/ssl-services* | N/A | Enables HTTPS or gRPC over SSL when connecting to the endpoints of services. | N/A | [ssl-services](https://github.com/nginx/kubernetes-ingress/tree/release-2026-lts/examples/ingress-resources/ssl-services) | | *nginx.org/grpc-services* | N/A | Enables gRPC for services. Note: requires HTTP/2 (see `http2` ConfigMap key); only works for Ingresses with TLS termination enabled. | N/A | [grpc-services](https://github.com/nginx/kubernetes-ingress/tree/release-2026-lts/examples/ingress-resources/grpc-services) | | *nginx.org/websocket-services* | N/A | Enables WebSocket for services. | N/A | [websocket](https://github.com/nginx/kubernetes-ingress/tree/release-2026-lts/examples/ingress-resources/websocket) | | *nginx.org/max-fails* | *max-fails* | Sets the value of the [max_fails](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#max_fails) parameter of the `server` directive. | *1* | | | *nginx.org/max-conns* | N\A | Sets the value of the [max_conns](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#max_conns) parameter of the `server` directive. | *0* | | | *nginx.org/upstream-zone-size* | *upstream-zone-size* | Sets the size of the shared memory [zone](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#zone) for upstreams. Shared memory zones are required and cannot be disabled. The special value 0 will be ignored. | *256K* | | | *nginx.org/fail-timeout* | *fail-timeout* | Sets the value of the [fail_timeout](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#fail_timeout) parameter of the `server` directive. | *10s* | | | *nginx.org/sticky-cookie-services* | N/A | Configures session persistence. This is the preferred annotation for session persistence configuration. If both `nginx.org/sticky-cookie-services` and `nginx.com/sticky-cookie-services` are set, the `nginx.org/` annotation takes precedence. | N/A | [session-persistence](https://github.com/nginx/kubernetes-ingress/tree/release-2026-lts/examples/ingress-resources/session-persistence) | | *nginx.com/sticky-cookie-services* | N/A | Configures session persistence. Maintained for backward compatibility. Use `nginx.org/sticky-cookie-services` instead. | N/A | [session-persistence](https://github.com/nginx/kubernetes-ingress/tree/release-2026-lts/examples/ingress-resources/session-persistence) | | *nginx.org/keepalive* | *keepalive* | Sets the value of the [keepalive](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive) directive. Note that `proxy_set_header Connection "";` is added to the generated configuration when the value > 0. | *0* | | | *nginx.com/health-checks* | N/A | Enables active health checks. | *False* | [health-checks](https://github.com/nginx/kubernetes-ingress/tree/release-2026-lts/examples/ingress-resources/health-checks) | | *nginx.com/health-checks-mandatory* | N/A | Configures active health checks as mandatory. | *False* | [health-checks](https://github.com/nginx/kubernetes-ingress/tree/release-2026-lts/examples/ingress-resources/health-checks) | | *nginx.com/health-checks-mandatory-queue* | N/A | When active health checks are mandatory, creates a queue where incoming requests are temporarily stored while NGINX Plus is checking the health of the endpoints after a configuration reload. | *0* | [health-checks](https://github.com/nginx/kubernetes-ingress/tree/release-2026-lts/examples/ingress-resources/health-checks) | | *nginx.com/slow-start* | N/A | Sets the upstream server [slow-start period](/nginx/admin-guide/load-balancer/http-load-balancer.md#server-slow-start). By default, slow-start is activated after a server becomes [available](/nginx/admin-guide/load-balancer/http-health-check.md#passive-health-checks) or [healthy](/nginx/admin-guide/load-balancer/http-health-check.md#active-health-checks). To enable slow-start for newly-added servers, configure [mandatory active health checks](https://github.com/nginx/kubernetes-ingress/tree/release-2026-lts/examples/ingress-resources/health-checks). | *"0s"* | | | *nginx.org/use-cluster-ip* | N/A | Enables using the Cluster IP and port of the service instead of the default behavior of using the IP and port of the pods. When this field is enabled, the fields that configure NGINX behavior related to multiple upstream servers (like `lb-method` and `next-upstream`) will have no effect, as NGINX Ingress Controller LTS will configure NGINX with only one upstream server that will match the service Cluster IP. | *False* | | ### Rate limiting |Annotation | ConfigMap Key | Description | Default | Example | | ---| ---| ---| ---| --- | | *nginx.org/limit-req-rate* | N/A | Enables request-rate-limiting for this ingress by creating a [limit_req_zone](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_zone) and matching [limit_req](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req) for each location. All servers/locations of one ingress share the same zone. Must have unit r/s or r/m. | N/A | 200r/s | | *nginx.org/limit-req-key* | N/A | The key to which the rate limit is applied. Can contain text, variables, or a combination of them. Variables must be surrounded by ${}. | ${binary_remote_addr} | ${binary_remote_addr} | | *nginx.org/limit-req-zone-size* | N/A | Configures the size of the created [limit_req_zone](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_zone). | 10m | 20m | | *nginx.org/limit-req-delay* | N/A | Configures the delay-parameter of the [limit_req](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req) directive. | 0 | 100 | | *nginx.org/limit-req-no-delay* | N/A | Configures the nodelay-parameter of the [limit_req](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req) directive. | false | true | | *nginx.org/limit-req-burst* | N/A | Configures the burst-parameter of the [limit_req](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req) directive. | N/A | 100 | | *nginx.org/limit-req-dry-run* | N/A | Enables the dry run mode. In this mode, the rate limit is not actually applied, but the number of excessive requests is accounted as usual in the shared memory zone. | false | true | | *nginx.org/limit-req-log-level* | N/A | Sets the desired logging level for cases when the server refuses to process requests due to rate exceeding, or delays request processing. Allowed values are info, notice, warn or error. | error | info | | *nginx.org/limit-req-reject-code* | N/A | Sets the status code to return in response to rejected requests. Must fall into the range 400..599. | 429 | 503 | | *nginx.org/limit-req-scale* | N/A | Enables a constant rate-limit by dividing the configured rate by the number of nginx-ingress pods currently serving traffic. This adjustment ensures that the rate-limit remains consistent, even as the number of nginx-pods fluctuates due to autoscaling. Note: This will not work properly if requests from a client are not evenly distributed accross all ingress pods (sticky sessions, long lived TCP-Connections with many requests etc.). In such cases using [zone-sync](/nic/lts/configuration/global-configuration/configmap-resource.md#zone-sync) instead would give better results. Enabling `zone-sync` will suppress this setting. | false | true | ### Snippets and custom templates | Annotation | ConfigMap Key | Description | Default | | ---| ---| ---| ---| | *nginx.org/location-snippets* | *location-snippets* | Sets a custom snippet in location context. | N/A | | *nginx.org/server-snippets* | *server-snippets* | Sets a custom snippet in server context. | N/A | -------------------------------------------------------------------------------- title: "Configuration" url: https://docs.nginx.com/nginx-ingress-controller/integrations/app-protect-dos/configuration/index.md -------------------------------------------------------------------------------- **Note:** Check out the complete [NGINX Ingress Controller with App Protect DoS example for VirtualServer](https://github.com/nginx/kubernetes-ingress/tree/v/examples/custom-resources/app-protect-dos) and the [NGINX Ingress Controller with App Protect DoS example for Ingress](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/app-protect-dos). ## App Protect DoS Configuration A `DosProtectedResource` is a [Custom Resource](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/) that holds the configuration of a collection of protected resources. An [Ingress](/nic/configuration/ingress-resources/basic-configuration), [VirtualServer and VirtualServerRoute](/nic/configuration/virtualserver-and-virtualserverroute-resources.md) can be protected by specifying a reference to the DosProtectedResource. 1. Create an `DosProtectedResource` Custom resource manifest. As an example: ```yaml apiVersion: appprotectdos.f5.com/v1beta1 kind: DosProtectedResource metadata: name: dos-protected spec: enable: true name: "webapp.example.com" apDosMonitor: uri: "webapp.example.com" protocol: "http1" timeout: 5 ``` 2. Enable App Protect DoS for an Ingress resource by adding an annotation to the Ingress. Set the value of the annotation to the qualified identifier(`namespace/name`) of a DosProtectedResource: ```yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: webapp-ingress annotations: appprotectdos.f5.com/app-protect-dos-resource: "default/dos-protected" ``` 3. Enable App Protect DoS on a VirtualServer resource by setting the `dos` field value to the qualified identifier(`namespace/name`) of a DosProtectedResource: ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServer metadata: name: webapp spec: host: webapp.example.com upstreams: - name: webapp service: webapp-svc port: 80 routes: - path: / dos: dos-protected action: pass: webapp ``` ## DoS Policy Configuration You can configure the policy for DoS by creating an `APDosPolicy` [Custom Resource](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/) and specifying the qualified identifier(`namespace/name`) of the `ApDosPolicy` in the `DosProtectedResource`. For example, say you want to use DoS Policy as shown below: ```json { mitigation_mode: "standard", signatures: "on", bad_actors: "on", automation_tools_detection: "on", tls_fingerprint: "on", } ``` Create an `APDosPolicy` resource with the policy defined in the `spec`, as shown below: ```yaml apiVersion: appprotectdos.f5.com/v1beta1 kind: APDosPolicy metadata: name: dospolicy spec: mitigation_mode: "standard" signatures: "on" bad_actors: "on" automation_tools_detection: "on" tls_fingerprint: "on" ``` Then add a reference in the `DosProtectedResource` to the `ApDosPolicy`: ```yaml apiVersion: appprotectdos.f5.com/v1beta1 kind: DosProtectedResource metadata: name: dos-protected spec: enable: true name: "my-dos" apDosMonitor: uri: "webapp.example.com" apDosPolicy: "default/dospolicy" ``` ## App Protect DoS Logs {#app-protect-dos-logs} You can set the [App Protect DoS Log configuration](/nap-dos/monitoring/types-of-logs.md) by creating an `APDosLogConf` [Custom Resource](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/) and specifying the qualified identifier(`namespace/name`) of the `ApDosLogConf` in the `DosProtectedResource`. For example, say you want to log state changing requests for your Ingress resources using App Protect DoS. The App Protect DoS log configuration looks like this: ```json { "filter": { "traffic-mitigation-stats": "all", "bad-actors": "top 10", "attack-signatures": "top 10" } } ``` You would add that config in the `spec` of your `APDosLogConf` resource as follows: ```yaml apiVersion: appprotectdos.f5.com/v1beta1 kind: APDosLogConf metadata: name: doslogconf spec: filter: traffic-mitigation-stats: all bad-actors: top 10 attack-signatures: top 10 ``` Then add a reference in the `DosProtectedResource` to the `APDosLogConf`: ```yaml apiVersion: appprotectdos.f5.com/v1beta1 kind: DosProtectedResource metadata: name: dos-protected spec: enable: true name: "my-dos" apDosMonitor: uri: "webapp.example.com" dosSecurityLog: enable: true apDosLogConf: "doslogconf" dosLogDest: "syslog-svc.default.svc.cluster.local:514" ``` ## Global Configuration NGINX Ingress Controller has a set of global configuration parameters that align with those available in the F5 DoS for NGINX module. See [ConfigMap keys](/nic/configuration/global-configuration/configmap-resource.md#modules) for the complete list. The App Protect parameters use the `app-protect-dos*` prefix. -------------------------------------------------------------------------------- title: "Configure F5 WAF for NGINX with NGINX Ingress Controller" url: https://docs.nginx.com/nginx-ingress-controller/integrations/app-protect-waf-v5/configuration/index.md -------------------------------------------------------------------------------- ## Overview This document explains how to use F5 NGINX Ingress Controller to configure [F5 WAF for NGINX v5](/waf/). **Note:** There are complete NGINX Ingress Controller with F5 WAF for NGINX [example resources on GitHub](https://github.com/nginx/kubernetes-ingress/tree/v/examples/custom-resources/app-protect-waf-v5). ## Global configuration NGINX Ingress Controller has global configuration parameters that match those in F5 WAF for NGINX. They are found in the [ConfigMap resource](/nic/configuration/global-configuration/configmap-resource.md#modules): the F5 WAF for NGINX parameters are prefixed with `app-protect*`. ## Enable F5 WAF for NGINX v5 F5 WAF for NGINX v5 can be enabled for VirtualServer, VirtualServerRoute, and Ingress resources through a Policy resource that references a WAF bundle or bundle source. Additional detail can be found in the [Policy Resource documentation](/nic/configuration/policy-resource/policy-reference.md#waf). **Note:** Policies that rely on NGINX subrequests (such as `ExternalAuth`, `APIKey`, `JWT` with remote JWKS fetching, `OIDC`, or `Cache` with `cacheBackgroundUpdate`) and F5 WAF for NGINX may not function as expected and may cause issues when applied together on the same route. --- ## F5 WAF for NGINX Bundles F5 WAF for NGINX bundles for VirtualServer custom resources are defined by creating policy bundles and putting them on a mounted volume accessible from NGINX Ingress Controller. Before applying a policy, a WAF policy bundle must be created, then copied to a volume mounted to `/etc/app_protect/bundles`. **Note:** Instead of manually placing bundles on a mounted volume, you can configure NGINX Ingress Controller to fetch bundles from a remote source automatically. See [Connect F5 WAF for NGINX to bundle sources](/nic/integrations/app-protect-waf-v5/bundle-sources.md) for details on NGINX One Console, NGINX Instance Manager, and HTTPS source types. **Note:** NGINX Ingress Controller supports `securityLogs` for policy bundles. Log bundles must also be copied to a volume mounted to `/etc/app_protect/bundles`. This example shows how a policy is configured by referencing a generated WAF Policy Bundle: ```yaml apiVersion: k8s.nginx.org/v1 kind: Policy metadata: name: spec: waf: enable: true apBundle: ".tgz" ``` This example shows the same policy as above but with a log bundle used for security log configuration: ```yaml apiVersion: k8s.nginx.org/v1 kind: Policy metadata: name: spec: waf: enable: true apBundle: ".tgz" securityLogs: - enable: true apLogBundle: ".tgz" logDest: "syslog:server=syslog-svc.default:514" ``` --- ## Configure NGINX Plus Ingress Controller using Virtual Server resources This example shows how to deploy NGINX Ingress Controller with NGINX Plus and F5 WAF for NGINX v5, deploy a simple web application, and then configure load balancing and WAF protection for that application using the VirtualServer resource. **Note:** You can find the files for this example on [GitHub](https://github.com/nginx/kubernetes-ingress/tree/v/examples/custom-resources/app-protect-waf-v5). ### Prerequisites 1. Follow the installation [instructions](/nic/integrations/app-protect-waf-v5/installation.md) to deploy NGINX Ingress Controller with NGINX Plus and F5 WAF for NGINX version 5. 2. Save the public IP address of NGINX Ingress Controller into a shell variable: ```shell IC_IP=XXX.YYY.ZZZ.III ``` 3. Save the HTTP port of NGINX Ingress Controller into a shell variable: ```shell IC_HTTP_PORT= ``` ### Deploy a web application Create the application deployment and service: ```shell kubectl apply -f https://raw.githubusercontent.com/nginx/kubernetes-ingress/v/examples/custom-resources/app-protect-waf-v5/webapp.yaml ``` ### Create the Syslog service Create the syslog service and pod for the F5 WAF for NGINX security logs: ```shell kubectl apply -f https://raw.githubusercontent.com/nginx/kubernetes-ingress/v/examples/custom-resources/app-protect-waf-v5/syslog.yaml ``` ### Deploy the WAF Policy **Note:** Configuration settings in the Policy resource enable WAF protection by configuring F5 WAF for NGINX with the log configuration created in the previous step. The policy bundle referenced as `your_policy_bundle_name.tgz` need to be created and placed in the `/etc/app_protect/bundles` volume first. Create and deploy the WAF policy. ```shell kubectl apply -f https://raw.githubusercontent.com/nginx/kubernetes-ingress/v/examples/custom-resources/app-protect-waf-v5/waf.yaml ``` ### Configure load balancing **Note:** VirtualServer references the `waf-policy` created in Step 3. 1. Create the VirtualServer Resource: ```shell kubectl apply -f https://raw.githubusercontent.com/nginx/kubernetes-ingress/v/examples/custom-resources/app-protect-waf-v5/virtual-server.yaml ``` ### Test the application To access the application, curl the coffee and the tea services. Use the `--resolve` option to set the Host header of a request with `webapp.example.com` 1. Send a request to the application: ```shell curl --resolve webapp.example.com:$IC_HTTP_PORT:$IC_IP http://webapp.example.com:$IC_HTTP_PORT/ ``` ```shell Server address: 10.12.0.18:80 Server name: webapp-7586895968-r26zn ``` 1. Try to send a request with a suspicious URL: ```shell curl --resolve webapp.example.com:$IC_HTTP_PORT:$IC_IP "http://webapp.example.com:$IC_HTTP_PORT/" ``` Expected output: ```text Request Rejected The requested URL was rejected. Please consult with your administrator. ... ``` ## Attach to an Ingress Save the following as `cafe.yaml`. The file defines the sample cafe application (`coffee` and `tea` Deployments and Services) and an Ingress. The `nginx.com/policies` annotation attaches the `waf-policy` Policy to every route on the Ingress. ```yaml apiVersion: apps/v1 kind: Deployment metadata: name: coffee namespace: default spec: replicas: 2 selector: matchLabels: app: coffee template: metadata: labels: app: coffee spec: containers: - name: coffee image: nginxdemos/hello:plain-text ports: - containerPort: 80 --- apiVersion: v1 kind: Service metadata: name: coffee-svc namespace: default spec: selector: app: coffee ports: - port: 80 targetPort: 80 name: http --- apiVersion: apps/v1 kind: Deployment metadata: name: tea namespace: default spec: replicas: 2 selector: matchLabels: app: tea template: metadata: labels: app: tea spec: containers: - name: tea image: nginxdemos/hello:plain-text ports: - containerPort: 80 --- apiVersion: v1 kind: Service metadata: name: tea-svc namespace: default spec: selector: app: tea ports: - port: 80 targetPort: 80 name: http --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: cafe-ingress namespace: default annotations: nginx.com/policies: "waf-policy" spec: ingressClassName: nginx rules: - host: cafe.example.com http: paths: - path: /tea pathType: Prefix backend: service: name: tea-svc port: number: 80 - path: /coffee pathType: Prefix backend: service: name: coffee-svc port: number: 80 ``` Apply the file: ```shell kubectl apply -f cafe.yaml ``` Send a normal request: ```shell curl --resolve cafe.example.com:$IC_HTTP_PORT:$IC_IP \ http://cafe.example.com:$IC_HTTP_PORT/coffee ``` Send a request that triggers the data guard violation: ```shell curl --resolve cafe.example.com:$IC_HTTP_PORT:$IC_IP \ "http://cafe.example.com:$IC_HTTP_PORT/coffee/" ``` The response body is `Request Rejected`. Under PLM, the Ingress-only App Protect annotations (`appprotect.f5.com/app-protect-policy` and `appprotect.f5.com/app-protect-security-log`) aren't supported. Use the `k8s.nginx.org/v1` Policy resource and the `nginx.com/policies` annotation instead, as shown above. ## Verify bundles on disk Confirm the compiled bundles are present in the ingress controller pod: ```shell NIC_POD=$(kubectl get pods --namespace nginx-ingress \ --selector app.kubernetes.io/name=nginx-ingress \ --output jsonpath='{.items[0].metadata.name}') kubectl exec --namespace nginx-ingress $NIC_POD --container nginx-ingress -- \ ls -ltr /etc/app_protect/bundles/ ``` Expected output: ```text total 1860 -rw------- 1 nginx nginx 1654 Aug 12 10:06 fetched_default_waf-policy_log_0.tgz -rw------- 1 nginx nginx 1898698 Aug 12 13:32 fetched_default_waf-policy_policy.tgz ``` Check the Policy status: ```shell kubectl describe policy waf-policy ``` A `State: Valid` and `Reason: AddedOrUpdated` status confirms the bundles were fetched successfully. ## Troubleshooting - **Policy status is `Warning` with reason `BundleFetchFailed`.** Run `kubectl describe appolicy --namespace security` and confirm `status.bundle.state` is `ready`. If PLM hasn't compiled the resource yet, the Policy fetch can't proceed. - **NGINX Ingress Controller reports the referenced namespace isn't watched.** If `controller.watchNamespace` is set, include the namespace that holds the `APPolicy` and `APLogConf` resources. If `controller.watchSecretNamespace` is set, include the PLM namespace so the controller can observe storage Secret rotation. -------------------------------------------------------------------------------- title: "Use Helm to Install NGINX Ingress Controller with NGINX Plus" url: https://docs.nginx.com/nginx-ingress-controller/install/helm/plus/index.md -------------------------------------------------------------------------------- This page describes how to use Helm to install F5 NGINX Ingress Controller with NGINX Plus. It explains the requirements for NGINX Ingress Controller, creating secrets, how to obtain and install the Helm chart, and what custom resource definitions (CRDs) are installed during the process. By following these instructions, you will finish with a functional NGINX Ingress Controller instance for your Kubernetes cluster. ## Before you begin - An active NGINX Plus subscription (Purchased or trial) - A [supported Kubernetes version](/nic/technical-specifications.md#supported-kubernetes-versions) - A functional Kubernetes cluster - [Helm 3.19+.](https://helm.sh/docs/intro/install) Throughout this page, you will see placeholder values indicated with angular brackets, such as **\**. Replace them accordingly for your installation. ### Quickstart If you do not require extra guidance to understand the installation process or have customization requirements, here are the steps to install NGINX Ingress Controller. You will require your NGINX Plus license JWT to follow these steps: if you do not have it, the steps are explained in the [Download your JSON web token](#download-your-json-web-token) section. **Note:** These commands should be run in the same directory as your **license.jwt** file. Create a secret for the NGINX Plus license: ```shell kubectl create secret generic nplus-license --from-file=license.jwt --type=nginx.com/license ``` Create a secret for the F5 registry: ```shell kubectl create secret docker-registry regcred \ --docker-server=private-registry.nginx.com \ --docker-username=$(cat license.jwt) \ --docker-password=none ``` Use Helm to install NGINX Ingress Controller with NGINX Plus: ```shell helm install oci://ghcr.io/nginx/charts/nginx-ingress \ --version \ --set controller.image.repository=private-registry.nginx.com/nginx-ic/nginx-plus-ingress \ --set controller.image.tag= \ --set controller.nginxplus=true \ --set controller.serviceAccount.imagePullSecretName=regcred \ --set controller.mgmt.licenseTokenSecretName=nplus-license ``` **Note:** The `edge` version **is not intended for production use**. It is intended for testing and development purposes only. If you'd like to test the latest changes in NGINX Ingress Controller before a new release, you can install the `edge` version, which is built from the `main` branch of the [NGINX Ingress Controller repository](https://github.com/nginx/kubernetes-ingress). You can install the `edge` version by specifying the `--version` flag with the value `0.0.0-edge`: ```shell helm install oci://ghcr.io/nginx/charts/nginx-ingress \ --version 0.0.0-edge \ --set controller.image.repository=private-registry.nginx.com/nginx-ic/nginx-plus-ingress \ --set controller.image.tag= \ --set controller.nginxplus=true \ --set controller.serviceAccount.imagePullSecretName=regcred \ --set controller.mgmt.licenseTokenSecretName=nplus-license ``` ## Download your JSON web token 1. Log in to [MyF5](https://my.f5.com/manage/s/). 1. Go to **My Products & Plans > Subscriptions** to see your active subscriptions. 1. Find your NGINX subscription, and select the **Subscription ID** for details. 1. Download the **JSON Web Token** file from the subscription page. ## Create license and registry secrets **Note:** The commands in the rest of this document should be run in the same directory as your **license.jwt** file. JWTs are sensitive information and should be stored securely. Delete them after use to prevent unauthorized access. Once you have obtained your license JWT, create a Kubernetes secret using `kubectl create`: ```shell kubectl create secret generic nplus-license --from-file license.jwt ``` #### Example output ```text secret/nplus-license created ``` Then create another Kubernetes secret to allow interactions with the F5 registry: ```shell kubectl create secret docker-registry regcred \ --docker-server=private-registry.nginx.com \ --docker-username=$(cat license.jwt) \ --docker-password=none ``` #### Example output ```text secret/regcred created ``` ## Install the Helm chart You have two options for installing the Helm chart: directly from the F5 registry, or using the source. ### F5 Registry To install NGINX Ingress Controller using the F5 registry, run this command with your release name: ``` helm install oci://ghcr.io/nginx/charts/nginx-ingress \ --version \ --set controller.image.repository=private-registry.nginx.com/nginx-ic/nginx-plus-ingress \ --set controller.image.tag= \ --set controller.nginxplus=true \ --set controller.serviceAccount.imagePullSecretName=regcred \ --set controller.mgmt.licenseTokenSecretName=nplus-license ``` #### Example output ```text Pulled: ghcr.io/nginx/charts/nginx-ingress: Digest: sha256:bb452d593c31b6be39f459f9604882e170227429821bac01e7ddd7da16d91ba1 NAME: h4-plus-registry LAST DEPLOYED: Fri Nov 28 14:47:15 2025 NAMESPACE: default STATUS: deployed REVISION: 1 DESCRIPTION: Install complete TEST SUITE: None NOTES: NGINX Ingress Controller has been installed. For release notes for this version please see: https://docs.nginx.com/nginx-ingress-controller/releases/ Installation and upgrade instructions: https://docs.nginx.com/nginx-ingress-controller/installation/installing-nic/installation-with-helm/ ``` ### From source To install NGINX Ingress Controller from source, first pull the chart by running this command: ```shell helm pull oci://ghcr.io/nginx/charts/nginx-ingress --untar --version ``` #### Example output ```text Pulled: ghcr.io/nginx/charts/nginx-ingress: Digest: sha256:bb452d593c31b6be39f459f9604882e170227429821bac01e7ddd7da16d91ba1 ``` Then use the `cd` command to change your working directory to _nginx-ingress_: ```shell cd nginx-ingress ``` Finally, install the chart with your release name with `helm install`: ```shell helm install . --set controller.image.repository=private-registry.nginx.com/nginx-ic/nginx-plus-ingress --set controller.nginxplus=true ``` #### Example output ```text NAME: h4-plus-source LAST DEPLOYED: Fri Nov 28 14:49:14 2025 NAMESPACE: default STATUS: deployed REVISION: 1 DESCRIPTION: Install complete TEST SUITE: None NOTES: NGINX Ingress Controller has been installed. For release notes for this version please see: https://docs.nginx.com/nginx-ingress-controller/releases/ Installation and upgrade instructions: https://docs.nginx.com/nginx-ingress-controller/installation/installing-nic/installation-with-helm/ ``` ## Verify the deployment To verify that NGINX Ingress Controller has been installed correctly, you can review `ingressclasses` with `kubectl get`: ```shell kubectl get ingressclasses ``` #### Example output ```text NAME CONTROLLER PARAMETERS AGE nginx nginx.org/ingress-controller 33s ``` ## Custom Resource Definitions When installing the chart, Helm will install the required CRDs. Without them, NGINX Ingress Controller pods will not become _Ready_. If you do not use the custom resources that require those CRDs, add the parameter `--skip-crds` in your `helm install` command. The following chart parameters should be set to `false`: - `controller.enableCustomResources` - `controller.appprotect.enable` - `controller.appprotectdos.enable` ## Next steps - [NGINX Ingress Controller Helm chart parameters](/nic/install/helm/parameters.md) - [Security recommendations](/nic/configuration/security.md) - [Basic configuration](/nic/configuration/ingress-resources/basic-configuration.md) - [Extensibility with NGINX Plus](/nic/overview/nginx-plus.md) -------------------------------------------------------------------------------- title: "Use Helm to Install NGINX Ingress Controller LTS with NGINX Plus" url: https://docs.nginx.com/nginx-ingress-controller/lts/install/helm/plus/index.md -------------------------------------------------------------------------------- This page describes how to use Helm to install F5 NGINX Ingress Controller LTS with NGINX Plus. It explains the requirements for NGINX Ingress Controller LTS, creating secrets, how to obtain and install the Helm chart, and what custom resource definitions (CRDs) are installed during the process. By following these instructions, you will finish with a functional NGINX Ingress Controller LTS instance for your Kubernetes cluster. ## Before you begin - An active NGINX Plus subscription (Purchased or trial) - A [supported Kubernetes version](/nic/lts/technical-specifications.md#supported-kubernetes-versions) - A functional Kubernetes cluster - [Helm 3.19+.](https://helm.sh/docs/intro/install) Throughout this page, you will see placeholder values indicated with angular brackets, such as **\**. Replace them accordingly for your installation. ### Quickstart If you do not require extra guidance to understand the installation process or have customization requirements, here are the steps to install NGINX Ingress Controller LTS. You will require your NGINX Plus license JWT to follow these steps: if you do not have it, the steps are explained in the [Download your JSON web token](#download-your-json-web-token) section. **note:** These commands should be run in the same directory as your **license.jwt** file. Create a secret for the NGINX Plus license: ```shell kubectl create secret generic nplus-license --from-file=license.jwt --type=nginx.com/license ``` Create a secret for the F5 registry: ```shell kubectl create secret docker-registry regcred \ --docker-server=private-registry.nginx.com \ --docker-username=$(cat license.jwt) \ --docker-password=none ``` Use Helm to install NGINX Ingress Controller LTS with NGINX Plus: ```shell helm install oci://ghcr.io/nginx/charts/nginx-ingress-lts \ --version \ --set controller.image.repository=private-registry.nginx.com/nginx-ic/lts/nginx-plus-ingress \ --set controller.image.tag= \ --set controller.nginxplus=true \ --set controller.serviceAccount.imagePullSecretName=regcred \ --set controller.mgmt.licenseTokenSecretName=nplus-license ``` ## Download your JSON web token 1. Log in to [MyF5](https://my.f5.com/manage/s/). 1. Go to **My Products & Plans > Subscriptions** to see your active subscriptions. 1. Find your NGINX subscription, and select the **Subscription ID** for details. 1. Download the **JSON Web Token** file from the subscription page. ## Create license and registry secrets **note:** The commands in the rest of this document should be run in the same directory as your **license.jwt** file. JWTs are sensitive information and should be stored securely. Delete them after use to prevent unauthorized access. Once you have obtained your license JWT, create a Kubernetes secret using `kubectl create`: ```shell kubectl create secret generic nplus-license --from-file=license.jwt --type=nginx.com/license ``` #### Example output ```text secret/nplus-license created ``` Then create another Kubernetes secret to allow interactions with the F5 registry: ```shell kubectl create secret docker-registry regcred \ --docker-server=private-registry.nginx.com \ --docker-username=$(cat license.jwt) \ --docker-password=none ``` #### Example output ```text secret/regcred created ``` ## Install the Helm chart You have two options for installing the Helm chart: directly from the F5 registry, or using the source. ### F5 Registry To install NGINX Ingress Controller LTS using the F5 registry, run this command with your release name: ``` helm install oci://ghcr.io/nginx/charts/nginx-ingress-lts \ --version \ --set controller.image.repository=private-registry.nginx.com/nginx-ic/lts/nginx-plus-ingress \ --set controller.image.tag= \ --set controller.nginxplus=true \ --set controller.serviceAccount.imagePullSecretName=regcred \ --set controller.mgmt.licenseTokenSecretName=nplus-license ``` #### Example output ```text Pulled: ghcr.io/nginx/charts/nginx-ingress: Digest: sha256:bb452d593c31b6be39f459f9604882e170227429821bac01e7ddd7da16d91ba1 NAME: h4-plus-registry LAST DEPLOYED: Fri Nov 28 14:47:15 2025 NAMESPACE: default STATUS: deployed REVISION: 1 DESCRIPTION: Install complete TEST SUITE: None NOTES: NGINX Ingress Controller LTS has been installed. For release notes for this version please see: https://docs.nginx.com/nginx-ingress-controller/releases/ Installation and upgrade instructions: https://docs.nginx.com/nginx-ingress-controller/installation/installing-nic/installation-with-helm/ ``` ### From source To install NGINX Ingress Controller LTS from source, first pull the chart by running this command: ```shell helm pull oci://ghcr.io/nginx/charts/nginx-ingress-lts --untar --version ``` #### Example output ```text Pulled: ghcr.io/nginx/charts/nginx-ingress-lts: Digest: sha256:bb452d593c31b6be39f459f9604882e170227429821bac01e7ddd7da16d91ba1 ``` Then use the `cd` command to change your working directory to _nginx-ingress_: ```shell cd nginx-ingress ``` Finally, install the chart with your release name with `helm install`: ```shell helm install . --set controller.image.repository=private-registry.nginx.com/nginx-ic/lts/nginx-plus-ingress --set controller.nginxplus=true ``` #### Example output ```text NAME: h4-plus-source LAST DEPLOYED: Fri Nov 28 14:49:14 2025 NAMESPACE: default STATUS: deployed REVISION: 1 DESCRIPTION: Install complete TEST SUITE: None NOTES: NGINX Ingress Controller LTS has been installed. For release notes for this version please see: https://docs.nginx.com/nginx-ingress-controller/releases/ Installation and upgrade instructions: https://docs.nginx.com/nginx-ingress-controller/installation/installing-nic/installation-with-helm/ ``` ## Verify the deployment To verify that NGINX Ingress Controller LTS has been installed correctly, you can review `ingressclasses` with `kubectl get`: ```shell kubectl get ingressclasses ``` #### Example output ```text NAME CONTROLLER PARAMETERS AGE nginx nginx.org/ingress-controller 33s ``` ## Custom Resource Definitions When installing the chart, Helm will install the required CRDs. Without them, NGINX Ingress Controller LTS pods will not become _Ready_. If you do not use the custom resources that require those CRDs, add the parameter `--skip-crds` in your `helm install` command. The following chart parameters should be set to `false`: - `controller.enableCustomResources` ## Next steps - [NGINX Ingress Controller LTS Helm chart parameters](/nic/lts/install/helm/parameters.md) - [Security recommendations](/nic/lts/configuration/security.md) - [Basic configuration](/nic/lts/configuration/ingress-resources/basic-configuration.md) - [Extensibility with NGINX Plus](/nic/lts/overview/nginx-plus.md) -------------------------------------------------------------------------------- title: "Policy specification" url: https://docs.nginx.com/nginx-ingress-controller/configuration/policy-resource/policy-specification/index.md -------------------------------------------------------------------------------- A `Policy` resource defines exactly one policy type under `.spec`. Supported policy types are: - `accessControl` - `rateLimit` - `apiKey` - `basicAuth` - `jwt` - `ingressMTLS` - `egressMTLS` - `oidc` - `oidcNative` - `cache` - `cors` - `waf` - `externalAuth` - `hsts` | Policy type | Description | VirtualServer / VirtualServerRoute | Ingress | | --- | --- | --- | --- | | [`accessControl`](/nic/configuration/policy-resource/policy-reference.md#accesscontrol) | Allows or denies requests based on the client IP address. | Yes | Yes, with `nginx.org/policies` | | [`cors`](/nic/configuration/policy-resource/policy-reference.md#cors) | Configures Cross-Origin Resource Sharing (CORS) headers. | Yes | Yes, with `nginx.org/policies` | | [`egressMTLS`](/nic/configuration/policy-resource/policy-reference.md#egressmtls) | Configures mutual TLS (mTLS) authentication and certificate verification for upstream connections. | Yes | Yes, with `nginx.org/policies` | | [`ingressMTLS`](/nic/configuration/policy-resource/policy-reference.md#ingressmtls) | Configures mTLS client certificate verification. | Yes | Yes, with `nginx.org/policies` | | [`waf`](/nic/configuration/policy-resource/policy-reference.md#waf) | Configures WAF and log configuration policies for [NGINX AppProtect](/nic/integrations/app-protect-waf/configuration.md). | Yes | Yes, with `nginx.com/policies` | | [`externalAuth`](/nic/configuration/policy-resource/policy-reference.md#externalauth) | Authenticates client requests using an external authentication server. | Yes | Yes, with `nginx.org/policies` | | [`rateLimit`](/nic/configuration/policy-resource/policy-reference.md#ratelimit) | Controls the request-processing rate for a defined key. | Yes | No | | [`apiKey`](/nic/configuration/policy-resource/policy-reference.md#apikey) | Authorizes requests that include a valid API key in a specified header or query parameter. | Yes | No | | [`basicAuth`](/nic/configuration/policy-resource/policy-reference.md#basicauth) | Authenticates client requests using HTTP Basic authentication credentials. | Yes | No | | [`jwt`](/nic/configuration/policy-resource/policy-reference.md#jwt-using-a-local-kubernetes-secret) | Authenticates client requests using JSON Web Tokens (JWT). Requires NGINX Plus. | Yes | No | | [`oidc`](/nic/configuration/policy-resource/policy-reference.md#oidc) | Configures NGINX Plus as a relying party for OpenID Connect (OIDC) authentication. | Yes | No | | [`oidcNative`](/nic/configuration/policy-resource/policy-reference.md#oidcnative) | Configures NGINX Plus as a relying party for OIDC authentication using the built-in native module. | Yes | Yes, with `nginx.com/policies` | | [`cache`](/nic/configuration/policy-resource/policy-reference.md#cache) | Configures proxy caching for serving cached content. | Yes | No | | [`hsts`](/nic/configuration/policy-resource/policy-reference.md#hsts) | Configures [HTTP Strict Transport Security](https://www.nginx.com/blog/http-strict-transport-security-hsts-and-nginx/) (HSTS) to enforce secure connections to the server. | Yes | No | **note:** NGINX Ingress Controller added Policy resource support for Ingress objects through the [`nginx.org/policies`](/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md) annotation in v5.4.0. ## Important rule: one policy type per resource A `Policy` resource must define exactly one policy type under `.spec`. If you need multiple behaviors, create multiple policies and reference them together. The following example is valid: ```yaml apiVersion: k8s.nginx.org/v1 kind: Policy metadata: name: allow-localhost spec: accessControl: allow: - 10.0.0.0/8 ``` The following example is **not** valid, because it defines two policy types in the same resource: ```yaml apiVersion: k8s.nginx.org/v1 kind: Policy metadata: name: invalid-policy spec: accessControl: allow: - 10.0.0.0/8 cors: allowOrigin: - https://example.com ``` ## What's next Learn how to [apply policies to resources](/nic/configuration/policy-resource/applying-policies.md). -------------------------------------------------------------------------------- title: "Policy specification" url: https://docs.nginx.com/nginx-ingress-controller/lts/configuration/policy-resource/policy-specification/index.md -------------------------------------------------------------------------------- Below is an example of a policy that allows access for clients from the subnet `10.0.0.0/8` and denies access for any other clients: ```yaml apiVersion: k8s.nginx.org/v1 kind: Policy metadata: name: allow-localhost spec: accessControl: allow: - 10.0.0.0/8 ``` |Field | Description | Type | Supported in VS/VSR | Supported in Ingress | | ---| ---| ---| --- | --- | |``accessControl`` | Allows or denies requests based on the client IP address. | [accessControl](#accesscontrol) | Yes | Yes | |``rateLimit`` | Controls the request-processing rate for a defined key. | [rateLimit](#ratelimit) | Yes | No | |``apiKey`` | Authorizes requests that include a valid API key in a specified header or query parameter. | [apiKey](#apikey) | Yes | No | |``basicAuth`` | Authenticates client requests using HTTP Basic authentication credentials. | [basicAuth](#basicauth) | Yes | No | |``jwt`` | Authenticates client requests using JSON Web Tokens. | [jwt](#jwt-using-a-local-kubernetes-secret) | Yes | No | |``ingressMTLS`` | Configures client certificate verification. | [ingressMTLS](#ingressmtls) | Yes | No | |``egressMTLS`` | Configures upstream authentication and certificate verification. | [egressMTLS](#egressmtls) | Yes | No | |``oidc`` | Configures NGINX Plus as a relying party for OpenID Connect (OIDC) authentication. | [OIDC](#oidc) | Yes | No | |``cache`` | Configures proxy caching for serving cached content. | [cache](#cache) | Yes | No | |``cors`` | Configures Cross-Origin Resource Sharing (CORS) headers. | [cors](#cors) | Yes | Yes | **note:** NGINX Ingress Controller LTS added Policy resource support for Ingress objects through the [`nginx.org/policies`](/nic/lts/configuration/ingress-resources/advanced-configuration-with-annotations.md) annotation in v5.4.0. A policy must include exactly one policy type. ## AccessControl The access control policy configures NGINX to deny or allow requests from clients with the specified IP addresses or subnets. For example, the following policy allows access for clients from the subnet `10.0.0.0/8` and denies access for any other clients: ```yaml accessControl: allow: - 10.0.0.0/8 ``` In contrast, the following policy does the opposite. It denies access for clients from `10.0.0.0/8` and allows access for any other clients: ```yaml accessControl: deny: - 10.0.0.0/8 ``` **note:** This feature uses the NGINX [ngx_http_access_module](http://nginx.org/en/docs/http/ngx_http_access_module.html). The NGINX Ingress Controller LTS access control policy supports either allow rules or deny rules, but not both, unlike the module itself. |Field | Description | Type | Required | | ---| ---| ---| --- | |``allow`` | Allows access for the specified networks or addresses. For example, ``192.168.1.1`` or ``10.1.1.0/16``. | ``[]string`` | No | |``deny`` | Denies access for the specified networks or addresses. For example, ``192.168.1.1`` or ``10.1.1.0/16``. | ``[]string`` | No | \* an accessControl must include either `allow` or `deny`. | ### AccessControl merging behavior A VirtualServer or VirtualServerRoute can reference multiple access control policies. For example, this configuration references two policies, each with a configured allow list: ```yaml policies: - name: allow-policy-one - name: allow-policy-two ``` When a resource references more than one access control policy, NGINX Ingress Controller LTS merges the contents into a single allow list or a single deny list. NGINX Ingress Controller LTS doesn't support referencing both allow and deny policies together, as shown in the following example. If a resource references both allow and deny lists, NGINX Ingress Controller LTS uses only the allow list policies. ```yaml policies: - name: deny-policy - name: allow-policy-one - name: allow-policy-two ``` ## RateLimit The rate limit policy configures NGINX to limit the processing rate of requests. For example, the following policy limits all subsequent requests from a single IP address once the rate exceeds 10 requests per second: ```yaml rateLimit: rate: 10r/s zoneSize: 10M key: ${binary_remote_addr} ``` **note:** This feature uses the NGINX [ngx_http_limit_req_module](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html). **note:** When you turn on the [zone sync feature](/nic/lts/configuration/global-configuration/configmap-resource.md#zone-sync) with NGINX Plus, NGINX Ingress Controller LTS synchronizes the rate limiting zone across all replicas in the cluster. This means all replicas know about requests that other replicas in the cluster have already rate limited. |Field | Description | Type | Required | | ---| ---| ---| --- | |``rate`` | The rate of requests permitted. The rate is specified in requests per second (r/s) or requests per minute (r/m). | ``string`` | Yes | |``key`` | The key to which the rate limit is applied. Can contain text, variables, or a combination of them. Variables must be surrounded by ``${}``. For example: ``${binary_remote_addr}``. Accepted variables are ``$binary_remote_addr``, ``$request_uri``,``$request_method``, ``$url``, ``$http_``, ``$args``, ``$arg_``, ``$cookie_``, ``$jwt_claim_``. | ``string`` | Yes | |``zoneSize`` | Size of the shared memory zone. Only positive values are allowed. Allowed suffixes are ``k`` or ``m``, if none are present ``k`` is assumed. | ``string`` | Yes | |``delay`` | The delay parameter specifies a limit at which excessive requests become delayed. If not set all excessive requests are delayed. | ``int`` | No | |``noDelay`` | Disables the delaying of excessive requests while requests are being limited. Overrides ``delay`` if both are set. | ``bool`` | No | |``burst`` | Excessive requests are delayed until their number exceeds the ``burst`` size, in which case the request is terminated with an error. | ``int`` | No | |``dryRun`` | Turns on dry run mode. In this mode, NGINX Ingress Controller LTS doesn't apply the rate limit, but it accounts for the number of excessive requests as usual in the shared memory zone. | ``bool`` | No | |``logLevel`` | Sets the desired logging level for cases when the server refuses to process requests due to rate exceeding, or delays request processing. Allowed values are ``info``, ``notice``, ``warn`` or ``error``. Default is ``error``. | ``string`` | No | |``rejectCode`` | Sets the status code to return in response to rejected requests. Must fall into the range ``400..599``. Default is ``503``. | ``int`` | No | |``scale`` | Keeps the rate limit constant by dividing the configured rate by the number of NGINX Ingress Controller LTS pods currently serving traffic. This adjustment keeps the rate limit consistent, even as the number of pods fluctuates due to autoscaling. **This doesn't work correctly if requests from a client aren't distributed evenly across all Ingress Controller pods** (for example, with sticky sessions or long-lived TCP connections with many requests). In these cases, [zone sync](/nic/lts/configuration/global-configuration/configmap-resource.md#zone-sync) gives better results. Turning on `zone-sync` suppresses this setting. | ``bool`` | No | |``condition`` | Add a condition to a rate-limit policy. | [ratelimit.condition](#ratelimitcondition) | No | **note:** For each policy referenced in a VirtualServer or its VirtualServerRoutes, NGINX Ingress Controller LTS generates a single rate limiting zone defined by the [`limit_req_zone`](http://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_zone) directive. If two VirtualServer resources reference the same policy, NGINX Ingress Controller LTS generates two different rate limiting zones, one zone per VirtualServer. ### RateLimit merging behavior A VirtualServer or VirtualServerRoute can reference multiple rate limit policies. For example, this configuration references two policies: ```yaml policies: - name: rate-limit-policy-one - name: rate-limit-policy-two ``` When a resource references more than one rate limit policy, NGINX Ingress Controller LTS configures NGINX to use all referenced rate limits. When you define multiple policies, each additional policy inherits the `dryRun`, `logLevel`, and `rejectCode` parameters from the first policy referenced (`rate-limit-policy-one`, in the example above). ## RateLimit.Condition RateLimit.Condition defines a condition for a rate limit policy. For example: ```yaml condition: jwt: claim: user_details.level match: premium default: true ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``jwt`` | defines a JWT condition to rate limit against. | [ratelimit.condition.jwt](#ratelimitconditionjwt) | No | |``variables`` | defines a Variable condition to rate limit against. | [ratelimit.condition.variables](#ratelimitconditionvariables) | No | |``default`` | sets the rate limit in this policy to be the default if no conditions are met. In a group of policies with the same condition, only one policy can be the default. | ``bool`` | No | **note:** Conditions (`jwt` or `variables`) are optional, but each policy can only have one. If conditions are used and a request doesn't match any of them, NGINX Ingress Controller LTS applies the `default` policy, if one is defined. Otherwise, if no `default` is set, the request isn't rate limited. Combine the rate limit policy with condition with one or more rate limit policies. For example, you can combine multiple rate limit policies that use [RateLimit.Condition.JWT](#ratelimitconditionjwt) to apply different tiers of rate limit based on the value of a JWT claim. For a practical example of tiered rate limiting by the value of a JWT claim, see the example in the [GitHub repository](https://github.com/nginx/kubernetes-ingress/tree/v/examples/custom-resources/rate-limit-tiered-jwt-claim/README.md). ## RateLimit.Condition.JWT RateLimit.Condition.JWT defines a condition for a rate limit by JWT claim. For example, the following condition applies a rate limit policy only to requests with a JWT claim `user_details.level` with a value `premium`: ```yaml jwt: claim: user_details.level match: premium ``` The rate limit policy applies only to requests that contain a JWT with the specified claim and value. For example, the following JWT payload matches the JWT condition: ```json { "user_details": { "level": "premium" }, "sub": "client1" } ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``claim`` | Claim is the JWT claim to be rate limit by. Nested claims should be separated by ".". | ``string`` | Yes | |``match`` | the value of the claim to match against. | ``string`` | Yes | ## RateLimit.Condition.Variables RateLimit.Condition.Variables defines a condition for a rate limit by NGINX variable. The following example defines a condition for a rate limit policy that applies only to requests with the request method with a value `GET`: ```yaml variables: - name: $request_method match: GET ``` **note:** NGINX Ingress Controller LTS currently supports only one variable at a time. |Field | Description | Type | Required | | ---| ---| ---| --- | |``name`` | the name of the NGINX variable to be rate limit by. | ``string`` | Yes | |``match`` | the value of the NGINX variable to match against. Values prefixed with the `~` character denote the following is a [regular expression](https://nginx.org/en/docs/http/ngx_http_map_module.html#map). | ``string`` | Yes | ## APIKey The API Key auth policy configures NGINX to authorize client requests based on the presence of a valid API Key in a header or query parameter specified in the policy. **note:** This feature uses the NGINX [ngx_http_auth_request_module](http://nginx.org/en/docs/http/ngx_http_auth_request_module.html) and [NGINX JavaScript (NJS)](https://nginx.org/en/docs/njs/). The policy stores API keys securely using SHA-256 hashing. When a client sends an API Key, NJS hashes it and compares it to the hashed API Key in the NGINX configuration. If the hashed keys match, the NJS subrequest issues a 204 No Content response to the `auth_request` directive, indicating successful authorization. If the client doesn't provide an API Key in the specified header or query parameter, NGINX returns a 401 Unauthorized response. If the client presents an invalid key in the expected header or query parameter, NGINX returns a 403 Forbidden response and denies access. You can use the [errorPages](/nic/lts/configuration/virtualserver-and-virtualserverroute-resources.md#errorpage) property on a route to change the default behavior for 401 or 403 errors. The policy requires at least one header or query parameter. The policy below configures NGINX Ingress Controller LTS to require the API Key `password` in the header "my-header". ```yaml apiKey: suppliedIn: header: - "my-header" clientSecret: api-key-secret ``` ```yaml apiVersion: v1 kind: Secret metadata: name: api-key-secret type: nginx.org/apikey data: client1: cGFzc3dvcmQ= # password ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``suppliedIn`` | `header` or `query`. | | Yes | |``suppliedIn.header`` | An array of headers that the API Key may appear in. | ``string[]`` | No | |``suppliedIn.query`` | An array of query params that the API Key may appear in. | ``string[]`` | No | |``clientSecret`` | The name of the Kubernetes secret that stores the API Key(s). It must be in the same namespace as the Policy resource. The secret must be of the type ``nginx.org/apikey``, and the API Key(s) must be stored in a key: val format where each key is a unique clientID and each value is a unique base64 encoded API Key | ``string`` | Yes | **important:** An APIKey policy must include at least one of the `suppliedIn.header` or `suppliedIn.query` parameters. You can also include both. ### APIKey merging behavior A VirtualServer or VirtualServerRoute can be associated with only one API Key policy per route or subroute. You can replace an API Key policy from a higher level with a different policy defined on a more specific route. For example, a VirtualServer can implement different API Key policies at various levels. In the following configuration, the server-wide `api-key-policy-server` applies to `/backend1` for authorization, because that route has no more specific policy. `/backend2` uses `api-key-policy-route`, defined at the route level. ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServer metadata: name: virtual-server spec: host: virtual-server.example.com policies: - name: api-key-policy-server upstreams: - name: backend2 service: backend2-svc port: 80 - name: backend1 service: backend1-svc port: 80 routes: - path: /backend1 action: pass: backend1 - path: /backend2 action: pass: backend2 policies: - name: api-key-policy-route ``` ## BasicAuth The basic auth policy configures NGINX to authenticate client requests using the [HTTP Basic authentication scheme](https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication). For example, the following policy rejects all requests that don't include a valid username and password combination in the HTTP header `Authentication`: ```yaml basicAuth: secret: htpasswd-secret realm: "My API" ``` **note:** This feature uses the NGINX [ngx_http_auth_basic_module](https://nginx.org/en/docs/http/ngx_http_auth_basic_module.html). |Field | Description | Type | Required | | ---| ---| ---| --- | |``secret`` | The name of the Kubernetes secret that stores the Htpasswd configuration. It must be in the same namespace as the Policy resource. The secret must be of the type ``nginx.org/htpasswd``, and the config must be stored in the secret under the key ``htpasswd``. Otherwise, NGINX Ingress Controller LTS rejects the secret as invalid. | ``string`` | Yes | |``realm`` | The realm for the basic authentication. | ``string`` | No | ### BasicAuth merging behavior A VirtualServer or VirtualServerRoute can reference multiple basic auth policies, but NGINX Ingress Controller LTS applies only the first one. It ignores every subsequent reference. For example, this configuration references two policies: ```yaml policies: - name: basic-auth-policy-one - name: basic-auth-policy-two ``` In this example, NGINX Ingress Controller LTS uses the configuration from the first policy reference, `basic-auth-policy-one`, and ignores `basic-auth-policy-two`. ## JWT using a local Kubernetes secret The JWT policy configures NGINX Plus to authenticate client requests using JSON Web Tokens. The following example policy rejects all requests that don't include a valid JWT in the HTTP header `token`: ```yaml jwt: secret: jwk-secret realm: "My API" token: $http_token ``` You can pass the JWT claims and JOSE headers to the upstream servers. For example: ```yaml action: proxy: upstream: webapp requestHeaders: set: - name: user value: ${jwt_claim_user} - name: alg value: ${jwt_header_alg} ``` This example uses the `requestHeaders` of [Action.Proxy](/nic/lts/configuration/virtualserver-and-virtualserverroute-resources.md#actionproxy) to set the values of two headers that NGINX passes to the upstream servers. The value of the `${jwt_claim_user}` variable is the `user` claim of a JWT. For other claims, use `${jwt_claim_name}`, where `name` is the name of the claim. Nested claims and claims that include a period (`.`) aren't supported. Similarly, use `${jwt_header_name}`, where `name` is the name of a header. This example uses the `alg` header. **note:** This feature uses the NGINX Plus [ngx_http_auth_jwt_module](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html). |Field | Description | Type | Required | | ---| ---| ---| --- | |``secret`` | The name of the Kubernetes secret that stores the JWK. It must be in the same namespace as the Policy resource. The secret must be of the type ``nginx.org/jwk``, and the JWK must be stored in the secret under the key ``jwk``. Otherwise, NGINX Ingress Controller LTS rejects the secret as invalid. | ``string`` | Yes | |``realm`` | The realm of the JWT. | ``string`` | Yes | |``token`` | The token specifies a variable that contains the JSON Web Token. By default the JWT is passed in the ``Authorization`` header as a Bearer Token. JWT may be also passed as a cookie or a part of a query string, for example: ``$cookie_auth_token``. Accepted variables are ``$http_``, ``$arg_``, ``$cookie_``. | ``string`` | No | ### JWT merging behavior A VirtualServer or VirtualServerRoute can reference multiple JWT policies, but NGINX Ingress Controller LTS applies only the first one. It ignores every subsequent reference. For example, this configuration references two policies: ```yaml policies: - name: jwt-policy-one - name: jwt-policy-two ``` In this example, NGINX Ingress Controller LTS uses the configuration from the first policy reference, `jwt-policy-one`, and ignores `jwt-policy-two`. ## JWT using JWKS from a remote location The JWT policy configures NGINX Plus to authenticate client requests using JSON Web Tokens. You can import the JWKS keys for a JWT policy from a URL, such as a remote server or an identity provider, so you don't have to copy and update them on the Ingress Controller pod. The following example policy rejects all requests that don't include a valid JWT in the HTTP header fetched from the identity provider: ```yaml jwt: realm: MyProductAPI token: $http_token jwksURI: keyCache: 1h ``` **note:** This feature uses the NGINX Plus directive [auth_jwt_key_request](http://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_request), part of [ngx_http_auth_jwt_module](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html). |Field | Description | Type | Required | Default | | ---| ---| ---| --- | --- | |``jwksURI`` | The remote URI where NGINX Ingress Controller LTS sends the request to retrieve the JSON Web Key set.| ``string`` | Yes | -- | |``keyCache`` | Turns on in-memory caching of JWKS (JSON Web Key Sets) obtained from the ``jwksURI`` and sets a valid time for expiration. | ``string`` | Yes | -- | |``realm`` | The realm of the JWT. | ``string`` | Yes | -- | |``token`` | The token specifies a variable that contains the JSON Web Token. By default the JWT is passed in the ``Authorization`` header as a Bearer Token. JWT may be also passed as a cookie or a part of a query string, for example: ``$cookie_auth_token``. Accepted variables are ``$http_``, ``$arg_``, ``$cookie_``. | ``string`` | No | -- | |``sniEnabled`` | Turns on SNI (Server Name Indication) for the JWT policy. Use this when the remote server requires SNI to serve the correct certificate. | ``bool`` | No | `false` | |``sniName`` | The SNI name to use when connecting to the remote server. If not set, NGINX Ingress Controller LTS uses the hostname from the ``jwksURI``. | ``string`` | No | -- | |``sslVerify`` | Turns on verification of the JWKS server SSL certificate. | ``bool`` | No | `false` | |``sslVerifyDepth`` | Sets the verification depth in the JWKS server certificates chain. | ``int`` | No | `1` | |``trustedCertSecret`` | The name of the Kubernetes secret that stores the CA certificate for JWKS server verification. It must be in the same namespace as the Policy resource. The secret must be of the type ``nginx.org/ca``, and the certificate must be stored in the secret under the key ``ca.crt``. | ``string`` | No | -- | **note:** NGINX Ingress Controller LTS turns on content caching by default for each JWT policy, with a default time of 12 hours. This improves resiliency by letting NGINX Ingress Controller LTS retrieve the JWKS (JSON Web Key Set) from the cache even after it expires. ### JWT merging behavior This behavior is similar to using a local Kubernetes secret. A VirtualServer or VirtualServerRoute can reference multiple JWT policies, but NGINX Ingress Controller LTS applies only the first one. It ignores every subsequent reference. For example, this configuration references two policies: ```yaml policies: - name: jwt-policy-one - name: jwt-policy-two ``` In this example, NGINX Ingress Controller LTS uses the configuration from the first policy reference, `jwt-policy-one`, and ignores `jwt-policy-two`. ## IngressMTLS The IngressMTLS policy configures client certificate verification. For example, the following policy verifies a client certificate using the CA certificate specified in `ingress-mtls-secret`: ```yaml ingressMTLS: clientCertSecret: ingress-mtls-secret verifyClient: "on" verifyDepth: 1 ``` Below is an example of `ingress-mtls-secret` using the secret type `nginx.org/ca`: ```yaml kind: Secret metadata: name: ingress-mtls-secret apiVersion: v1 type: nginx.org/ca data: ca.crt: ``` A VirtualServer that references an IngressMTLS policy must: - Turn on [TLS termination](/nic/lts/configuration/virtualserver-and-virtualserverroute-resources.md#virtualservertls). - Reference the policy in the VirtualServer [`spec`](/nic/lts/configuration/virtualserver-and-virtualserverroute-resources.md#virtualserver-specification). You can't reference an IngressMTLS policy in a [`route`](/nic/lts/configuration/virtualserver-and-virtualserverroute-resources.md#virtualserverroute) or in a VirtualServerRoute [`subroute`](/nic/lts/configuration/virtualserver-and-virtualserverroute-resources.md#virtualserverroutesubroute). If a resource doesn't meet these conditions, NGINX sends the `500` status code to clients. You can pass the client certificate details, including the certificate, to the upstream servers. For example: ```yaml action: proxy: upstream: webapp requestHeaders: set: - name: client-cert-subj-dn value: ${ssl_client_s_dn} # subject DN - name: client-cert value: ${ssl_client_escaped_cert} # client certificate in the PEM format (urlencoded) ``` This example uses the `requestHeaders` of [Action.Proxy](/nic/lts/configuration/virtualserver-and-virtualserverroute-resources.md#actionproxy) to set the values of the two headers that NGINX passes to the upstream servers. See the [list of embedded variables](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#variables) that `ngx_http_ssl_module` supports, which you can use to pass the client certificate details. **note:** This feature uses the NGINX [ngx_http_ssl_module](https://nginx.org/en/docs/http/ngx_http_ssl_module.html). ### Configure a certificate revocation list The IngressMTLS policy supports configuring a certificate revocation list (CRL) for your policy, in one of two ways. **note:** You can use only one of these configuration options at a time. 1. Add the `ca.crl` field to the `nginx.org/ca` secret type, which accepts a base64 encoded certificate revocation list. Example: ```yaml kind: Secret metadata: name: ingress-mtls-secret apiVersion: v1 type: nginx.org/ca data: ca.crt: ca.crl: ``` 2. Add the `crlFileName` field to your IngressMTLS policy spec with the name of the CRL file. **note:** Use this configuration option only when your CRL is larger than 1 MiB. Otherwise, use the `nginx.org/ca` secret type to manage your CRL. Example: ```yaml apiVersion: k8s.nginx.org/v1 kind: Policy metadata: name: ingress-mtls-policy spec: ingressMTLS: clientCertSecret: ingress-mtls-secret crlFileName: webapp.crl verifyClient: "on" verifyDepth: 1 ``` **important:** When you configure a CRL with the `ingressMTLS.crlFileName` field, keep this additional context in mind: - NGINX Ingress Controller LTS expects the CRL, in this case `webapp.crl`, to be in `/etc/nginx/secrets`. Add a volume mount to the NGINX Ingress Controller LTS deployment to add your CRL to `/etc/nginx/secrets`. - When you update the content of your CRL, for example after you revoke a new certificate, NGINX needs to reload to pick up the latest changes. Depending on your environment, this may require you to update the name of your CRL and apply this update to your `ingress-mtls.yaml` policy so NGINX picks up the latest CRL. See the Kubernetes documentation on [volumes](https://kubernetes.io/docs/concepts/storage/volumes/) to find the best implementation for your environment. |Field | Description | Type | Required | | ---| ---| ---| --- | |``clientCertSecret`` | The name of the Kubernetes secret that stores the CA certificate. It must be in the same namespace as the Policy resource. The secret must be of the type ``nginx.org/ca``, and the certificate must be stored in the secret under the key ``ca.crt``. Otherwise, NGINX Ingress Controller LTS rejects the secret as invalid. | ``string`` | Yes | |``verifyClient`` | Verification for the client. Possible values are ``"on"``, ``"off"``, ``"optional"``, ``"optional_no_ca"``. The default is ``"on"``. | ``string`` | No | |``verifyDepth`` | Sets the verification depth in the client certificates chain. The default is ``1``. | ``int`` | No | |``crlFileName`` | The file name of the Certificate Revocation List. NGINX Ingress Controller LTS looks for this file in `/etc/nginx/secrets`. | ``string`` | No | ### IngressMTLS merging behavior A VirtualServer can reference only a single IngressMTLS policy, and NGINX Ingress Controller LTS ignores every subsequent reference. For example, this configuration references two policies: ```yaml policies: - name: ingress-mtls-policy-one - name: ingress-mtls-policy-two ``` In this example, NGINX Ingress Controller LTS uses the configuration from the first policy reference, `ingress-mtls-policy-one`, and ignores `ingress-mtls-policy-two`. ## EgressMTLS The EgressMTLS policy configures upstream authentication and certificate verification. For example, the following policy uses `egress-mtls-secret` to authenticate with the upstream application and `egress-trusted-ca-secret` to verify the certificate of the application: ```yaml egressMTLS: tlsSecret: egress-mtls-secret trustedCertSecret: egress-trusted-ca-secret verifyServer: on verifyDepth: 2 ``` **note:** This feature uses the NGINX [ngx_http_proxy_module](https://nginx.org/en/docs/http/ngx_http_proxy_module.html). |Field | Description | Type | Required | | ---| ---| ---| --- | |``tlsSecret`` | The name of the Kubernetes secret that stores the TLS certificate and key. It must be in the same namespace as the Policy resource. The secret must be of the type ``kubernetes.io/tls``, the certificate must be stored in the secret under the key ``tls.crt``, and the key must be stored under the key ``tls.key``. Otherwise, NGINX Ingress Controller LTS rejects the secret as invalid. | ``string`` | No | |``trustedCertSecret`` | The name of the Kubernetes secret that stores the CA certificate. It must be in the same namespace as the Policy resource. The secret must be of the type ``nginx.org/ca``, and the certificate must be stored in the secret under the key ``ca.crt``. Otherwise, NGINX Ingress Controller LTS rejects the secret as invalid. | ``string`` | No | |``verifyServer`` | Turns on verification of the upstream HTTPS server certificate. | ``bool`` | No | |``verifyDepth`` | Sets the verification depth in the proxied HTTPS server certificates chain. The default is ``1``. | ``int`` | No | |``sessionReuse`` | Turns on reuse of SSL sessions to the upstreams. The default is ``true``. | ``bool`` | No | |``serverName`` | Turns on passing of the server name through the ``Server Name Indication`` extension. | ``bool`` | No | |``sslName`` | Lets you override the server name used to verify the certificate of the upstream HTTPS server. | ``string`` | No | |``ciphers`` | Specifies the enabled ciphers for requests to an upstream HTTPS server. The default is ``DEFAULT``. | ``string`` | No | |``protocols`` | Specifies the protocols for requests to an upstream HTTPS server. Accepted values are ``SSLv2``, ``SSLv3``, ``TLSv1``, ``TLSv1.1``, ``TLSv1.2``, and ``TLSv1.3`` (case-insensitive). The default is ``TLSv1 TLSv1.1 TLSv1.2``. | ``string`` | No | ### EgressMTLS merging behavior A VirtualServer or VirtualServerRoute can reference multiple EgressMTLS policies, but NGINX Ingress Controller LTS applies only the first one. It ignores every subsequent reference. For example, this configuration references two policies: ```yaml policies: - name: egress-mtls-policy-one - name: egress-mtls-policy-two ``` In this example, NGINX Ingress Controller LTS uses the configuration from the first policy reference, `egress-mtls-policy-one`, and ignores `egress-mtls-policy-two`. ## OIDC **tip:** This feature is turned off by default. To turn it on, set the [enable-oidc](/nic/lts/configuration/global-configuration/command-line-arguments.md#cmdoption-enable-oidc) command-line argument of NGINX Ingress Controller LTS. The OIDC policy configures NGINX Plus as a relying party for OpenID Connect authentication. For example, the following policy uses the client ID `nginx-plus` and the client secret `oidc-secret` to authenticate with the OpenID Connect provider `https://idp.example.com`: ```yaml spec: oidc: clientID: nginx-plus clientSecret: oidc-secret authEndpoint: https://idp.example.com/openid-connect/auth tokenEndpoint: https://idp.example.com/openid-connect/token jwksURI: https://idp.example.com/openid-connect/certs endSessionEndpoint: https://idp.example.com/openid-connect/logout postLogoutRedirectURI: / accessTokenEnable: true pkceEnable: false ``` NGINX Plus passes the ID of an authenticated user to the backend in the HTTP header `username`. **note:** This feature uses the [reference implementation](https://github.com/nginxinc/nginx-openid-connect/) of NGINX Plus as a relying party for OpenID Connect authentication. ### Prerequisites To use OIDC, turn on [zone synchronization](/nginx/admin-guide/high-availability/zone_sync.md). If you don't set up zone synchronization, NGINX Plus fails to reload. You also need to configure a resolver, which NGINX Plus uses to resolve the IDP authorization endpoint. You can find an example configuration [in the GitHub repository](https://github.com/nginx/kubernetes-ingress/blob/v/examples/custom-resources/oidc#step-7---configure-nginx-plus-zone-synchronization-and-resolver). **warning:** The configuration in the example doesn't turn on TLS, so synchronization between replicas happens in clear text. This can expose tokens. ### Limitations The OIDC policy defines a few internal locations that you can't customize: `/_jwks_uri`, `/_token`, `/_refresh`, `/_id_token_validation`, `/logout`. In addition, `/_codexch` is the default value for the redirect URI, and `/_logout` is the default value for the post logout redirect URI. You can customize both. Specifying one of these locations as a route in the VirtualServer or VirtualServerRoute causes a collision, and NGINX Plus fails to reload. |Field | Description | Type | Required | | ---| ---| ---| --- | |``clientID`` | The client ID provided by your OpenID Connect provider. | ``string`` | Yes | |``clientSecret`` | The name of the Kubernetes secret that stores the client secret provided by your OpenID Connect provider. It must be in the same namespace as the Policy resource. The secret must be of the type ``nginx.org/oidc``, and the secret stored under the key ``client-secret``. Otherwise, NGINX Ingress Controller LTS rejects the secret as invalid. If you enable PKCE, don't configure this field. | ``string`` | Yes | |``authEndpoint`` | URL for the authorization endpoint provided by your OpenID Connect provider. | ``string`` | Yes | |``authExtraArgs`` | A list of extra URL arguments to pass to the authorization endpoint provided by your OpenID Connect provider. Arguments must be URL encoded, multiple arguments may be included in the list, for example ``[ arg1=value1, arg2=value2 ]`` | ``string[]`` | No | |``tokenEndpoint`` | URL for the token endpoint provided by your OpenID Connect provider. | ``string`` | Yes | |``endSessionEndpoint`` | URL provided by your OpenID Connect provider to request the end user be logged out. | ``string`` | No | |``jwksURI`` | URL for the JSON Web Key Set (JWK) document provided by your OpenID Connect provider. | ``string`` | Yes | |``scope`` | List of OpenID Connect scopes. The scope ``openid`` always needs to be present and others can be added concatenating them with a ``+`` sign, for example ``openid+profile+email``, ``openid+email+userDefinedScope``. The default is ``openid``. | ``string`` | No | |``redirectURI`` | Lets you override the default redirect URI. The default is ``/_codexch``. | ``string`` | No | |``postLogoutRedirectURI`` | URI to redirect to after the logout has been performed. Requires ``endSessionEndpoint``. The default is ``/_logout``. | ``string`` | No | |``zoneSyncLeeway`` | Specifies the maximum timeout in milliseconds for synchronizing ID/access tokens and shared values between Ingress Controller pods. The default is ``200``. | ``int`` | No | |``accessTokenEnable`` | Option of whether Bearer token is used to authorize NGINX to access protected backend. | ``boolean`` | No | |``pkceEnable`` | Turns on Proof Key for Code Exchange. The OpenID client needs to be in public mode. `clientSecret` is not used in this mode. | ``boolean`` | No | |``sslVerify`` | Use this option to turn on TLS verification when calls are made to the IDP endpoints. | ``boolean`` | No | |``verifyDepth`` | Sets the verification depth in the proxied HTTPS server certificates chain. The default is ``1``. | ``int`` | No | |``trustedCertSecret`` | The name of the Kubernetes secret that stores the CA certificate. It must be in the same namespace as the Policy resource. The secret must be of the type ``nginx.org/ca``, and the certificate must be stored in the secret under the key ``ca.crt``. Otherwise, NGINX Ingress Controller LTS rejects the secret as invalid. | ``string`` | No | **note:** Only one OIDC policy can be referenced in a VirtualServer and its VirtualServerRoutes. However, you can still apply the same policy to different routes in the VirtualServer and VirtualServerRoutes. ### OIDC merging behavior A VirtualServer or VirtualServerRoute can reference only a single OIDC policy, and NGINX Ingress Controller LTS ignores every subsequent reference. For example, this configuration references two policies: ```yaml policies: - name: oidc-policy-one - name: oidc-policy-two ``` In this example, NGINX Ingress Controller LTS uses the configuration from the first policy reference, `oidc-policy-one`, and ignores `oidc-policy-two`. ## Cache The cache policy configures proxy caching, which improves performance by storing and serving cached responses to clients instead of proxying every request to upstream servers. For example, the following policy creates a cache zone named "mycache" with 10 MB of memory allocated, and caches all GET response codes for 30 seconds: ```yaml cache: cacheZoneName: "mycache" cacheZoneSize: "10m" allowedCodes: ["any"] allowedMethods: ["GET"] time: "30s" ``` Here's an example with more specific configuration: ```yaml cache: cacheZoneName: "mycache" cacheZoneSize: "100m" allowedCodes: [200, 301, 302] allowedMethods: ["GET", "POST"] time: "5m" levels: "1:2" overrideUpstreamCache: true inactive: "60m" useTempPath: false maxSize: "10g" minFree: "1g" manager: files: 100 sleep: "50ms" threshold: "200ms" cacheKey: "$scheme$host$request_uri" cacheUseStale: [ "error", "timeout", "updating", "http_500" ] cacheRevalidate: true cacheBackgroundUpdate: true cacheMinUses: 1 lock: enable: true timeout: "5s" age: "30s" conditions: noCache: [ "$cookie_nocache", "$arg_nocache" ] bypass: [ "$http_authorization" ] ``` **note:** This feature uses the NGINX [ngx_http_proxy_module](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_path) `proxy_cache_path` and related directives. |Field | Description | Type | Required | | --- | ---| ---| --- | |``cacheZoneName`` | CacheZoneName defines the name of the cache zone. Must start with a lowercase letter,followed by alphanumeric characters or underscores, and end with an alphanumeric character. Single lowercase letters are also allowed. Examples: "cache", "my_cache", "cache1". | ``string`` | Yes | |``cacheZoneSize`` | CacheZoneSize defines the size of the cache zone. Must be a number followed by a size unit: 'k' for kilobytes, 'm' for megabytes, or 'g' for gigabytes. Examples: "10m", "1g", "512k". | ``string`` | Yes | |``allowedCodes`` | AllowedCodes defines which HTTP response codes should be cached. Accepts either: - The string "any" to cache all response codes (must be the only element) - A list of HTTP status codes as integers (100-599) Examples: ["any"], [200, 301, 404], [200]. Invalid: ["any", 200] (cannot mix "any" with specific codes). | ``[]IntOrString`` | No | |``time`` | The default cache time for responses. Required when allowedCodes is specified. Must be a number followed by a time unit: 's' for seconds, 'm' for minutes, 'h' for hours, 'd' for days. Examples: "30s", "5m", "1h", "2d". | ``string`` | No | |``allowedMethods`` | AllowedMethods defines which HTTP methods should be cached. Only "GET", "HEAD", and "POST" are supported by the NGINX `proxy_cache_methods` directive. GET and HEAD are always cached by default even if not specified. Maximum of 3 items allowed. Examples: ["GET"], ["GET", "HEAD", "POST"]. Invalid methods: PUT, DELETE, PATCH, and so on. | ``[]string`` | No | |``levels`` | Levels defines the cache directory hierarchy levels for storing cached files. Must be in format "X:Y" or "X:Y:Z" where X, Y, Z are either 1 or 2. This controls the number of subdirectory levels and their name lengths. Examples: "1:2", "2:2", "1:2:2". Invalid: "3:1", "1:3", "1:2:3". | ``string`` | No | |``overrideUpstreamCache`` | OverrideUpstreamCache controls whether to override upstream cache headers (using the `proxy_ignore_headers` directive). When true, NGINX ignores cache-related headers from upstream servers like Cache-Control, Expires, and so on. Default: false. | ``bool`` | No | |``cachePurgeAllow`` | CachePurgeAllow defines IP addresses or CIDR blocks allowed to purge cache. Examples: ["192.168.1.100", "10.0.0.0/8", "::1"]. | ``[]string`` | No | |``cacheKey`` | CacheKey defines a key for caching (`proxy_cache_key`). By default, "$scheme$proxy_host$uri". Must not contain command execution patterns: $(, `, ;, &&, || | ``string`` | No | |``cacheUseStale`` | CacheUseStale determines in which cases a stale cached response can be used (`proxy_cache_use_stale`). Valid parameters: error, timeout, invalid_header, updating, http_500, http_502, http_503, http_504, http_403, http_404, http_429, off. | ``[]string`` | No | |``cacheRevalidate`` | CacheRevalidate turns on revalidation of expired cache items using conditional requests (`proxy_cache_revalidate`). Uses "If-Modified-Since" and "If-None-Match" header fields. | ``bool`` | No | |``cacheBackgroundUpdate`` | CacheBackgroundUpdate lets NGINX start a background subrequest to update an expired cache item (`proxy_cache_background_update`). NGINX returns a stale cached response to the client while it updates the cache. | ``bool`` | No | |``cacheMinUses`` | CacheMinUses sets the number of requests after which NGINX Ingress Controller LTS caches the response (`proxy_cache_min_uses`). | ``integer`` | No | |``inactive`` | Inactive sets the time after which cached data that are not accessed get removed from the cache (inactive parameter). By default, inactive is set to 10 minutes. | ``string`` | No | |``maxSize`` | MaxSize sets the maximum cache size (max_size parameter). When the size is exceeded, the cache manager removes the least recently used data. | ``string`` | No | |``minFree`` | MinFree sets the minimum amount of free space required on the file system with cache (min_free parameter). When there is not enough free space, the cache manager removes the least recently used data. | ``string`` | No | |``useTempPath`` | UseTempPath controls whether temporary files and the cache are put on different file systems (use_temp_path parameter). If set to false, NGINX puts temporary files directly in the cache directory (use_temp_path=off). Default: false (use_temp_path=off, which puts temp files directly in the cache directory for better performance). | ``bool`` | No | |``manager`` | Manager configures the cache manager process parameters (manager_files, manager_sleep, manager_threshold). | ``object`` | No | |``manager.files`` | Files sets the maximum number of files that the cache manager deletes in one iteration. During one iteration, the cache manager deletes no more than manager_files items (by default, 100). | ``integer`` | No | |``manager.sleep`` | Sleep sets the pause between cache manager iterations. Between iterations, a pause configured by manager_sleep (by default, 50 milliseconds) is made. | ``string`` | No | |``manager.threshold`` | Threshold sets the maximum duration of one cache manager iteration. The duration of one iteration is limited by manager_threshold (by default, 200 milliseconds). | ``string`` | No | |``lock`` | Lock configures cache locking to prevent multiple identical requests from populating the same cache element simultaneously. | ``object`` | No | |``lock.enable`` | Enable sets whether cache locking is turned on (`proxy_cache_lock`). When on, only one request at a time can populate a new cache element according to the `proxy_cache_key`. | ``bool`` | No | |``lock.timeout`` | Timeout sets a timeout for `proxy_cache_lock`. When the time expires, NGINX passes the request to the proxied server, but it doesn't cache the response. | ``string`` | No | |``lock.age`` | Age sets the maximum time a cache lock can be held (`proxy_cache_lock_age`). If the last request passed to the proxied server for populating a new cache element hasn't completed within the specified time, NGINX may pass one more request. | ``string`` | No | |``conditions`` | Conditions defines when responses should not be cached or taken from cache. | ``object`` | No | |``conditions.noCache`` | NoCache defines conditions under which the response won't be saved to a cache (`proxy_no_cache`). If at least one value of the string parameters isn't empty and isn't equal to "0", NGINX doesn't save the response. | ``[]string`` | No | |``conditions.bypass`` | Bypass defines conditions under which the response won't be taken from a cache (`proxy_cache_bypass`). If at least one value of the string parameters isn't empty and isn't equal to "0", NGINX doesn't take the response from the cache. | ``[]string`` | No | ### Cache merging behavior A VirtualServer or VirtualServerRoute can reference multiple cache policies, but NGINX Ingress Controller LTS applies only the first one. It ignores every subsequent reference. ## CORS The CORS policy configures Cross-Origin Resource Sharing (CORS) headers. **note:** This feature uses the NGINX `add_header` directive. Below is an example of a CORS policy configuring all the available options: ```yaml apiVersion: k8s.nginx.org/v1 kind: Policy metadata: name: cors-policy spec: cors: allowOrigin: - "https://test.example.com" - "https://app.example.com" - "https://admin.example.com" allowMethods: - "GET" - "POST" - "PUT" allowHeaders: - "Content-Type" - "Authorization" - "X-Requested-With" - "X-API-Key" allowCredentials: true exposeHeaders: - "X-Total-Count" - "X-Page-Size" - "X-RateLimit-Remaining" - "X-RateLimit-Reset" maxAge: 3600 ``` |Field | Description | Type | Required | | --- | ---| ---| --- | |``allowOrigin`` | AllowOrigin defines the origins that are allowed to make cross-origin requests. Can be exact domains, single wildcards, or `*` for all origins. Examples: ["https://example.com", "https://*.mydomain.com", "*"] Security: When allowCredentials is true, wildcard "*" is not allowed. The server must specify explicit origins for credentialed requests. |``array[string]`` | Yes | |``allowMethods`` | AllowMethods defines the HTTP methods that are allowed for cross-origin requests. | ``array[string]`` | No | |``allowHeaders`` | AllowHeaders defines the headers that are allowed in cross-origin requests. Common safe headers: ["Accept", "Accept-Language", "Content-Language", "Content-Type"] Custom headers: ["Authorization", "X-Requested-With", "X-Custom-Header"] | ``array[string]`` | No | |``allowCredentials`` | AllowCredentials indicates whether the response to the request can be exposed when the credentials flag is true. When used as part of a response to a preflight request, this indicates whether the actual request can be made using credentials. | ``boolean`` | No | |``exposeHeaders`` | ExposeHeaders defines the headers that browsers are allowed to access. Use this field to expose additional custom headers to the browser. Example: ["X-Total-Count", "X-Page-Size", "X-RateLimit-Remaining"] Note: Set-Cookie headers cannot be exposed through CORS per official MDN specification. | ``array[string]`` | No | |``maxAge`` | MaxAge defines how long (in seconds) the results of a preflight request can be cached. Default: 86400 (24 hours). | ``integer`` | No | **note:** If CORS is currently configured in deployments using `snippets` or `responseHeaders.add`, migrate those settings to the CORS policy and remove the duplicate configuration. ### CORS merging behavior A VirtualServer or VirtualServerRoute can reference multiple CORS policies, but NGINX Ingress Controller LTS applies only the first one. It ignores every subsequent reference. ## What's next Learn how to [manage Policy resources with kubectl](/nic/lts/configuration/policy-resource/using-policy.md). -------------------------------------------------------------------------------- title: "Technical specifications" url: https://docs.nginx.com/nginx-ingress-controller/lts/technical-specifications/index.md -------------------------------------------------------------------------------- This page describes technical specifications for F5 NGINX Ingress Controller LTS, such as its version compatibility with Kubernetes and other NGINX software. ## Supported NGINX Ingress Controller LTS versions NGINX Ingress Controller LTS is a feature-frozen release supported for 36 months from the date of release. Security patches and stability bug fixes are delivered as patch revisions throughout the support period. We test NGINX Ingress Controller LTS on a range of Kubernetes platforms for each release, and list them in the [Changelog](/nic/lts/changelog). We provide technical support for NGINX Ingress Controller LTS on any Kubernetes platform that is currently supported by its provider, and that passes the [Kubernetes conformance tests](https://www.cncf.io/certification/software-conformance/). We provide technical support for F5 customers using NGINX Ingress Controller LTS for 36 months from the date of release. NGINX Ingress Controller supports the following versions of [NGINX Plus](/nginx/) and [Kubernetes](https://kubernetes.io/): | NIC LTS version | Kubernetes versions tested | NIC Helm Chart version | NGINX Plus version | End of Technical Support | | --- | --- | --- | --- | --- | | | 1.30 - 1.37 | | R37.0.6.2 | June 4, 2029 | ## Supported Docker images We provide the following Docker images, which include NGINX Plus bundled with the Ingress Controller binary. ### Images with NGINX Plus for LTS 2026 NGINX Ingress Controller LTS image include NGINX Plus LTS R37.0 | Name | Base image |
Additional modules
| F5 Container Registry Image | Architectures | Based on CR | | ---| ---| --- | --- | --- | --- | |Debian-based image | ``debian:13-slim`` | NJS (NGINX JavaScript)
OpenTelemetry
Agent (NGINX Agent 3) | ``private-registry.nginx.com/nginx-ic/lts/nginx-plus-ingress: `` | arm64
amd64 | 5.4.3 | -------------------------------------------------------------------------------- title: "Technical specifications" url: https://docs.nginx.com/nginx-ingress-controller/technical-specifications/index.md -------------------------------------------------------------------------------- This page describes technical specifications for F5 NGINX Ingress Controller, such as its version compatibility with Kubernetes and other NGINX software. ## Supported NGINX Ingress Controller versions We recommend using the latest release of NGINX Ingress Controller, and provides software updates for the most recent release. We test NGINX Ingress Controller on a range of Kubernetes platforms for each release, and list them in the [Changelog](/nic/changelog). We provide technical support for NGINX Ingress Controller on any Kubernetes platform that is currently supported by its provider, and that passes the [Kubernetes conformance tests](https://www.cncf.io/certification/software-conformance/). We provide technical support for F5 customers who are using the most recent version of NGINX Ingress Controller, and any version released within two years of the current release. NGINX Ingress Controller supports the following versions of [NGINX Plus](/nginx/) and [Kubernetes](https://kubernetes.io/): | NIC version | Kubernetes versions tested | NIC Helm Chart version | NIC Operator version | NGINX / NGINX Plus version | | --- | --- | --- | --- | --- | | | 1.30 - 1.37 | | | 1.31.6 / R37.1.1.2 | | 5.5.4 | 1.29 - 1.36 | 2.6.4 | 3.6.5 | 1.31.3 / R37.0.3.1 | | 5.4.3 | 1.28 - 1.35 | 2.5.3 | 3.5.4 | 1.31.1 / R37.0.1.1 | | 5.3.4 | 1.27 - 1.35 | 2.4.4 | 3.4.4 | 1.29.5 / R36 P2 | | 5.2.1 | 1.27 - 1.34 | 2.3.1 | 3.3.1 | 1.29.1 / R35 | | 5.1.1 | 1.25 - 1.33 | 2.2.2 | 3.2.3 | 1.29.1 / R35 | | 5.0.0 | 1.25 - 1.32 | 2.1.0 | 3.1.0 | 1.27.4 / R34 | | 4.0.1 | 1.25 - 1.32 | 2.0.1 | 3.0.1 | 1.27.4 / R33 P2 | | 3.7.2 | 1.25 - 1.31 | 1.4.2 | 2.4.2 | 1.27.2 / R32 P1 | ## Supported Docker images We provide the following Docker images, which include NGINX or NGINX Plus bundled with the Ingress Controller binary. ### Images with NGINX _All images include NGINX 1.31.6._ |
Name
|
Base image
| DockerHub image | Architectures | | ---| --- | --- | --- | |Alpine-based image | ``alpine:3.24`` | ``nginx/nginx-ingress:-alpine`` | arm64
amd64 | |Debian-based image | ``debian:13-slim`` | ``nginx/nginx-ingress:`` | arm64
amd64 | |Ubi-based image | ``redhat/ubi10-minimal`` | ``nginx/nginx-ingress:-ubi`` | arm64
amd64 | ### Images with NGINX Plus NGINX Plus images include NGINX Plus R37.1.1.2. #### F5 Container registry NGINX Plus images are available through the F5 Container registry `private-registry.nginx.com`, explained in the [Download NGINX Ingress Controller from the F5 Registry](/nic/install/images/registry-download.md) and [Add an NGINX Ingress Controller image to your cluster](/nic/install/images/add-image-to-cluster.md) topics. | Name | Base image |
Additional modules
| F5 Container Registry Image | Architectures | | ---| ---| --- | --- | --- | |Alpine-based image | ``alpine:3.24`` | NJS (NGINX JavaScript)
OpenTelemetry
NGINX Agent 3 | `nginx-ic/nginx-plus-ingress:-alpine` | arm64
amd64 | |Alpine-based image with FIPS inside | ``alpine:3.24`` | NJS (NGINX JavaScript)
OpenTelemetry
FIPS module and OpenSSL configuration
NGINX Agent 3 | `nginx-ic/nginx-plus-ingress:-alpine-fips` | arm64
amd64 | |Alpine-based image with F5 WAF for NGINX & FIPS inside | ``alpine:3.24`` | F5 WAF for NGINX
NJS (NGINX JavaScript)
OpenTelemetry
FIPS module and OpenSSL configuration
NGINX Agent 2 | `nginx-ic-nap/nginx-plus-ingress:-alpine-fips` | amd64 | |Alpine-based image with F5 WAF for NGINX & FIPS inside and NGINX Agent 3 | ``alpine:3.24`` | F5 WAF for NGINX
NJS (NGINX JavaScript)
OpenTelemetry
FIPS module and OpenSSL configuration
NGINX Agent 3 | `nginx-ic-nap/nginx-plus-ingress:-alpine-fips-agent` | amd64 | |Alpine-based image with F5 WAF for NGINX v5 & FIPS inside | ``alpine:3.24`` | F5 WAF for NGINX v5
NJS (NGINX JavaScript)
OpenTelemetry
FIPS module and OpenSSL configuration
NGINX Agent 2 | `nginx-ic-nap-v5/nginx-plus-ingress:-alpine-fips` | amd64 | |Alpine-based image with F5 WAF for NGINX v5 & FIPS inside and NGINX Agent 3 | ``alpine:3.24`` | F5 WAF for NGINX v5
NJS (NGINX JavaScript)
OpenTelemetry
FIPS module and OpenSSL configuration
NGINX Agent 3 | `nginx-ic-nap-v5/nginx-plus-ingress:-alpine-fips-agent` | amd64 | |Debian-based image | ``debian:13-slim`` | NJS (NGINX JavaScript)
OpenTelemetry
NGINX Agent 3 | `nginx-ic/nginx-plus-ingress:` | arm64
amd64 | |Debian-based image with F5 WAF for NGINX | ``debian:13-slim`` | F5 WAF for NGINX
NJS (NGINX JavaScript)
OpenTelemetry
NGINX Agent 2 | `nginx-ic-nap/nginx-plus-ingress:` | amd64 | |Debian-based image with F5 WAF for NGINX and NGINX Agent 3 | ``debian:13-slim`` | F5 WAF for NGINX
NJS (NGINX JavaScript)
OpenTelemetry
NGINX Agent 3 | `nginx-ic-nap/nginx-plus-ingress:-agent` | amd64 | |Debian-based image with F5 WAF for NGINX v5 | ``debian:13-slim`` | F5 WAF for NGINX v5
NJS (NGINX JavaScript)
OpenTelemetry
NGINX Agent 2 | `nginx-ic-nap-v5/nginx-plus-ingress:` | amd64 | |Debian-based image with F5 WAF for NGINX v5 and NGINX Agent 3 | ``debian:13-slim`` | F5 WAF for NGINX v5
NJS (NGINX JavaScript)
OpenTelemetry
NGINX Agent 3 | `nginx-ic-nap-v5/nginx-plus-ingress:-agent` | amd64 | |Debian-based image with F5 DoS for NGINX | ``debian:13-slim`` | F5 DoS for NGINX
NJS (NGINX JavaScript)
OpenTelemetry
NGINX Agent 3 | `nginx-ic-dos/nginx-plus-ingress:` | amd64 | |Debian-based image with F5 WAF for NGINX and F5 DoS for NGINX | ``debian:13-slim`` | F5 WAF for NGINX and F5 DoS for NGINX
NJS (NGINX JavaScript)
OpenTelemetry
NGINX Agent 2 | `nginx-ic-nap-dos/nginx-plus-ingress:` | amd64 | |Debian-based image with F5 WAF for NGINX, F5 DoS for NGINX and NGINX Agent 3 | ``debian:13-slim`` | F5 WAF for NGINX and F5 DoS for NGINX
NJS (NGINX JavaScript)
OpenTelemetry
NGINX Agent 3 | `nginx-ic-nap-dos/nginx-plus-ingress:-agent` | amd64 | |Ubi-based image | ``redhat/ubi10-minimal`` | NJS (NGINX JavaScript)
OpenTelemetry
NGINX Agent 3 | `nginx-ic/nginx-plus-ingress:-ubi` | arm64
amd64 | |Ubi-based image with F5 WAF for NGINX | ``redhat/ubi10-minimal`` | F5 WAF for NGINX
NJS (NGINX JavaScript)
OpenTelemetry
NGINX Agent 2 | `nginx-ic-nap/nginx-plus-ingress:-ubi` | amd64 | |Ubi-based image with F5 WAF for NGINX and NGINX Agent 3 | ``redhat/ubi10-minimal`` | F5 WAF for NGINX
NJS (NGINX JavaScript)
OpenTelemetry
NGINX Agent 3 | `nginx-ic-nap/nginx-plus-ingress:-ubi-agent` | amd64 | |Ubi-based image with F5 WAF for NGINX v5 | ``redhat/ubi10-minimal`` | F5 WAF for NGINX v5
NJS (NGINX JavaScript)
OpenTelemetry
NGINX Agent 2 | `nginx-ic-nap-v5/nginx-plus-ingress:-ubi` | amd64 | |Ubi-based image with F5 WAF for NGINX v5 and NGINX Agent 3 | ``redhat/ubi10-minimal`` | F5 WAF for NGINX v5
NJS (NGINX JavaScript)
OpenTelemetry
NGINX Agent 3 | `nginx-ic-nap-v5/nginx-plus-ingress:-ubi-agent` | amd64 | |Ubi-based image with F5 DoS for NGINX | ``redhat/ubi10-minimal`` | F5 DoS for NGINX
NJS (NGINX JavaScript)
OpenTelemetry
NGINX Agent 3 | `nginx-ic-dos/nginx-plus-ingress:-ubi` | amd64 | |Ubi-based image with F5 WAF for NGINX and F5 DoS for NGINX | ``redhat/ubi10-minimal`` | F5 WAF for NGINX and F5 DoS for NGINX
NJS (NGINX JavaScript)
OpenTelemetry
NGINX Agent 2 | `nginx-ic-nap-dos/nginx-plus-ingress:-ubi` | amd64 | |Ubi-based image with F5 WAF for NGINX, F5 DoS for NGINX and NGINX Agent 3 | ``redhat/ubi10-minimal`` | F5 WAF for NGINX and F5 DoS for NGINX
NJS (NGINX JavaScript)
OpenTelemetry
NGINX Agent 3 | `nginx-ic-nap-dos/nginx-plus-ingress:-ubi-agent` | amd64 | ### Custom images You can customize an existing Dockerfile or use it as a reference to create a new one, which is necessary when: - Choosing a different base image. - Installing additional NGINX modules. ## Supported Helm versions NGINX Ingress Controller can be [installed](/nic/install/helm.md) using Helm 3.0 or later. ## Supported F5 WAF for NGINX versions **Note:** F5 WAF for NGINX package based installation (previously NGINX App Protect WAF v4) is not supported when `readOnlyRootFilesystem` is enabled. NGINX Ingress Controller supports the following versions of [F5 WAF for NGINX](https://docs.nginx.com/waf/): | NIC Version | NAP-WAF Version | Config Manager | Enforcer | | ------------------- | --------------- | -------------- | -------- | | | 37.1+ | | | | 5.5.4 | 37.0+5.690.0 | 5.14.0 | 5.14.0 | | 5.4.3 | 37.0+5.635.1 | 5.13.1 | 5.13.1 | | 5.3.4 | 35+5.527.0 | 5.11.2 | 5.11.2 | | 5.2.1 | 35+5.527.0 | 5.9.0 | 5.9.0 | | 5.1.1 | 35+5.498 | 5.8.0 | 5.8.0 | | 5.0.0 | 34+5.342 | 5.6.0 | 5.6.0 | | 4.0.1 | 33+5.264 | 5.5.0 | 5.5.0 | | 3.7.2 | 32+5.1 | 5.3.0 | 5.3.0 | | 3.6.2 | 32+5.48 | 5.2.0 | 5.2.0 | -------------------------------------------------------------------------------- title: "The design of NGINX Ingress Controller" url: https://docs.nginx.com/nginx-ingress-controller/overview/design/index.md -------------------------------------------------------------------------------- This document explains how the F5 NGINX Ingress Controller is designed, and how it differs when using NGINX or NGINX Plus. The intended audience for this information is primarily the two following groups: - _Operators_ who want to know how the software works and understand how it can fail. - _Developers_ who want to [contribute](https://github.com/nginx/kubernetes-ingress/blob/main/CONTRIBUTING.md) to the project. We assume that the reader is familiar with core Kubernetes concepts, such as Pods, Deployments, Services, and Endpoints. For an understanding of how NGINX itself works, you can read the ["Inside NGINX: How We Designed for Performance & Scale"](https://www.nginx.com/blog/inside-nginx-how-we-designed-for-performance-scale/) blog post. For conciseness in diagrams, NGINX Ingress Controller is often labeled "IC" on this page. ## NGINX Ingress Controller at a high level This figure depicts an example of NGINX Ingress Controller exposing two web applications within a Kubernetes cluster to clients on the internet: ![image](/nic/ic-high-level.png) **Note:** For simplicity, necessary Kubernetes resources like Deployments and Services aren't shown, which Admin and the users also need to create. The figure shows: - A _Kubernetes cluster_. - Cluster users _Admin_, _User A_ and _User B_, which use the cluster via the _Kubernetes API_. - _Clients A_ and _Clients B_, which connect to the _Applications A_ and _B_ deployed by the corresponding users. - _NGINX Ingress Controller_, deployed in a pod with the namespace _nginx-ingress_ and configured using the _ConfigMap resource_ _nginx-ingress_. A single pod is depicted; at least two pods are typically deployed for redundancy. _NGINX Ingress Controller_ uses the _Kubernetes API_ to get the latest Ingress resources created in the cluster and then configures _NGINX_ according to those resources. - _Application A_ with two pods deployed in the _namespace A_ by _User A_. To expose the application to its clients (_Clients A_) via the host `a.example.com`, _User A_ creates _Ingress A_. - _Application B_ with one pod deployed in the _namespace B_ by _User B_. To expose the application to its clients (_Clients B_) via the host `b.example.com`, _User B_ creates _VirtualServer B_. - _Public Endpoint_, which fronts the _NGINX Ingress Controller_ pod(s). This is typically a standalone TCP load balancer (Cloud, software, or hardware) or a combination of a load balancer with a NodePort service. _Clients A_ and _B_ connect to their applications via the _Public Endpoint_. The yellow and purple arrows represent connections related to the client traffic, and the black arrows represent access to the Kubernetes API. ## The NGINX Ingress Controller pod The NGINX Ingress Controller pod consists of a single container, which includes the following: - The _NGINX Ingress Controller process_, which configures NGINX according to Ingress and other resources created in the cluster. - The _NGINX master process_, which controls NGINX worker processes. - _NGINX worker processes_, which handle the client traffic and load balance the traffic to the backend applications. The following is an architectural diagram depicting how those processes interact together and with some external entities: ![image](/nic/ic-pod.png) This table describes each connection, starting with its type: | # | Protocols | Description | | --- | --- | --- | |1|HTTP| _Prometheus_ fetches NGINX Ingress Controller and NGINX metrics with an NGINX Ingress Controller HTTP endpoint (Default `:9113/metrics`). **Note**: *Prometheus* is not required and the endpoint can be turned off. | |2|HTTPS| _NGINX Ingress Controller_ reads the _Kubernetes API_ for the latest versions of the resources in the cluster and writes to the API to update the handled resources' statuses and emit events. |3|HTTP| _Kubelet_ checks the _NGINX Ingress Controller_ readiness probe (Default `:8081/nginx-ready`) to consider the _NGINX Ingress Controller_ pod [ready](https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#pod-conditions). |4|File I/O| When _NGINX Ingress Controller_ starts, it reads the _configuration templates_ from the filesystem necessary for configuration generation. The templates are located in the `/` directory of the container and have the `.tmpl` extension |5|File I/O| _NGINX Ingress Controller_ writes logs to *stdout* and *stderr*, which are collected by the container runtime. |6|File I/O| _NGINX Ingress Controller_ generates NGINX *configuration* based on the resources created in the cluster (See [NGINX Ingress Controller is a Kubernetes Controller](#nginx-ingress-controller-is-a-kubernetes-controller)) and writes it on the filesystem in the `/etc/nginx` folder. The configuration files have a `.conf` extension. |7|File I/O| _NGINX Ingress Controller_ writes _TLS certificates_ and _keys_ from any [TLS Secrets](https://kubernetes.io/docs/concepts/configuration/secret/#tls-secrets) referenced in the Ingress and other resources to the filesystem. |8|HTTP| _NGINX Ingress Controller_ fetches the [NGINX metrics](https://nginx.org/en/docs/http/ngx_http_stub_status_module.html#stub_status) via the `unix:/var/lib/nginx/nginx-status.sock` UNIX socket and converts it to Prometheus format used in #1. |9|HTTP| To verify a successful configuration reload, _NGINX Ingress Controller_ ensures at least one _NGINX worker_ has the new configuration. To do that, the *IC* checks a particular endpoint via the `unix:/var/lib/nginx/nginx-config-version.sock` UNIX socket. |10|N/A| To start NGINX, NGINX Ingress Controller runs the `nginx` command, which launches the _NGINX master_. |11|Signal| To reload NGINX, the _NGINX Ingress Controller_ runs the `nginx -s reload` command, which validates the configuration and sends the [reload signal](https://nginx.org/en/docs/control.html) to the *NGINX master*. |12|Signal| To shutdown NGINX, the _NGINX Ingress Controller_ executes `nginx -s quit` command, which sends the graceful shutdown signal to the *NGINX master*. |13|File I/O| The _NGINX master_ sends logs to its _stdout_ and _stderr_, which are collected by the container runtime. |14|File I/O| The _NGINX master_ reads the _TLS cert and keys_ referenced in the configuration when it starts or reloads. |15|File I/O| The _NGINX master_ reads _configuration files_ when it starts or during a reload. |16|Signal| The _NGINX master_ controls the [lifecycle of _NGINX workers_](https://nginx.org/en/docs/control.html#reconfiguration) it creates workers with the new configuration and shutdowns workers with the old configuration. |17|File I/O| An _NGINX worker_ writes logs to its _stdout_ and _stderr_, which are collected by the container runtime. |18|UDP| An _NGINX worker_ sends the HTTP upstream server response latency logs via the Syslog protocol over the UNIX socket `/var/lib/nginx/nginx-syslog.sock` to _NGINX Ingress Controller_. In turn, _NGINX Ingress Controller_ analyzes and transforms the logs into Prometheus metrics. |19|HTTP,HTTPS,TCP,UDP| A _client_ sends traffic to and receives traffic from any of the _NGINX workers_ on ports 80 and 443 and any additional ports exposed by the [GlobalConfiguration resource](/nic//configuration/global-configuration/globalconfiguration-resource.md). |20|HTTP,HTTPS,TCP,UDP| An _NGINX worker_ sends traffic to and receives traffic from the _backends_. |21|HTTP| _Admin_ can connect to the [NGINX stub_status](http://nginx.org/en/docs/http/ngx_http_stub_status_module.html#stub_status) using port 8080 via an _NGINX worker_. By default, NGINX only allows connections from `localhost`. ### Differences with NGINX Plus The previous diagram depicts NGINX Ingress Controller using NGINX. NGINX Ingress Controller with NGINX Plus has the following differences: - To configure NGINX Plus, NGINX Ingress Controller uses [configuration reloads](#reloading-nginx) and the [NGINX Plus API](http://nginx.org/en/docs/http/ngx_http_api_module.html#api). This allows NGINX Ingress Controller to dynamically change the upstream servers. - Instead of the stub status metrics, the extended metrics available from the NGINX Plus API are used. - In addition to TLS certs and keys, NGINX Ingress Controller writes JWKs from the secrets of the type `nginx.org/jwk`, and NGINX workers read them. ## The NGINX Ingress Controller process This section covers the architecture of the NGINX Ingress Controller process, including: - How NGINX Ingress Controller processes a new Ingress resource created by a user. - A summary of how NGINX Ingress Controller works in relation to others Kubernetes Controllers. - The different components of the IC process. ### Processing a new Ingress resource The following diagram depicts how NGINX Ingress Controller processes a new Ingress resource. The the NGINX master and worker processes are represented as a single rectangle, _NGINX_ for simplicity. VirtualServer and VirtualServerRoute resources are indicated similarly. ![image](/nic/ic-process.png) Processing a new Ingress resource involves the following steps: each step corresponds to the arrow on the diagram with the same number: 1. _User_ creates a new Ingress resource. 1. The NGINX Ingress Controller process has a _Cache_ of the resources in the cluster. The _Cache_ includes only the resources NGINX Ingress Controller is concerned with such as Ingresses. The _Cache_ stays in sync with the Kubernetes API by [watching for changes to the resources](https://kubernetes.io/docs/reference/using-api/api-concepts/#efficient-detection-of-changes). 1. Once the _Cache_ has the new Ingress resource, it notifies the _Control Loop_ about the changed resource. 1. The _Control Loop_ gets the latest version of the Ingress resource from the _Cache_. Since the Ingress resource references other resources, such as TLS Secrets, the _Control loop_ gets the latest versions of those referenced resources as well. 1. The _Control Loop_ generates TLS certificates and keys from the TLS Secrets and writes them to the filesystem. 1. The _Control Loop_ generates and writes the NGINX _configuration files_, which correspond to the Ingress resource, and writes them to the filesystem. 1. The _Control Loop_ reloads _NGINX_ and waits for _NGINX_ to successfully reload. As part of the reload: 1. _NGINX_ reads the _TLS certs and keys_. 1. _NGINX_ reads the _configuration files_. 1. The _Control Loop_ emits an event for the Ingress resource and updates its status. If the reload fails, the event includes the error message. ### NGINX Ingress Controller is a Kubernetes controller With the context from the previous sections, we can generalize how NGINX Ingress Controller works: *NGINX Ingress Controller constantly processes both new resources and changes to the existing resources in the cluster. As a result, the NGINX configuration stays up-to-date with the resources in the cluster.* NGINX Ingress Controller is an example of a [Kubernetes Controller](https://kubernetes.io/docs/concepts/architecture/controller/): NGINX Ingress Controller runs a control loop that ensures NGINX is configured according to the desired state (Ingresses and other resources). The desired state is based on the following built-in Kubernetes resources and Custom Resources (CRs): - Layer 7 Load balancing configuration: - Ingresses - VirtualServers (CR) - VirtualServerRoutes (CR) - Layer 7 policies: - Policies (CR) - Layer 4 load balancing configuration: - TransportServers (CR) - Service discovery: - Services - Endpoints - Pods - Secret configuration: - Secrets - Global Configuration: - ConfigMap (only one resource) - GlobalConfiguration (CR, only one resource) NGINX Ingress Controller can watch additional Custom Resources, which are less common and not enabled by default: - [F5 WAF for NGINX resources](/nic/integrations/app-protect-dos/configuration) (APPolicies, APLogConfs, APUserSigs) - IngressLink resource (only one resource) ## NGINX Ingress Controller process components In this section, we describe the components of the NGINX Ingress Controller process and how they interact, including: 1. How NGINX Ingress Controller watches for resources changes. 1. The main components of the NGINX Ingress Controller _Control Loop_. 1. How those components process a resource change. 1. Additional components that are crucial for processing changes. NGINX Ingress Controller is written in [Go](https://golang.org/) and relies heavily on the [Go client for Kubernetes](https://github.com/kubernetes/client-go). Where relevant, we include links to the source code on GitHub. ### Resource caches In an earlier section, [Processing a New Ingress Resource](#processing-a-new-ingress-resource), we mentioned that NGINX Ingress Controller has a cache of the resources in the cluster that stays in sync with the Kubernetes API by watching them for changes. We also mentioned that once the cache is updated, it notifies the control loop about the changed resources. The cache is actually a collection of *informers*. The following diagram shows how changes to resources are processed by NGINX Ingress Controller. ![image](/nic/ic-process-components.png) - For every resource type that NGINX Ingress Controller monitors, it creates an [_Informer_](https://pkg.go.dev/k8s.io/client-go@v0.21.0/tools/cache#SharedInformer). The _Informer_ includes a _Store_ that holds the resources of that type. To keep the _Store_ in sync with the latest versions of the resources in the cluster, the _Informer_ calls the Watch and List _Kubernetes APIs_ for that resource type (see the arrow _1. Watch and List_ on the diagram). - When a change happens in the cluster (for example, a new resource is created), the _Informer_ updates its _Store_ and invokes [_Handlers_](https://pkg.go.dev/k8s.io/client-go@v0.21.0/tools/cache#ResourceEventHandler) (See the arrow _2. Invoke_) for that _Informer_. - NGINX Ingress Controller registers _Handlers_ for every _Informer_. Most of the time, a _Handler_ creates an entry for the affected resource in the _Workqueue_ where a workqueue element includes the type of the resource and its namespace and name (See the arrow _3. Put_). - The _Workqueue_ always tries to drain itself: if there is an element at the front, the queue will remove the element and send it to the _Controller_ by calling a callback function (See the arrow _4. Send_). - The _Controller_ is the primary component of NGINX Ingress Controller, which represents the _Control Loop_, explained in [The Control Loop](#the-control-loop) section. To process a workqueue element, the _Controller_ component gets the latest version of the resource from the _Store_ (See the arrow _5. Get_), reconfigures _NGINX_ according to the resource (See the arrow _6. Reconfigure*_, updates the resource status, and emits an event via the _Kubernetes API_ (See the arrow _7. Update status and emit event_). ### The control loop This section discusses the main components of NGINX Ingress Controller, which comprise the control loop: - [Controller](https://github.com/nginx/kubernetes-ingress/blob/v1.11.0/internal/k8s/controller.go#L90) - Runs the NGINX Ingress Controller control loop. - Instantiates _Informers_, _Handlers_, the _Workqueue_ and additional helper components. - Includes the sync method), which is called by the _Workqueue_ to process a changed resource. - Passes changed resources to _Configurator_ to re-configure NGINX. - [Configurator](https://github.com/nginx/kubernetes-ingress/blob/v1.11.0/internal/configs/configurator.go#L95) - Generates NGINX configuration files, TLS and cert keys, and JWKs based on the Kubernetes resource. - Uses _Manager_ to write the generated files and reload NGINX. - [Manager](https://github.com/nginx/kubernetes-ingress/blob/v1.11.0/internal/nginx/manager.go#L52) - Controls the lifecycle of NGINX (starting, reloading, quitting). See [Reloading NGINX](#reloading-nginx) for more details about reloading. - Manages the configuration files, TLS keys and certs, and JWKs. The following diagram shows how the three components interact: ![image](/nic/control-loop.png) #### The Controller sync method The Controller [sync](https://github.com/nginx/kubernetes-ingress/blob/v1.11.0/internal/k8s/controller.go#L663) method is called by the _Workqueue_ to process a change of a resource. The method determines the _kind_ of the resource and calls the appropriate _sync_ method (Such as _syncIngress_ for Ingress resources). To explain how the sync methods work, we will examine the most important one: the _syncIngress_ method, and describe how it processes a new Ingress resource. ![image](/nic/controller-sync.png) 1. The _Workqueue_ calls the _sync_ method and passes a workqueue element to it that includes the changed resource _kind_ and _key_ (The key is the resource namespace/name such as “default/cafe-ingress”). 1. Using the _kind_, the _sync_ method calls the appropriate sync method and passes the resource key. For Ingress resources, the method is _syncIngress_. 1. _syncIngress_ gets the Ingress resource from the *Ingress Store* using the key. The _Store_ is controlled by the _Ingress Informer_. In the code, we use the helper _storeToIngressLister_ type that wraps the _Store_. 1. _syncIngress_ calls _AddOrUpdateIngress_ of the _Configuration_, passing the Ingress along. The [Configuration](https://github.com/nginx/kubernetes-ingress/blob/v1.11.0/internal/k8s/configuration.go#L320) is a component that represents a valid collection of load balancing configuration resources (Ingresses, VirtualServers, VirtualServerRoutes, TransportServers), ready to be converted to the NGINX configuration (see the [Configuration section](#configuration) for more details). _AddOrUpdateIngress_ returns a list of [ResourceChanges](https://github.com/nginx/kubernetes-ingress/blob/v1.11.0/internal/k8s/configuration.go#L59), which must be reflected in the NGINX config. Typically, for a new Ingress resource, the _Configuration_ returns only a single _ResourceChange_. 1. _syncIngress_ calls _processChanges_, which processes the single Ingress _ResourceChange_. 1. _processChanges_ creates an extended Ingress resource (_IngressEx_) that includes the original Ingress resource and its dependencies, such as Endpoints and Secrets, to generate the NGINX configuration. For simplicity, we don’t show this step on the diagram. 1. _processChanges_ calls _AddOrUpdateIngress_ of the _Configurator_ and passes the extended Ingress resource. 1. _Configurator_ generates an NGINX configuration file based on the extended Ingress resource, then: 1. Calls _Manager’s CreateConfig()_ to update the config for the Ingress resource. 1. Calls _Manager’s Reload()_ to reload NGINX. 1. The reload status is propagated from _Manager_ to _processChanges_, and is either a success or a failure with an error message. 1. _processChanges_ calls _updateRegularIngressStatusAndEvent_ to update the status of the Ingress resource and emit an event with the status of the reload: both make an API call to the Kubernetes API. **Additional notes**: - Many details are not included for conciseness: the source code provides the most granular detail. - The _syncVirtualServer_, _syncVirtualServerRoute_, and _syncTransportServer_ methods are similar to _syncIngress_, while other sync methods are different. However, those methods typically find the affected Ingress, VirtualServer, and TransportServer resources and regenerate the configuration for them. - The _Workqueue_ has only a single worker thread that calls the sync method synchronously, meaning the _Control Loop_ processes only one change at a time. #### Helper components There are two additional helper components crucial for processing changes: _Configuration_ and _LocalSecretStore_. ##### Configuration [_Configuration_](https://github.com/nginx/kubernetes-ingress/blob/v1.11.0/internal/k8s/configuration.go#L320) holds the latest valid state of the NGINX Ingress Controller load balancing configuration resources: Ingresses, VirtualServers, VirtualServerRoutes, TransportServers, and GlobalConfiguration. The _Configuration_ supports add, update and delete operations on the resources. When you invoke these operations on a resource in the Configuration, it performs the following: 1. Validates the object (For add or update) 1. Calculates the changes to the affected resources that are necessary to propagate to the NGINX configuration, returning the changes to the caller. For example, when you add a new Ingress resource, the _Configuration_ returns a change requiring NGINX Ingress Controller to add the configuration for that Ingress to the NGINX configuration files. If you made an existing Ingress resource invalid, the _Configuration_ returns a change requiring NGINX Ingress Controller to remove the configuration for that Ingress from the NGINX configuration files. Additionally, the _Configuration_ ensures that only one Ingress/VirtualServer/TransportServer (TLS Passthrough) resource holds a particular host (For example, cafe.example.com) and only one TransportServer (TCP/UDP) holds a particular listener (Such as port 53 for UDP). This ensures that no host or listener collisions happen in the NGINX configuration. Ultimately, NGINX Ingress Controller ensures the NGINX config on the filesystem reflects the state of the objects in the _Configuration_ at any point in time. ##### LocalSecretStore [_LocalSecretStore_](https://github.com/nginx/kubernetes-ingress/blob/v1.11.0/internal/k8s/secrets/store.go#L32) (of the _SecretStore_ interface) holds the valid Secret resources and keeps the corresponding files on the filesystem in sync with them. Secrets are used to hold TLS certificates and keys (type `kubernetes.io/tls`), CAs (`nginx.org/ca`), JWKs (`nginx.org/jwk`), and client secrets for an OIDC provider (`nginx.org/oidc`). When _Controller_ processes a change to a configuration resource like Ingress, it creates an extended version of a resource that includes the dependencies (Such as Secrets) necessary to generate the NGINX configuration. _LocalSecretStore_ allows _Controller_ to reference the filesystem for a secret using the secret key (namespace/name). ## Reloading NGINX The following sections describe how NGINX reloads and how NGINX Ingress Controller specifically affects this process. ### How NGINX reloads work Reloading NGINX is necessary to apply new configuration changes and occurs with these steps: 1. The administrator sends a HUP (hangup) signal to the NGINX master process to trigger a reload. 1. The master process brings down the worker processes with the old configuration and starts worker processes with the new configuration. 1. The administrator verifies the reload has successfully finished. The [NGINX documentation](https://nginx.org/en/docs/control.html#reconfiguration) has more details about reloading. #### How to reload NGINX and confirm success The NGINX binary (`nginx`) supports the reload operation with the `-s reload` option. When you run this option: 1. It validates the new NGINX configuration and exits if it is invalid printing the error messages to the stderr. 1. It sends a HUP signal to the NGINX master process and exits. As an alternative, you can send a HUP signal to the NGINX master process directly. Once the reload operation has been invoked with `nginx -s reload`, there is no wait period for NGINX to finish reloading. This means it is the responsibility of an administator to check it is finished, for which there are a few options: - Check if the master process created new worker processes. Two ways are by running `ps` or reading the `/proc` file system. - Send an HTTP request to NGINX, to see if a new worker process responds. This signifies that NGINX reloaded successfully: this method requires additional NGINX configuration, explained below. NGINX reloads take roughly 200ms. The factors affecting reload time are configuration size and details, the number of TLS certificates/keys, enabled modules, and available CPU resources. #### Potential problems Most of the time, if `nginx -s reload` executes, the reload will also succeed. In the rare case a reload fails, the NGINX master process will print the an error message. This is an example: ```shell 2022/07/09 00:56:42 [emerg] 1353#1353: limit_req "one" uses the "$remote_addr" key while previously it used the "$binary_remote_addr" key ``` The operation is graceful; reloading doesn't lead to any traffic loss by NGINX. However, frequent reloads can lead to high memory utilization and potential OOM (Out-Of-Memory) errors, resulting in traffic loss. This can most likely happen if you (1) proxy traffic that utilizes long-lived connections (ex: Websockets, gRPC) and (2) reload frequently. In these scenarios, you can end up with multiple generations of NGINX worker processes that are shutting down which will force old workers to shut down after the timeout). Eventually, all those worker processes can exhaust the system's available memory. Old NGINX workers will not shut down until all connections are terminated either by clients or backends, unless you configure [worker_shutdown_timeout](https://nginx.org/en/docs/ngx_core_module.html#worker_shutdown_timeout). Since both the old and new NGINX worker processes coexist during a reload, reloading can lead to two spikes in memory utilization. With a lack of available memory, the NGINX master process can fail to create new worker processes. ### Reloading in NGINX Ingress Controller NGINX Ingress Controller reloads NGINX to apply configuration changes. To facilitate reloading, NGINX Ingress Controller configures a server listening on the Unix socket `unix:/var/lib/nginx/nginx-config-version.sock` that responds with the configuration version for `/configVersion` URI. NGINX Ingress Controller writes the configuration to `/etc/nginx/config-version.conf`. Reloads occur with this sequence of steps: 1. NGINX Ingress Controller updates generated configuration files, including any secrets. 1. NGINX Ingress Controller updates the config version in `/etc/nginx/config-version.conf`. 1. NGINX Ingress Controller runs `nginx -s reload`. If the command fails, NGINX Ingress Controller logs the error and considers the reload failed. 1. If the command succeeds, NGINX Ingress Controller periodically checks for the config version by sending an HTTP request to the config version server on `unix:/var/lib/nginx/nginx-config-version.sock`. 1. Once NGINX Ingress Controller sees the correct config version returned by NGINX, it considers the reload successful. If it doesn't see the correct configuration version after the configurable timeout ( [`-nginx-reload-timeout`](/nic/configuration/global-configuration/command-line-arguments)), NGINX Ingress Controller considers the reload failed. The [NGINX Ingress Controller Control Loop](#the-control-loop) stops during a reload so that it cannot affect configuration files or reload NGINX until the current reload succeeds or fails. ### When NGINX Ingress Controller reloads NGINX NGINX Ingress Controller reloads NGINX every time the Control Loop processes a change that affects the generated NGINX configuration. In general, every time a monitored resource is changed, NGINX Ingress Controller will regenerate the configuration and reload NGINX. There are three special cases: - *Start*. When NGINX Ingress Controller starts, it processes all resources in the cluster and only then reloads NGINX. This avoids a "reload storm" by reloading only once. - *Batch updates*. When NGINX Ingress Controller receives a number of concurrent requests from the Kubernetes API, it will pause NGINX reloads until the task queue is empty. This reduces the number of reloads to minimize the impact of batch updates and reduce the risk of OOM (Out of Memory) errors. - *NGINX Plus*. If NGINX Ingress Controller is using NGINX Plus, it will not reload NGINX Plus for changes to the Endpoints resources. In this case, NGINX Ingress Controller will use the NGINX Plus API to update the corresponding upstreams and skip reloading. -------------------------------------------------------------------------------- title: "The design of NGINX Ingress Controller LTS" url: https://docs.nginx.com/nginx-ingress-controller/lts/overview/design/index.md -------------------------------------------------------------------------------- This document explains how the F5 NGINX Ingress Controller LTS is designed. The intended audience for this information is primarily the two following groups: - _Operators_ who want to know how the software works and understand how it can fail. - _Developers_ who want to [contribute](https://github.com/nginx/kubernetes-ingress/blob/main/CONTRIBUTING.md) to the project. We assume that the reader is familiar with core Kubernetes concepts, such as Pods, Deployments, Services, and Endpoints. For an understanding of how NGINX itself works, you can read the ["Inside NGINX: How We Designed for Performance & Scale"](https://www.nginx.com/blog/inside-nginx-how-we-designed-for-performance-scale/) blog post. For conciseness in diagrams, NGINX Ingress Controller LTS is often labeled "IC" on this page. ## NGINX Ingress Controller LTS at a high level This figure depicts an example of NGINX Ingress Controller LTS exposing two web applications within a Kubernetes cluster to clients on the internet: ![image](/nic/ic-high-level.png) **note:** For simplicity, necessary Kubernetes resources like Deployments and Services aren't shown, which Admin and the users also need to create. The figure shows: - A _Kubernetes cluster_. - Cluster users _Admin_, _User A_ and _User B_, which use the cluster via the _Kubernetes API_. - _Clients A_ and _Clients B_, which connect to the _Applications A_ and _B_ deployed by the corresponding users. - _NGINX Ingress Controller LTS_, deployed in a pod with the namespace _nginx-ingress_ and configured using the _ConfigMap resource_ _nginx-ingress_. A single pod is depicted; at least two pods are typically deployed for redundancy. _NGINX Ingress Controller LTS_ uses the _Kubernetes API_ to get the latest Ingress resources created in the cluster and then configures _NGINX_ according to those resources. - _Application A_ with two pods deployed in the _namespace A_ by _User A_. To expose the application to its clients (_Clients A_) via the host `a.example.com`, _User A_ creates _Ingress A_. - _Application B_ with one pod deployed in the _namespace B_ by _User B_. To expose the application to its clients (_Clients B_) via the host `b.example.com`, _User B_ creates _VirtualServer B_. - _Public Endpoint_, which fronts the _NGINX Ingress Controller LTS_ pod(s). This is typically a standalone TCP load balancer (Cloud, software, or hardware) or a combination of a load balancer with a NodePort service. _Clients A_ and _B_ connect to their applications via the _Public Endpoint_. The yellow and purple arrows represent connections related to the client traffic, and the black arrows represent access to the Kubernetes API. ## The NGINX Ingress Controller LTS pod The NGINX Ingress Controller LTS pod consists of a single container, which includes the following: - The _NGINX Ingress Controller LTS process_, which configures NGINX according to Ingress and other resources created in the cluster. - The _NGINX master process_, which controls NGINX worker processes. - _NGINX worker processes_, which handle the client traffic and load balance the traffic to the backend applications. The following is an architectural diagram depicting how those processes interact together and with some external entities: ![image](/nic/ic-pod.png) This table describes each connection, starting with its type: | # | Protocols | Description | | --- | --- | --- | |1|HTTP| _Prometheus_ fetches NGINX Ingress Controller LTS and NGINX metrics with an NGINX Ingress Controller LTS HTTP endpoint (Default `:9113/metrics`). **Note**: *Prometheus* is not required and the endpoint can be turned off. | |2|HTTPS| _NGINX Ingress Controller LTS_ reads the _Kubernetes API_ for the latest versions of the resources in the cluster and writes to the API to update the handled resources' statuses and emit events. |3|HTTP| _Kubelet_ checks the _NGINX Ingress Controller LTS_ readiness probe (Default `:8081/nginx-ready`) to consider the _NGINX Ingress Controller LTS_ pod [ready](https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#pod-conditions). |4|File I/O| When _NGINX Ingress Controller LTS_ starts, it reads the _configuration templates_ from the filesystem necessary for configuration generation. The templates are located in the `/` directory of the container and have the `.tmpl` extension |5|File I/O| _NGINX Ingress Controller LTS_ writes logs to *stdout* and *stderr*, which are collected by the container runtime. |6|File I/O| _NGINX Ingress Controller LTS_ generates NGINX *configuration* based on the resources created in the cluster (See [NGINX Ingress Controller LTS is a Kubernetes Controller](#nginx-ingress-controller-is-a-kubernetes-controller)) and writes it on the filesystem in the `/etc/nginx` folder. The configuration files have a `.conf` extension. |7|File I/O| _NGINX Ingress Controller LTS_ writes _TLS certificates_ and _keys_ from any [TLS Secrets](https://kubernetes.io/docs/concepts/configuration/secret/#tls-secrets) referenced in the Ingress and other resources to the filesystem. |8|HTTP| _NGINX Ingress Controller LTS_ fetches the [NGINX metrics](https://nginx.org/en/docs/http/ngx_http_api_module.html) via the NGINX Plus API. |9|HTTP| To verify a successful configuration reload, _NGINX Ingress Controller LTS_ ensures at least one _NGINX worker_ has the new configuration. To do that, the *IC* checks a particular endpoint via the `unix:/var/lib/nginx/nginx-config-version.sock` UNIX socket. |10|N/A| To start NGINX, NGINX Ingress Controller LTS runs the `nginx` command, which launches the _NGINX master_. |11|Signal| To reload NGINX, the _NGINX Ingress Controller LTS_ runs the `nginx -s reload` command, which validates the configuration and sends the [reload signal](https://nginx.org/en/docs/control.html) to the *NGINX master*. |12|Signal| To shutdown NGINX, the _NGINX Ingress Controller LTS_ executes `nginx -s quit` command, which sends the graceful shutdown signal to the *NGINX master*. |13|File I/O| The _NGINX master_ sends logs to its _stdout_ and _stderr_, which are collected by the container runtime. |14|File I/O| The _NGINX master_ reads the _TLS cert and keys_ referenced in the configuration when it starts or reloads. |15|File I/O| The _NGINX master_ reads _configuration files_ when it starts or during a reload. |16|Signal| The _NGINX master_ controls the [lifecycle of _NGINX workers_](https://nginx.org/en/docs/control.html#reconfiguration) it creates workers with the new configuration and shutdowns workers with the old configuration. |17|File I/O| An _NGINX worker_ writes logs to its _stdout_ and _stderr_, which are collected by the container runtime. |18|UDP| An _NGINX worker_ sends the HTTP upstream server response latency logs via the Syslog protocol over the UNIX socket `/var/lib/nginx/nginx-syslog.sock` to _NGINX Ingress Controller LTS_. In turn, _NGINX Ingress Controller LTS_ analyzes and transforms the logs into Prometheus metrics. |19|HTTP,HTTPS,TCP,UDP| A _client_ sends traffic to and receives traffic from any of the _NGINX workers_ on ports 80 and 443 and any additional ports exposed by the [GlobalConfiguration resource](/nic/lts/configuration/global-configuration/globalconfiguration-resource.md). |20|HTTP,HTTPS,TCP,UDP| An _NGINX worker_ sends traffic to and receives traffic from the _backends_. |21|HTTP| _Admin_ can connect to the [NGINX stub_status](http://nginx.org/en/docs/http/ngx_http_stub_status_module.html#stub_status) using port 8080 via an _NGINX worker_. By default, NGINX only allows connections from `localhost`. ### NGINX Plus capabilities NGINX Ingress Controller LTS uses NGINX Plus and provides the following capabilities: - To configure NGINX Plus, NGINX Ingress Controller LTS uses [configuration reloads](#reloading-nginx) and the [NGINX Plus API](http://nginx.org/en/docs/http/ngx_http_api_module.html#api). This allows NGINX Ingress Controller LTS to dynamically change the upstream servers. - Extended metrics available from the NGINX Plus API are used instead of stub status metrics. - In addition to TLS certs and keys, NGINX Ingress Controller LTS writes JWKs from the secrets of the type `nginx.org/jwk`, and NGINX workers read them. ## The NGINX Ingress Controller LTS process This section covers the architecture of the NGINX Ingress Controller LTS process, including: - How NGINX Ingress Controller LTS processes a new Ingress resource created by a user. - A summary of how NGINX Ingress Controller LTS works in relation to others Kubernetes Controllers. - The different components of the IC process. ### Processing a new Ingress resource The following diagram depicts how NGINX Ingress Controller LTS processes a new Ingress resource. The the NGINX master and worker processes are represented as a single rectangle, _NGINX_ for simplicity. VirtualServer and VirtualServerRoute resources are indicated similarly. ![image](/nic/ic-process.png) Processing a new Ingress resource involves the following steps: each step corresponds to the arrow on the diagram with the same number: 1. _User_ creates a new Ingress resource. 1. The NGINX Ingress Controller LTS process has a _Cache_ of the resources in the cluster. The _Cache_ includes only the resources NGINX Ingress Controller LTS is concerned with such as Ingresses. The _Cache_ stays in sync with the Kubernetes API by [watching for changes to the resources](https://kubernetes.io/docs/reference/using-api/api-concepts/#efficient-detection-of-changes). 1. Once the _Cache_ has the new Ingress resource, it notifies the _Control Loop_ about the changed resource. 1. The _Control Loop_ gets the latest version of the Ingress resource from the _Cache_. Since the Ingress resource references other resources, such as TLS Secrets, the _Control loop_ gets the latest versions of those referenced resources as well. 1. The _Control Loop_ generates TLS certificates and keys from the TLS Secrets and writes them to the filesystem. 1. The _Control Loop_ generates and writes the NGINX _configuration files_, which correspond to the Ingress resource, and writes them to the filesystem. 1. The _Control Loop_ reloads _NGINX_ and waits for _NGINX_ to successfully reload. As part of the reload: 1. _NGINX_ reads the _TLS certs and keys_. 1. _NGINX_ reads the _configuration files_. 1. The _Control Loop_ emits an event for the Ingress resource and updates its status. If the reload fails, the event includes the error message. ### NGINX Ingress Controller LTS is a Kubernetes controller With the context from the previous sections, we can generalize how NGINX Ingress Controller LTS works: *NGINX Ingress Controller LTS constantly processes both new resources and changes to the existing resources in the cluster. As a result, the NGINX configuration stays up-to-date with the resources in the cluster.* NGINX Ingress Controller LTS is an example of a [Kubernetes Controller](https://kubernetes.io/docs/concepts/architecture/controller/): NGINX Ingress Controller LTS runs a control loop that ensures NGINX is configured according to the desired state (Ingresses and other resources). The desired state is based on the following built-in Kubernetes resources and Custom Resources (CRs): - Layer 7 Load balancing configuration: - Ingresses - VirtualServers (CR) - VirtualServerRoutes (CR) - Layer 7 policies: - Policies (CR) - Layer 4 load balancing configuration: - TransportServers (CR) - Service discovery: - Services - Endpoints - Pods - Secret configuration: - Secrets - Global Configuration: - ConfigMap (only one resource) - GlobalConfiguration (CR, only one resource) NGINX Ingress Controller LTS can watch additional Custom Resources, which are less common and not enabled by default: - IngressLink resource (only one resource) ## NGINX Ingress Controller LTS process components In this section, we describe the components of the NGINX Ingress Controller LTS process and how they interact, including: 1. How NGINX Ingress Controller LTS watches for resources changes. 1. The main components of the NGINX Ingress Controller LTS _Control Loop_. 1. How those components process a resource change. 1. Additional components that are crucial for processing changes. NGINX Ingress Controller LTS is written in [Go](https://golang.org/) and relies heavily on the [Go client for Kubernetes](https://github.com/kubernetes/client-go). Where relevant, we include links to the source code on GitHub. ### Resource caches In an earlier section, [Processing a New Ingress Resource](#processing-a-new-ingress-resource), we mentioned that NGINX Ingress Controller LTS has a cache of the resources in the cluster that stays in sync with the Kubernetes API by watching them for changes. We also mentioned that once the cache is updated, it notifies the control loop about the changed resources. The cache is actually a collection of *informers*. The following diagram shows how changes to resources are processed by NGINX Ingress Controller LTS. ![image](/nic/ic-process-components.png) - For every resource type that NGINX Ingress Controller LTS monitors, it creates an [_Informer_](https://pkg.go.dev/k8s.io/client-go@v0.21.0/tools/cache#SharedInformer). The _Informer_ includes a _Store_ that holds the resources of that type. To keep the _Store_ in sync with the latest versions of the resources in the cluster, the _Informer_ calls the Watch and List _Kubernetes APIs_ for that resource type (see the arrow _1. Watch and List_ on the diagram). - When a change happens in the cluster (for example, a new resource is created), the _Informer_ updates its _Store_ and invokes [_Handlers_](https://pkg.go.dev/k8s.io/client-go@v0.21.0/tools/cache#ResourceEventHandler) (See the arrow _2. Invoke_) for that _Informer_. - NGINX Ingress Controller LTS registers _Handlers_ for every _Informer_. Most of the time, a _Handler_ creates an entry for the affected resource in the _Workqueue_ where a workqueue element includes the type of the resource and its namespace and name (See the arrow _3. Put_). - The _Workqueue_ always tries to drain itself: if there is an element at the front, the queue will remove the element and send it to the _Controller_ by calling a callback function (See the arrow _4. Send_). - The _Controller_ is the primary component of NGINX Ingress Controller LTS, which represents the _Control Loop_, explained in [The Control Loop](#the-control-loop) section. To process a workqueue element, the _Controller_ component gets the latest version of the resource from the _Store_ (See the arrow _5. Get_), reconfigures _NGINX_ according to the resource (See the arrow _6. Reconfigure*_, updates the resource status, and emits an event via the _Kubernetes API_ (See the arrow _7. Update status and emit event_). ### The control loop This section discusses the main components of NGINX Ingress Controller LTS, which comprise the control loop: - [Controller](https://github.com/nginx/kubernetes-ingress/blob/v1.11.0/internal/k8s/controller.go#L90) - Runs the NGINX Ingress Controller LTS control loop. - Instantiates _Informers_, _Handlers_, the _Workqueue_ and additional helper components. - Includes the sync method), which is called by the _Workqueue_ to process a changed resource. - Passes changed resources to _Configurator_ to re-configure NGINX. - [Configurator](https://github.com/nginx/kubernetes-ingress/blob/v1.11.0/internal/configs/configurator.go#L95) - Generates NGINX configuration files, TLS and cert keys, and JWKs based on the Kubernetes resource. - Uses _Manager_ to write the generated files and reload NGINX. - [Manager](https://github.com/nginx/kubernetes-ingress/blob/v1.11.0/internal/nginx/manager.go#L52) - Controls the lifecycle of NGINX (starting, reloading, quitting). See [Reloading NGINX](#reloading-nginx) for more details about reloading. - Manages the configuration files, TLS keys and certs, and JWKs. The following diagram shows how the three components interact: ![image](/nic/control-loop.png) #### The Controller sync method The Controller [sync](https://github.com/nginx/kubernetes-ingress/blob/v1.11.0/internal/k8s/controller.go#L663) method is called by the _Workqueue_ to process a change of a resource. The method determines the _kind_ of the resource and calls the appropriate _sync_ method (Such as _syncIngress_ for Ingress resources). To explain how the sync methods work, we will examine the most important one: the _syncIngress_ method, and describe how it processes a new Ingress resource. ![image](/nic/controller-sync.png) 1. The _Workqueue_ calls the _sync_ method and passes a workqueue element to it that includes the changed resource _kind_ and _key_ (The key is the resource namespace/name such as “default/cafe-ingress”). 1. Using the _kind_, the _sync_ method calls the appropriate sync method and passes the resource key. For Ingress resources, the method is _syncIngress_. 1. _syncIngress_ gets the Ingress resource from the *Ingress Store* using the key. The _Store_ is controlled by the _Ingress Informer_. In the code, we use the helper _storeToIngressLister_ type that wraps the _Store_. 1. _syncIngress_ calls _AddOrUpdateIngress_ of the _Configuration_, passing the Ingress along. The [Configuration](https://github.com/nginx/kubernetes-ingress/blob/v1.11.0/internal/k8s/configuration.go#L320) is a component that represents a valid collection of load balancing configuration resources (Ingresses, VirtualServers, VirtualServerRoutes, TransportServers), ready to be converted to the NGINX configuration (see the [Configuration section](#configuration) for more details). _AddOrUpdateIngress_ returns a list of [ResourceChanges](https://github.com/nginx/kubernetes-ingress/blob/v1.11.0/internal/k8s/configuration.go#L59), which must be reflected in the NGINX config. Typically, for a new Ingress resource, the _Configuration_ returns only a single _ResourceChange_. 1. _syncIngress_ calls _processChanges_, which processes the single Ingress _ResourceChange_. 1. _processChanges_ creates an extended Ingress resource (_IngressEx_) that includes the original Ingress resource and its dependencies, such as Endpoints and Secrets, to generate the NGINX configuration. For simplicity, we don’t show this step on the diagram. 1. _processChanges_ calls _AddOrUpdateIngress_ of the _Configurator_ and passes the extended Ingress resource. 1. _Configurator_ generates an NGINX configuration file based on the extended Ingress resource, then: 1. Calls _Manager’s CreateConfig()_ to update the config for the Ingress resource. 1. Calls _Manager’s Reload()_ to reload NGINX. 1. The reload status is propagated from _Manager_ to _processChanges_, and is either a success or a failure with an error message. 1. _processChanges_ calls _updateRegularIngressStatusAndEvent_ to update the status of the Ingress resource and emit an event with the status of the reload: both make an API call to the Kubernetes API. **Additional notes**: - Many details are not included for conciseness: the source code provides the most granular detail. - The _syncVirtualServer_, _syncVirtualServerRoute_, and _syncTransportServer_ methods are similar to _syncIngress_, while other sync methods are different. However, those methods typically find the affected Ingress, VirtualServer, and TransportServer resources and regenerate the configuration for them. - The _Workqueue_ has only a single worker thread that calls the sync method synchronously, meaning the _Control Loop_ processes only one change at a time. #### Helper components There are two additional helper components crucial for processing changes: _Configuration_ and _LocalSecretStore_. ##### Configuration [_Configuration_](https://github.com/nginx/kubernetes-ingress/blob/v1.11.0/internal/k8s/configuration.go#L320) holds the latest valid state of the NGINX Ingress Controller LTS load balancing configuration resources: Ingresses, VirtualServers, VirtualServerRoutes, TransportServers, and GlobalConfiguration. The _Configuration_ supports add, update and delete operations on the resources. When you invoke these operations on a resource in the Configuration, it performs the following: 1. Validates the object (For add or update) 1. Calculates the changes to the affected resources that are necessary to propagate to the NGINX configuration, returning the changes to the caller. For example, when you add a new Ingress resource, the _Configuration_ returns a change requiring NGINX Ingress Controller LTS to add the configuration for that Ingress to the NGINX configuration files. If you made an existing Ingress resource invalid, the _Configuration_ returns a change requiring NGINX Ingress Controller LTS to remove the configuration for that Ingress from the NGINX configuration files. Additionally, the _Configuration_ ensures that only one Ingress/VirtualServer/TransportServer (TLS Passthrough) resource holds a particular host (For example, cafe.example.com) and only one TransportServer (TCP/UDP) holds a particular listener (Such as port 53 for UDP). This ensures that no host or listener collisions happen in the NGINX configuration. Ultimately, NGINX Ingress Controller LTS ensures the NGINX config on the filesystem reflects the state of the objects in the _Configuration_ at any point in time. ##### LocalSecretStore [_LocalSecretStore_](https://github.com/nginx/kubernetes-ingress/blob/v1.11.0/internal/k8s/secrets/store.go#L32) (of the _SecretStore_ interface) holds the valid Secret resources and keeps the corresponding files on the filesystem in sync with them. Secrets are used to hold TLS certificates and keys (type `kubernetes.io/tls`), CAs (`nginx.org/ca`), JWKs (`nginx.org/jwk`), and client secrets for an OIDC provider (`nginx.org/oidc`). When _Controller_ processes a change to a configuration resource like Ingress, it creates an extended version of a resource that includes the dependencies (Such as Secrets) necessary to generate the NGINX configuration. _LocalSecretStore_ allows _Controller_ to reference the filesystem for a secret using the secret key (namespace/name). ## Reloading NGINX The following sections describe how NGINX reloads and how NGINX Ingress Controller LTS specifically affects this process. ### How NGINX reloads work Reloading NGINX is necessary to apply new configuration changes and occurs with these steps: 1. The administrator sends a HUP (hangup) signal to the NGINX master process to trigger a reload. 1. The master process brings down the worker processes with the old configuration and starts worker processes with the new configuration. 1. The administrator verifies the reload has successfully finished. The [NGINX documentation](https://nginx.org/en/docs/control.html#reconfiguration) has more details about reloading. #### How to reload NGINX and confirm success The NGINX binary (`nginx`) supports the reload operation with the `-s reload` option. When you run this option: 1. It validates the new NGINX configuration and exits if it is invalid printing the error messages to the stderr. 1. It sends a HUP signal to the NGINX master process and exits. As an alternative, you can send a HUP signal to the NGINX master process directly. Once the reload operation has been invoked with `nginx -s reload`, there is no wait period for NGINX to finish reloading. This means it is the responsibility of an administator to check it is finished, for which there are a few options: - Check if the master process created new worker processes. Two ways are by running `ps` or reading the `/proc` file system. - Send an HTTP request to NGINX, to see if a new worker process responds. This signifies that NGINX reloaded successfully: this method requires additional NGINX configuration, explained below. NGINX reloads take roughly 200ms. The factors affecting reload time are configuration size and details, the number of TLS certificates/keys, enabled modules, and available CPU resources. #### Potential problems Most of the time, if `nginx -s reload` executes, the reload will also succeed. In the rare case a reload fails, the NGINX master process will print the an error message. This is an example: ```shell 2022/07/09 00:56:42 [emerg] 1353#1353: limit_req "one" uses the "$remote_addr" key while previously it used the "$binary_remote_addr" key ``` The operation is graceful; reloading doesn't lead to any traffic loss by NGINX. However, frequent reloads can lead to high memory utilization and potential OOM (Out-Of-Memory) errors, resulting in traffic loss. This can most likely happen if you (1) proxy traffic that utilizes long-lived connections (ex: Websockets, gRPC) and (2) reload frequently. In these scenarios, you can end up with multiple generations of NGINX worker processes that are shutting down which will force old workers to shut down after the timeout). Eventually, all those worker processes can exhaust the system's available memory. Old NGINX workers will not shut down until all connections are terminated either by clients or backends, unless you configure [worker_shutdown_timeout](https://nginx.org/en/docs/ngx_core_module.html#worker_shutdown_timeout). Since both the old and new NGINX worker processes coexist during a reload, reloading can lead to two spikes in memory utilization. With a lack of available memory, the NGINX master process can fail to create new worker processes. ### Reloading in NGINX Ingress Controller LTS NGINX Ingress Controller LTS reloads NGINX to apply configuration changes. To facilitate reloading, NGINX Ingress Controller LTS configures a server listening on the Unix socket `unix:/var/lib/nginx/nginx-config-version.sock` that responds with the configuration version for `/configVersion` URI. NGINX Ingress Controller LTS writes the configuration to `/etc/nginx/config-version.conf`. Reloads occur with this sequence of steps: 1. NGINX Ingress Controller LTS updates generated configuration files, including any secrets. 1. NGINX Ingress Controller LTS updates the config version in `/etc/nginx/config-version.conf`. 1. NGINX Ingress Controller LTS runs `nginx -s reload`. If the command fails, NGINX Ingress Controller LTS logs the error and considers the reload failed. 1. If the command succeeds, NGINX Ingress Controller LTS periodically checks for the config version by sending an HTTP request to the config version server on `unix:/var/lib/nginx/nginx-config-version.sock`. 1. Once NGINX Ingress Controller LTS sees the correct config version returned by NGINX, it considers the reload successful. If it doesn't see the correct configuration version after the configurable timeout ( [`-nginx-reload-timeout`](/nic/lts/configuration/global-configuration/command-line-arguments)), NGINX Ingress Controller LTS considers the reload failed. The [NGINX Ingress Controller LTS Control Loop](#the-control-loop) stops during a reload so that it cannot affect configuration files or reload NGINX until the current reload succeeds or fails. ### When NGINX Ingress Controller LTS reloads NGINX NGINX Ingress Controller LTS reloads NGINX every time the Control Loop processes a change that affects the generated NGINX configuration. In general, every time a monitored resource is changed, NGINX Ingress Controller LTS will regenerate the configuration and reload NGINX. There are three special cases: - *Start*. When NGINX Ingress Controller LTS starts, it processes all resources in the cluster and only then reloads NGINX. This avoids a "reload storm" by reloading only once. - *Batch updates*. When NGINX Ingress Controller LTS receives a number of concurrent requests from the Kubernetes API, it will pause NGINX reloads until the task queue is empty. This reduces the number of reloads to minimize the impact of batch updates and reduce the risk of OOM (Out of Memory) errors. - *Endpoint updates*. NGINX Ingress Controller LTS uses the NGINX Plus API to update the corresponding upstreams for endpoint changes and skips reloading. -------------------------------------------------------------------------------- title: "Troubleshooting common issues" url: https://docs.nginx.com/nginx-ingress-controller/lts/troubleshooting/troubleshoot-common/index.md -------------------------------------------------------------------------------- This page describes how to troubleshoot common issues with NGINX Ingress Controller LTS. Instruction for specific resources is available in the [Troubleshooting](/nic/lts/troubleshooting/) section. ## Common issues | Problem Area | Symptom | Troubleshooting Method | Common Cause | |-----|-----|-----|-----| | Startup | NGINX Ingress Controller LTS fails to start. | Check the logs. | Misconfigured RBAC, a missing default server TLS Secret.| | Ingress resource and annotations | The configuration is not applied | Check the events of the Ingress resource, check the logs, check the generated config. | Invalid values of annotations. | | VirtualServer and VirtualServerRoute resources | The configuration is not applied. | Check the events of the VirtualServer and VirtualServerRoutes, check the logs, check the generated config. | VirtualServer or VirtualServerRoute is invalid. | | Policy resource | The configuration is not applied. | Check the events of the Policy resource as well as the events of the VirtualServers that reference that policy, check the logs, check the generated config. | Policy is invalid. | | ConfigMap keys | The configuration is not applied. | Check the events of the ConfigMap, check the logs, check the generated config. | Invalid values of ConfigMap keys. | | NGINX | NGINX responds with unexpected responses. | Check the logs, check the generated config, check the live activity dashboard, run NGINX in the debug mode. | Unhealthy backend pods, a misconfigured backend service. | ## Troubleshooting Methods This section explains how to gather additional information for troubleshooting. The commands examples make the following assumptions: - That NGINX Ingress Controller LTS is deployed in the namespace `nginx-ingress`. - `` is the name of one of the NGINX Ingress Controller LTS pods. ### Check NGINX Ingress Controller LTS logs To check NGINX Ingress Controller LTS logs, which include both information from NGINX Ingress Controller LTS and NGINX's access and error logs, run the following command: ```shell kubectl logs -n nginx-ingress ``` ### Check the generated configuration files For each Ingress/VirtualServer resource, NGINX Ingress Controller LTS generates a corresponding NGINX configuration file in the `/etc/nginx/conf.d folder`. Additionally, NGINX Ingress Controller LTS generates the main configuration file `/etc/nginx/nginx.conf`, which includes all the configurations files from `/etc/nginx/conf.d`. The configuration for a VirtualServerRoute resource is located in the configuration file of the VirtualServer that references the resource. You can view the content of the main configuration file by running: ```shell kubectl exec -n nginx-ingress -- cat /etc/nginx/nginx.conf ``` Similarly, you can view the content of any generated configuration file in the `/etc/nginx/conf.d` folder. You can also print all NGINX configuration files together: ```shell kubectl exec -n nginx-ingress -- nginx -T ``` However, this command will fail if any of the configuration files is not valid. ### Check the Live Activity Monitoring Dashboard The live activity monitoring dashboard shows the real-time information about NGINX Plus and the applications it is load balancing, which is helpful for troubleshooting. To access the dashboard, read the [Status Page](/nic/lts/logging-and-monitoring/status-page.md) topic. ### Enable debugging for NGINX Ingress Controller LTS For additional NGINX Ingress Controller LTS debugging, you can enable debug settings to get more verbose logging. Increasing the debug log levels for NGINX Ingress Controller LTS will also apply to NGINX itself. There are two places to configure more verbose logging for NGINX Ingress Controller LTS: 1. Command line arguments 1. Configmap settings **Command line arguments** When using `manifest` for deployment, use the command line argument `-nginx-debug` in your deployment, daemonset, or statefulset. You can add the `-log-level` parameter to increase the verbosity of the NGINX Ingress Controller LTS process. Here is a small snippet of setting these command line arguments in the `args` section of a deployment: ```yaml args: - -nginx-configmaps=$(POD_NAMESPACE)/nginx-config - -enable-cert-manager - -nginx-debug - -log-level=debug ``` **ConfigMap settings** You can configure `error-log-level` in NGINX Ingress Controller LTS `configMap`: ```yaml kind: ConfigMap apiVersion: v1 metadata: name: nginx-config namespace: nginx-ingress data: error-log-level: "debug" ``` **Using Helm** If you are using `helm`, you can adjust these two settings: ```none controller.nginxDebug = true or false controller.loglevel = fatal, error, warn, info, debug or trace ``` For example, if using a `values.yaml` file: ```yaml ## Enables debugging for NGINX. Uses the nginx-debug binary. Requires error-log-level: debug in ConfigMap via `controller.config.entries`. nginxDebug: true ## The log level of the Ingress Controller. logLevel: debug ``` This is a more complete example `values.yaml` file: ```yaml controller: kind: Deployment nginxDebug: true logLevel: debug annotations: nginx: ingress-prod pod: annotations: prometheus.io/scrape: "true" prometheus.io/port: "9113" prometheus.io/scheme: http extraLabels: env: prod-weset nginxplus: plus image: repository: private-registry.nginx.com/nginx-ic/lts/nginx-plus-ingress tag: # NGINX Configmap config: entries: error-log-level: "debug" proxy_connet_timeout: "5s" http-snippets: | underscores_in_headers on; ingressClass: nginx ``` Enable the `nginx-debug` CLI argument and change the `error-log-level` to `debug` to capture more output for debugging. **note:** It is recommended to only enable `nginx-debug` CLI and the `error-log-level` when debugging. #### Example debug NGINX Ingress Controller LTS Output These logs show some of the additional entries when debugging is enabled for NGINX Ingress Controller LTS. ```shell I1026 15:39:03.269092 1 manager.go:301] Reloading nginx with configVersion: 1 I1026 15:39:03.269115 1 utils.go:17] executing /usr/sbin/nginx-debug -s reload -e stderr 2022/10/26 15:39:03 [notice] 19#19: signal 1 (SIGHUP) received from 42, reconfiguring 2022/10/26 15:39:03 [debug] 19#19: wake up, sigio 0 2022/10/26 15:39:03 [notice] 19#19: reconfiguring 2022/10/26 15:39:03 [debug] 19#19: posix_memalign: 000056362AF0A420:16384 @16 2022/10/26 15:39:03 [debug] 19#19: add cleanup: 000056362AF0C318 2022/10/26 15:39:03 [debug] 19#19: posix_memalign: 000056362AF48230:16384 @16 2022/10/26 15:39:03 [debug] 19#19: malloc: 000056362AF00DE0:4096 2022/10/26 15:39:03 [debug] 19#19: read: 46, 000056362AF00DE0, 3090, 0 2022/10/26 15:39:03 [debug] 19#19: posix_memalign: 000056362AF58670:16384 @16 2022/10/26 15:39:03 [debug] 19#19: malloc: 000056362AF12440:4280 2022/10/26 15:39:03 [debug] 19#19: malloc: 000056362AF13500:4280 2022/10/26 15:39:03 [debug] 19#19: malloc: 000056362AF145C0:4280 2022/10/26 15:39:03 [debug] 19#19: malloc: 000056362AF5C680:4280 2022/10/26 15:39:03 [debug] 19#19: malloc: 000056362AF5D740:4280 2022/10/26 15:39:03 [debug] 19#19: malloc: 000056362AF5E800:4280 2022/10/26 15:39:03 [debug] 19#19: posix_memalign: 000056362AF5F8C0:16384 @16 2022/10/26 15:39:03 [debug] 19#19: malloc: 000056362AF41500:4096 2022/10/26 15:39:03 [debug] 19#19: malloc: 000056362AF638D0:8192 2022/10/26 15:39:03 [debug] 19#19: include /etc/nginx/mime.types 2022/10/26 15:39:03 [debug] 19#19: include /etc/nginx/mime.types 2022/10/26 15:39:03 [debug] 19#19: malloc: 000056362AF658E0:4096 2022/10/26 15:39:03 [debug] 19#19: malloc: 000056362AF668F0:5349 2022/10/26 15:39:03 [debug] 19#19: read: 47, 000056362AF658E0, 4096, 0 2022/10/26 15:39:03 [debug] 19#19: malloc: 000056362AF67DE0:4096 2022/10/26 15:39:03 [debug] 19#19: read: 47, 000056362AF658E1, 1253, 4096 2022/10/26 15:39:03 [debug] 19#19: posix_memalign: 000056362AF68DF0:16384 @16 2022/10/26 15:39:03 [debug] 19#19: posix_memalign: 000056362AF6CE00:16384 @16 2022/10/26 15:39:03 [debug] 19#19: malloc: 000056362AF70E10:524288 2022/10/26 15:39:03 [debug] 19#19: malloc: 000056362AFF0E20:524288 2022/10/26 15:39:03 [debug] 19#19: malloc: 000056362B070E30:524288 2022/10/26 15:39:03 [debug] 19#19: malloc: 000056362B0F0E40:400280 ``` Once you have completed your debugging process, you can change the values back to the original values. -------------------------------------------------------------------------------- title: "Troubleshooting common issues" url: https://docs.nginx.com/nginx-ingress-controller/troubleshooting/troubleshoot-common/index.md -------------------------------------------------------------------------------- This page describes how to troubleshoot common issues with NGINX Ingress Controller. Instruction for specific resources is available in the [Troubleshooting](/nic/troubleshooting/) section. ## Common issues | Problem Area | Symptom | Troubleshooting Method | Common Cause | |-----|-----|-----|-----| | Startup | NGINX Ingress Controller fails to start. | Check the logs. | Misconfigured RBAC, a missing default server TLS Secret.| | Ingress resource and annotations | The configuration is not applied | Check the events of the Ingress resource, check the logs, check the generated config. | Invalid values of annotations. | | VirtualServer and VirtualServerRoute resources | The configuration is not applied. | Check the events of the VirtualServer and VirtualServerRoutes, check the logs, check the generated config. | VirtualServer or VirtualServerRoute is invalid. | | Policy resource | The configuration is not applied. | Check the events of the Policy resource as well as the events of the VirtualServers that reference that policy, check the logs, check the generated config. | Policy is invalid. | | ConfigMap keys | The configuration is not applied. | Check the events of the ConfigMap, check the logs, check the generated config. | Invalid values of ConfigMap keys. | | NGINX | NGINX responds with unexpected responses. | Check the logs, check the generated config, check the live activity dashboard (NGINX Plus only), run NGINX in the debug mode. | Unhealthy backend pods, a misconfigured backend service. | ## Troubleshooting Methods This section explains how to gather additional information for troubleshooting. The commands examples make the following assumptions: - That NGINX Ingress Controller is deployed in the namespace `nginx-ingress`. - `` is the name of one of the NGINX Ingress Controller pods. ### Check NGINX Ingress Controller logs To check NGINX Ingress Controller logs, which include both information from NGINX Ingress Controller and NGINX's access and error logs, run the following command: ```shell kubectl logs -n nginx-ingress ``` ### Check the generated configuration files For each Ingress/VirtualServer resource, NGINX Ingress Controller generates a corresponding NGINX configuration file in the `/etc/nginx/conf.d folder`. Additionally, NGINX Ingress Controller generates the main configuration file `/etc/nginx/nginx.conf`, which includes all the configurations files from `/etc/nginx/conf.d`. The configuration for a VirtualServerRoute resource is located in the configuration file of the VirtualServer that references the resource. You can view the content of the main configuration file by running: ```shell kubectl exec -n nginx-ingress -- cat /etc/nginx/nginx.conf ``` Similarly, you can view the content of any generated configuration file in the `/etc/nginx/conf.d` folder. You can also print all NGINX configuration files together: ```shell kubectl exec -n nginx-ingress -- nginx -T ``` However, this command will fail if any of the configuration files is not valid. ### Check the Live Activity Monitoring Dashboard The live activity monitoring dashboard shows the real-time information about NGINX Plus and the applications it is load balancing, which is helpful for troubleshooting. To access the dashboard, read the [Status Page](/nic/logging-and-monitoring/status-page.md) topic. ### Enable debugging for NGINX Ingress Controller For additional NGINX Ingress Controller debugging, you can enable debug settings to get more verbose logging. Increasing the debug log levels for NGINX Ingress Controller will also apply to NGINX itself. There are two places to configure more verbose logging for NGINX Ingress Controller: 1. Command line arguments 1. Configmap settings **Command line arguments** When using `manifest` for deployment, use the command line argument `-nginx-debug` in your deployment, daemonset, or statefulset. You can add the `-log-level` parameter to increase the verbosity of the NGINX Ingress Controller process. Here is a small snippet of setting these command line arguments in the `args` section of a deployment: ```yaml args: - -nginx-configmaps=$(POD_NAMESPACE)/nginx-config - -enable-cert-manager - -nginx-debug - -log-level=debug ``` **ConfigMap settings** You can configure `error-log-level` in NGINX Ingress Controller `configMap`: ```yaml kind: ConfigMap apiVersion: v1 metadata: name: nginx-config namespace: nginx-ingress data: error-log-level: "debug" ``` **Using Helm** If you are using `helm`, you can adjust these two settings: ```none controller.nginxDebug = true or false controller.loglevel = fatal, error, warn, info, debug or trace ``` For example, if using a `values.yaml` file: ```yaml ## Enables debugging for NGINX. Uses the nginx-debug binary. Requires error-log-level: debug in ConfigMap via `controller.config.entries`. nginxDebug: true ## The log level of the Ingress Controller. logLevel: debug ``` This is a more complete example `values.yaml` file: ```yaml controller: kind: Deployment nginxDebug: true logLevel: debug annotations: nginx: ingress-prod pod: annotations: prometheus.io/scrape: "true" prometheus.io/port: "9113" prometheus.io/scheme: http extraLabels: env: prod-weset nginxplus: plus image: repository: nginx/nginx-ingress tag: # NGINX Configmap config: entries: error-log-level: "debug" proxy_connet_timeout: "5s" http-snippets: | underscores_in_headers on; ingressClass: nginx ``` Enable the `nginx-debug` CLI argument and change the `error-log-level` to `debug` to capture more output for debugging. **Note:** It is recommended to only enable `nginx-debug` CLI and the `error-log-level` when debugging. #### Example debug NGINX Ingress Controller Output These logs show some of the additional entries when debugging is enabled for NGINX Ingress Controller. ```shell I1026 15:39:03.269092 1 manager.go:301] Reloading nginx with configVersion: 1 I1026 15:39:03.269115 1 utils.go:17] executing /usr/sbin/nginx-debug -s reload -e stderr 2022/10/26 15:39:03 [notice] 19#19: signal 1 (SIGHUP) received from 42, reconfiguring 2022/10/26 15:39:03 [debug] 19#19: wake up, sigio 0 2022/10/26 15:39:03 [notice] 19#19: reconfiguring 2022/10/26 15:39:03 [debug] 19#19: posix_memalign: 000056362AF0A420:16384 @16 2022/10/26 15:39:03 [debug] 19#19: add cleanup: 000056362AF0C318 2022/10/26 15:39:03 [debug] 19#19: posix_memalign: 000056362AF48230:16384 @16 2022/10/26 15:39:03 [debug] 19#19: malloc: 000056362AF00DE0:4096 2022/10/26 15:39:03 [debug] 19#19: read: 46, 000056362AF00DE0, 3090, 0 2022/10/26 15:39:03 [debug] 19#19: posix_memalign: 000056362AF58670:16384 @16 2022/10/26 15:39:03 [debug] 19#19: malloc: 000056362AF12440:4280 2022/10/26 15:39:03 [debug] 19#19: malloc: 000056362AF13500:4280 2022/10/26 15:39:03 [debug] 19#19: malloc: 000056362AF145C0:4280 2022/10/26 15:39:03 [debug] 19#19: malloc: 000056362AF5C680:4280 2022/10/26 15:39:03 [debug] 19#19: malloc: 000056362AF5D740:4280 2022/10/26 15:39:03 [debug] 19#19: malloc: 000056362AF5E800:4280 2022/10/26 15:39:03 [debug] 19#19: posix_memalign: 000056362AF5F8C0:16384 @16 2022/10/26 15:39:03 [debug] 19#19: malloc: 000056362AF41500:4096 2022/10/26 15:39:03 [debug] 19#19: malloc: 000056362AF638D0:8192 2022/10/26 15:39:03 [debug] 19#19: include /etc/nginx/mime.types 2022/10/26 15:39:03 [debug] 19#19: include /etc/nginx/mime.types 2022/10/26 15:39:03 [debug] 19#19: malloc: 000056362AF658E0:4096 2022/10/26 15:39:03 [debug] 19#19: malloc: 000056362AF668F0:5349 2022/10/26 15:39:03 [debug] 19#19: read: 47, 000056362AF658E0, 4096, 0 2022/10/26 15:39:03 [debug] 19#19: malloc: 000056362AF67DE0:4096 2022/10/26 15:39:03 [debug] 19#19: read: 47, 000056362AF658E1, 1253, 4096 2022/10/26 15:39:03 [debug] 19#19: posix_memalign: 000056362AF68DF0:16384 @16 2022/10/26 15:39:03 [debug] 19#19: posix_memalign: 000056362AF6CE00:16384 @16 2022/10/26 15:39:03 [debug] 19#19: malloc: 000056362AF70E10:524288 2022/10/26 15:39:03 [debug] 19#19: malloc: 000056362AFF0E20:524288 2022/10/26 15:39:03 [debug] 19#19: malloc: 000056362B070E30:524288 2022/10/26 15:39:03 [debug] 19#19: malloc: 000056362B0F0E40:400280 ``` Once you have completed your debugging process, you can change the values back to the original values. -------------------------------------------------------------------------------- title: "View the NGINX status page" url: https://docs.nginx.com/nginx-ingress-controller/logging-and-monitoring/status-page/index.md -------------------------------------------------------------------------------- This document explains how to get access to the stub status in NGINX and the dashboard in NGINX Plus. NGINX comes with a status page that reports basic metrics about NGINX called the [stub status](https://nginx.org/en/docs/http/ngx_http_stub_status_module.html). NGINX Plus comes with a [dashboard](/nginx/admin-guide/monitoring/live-activity-monitoring.md) that reports key load-balancing and performance metrics. F5 DoS for NGINX comes with a [dashboard](/nap-dos/monitoring/live-activity-monitoring.md) that shows the status and information of the protected objects. This doc shows how to get access to the stub status/dashboard. ## Accessing Stub Status Prerequisites: 1. The stub status is enabled by default. Ensure that the `nginx-status` [command-line argument](/nic/configuration/global-configuration/command-line-arguments.md) is not set to false. 1. The stub status is available on port 8080 by default. It is customizable by the `nginx-status-port` command-line argument. If yours is not on 8080, modify the kubectl proxy command below. To access the status: 1. Use the `kubectl port-forward` command to forward connections to port 8080 on your local machine to port 8080 of an NGINX Ingress Controller pod (replace `` with the actual name of a pod):. ```shell kubectl port-forward 8080:8080 --namespace=nginx-ingress ``` 1. Open your browser at [http://127.0.0.1:8080/stub_status](http://127.0.0.1:8080/stub_status) to access the status. If you want to access the stub status externally (without `kubectl port-forward`): 1. Configure `-nginx-status-allow-cidrs` command-line argument with IP/CIDR blocks for which you want to allow access to the status. By default, the access is allowed for `127.0.0.1,::1`. 1. Use the IP/port through which the Ingress Controller pod/pods are available to connect the stub status at the `/stub_status` path. ## Accessing Live Activity Monitoring Dashboard Prerequisites: 1. The dashboard is enabled by default. Ensure that the `nginx-status` [command-line argument](/nic/configuration/global-configuration/command-line-arguments.md) is not set to false. 1. The dashboard is available on port 8080 by default. It is customizable by the `nginx-status-port` command-line argument. If yours is not on 8080, modify the kubectl proxy command below. To access the dashboard: 1. Use the `kubectl port-forward` command to forward connections to port 8080 on your local machine to port 8080 of an NGINX Plus Ingress Controller pod (replace `` with the actual name of a pod): ```shell kubectl port-forward 8080:8080 --namespace=nginx-ingress ``` 1. Open your browser at to access the dashboard. 1. App Protect DoS: Open your browser at to access the dashboard. If you want to access the dashboard externally (without `kubectl port-forward`): 1. Configure `-nginx-status-allow-cidrs` command-line argument with IP/CIDR blocks for which you want to allow access to the dashboard. By default, the access is allowed for `127.0.0.1,::1`. 1. Use the IP/port through which the Ingress Controller pod/pods are available to connect the dashboard at the `/dashboard.html` path. **Note**: The [API](https://nginx.org/en/docs/http/ngx_http_api_module.html), which the dashboard uses to get the metrics, is also accessible: use the `/api` path. For App Protect DoS use the `/api/dos` path. Note that the API is configured in the read-only mode. -------------------------------------------------------------------------------- title: "View the NGINX status page" url: https://docs.nginx.com/nginx-ingress-controller/lts/logging-and-monitoring/status-page/index.md -------------------------------------------------------------------------------- This document explains how to get access to the dashboard in NGINX Plus. NGINX Plus comes with a [dashboard](/nginx/admin-guide/monitoring/live-activity-monitoring.md) that reports key load-balancing and performance metrics. ## Accessing Live Activity Monitoring Dashboard Prerequisites: 1. The dashboard is enabled by default. Ensure that the `nginx-status` [command-line argument](/nic/lts/configuration/global-configuration/command-line-arguments.md) is not set to false. 1. The dashboard is available on port 8080 by default. It is customizable by the `nginx-status-port` command-line argument. If yours is not on 8080, modify the kubectl proxy command below. To access the dashboard: 1. Use the `kubectl port-forward` command to forward connections to port 8080 on your local machine to port 8080 of an NGINX Plus Ingress Controller pod (replace `` with the actual name of a pod): ```shell kubectl port-forward 8080:8080 --namespace=nginx-ingress ``` 1. Open your browser at to access the dashboard. If you want to access the dashboard externally (without `kubectl port-forward`): 1. Configure `-nginx-status-allow-cidrs` command-line argument with IP/CIDR blocks for which you want to allow access to the dashboard. By default, the access is allowed for `127.0.0.1,::1`. 1. Use the IP/port through which the Ingress Controller pod/pods are available to connect the dashboard at the `/dashboard.html` path. **Note**: The [API](https://nginx.org/en/docs/http/ngx_http_api_module.html), which the dashboard uses to get the metrics, is also accessible: use the `/api` path. For App Protect DoS use the `/api/dos` path. Note that the API is configured in the read-only mode. -------------------------------------------------------------------------------- title: "Upgrade NGINX Ingress Controller to F5 WAF for NGINX with PLM" url: https://docs.nginx.com/nginx-ingress-controller/install/plm-upgrade/index.md -------------------------------------------------------------------------------- Use this guide to upgrade an existing F5 NGINX Ingress Controller + F5 WAF for NGINX deployment from in-pod App Protect policy compilation to Policy Lifecycle Management (PLM). PLM compiles `APPolicy` and `APLogConf` resources in a dedicated controller and stores the resulting bundles in an in-cluster S3-compatible object store. NGINX Ingress Controller then fetches the compiled bundles instead of compiling them in the data plane. Under PLM, the fields you already write on the NGINX Ingress Controller `Policy` resource (`waf.apPolicy` and `waf.securityLogs[].apLogConf`) continue to work unchanged. You don't need to rewrite the `k8s.nginx.org/v1` Policy manifest for a VirtualServer or Ingress during the upgrade. By the end of this guide, you'll have: - The PLM backend installed alongside your existing NGINX Ingress Controller deployment. - NGINX Ingress Controller upgraded to a PLM-capable release with PLM storage configured. - Existing `APPolicy` and `APLogConf` resources adopted by PLM and compiled into bundles. - WAF-protected traffic served by NGINX Ingress Controller, with bundles fetched from PLM storage instead of compiled in the data plane. ## Before you begin Before you start, make sure you have: - An existing NGINX Ingress Controller + F5 WAF for NGINX deployment running in your cluster. - Existing `APPolicy` and `APLogConf` resources served by the `appprotect.f5.com/v1beta1` CRDs shipped with your current NGINX Ingress Controller installation. - `kubectl` and Helm access to the cluster. - Credentials for `private-registry.nginx.com`. Record your current values before you begin: | Value | Where to find it | |---|---| | NGINX Ingress Controller release name | `helm list --namespace ` | | NGINX Ingress Controller chart version | `helm list --namespace ` | | Existing NGINX Ingress Controller values | `helm get values --namespace ` | | Existing `APPolicy` resources | `kubectl get appolicy --all-namespaces` | | Existing `APLogConf` resources | `kubectl get aplogconf --all-namespaces` | This guide uses `nic` as the NGINX Ingress Controller release name, `nginx-ingress` as the NGINX Ingress Controller namespace, `plm-system` as the PLM namespace, and `plm` as the PLM release name. Replace these with your own values consistently throughout. ## Deploy PLM infrastructure Install the PLM backend into the `plm-system` namespace. The install adds the `appprotect.f5.com/v1` versions to the existing `appprotect.f5.com` CRDs. Because the v1 versions are a superset of v1beta1, adding them doesn't affect NGINX Ingress Controller while it still watches v1beta1. The Policy Lifecycle Manager (PLM) backend runs as a Kubernetes operator. It watches WAF custom resources and compiles WAF policies into bundles. The Policy Controller delegates compilation to a separate compiler service over gRPC. The resulting bundles are stored in an embedded SeaweedFS S3-compatible object store. F5 WAF for NGINX is installed using a separate Helm chart from your NGINX data plane. The steps in this section install only the F5 WAF for NGINX PLM components and do not affect your existing NGINX installation. ### Create the registry pull secret Create a namespace for the PLM components, store your JWT in a Kubernetes Secret, then create the registry pull secret for the private F5 container registry. 1. Create the namespace and store your JWT. The following commands assume your JWT file is named `license.jwt`: ```shell kubectl create namespace plm-system kubectl create secret generic jwt-reg-secret \ --namespace plm-system \ --from-file=license.jwt ``` 2. Retrieve the JWT from the Secret and create the registry pull secret: ```shell JWT=$(kubectl get secret jwt-reg-secret \ --namespace plm-system \ -o jsonpath='{.data.license\.jwt}' | base64 -d) kubectl create secret docker-registry regcred \ --namespace plm-system \ --docker-server=private-registry.nginx.com \ --docker-username="$JWT" \ --docker-password=none \ --dry-run=client --output yaml | kubectl apply -f - ``` ### Install the Policy Controller Create a values file for the Helm installation. The `securityUpdatesRepo.cert` and `securityUpdatesRepo.key` fields are optional. They are only required if your signature repository needs certificate-based authentication. The Policy Controller starts successfully with these fields left empty. If your signature repository requires them, replace `` and `` with the base64-encoded contents of your `nginx-repo.crt` and `nginx-repo.key` files. To encode them, run: ```shell base64 --wrap=0 < nginx-repo.crt base64 --wrap=0 < nginx-repo.key ``` Create `/tmp/plm-values.yaml`: ```yaml imagePullSecrets: - name: regcred securityUpdatesRepo: cert: "" # optional: only needed for authenticated signature repository access key: "" # optional: only needed for authenticated signature repository access policyController: image: tag: "" compiler: image: tag: "" seaweedfsOperatorConfig: seaweedfs: image: tag: "" seaweedfs-operator: image: tag: "" pullSecrets: regcred ``` #### Enable TLS for PLM storage (optional) By default, communication between PLM components and the SeaweedFS object store uses unencrypted HTTP. To enable TLS, add a `certificates` block to `/tmp/plm-values.yaml`: ```yaml seaweedfsOperatorConfig: seaweedfs: certificates: enabled: true ``` **Note:** The PLM chart does not generate certificates. You must create the five Secrets listed in the commands below before running `helm upgrade --install`. If any Secret is missing, the SeaweedFS pods will fail to mount their certificates and will not start. **Note:** If you're enabling TLS on an existing installation, the storage backend restarts and objects written before the switch can become orphaned. See the [APPolicy shows `invalid` with `unexpected EOF` after enabling TLS](#troubleshoot-the-deployment) entry in the troubleshooting section. A fresh installation with TLS enabled from the start doesn't have this issue. Create the Secrets from your CA and certificate files before installing. The chart expects Secret names in the form `-f5-waf-seaweedfs-` — for the `plm` release name used in this tutorial, those are: ```shell kubectl create secret generic plm-f5-waf-seaweedfs-ca-cert \ --namespace plm-system \ --from-file=tls.crt= \ --from-file=ca.crt= kubectl create secret tls plm-f5-waf-seaweedfs-master-cert \ --namespace plm-system \ --cert= \ --key= kubectl create secret tls plm-f5-waf-seaweedfs-volume-cert \ --namespace plm-system \ --cert= \ --key= kubectl create secret tls plm-f5-waf-seaweedfs-filer-cert \ --namespace plm-system \ --cert= \ --key= kubectl create secret tls plm-f5-waf-seaweedfs-client-cert \ --namespace plm-system \ --cert= \ --key= ``` The CA Secret requires both `tls.crt` and `ca.crt` keys, both pointing to the same CA certificate file. The PLM chart mounts the CA using `tls.crt` into the Policy Controller, compiler, and SeaweedFS pods. The data plane's S3 client reads `ca.crt` from the same Secret when verifying the storage endpoint. The four component Secrets use `kubectl create secret tls`, which produces `tls.crt` and `tls.key` — no `ca.crt` key is needed for them. Replace each `` placeholder with the path to the corresponding certificate and key file from your PKI. The CA must sign all component certificates. If you don't have an existing PKI, generate a CA and sign the five component certificates before proceeding. #### Install the chart Add the NGINX Helm repository and install the chart: ```shell helm repo add nginx-stable https://helm.nginx.com/stable helm repo update nginx-stable helm upgrade --install plm nginx-stable/f5-waf-policy-controller \ --version \ --namespace plm-system \ --values /tmp/plm-values.yaml ``` To see all available configuration options for the PLM chart, run: ```shell helm show values nginx-stable/f5-waf-policy-controller --version ``` ### Verify the deployment Wait for all PLM components to become ready. The Policy Controller's init container waits for both the compiler service and the SeaweedFS S3 endpoint to be available before it starts, so the controller pod will show `Init:0/1` until SeaweedFS is ready. Wait for the SeaweedFS storage backend: ```shell kubectl rollout status deployment/plm-seaweedfs-operator \ --namespace plm-system --timeout=120s ``` The SeaweedFS operator creates the SeaweedFS pods after it reconciles the SeaweedFS custom resource, so there is a window where the operator deployment is ready but no SeaweedFS pods exist yet. Poll until the pods appear and are ready: ```shell end=$((SECONDS + 300)) until kubectl wait pods \ --selector app.kubernetes.io/name=seaweedfs \ --for=condition=Ready \ --namespace plm-system \ --timeout=10s 2>/dev/null; do if [ $SECONDS -ge $end ]; then echo "Timed out waiting for SeaweedFS pods" exit 1 fi sleep 5 done ``` Wait for the Policy Controller: ```shell kubectl rollout status deployment/plm-f5-waf-policy-controller \ --namespace plm-system --timeout=180s ``` Confirm all pods are running: ```shell kubectl get pods --namespace plm-system ``` Example output: ```text NAME READY STATUS RESTARTS plm-f5-waf-compiler-service-xxxxx 1/1 Running 0 plm-f5-waf-policy-controller-xxxxx 1/1 Running 0 plm-seaweedfs-operator-xxxxx 1/1 Running 0 plm-f5-waf-seaweed-master-0 1/1 Running 0 plm-f5-waf-seaweed-filer-0 1/1 Running 0 plm-f5-waf-seaweed-volume-0 1/1 Running 0 plm-f5-waf-seaweed-volume-1 1/1 Running 0 plm-f5-waf-seaweed-volume-2 1/1 Running 0 ``` Confirm the CRDs are present: ```shell kubectl get crd | grep appprotect.f5.com ``` Expected output: ```text aplogconfs.appprotect.f5.com appolicies.appprotect.f5.com apsignatures.appprotect.f5.com apusersigs.appprotect.f5.com ``` All eight pods running and all four CRDs present confirms the PLM backend is ready. ### Update the CRDs **Note:** Skip this step on a fresh install — Helm installs the CRDs automatically. Only follow these steps when upgrading an existing PLM installation. When upgrading PLM, apply the CRDs manually before running `helm upgrade`: ```shell kubectl apply -f https://raw.githubusercontent.com/nginx/waf-policy-controller//manifests/1-deploy-crds.yaml ``` ### Troubleshoot the deployment These are the most common failures during PLM installation, roughly in order of likelihood. #### Pods stuck in `ImagePullBackOff` The JWT is wrong, expired, or contains a line break. Check the events log: ```shell kubectl get events --namespace plm-system --field-selector reason=Failed ``` Use the full JWT string as the registry username. Use the literal string `none` as the password. #### Policy Controller stuck in `Init:0/1` The `Init:0/1` state is expected during startup. The init container waits for the compiler service and the S3 endpoint before it starts. If the pod stays in `Init:0/1` for more than a few minutes, check that the SeaweedFS pods are `Running`: ```shell kubectl get pods --namespace plm-system --selector app.kubernetes.io/name=seaweedfs ``` The most common cause is PVCs stuck in `Pending` because the cluster has no default StorageClass. #### SeaweedFS pods `Pending` SeaweedFS pods stay `Pending` when the cluster has no default StorageClass or insufficient capacity. Check the PVCs and available storage classes: ```shell kubectl get pvc --namespace plm-system kubectl get storageclass ``` #### `APPolicy` shows `invalid` with `unexpected EOF` after enabling TLS Enabling TLS on an existing installation restarts the storage backend. Objects written before TLS was enabled can become orphaned. Check the filer log: ```shell kubectl logs --namespace plm-system plm-f5-waf-seaweed-filer-0 | grep "not found" ``` If the output contains `volume N not found`, orphaned objects exist. Delete the affected `APPolicy` resource and reapply it. The Policy Controller regenerates the bundle. #### Helm install fails on a ClusterRole If the error references `seaweed-editor-role` or `seaweed-viewer-role`, another PLM installation already exists in the cluster. Only one PLM installation is supported per cluster. Remove the existing release before installing. #### Check the Policy Controller logs Use the Policy Controller logs to diagnose any policy-related failure: ```shell kubectl logs --namespace plm-system deploy/plm-f5-waf-policy-controller -c policy-controller ``` **Note:** The `-c policy-controller` flag is required because the pod has more than one container. The containers are distroless, so `kubectl exec` isn't available for interactive debugging. After the install completes, PLM adopts every existing `APPolicy` and `APLogConf` resource in the cluster. It does this by adding the `appprotect.f5.com/finalizer` finalizer and compiling each resource against its current signature package. Your running NGINX Ingress Controller continues to compile the same resources in-pod. The two mechanisms operate independently until you upgrade NGINX Ingress Controller. Confirm PLM has compiled the existing resources: ```shell kubectl get appolicy --all-namespaces \ --output custom-columns='NAMESPACE:.metadata.namespace,NAME:.metadata.name,STATE:.status.bundle.state' kubectl get aplogconf --all-namespaces \ --output custom-columns='NAMESPACE:.metadata.namespace,NAME:.metadata.name,STATE:.status.bundle.state' ``` Every resource should report `STATE: ready`. If any resource isn't `ready`, inspect the PLM policy-controller logs, resolve the compilation errors, and then continue. ## Look up the PLM storage endpoint and credentials NGINX Ingress Controller connects to PLM's SeaweedFS filer over S3 to fetch compiled bundles. Before you run `helm upgrade`, collect these four values from the PLM installation: 1. **PLM storage URL**: the SeaweedFS filer endpoint (HTTPS or HTTP). 2. **Credentials Secret**: the S3 credentials Secret. The access key ID is `admin` by default. The secret access key is in the `seaweedfs_admin_secret` field. 3. **CA Secret** (HTTPS only): verifies the SeaweedFS filer certificate. 4. **Client TLS Secret** (mutual TLS only): presented by NGINX Ingress Controller when it connects to the filer. List the Services PLM created and identify the filer: ```shell kubectl get service --namespace plm-system ``` Expected output includes an entry similar to: ```text NAME TYPE CLUSTER-IP PORT(S) plm-f5-waf-seaweed-filer ClusterIP 10.0.0.10 8333/TCP,9333/TCP,... ``` Assemble the URL from the service name, namespace, and port. Use `9333` for HTTPS and `8333` for HTTP: - HTTPS (mTLS): `https://plm-f5-waf-seaweed-filer.plm-system.svc.cluster.local:9333` - HTTP: `http://plm-f5-waf-seaweed-filer.plm-system.svc.cluster.local:8333` List the Secrets PLM created: ```shell kubectl get secret --namespace plm-system ``` The default PLM install creates three Secrets that NGINX Ingress Controller references: - `plm-f5-waf-seaweedfs-auth`: SeaweedFS credentials. - `plm-f5-waf-seaweedfs-ca-cert`: CA certificate for the HTTPS filer. - `plm-f5-waf-seaweedfs-client-cert`: client TLS certificate for mTLS. Record the Secret references in `/` form. When you run `helm upgrade`, pass all four values to NGINX Ingress Controller using `--set controller.appprotect.plmStorage.*` flags. ## Apply the NGINX Ingress Controller CRDs Before you run `helm upgrade`, apply the NGINX Ingress Controller CRDs from the bundled manifest for your target release. The `deploy/crds.yaml` bundle contains every CRD the controller needs (`VirtualServer`, `VirtualServerRoute`, `Policy`, `TransportServer`, `GlobalConfiguration`, `DNSEndpoint`). The bundle deliberately excludes the App Protect CRDs, which PLM owns. ```shell kubectl apply -f https://raw.githubusercontent.com/nginx/kubernetes-ingress/v/deploy/crds.yaml ``` ## Section 1: Upgrade a VirtualServer-based deployment Follow this section if your existing NGINX Ingress Controller deployment routes traffic through `k8s.nginx.org/v1` VirtualServer resources that reference a Policy resource with `waf.apPolicy` and `waf.securityLogs[].apLogConf` fields. ### Upgrade NGINX Ingress Controller with PLM storage Upgrade the NGINX Ingress Controller release. `--reuse-values` preserves your existing configuration. The `--set` flags overlay PLM storage on top. `--skip-crds` prevents Helm from touching CRDs, because you applied the NGINX Ingress Controller CRDs in the previous step and PLM owns the App Protect CRDs. This example uses HTTPS PLM storage with mutual TLS. For HTTP storage, set `controller.appprotect.plmStorage.url` to `http://:8333` and omit the `caSecret` and `clientSSLSecret` flags. ```shell helm upgrade nic nginx-stable/nginx-ingress \ --namespace nginx-ingress \ --skip-crds \ --reuse-values \ --set controller.appprotect.enable=true \ --set controller.appprotect.v5=true \ --set controller.image.repository="private-registry.nginx.com/nginx-ic-nap-v5/nginx-plus-ingress" \ --set controller.image.tag="" \ --set controller.appprotect.plmStorage.url="https://plm-f5-waf-seaweed-filer.plm-system.svc.cluster.local:9333" \ --set controller.appprotect.plmStorage.credentialsSecret="plm-system/plm-f5-waf-seaweedfs-auth" \ --set controller.appprotect.plmStorage.caSecret="plm-system/plm-f5-waf-seaweedfs-ca-cert" \ --set controller.appprotect.plmStorage.clientSSLSecret="plm-system/plm-f5-waf-seaweedfs-client-cert" \ --set controller.appprotect.plmStorage.insecureSkipVerify=false ``` Wait for the rollout to complete: ```shell kubectl rollout status deployment/nic-nginx-ingress-controller \ --namespace nginx-ingress \ --timeout=180s ``` Confirm NGINX Ingress Controller is watching the v1 CRDs by inspecting the controller log: ```shell kubectl logs deployment/nic-nginx-ingress-controller \ --namespace nginx-ingress \ --container nginx-ingress | grep 'appprotect.f5.com/v' ``` Expected output includes: ```text Using appprotect.f5.com/v1 CRDs ``` ### Verify Policy status Check each WAF Policy referenced by a VirtualServer: ```shell kubectl get policy --all-namespaces \ --output custom-columns='NAMESPACE:.metadata.namespace,NAME:.metadata.name,STATE:.status.state,REASON:.status.reason' ``` Expected output after the first successful fetch: ```text NAMESPACE NAME STATE REASON default waf-policy Valid AddedOrUpdated ``` If a Policy remains in `Warning` with `BundlePending`, the referenced `APPolicy` or `APLogConf` isn't yet `ready` in PLM. Confirm that PLM has compiled the resource, then retry. ### Verify traffic Send a normal request to the VirtualServer and confirm the application responds: ```shell curl --resolve webapp.example.com:$IC_HTTP_PORT:$IC_IP \ http://webapp.example.com:$IC_HTTP_PORT/ ``` Then send a request that triggers the configured WAF violation and confirm the response is `Request Rejected`. ### Confirm bundles come from PLM storage Check the bundle files in the ingress controller pod: ```shell NIC_POD=$(kubectl get pods --namespace nginx-ingress \ --selector app.kubernetes.io/name=nginx-ingress \ --output jsonpath='{.items[0].metadata.name}') kubectl exec --namespace nginx-ingress $NIC_POD --container nginx-ingress -- \ ls -l /etc/app_protect/bundles/ ``` Files named `fetched___policy.tgz` and `fetched___log_.tgz` confirm that the bundles were fetched from PLM storage. ## Section 2: Upgrade an Ingress-based deployment Follow this section if your existing NGINX Ingress Controller deployment routes traffic through Kubernetes Ingress resources. The procedure is the same as Section 1, with one additional Ingress-specific step. ### Audit Ingress annotations Under PLM, NGINX Ingress Controller doesn't support the App Protect Ingress annotations: - `appprotect.f5.com/app-protect-policy` - `appprotect.f5.com/app-protect-security-log` - `appprotect.f5.com/app-protect-security-log-enable` An Ingress that uses these annotations is accepted but produces a warning after the upgrade. WAF isn't applied to that route. Before you turn on PLM storage, migrate every such Ingress to a `k8s.nginx.org/v1` Policy resource. ### Upgrade NGINX Ingress Controller with PLM storage Run the same `helm upgrade` command shown in [Section 1, step 1](#1-upgrade-nginx-ingress-controller-with-plm-storage). ### Verify Ingress traffic For each Ingress, send a normal request and confirm the application responds. Then send a request that matches a WAF violation and confirm the response is `Request Rejected`. ### Confirm bundles come from PLM storage Use the same procedure as [Section 1, step 4](#4-confirm-bundles-come-from-plm-storage). ## Troubleshooting - **Policy stays in `BundlePending` after the upgrade.** The referenced `APPolicy` or `APLogConf` isn't `ready` in PLM. Run `kubectl describe appolicy ` and inspect the PLM policy-controller logs. - **NGINX Ingress Controller reports the referenced namespace isn't watched.** If the deployment sets `controller.watchNamespace`, include the namespace of every `APPolicy` and `APLogConf` resource. Also include the PLM namespace in `controller.watchSecretNamespace` so NGINX Ingress Controller can observe storage Secret rotation. - **Helm upgrade fails with a CRD conflict.** Confirm PLM is installed and its v1 CRDs are present, then run the upgrade with `--skip-crds`. -------------------------------------------------------------------------------- title: "Apply policies to resources" url: https://docs.nginx.com/nginx-ingress-controller/configuration/policy-resource/applying-policies/index.md -------------------------------------------------------------------------------- You can reference policies from VirtualServer, VirtualServerRoute, and Ingress resources. How you attach a policy depends on the resource type. ## VirtualServer You can attach policies at: - `spec.policies` for server-wide behavior - `spec.routes[].policies` for route-specific behavior Example: ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServer metadata: name: cafe spec: host: cafe.example.com policies: - name: access-policy upstreams: - name: coffee service: coffee-svc port: 80 routes: - path: /coffee policies: - name: route-cors-policy action: pass: coffee ``` ## VirtualServerRoute You can attach policies at: - `spec.subroutes[].policies` Example: ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServerRoute metadata: name: tea spec: host: cafe.example.com upstreams: - name: tea service: tea-svc port: 80 subroutes: - path: /tea policies: - name: subroute-policy action: pass: tea ``` ## Ingress Ingress uses annotations instead of a `policies` field. Supported annotations are: - `nginx.org/policies` - `nginx.com/policies` Example: ```yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: webapp annotations: nginx.org/policies: access-policy,cors-policy spec: ingressClassName: nginx rules: - host: webapp.example.com http: paths: - path: / pathType: Prefix backend: service: name: webapp port: number: 80 ``` ## What's next Learn about [ingress-specific policy behavior](/nic/configuration/policy-resource/ingress-specific-behavior.md). -------------------------------------------------------------------------------- title: "Compile F5 WAF for NGINX policies using NGINX Instance Manager" url: https://docs.nginx.com/nginx-ingress-controller/integrations/app-protect-waf-v5/compile-waf-policies/index.md -------------------------------------------------------------------------------- ## Overview This guide describes how to use F5 NGINX Instance Manager to compile F5 WAF for NGINX Policies for use with NGINX Ingress Controller. F5 WAF for NGINX uses policies to configure which security features are set. When these policies are changed, they need to be compiled so that the engine can begin to use them. Compiling policies can take a large amount of time and resources. You can do this with the NGINX Instance Manager. This reduces the impact on a NGINX Ingress Controller deployment. By using NGINX Instance Manager to compile WAF policies, the policy bundle can also be used immediately by NGINX Ingress Controller without reloading. **Note:** NGINX Ingress Controller can fetch compiled bundles directly from NGINX Instance Manager or NGINX One Console without the manual download and upload steps described on this page. See [Connect F5 WAF for NGINX to bundle sources](/nic/integrations/app-protect-waf-v5/bundle-sources.md) for details on automated bundle fetching with optional polling for updates. The following steps describe how to use the NGINX Instance Manager API to create a new security policy, compile a bundle, then add it to NGINX Ingress Controller. ## Before you begin - A working [NGINX Instance Manager](/nim/deploy/) instance. - An [NGINX Instance Manager user](/nim/admin-guide/rbac/overview-rbac.md) for API requests. - A NGINX Ingress Controller [deployment with F5 WAF for NGINX](/nic/integrations/app-protect-waf/installation.md). ## Create a new security policy **Note:** You can skip this step if you intend to use an existing security policy. Create a [new security policy](/nim/waf-integration/policies-and-logs/policies/create-policy.md) using the API: this will require the use of a tool such as [`curl`](https://curl.se/) or [Postman](https://www.postman.com/) Create the file `simple-policy.json` with the contents below: ```json { "metadata": { "name": "Nginxbundletest", "displayName": "Nginxbundletest", "description": "Ignore cross-site scripting is a security policy that intentionally ignores cross site scripting." }, "content": "ewoJInBvbGljeSI6IHsKCQkibmFtZSI6ICJzaW1wbGUtYmxvY2tpbmctcG9saWN5IiwKCQkic2lnbmF0dXJlcyI6IFsKCQkJewoJCQkJInNpZ25hdHVyZUlkIjogMjAwMDAxODM0LAoJCQkJImVuYWJsZWQiOiBmYWxzZQoJCQl9CgkJXSwKCQkidGVtcGxhdGUiOiB7CgkJCSJuYW1lIjogIlBPTElDWV9URU1QTEFURV9OR0lOWF9CQVNFIgoJCX0sCgkJImFwcGxpY2F0aW9uTGFuZ3VhZ2UiOiAidXRmLTgiLAoJCSJlbmZvcmNlbWVudE1vZGUiOiAiYmxvY2tpbmciCgl9Cn0=" } ``` **Note:** The `content` value must be base64 encoded or you will encounter an error. Upload the policy JSON files with the API, which is the same method to create the bundle later. In the same directory you created `simple-policy.json`, create a POST request for NGINX Instance Manager using the API. ```shell curl -X POST https://{{NMS_FQDN}}/api/platform/v1/security/policies \ -H "Authorization: Bearer " \ -H "Content-Type: application/json" \ -d @simple-policy.json ``` You should receive an API response similar to the following output, indicating the policy has been successfully created. ```json { "metadata": { "created": "2024-06-12T20:28:08.152171922Z", "description": "Ignore cross-site scripting is a security policy that intentionally ignores cross site scripting.", "displayName": "Nginxbundletest", "externalId": "", "externalIdType": "", "modified": "2024-06-12T20:28:08.152171922Z", "name": "Nginxbundletest", "revisionTimestamp": "2024-06-12T20:28:08.152171922Z", "uid": "6af9f261-658b-4be1-b07a-cebd83e917a1" }, "selfLink": { "rel": "/api/platform/v1/security/policies/6af9f261-658b-4be1-b07a-cebd83e917a1" } } ``` **Note:** Take note of the *uid* field: `"uid": "6af9f261-658b-4be1-b07a-cebd83e917a1"` It is one of two unique IDs we will use to download the bundle: it will be referenced as *policy-UID*. --- ## Create a new security bundle Once you have created (Or selected) a security policy, [create a security bundle](/nim/waf-integration/policies-and-logs/bundles/create-bundle.md) using the API. The version in the bundle you create **must** match the WAF compiler version you intend to use. You can check which version is installed in NGINX Instance Manager by checking the operating system packages. If the wrong version is noted in the JSON payload, you will receive an error similar to below: ```text {"code":13018,"message":"Error compiling the security policy set: One or more of the specified compiler versions does not exist. Check the compiler versions, then try again."} ``` Create the file `security-policy-bundles.json`: ```json { "bundles": [ { "appProtectWAFVersion": "", "policyName": "Nginxbundletest", "policyUID": "", "attackSignatureVersionDateTime": "latest", "threatCampaignVersionDateTime": "latest" } ] } ``` The *policyUID* value is left blank, as it is generated as part of the creating the bundle. Send a POST request to create the bundle through the API: ```shell curl -X POST https://{{NMS_FQDN}}/api/platform/v1/security/policies/bundles \ -H "Authorization: Bearer " \ -H "Content-Type: application/json" \ -d @security-policy-bundles.json ``` You should receive a response similar to the following: ```json { "items": [ { "compilationStatus": { "message": "", "status": "compiling" }, "content": "", "metadata": { "appProtectWAFVersion": "", "attackSignatureVersionDateTime": "2024.02.21", "created": "2024-06-12T13:28:20.023775785-07:00", "modified": "2024-06-12T13:28:20.023775785-07:00", "policyName": "Nginxbundletest", "policyUID": "6af9f261-658b-4be1-b07a-cebd83e917a1", "threatCampaignVersionDateTime": "2024.02.25", "uid": "cbdf9577-6d81-43d6-8ce1-2e3d4714e8b5" } } ] } ``` You can use the API to list the security bundles, verifying the new addition: ```shell curl --location 'https://127.0.0.1/api/platform/v1/security/policies/bundles' \ -H "Authorization: Bearer " ``` ```json { "items": [ { "compilationStatus": { "message": "", "status": "compiled" }, "content": "", "metadata": { "appProtectWAFVersion": "", "attackSignatureVersionDateTime": "2024.02.21", "created": "2024-06-13T09:09:10.809-07:00", "modified": "2024-06-13T09:09:20-07:00", "policyName": "Nginxbundletest", "policyUID": "ec8681eb-1e25-4b71-93bd-b91f67c5ac99", "threatCampaignVersionDateTime": "2024.02.25", "uid": "de08b324-99d8-4155-b2eb-fe687b21034e" } } ] } ``` **Note:** Take note of the *uid* field: `"uid": "de08b324-99d8-4155-b2eb-fe687b21034e"` It is one of two unique IDs we will use to download the bundle: it will be referenced as *bundle-UID*. --- ## Download the security policy bundle Use a GET request to download the security bundle using the policy and bundle IDs: ```shell curl -X GET "https://{NMS_FQDN}/api/platform/v1/security/policies//bundles/" -H "Authorization: Bearer " | jq -r '.content' | base64 -d > security-policy-bundle.tgz ``` This GET request uses the policy and bundle IDs from the previous examples: ```shell curl -X GET -k 'https://127.0.0.1/api/platform/v1/security/policies/6af9f261-658b-4be1-b07a-cebd83e917a1/bundles/de08b324-99d8-4155-b2eb-fe687b21034e' \ -H "Authorization: Basic " \ | jq -r '.content' | base64 -d > security-policy-bundle.tgz ``` ## Download the security log bundle Use a GET request to download the `secops_dashboard` security log bundle. The security log bundle adjusts the format of the policy events to be compatible with NGINX Instance Manager: ```shell curl -X GET "https://{NMS_FQDN}/api/platform/v1/security/logprofiles/secops_dashboard//bundle" -H "Authorization: Bearer " | jq -r .compiledBundle | base64 -d > secops_dashboard.tgz ``` --- ## Add volumes and volumeMounts To use WAF security bundles, your NGINX Ingress Controller instance must have *volumes* and *volumeMounts*. Precise paths are used to detect when bundles are uploaded to the cluster. Here is an example of what to add: ```yaml volumes: - name: persistentVolumeClaim: claimName: volumeMounts: - name: mountPath: /etc/app_protect/bundles ``` A full example of a deployment file with `volumes` and `volumeMounts` could look like the following: ```yaml apiVersion: apps/v1 kind: Deployment metadata: name: nginx-ingress namespace: nginx-ingress spec: replicas: 1 selector: matchLabels: app: nginx-ingress template: metadata: labels: app: nginx-ingress app.kubernetes.io/name: nginx-ingress #annotations: #prometheus.io/scrape: "true" #prometheus.io/port: "9113" #prometheus.io/scheme: http spec: serviceAccountName: nginx-ingress automountServiceAccountToken: true securityContext: seccompProfile: type: RuntimeDefault volumes: - name: nginx-bundle-mount emptydir: {} containers: - image: imagePullPolicy: IfNotPresent name: nginx-ingress ports: - name: http containerPort: 80 - name: https containerPort: 443 - name: readiness-port containerPort: 8081 - name: prometheus containerPort: 9113 readinessProbe: httpGet: path: /nginx-ready port: readiness-port periodSeconds: 1 resources: requests: cpu: "100m" memory: "128Mi" #limits: # cpu: "1" # memory: "1Gi" securityContext: allowPrivilegeEscalation: false runAsUser: 101 #nginx runAsNonRoot: true capabilities: drop: - ALL add: - NET_BIND_SERVICE volumeMounts: - name: bundle-mount mountPath: /etc/app_protect/bundles env: - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name args: - -nginx-configmaps=$(POD_NAMESPACE)/nginx-config - -report-ingress-status - -external-service=nginx-ingress ``` --- ## Upload the security log bundle Upload the security log bundle binary file to the NGINX Ingress Controller pods. #### Helm ```shell kubectl cp /your/local/path/secops_dashboard.tgz /:etc/app_protect/bundles/secops_dashboard.tgz -c nginx-ingress ``` #### Manifests ```shell kubectl cp /your/local/path/secops_dashboard.tgz /:etc/app_protect/bundles/secops_dashboard.tgz -c nginx-plus-ingress ``` ## Upload the security policy bundle Upload the binary file to the NGINX Ingress Controller pods. #### Helm ```shell kubectl cp /your/local/path/.tgz /:etc/app_protect/bundles.tgz -c nginx-ingress ``` #### Manifests ```shell kubectl cp /your/local/path/.tgz /:etc/app_protect/bundles.tgz -c nginx-plus-ingress ``` --- ## Create WAF policy To process a bundle, you must create a new WAF policy. This policy is added to `/etc/app_protect/bundles`, allowing NGINX Ingress Controller to load it into WAF. The example below shows the required WAF policy, for the *apBundle* field you must use the [security bundle](#download-the-security-policy-bundle) binary file (a tarball). The *apLogBundle* field contains the `secops_dashboard.tgz` [file](#download-the-security-log-bundle). ```yaml apiVersion: k8s.nginx.org/v1 kind: Policy metadata: name: spec: waf: enable: true apBundle: ".tgz" securityLogs: - enable: true apLogBundle: "secops_dashboard.tgz" logDest: "" ``` --- ## Create VirtualServer resource and apply policy Once the WAF policy has been created, link it to your *virtualServer resource*. ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServer metadata: name: webapp spec: host: webapp.example.com policies: - name: upstreams: - name: webapp service: webapp-svc port: 80 routes: - path: / action: pass: webapp ``` Your `VirtualServer` should now apply the generated security policy to your traffic and emit security events to NGINX Instance Manager. -------------------------------------------------------------------------------- title: "ConfigMap resources" url: https://docs.nginx.com/nginx-ingress-controller/configuration/global-configuration/configmap-resource/index.md -------------------------------------------------------------------------------- When using F5 NGINX Ingress Controller, you can customize or fine tune NGINX behavior using ConfigMap resources. Examples include setting the number of worker processes or customizing the access log format. ## Using ConfigMap 1. The [Installation with Manifests](/nic/install/manifests.md) documentation deploy an empty ConfigMap while the default installation manifests specify it in the command-line arguments of the Ingress Controller. However, if you customized the manifests, to use ConfigMap, make sure to specify the ConfigMap resource to use the [command-line arguments](/nic/configuration/global-configuration/command-line-arguments) of NGINX Ingress Controller. 1. Create a ConfigMap file with the name *nginx-config.yaml* and set the values that make sense for your setup: ```yaml kind: ConfigMap apiVersion: v1 metadata: name: nginx-config namespace: nginx-ingress data: proxy-connect-timeout: "10s" proxy-read-timeout: "10s" client-max-body-size: "2m" ``` See the section [Summary of ConfigMap Keys](#configmap-keys) for the explanation of the available ConfigMap keys (such as `proxy-connect-timeout` in this example). 1. Create a new (or update the existing) ConfigMap resource: ```shell kubectl apply -f nginx-config.yaml ``` The NGINX configuration will be updated. --- ## ConfigMap and Ingress annotations ConfigMap applies globally, meaning that it affects every Ingress resource. In contrast, annotations always apply to their Ingress resource. Annotations can override some ConfigMap keys: an example is that the `nginx.org/proxy-connect-timeout` annotations overrides the `proxy-connect-timeout` ConfigMap key. For more information, view the [Advanced configuration with annotations](/nic/configuration/ingress-resources/advanced-configuration-with-annotations) topic. ## ConfigMap and VirtualServer/VirtualServerRoute resources The ConfigMap affects every VirtualServer and VirtualServerRoute resources. However, the fields of those resources allow overriding some ConfigMap keys. For example, the `connect-timeout` field of the `upstream` overrides the `proxy-connect-timeout` ConfigMap key. For more information, view the [VirtualServer and VirtualServerRoute resources](/nic/configuration/virtualserver-and-virtualserverroute-resources) topic. ## ConfigMap keys ### Ingress Controller (Unrelated to NGINX Configuration) |ConfigMap Key | Description | Default | Example | | ---| ---| ---| --- | |*external-status-address* | Sets the address to be reported in the status of Ingress resources. Requires the *-report-status* command-line argument. Overrides the *-external-service* argument. | N/A | [Reporting resource status](/nic/configuration/global-configuration/reporting-resources-status) | ### General customization |ConfigMap Key | Description | Default | Example | | ---| ---| ---| --- | |*proxy-connect-timeout* | Sets the value of the [proxy_connect_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_connect_timeout) and [grpc_connect_timeout](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_connect_timeout) directive. | *60s* | | |*proxy-read-timeout* | Sets the value of the [proxy_read_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_read_timeout) and [grpc_read_timeout](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_read_timeout) directive. | *60s* | | |*proxy-send-timeout* | Sets the value of the [proxy_send_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_send_timeout) and [grpc_send_timeout](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_send_timeout) directive. | *60s* | | |*client-max-body-size* | Sets the value of the [client_max_body_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size) directive. | *1m* | | |*client-body-buffer-size* | Sets the value of the [client_body_buffer_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_buffer_size) directive. | N/A | | |*proxy-buffering* | Enables or disables [buffering of responses](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffering) from the proxied server. | *True* | | |*proxy-buffers* | Sets the value of the [proxy_buffers](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffers) directive. | Depends on the platform. | | |*proxy-buffer-size* | Sets the value of the [proxy_buffer_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffer_size) and [grpc_buffer_size](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_buffer_size) directives. | Depends on the platform. | | |*proxy-busy-buffers-size* | Sets the value of the [proxy_busy_buffers_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_busy_buffers_size) directive. | Depends on the platform. | | |*proxy-max-temp-file-size* | Sets the value of the [proxy_max_temp_file_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_max_temp_file_size) directive. | *1024m* | | |*set-real-ip-from* | Sets the value of the [set_real_ip_from](https://nginx.org/en/docs/http/ngx_http_realip_module.html#set_real_ip_from) directive. | N/A | | |*real-ip-header* | Sets the value of the [real_ip_header](https://nginx.org/en/docs/http/ngx_http_realip_module.html#real_ip_header) directive. | *X-Real-IP* | | |*real-ip-recursive* | Enables or disables the [real_ip_recursive](https://nginx.org/en/docs/http/ngx_http_realip_module.html#real_ip_recursive) directive. | *False* | | |*default-server-return* | Configures the [return](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#return) directive in the default server, which handles a client request if none of the hosts of Ingress or VirtualServer resources match. The default value configures NGINX to return a 404 error page. You can configure a fixed response or a redirect. For example, *default-server-return: 302 https://nginx.org* will redirect a client to *https://nginx.org*. When [-allow-empty-ingress-host](https://docs.nginx.com/nginx-ingress-controller/configuration/global-configuration/command-line-arguments/#cmdoption-allow-empty-ingress-host) is enabled and an empty-host Ingress is active, this directive applies to requests that do not match any Ingress path, unless the Ingress defines a `/` path or `spec.defaultBackend`. | *404* | | |*server-tokens* | Enables or disables the [server_tokens](https://nginx.org/en/docs/http/ngx_http_core_module.html#server_tokens) directive. Additionally, with the NGINX Plus, you can specify a custom string value, including the empty string value, which disables the emission of the “Server” field. | *True* | | |*worker-processes* | Sets the value of the [worker_processes](https://nginx.org/en/docs/ngx_core_module.html#worker_processes) directive. | *auto* | | |*worker-rlimit-nofile* | Sets the value of the [worker_rlimit_nofile](https://nginx.org/en/docs/ngx_core_module.html#worker_rlimit_nofile) directive. | N/A | | |*worker-connections* | Sets the value of the [worker_connections](https://nginx.org/en/docs/ngx_core_module.html#worker_connections) directive. | *1024* | | |*worker-cpu-affinity* | Sets the value of the [worker_cpu_affinity](https://nginx.org/en/docs/ngx_core_module.html#worker_cpu_affinity) directive. | N/A | | |*worker-shutdown-timeout* | Sets the value of the [worker_shutdown_timeout](https://nginx.org/en/docs/ngx_core_module.html#worker_shutdown_timeout) directive. | N/A | | |*server-names-hash-bucket-size* | Sets the value of the [server_names_hash_bucket_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#server_names_hash_bucket_size) directive. | *256* | | |*server-names-hash-max-size* | Sets the value of the [server_names_hash_max_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#server_names_hash_max_size) directive. | *1024* | | |*map-hash-bucket-size* | Sets the value of the [map_hash_bucket_size](http://nginx.org/en/docs/http/ngx_http_map_module.html#map_hash_bucket_size) directive.| *256* | | |*map-hash-max-size* | Sets the value of the [map_hash_max_size](http://nginx.org/en/docs/http/ngx_http_map_module.html#map_hash_max_size) directive. | *2048* | | |*resolver-addresses* | Sets the value of the [resolver](https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver) addresses. Note: If you use a DNS name (for example, *kube-dns.kube-system.svc.cluster.local* ) as a resolver address, NGINX Plus will resolve it using the system resolver during the start and on every configuration reload. If the name cannot be resolved or the DNS server doesn't respond, NGINX Plus will fail to start or reload. To avoid this, we recommend using IP addresses as resolver addresses instead of DNS names. Supported in NGINX Plus only. | N/A | [Support for Type ExternalName Services](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/externalname-services). | |*resolver-ipv6* | Enables IPv6 resolution in the resolver. Supported in NGINX Plus only. | *True* | [Support for Type ExternalName Services](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/externalname-services). | |*resolver-valid* | Sets the time NGINX caches the resolved DNS records. Supported in NGINX Plus only. | TTL value of a DNS record | [Support for Type ExternalName Services](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/externalname-services). | |*resolver-timeout* | Sets the [resolver_timeout](https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver_timeout) for name resolution. Supported in NGINX Plus only. | *30s* | [Support for Type ExternalName Services](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/externalname-services). | |*keepalive-timeout* | Sets the value of the [keepalive_timeout](https://nginx.org/en/docs/http/ngx_http_core_module.html#keepalive_timeout) directive. | *75s* | | |*keepalive-requests* | Sets the value of the [keepalive_requests](https://nginx.org/en/docs/http/ngx_http_core_module.html#keepalive_requests) directive. | *1000* | | |*variables-hash-bucket-size* | Sets the value of the [variables_hash_bucket_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#variables_hash_bucket_size) directive. | *256* | | |*variables-hash-max-size* | Sets the value of the [variables-hash-max-size](https://nginx.org/en/docs/http/ngx_http_core_module.html#variables_hash_max_size) directive. | *1024* | | ### Logging |ConfigMap Key | Description | Default | Example | | ---| ---| ---| --- | |*error-log-level* | Sets the global [error log level](https://nginx.org/en/docs/ngx_core_module.html#error_log) for NGINX. | *notice* | | |*access-log* | Sets the directive [access log](https://nginx.org/en/docs/http/ngx_http_log_module.html#access_log). A syslog destination is the only valid value. The value will be set to its default in-case user tries to configure it with location other than a syslog. | ``/dev/stdout main`` | ``syslog:server=localhost:514`` | |*access-log-off* | Disables the [access log](https://nginx.org/en/docs/http/ngx_http_log_module.html#access_log). | *False* | | |*default-server-access-log-off* | Disables the [access log](https://nginx.org/en/docs/http/ngx_http_log_module.html#access_log) for the default server. If access log is disabled globally (*access-log-off: "True"*), then the default server access log is always disabled. | *False* | | |*log-format* | Sets the custom [log format](https://nginx.org/en/docs/http/ngx_http_log_module.html#log_format) for HTTP and HTTPS traffic. For convenience, it is possible to define the log format across multiple lines (each line separated by *\n*). In that case, the Ingress Controller will replace every *\n* character with a space character. All *'* characters must be escaped. | See the [template file](https://github.com/nginx/kubernetes-ingress/blob/v/internal/configs/version1/nginx.tmpl) for the access log. | [Custom Log Format](https://github.com/nginx/kubernetes-ingress/tree/v/examples/shared-examples/custom-log-format). | |*log-format-escaping* | Sets the characters escaping for the variables of the log format. Supported values: *json* (JSON escaping), *default* (the default escaping) *none* (disables escaping). | *default* | | |*stream-log-format* | Sets the custom [log format](https://nginx.org/en/docs/stream/ngx_stream_log_module.html#log_format) for TCP, UDP, and TLS Passthrough traffic. For convenience, it is possible to define the log format across multiple lines (each line separated by *\n*). In that case, the Ingress Controller will replace every *\n* character with a space character. All *'* characters must be escaped. | See the [template file](https://github.com/nginx/kubernetes-ingress/blob/v/internal/configs/version1/nginx.tmpl). | | |*stream-log-format-escaping* | Sets the characters escaping for the variables of the stream log format. Supported values: *json* (JSON escaping), *default* (the default escaping) *none* (disables escaping). | *default* | | ### Header manipulation **Note:** `disable-forwarded-headers` removes the hardcoded X-Forwarded-* headers directives allowing users to manually manage forwarded headers using add-header. This setting is available only when `--enable-snippets` is enabled, due to the security implications of manually managing forwarded headers. Only enable disable-forwarded-headers if: - You fully trust all upstream proxies in your request path, and - You have validated that your infrastructure sanitizes or overwrites these headers before they reach NGINX (for example, via an external load balancer), or you want to explicitly set these values yourself. |ConfigMap Key | Description | Default | | ---| ---| ---| |*proxy-hide-headers* | Sets the value of one or more [proxy_hide_header](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_hide_header) directives. Example: *"nginx.org/proxy-hide-headers": "header-a,header-b"* | N/A | |*proxy-pass-headers* | Sets the value of one or more [proxy_pass_header](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass_header) directives. Example: *"nginx.org/proxy-pass-headers": "header-a,header-b"* | N/A | |*add-header* | Adds one or more response headers with the [add_header](https://nginx.org/en/docs/http/ngx_http_headers_module.html#add_header) directive in the `http` context. Use the format *Header-Name: value[:always]* and separate entries with commas. Example: `X-Frame-Options: DENY` or `X-Frame-Options: DENY: always`. | N/A | |*add-header-inherit* | Controls how [add_header_inherit](https://nginx.org/en/docs/http/ngx_http_headers_module.html#add_header_inherit) applies inherited response headers. Allowed values are *on*, *off*, and *merge*. | N/A | |*disable-forwarded-headers* | Disables automatic injection of standard `X-Forwarded-*` headers (`X-Forwarded-For`, `X-Forwarded-Host`, `X-Forwarded-Port`, and `X-Forwarded-Proto`), allowing custom header management. Requires `-enable-snippets` CLI flag to be enabled.| *false* | ### Auth and SSL/TLS |ConfigMap Key | Description | Default | | ---| ---| ---| |*redirect-to-https* | Sets a redirect rule based on the value of the *http_x_forwarded_proto* header on the server block to force incoming traffic to be over HTTPS. Useful when terminating SSL in a load balancer in front of the Ingress Controller — see [115](https://github.com/nginx/kubernetes-ingress/issues/115). The redirect code can be configured with the `http-redirect-code` key. | *False* | |*ssl-redirect* | Sets a redirect rule for all incoming HTTP traffic to force incoming traffic over HTTPS when TLS is configured. The redirect code can be configured with the `http-redirect-code` key. | *True* | |*http-redirect-code* | Sets the HTTP redirect code for HTTPS redirects. Supported codes: 301, 302, 307, 308. | *301* | |*hsts* | Enables [HTTP Strict Transport Security (HSTS)](https://www.nginx.com/blog/http-strict-transport-security-hsts-and-nginx/) : the HSTS header is added to the responses from backends. The *preload* directive is included in the header. Not applicable to VS/VSR. | *False* | |*hsts-max-age* | Sets the value of the *max-age* directive of the HSTS header. Not applicable to VS/VSR | *2592000* (1 month) | |*hsts-include-subdomains* | Adds the *includeSubDomains* directive to the HSTS header. Not applicable to VS/VSR | *False* | |*hsts-behind-proxy* | Enables HSTS based on the value of the *http_x_forwarded_proto* request header. Should only be used when TLS termination is configured in a load balancer (proxy) in front of the Ingress Controller. Note: to control redirection from HTTP to HTTPS configure the *nginx.org/redirect-to-https* annotation. Not applicable to VS/VSR. | *False* | |*ssl-protocols* | Sets the value of the [ssl_protocols](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_protocols) directive. | *TLSv1 TLSv1.1 TLSv1.2* | |*ssl-prefer-server-ciphers* | Enables or disables the [ssl_prefer_server_ciphers](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_prefer_server_ciphers) directive. | *False* | |*ssl-ciphers* | Sets the value of the [ssl_ciphers](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_ciphers) directive. | *HIGH:!aNULL:!MD5* | |*ssl-dhparam-file* | Sets the content of the dhparam file. The controller will create the file and set the value of the [ssl_dhparam](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_dhparam) directive with the path of the file. | N/A | ### Listeners |ConfigMap Key | Description | Default | Example | | ---| ---| ---| --- | |*http2* | Enables HTTP/2 in servers with SSL enabled. | *False* | | |*proxy-protocol* | Enables PROXY Protocol for incoming connections. | *False* | [Proxy Protocol](https://github.com/nginx/kubernetes-ingress/tree/v/examples/shared-examples/proxy-protocol). | ### Backend services (Upstreams) |ConfigMap Key | Description | Default | | ---| ---| ---| |*lb-method* | Sets the [load balancing method](/nginx/admin-guide/load-balancer/http-load-balancer.md#choosing-a-load-balancing-method). To use the round-robin method, specify *"round_robin"*. | *"random two least_conn"* | |*max-fails* | Sets the value of the [max_fails](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#max_fails) parameter of the *server* directive. | *1* | |*upstream-zone-size* | Sets the size of the shared memory [zone](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#zone) for upstreams. For NGINX, the special value 0 disables the shared memory zones. For NGINX Plus, shared memory zones are required and cannot be disabled. The special value 0 will be ignored. | *256k* for NGINX, *512k* for NGINX Plus | |*fail-timeout* | Sets the value of the [fail_timeout](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#fail_timeout) parameter of the *server* directive. | *10s* | |*keepalive* | Sets the value of the [keepalive](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive) directive. Note that *proxy_set_header Connection "";* is added to the generated configuration when the value > 0. | *0* | ### Zone Sync Zone Sync enables the [ngx_stream_zone_sync_module](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html) in NGINX Ingress Controller when NGINX Plus is used. Multiple replicas are required to effectively utililise this functionality. More information is available in the [How NGINX Plus Performs Zone Synchronization](/nginx/admin-guide/high-availability/zone_sync_details.md) topic. Zone synchronization with TLS for NGINX Ingress Controller is not yet available with ConfigMap. If you would like to enable Zone Sync with TLS, please remove `zone-sync` from ConfigMap and add Zone Sync parameters via [`stream-snippets`](/nic/configuration/ingress-resources/advanced-configuration-with-snippets.md) similar to [this example](https://github.com/nginx/kubernetes-ingress/blob/v4.0.1/examples/custom-resources/oidc/nginx-config.yaml) and adding the [zone_sync_ssl directive](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_ssl) along with any other TLS parameters to the `stream-snippets`. You will also need to manually add the headless service, such as in [this example](https://github.com/nginx/kubernetes-ingress/blob/v4.0.1/examples/custom-resources/oidc/nginx-ingress-headless.yaml). **Note:** If you previously installed OIDC or used the `zone_sync` directive with `stream-snippets` in [v4.0.1](https://github.com/nginx/kubernetes-ingress/tree/v4.0.1) or earlier, and you plan to enable the `zone-sync` ConfigMap key, the `zone_sync` directive should be removed from `stream-snippets`. If you encounter the error `error [emerg] 13#13: "zone_sync" directive is duplicate in /etc/nginx/nginx.conf:164` it is likely due to `zone_sync` being enabled in both `stream-snippets` and the ConfigMap. Once upgraded, remove the [old headless service](https://github.com/nginx/kubernetes-ingress/blob/v4.0.1/examples/custom-resources/oidc/nginx-ingress-headless.yaml) deployed for OIDC. |ConfigMap Key | Description | Default | | ---| ---| ---| |*zone-sync* | Enables zone synchronization between NGINX Ingress Controller Pods. This autogenerates a [zone_sync_server](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_server) and a headless service using the `ReplicaSet`, `DaemonSet` or `StatefulSet` name. Please note that this headless service will be automatically cleaned up when uninstalling via Helm or by removing the value from the ConfigMap. The headless service will need to be manually removed if the `controller.customConfigMap` value is set via Helm or the deployment is uninstalled via Manifests. Each Ingress Controller manages its own headless service. NGINX Plus Required. | *False* | |*zone-sync-port* | Specifies the optional port on which NGINX Ingress Controller listens for zone sync traffic. NGINX Plus & `zone-sync` Required. | *12345* | |*zone-sync-resolver-addresses* | Configures optional addresses used in the [resolver](https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver) directive for zone-sync. This field takes a comma separated list of addresses. NGINX Plus & `zone-sync` Required | `kube-dns.kube-system.svc.cluster.local` | |*zone-sync-resolver-ipv6* | Configures whether the optional [resolver](https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver) directive for zone-sync will look up IPv6 addresses. NGINX Plus & `zone-sync` Required | `true` | |*zone-sync-resolver-valid* | Configures an [NGINX time](https://nginx.org/en/docs/syntax.html) that the optional [resolver](https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver) directive for zone-sync will override the TTL value of responses from nameservers with. NGINX Plus & `zone-sync` Required | `5s` | ### OIDC (OpenID Connect) Timeouts For more information on timeouts, see [here](https://github.com/nginxinc/nginx-openid-connect?tab=readme-ov-file#configuring-the-key-value-store) | ConfigMap Key | Description | Default | | ------------- | ------------| ------- | | *oidc-pkce-timeout* | Sets the timeout for PKCE (Proof Key for Code Exchange) in OIDC. | `90s` | | *oidc-id-tokens-timeout* | Sets the timeout for ID tokens in OIDC. | `1h` | | *oidc-access-tokens-timeout* | Sets the timeout for access tokens in OIDC. | `1h` | | *oidc-refresh-tokens-timeout* | Sets the timeout for refresh tokens in OIDC. | `8h` | | *oidc-sids-timeout* | Sets the timeout for session IDs in OIDC. | `8h` | ### OIDC (OpenID Connect) ZoneSizes For more information on zonesizes, see [here](https://github.com/nginxinc/nginx-openid-connect?tab=readme-ov-file#configuring-the-key-value-store) | ConfigMap Key | Description | Default | | ------------- | ------------| ------- | | *oidc-pkce-zone-size* | Sets the zonesize for PKCE (Proof Key for Code Exchange) in OIDC. | `128K` | | *oidc-id-tokens-zone-size* | Sets the zonesize for ID tokens in OIDC. | `1M` | | *oidc-access-tokens-zone-size* | Sets the zonesize for access tokens in OIDC. | `1M` | | *oidc-refresh-tokens-zone-size* | Sets the zonesize for refresh tokens in OIDC. | `1M` | | *oidc-sids-zone-size* | Sets the zonesize for session IDs in OIDC. | `1M` | ### Snippets and custom templates |ConfigMap Key | Description | Default | Example | | ---| ---| ---| --- | |*main-snippets* | Sets a custom snippet in main context. | N/A | | |*http-snippets* | Sets a custom snippet in http context. | N/A | | |*location-snippets* | Sets a custom snippet in location context. | N/A | | |*server-snippets* | Sets a custom snippet in server context. | N/A | | |*stream-snippets* | Sets a custom snippet in stream context. | N/A | [Support for TCP/UDP Load Balancing](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/tcp-udp). | |*main-template* | Sets the main NGINX configuration template. | By default the template is read from the file in the container. | [Custom Templates](/nic/configuration/global-configuration/custom-templates.md). | |*ingress-template* | Sets the NGINX configuration template for an Ingress resource. | By default the template is read from the file on the container. | [Custom Templates](/nic/configuration/global-configuration/custom-templates.md). | |*virtualserver-template* | Sets the NGINX configuration template for an VirtualServer resource. | By default the template is read from the file on the container. | [Custom Templates](/nic/configuration/global-configuration/custom-templates.md). | |*transportserver-template* | Sets the NGINX configuration template for a TransportServer resource. | By default the template is read from the file on the container. | [Custom Templates](/nic/configuration/global-configuration/custom-templates.md) | ### Modules |ConfigMap Key | Description | Default | Example | | ---| ---| ---| --- | |*otel-exporter-endpoint* | OTLP/gRPC endpoint that will accept [OpenTelemetry](https://opentelemetry.io) data. Set `otel-trace-in-http` to *"true"* to enable OpenTelemetry at the global level. | N/A | *"https://otel-collector:4317"* | |*otel-exporter-header-name* | The name of a custom HTTP header to add to telemetry export request. `otel-exporter-endpoint` and `otel-exporter-header-value` required. | N/A | *"X-custom-header"* | |*otel-exporter-header-value* | The value of a custom HTTP header to add to telemetry export request. `otel-exporter-endpoint` and `otel-exporter-header-name` required. | N/A | *"custom-value"* | |*otel-service-name* | Sets the `service.name` attribute of the OTel resource. `otel-exporter-endpoint` required. | N/A | *"nginx-ingress-controller:nginx"* | | *otel-trace-in-http* | Enables [OpenTelemetry](https://opentelemetry.io) globally (for all Ingress, VirtualServer and VirtualServerRoute resources). Set this to *"false"* to enable OpenTelemetry for individual routes with snippets. `otel-exporter-endpoint` required. | *"false"* | *"true"* | |*opentracing* | Removed in v5.0.0. Enables [OpenTracing](https://opentracing.io) globally (for all Ingress, VirtualServer and VirtualServerRoute resources). Note: requires the Ingress Controller image with OpenTracing module and a tracer. See the [docs](/nic/logging-and-monitoring/opentracing.md) for more information. | *False* | | |*opentracing-tracer* | Removed in v5.0.0. Sets the path to the vendor tracer binary plugin. | N/A | | |*opentracing-tracer-config* | Removed in v5.0.0. Sets the tracer configuration in JSON format. | N/A | | |*app-protect-compressed-requests-action* | Sets the *app_protect_compressed_requests_action* [global directive](/nginx-app-protect/configuration/#global-directives). | *drop* | | |*app-protect-cookie-seed* | Sets the *app_protect_cookie_seed* [global directive](/nginx-app-protect/configuration/#global-directives). | Random automatically generated string | | |*app-protect-failure-mode-action* | Sets the *app_protect_failure_mode_action* [global directive](/nginx-app-protect/configuration/#global-directives). | *pass* | | |*app-protect-cpu-thresholds* | Sets the *app_protect_cpu_thresholds* [global directive](/nginx-app-protect/configuration/#global-directives). | *high=100 low=100* | | |*app-protect-physical-memory-util-thresholds* | Sets the *app_protect_physical_memory_util_thresholds* [global directive](/nginx-app-protect/configuration/#global-directives). | *high=100 low=100* | | |`app-protect-reconnect-period-seconds` | Sets the `app_protect_reconnect_period_seconds` [global directive](/nginx-app-protect/configuration/#global-directives). | `5` | | |*app-protect-dos-log-format* | Sets the custom [log format](https://nginx.org/en/docs/http/ngx_http_log_module.html#log_format) for Dos Access log traffic. For convenience, it is possible to define the log format across multiple lines (each line separated by *\n*). In that case, the Ingress Controller will replace every *\n* character with a space character. All *'* characters must be escaped. | `, vs_name_al=$app_protect_dos_vs_name, ip=$remote_addr, tls_fp=$app_protect_dos_tls_fp, outcome=$app_protect_dos_outcome, reason=$app_protect_dos_outcome_reason, policy_name=$app_protect_dos_policy_name, dos_version=$app_protect_dos_version, ip_tls=$remote_addr:$app_protect_dos_tls_fp,` | | |*app-protect-dos-log-format-escaping* | Sets the characters escaping for the variables of the stream log format. Supported values: *json* (JSON escaping), *default* (the default escaping) *none* (disables escaping). | *default* | | |*app-protect-dos-arb-fqdn* | Sets the *app-protect-dos-arb-fqdn* [directive](/nginx-app-protect-dos/directives-and-policy/learn-about-directives-and-policy/#arbitrator-fqdn-directive-app_protect_dos_arb_fqdn). | *svc-appprotect-dos-arb* | | -------------------------------------------------------------------------------- title: "ConfigMap resources" url: https://docs.nginx.com/nginx-ingress-controller/lts/configuration/global-configuration/configmap-resource/index.md -------------------------------------------------------------------------------- When using F5 NGINX Ingress Controller LTS, you can customize or fine tune NGINX behavior using ConfigMap resources. Examples include setting the number of worker processes or customizing the access log format. ## Using ConfigMap 1. The [Installation with Manifests](/nic/lts/install/manifests.md) documentation deploy an empty ConfigMap while the default installation manifests specify it in the command-line arguments of the Ingress Controller. However, if you customized the manifests, to use ConfigMap, make sure to specify the ConfigMap resource to use the [command-line arguments](/nic/lts/configuration/global-configuration/command-line-arguments) of NGINX Ingress Controller LTS. 1. Create a ConfigMap file with the name *nginx-config.yaml* and set the values that make sense for your setup: ```yaml kind: ConfigMap apiVersion: v1 metadata: name: nginx-config namespace: nginx-ingress data: proxy-connect-timeout: "10s" proxy-read-timeout: "10s" client-max-body-size: "2m" ``` See the section [Summary of ConfigMap Keys](#configmap-keys) for the explanation of the available ConfigMap keys (such as `proxy-connect-timeout` in this example). 1. Create a new (or update the existing) ConfigMap resource: ```shell kubectl apply -f nginx-config.yaml ``` The NGINX configuration will be updated. --- ## ConfigMap and Ingress annotations ConfigMap applies globally, meaning that it affects every Ingress resource. In contrast, annotations always apply to their Ingress resource. Annotations can override some ConfigMap keys: an example is that the `nginx.org/proxy-connect-timeout` annotations overrides the `proxy-connect-timeout` ConfigMap key. For more information, view the [Advanced configuration with annotations](/nic/lts/configuration/ingress-resources/advanced-configuration-with-annotations) topic. ## ConfigMap and VirtualServer/VirtualServerRoute resources The ConfigMap affects every VirtualServer and VirtualServerRoute resources. However, the fields of those resources allow overriding some ConfigMap keys. For example, the `connect-timeout` field of the `upstream` overrides the `proxy-connect-timeout` ConfigMap key. For more information, view the [VirtualServer and VirtualServerRoute resources](/nic/lts/configuration/virtualserver-and-virtualserverroute-resources) topic. ## ConfigMap keys ### Ingress Controller (Unrelated to NGINX Configuration) |ConfigMap Key | Description | Default | Example | | ---| ---| ---| --- | |*external-status-address* | Sets the address to be reported in the status of Ingress resources. Requires the *-report-status* command-line argument. Overrides the *-external-service* argument. | N/A | [Reporting resource status](/nic/lts/configuration/global-configuration/reporting-resources-status) | ### General customization |ConfigMap Key | Description | Default | Example | | ---| ---| ---| --- | |*proxy-connect-timeout* | Sets the value of the [proxy_connect_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_connect_timeout) and [grpc_connect_timeout](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_connect_timeout) directive. | *60s* | | |*proxy-read-timeout* | Sets the value of the [proxy_read_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_read_timeout) and [grpc_read_timeout](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_read_timeout) directive. | *60s* | | |*proxy-send-timeout* | Sets the value of the [proxy_send_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_send_timeout) and [grpc_send_timeout](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_send_timeout) directive. | *60s* | | |*client-max-body-size* | Sets the value of the [client_max_body_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size) directive. | *1m* | | |*client-body-buffer-size* | Sets the value of the [client_body_buffer_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_buffer_size) directive. | N/A | | |*proxy-buffering* | Enables or disables [buffering of responses](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffering) from the proxied server. | *True* | | |*proxy-buffers* | Sets the value of the [proxy_buffers](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffers) directive. | Depends on the platform. | | |*proxy-buffer-size* | Sets the value of the [proxy_buffer_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffer_size) and [grpc_buffer_size](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_buffer_size) directives. | Depends on the platform. | | |*proxy-busy-buffers-size* | Sets the value of the [proxy_busy_buffers_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_busy_buffers_size) directive. | Depends on the platform. | | |*proxy-max-temp-file-size* | Sets the value of the [proxy_max_temp_file_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_max_temp_file_size) directive. | *1024m* | | |*set-real-ip-from* | Sets the value of the [set_real_ip_from](https://nginx.org/en/docs/http/ngx_http_realip_module.html#set_real_ip_from) directive. | N/A | | |*real-ip-header* | Sets the value of the [real_ip_header](https://nginx.org/en/docs/http/ngx_http_realip_module.html#real_ip_header) directive. | *X-Real-IP* | | |*real-ip-recursive* | Enables or disables the [real_ip_recursive](https://nginx.org/en/docs/http/ngx_http_realip_module.html#real_ip_recursive) directive. | *False* | | |*default-server-return* | Configures the [return](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#return) directive in the default server, which handles a client request if none of the hosts of Ingress or VirtualServer resources match. The default value configures NGINX to return a 404 error page. You can configure a fixed response or a redirect. For example, *default-server-return: 302 https://nginx.org* will redirect a client to *https://nginx.org*. | *404* | | |*server-tokens* | Enables or disables the [server_tokens](https://nginx.org/en/docs/http/ngx_http_core_module.html#server_tokens) directive. You can specify a custom string value, including the empty string value, which disables the emission of the “Server” field. | *True* | | |*worker-processes* | Sets the value of the [worker_processes](https://nginx.org/en/docs/ngx_core_module.html#worker_processes) directive. | *auto* | | |*worker-rlimit-nofile* | Sets the value of the [worker_rlimit_nofile](https://nginx.org/en/docs/ngx_core_module.html#worker_rlimit_nofile) directive. | N/A | | |*worker-connections* | Sets the value of the [worker_connections](https://nginx.org/en/docs/ngx_core_module.html#worker_connections) directive. | *1024* | | |*worker-cpu-affinity* | Sets the value of the [worker_cpu_affinity](https://nginx.org/en/docs/ngx_core_module.html#worker_cpu_affinity) directive. | N/A | | |*worker-shutdown-timeout* | Sets the value of the [worker_shutdown_timeout](https://nginx.org/en/docs/ngx_core_module.html#worker_shutdown_timeout) directive. | N/A | | |*server-names-hash-bucket-size* | Sets the value of the [server_names_hash_bucket_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#server_names_hash_bucket_size) directive. | *256* | | |*server-names-hash-max-size* | Sets the value of the [server_names_hash_max_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#server_names_hash_max_size) directive. | *1024* | | |*map-hash-bucket-size* | Sets the value of the [map_hash_bucket_size](http://nginx.org/en/docs/http/ngx_http_map_module.html#map_hash_bucket_size) directive.| *256* | | |*map-hash-max-size* | Sets the value of the [map_hash_max_size](http://nginx.org/en/docs/http/ngx_http_map_module.html#map_hash_max_size) directive. | *2048* | | |*resolver-addresses* | Sets the value of the [resolver](https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver) addresses. Note: If you use a DNS name (for example, *kube-dns.kube-system.svc.cluster.local* ) as a resolver address, NGINX Plus will resolve it using the system resolver during the start and on every configuration reload. If the name cannot be resolved or the DNS server doesn't respond, NGINX Plus will fail to start or reload. To avoid this, we recommend using IP addresses as resolver addresses instead of DNS names. | N/A | [Support for Type ExternalName Services](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/externalname-services). | |*resolver-ipv6* | Enables IPv6 resolution in the resolver. | *True* | [Support for Type ExternalName Services](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/externalname-services). | |*resolver-valid* | Sets the time NGINX caches the resolved DNS records. | TTL value of a DNS record | [Support for Type ExternalName Services](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/externalname-services). | |*resolver-timeout* | Sets the [resolver_timeout](https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver_timeout) for name resolution. | *30s* | [Support for Type ExternalName Services](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/externalname-services). | |*keepalive-timeout* | Sets the value of the [keepalive_timeout](https://nginx.org/en/docs/http/ngx_http_core_module.html#keepalive_timeout) directive. | *75s* | | |*keepalive-requests* | Sets the value of the [keepalive_requests](https://nginx.org/en/docs/http/ngx_http_core_module.html#keepalive_requests) directive. | *1000* | | |*variables-hash-bucket-size* | Sets the value of the [variables_hash_bucket_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#variables_hash_bucket_size) directive. | *256* | | |*variables-hash-max-size* | Sets the value of the [variables-hash-max-size](https://nginx.org/en/docs/http/ngx_http_core_module.html#variables_hash_max_size) directive. | *1024* | | ### Logging |ConfigMap Key | Description | Default | Example | | ---| ---| ---| --- | |*error-log-level* | Sets the global [error log level](https://nginx.org/en/docs/ngx_core_module.html#error_log) for NGINX. | *notice* | | |*access-log* | Sets the directive [access log](https://nginx.org/en/docs/http/ngx_http_log_module.html#access_log). A syslog destination is the only valid value. The value will be set to its default in-case user tries to configure it with location other than a syslog. | ``/dev/stdout main`` | ``syslog:server=localhost:514`` | |*access-log-off* | Disables the [access log](https://nginx.org/en/docs/http/ngx_http_log_module.html#access_log). | *False* | | |*default-server-access-log-off* | Disables the [access log](https://nginx.org/en/docs/http/ngx_http_log_module.html#access_log) for the default server. If access log is disabled globally (*access-log-off: "True"*), then the default server access log is always disabled. | *False* | | |*log-format* | Sets the custom [log format](https://nginx.org/en/docs/http/ngx_http_log_module.html#log_format) for HTTP and HTTPS traffic. For convenience, it is possible to define the log format across multiple lines (each line separated by *\n*). In that case, the Ingress Controller will replace every *\n* character with a space character. All *'* characters must be escaped. | See the [template file](https://github.com/nginx/kubernetes-ingress/blob/v/internal/configs/version1/nginx.tmpl) for the access log. | [Custom Log Format](https://github.com/nginx/kubernetes-ingress/tree/v/examples/shared-examples/custom-log-format). | |*log-format-escaping* | Sets the characters escaping for the variables of the log format. Supported values: *json* (JSON escaping), *default* (the default escaping) *none* (disables escaping). | *default* | | |*stream-log-format* | Sets the custom [log format](https://nginx.org/en/docs/stream/ngx_stream_log_module.html#log_format) for TCP, UDP, and TLS Passthrough traffic. For convenience, it is possible to define the log format across multiple lines (each line separated by *\n*). In that case, the Ingress Controller will replace every *\n* character with a space character. All *'* characters must be escaped. | See the [template file](https://github.com/nginx/kubernetes-ingress/blob/v/internal/configs/version1/nginx.tmpl). | | |*stream-log-format-escaping* | Sets the characters escaping for the variables of the stream log format. Supported values: *json* (JSON escaping), *default* (the default escaping) *none* (disables escaping). | *default* | | ### Request URI/Header manipulation |ConfigMap Key | Description | Default | | ---| ---| ---| |*proxy-hide-headers* | Sets the value of one or more [proxy_hide_header](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_hide_header) directives. Example: *"nginx.org/proxy-hide-headers": "header-a,header-b"* | N/A | |*proxy-pass-headers* | Sets the value of one or more [proxy_pass_header](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass_header) directives. Example: *"nginx.org/proxy-pass-headers": "header-a,header-b"* | N/A | ### Auth and SSL/TLS |ConfigMap Key | Description | Default | | ---| ---| ---| |*redirect-to-https* | Sets a redirect rule based on the value of the *http_x_forwarded_proto* header on the server block to force incoming traffic to be over HTTPS. Useful when terminating SSL in a load balancer in front of the Ingress Controller — see [115](https://github.com/nginx/kubernetes-ingress/issues/115). The redirect code can be configured with the `http-redirect-code` key. | *False* | |*ssl-redirect* | Sets a redirect rule for all incoming HTTP traffic to force incoming traffic over HTTPS when TLS is configured. The redirect code can be configured with the `http-redirect-code` key. | *True* | |*http-redirect-code* | Sets the HTTP redirect code for HTTPS redirects. Supported codes: 301, 302, 307, 308. | *301* | |*hsts* | Enables [HTTP Strict Transport Security (HSTS)](https://www.nginx.com/blog/http-strict-transport-security-hsts-and-nginx/) : the HSTS header is added to the responses from backends. The *preload* directive is included in the header. | *False* | |*hsts-max-age* | Sets the value of the *max-age* directive of the HSTS header. | *2592000* (1 month) | |*hsts-include-subdomains* | Adds the *includeSubDomains* directive to the HSTS header. | *False* | |*hsts-behind-proxy* | Enables HSTS based on the value of the *http_x_forwarded_proto* request header. Should only be used when TLS termination is configured in a load balancer (proxy) in front of the Ingress Controller. Note: to control redirection from HTTP to HTTPS configure the *nginx.org/redirect-to-https* annotation. | *False* | |*ssl-protocols* | Sets the value of the [ssl_protocols](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_protocols) directive. | *TLSv1 TLSv1.1 TLSv1.2* | |*ssl-prefer-server-ciphers* | Enables or disables the [ssl_prefer_server_ciphers](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_prefer_server_ciphers) directive. | *False* | |*ssl-ciphers* | Sets the value of the [ssl_ciphers](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_ciphers) directive. | *HIGH:!aNULL:!MD5* | |*ssl-dhparam-file* | Sets the content of the dhparam file. The controller will create the file and set the value of the [ssl_dhparam](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_dhparam) directive with the path of the file. | N/A | ### Listeners |ConfigMap Key | Description | Default | Example | | ---| ---| ---| --- | |*http2* | Enables HTTP/2 in servers with SSL enabled. | *False* | | |*proxy-protocol* | Enables PROXY Protocol for incoming connections. | *False* | [Proxy Protocol](https://github.com/nginx/kubernetes-ingress/tree/v/examples/shared-examples/proxy-protocol). | ### Backend services (Upstreams) |ConfigMap Key | Description | Default | | ---| ---| ---| |*lb-method* | Sets the [load balancing method](/nginx/admin-guide/load-balancer/http-load-balancer.md#choosing-a-load-balancing-method). To use the round-robin method, specify *"round_robin"*. | *"random two least_conn"* | |*max-fails* | Sets the value of the [max_fails](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#max_fails) parameter of the *server* directive. | *1* | |*upstream-zone-size* | Sets the size of the shared memory [zone](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#zone) for upstreams. Shared memory zones are required and cannot be disabled. The special value 0 will be ignored. | *512k* for NGINX Plus | |*fail-timeout* | Sets the value of the [fail_timeout](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#fail_timeout) parameter of the *server* directive. | *10s* | |*keepalive* | Sets the value of the [keepalive](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive) directive. Note that *proxy_set_header Connection "";* is added to the generated configuration when the value > 0. | *0* | ### Zone Sync Zone Sync enables the [ngx_stream_zone_sync_module](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html) in NGINX Ingress Controller LTS. Multiple replicas are required to effectively utililise this functionality. More information is available in the [How NGINX Plus Performs Zone Synchronization](/nginx/admin-guide/high-availability/zone_sync_details.md) topic. Zone synchronization with TLS for NGINX Ingress Controller LTS is not yet available with ConfigMap. If you would like to enable Zone Sync with TLS, please remove `zone-sync` from ConfigMap and add Zone Sync parameters via [`stream-snippets`](/nic/lts/configuration/ingress-resources/advanced-configuration-with-snippets.md) similar to [this example](https://github.com/nginx/kubernetes-ingress/blob/v4.0.1/examples/custom-resources/oidc/nginx-config.yaml) and adding the [zone_sync_ssl directive](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_ssl) along with any other TLS parameters to the `stream-snippets`. You will also need to manually add the headless service, such as in [this example](https://github.com/nginx/kubernetes-ingress/blob/v4.0.1/examples/custom-resources/oidc/nginx-ingress-headless.yaml). **caution:** If you previously installed OIDC or used the `zone_sync` directive with `stream-snippets` in [v4.0.1](https://github.com/nginx/kubernetes-ingress/tree/v4.0.1) or earlier, and you plan to enable the `zone-sync` ConfigMap key, the `zone_sync` directive should be removed from `stream-snippets`. If you encounter the error `error [emerg] 13#13: "zone_sync" directive is duplicate in /etc/nginx/nginx.conf:164` it is likely due to `zone_sync` being enabled in both `stream-snippets` and the ConfigMap. Once upgraded, remove the [old headless service](https://github.com/nginx/kubernetes-ingress/blob/v4.0.1/examples/custom-resources/oidc/nginx-ingress-headless.yaml) deployed for OIDC. |ConfigMap Key | Description | Default | | ---| ---| ---| |*zone-sync* | Enables zone synchronization between NGINX Ingress Controller LTS Pods. This autogenerates a [zone_sync_server](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_server) and a headless service using the `ReplicaSet`, `DaemonSet` or `StatefulSet` name. Please note that this headless service will be automatically cleaned up when uninstalling via Helm or by removing the value from the ConfigMap. The headless service will need to be manually removed if the `controller.customConfigMap` value is set via Helm or the deployment is uninstalled via Manifests. Each Ingress Controller manages its own headless service. | *False* | |*zone-sync-port* | Specifies the optional port on which NGINX Ingress Controller LTS listens for zone sync traffic. `zone-sync` Required. | *12345* | |*zone-sync-resolver-addresses* | Configures optional addresses used in the [resolver](https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver) directive for zone-sync. This field takes a comma separated list of addresses. `zone-sync` Required | `kube-dns.kube-system.svc.cluster.local` | |*zone-sync-resolver-ipv6* | Configures whether the optional [resolver](https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver) directive for zone-sync will look up IPv6 addresses. `zone-sync` Required | `true` | |*zone-sync-resolver-valid* | Configures an [NGINX time](https://nginx.org/en/docs/syntax.html) that the optional [resolver](https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver) directive for zone-sync will override the TTL value of responses from nameservers with. `zone-sync` Required | `5s` | ### OIDC (OpenID Connect) Timeouts For more information on timeouts, see [here](https://github.com/nginxinc/nginx-openid-connect?tab=readme-ov-file#configuring-the-key-value-store) | ConfigMap Key | Description | Default | | ------------- | ------------| ------- | | *oidc-pkce-timeout* | Sets the timeout for PKCE (Proof Key for Code Exchange) in OIDC. | `90s` | | *oidc-id-tokens-timeout* | Sets the timeout for ID tokens in OIDC. | `1h` | | *oidc-access-tokens-timeout* | Sets the timeout for access tokens in OIDC. | `1h` | | *oidc-refresh-tokens-timeout* | Sets the timeout for refresh tokens in OIDC. | `24h` | | *oidc-sids-timeout* | Sets the timeout for session IDs in OIDC. | `24h` | ### Snippets and custom templates |ConfigMap Key | Description | Default | Example | | ---| ---| ---| --- | |*main-snippets* | Sets a custom snippet in main context. | N/A | | |*http-snippets* | Sets a custom snippet in http context. | N/A | | |*location-snippets* | Sets a custom snippet in location context. | N/A | | |*server-snippets* | Sets a custom snippet in server context. | N/A | | |*stream-snippets* | Sets a custom snippet in stream context. | N/A | [Support for TCP/UDP Load Balancing](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/tcp-udp). | |*main-template* | Sets the main NGINX configuration template. | By default the template is read from the file in the container. | [Custom Templates](/nic/lts/configuration/global-configuration/custom-templates.md). | |*ingress-template* | Sets the NGINX configuration template for an Ingress resource. | By default the template is read from the file on the container. | [Custom Templates](/nic/lts/configuration/global-configuration/custom-templates.md). | |*virtualserver-template* | Sets the NGINX configuration template for an VirtualServer resource. | By default the template is read from the file on the container. | [Custom Templates](/nic/lts/configuration/global-configuration/custom-templates.md). | |*transportserver-template* | Sets the NGINX configuration template for a TransportServer resource. | By default the template is read from the file on the container. | [Custom Templates](/nic/lts/configuration/global-configuration/custom-templates.md) | ### Modules |ConfigMap Key | Description | Default | Example | | ---| ---| ---| --- | |*otel-exporter-endpoint* | OTLP/gRPC endpoint that will accept [OpenTelemetry](https://opentelemetry.io) data. Set `otel-trace-in-http` to *"true"* to enable OpenTelemetry at the global level. | N/A | *"https://otel-collector:4317"* | |*otel-exporter-header-name* | The name of a custom HTTP header to add to telemetry export request. `otel-exporter-endpoint` and `otel-exporter-header-value` required. | N/A | *"X-custom-header"* | |*otel-exporter-header-value* | The value of a custom HTTP header to add to telemetry export request. `otel-exporter-endpoint` and `otel-exporter-header-name` required. | N/A | *"custom-value"* | |*otel-service-name* | Sets the `service.name` attribute of the OTel resource. `otel-exporter-endpoint` required. | N/A | *"nginx-ingress-controller:nginx"* | |*otel-trace-in-http* | Enables [OpenTelemetry](https://opentelemetry.io) globally (for all Ingress, VirtualServer and VirtualServerRoute resources). Set this to *"false"* to enable OpenTelemetry for individual routes with snippets. `otel-exporter-endpoint` required. | *"false"* | *"true"* | -------------------------------------------------------------------------------- title: "Create a license Secret" url: https://docs.nginx.com/nginx-ingress-controller/install/license-secret/index.md -------------------------------------------------------------------------------- This document explains how to create and use a license secret for F5 NGINX Ingress Controller. # Overview NGINX Plus Ingress Controller requires a valid JSON Web Token (JWT) to download the container image from the F5 registry. From version 4.0.0, this JWT token is also required to run NGINX Plus. This requirement is part of F5’s broader licensing program and aligns with industry best practices. The JWT will streamline subscription renewals and usage reporting, helping you manage your NGINX Plus subscription more efficiently. The [telemetry](#telemetry) data we collect helps us improve our products and services to better meet your needs. The JWT is required for validating your subscription and reporting telemetry data. For environments connected to the internet, telemetry is automatically sent to F5’s licensing endpoint. In offline environments, telemetry is routed through [NGINX Instance Manager](/nim/). By default usage is reported every hour and also whenever NGINX is reloaded. **Note:** Read the [subscription licenses topic](/solutions/about-subscription-licenses.md#for-internet-connected-environments) for a list of IPs associated with F5's licensing endpoint (`product.connect.nginx.com`). ## Set up your NGINX Plus license ### Download the JWT 1. Log in to [MyF5](https://my.f5.com/manage/s/). 2. Go to **My Products & Plans > Subscriptions** to see your active subscriptions. 3. Find your NGINX products or services subscription, and select the **Subscription ID** for details. 4. Download the **JSON Web Token (JWT)** from the subscription page. **Note:** The Connectivity Stack for Kubernetes JWT does not work with NGINX Plus reporting. A regular NGINX Plus instance JWT must be used. ### Create the Secret The JWT needs to be configured before deploying NGINX Ingress Controller. It must be stored in a Kubernetes Secret of type `nginx.com/license` in the same namespace as your NGINX Ingress Controller pod(s). Create the Secret with the following command: ```shell kubectl create secret generic license-token --from-file=license.jwt= --type=nginx.com/license -n ``` Once created, you can download the `.jwt` file. **Note:** For security, follow these practices with JSON Web Tokens (JWTs), passwords, and shell history: 1. **JWTs:** JWTs are sensitive information. Store them securely. Delete them after use to prevent unauthorized access. 1. **Shell history:** Commands that include JWTs or passwords are recorded in the history of your shell, in plain text. Clear your shell history after running such commands. For example, if you use bash, you can delete commands in your `~/.bash_history` file. Alternatively, you can run the `history -c` command to erase your shell history. Follow these practices to help ensure the security of your system and data. ### Update the Secret If you've already deployed NGINX Ingress Controller and need to rotate or renew the JWT (for example, when the existing token is about to expire or has been replaced), update the existing Secret in place. First, take your new JWT license token, and save it to your existing `license.jwt` file. Next, use the following command to generate the updated Secret manifest and apply it: ```shell kubectl create secret generic license-token \ --save-config \ --dry-run=client \ --from-file=license.jwt= \ --type=nginx.com/license \ -o yaml | \ kubectl apply -f - ``` Notes: - Replace `license.jwt` on the `--from-file` flag with the path to your renewed JWT file if it's not in the current directory. - If your Secret resides in a specific namespace, include `-n ` on the `kubectl create secret` command so the generated YAML contains the correct namespace. - Ensure the Secret name (`license-token` by default) matches the name referenced by your Helm values or Management ConfigMap. - After the Secret is updated, the mounted Secret volume in the Pod is refreshed automatically by Kubernetes. NGINX Plus applies the updated license automatically. If you do not see the update take effect after a short period, restart the Ingress Controller Pod(s) to force a re-read of the Secret. ### Add the license Secret to your deployment If using a name other than the default `license-token`, provide the name of this Secret when installing NGINX Ingress Controller: #### Helm Specify the Secret name using the `controller.mgmt.licenseTokenSecretName` Helm value. For detailed guidance on creating the Management block with Helm, refer to the [Helm installation topics](/nic/install/helm/). #### Manifests Specify the Secret name in the `license-token-secret-name` Management ConfigMap key. For detailed guidance on creating the Management ConfigMap, refer to the [Management ConfigMap Resource Documentation](/nic/configuration/global-configuration/mgmt-configmap-resource/). If you are reporting to the default licensing endpoint, then you can now proceed with [installing NGINX Ingress Controller](/nic/install/). Otherwise, follow the steps below to configure reporting to NGINX Instance Manager ### Create report for NGINX Instance Manager {#nim} If you are deploying NGINX Ingress Controller in an "air-gapped" environment you will need to report to [NGINX Instance Manager](/nim/) instead of the default licensing endpoint. First, you must specify the endpoint of your NGINX Instance Manager. #### Helm Specify the endpoint using the `controller.mgmt.usageReport.endpoint` helm value. #### Manifests Specify the endpoint in the `usage-report-endpoint` Management ConfigMap key. #### Configure SSL certificates and SSL trusted certificates {#nim-cert} To configure SSL certificates or SSL trusted certificates, extra steps are necessary. To use Client Auth with NGINX Instance Manager, first create a Secret of type `kubernetes.io/tls` in the same namespace as the NGINX Ingress Controller pods. ```shell kubectl create secret tls ssl-certificate --cert= --key= -n ``` To provide a SSL trusted certificate, and an optional Certificate Revocation List, create a Secret of type `nginx.org/ca` in the Namespace that the NIC Pod(s) are in. ```shell kubectl create secret generic ssl-trusted-certificate \ --from-file=ca.crt= \ --from-file=ca.crl= \ # optional --type=nginx.org/ca ``` Providing an optional CRL (certificate revocation list) will configure the [`ssl_crl`](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_crl) directive. #### Helm Specify the SSL certificate Secret name using the `controller.mgmt.sslCertificateSecretName` Helm value. Specify the SSL trusted certificate Secret name using the `controller.mgmt.sslTrustedCertificateSecretName` Helm value. #### Manifests Specify the SSL certificate Secret name in the `ssl-certificate-secret-name` management ConfigMap key. Specify the SSL trusted certificate Secret name in the `ssl-trusted-certificate-secret-name` management ConfigMap key. Once these Secrets are created and configured, you can now [install NGINX Ingress Controller ](/nic/install/). ## What’s reported and how it’s protected {#telemetry} NGINX Plus reports the following data every hour by default: - **NGINX version and status**: The version of NGINX Plus running on the instance. - **Instance UUID**: A unique identifier for each NGINX Plus instance. - **Traffic data**: - **Bytes received from and sent to clients**: HTTP and stream traffic volume between clients and NGINX Plus. - **Bytes received from and sent to upstreams**: HTTP and stream traffic volume between NGINX Plus and upstream servers. - **Client connections**: The number of accepted client connections (HTTP and stream traffic). - **Requests handled**: The total number of HTTP requests processed. - **NGINX uptime**: The number of reloads and worker connections during uptime. - **Usage report timestamps**: Start and end times for each usage report. - **Kubernetes node details**: Information about Kubernetes nodes. ### Security and privacy of reported data All communication between your NGINX Plus instances, NGINX Instance Manager, and F5’s licensing endpoint (`product.connect.nginx.com`) is protected using **SSL/TLS** encryption. Only **operational metrics** are reported — no **personally identifiable information (PII)** or **sensitive customer data** is transmitted. -------------------------------------------------------------------------------- title: "Create a license Secret" url: https://docs.nginx.com/nginx-ingress-controller/lts/install/license-secret/index.md -------------------------------------------------------------------------------- This document explains how to create and use a license secret for F5 NGINX Ingress Controller LTS. # Overview NGINX Plus Ingress Controller requires a valid JSON Web Token (JWT) to download the container image from the F5 registry. This JWT token is also required to run NGINX Plus. This requirement is part of F5’s broader licensing program and aligns with industry best practices. The JWT will streamline subscription renewals and usage reporting, helping you manage your NGINX Plus subscription more efficiently. The [telemetry](#telemetry) data we collect helps us improve our products and services to better meet your needs. The JWT is required for validating your subscription and reporting telemetry data. For environments connected to the internet, telemetry is automatically sent to F5’s licensing endpoint. In offline environments, telemetry is routed through [NGINX Instance Manager](/nim/). By default usage is reported every hour and also whenever NGINX is reloaded. **note:** Read the [subscription licenses topic](/solutions/about-subscription-licenses.md#for-internet-connected-environments) for a list of IPs associated with F5's licensing endpoint (`product.connect.nginx.com`). ## Set up your NGINX Plus license ### Download the JWT 1. Log in to [MyF5](https://my.f5.com/manage/s/). 2. Go to **My Products & Plans > Subscriptions** to see your active subscriptions. 3. Find your NGINX products or services subscription, and select the **Subscription ID** for details. 4. Download the **JSON Web Token (JWT)** from the subscription page. **Note:** The Connectivity Stack for Kubernetes JWT does not work with NGINX Plus reporting. A regular NGINX Plus instance JWT must be used. ### Create the Secret The JWT needs to be configured before deploying NGINX Ingress Controller LTS. It must be stored in a Kubernetes Secret of type `nginx.com/license` in the same namespace as your NGINX Ingress Controller LTS pod(s). Create the Secret with the following command: ```shell kubectl create secret generic license-token --from-file=license.jwt= --type=nginx.com/license -n ``` Once created, you can download the `.jwt` file. **Note:** For security, follow these practices with JSON Web Tokens (JWTs), passwords, and shell history: 1. **JWTs:** JWTs are sensitive information. Store them securely. Delete them after use to prevent unauthorized access. 1. **Shell history:** Commands that include JWTs or passwords are recorded in the history of your shell, in plain text. Clear your shell history after running such commands. For example, if you use bash, you can delete commands in your `~/.bash_history` file. Alternatively, you can run the `history -c` command to erase your shell history. Follow these practices to help ensure the security of your system and data. ### Update the Secret If you've already deployed NGINX Ingress Controller LTS and need to rotate or renew the JWT (for example, when the existing token is about to expire or has been replaced), update the existing Secret in place. First, take your new JWT license token, and save it to your existing `license.jwt` file. Next, use the following command to generate the updated Secret manifest and apply it: ```shell kubectl create secret generic license-token \ --save-config \ --dry-run=client \ --from-file=license.jwt= \ --type=nginx.com/license \ -o yaml | \ kubectl apply -f - ``` Notes: - Replace `license.jwt` on the `--from-file` flag with the path to your renewed JWT file if it's not in the current directory. - If your Secret resides in a specific namespace, include `-n ` on the `kubectl create secret` command so the generated YAML contains the correct namespace. - Ensure the Secret name (`license-token` by default) matches the name referenced by your Helm values or Management ConfigMap. - After the Secret is updated, the mounted Secret volume in the Pod is refreshed automatically by Kubernetes. NGINX Plus applies the updated license automatically. If you do not see the update take effect after a short period, restart the Ingress Controller Pod(s) to force a re-read of the Secret. ### Add the license Secret to your deployment If using a name other than the default `license-token`, provide the name of this Secret when installing NGINX Ingress Controller LTS: #### Helm Specify the Secret name using the `controller.mgmt.licenseTokenSecretName` Helm value. For detailed guidance on creating the Management block with Helm, refer to the [Helm installation topics](/nic/lts/install/helm/). #### Manifests Specify the Secret name in the `license-token-secret-name` Management ConfigMap key. For detailed guidance on creating the Management ConfigMap, refer to the [Management ConfigMap Resource Documentation](/nic/lts/configuration/global-configuration/mgmt-configmap-resource/). If you are reporting to the default licensing endpoint, then you can now proceed with [installing NGINX Ingress Controller LTS](/nic/lts/install/). Otherwise, follow the steps below to configure reporting to NGINX Instance Manager ### Create report for NGINX Instance Manager {#nim} If you are deploying NGINX Ingress Controller LTS in an "air-gapped" environment you will need to report to [NGINX Instance Manager](/nim/) instead of the default licensing endpoint. First, you must specify the endpoint of your NGINX Instance Manager. #### Helm Specify the endpoint using the `controller.mgmt.usageReport.endpoint` helm value. #### Manifests Specify the endpoint in the `usage-report-endpoint` Management ConfigMap key. #### Configure SSL certificates and SSL trusted certificates {#nim-cert} To configure SSL certificates or SSL trusted certificates, extra steps are necessary. To use Client Auth with NGINX Instance Manager, first create a Secret of type `kubernetes.io/tls` in the same namespace as the NGINX Ingress Controller LTS pods. ```shell kubectl create secret tls ssl-certificate --cert= --key= -n ``` To provide a SSL trusted certificate, and an optional Certificate Revocation List, create a Secret of type `nginx.org/ca` in the Namespace that the NIC Pod(s) are in. ```shell kubectl create secret generic ssl-trusted-certificate \ --from-file=ca.crt= \ --from-file=ca.crl= \ # optional --type=nginx.org/ca ``` Providing an optional CRL (certificate revocation list) will configure the [`ssl_crl`](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_crl) directive. #### Helm Specify the SSL certificate Secret name using the `controller.mgmt.sslCertificateSecretName` Helm value. Specify the SSL trusted certificate Secret name using the `controller.mgmt.sslTrustedCertificateSecretName` Helm value. #### Manifests Specify the SSL certificate Secret name in the `ssl-certificate-secret-name` management ConfigMap key. Specify the SSL trusted certificate Secret name in the `ssl-trusted-certificate-secret-name` management ConfigMap key. Once these Secrets are created and configured, you can now [install NGINX Ingress Controller LTS](/nic/lts/install/). ## What’s reported and how it’s protected {#telemetry} NGINX Plus reports the following data every hour by default: - **NGINX version and status**: The version of NGINX Plus running on the instance. - **Instance UUID**: A unique identifier for each NGINX Plus instance. - **Traffic data**: - **Bytes received from and sent to clients**: HTTP and stream traffic volume between clients and NGINX Plus. - **Bytes received from and sent to upstreams**: HTTP and stream traffic volume between NGINX Plus and upstream servers. - **Client connections**: The number of accepted client connections (HTTP and stream traffic). - **Requests handled**: The total number of HTTP requests processed. - **NGINX uptime**: The number of reloads and worker connections during uptime. - **Usage report timestamps**: Start and end times for each usage report. - **Kubernetes node details**: Information about Kubernetes nodes. ### Security and privacy of reported data All communication between your NGINX Plus instances, NGINX Instance Manager, and F5’s licensing endpoint (`product.connect.nginx.com`) is protected using **SSL/TLS** encryption. Only **operational metrics** are reported — no **personally identifiable information (PII)** or **sensitive customer data** is transmitted. -------------------------------------------------------------------------------- title: "Custom annotations" url: https://docs.nginx.com/nginx-ingress-controller/configuration/ingress-resources/custom-annotations/index.md -------------------------------------------------------------------------------- This topic explains how you can use custom annotations with F5 NGINX Ingress Controller. Custom annotations enable you to quickly extend the Ingress resource to support many advanced features of NGINX, such as rate limiting, caching, etc. ## Overview NGINX Ingress Controller supports a number of annotations for the Ingress resource that fine tune NGINX configuration (for example, connection timeouts) or enable additional features (for example, JWT validation). The complete list of annotations is available [here](/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md). The annotations are provided only for the most common features and use cases, meaning that not every NGINX feature or a customization option is available through the annotations. Additionally, even if an annotation is available, it might not give you the satisfactory level of control of a particular NGINX feature. Custom annotations allow you to add an annotation for an NGINX feature that is not available as a regular annotation. In contrast with regular annotations, to add a custom annotation, you don't need to modify the Ingress Controller source code -- just modify the template. Additionally, with a custom annotation, you get full control of how the feature is implemented in NGINX configuration. ## Usage The Ingress Controller generates NGINX configuration for Ingress resources by executing a configuration template. See [NGINX template](https://github.com/nginx/kubernetes-ingress/blob/v/internal/configs/version1/nginx.ingress.tmpl) or [NGINX Plus template](https://github.com/nginx/kubernetes-ingress/blob/v/internal/configs/version1/nginx-plus.ingress.tmpl). To support custom annotations, the template has access to the information about the Ingress resource - its *name*, *namespace* and *annotations*. It is possible to check if a particular annotation present in the Ingress resource and conditionally insert NGINX configuration directives at multiple NGINX contexts - `http`, `server`, `location` or `upstream`. Additionally, you can get the value that is set to the annotation. Consider the following excerpt from the template, which was extended to support two custom annotations: ```jinja2 # This is the configuration for {{$.Ingress.Name}}/{{$.Ingress.Namespace}} {{if index $.Ingress.Annotations "custom.nginx.org/feature-a"}} # Insert config for feature A if the annotation is set {{end}} {{with $value := index $.Ingress.Annotations "custom.nginx.org/feature-b"}} # Insert config for feature B if the annotation is set # Print the value assigned to the annotation: {{$value}} {{end}} ``` Consider the following Ingress resource and note how we set two annotations: ```yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: example-ingress namespace: production annotations: custom.nginx.org/feature-a: "on" custom.nginx.org/feature-b: "512" spec: rules: - host: example.com . . . ``` Assuming that the Ingress Controller is using that customized template, it will generate a config for the Ingress resource that will include the following part, generated by our template excerpt: ```yaml # This is the configuration for cafe-ingress/default # Insert config for feature A if the annotation is set # Insert config for feature B if the annotation is set # Print the value assigned to the annotation: 512 ``` **Notes**: - You can customize the template to insert you custom annotations via [custom templates](/nic//configuration/global-configuration/custom-templates.md). - The Ingress Controller uses go templates to generate NGINX config. You can read more information about go templates [here](https://golang.org/pkg/text/template/). See the examples in the next section that use custom annotations to configure NGINX features. ### Custom Annotations with Mergeable Ingress Resources A Mergeable Ingress resource consists of multiple Ingress resources - one master and one or several minions. Read more about Mergeable Ingress resources [here](/nic/configuration/ingress-resources/cross-namespace-configuration.md). If you'd like to use custom annotations with Mergeable Ingress resources, please keep the following in mind: - Custom annotations can be used in the Master and in Minions. For Minions, you can access them in the template only when processing locations. If you access `$.Ingress` anywhere in the Ingress template, you will get the master Ingress resource. To access a Minion Ingress resource, use `$location.MinionIngress`. However, it is only available when processing locations: ```jinja2 {{range $location := $server.Locations}} location {{$location.Path}} { {{with $location.MinionIngress}} # location for minion {{$location.MinionIngress.Namespace}}/{{$location.MinionIngress.Name}} {{end}} } {{end}} ``` **Note**: `$location.MinionIngress` is a pointer. When a regular Ingress resource is processed in the template, the value of the pointer is `nil`. Thus, it is important that you check that `$location.MinionIngress` is not `nil` as in the example above using the `with` action. - Minions do not inherent custom annotations of the master. ### Helper Functions Helper functions can be used in the Ingress template to parse the values of custom annotations. | Function | Input Arguments | Return Arguments | Description | | ---| ---| ---| --- | | ``split`` | ``s, sep string`` | ``[]string`` | Splits the string ``s`` into a slice of strings separated by the ``sep``. | | ``trim`` | ``s string`` | ``string`` | Trims the trailing and leading whitespace from the string ``s``. | | ``contains`` | ``s, substr string`` | ``bool`` | Tests whether the string ``substr`` is a substring of the string ``s``. | | ``hasPrefix`` | ``s, prefix string`` | ``bool`` | Tests whether the string ``prefix`` is a prefix of the string ``s``. | | ``hasSuffix`` | ``s, suffix string`` | ``bool`` | Tests whether the string ``suffix`` is a suffix of the string ``s``. | | ``toLower`` | ``s string`` | ``bool`` | Converts all letters in the string ``s`` to their lower case. | | ``toUpper`` | ``s string`` | ``bool`` | Converts all letters in the string ``s`` to their upper case. | | ``replaceAll`` | ``s, old, new string`` | ``string`` | Replaces all occurrences of ``old`` with ``new`` in the string ``s``. | Consider the following custom annotation `custom.nginx.org/allowed-ips`, which expects a comma-separated list of IP addresses: ```yaml annotations: custom.nginx.org/allowed-ips: "192.168.1.3, 10.0.0.13" ``` The helper functions can parse the value of the `custom.nginx.org/allowed-ips` annotation, so that in the template you can use each IP address separately. Consider the following template excerpt: ```jinja2 {{range $ip := split (index $.Ingress.Annotations "custom.nginx.org/allowed-ips") ","}} allow {{trim $ip}}; {{end}} deny all; ``` The template excerpt will generate the following configuration: ``` allow 192.168.1.3; allow 10.0.0.13; deny all; ``` ## Example See the [custom annotations example](https://github.com/nginx/kubernetes-ingress/blob/v/examples/ingress-resources/custom-annotations). -------------------------------------------------------------------------------- title: "Custom annotations" url: https://docs.nginx.com/nginx-ingress-controller/lts/configuration/ingress-resources/custom-annotations/index.md -------------------------------------------------------------------------------- This topic explains how you can use custom annotations with F5 NGINX Ingress Controller LTS. Custom annotations enable you to quickly extend the Ingress resource to support many advanced features of NGINX, such as rate limiting, caching, etc. ## Overview NGINX Ingress Controller LTS supports a number of annotations for the Ingress resource that fine tune NGINX configuration (for example, connection timeouts) or enable additional features (for example, JWT validation). The complete list of annotations is available [here](/nic/lts/configuration/ingress-resources/advanced-configuration-with-annotations.md). The annotations are provided only for the most common features and use cases, meaning that not every NGINX feature or a customization option is available through the annotations. Additionally, even if an annotation is available, it might not give you the satisfactory level of control of a particular NGINX feature. Custom annotations allow you to add an annotation for an NGINX feature that is not available as a regular annotation. In contrast with regular annotations, to add a custom annotation, you don't need to modify the Ingress Controller source code -- just modify the template. Additionally, with a custom annotation, you get full control of how the feature is implemented in NGINX configuration. ## Usage The Ingress Controller generates NGINX configuration for Ingress resources by executing a configuration template. See [NGINX Plus template](https://github.com/nginx/kubernetes-ingress/blob/release-2026-lts/internal/configs/version1/nginx-plus.ingress.tmpl). To support custom annotations, the template has access to the information about the Ingress resource - its *name*, *namespace* and *annotations*. It is possible to check if a particular annotation present in the Ingress resource and conditionally insert NGINX configuration directives at multiple NGINX contexts - `http`, `server`, `location` or `upstream`. Additionally, you can get the value that is set to the annotation. Consider the following excerpt from the template, which was extended to support two custom annotations: ```jinja2 # This is the configuration for {{$.Ingress.Name}}/{{$.Ingress.Namespace}} {{if index $.Ingress.Annotations "custom.nginx.org/feature-a"}} # Insert config for feature A if the annotation is set {{end}} {{with $value := index $.Ingress.Annotations "custom.nginx.org/feature-b"}} # Insert config for feature B if the annotation is set # Print the value assigned to the annotation: {{$value}} {{end}} ``` Consider the following Ingress resource and note how we set two annotations: ```yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: example-ingress namespace: production annotations: custom.nginx.org/feature-a: "on" custom.nginx.org/feature-b: "512" spec: rules: - host: example.com . . . ``` Assuming that the Ingress Controller is using that customized template, it will generate a config for the Ingress resource that will include the following part, generated by our template excerpt: ```yaml # This is the configuration for cafe-ingress/default # Insert config for feature A if the annotation is set # Insert config for feature B if the annotation is set # Print the value assigned to the annotation: 512 ``` **Notes**: - You can customize the template to insert you custom annotations via [custom templates](/nic/lts/configuration/global-configuration/custom-templates.md). - The Ingress Controller uses go templates to generate NGINX config. You can read more information about go templates [here](https://golang.org/pkg/text/template/). See the examples in the next section that use custom annotations to configure NGINX features. ### Custom Annotations with Mergeable Ingress Resources A Mergeable Ingress resource consists of multiple Ingress resources - one master and one or several minions. Read more about Mergeable Ingress resources [here](/nic/lts/configuration/ingress-resources/cross-namespace-configuration.md). If you'd like to use custom annotations with Mergeable Ingress resources, please keep the following in mind: - Custom annotations can be used in the Master and in Minions. For Minions, you can access them in the template only when processing locations. If you access `$.Ingress` anywhere in the Ingress template, you will get the master Ingress resource. To access a Minion Ingress resource, use `$location.MinionIngress`. However, it is only available when processing locations: ```jinja2 {{range $location := $server.Locations}} location {{$location.Path}} { {{with $location.MinionIngress}} # location for minion {{$location.MinionIngress.Namespace}}/{{$location.MinionIngress.Name}} {{end}} } {{end}} ``` **Note**: `$location.MinionIngress` is a pointer. When a regular Ingress resource is processed in the template, the value of the pointer is `nil`. Thus, it is important that you check that `$location.MinionIngress` is not `nil` as in the example above using the `with` action. - Minions do not inherent custom annotations of the master. ### Helper Functions Helper functions can be used in the Ingress template to parse the values of custom annotations. | Function | Input Arguments | Return Arguments | Description | | ---| ---| ---| --- | | ``split`` | ``s, sep string`` | ``[]string`` | Splits the string ``s`` into a slice of strings separated by the ``sep``. | | ``trim`` | ``s string`` | ``string`` | Trims the trailing and leading whitespace from the string ``s``. | | ``contains`` | ``s, substr string`` | ``bool`` | Tests whether the string ``substr`` is a substring of the string ``s``. | | ``hasPrefix`` | ``s, prefix string`` | ``bool`` | Tests whether the string ``prefix`` is a prefix of the string ``s``. | | ``hasSuffix`` | ``s, suffix string`` | ``bool`` | Tests whether the string ``suffix`` is a suffix of the string ``s``. | | ``toLower`` | ``s string`` | ``bool`` | Converts all letters in the string ``s`` to their lower case. | | ``toUpper`` | ``s string`` | ``bool`` | Converts all letters in the string ``s`` to their upper case. | | ``replaceAll`` | ``s, old, new string`` | ``string`` | Replaces all occurrences of ``old`` with ``new`` in the string ``s``. | Consider the following custom annotation `custom.nginx.org/allowed-ips`, which expects a comma-separated list of IP addresses: ```yaml annotations: custom.nginx.org/allowed-ips: "192.168.1.3, 10.0.0.13" ``` The helper functions can parse the value of the `custom.nginx.org/allowed-ips` annotation, so that in the template you can use each IP address separately. Consider the following template excerpt: ```jinja2 {{range $ip := split (index $.Ingress.Annotations "custom.nginx.org/allowed-ips") ","}} allow {{trim $ip}}; {{end}} deny all; ``` The template excerpt will generate the following configuration: ``` allow 192.168.1.3; allow 10.0.0.13; deny all; ``` ## Example See the [custom annotations example](https://github.com/nginx/kubernetes-ingress/tree/release-2026-lts/examples/ingress-resources/custom-annotations). -------------------------------------------------------------------------------- title: "DoS protected resource specification" url: https://docs.nginx.com/nginx-ingress-controller/integrations/app-protect-dos/dos-protected/index.md -------------------------------------------------------------------------------- F5 DoS for NGINX protected resource specification **Note:** This feature is only available using the NGINX Plus [F5 DoS for NGINX Module](/nap-dos/deployment-guide/learn-about-deployment.md). ## DoS Protected resource specification Below is an example of a DoS protected resource. ```yaml apiVersion: appprotectdos.f5.com/v1beta1 kind: DosProtectedResource metadata: name: dos-protected spec: enable: true name: "my-dos" apDosMonitor: uri: "webapp.example.com" ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``enable`` | Enables F5 DoS for NGINX, Default value: false. | ``bool`` | No | |``name`` | Name of the protected object, max of 63 characters. | ``string`` | Yes | |``dosAccessLogDest`` | The log destination for the access log with dos log format. Accepted variables are ``:`` or ``stderr``. | ``string`` | No | |``apDosMonitor.uri`` | The destination to the desired protected object. [App Protect DoS monitor](#dosprotectedresourceapdosmonitor) Default value: None, URL will be extracted from the first request which arrives and taken from "Host" header or from destination ip+port. | ``string`` | No | |``apDosMonitor.protocol`` | Determines if the server listens on http1 / http2 / grpc / websocket. [App Protect DoS monitor](#dosprotectedresourceapdosmonitor) Default value: http1. | ``enum`` | No | |``apDosMonitor.timeout`` | Determines how long (in seconds) should F5 DoS for NGINX wait for a response. [App Protect DoS monitor](#dosprotectedresourceapdosmonitor) Default value: 10 seconds for http1/http2 and 5 seconds for grpc. | ``int64`` | No | |``apDosPolicy`` | The [App Protect DoS policy](#dosprotectedresourceapdospolicy) of the dos. Accepts an optional namespace. | ``string`` | No | |``dosSecurityLog.enable`` | Enables security log. | ``bool`` | No | |``dosSecurityLog.apDosLogConf`` | The [App Protect DoS log conf](/nic/integrations/app-protect-dos/configuration.md#app-protect-dos-logs) resource. Accepts an optional namespace. | ``string`` | No | |``dosSecurityLog.dosLogDest`` | The log destination for the security log. Accepted variables are ``:`` or ``stderr``. Default is ``127.0.0.1:514``. | ``string`` | No | |``allowList`` | List of allowed IP addresses and subnet masks. Each entry is represented by an `IPWithMask` string. | ``[]AllowListEntry`` | No | ### DosProtectedResource.apDosPolicy The `apDosPolicy` is a reference (qualified identifier in the format `namespace/name`) to the policy configuration defined as an `ApDosPolicy`. ### DosProtectedResource.apDosMonitor This is how F5 DoS for NGINX monitors the stress level of the protected object. The monitor requests are sent from localhost (127.0.0.1). ### Invalid DoS Protected resources NGINX will treat a DoS protected resource as invalid if one of the following conditions is met: - The DoS protected resource doesn't pass the [comprehensive validation](#comprehensive-validation). - The DoS protected resource isn't present in the cluster. ### Validation Two types of validation are available for the DoS protected resource: - *Structural validation*, done by `kubectl` and the Kubernetes API server. - *Comprehensive validation*, done by NGINX Ingress Controller. #### Structural validation The custom resource definition for the DoS protected resource includes a structural OpenAPI schema, which describes the type of every field of the resource. If you try to create (or update) a resource that violates the structural schema -- for example, the resource uses a string value instead of a bool in the `enable` field -- `kubectl` and the Kubernetes API server will reject the resource. - Example of `kubectl` validation: ```shell kubectl apply -f apdos-protected.yaml ``` ```shell error: error validating "examples/app-protect-dos/apdos-protected.yaml": error validating data: ValidationError(DosProtectedResource.spec.enable): invalid type for com.f5.appprotectdos.v1beta1.DosProtectedResource.spec.enable: got "string", expected "boolean"; if you choose to ignore these errors, turn validation off with --validate=false ``` - Example of Kubernetes API server validation: ```shell kubectl apply -f access-control-policy-allow.yaml --validate=false ``` ```shell The DosProtectedResource "dos-protected" is invalid: spec.enable: Invalid value: "string": spec.enable in body must be of type boolean: "string" ``` If a resource passes structural validation, then NGINX Ingress Controller will start comprehensive validation. #### Comprehensive validation NGINX Ingress Controller validates the fields of a DoS protected resource. If a resource is invalid, NGINX Ingress Controller will reject it. The resource will continue to exist in the cluster, but NGINX Ingress Controller will ignore it. You can use `kubectl` to check if NGINX Ingress Controller successfully applied a DoS protected resource configuration. For our example `dos-protected` DoS protected resource, we can run: ```shell kubectl describe dosprotectedresource dos-protected ``` ```shell Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal AddedOrUpdated 12s (x2 over 18h) nginx-ingress-controller Configuration for default/dos-protected was added or updated ``` Note how the events section includes a Normal event with the AddedOrUpdated reason that informs us that the configuration was successfully applied. If you create an invalid resource, NGINX Ingress Controller will reject it and emit a Rejected event. For example, if you create a dos protected resource `dos-protected` with an invalid URI `bad` in the `dosSecurityLog/dosLogDest` field, you will get: ```shell kubectl describe policy webapp-policy ``` ```shell Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning Rejected 2s nginx-ingress-controller error validating DosProtectedResource: dos-protected invalid field: dosSecurityLog/dosLogDest err: invalid log destination: bad, must follow format: : or stderr ``` The events section has Warning event with the rejection error in the message. **Note:** If you invalidate an existing resource, NGINX Ingress Controller will reject it. -------------------------------------------------------------------------------- title: "Enable OpenTelemetry" url: https://docs.nginx.com/nginx-ingress-controller/logging-and-monitoring/opentelemetry/index.md -------------------------------------------------------------------------------- This topic describes how to enable [OpenTelemetry](https://opentelemetry.io/) for F5 NGINX Ingress Controller using the [native NGINX module](https://nginx.org/en/docs/ngx_otel_module.html). ## Before you begin To complete this guide, you need the following pre-requisites: - An [NGINX Ingress Controller installation](/nic/install/) with OpenTelemetry (v5.1.0+) ## Load the OpenTelemetry module To enable OpenTelemetry, you must first load the module by adding the [_otel-exporter-endpoint_ ConfigMap key](/nic/configuration/global-configuration/configmap-resource.md#modules), which takes an endpoint argument. The following is an example of a OpenTelemetry collector running in your cluster as the target for exporting data: ```yaml otel-exporter-endpoint: "http://otel-collector.default.svc.cluster.local:4317" ``` A complete ConfigMap example with all OpenTelemetry options could look as follows: *[Code: https://raw.githubusercontent.com/nginx/kubernetes-ingress/refs/heads/main/examples/shared-examples/otel/nginx-config.yaml]* ## Enable OpenTelemetry Once you have loaded the module, you can now enable OpenTelemetry. You can configure it globally for all resources, or on a per resource basis. ### Global To enable OpenTelemetry for all resources, set the _otel-trace-in-http_ ConfigMap key to `true`: ```yaml otel-trace-in-http: "true" ``` ### Per resource You can configure OpenTelemetry on a per resource basis in NGINX Ingress Controller. For this functionality, you must [enable snippets](/nic/configuration/ingress-resources/advanced-configuration-with-snippets.md) with the `-enable-snippets` command-line argument. Based on the state of global configuration, you can selectively enable or disable metrics for each resource. #### Enable a specific resource or path With OpenTelemetry **disabled** globally, you can enable it for a specific resource using the server snippet annotation: ```yaml nginx.org/server-snippets: | otel_trace on; ``` You can enable it for specific paths using [Mergeable Ingress resources](/nic/configuration/ingress-resources/cross-namespace-configuration.md). Use the server snippet annotation for the paths of a specific Minion Ingress resource: ```yaml nginx.org/location-snippets: | otel_trace on; ``` #### Disable a specific resource or path With OpenTelemetry **enabled** globally, you can disable it for a specific resource using the server snippet annotation: ```yaml nginx.org/server-snippets: | otel_trace off; ``` You can disable it for specific paths using [Mergeable Ingress resources](/nic/configuration/ingress-resources/cross-namespace-configuration.md). Use the server snippet annotation for the paths of a specific Minion Ingress resource: ```yaml nginx.org/location-snippets: | otel_trace off; ``` ## Customize OpenTelemetry **Note:** You cannot modify the additional directives in the _otel_exporter_ block using snippets. You can customize OpenTelemetry through the supported [OpenTelemetry module directives](https://nginx.org/en/docs/ngx_otel_module.html). Use the `location-snippets` ConfigMap keys or annotations to insert those directives into the generated NGINX configuration. -------------------------------------------------------------------------------- title: "Enable OpenTelemetry" url: https://docs.nginx.com/nginx-ingress-controller/lts/logging-and-monitoring/opentelemetry/index.md -------------------------------------------------------------------------------- This topic describes how to enable [OpenTelemetry](https://opentelemetry.io/) for F5 NGINX Ingress Controller LTS using the [native NGINX module](https://nginx.org/en/docs/ngx_otel_module.html). ## Before you begin To complete this guide, you need the following pre-requisites: - An [NGINX Ingress Controller LTS installation](/nic/lts/install/) with OpenTelemetry (v5.1.0+) ## Load the OpenTelemetry module To enable OpenTelemetry, you must first load the module by adding the [_otel-exporter-endpoint_ ConfigMap key](/nic/lts/configuration/global-configuration/configmap-resource.md#modules), which takes an endpoint argument. The following is an example of a OpenTelemetry collector running in your cluster as the target for exporting data: ```yaml otel-exporter-endpoint: "http://otel-collector.default.svc.cluster.local:4317" ``` A complete ConfigMap example with all OpenTelemetry options could look as follows: *[Code: https://raw.githubusercontent.com/nginx/kubernetes-ingress/refs/heads/main/examples/shared-examples/otel/nginx-config.yaml]* ## Enable OpenTelemetry Once you have loaded the module, you can now enable OpenTelemetry. You can configure it globally for all resources, or on a per resource basis. ### Global To enable OpenTelemetry for all resources, set the _otel-trace-in-http_ ConfigMap key to `true`: ```yaml otel-trace-in-http: "true" ``` ### Per resource You can configure OpenTelemetry on a per resource basis in NGINX Ingress Controller LTS. For this functionality, you must [enable snippets](/nic/lts/configuration/ingress-resources/advanced-configuration-with-snippets.md) with the `-enable-snippets` command-line argument. Based on the state of global configuration, you can selectively enable or disable metrics for each resource. #### Enable a specific resource or path With OpenTelemetry **disabled** globally, you can enable it for a specific resource using the server snippet annotation: ```yaml nginx.org/server-snippets: | otel_trace on; ``` You can enable it for specific paths using [Mergeable Ingress resources](/nic/lts/configuration/ingress-resources/cross-namespace-configuration.md). Use the server snippet annotation for the paths of a specific Minion Ingress resource: ```yaml nginx.org/location-snippets: | otel_trace on; ``` #### Disable a specific resource or path With OpenTelemetry **enabled** globally, you can disable it for a specific resource using the server snippet annotation: ```yaml nginx.org/server-snippets: | otel_trace off; ``` You can disable it for specific paths using [Mergeable Ingress resources](/nic/lts/configuration/ingress-resources/cross-namespace-configuration.md). Use the server snippet annotation for the paths of a specific Minion Ingress resource: ```yaml nginx.org/location-snippets: | otel_trace off; ``` ## Customize OpenTelemetry **note:** You cannot modify the additional directives in the _otel_exporter_ block using snippets. You can customize OpenTelemetry through the supported [OpenTelemetry module directives](https://nginx.org/en/docs/ngx_otel_module.html). Use the `location-snippets` ConfigMap keys or annotations to insert those directives into the generated NGINX configuration. -------------------------------------------------------------------------------- title: "Extensibility with NGINX Plus" url: https://docs.nginx.com/nginx-ingress-controller/lts/overview/nginx-plus/index.md -------------------------------------------------------------------------------- This document explains how F5 NGINX Plus can extend the functionality of F5 NGINX Ingress Controller LTS. NGINX Ingress Controller LTS works with [NGINX Plus](https://www.nginx.com/products/nginx/), a commercial closed source version of NGINX which has additional features and support from NGINX Inc. NGINX Ingress Controller LTS can leverage functionality from NGINX Plus to extend its base capabilities. --- ## Additional features - _Real-time metrics_: Metrics for NGINX Plus and application performance are available through the API or the [NGINX Status Page](/nic/lts/logging-and-monitoring/status-page). These metrics can also be exported to [Prometheus](/nic/lts/logging-and-monitoring/prometheus). - _Additional load balancing methods_: The `least_time` and `random two least_time` methods and their derivatives become available. The NGINX [`ngx_http_upstream_module` documentation](https://nginx.org/en/docs/http/ngx_http_upstream_module.html) has the complete list of load balancing methods. - _Session persistence_: While the *sticky cookie* method is available in both NGINX and NGINX Plus, NGINX Plus provides additional session persistence methods, including *sticky route* and *sticky learn*. See the [Ingress Resource](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/session-persistence) and [Custom Resource](https://github.com/nginx/kubernetes-ingress/tree/v/examples/custom-resources/session-persistence) examples. - _Active health checks_: See the [Ingress Resource](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/health-checks) and [Custom Resource](https://github.com/nginx/kubernetes-ingress/tree/v/examples/custom-resources/health-checks) examples. - _JWT validation_: See the [Ingress Resource](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/jwt) and [Custom Resource](https://github.com/nginx/kubernetes-ingress/tree/v/examples/custom-resources/jwt) examples. For a comprehensive guide of NGINX Plus features available with Ingress resources, see the [ConfigMap](/nic/lts/configuration/global-configuration/configmap-resource) and [Annotations](/nic/lts/configuration/ingress-resources/advanced-configuration-with-annotations) documentation. **note:** NGINX Plus features are configured for Ingress resources using Annotations that start with `nginx.com`. For a comprehensive guide of NGINX Plus features available with custom resources, see the [Policy](/nic/lts/configuration/policy-resource), [VirtualServer](/nic/lts/configuration/virtualserver-and-virtualserverroute-resources) and [TransportServer](/nic/lts/configuration/transportserver-resource) documentation. --- ## Dynamic reconfiguration NGINX Ingress Controller LTS updates the configuration of the load balancer to reflect changes every time the number of pods exposed through an Ingress resource changes. When using NGINX, the configuration file must be changed then reloaded. For NGINX Plus, its dynamic reconfiguration is utilized, updating NGINX Plus without reloading. This avoids the increase of memory usage caused by reloads (Particularly with large volumes of client requests) and when load balancing applications with long-lived connections (Such as those using WebSockets or handling file uploads, downloads or streaming). -------------------------------------------------------------------------------- title: "Extensibility with NGINX Plus" url: https://docs.nginx.com/nginx-ingress-controller/overview/nginx-plus/index.md -------------------------------------------------------------------------------- This document explains how F5 NGINX Plus can extend the functionality of F5 NGINX Ingress Controller. NGINX Ingress Controller works with [NGINX](https://nginx.org/) as well as [NGINX Plus](https://www.nginx.com/products/nginx/), a commercial closed source version of NGINX which has additional features and support from NGINX Inc. NGINX Ingress Controller can leverage functionality from NGINX Plus to extend its base capabilities. --- ## Additional features - _Real-time metrics_: Metrics for NGINX Plus and application performance are available through the API or the [NGINX Status Page](/nic/logging-and-monitoring/status-page). These metrics can also be exported to [Prometheus](/nic/logging-and-monitoring/prometheus). - _Additional load balancing methods_: The `least_time` and `random two least_time` methods and their derivatives become available. The NGINX [`ngx_http_upstream_module` documentation](https://nginx.org/en/docs/http/ngx_http_upstream_module.html) has the complete list of load balancing methods. - _Session persistence_: While the *sticky cookie* method is available in both NGINX and NGINX Plus, NGINX Plus provides additional session persistence methods, including *sticky route* and *sticky learn*. See the [Ingress Resource](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/session-persistence) and [Custom Resource](https://github.com/nginx/kubernetes-ingress/tree/v/examples/custom-resources/session-persistence) examples. - _Active health checks_: See the [Ingress Resource](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/health-checks) and [Custom Resource](https://github.com/nginx/kubernetes-ingress/tree/v/examples/custom-resources/health-checks) examples. - _JWT validation_: See the [Ingress Resource](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/jwt) and [Custom Resource](https://github.com/nginx/kubernetes-ingress/tree/v/examples/custom-resources/jwt) examples. For a comprehensive guide of NGINX Plus features available with Ingress resources, see the [ConfigMap](/nic/configuration/global-configuration/configmap-resource) and [Annotations](/nic/configuration/ingress-resources/advanced-configuration-with-annotations) documentation. **Note:** NGINX Plus features are configured for Ingress resources using Annotations that start with `nginx.com`. For a comprehensive guide of NGINX Plus features available with custom resources, see the [Policy](/nic/configuration/policy-resource), [VirtualServer](/nic/configuration/virtualserver-and-virtualserverroute-resources) and [TransportServer](/nic/configuration/transportserver-resource) documentation. --- ## Dynamic reconfiguration NGINX Ingress Controller updates the configuration of the load balancer to reflect changes every time the number of pods exposed through an Ingress resource changes. When using NGINX, the configuration file must be changed then reloaded. For NGINX Plus, its dynamic reconfiguration is utilized, updating NGINX Plus without reloading. This avoids the increase of memory usage caused by reloads (Particularly with large volumes of client requests) and when load balancing applications with long-lived connections (Such as those using WebSockets or handling file uploads, downloads or streaming). -------------------------------------------------------------------------------- title: "Manage Policy resources with kubectl" url: https://docs.nginx.com/nginx-ingress-controller/lts/configuration/policy-resource/using-policy/index.md -------------------------------------------------------------------------------- Use standard `kubectl` commands to work with Policy resources, just as you would with built-in Kubernetes resources. For example, the following command creates a Policy resource defined in `access-control-policy-allow.yaml` with the name `webapp-policy`: ```shell kubectl apply -f access-control-policy-allow.yaml policy.k8s.nginx.org/webapp-policy configured ``` Get the resource by running: ```shell kubectl get policy webapp-policy NAME AGE webapp-policy 27m ``` For `kubectl get` and similar commands, you can also use the short name `pol` instead of `policy`. ## Attach policies to a resource You can attach policies to VirtualServer, VirtualServerRoute, and Ingress resources. For example: - VirtualServer: ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServer metadata: name: cafe namespace: cafe spec: host: cafe.example.com tls: secret: cafe-secret policies: # spec policies - name: policy1 upstreams: - name: coffee service: coffee-svc port: 80 routes: - path: /tea policies: # route policies - name: policy2 namespace: cafe route: tea/tea - path: /coffee policies: # route policies - name: policy3 namespace: cafe action: pass: coffee ``` For VirtualServer, you can apply a policy: * to all routes (spec policies) * to a specific route (route policies) Route policies of the same type override spec policies. In the example above, if `policy-1` and `policy-3` are both `accessControl` policies, NGINX applies `policy-3` to requests for `cafe.example.com/coffee`. NGINX enforces this override: the spec policies apply in the `server` context of the configuration, and the route policies apply in the `location` context. As a result, the route policies of the same type take precedence. - VirtualServerRoute, referenced by the VirtualServer above: ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServerRoute metadata: name: tea namespace: tea spec: host: cafe.example.com upstreams: - name: tea service: tea-svc port: 80 subroutes: # subroute policies - path: /tea policies: - name: policy4 namespace: tea action: pass: tea ``` For VirtualServerRoute, you can apply a policy to a subroute (subroute policies). Subroute policies of the same type override spec policies. In the example above, if `policy-1` (in the VirtualServer) and `policy-4` are both `accessControl` policies, NGINX applies `policy-4` to requests for `cafe.example.com/tea`. As with the VirtualServer, NGINX enforces this override. Subroute policies always override route policies, regardless of type. For example, NGINX Ingress Controller LTS ignores `policy-2` from the VirtualServer route for the `/tea` subroute, because the subroute has its own policies, `policy4` in this case. If the subroute had no policies, NGINX Ingress Controller LTS would apply `policy-2` instead. NGINX Ingress Controller LTS enforces this override: the `location` context for the subroute has either route policies or subroute policies, but never both. - Ingress: ```yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: cafe-ingress annotations: nginx.org/policies: "webapp-policy" spec: ingressClassName: nginx tls: - hosts: - cafe.example.com secretName: tls-secret rules: - host: cafe.example.com http: paths: - path: /tea pathType: Prefix backend: service: name: tea-svc port: number: 80 - path: /coffee pathType: Prefix backend: service: name: coffee-svc port: number: 80 ``` For Ingress, you can apply policies: * to a single Ingress * to a master Ingress, where minion Ingresses inherit the policies * to minion Ingresses, where minion policies override master policies ## Invalid policies NGINX treats a policy as invalid if any of the following conditions is true: - The policy doesn't pass [comprehensive validation](#comprehensive-validation). - The policy isn't present in the cluster. - The policy doesn't meet its type-specific requirements. For example, an `ingressMTLS` policy requires TLS termination turned on in the VirtualServer. For an invalid policy, NGINX returns the 500 status code for client requests, following these rules: - If a policy is referenced in a VirtualServer `route` or a VirtualServerRoute `subroute`, NGINX returns the 500 status code for requests to the URIs of that route or subroute. - If a policy is referenced in the VirtualServer `spec`, NGINX returns the 500 status code for requests to all URIs of that VirtualServer. If a policy is invalid, the VirtualServer or VirtualServerRoute gets the [status](/nic/lts/configuration/global-configuration/reporting-resources-status.md#virtualserver-and-virtualserverroute-resources) state `Warning`, with a message that explains why the policy is invalid. ## Validation Two types of validation are available for the Policy resource: - *Structural validation*, done by `kubectl` and the Kubernetes API server. - *Comprehensive validation*, done by NGINX Ingress Controller LTS. ### Structural validation The custom resource definition for the Policy includes a structural OpenAPI schema, which describes the type of every field of the resource. If you try to create or update a resource that violates the structural schema, for example, if the resource uses a string value instead of an array of strings in the `allow` field, `kubectl` and the Kubernetes API server reject the resource. - Example of `kubectl` validation: ```shell kubectl apply -f access-control-policy-allow.yaml error: error validating "access-control-policy-allow.yaml": error validating data: ValidationError(Policy.spec.accessControl.allow): invalid type for org.nginx.k8s.v1.Policy.spec.accessControl.allow: got "string", expected "array"; if you choose to ignore these errors, turn validation off with --validate=false ``` - Example of Kubernetes API server validation: ```shell kubectl apply -f access-control-policy-allow.yaml --validate=false The Policy "webapp-policy" is invalid: spec.accessControl.allow: Invalid value: "string": spec.accessControl.allow in body must be of type array: "string" ``` If a resource passes structural validation, NGINX Ingress Controller LTS's comprehensive validation runs next. ### Comprehensive validation NGINX Ingress Controller LTS validates the fields of a Policy resource. If a resource is invalid, NGINX Ingress Controller LTS rejects it. The resource continues to exist in the cluster, but NGINX Ingress Controller LTS ignores it. Use `kubectl` to check whether NGINX Ingress Controller LTS successfully applied a Policy configuration. For the example `webapp-policy` Policy, run: ```shell kubectl describe pol webapp-policy . . . Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal AddedOrUpdated 11s nginx-ingress-controller Policy default/webapp-policy was added or updated ``` The `events` section includes a `Normal` event with the `AddedOrUpdated` reason, which tells you the configuration applied successfully. If you create an invalid resource, NGINX Ingress Controller LTS rejects it and emits a Rejected event. For example, if you create a Policy `webapp-policy` with an invalid IP `10.0.0.` in the `allow` field, you get: ```shell kubectl describe policy webapp-policy . . . Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning Rejected 7s nginx-ingress-controller Policy default/webapp-policy is invalid and was rejected: spec.accessControl.allow[0]: Invalid value: "10.0.0.": must be a CIDR or IP ``` The events section includes a `Warning` event with the `Rejected` reason. This information is also available in the `status` field of the Policy resource. Note the Status section of the Policy: ```shell kubectl describe pol webapp-policy . . . Status: Message: Policy default/webapp-policy is invalid and was rejected: spec.accessControl.allow[0]: Invalid value: "10.0.0.": must be a CIDR or IP Reason: Rejected State: Invalid ``` **Note:** If you make an existing resource invalid, NGINX Ingress Controller LTS rejects it. -------------------------------------------------------------------------------- title: "Management ConfigMap resource" url: https://docs.nginx.com/nginx-ingress-controller/configuration/global-configuration/mgmt-configmap-resource/index.md -------------------------------------------------------------------------------- When using F5 NGINX Ingress Controller with NGINX Plus, it is required to pass a [command line argument](/nic/configuration/global-configuration/command-line-arguments) to NGINX Ingress Controller, `--mgmt-configmap=` which specifies the ConfigMap to use. The minimal required ConfigMap must have a `license-token-secret-name` key. Helm users will not need to create this map or pass the argument, it will be created with a Helm install. --- 1. Create a ConfigMap file with the name *nginx-config-mgmt.yaml* and set the values that make sense for your setup: ```yaml apiVersion: v1 kind: ConfigMap metadata: name: nginx-config-mgmt namespace: nginx-ingress data: license-token-secret-name: "license-token" ``` 1. Create a new (or update the existing) ConfigMap resource: ```shell kubectl apply -f nginx-config-mgmt.yaml ``` The [NGINX Management](https://nginx.org/en/docs/ngx_mgmt_module.html) block configuration will be updated. ## Management ConfigMap keys |ConfigMap Key | Description | Default | | ---| ---| ---| |*license-token-secret-name* | Configures the secret used in the [license_token](https://nginx.org/en/docs/ngx_mgmt_module.html#license_token) directive. This key assumes the secret is in the Namespace that NGINX Ingress Controller is deployed in. The secret must be of type `nginx.com/license` with the base64 encoded JWT in the `license.jwt` key. | N/A | |*ssl-verify* | Configures the [ssl_verify](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_verify) directive, which enables or disables verification of the usage reporting endpoint certificate. | `true` | |*enforce-initial-report* | Configures the [enforce_initial_report](https://nginx.org/en/docs/ngx_mgmt_module.html#enforce_initial_report) directive, which enables or disables the 180-day grace period for sending the initial usage report. | `false` | |*usage-report-endpoint* | Configures the endpoint of the [usage_report](https://nginx.org/en/docs/ngx_mgmt_module.html#usage_report) directive. This is used to configure the endpoint NGINX uses to send usage reports to NIM. | `product.connect.nginx.com` | |*usage-report-interval* | Configures the interval of the [usage_report](https://nginx.org/en/docs/ngx_mgmt_module.html#usage_report) directive. This specifies the frequency that usage reports are sent. This field takes an [NGINX time](https://nginx.org/en/docs/syntax.html). | `1h` | |*usage-report-proxy-host* | Configures the host name of the [proxy](https://nginx.org/en/docs/ngx_mgmt_module.html#proxy) directive with optional port. | N/A | |*ssl-trusted-certificate-secret-name* | Configures the secret used to create the file(s) referenced the in [ssl_trusted_certifcate](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_trusted_certificate), and [ssl_crl](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_crl) directives. This key assumes the secret is in the Namespace that NGINX Ingress Controller is deployed in. The secret must be of type `nginx.org/ca`, where the `ca.crt` key contains a base64 encoded trusted cert, and the optional `ca.crl` key can contain a base64 encoded CRL. If the optional `ca.crl` key is supplied, it will configure the NGINX `ssl_crl` directive. | N/A | |*ssl-certificate-secret-name* | Configures the secret used to create the `ssl_certificate` and `ssl_certificate_key` directives. This key assumes the secret is in the Namespace that NGINX Ingress Controller is deployed in. The secret must be of type `kubernetes.io/tls`| N/A | |*resolver-addresses* | Configures addresses used in the mgmt block [resolver](https://nginx.org/en/docs/ngx_mgmt_module.html#resolver) directive. This field takes a comma separated list of addresses. | N/A | |*resolver-ipv6* | Configures whether the mgmt block [resolver](https://nginx.org/en/docs/ngx_mgmt_module.html#resolver) directive will look up IPv6 addresses. | `true` | |*resolver-valid* | Configures an [NGINX time](https://nginx.org/en/docs/syntax.html) that the mgmt block [resolver](https://nginx.org/en/docs/ngx_mgmt_module.html#resolver) directive will override the TTL value of responses from nameservers with. | N/A | -------------------------------------------------------------------------------- title: "Management ConfigMap resource" url: https://docs.nginx.com/nginx-ingress-controller/lts/configuration/global-configuration/mgmt-configmap-resource/index.md -------------------------------------------------------------------------------- When using F5 NGINX Ingress Controller LTS, it is required to pass a [command line argument](/nic/lts/configuration/global-configuration/command-line-arguments) to NGINX Ingress Controller LTS, `--mgmt-configmap=` which specifies the ConfigMap to use. The minimal required ConfigMap must have a `license-token-secret-name` key. Helm users will not need to create this map or pass the argument, it will be created with a Helm install. --- 1. Create a ConfigMap file with the name *nginx-config-mgmt.yaml* and set the values that make sense for your setup: ```yaml apiVersion: v1 kind: ConfigMap metadata: name: nginx-config-mgmt namespace: nginx-ingress data: license-token-secret-name: "license-token" ``` 1. Create a new (or update the existing) ConfigMap resource: ```shell kubectl apply -f nginx-config-mgmt.yaml ``` The [NGINX Management](https://nginx.org/en/docs/ngx_mgmt_module.html) block configuration will be updated. ## Management ConfigMap keys |ConfigMap Key | Description | Default | | ---| ---| ---| |*license-token-secret-name* | Configures the secret used in the [license_token](https://nginx.org/en/docs/ngx_mgmt_module.html#license_token) directive. This key assumes the secret is in the Namespace that NGINX Ingress Controller LTS is deployed in. The secret must be of type `nginx.com/license` with the base64 encoded JWT in the `license.jwt` key. | N/A | |*ssl-verify* | Configures the [ssl_verify](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_verify) directive, which enables or disables verification of the usage reporting endpoint certificate. | `true` | |*enforce-initial-report* | Configures the [enforce_initial_report](https://nginx.org/en/docs/ngx_mgmt_module.html#enforce_initial_report) directive, which enables or disables the 180-day grace period for sending the initial usage report. | `false` | |*usage-report-endpoint* | Configures the endpoint of the [usage_report](https://nginx.org/en/docs/ngx_mgmt_module.html#usage_report) directive. This is used to configure the endpoint NGINX uses to send usage reports to NIM. | `product.connect.nginx.com` | |*usage-report-interval* | Configures the interval of the [usage_report](https://nginx.org/en/docs/ngx_mgmt_module.html#usage_report) directive. This specifies the frequency that usage reports are sent. This field takes an [NGINX time](https://nginx.org/en/docs/syntax.html). | `1h` | |*usage-report-proxy-host* | Configures the host name of the [proxy](https://nginx.org/en/docs/ngx_mgmt_module.html#proxy) directive with optional port. | N/A | |*ssl-trusted-certificate-secret-name* | Configures the secret used to create the file(s) referenced the in [ssl_trusted_certifcate](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_trusted_certificate), and [ssl_crl](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_crl) directives. This key assumes the secret is in the Namespace that NGINX Ingress Controller LTS is deployed in. The secret must be of type `nginx.org/ca`, where the `ca.crt` key contains a base64 encoded trusted cert, and the optional `ca.crl` key can contain a base64 encoded CRL. If the optional `ca.crl` key is supplied, it will configure the NGINX `ssl_crl` directive. | N/A | |*ssl-certificate-secret-name* | Configures the secret used to create the `ssl_certificate` and `ssl_certificate_key` directives. This key assumes the secret is in the Namespace that NGINX Ingress Controller LTS is deployed in. The secret must be of type `kubernetes.io/tls`| N/A | |*resolver-addresses* | Configures addresses used in the mgmt block [resolver](https://nginx.org/en/docs/ngx_mgmt_module.html#resolver) directive. This field takes a comma separated list of addresses. | N/A | |*resolver-ipv6* | Configures whether the mgmt block [resolver](https://nginx.org/en/docs/ngx_mgmt_module.html#resolver) directive will look up IPv6 addresses. | `true` | |*resolver-valid* | Configures an [NGINX time](https://nginx.org/en/docs/syntax.html) that the mgmt block [resolver](https://nginx.org/en/docs/ngx_mgmt_module.html#resolver) directive will override the TTL value of responses from nameservers with. | N/A | -------------------------------------------------------------------------------- title: "Security recommendations" url: https://docs.nginx.com/nginx-ingress-controller/configuration/security/index.md -------------------------------------------------------------------------------- F5 NGINX Ingress Controller follows Kubernetes best practices: this page outlines configuration specific to NGINX Ingress Controller you may require, including links to examples in the [GitHub repository](https://github.com/nginx/kubernetes-ingress/tree/v/examples). For general guidance, we recommend the official Kubernetes documentation for [Securing a Cluster](https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/). ## Kubernetes recommendations ### RBAC and Service Accounts Kubernetes uses [RBAC](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) to control the resources and operations available to different types of users. NGINX Ingress Controller requires RBAC to configure a [ServiceUser](https://kubernetes.io/docs/concepts/security/service-accounts/#default-service-accounts), and provides least privilege access in its standard deployment configurations: - [Helm](https://github.com/nginx/kubernetes-ingress/blob/v/deployments/rbac/rbac.yaml) - [Manifests](https://github.com/nginx/kubernetes-ingress/blob/v/deployments/rbac/rbac.yaml) By default, the ServiceAccount has access to all Secret resources in the cluster. ### Secrets [Secrets](https://kubernetes.io/docs/concepts/configuration/secret/) are required by NGINX Ingress Controller for certificates and privacy keys, which Kubernetes stores unencrypted by default. We recommend following the [Kubernetes documentation](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/) to store these Secrets using at-rest encryption. ## NGINX Ingress Controller recommendations ### Configure root filesystem as read-only **Note:** This feature has its own documentation in [F5 WAF for NGINX](/waf/configure/kubernetes-read-only.md) documentation. It is compatible with a Kubernetes deployment: it **is not** compatible with [F5 DoS for NGINX](/nap-dos/). NGINX Ingress Controller is designed to be resilient against attacks in various ways, such as running the service as non-root to avoid changes to files. We recommend setting filesystems on all containers to read-only, this includes `nginx-ingress-controller`, though also includes `waf-enforcer` and `waf-config-mgr` when F5 WAF for NGINXv5 is in use. This is so that the attack surface is further reduced by limiting changes to binaries and libraries. This is not enabled by default, but can be enabled with **Helm** using the [**readOnlyRootFilesystem**](/nic/install/helm.md#configuration) argument in security contexts on all containers: `nginx-ingress-controller`, `waf_enforcer` and `waf_config_mgr`. For **Manifests**, uncomment the following sections of the deployment and add sections for `waf-enforcer` and `waf-config-mgr` containers: - `readOnlyRootFilesystem: true` - The entire **volumeMounts** section - The entire **initContainers** section The block below shows the code you will look for: ```yaml # volumes: # - name: nginx-etc # emptyDir: {} # - name: nginx-cache # do not set this value in statefulset if volumeclaimtemplate is set # emptyDir: {} # do not set this value in statefulset if volumeclaimtemplate is set # - name: nginx-lib # emptyDir: {} # - name: nginx-lib-state # emptyDir: {} # - name: nginx-log # emptyDir: {} . . . # readOnlyRootFilesystem: true . . . # volumeMounts: # - mountPath: /etc/nginx # name: nginx-etc # - mountPath: /var/cache/nginx # name: nginx-cache # - mountPath: /var/lib/nginx # name: nginx-lib # - mountPath: /var/lib/nginx/state # name: nginx-lib-state # - mountPath: /var/log/nginx # name: nginx-log ``` - Add **waf-enforcer** and **waf-config-mgr** container sections - Add `readOnlyFilesystem: true` in both containers security context sections ### Prometheus If Prometheus metrics are [enabled](/nic/logging-and-monitoring/prometheus.md), we recommend [using HTTPS](/nic/configuration/global-configuration/command-line-arguments.md#cmdoption-prometheus-tls-secret). ### Snippets Snippets allow raw NGINX configuration to be inserted into resources. They are intended for advanced NGINX users and could create vulnerabilities in a cluster if misused. Snippets are disabled by default. To use snippets, set the [**enable-snippets**]() command-line argument. **Note:** Snippets are **always** enabled for ConfigMap. For more information, read the following: - [Advanced configuration using Snippets](/nic/configuration/ingress-resources/advanced-configuration-with-snippets.md) - [Using Snippets with VirtualServer/VirtualServerRoute](/nic/configuration/virtualserver-and-virtualserverroute-resources.md#using-snippets) - [Using Snippets with TransportServer](/nic/configuration/transportserver-resource.md#using-snippets) - [ConfigMap snippets and custom templates](/nic/configuration/global-configuration/configmap-resource.md#snippets-and-custom-templates) -------------------------------------------------------------------------------- title: "Security recommendations" url: https://docs.nginx.com/nginx-ingress-controller/lts/configuration/security/index.md -------------------------------------------------------------------------------- F5 NGINX Ingress Controller LTS follows Kubernetes best practices: this page outlines configuration specific to NGINX Ingress Controller LTS you may require, including links to examples in the [GitHub repository](https://github.com/nginx/kubernetes-ingress/tree/v/examples). For general guidance, we recommend the official Kubernetes documentation for [Securing a Cluster](https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/). ## Kubernetes recommendations ### RBAC and Service Accounts Kubernetes uses [RBAC](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) to control the resources and operations available to different types of users. NGINX Ingress Controller LTS requires RBAC to configure a [ServiceUser](https://kubernetes.io/docs/concepts/security/service-accounts/#default-service-accounts), and provides least privilege access in its standard deployment configurations: - [Helm](https://github.com/nginx/kubernetes-ingress/blob/v/deployments/rbac/rbac.yaml) - [Manifests](https://github.com/nginx/kubernetes-ingress/blob/v/deployments/rbac/rbac.yaml) By default, the ServiceAccount has access to all Secret resources in the cluster. ### Secrets [Secrets](https://kubernetes.io/docs/concepts/configuration/secret/) are required by NGINX Ingress Controller LTS for certificates and privacy keys, which Kubernetes stores unencrypted by default. We recommend following the [Kubernetes documentation](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/) to store these Secrets using at-rest encryption. ## NGINX Ingress Controller LTS recommendations ### Configure root filesystem as read-only NGINX Ingress Controller LTS is designed to be resilient against attacks in various ways, such as running the service as non-root to avoid changes to files. We recommend setting filesystems on `nginx-ingress-controller` container to read-only. This is so that the attack surface is further reduced by limiting changes to binaries and libraries. This is not enabled by default, but can be enabled with **Helm** using the [**readOnlyRootFilesystem**](/nic/lts/install/helm.md#configuration) argument in security contexts. For **Manifests**, uncomment the following sections of the deployment: - `readOnlyRootFilesystem: true` - The entire **volumeMounts** section - The entire **initContainers** section The block below shows the code you will look for: ```yaml # volumes: # - name: nginx-etc # emptyDir: {} # - name: nginx-cache # do not set this value in statefulset if volumeclaimtemplate is set # emptyDir: {} # do not set this value in statefulset if volumeclaimtemplate is set # - name: nginx-lib # emptyDir: {} # - name: nginx-lib-state # emptyDir: {} # - name: nginx-log # emptyDir: {} . . . # readOnlyRootFilesystem: true . . . # volumeMounts: # - mountPath: /etc/nginx # name: nginx-etc # - mountPath: /var/cache/nginx # name: nginx-cache # - mountPath: /var/lib/nginx # name: nginx-lib # - mountPath: /var/lib/nginx/state # name: nginx-lib-state # - mountPath: /var/log/nginx # name: nginx-log ``` ### Prometheus If Prometheus metrics are [enabled](/nic/lts/logging-and-monitoring/prometheus.md), we recommend [using HTTPS](/nic/lts/configuration/global-configuration/command-line-arguments.md#cmdoption-prometheus-tls-secret). ### Snippets Snippets allow raw NGINX configuration to be inserted into resources. They are intended for advanced NGINX users and could create vulnerabilities in a cluster if misused. Snippets are disabled by default. To use snippets, set the [**enable-snippets**]() command-line argument. **caution:** Snippets are **always** enabled for ConfigMap. For more information, read the following: - [Advanced configuration using Snippets](/nic/lts/configuration/ingress-resources/advanced-configuration-with-snippets.md) - [Using Snippets with VirtualServer/VirtualServerRoute](/nic/lts/configuration/virtualserver-and-virtualserverroute-resources.md#using-snippets) - [Using Snippets with TransportServer](/nic/lts/configuration/transportserver-resource.md#using-snippets) - [ConfigMap snippets and custom templates](/nic/lts/configuration/global-configuration/configmap-resource.md#snippets-and-custom-templates) -------------------------------------------------------------------------------- title: "Troubleshooting Ingress resources" url: https://docs.nginx.com/nginx-ingress-controller/lts/troubleshooting/troubleshoot-ingress/index.md -------------------------------------------------------------------------------- This page describes how to troubleshoot NGINX Ingress Controller LTS Policy Resources. ## Ingress resources After you create or update an Ingress resource, you can immediately check if the NGINX configuration for that Ingress resource was successfully applied by NGINX: ```shell kubectl describe ing cafe-ingress ``` ```shell Name: cafe-ingress Namespace: default Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal AddedOrUpdated 12s nginx-ingress-controller Configuration for default/cafe-ingress was added or updated ``` The events section has a *Normal* event with the *AddedOrUpdated reason*, indicating the policy was successfully accepted. -------------------------------------------------------------------------------- title: "Troubleshooting Ingress resources" url: https://docs.nginx.com/nginx-ingress-controller/troubleshooting/troubleshoot-ingress/index.md -------------------------------------------------------------------------------- This page describes how to troubleshoot NGINX Ingress Controller Policy Resources. ## Ingress resources After you create or update an Ingress resource, you can immediately check if the NGINX configuration for that Ingress resource was successfully applied by NGINX: ```shell kubectl describe ing cafe-ingress ``` ```shell Name: cafe-ingress Namespace: default Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal AddedOrUpdated 12s nginx-ingress-controller Configuration for default/cafe-ingress was added or updated ``` The events section has a *Normal* event with the *AddedOrUpdated reason*, indicating the policy was successfully accepted. -------------------------------------------------------------------------------- title: "Troubleshooting Policy resources" url: https://docs.nginx.com/nginx-ingress-controller/lts/troubleshooting/troubleshoot-configmap-policy/index.md -------------------------------------------------------------------------------- This page describes how to troubleshoot NGINX Ingress Controller LTS Policy Resources. ## Policy Resources After you create or update a Policy resource, you can use `kubectl describe` to check whether or not NGINX Ingress Controller LTS accepted the policy: ```shell kubectl describe pol webapp-policy ``` ```shell Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal AddedOrUpdated 11s nginx-ingress-controller Policy default/webapp-policy was added or updated ``` The events section has a *Normal* event with the *AddedOrUpdated reason*, indicating the policy was successfully accepted. However, the fact that a policy was accepted doesn’t guarantee that the NGINX configuration was successfully applied. To verify the configuration applied, check the events of the [VirtualServer and VirtualServerRoute resources](/nic/lts/troubleshooting/troubleshoot-virtualserver.md) that reference the policy. ## ConfigMap Resources After you update the ConfigMap resource, you can immediately check if the configuration was successfully applied by NGINX: ```shell kubectl describe configmap nginx-config -n nginx-ingress Name: nginx-config Namespace: nginx-ingress Labels: Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Updated 11s (x2 over 26m) nginx-ingress-controller Configuration from nginx-ingress/nginx-config was updated ``` Similar to *Policies*, the events section has a *Normal* event with the *AddedOrUpdated reason*, indicating the policy was successfully accepted. -------------------------------------------------------------------------------- title: "Troubleshooting Policy resources" url: https://docs.nginx.com/nginx-ingress-controller/troubleshooting/troubleshoot-configmap-policy/index.md -------------------------------------------------------------------------------- This page describes how to troubleshoot NGINX Ingress Controller Policy Resources. ## Policy Resources After you create or update a Policy resource, you can use `kubectl describe` to check whether or not NGINX Ingress Controller accepted the policy: ```shell kubectl describe pol webapp-policy ``` ```shell Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal AddedOrUpdated 11s nginx-ingress-controller Policy default/webapp-policy was added or updated ``` The events section has a *Normal* event with the *AddedOrUpdated reason*, indicating the policy was successfully accepted. However, the fact that a policy was accepted doesn’t guarantee that the NGINX configuration was successfully applied. To verify the configuration applied, check the events of the [VirtualServer and VirtualServerRoute resources](/nic/troubleshooting/troubleshoot-virtualserver.md) that reference the policy. ## ConfigMap Resources After you update the ConfigMap resource, you can immediately check if the configuration was successfully applied by NGINX: ```shell kubectl describe configmap nginx-config -n nginx-ingress Name: nginx-config Namespace: nginx-ingress Labels: Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Updated 11s (x2 over 26m) nginx-ingress-controller Configuration from nginx-ingress/nginx-config was updated ``` Similar to *Policies*, the events section has a *Normal* event with the *AddedOrUpdated reason*, indicating the policy was successfully accepted. -------------------------------------------------------------------------------- title: "Advanced configuration with Snippets" url: https://docs.nginx.com/nginx-ingress-controller/configuration/ingress-resources/advanced-configuration-with-snippets/index.md -------------------------------------------------------------------------------- Snippets allow you to insert raw NGINX config into different contexts of the NGINX configurations that F5 NGINX Ingress Controller generates. Snippets are intended for advanced NGINX users who need more control over the generated NGINX configuration, and can be used in cases where Annotations and ConfigMap entries would not apply. ## Disadvantages of snippets Snippets are configured [using Annotations](/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md#snippets-and-custom-templates), but are disabled by default due to their complexity. They are also available through the [ConfigMap](/nic/configuration/global-configuration/configmap-resource.md#snippets-and-custom-templates) resource. To use snippets, set the [`enable-snippets`](/nic/configuration/global-configuration/command-line-arguments.md#cmdoption-enable-snippets) command-line argument. Snippets have the following disadvantages: - *Complexity*. Snippets require you to: - Understand NGINX configuration primitives and implement a correct NGINX configuration. - Understand how NGINX Ingress Controller generates NGINX configuration so that a snippet doesn't interfere with the other features in the configuration. - *Decreased robustness*. An incorrect snippet can invalidate NGINX configuration, causing reload failures. Until the snippet is fixed, it will prevent any new configuration updates, including updates for the other Ingress resources. - *Security implications*. Snippets give access to NGINX configuration primitives, which are not validated by NGINX Ingress Controller. For example, a snippet can configure NGINX to serve the TLS certificates and keys used for TLS termination for Ingress resources. **Note:** If the NGINX configuration includes an invalid snippet, NGINX will continue to operate with the last valid configuration. ## Using snippets The example below shows how to use snippets to customize the NGINX configuration template using annotations. ```yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: cafe-ingress-with-snippets annotations: nginx.org/server-snippets: | location / { return 302 /coffee; } nginx.org/location-snippets: | add_header my-test-header test-value; spec: rules: - host: cafe.example.com http: paths: - path: /tea pathType: Prefix backend: service: name: tea-svc port: number: 80 - path: /coffee pathType: Prefix backend: service: name: coffee-svc port: number: 80 ``` These snippets generate the following NGINX configuration: **Note:** The example is shortened for conciseness. ```nginx server { listen 80; location / { return 302 /coffee; } location /coffee { proxy_http_version 1.1; add_header my-test-header test-value; ... proxy_pass http://default-cafe-ingress-with-snippets-cafe.example.com-coffee-svc-80; } location /tea { proxy_http_version 1.1; add_header my-test-header test-value; ... proxy_pass http://default-cafe-ingress-with-snippets-cafe.example.com-tea-svc-80; } } ``` ## Troubleshooting If a snippet includes an invalid NGINX configuration, NGINX Ingress Controller will fail to reload NGINX. The error will be reported in NGINX Ingress Controller logs and an event with the error will be associated with the Ingress resource: An example of an error from the logs: ```text [emerg] 31#31: unknown directive "badd_header" in /etc/nginx/conf.d/default-cafe-ingress-with-snippets.conf:54 Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"cafe-ingress-with-snippets", UID:"f9656dc9-63a6-41dd-a499-525b0e0309bb", APIVersion:"extensions/v1beta1", ResourceVersion:"2322030", FieldPath:""}): type: 'Warning' reason: 'AddedOrUpdatedWithError' Configuration for default/cafe-ingress-with-snippets was added or updated, but not applied: Error reloading NGINX for default/cafe-ingress-with-snippets: nginx reload failed: Command /usr/sbin/nginx -s reload stdout: "" stderr: "nginx: [emerg] unknown directive \"badd_header\" in /etc/nginx/conf.d/default-cafe-ingress-with-snippets.conf:54\n" finished with error: exit status 1 ``` An example of an event with an error (you can view events associated with the Ingress by running `kubectl describe -n nginx-ingress ingress nginx-ingress`): ```text Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal AddedOrUpdated 52m (x3 over 61m) nginx-ingress-controller Configuration for default/cafe-ingress-with-snippets was added or updated finished with error: exit status 1 Warning AddedOrUpdatedWithError 54s (x2 over 89s) nginx-ingress-controller Configuration for default/cafe-ingress-with-snippets was added or updated, but not applied: Error reloading NGINX for default/cafe-ingress-with-snippets: nginx reload failed: Command /usr/sbin/nginx -s reload stdout: "" stderr: "nginx: [emerg] unknown directive \"badd_header\" in /etc/nginx/conf.d/default-cafe-ingress-with-snippets.conf:54\n" finished with error: exit status 1 ``` Additionally, to help troubleshoot snippets, a number of Prometheus metrics show the stats about failed reloads – `controller_nginx_last_reload_status` and `controller_nginx_reload_errors_total`. -------------------------------------------------------------------------------- title: "Advanced configuration with Snippets" url: https://docs.nginx.com/nginx-ingress-controller/lts/configuration/ingress-resources/advanced-configuration-with-snippets/index.md -------------------------------------------------------------------------------- Snippets allow you to insert raw NGINX config into different contexts of the NGINX configurations that F5 NGINX Ingress Controller LTS generates. Snippets are intended for advanced NGINX users who need more control over the generated NGINX configuration, and can be used in cases where Annotations and ConfigMap entries would not apply. ## Disadvantages of snippets Snippets are configured [using Annotations](/nic/lts/configuration/ingress-resources/advanced-configuration-with-annotations.md#snippets-and-custom-templates), but are disabled by default due to their complexity. They are also available through the [ConfigMap](/nic/lts/configuration/global-configuration/configmap-resource.md#snippets-and-custom-templates) resource. To use snippets, set the [`enable-snippets`](/nic/lts/configuration/global-configuration/command-line-arguments.md#cmdoption-enable-snippets) command-line argument. Snippets have the following disadvantages: - *Complexity*. Snippets require you to: - Understand NGINX configuration primitives and implement a correct NGINX configuration. - Understand how NGINX Ingress Controller LTS generates NGINX configuration so that a snippet doesn't interfere with the other features in the configuration. - *Decreased robustness*. An incorrect snippet can invalidate NGINX configuration, causing reload failures. Until the snippet is fixed, it will prevent any new configuration updates, including updates for the other Ingress resources. - *Security implications*. Snippets give access to NGINX configuration primitives, which are not validated by NGINX Ingress Controller LTS. For example, a snippet can configure NGINX to serve the TLS certificates and keys used for TLS termination for Ingress resources. **note:** If the NGINX configuration includes an invalid snippet, NGINX will continue to operate with the last valid configuration. ## Using snippets The example below shows how to use snippets to customize the NGINX configuration template using annotations. ```yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: cafe-ingress-with-snippets annotations: nginx.org/server-snippets: | location / { return 302 /coffee; } nginx.org/location-snippets: | add_header my-test-header test-value; spec: rules: - host: cafe.example.com http: paths: - path: /tea pathType: Prefix backend: service: name: tea-svc port: number: 80 - path: /coffee pathType: Prefix backend: service: name: coffee-svc port: number: 80 ``` These snippets generate the following NGINX configuration: **note:** The example is shortened for conciseness. ```nginx server { listen 80; location / { return 302 /coffee; } location /coffee { proxy_http_version 1.1; add_header my-test-header test-value; ... proxy_pass http://default-cafe-ingress-with-snippets-cafe.example.com-coffee-svc-80; } location /tea { proxy_http_version 1.1; add_header my-test-header test-value; ... proxy_pass http://default-cafe-ingress-with-snippets-cafe.example.com-tea-svc-80; } } ``` ## Troubleshooting If a snippet includes an invalid NGINX configuration, NGINX Ingress Controller LTS will fail to reload NGINX. The error will be reported in NGINX Ingress Controller LTS logs and an event with the error will be associated with the Ingress resource: An example of an error from the logs: ```text [emerg] 31#31: unknown directive "badd_header" in /etc/nginx/conf.d/default-cafe-ingress-with-snippets.conf:54 Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"cafe-ingress-with-snippets", UID:"f9656dc9-63a6-41dd-a499-525b0e0309bb", APIVersion:"extensions/v1beta1", ResourceVersion:"2322030", FieldPath:""}): type: 'Warning' reason: 'AddedOrUpdatedWithError' Configuration for default/cafe-ingress-with-snippets was added or updated, but not applied: Error reloading NGINX for default/cafe-ingress-with-snippets: nginx reload failed: Command /usr/sbin/nginx -s reload stdout: "" stderr: "nginx: [emerg] unknown directive \"badd_header\" in /etc/nginx/conf.d/default-cafe-ingress-with-snippets.conf:54\n" finished with error: exit status 1 ``` An example of an event with an error (you can view events associated with the Ingress by running `kubectl describe -n nginx-ingress ingress nginx-ingress`): ```text Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal AddedOrUpdated 52m (x3 over 61m) nginx-ingress-controller Configuration for default/cafe-ingress-with-snippets was added or updated finished with error: exit status 1 Warning AddedOrUpdatedWithError 54s (x2 over 89s) nginx-ingress-controller Configuration for default/cafe-ingress-with-snippets was added or updated, but not applied: Error reloading NGINX for default/cafe-ingress-with-snippets: nginx reload failed: Command /usr/sbin/nginx -s reload stdout: "" stderr: "nginx: [emerg] unknown directive \"badd_header\" in /etc/nginx/conf.d/default-cafe-ingress-with-snippets.conf:54\n" finished with error: exit status 1 ``` Additionally, to help troubleshoot snippets, a number of Prometheus metrics show the stats about failed reloads – `controller_nginx_last_reload_status` and `controller_nginx_reload_errors_total`. -------------------------------------------------------------------------------- title: "Configuration examples" url: https://docs.nginx.com/nginx-ingress-controller/configuration/configuration-examples/index.md -------------------------------------------------------------------------------- Our [GitHub repo](https://github.com/nginx/kubernetes-ingress) includes a number of configuration examples: - [*Examples of Custom Resources*](https://github.com/nginx/kubernetes-ingress/tree/v/examples/custom-resources) show how to advanced NGINX features by using VirtualServer, VirtualServerRoute, TransportServer and Policy Custom Resources. - [*Examples of Ingress Resources*](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources) show how to use advanced NGINX features in Ingress resources with annotations. -------------------------------------------------------------------------------- title: "Configuration examples" url: https://docs.nginx.com/nginx-ingress-controller/lts/configuration/configuration-examples/index.md -------------------------------------------------------------------------------- Our [GitHub repo](https://github.com/nginx/kubernetes-ingress) includes a number of configuration examples: - [*Examples of Custom Resources*](https://github.com/nginx/kubernetes-ingress/tree/release-2026-lts/examples/custom-resources) show how to advanced NGINX features by using VirtualServer, VirtualServerRoute, TransportServer and Policy Custom Resources. - [*Examples of Ingress Resources*](https://github.com/nginx/kubernetes-ingress/tree/release-2026-lts/examples/ingress-resources) show how to use advanced NGINX features in Ingress resources with annotations. -------------------------------------------------------------------------------- title: "Enable Prometheus metrics" url: https://docs.nginx.com/nginx-ingress-controller/logging-and-monitoring/prometheus/index.md -------------------------------------------------------------------------------- This topic describes how to enable [Prometheus metrics](https://prometheus.io/) for F5 NGINX Ingress Controller. The metrics exposed include NGINX Ingress Controller data, as well as NGINX Open Source and NGINX Plus. ## Enabling Metrics ### Using Helm To enable Prometheus metrics when using *Helm* to install NGINX Ingress Controller, configure the `prometheus.*` parameters of the Helm chart. See the [Installation with Helm](/nic/install/helm.md) topic. #### Using ServiceMonitor When deploying with *Helm*, you can deploy a `Service` and `ServiceMonitor` resource using the `prometheus.service.*` and `prometheus.serviceMonitor.*` parameters. When these resources are deployed, Prometheus metrics exposed by NGINX Ingress Controller can be captured and enumerated using a `Prometheus` resource alongside a Prometheus Operator deployment. To view metrics captured this way, you will need: * A working [Prometheus resource and Prometheus Operator](https://prometheus-operator.dev/docs/getting-started/introduction/) * The latest ServiceMonitor CRD from the [prometheus-operator](https://github.com/prometheus-operator/prometheus-operator) repository: ```shell LATEST=$(curl -s https://api.github.com/repos/prometheus-operator/prometheus-operator/releases/latest | jq -cr .tag_name) curl https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/$LATEST/example/prometheus-operator-crd/monitoring.coreos.com_servicemonitors.yaml | kubectl create -f - ``` ### Using Manifests If you're using *Kubernetes manifests* (Deployment, DaemonSet, or StatefulSet) to install the Ingress Controller, to enable Prometheus metrics: 1. Run the Ingress Controller with the `-enable-prometheus-metrics` [command-line argument](/nic/configuration/global-configuration/command-line-arguments.md). As a result, the Ingress Controller will expose NGINX or NGINX Plus metrics in the Prometheus format via the path `/metrics` on port `9113` (customizable via the `-prometheus-metrics-listen-port` command-line argument). 1. To enable TLS for the Prometheus endpoint, configure the `-prometheus-tls-secret` cli argument with the namespace and name of a TLS Secret. 1. Add the Prometheus port to the list of the ports of the Ingress Controller container in the template of the Ingress Controller pod: ```yaml - name: prometheus containerPort: 9113 ``` 1. Make Prometheus aware of the Ingress Controller targets by adding the following annotations to the template of the Ingress Controller pod (note: this assumes your Prometheus is configured to discover targets by analyzing the annotations of pods): ```yaml annotations: prometheus.io/scrape: "true" prometheus.io/port: "9113" prometheus.io/scheme: http ``` ## Available Metrics The Ingress Controller exports the following metrics: - NGINX/NGINX Plus metrics: - Exported by NGINX/NGINX Plus. Refer to the [NGINX Prometheus Exporter developer docs](https://github.com/nginx/nginx-prometheus-exporter#exported-metrics) to find more information about the exported metrics. - There is a Grafana dashboard for NGINX Plus metrics located in the root repo folder. - Calculated by the Ingress Controller: - `controller_upstream_server_response_latency_ms_count`. Bucketed response times from when NGINX establishes a connection to an upstream server to when the last byte of the response body is received by NGINX. **Note**: The metric for the upstream isn't available until traffic is sent to the upstream. The metric isn't enabled by default. To enable the metric, set the `-enable-latency-metrics` command-line argument. - Ingress Controller metrics - `controller_nginx_reloads_total`. Number of successful NGINX reloads. This includes the label `reason` with 2 possible values `endpoints` (the reason for the reload was an endpoints update) and `other` (the reload was caused by something other than an endpoint update like an ingress update). - `controller_nginx_reload_errors_total`. Number of unsuccessful NGINX reloads. - `controller_nginx_last_reload_status`. Status of the last NGINX reload, 0 meaning down and 1 up. - `controller_nginx_last_reload_milliseconds`. Duration in milliseconds of the last NGINX reload. - `controller_nginx_worker_processes_total`. Number of NGINX worker processes. This metric includes the constant label `generation` with two possible values `old` (the shutting down processes of the old generations) or `current` (the processes of the current generation). - `controller_ingress_resources_total`. Number of handled Ingress resources. This metric includes the label type, that groups the Ingress resources by their type (regular, [minion or master](/nic/configuration/ingress-resources/cross-namespace-configuration.md). **Note**: The metric doesn't count minions without a master. - `controller_virtualserver_resources_total`. Number of handled VirtualServer resources. - `controller_virtualserverroute_resources_total`. Number of handled VirtualServerRoute resources. **Note**: The metric counts only VirtualServerRoutes that have a reference from a VirtualServer. - `location_zone` (upstream services) metrics: - `location_zone_sent`. Number of bytes sent to clients. - `location_zone_received`. Number of bytes received from clients. - `location_zone_requests`. Total number of client requests. - `location_zone_responses`. Total number of responses sent to clients. - `location_zone_responses_codes`. Total number of responses sent to clients. - `location_zone_sent`. Number of bytes sent to clients. - `controller_transportserver_resources_total`. Number of handled TransportServer resources. This metric includes the label type, that groups the TransportServer resources by their type (passthrough, tcp or udp). - Workqueue metrics. **Note**: the workqueue is a queue used by the Ingress Controller to process changes to the relevant resources in the cluster like Ingress resources. The Ingress Controller uses only one queue. The metrics for that queue will have the label `name="taskQueue"` - `workqueue_depth`. Current depth of the workqueue. - `workqueue_queue_duration_second`. How long in seconds an item stays in the workqueue before being requested. - `workqueue_work_duration_seconds`. How long in seconds processing an item from the workqueue takes. **Note**: all metrics have the namespace `nginx_ingress`. For example, `nginx_ingress_controller_nginx_reloads_total`. **Note**: all metrics include the label `class`, which is set to the class of the Ingress Controller. The class is configured via the `-ingress-class` command-line argument. -------------------------------------------------------------------------------- title: "Enable Prometheus metrics" url: https://docs.nginx.com/nginx-ingress-controller/lts/logging-and-monitoring/prometheus/index.md -------------------------------------------------------------------------------- This topic describes how to enable [Prometheus metrics](https://prometheus.io/) for F5 NGINX Ingress Controller LTS. The metrics exposed include NGINX Ingress Controller LTS data and NGINX Plus metrics. ## Enabling Metrics ### Using Helm To enable Prometheus metrics when using *Helm* to install NGINX Ingress Controller LTS, configure the `prometheus.*` parameters of the Helm chart. See the [Installation with Helm](/nic/lts/install/helm.md) topic. #### Using ServiceMonitor When deploying with *Helm*, you can deploy a `Service` and `ServiceMonitor` resource using the `prometheus.service.*` and `prometheus.serviceMonitor.*` parameters. When these resources are deployed, Prometheus metrics exposed by NGINX Ingress Controller LTS can be captured and enumerated using a `Prometheus` resource alongside a Prometheus Operator deployment. To view metrics captured this way, you will need: * A working [Prometheus resource and Prometheus Operator](https://prometheus-operator.dev/docs/getting-started/introduction/) * The latest ServiceMonitor CRD from the [prometheus-operator](https://github.com/prometheus-operator/prometheus-operator) repository: ```shell LATEST=$(curl -s https://api.github.com/repos/prometheus-operator/prometheus-operator/releases/latest | jq -cr .tag_name) curl https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/$LATEST/example/prometheus-operator-crd/monitoring.coreos.com_servicemonitors.yaml | kubectl create -f - ``` ### Using Manifests If you're using *Kubernetes manifests* (Deployment, DaemonSet, or StatefulSet) to install the Ingress Controller, to enable Prometheus metrics: 1. Run the Ingress Controller with the `-enable-prometheus-metrics` [command-line argument](/nic/lts/configuration/global-configuration/command-line-arguments.md). As a result, the Ingress Controller will expose NGINX Plus metrics in the Prometheus format via the path `/metrics` on port `9113` (customizable via the `-prometheus-metrics-listen-port` command-line argument). 1. To enable TLS for the Prometheus endpoint, configure the `-prometheus-tls-secret` cli argument with the namespace and name of a TLS Secret. 1. Add the Prometheus port to the list of the ports of the Ingress Controller container in the template of the Ingress Controller pod: ```yaml - name: prometheus containerPort: 9113 ``` 1. Make Prometheus aware of the Ingress Controller targets by adding the following annotations to the template of the Ingress Controller pod (note: this assumes your Prometheus is configured to discover targets by analyzing the annotations of pods): ```yaml annotations: prometheus.io/scrape: "true" prometheus.io/port: "9113" prometheus.io/scheme: http ``` ## Available Metrics The Ingress Controller exports the following metrics: - NGINX Plus metrics: - Exported by NGINX Plus. Refer to the [NGINX Prometheus Exporter developer docs](https://github.com/nginx/nginx-prometheus-exporter#exported-metrics) to find more information about the exported metrics. - There is a Grafana dashboard for NGINX Plus metrics located in the root repo folder. - Calculated by the Ingress Controller: - `controller_upstream_server_response_latency_ms_count`. Bucketed response times from when NGINX establishes a connection to an upstream server to when the last byte of the response body is received by NGINX. **Note**: The metric for the upstream isn't available until traffic is sent to the upstream. The metric isn't enabled by default. To enable the metric, set the `-enable-latency-metrics` command-line argument. - Ingress Controller metrics - `controller_nginx_reloads_total`. Number of successful NGINX reloads. This includes the label `reason` with 2 possible values `endpoints` (the reason for the reload was an endpoints update) and `other` (the reload was caused by something other than an endpoint update like an ingress update). - `controller_nginx_reload_errors_total`. Number of unsuccessful NGINX reloads. - `controller_nginx_last_reload_status`. Status of the last NGINX reload, 0 meaning down and 1 up. - `controller_nginx_last_reload_milliseconds`. Duration in milliseconds of the last NGINX reload. - `controller_nginx_worker_processes_total`. Number of NGINX worker processes. This metric includes the constant label `generation` with two possible values `old` (the shutting down processes of the old generations) or `current` (the processes of the current generation). - `controller_ingress_resources_total`. Number of handled Ingress resources. This metric includes the label type, that groups the Ingress resources by their type (regular, [minion or master](/nic/lts/configuration/ingress-resources/cross-namespace-configuration.md). **Note**: The metric doesn't count minions without a master. - `controller_virtualserver_resources_total`. Number of handled VirtualServer resources. - `controller_virtualserverroute_resources_total`. Number of handled VirtualServerRoute resources. **Note**: The metric counts only VirtualServerRoutes that have a reference from a VirtualServer. - `location_zone` (upstream services) metrics: - `location_zone_sent`. Number of bytes sent to clients. - `location_zone_received`. Number of bytes received from clients. - `location_zone_requests`. Total number of client requests. - `location_zone_responses`. Total number of responses sent to clients. - `location_zone_responses_codes`. Total number of responses sent to clients. - `location_zone_sent`. Number of bytes sent to clients. - `controller_transportserver_resources_total`. Number of handled TransportServer resources. This metric includes the label type, that groups the TransportServer resources by their type (passthrough, tcp or udp). - Workqueue metrics. **Note**: the workqueue is a queue used by the Ingress Controller to process changes to the relevant resources in the cluster like Ingress resources. The Ingress Controller uses only one queue. The metrics for that queue will have the label `name="taskQueue"` - `workqueue_depth`. Current depth of the workqueue. - `workqueue_queue_duration_second`. How long in seconds an item stays in the workqueue before being requested. - `workqueue_work_duration_seconds`. How long in seconds processing an item from the workqueue takes. **Note**: all metrics have the namespace `nginx_ingress`. For example, `nginx_ingress_controller_nginx_reloads_total`. **Note**: all metrics include the label `class`, which is set to the class of the Ingress Controller. The class is configured via the `-ingress-class` command-line argument. -------------------------------------------------------------------------------- title: "F5 BIG-IP" url: https://docs.nginx.com/nginx-ingress-controller/integrations/f5-ingresslink/index.md -------------------------------------------------------------------------------- Learn how to use F5 IngressLink with NGINX Ingress Controller to configure your F5 BIG-IP device. F5 IngressLink is an integration between NGINX Ingress Controller and [F5 BIG-IP Container Ingress Services](https://clouddocs.f5.com/containers/latest/) (CIS) that configures an F5 BIG-IP device as a load balancer for NGINX Ingress Controller pods. ## Install NGINX Ingress Controller with the integration enabled The steps to enable the integration depend on the option chosen to install NGINX Ingress Controller: Using [Manifests](/nic/install/manifests) or using the [Helm chart](/nic/install/helm). ### Installation using manifests 1. Create a service for the Ingress Controller pods for ports 80 and 443. For example: ```yaml apiVersion: v1 kind: Service metadata: name: nginx-ingress-ingresslink namespace: nginx-ingress labels: app: ingresslink spec: ports: - port: 80 targetPort: 80 protocol: TCP name: http - port: 443 targetPort: 443 protocol: TCP name: https selector: app: nginx-ingress ``` Note the label `app: ingresslink`. We will use it in the [Configure CIS](#configure-cis) step. 1. In the [ConfigMap resource](/nic/configuration/global-configuration/configmap-resource) enable the proxy protocol, which the BIG-IP system will use to pass the client IP and port information to NGINX. For the `set-real-ip-from` key, use the subnet of the IP which the BIG-IP system uses to send traffic to NGINX: ```yaml proxy-protocol: "True" real-ip-header: "proxy_protocol" set-real-ip-from: "0.0.0.0/0" ``` 1. Deploy NGINX Ingress Controller with additional [command-line arguments](/nic/configuration/global-configuration/command-line-arguments): ```yaml args: - -ingresslink=nginx-ingress - -report-ingress-status . . . ``` where `ingresslink` references the name of the IngressLink resource from step 1, and `report-ingress-status` enables [reporting ingress statuses](/nic/configuration/global-configuration/reporting-resources-status#ingress-resources). ### Installation using Helm Install a Helm release with the following values: ```yaml controller: config: entries: proxy-protocol: "True" real-ip-header: "proxy_protocol" set-real-ip-from: "0.0.0.0/0" reportIngressStatus: ingressLink: nginx-ingress service: type: ClusterIP externalTrafficPolicy: Cluster extraLabels: app: ingresslink ``` We will use the `ingressLink` and `extraLabels` parameter values to configure CIS in the next section. For the `set-real-ip-from` key, use the subnet of the IP which the BIG-IP system uses to send traffic to NGINX. ## Configure CIS To enable the integration, F5 BIG-IP Container Ingress Services must be deployed in the cluster and configured to support the integration. Follow the instructions on the [CIS documentation portal](https://clouddocs.f5.com/containers/latest/userguide/ingresslink/#configuring-ingresslink). Make sure that: - The name of the IngressLink resource is the same as the one used during the installation of NGINX Ingress Controller (`nginx-ingress` in the previous example). - The selector in the IngressLink resource is the same as the Service labels configured during Ingress Controller installation (`app: ingresslink` in the previous example). - The IngressLink must belong to the same namespace as the Ingress Controller pod (`nginx-ingress` or the namespace used for installing the Helm chart). -------------------------------------------------------------------------------- title: "F5 BIG-IP" url: https://docs.nginx.com/nginx-ingress-controller/lts/integrations/f5-ingresslink/index.md -------------------------------------------------------------------------------- Learn how to use F5 IngressLink with NGINX Ingress Controller LTS to configure your F5 BIG-IP device. F5 IngressLink is an integration between NGINX Ingress Controller LTS and [F5 BIG-IP Container Ingress Services](https://clouddocs.f5.com/containers/latest/) (CIS) that configures an F5 BIG-IP device as a load balancer for NGINX Ingress Controller LTS pods. ## Install NGINX Ingress Controller LTS with the integration enabled The steps to enable the integration depend on the option chosen to install NGINX Ingress Controller LTS: Using [Manifests](/nic/lts/install/manifests) or using the [Helm chart](/nic/lts/install/helm). ### Installation using manifests 1. Create a service for the Ingress Controller pods for ports 80 and 443. For example: ```yaml apiVersion: v1 kind: Service metadata: name: nginx-ingress-ingresslink namespace: nginx-ingress labels: app: ingresslink spec: ports: - port: 80 targetPort: 80 protocol: TCP name: http - port: 443 targetPort: 443 protocol: TCP name: https selector: app: nginx-ingress ``` Note the label `app: ingresslink`. We will use it in the [Configure CIS](#configure-cis) step. 1. In the [ConfigMap resource](/nic/lts/configuration/global-configuration/configmap-resource) enable the proxy protocol, which the BIG-IP system will use to pass the client IP and port information to NGINX. For the `set-real-ip-from` key, use the subnet of the IP which the BIG-IP system uses to send traffic to NGINX: ```yaml proxy-protocol: "True" real-ip-header: "proxy_protocol" set-real-ip-from: "0.0.0.0/0" ``` 1. Deploy NGINX Ingress Controller LTS with additional [command-line arguments](/nic/lts/configuration/global-configuration/command-line-arguments): ```yaml args: - -ingresslink=nginx-ingress - -report-ingress-status . . . ``` where `ingresslink` references the name of the IngressLink resource from step 1, and `report-ingress-status` enables [reporting ingress statuses](/nic/lts/configuration/global-configuration/reporting-resources-status#ingress-resources). ### Installation using Helm Install a Helm release with the following values: ```yaml controller: config: entries: proxy-protocol: "True" real-ip-header: "proxy_protocol" set-real-ip-from: "0.0.0.0/0" reportIngressStatus: ingressLink: nginx-ingress service: type: ClusterIP externalTrafficPolicy: Cluster extraLabels: app: ingresslink ``` We will use the `ingressLink` and `extraLabels` parameter values to configure CIS in the next section. For the `set-real-ip-from` key, use the subnet of the IP which the BIG-IP system uses to send traffic to NGINX. ## Configure CIS To enable the integration, F5 BIG-IP Container Ingress Services must be deployed in the cluster and configured to support the integration. Follow the instructions on the [CIS documentation portal](https://clouddocs.f5.com/containers/latest/userguide/ingresslink/#configuring-ingresslink). Make sure that: - The name of the IngressLink resource is the same as the one used during the installation of NGINX Ingress Controller LTS (`nginx-ingress` in the previous example). - The selector in the IngressLink resource is the same as the Service labels configured during Ingress Controller installation (`app: ingresslink` in the previous example). - The IngressLink must belong to the same namespace as the Ingress Controller pod (`nginx-ingress` or the namespace used for installing the Helm chart). -------------------------------------------------------------------------------- title: "NGINX Ingress Controller Helm chart parameters" url: https://docs.nginx.com/nginx-ingress-controller/install/helm/parameters/index.md -------------------------------------------------------------------------------- This page describes the Helm chart parameters of F5 NGINX Ingress Controllers. It is a reference for each parameter name, a description of the parameter, and any default values. ## NGINX Ingress Controller Helm chart parameters **Note:** The [values.yaml](https://github.com/nginx/kubernetes-ingress/blob/main/charts/nginx-ingress/values.yaml) file of the Helm chart is a single source of truth and includes parameter descriptions: the table may become outdated. The [values.schema.json](https://github.com/nginx/kubernetes-ingress/blob/main/charts/nginx-ingress/values.schema.json) file can be used to inspect how values are formatted. | Parameter | Description | Default | | ----------| ----------- | ------- | | **controller.name** | The name of the NGINX Ingress Controller deployment, daemonset, or statefulset. | Autogenerated | | **controller.kind** | The kind of the NGINX Ingress Controller installation - deployment, daemonset, or statefulset. | deployment | | **controller.annotations** | Allows for setting of `annotations` for deployment, daemonset, or statefulset. | {} | | **controller.nginxplus** | Deploys NGINX Ingress Controller for NGINX Plus. | false | | **controller.mgmt.licenseTokenSecretName** | Configures the secret used in the [license_token](https://nginx.org/en/docs/ngx_mgmt_module.html#license_token) directive. This key assumes the secret is in the Namespace that NGINX Ingress Controller is deployed in. The secret must be of type `nginx.com/license` with the base64 encoded JWT in the `license.jwt` key. | license-token | | **controller.mgmt.enforceInitialReport** | Configures the [enforce_initial_report](https://nginx.org/en/docs/ngx_mgmt_module.html#enforce_initial_report) directive, which enables or disables the 180-day grace period for sending the initial usage report. | false | | **controller.mgmt.usageReport.endpoint** | Configures the endpoint of the [usage_report](https://nginx.org/en/docs/ngx_mgmt_module.html#usage_report) directive. This is used to configure the endpoint NGINX uses to send usage reports to NGINX Instance Manager . | product.connect.nginx.com | | **controller.mgmt.usageReport.interval** | Configures the interval of the [usage_report](https://nginx.org/en/docs/ngx_mgmt_module.html#usage_report) directive. This specifies the frequency that usage reports are sent. Only seconds(s), minutes(m), and hours(h) are allowed and must be between 60s and 24h. | 1h | | **controller.mgmt.usageReport.proxyHost** | Configures the host name of the [proxy](https://nginx.org/en/docs/ngx_mgmt_module.html#proxy) directive with optional port. | N/A | | **controller.mgmt.usageReport.proxyCredentialsSecretName** | Configures the [proxy_username](https://nginx.org/en/docs/ngx_mgmt_module.html#proxy_username) directive as well as the [proxy_password](https://nginx.org/en/docs/ngx_mgmt_module.html#proxy_password) directive using a Kubernetes Opaque Secret. The Secret must contain `username` and `password` fields. | N/A | | **controller.mgmt.sslVerify** | Configures the [ssl_verify](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_verify) directive, which enables or disables verification of the usage reporting endpoint certificate. | true | | **controller.mgmt.resolver.ipv6** | Configures whether the mgmt block [resolver](https://nginx.org/en/docs/ngx_mgmt_module.html#resolver) directive will look up IPv6 addresses. | true | | **controller.mgmt.resolver.valid** | Configures an [NGINX time](https://nginx.org/en/docs/syntax.html) that the mgmt block [resolver](https://nginx.org/en/docs/ngx_mgmt_module.html#resolver) directive will override the TTL value of responses from nameservers with. | N/A | | **controller.mgmt.resolver.addresses** | Configures addresses used in the mgmt block [resolver](https://nginx.org/en/docs/ngx_mgmt_module.html#resolver) directive. This field takes a list of addresses. | N/A | | **controller.mgmt.sslCertificateSecretName** | Configures the secret used to create the `ssl_certificate` and `ssl_certificate_key` directives. This key assumes the secret is in the Namespace that NGINX Ingress Controller is deployed in. The secret must be of type `kubernetes.io/tls` | N/A | | **controller.mgmt.sslTrustedCertificateSecretName** | Configures the secret used to create the file(s) referenced the in [ssl_trusted_certifcate](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_trusted_certificate), and [ssl_crl](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_crl) directives. This key assumes the secret is in the Namespace that NGINX Ingress Controller is deployed in. The secret must be of type `nginx.org/ca`, where the `ca.crt` key contains a base64 encoded trusted cert, and the optional `ca.crl` key can contain a base64 encoded CRL. If the optional `ca.crl` key is supplied, it will configure the NGINX `ssl_crl` directive. | N/A | | **controller.mgmt.configMapName** | Allows changing the name of the MGMT config map. The name should not include a namespace| Autogenerated | | **controller.nginxReloadTimeout** | The timeout in milliseconds which the NGINX Ingress Controller will wait for a successful NGINX reload after a change or at the initial start. | 60000 | | **controller.hostNetwork** | Enables NGINX Ingress Controller pods to use the host's network namespace. | false | | **controller.dnsPolicy** | DNS policy for NGINX Ingress Controller pods. | ClusterFirst | | **controller.nginxDebug** | Enables debugging for NGINX. Uses the `nginx-debug` binary. Requires `error-log-level: debug` in the ConfigMap via `controller.config.entries`. | false | | **controller.enableConfigSafety** | Enhances the stability and reliability of the NGINX Ingress Controller. When this feature is turned on, it ensures the validation of new configurations before reloading nginx. If a configuration is deemed invalid, the new config for the associated resource is rejected and the old working config is restored. This is an experimental feature. Behavior and configuration may change in future releases. When turned on, users can experience delayed pod startup times as resources are validated prior to being written. Delays are directly proportional to the number and complexity of resources. | false | | **controller.logLevel** | The log level of NGINX Ingress Controller. | info | | **controller.logFormat** | The log format of NGINX Ingress Controller. | glog | | **controller.directiveAutoAdjust** | Automatically adjusts NGINX buffer directives to prevent configuration errors. | false | | **controller.image.digest** | The image digest of NGINX Ingress Controller. | None | | **controller.image.repository** | The image repository of NGINX Ingress Controller. | nginx/nginx-ingress | | **controller.image.tag** | The tag of the NGINX Ingress Controller image. | | | **controller.image.pullPolicy** | The pull policy for the NGINX Ingress Controller image. | IfNotPresent | | **controller.lifecycle** | The lifecycle of the NGINX Ingress Controller pods. | {} | | **controller.customConfigMap** | The name of the custom ConfigMap used by the NGINX Ingress Controller. If set, then the default config is ignored. | "" | | **controller.config.name** | The name of the ConfigMap used by the NGINX Ingress Controller. | Autogenerated | | **controller.config.annotations** | The annotations of the NGINX Ingress Controller configmap. | {} | | **controller.config.entries** | The entries of the ConfigMap for customizing NGINX configuration. See [ConfigMap resource docs](/nic/configuration/global-configuration/configmap-resource.md) for the list of supported ConfigMap keys. | {} | | **controller.customPorts** | A list of custom ports to expose on the NGINX Ingress Controller pod. Follows the conventional Kubernetes yaml syntax for container ports. | [] | | **controller.defaultTLS.cert** | The base64-encoded TLS certificate for the default HTTPS server. **Note:** It is recommended that you specify your own certificate. Alternatively, omitting the default server secret completely will configure NGINX to reject TLS connections to the default server. | | **controller.defaultTLS.key** | The base64-encoded TLS key for the default HTTPS server. **Note:** It is recommended that you specify your own key. Alternatively, omitting the default server secret completely will configure NGINX to reject TLS connections to the default server. | | **controller.defaultTLS.secret** | The secret with a TLS certificate and key for the default HTTPS server. The value must follow the following format: `/`. Used as an alternative to specifying a certificate and key using `controller.defaultTLS.cert` and `controller.defaultTLS.key` parameters. **Note:** Alternatively, omitting the default server secret completely will configure NGINX to reject TLS connections to the default server. When `controller.allowEmptyIngressHost` is enabled, this secret also provides TLS for Ingress resources without a `host`. | None | | **controller.wildcardTLS.cert** | The base64-encoded TLS certificate for every Ingress/VirtualServer host that has TLS enabled but no secret specified. If the parameter is not set, for such Ingress/VirtualServer hosts NGINX will break any attempt to establish a TLS connection. | None | | **controller.wildcardTLS.key** | The base64-encoded TLS key for every Ingress/VirtualServer host that has TLS enabled but no secret specified. If the parameter is not set, for such Ingress/VirtualServer hosts NGINX will break any attempt to establish a TLS connection. | None | | **controller.wildcardTLS.secret** | The secret with a TLS certificate and key for every Ingress/VirtualServer host that has TLS enabled but no secret specified. The value must follow the following format: `/`. Used as an alternative to specifying a certificate and key using `controller.wildcardTLS.cert` and `controller.wildcardTLS.key` parameters. | None | | **controller.nodeSelector** | The node selector for pod assignment for NGINX Ingress Controller pods. | {} | | **controller.terminationGracePeriodSeconds** | The termination grace period of the NGINX Ingress Controller pod. | 30 | | **controller.tolerations** | The tolerations of NGINX Ingress Controller pods. | [] | | **controller.affinity** | The affinity of NGINX Ingress Controller pods. | {} | | **controller.topologySpreadConstraints** | The topology spread constraints of the NGINX Ingress Controller pods. | {} | | **controller.env** | The additional environment variables to be set on the NGINX Ingress Controller pods. | [] | | **controller.volumes** | The volumes of NGINX Ingress Controller pods. | [] | | **controller.volumeMounts** | The volumeMounts of NGINX Ingress Controller pods. | [] | | **controller.initContainers** | InitContainers for NGINX Ingress Controller pods. | [] | | **controller.extraContainers** | Extra (eg. sidecar) containers for NGINX Ingress Controller pods. | [] | | **controller.podSecurityContext**| The SecurityContext for Ingress Controller pods. | "seccompProfile": {"type": "RuntimeDefault"} | | **controller.securityContext** | The SecurityContext for Ingress Controller container. | {} | | **controller.initContainerSecurityContext** | The SecurityContext for NGINX Ingress Controller init container when `readOnlyRootFilesystem` is enabled by either setting `controller.securityContext.readOnlyRootFilesystem` or `controller.readOnlyRootFilesystem`to `true`. | {} | | **controller.resources** | The resources of NGINX Ingress Controller pods. | requests: cpu=100m,memory=128Mi | | **controller.initContainerResources** | The resources of the init container which is used when `readOnlyRootFilesystem` is enabled by either setting `controller.securityContext.readOnlyRootFilesystem` or `controller.readOnlyRootFilesystem`to `true`. | requests: cpu=100m,memory=128Mi | | **controller.replicaCount** | The number of replicas of the NGINX Ingress Controller deployment. | 1 | | **controller.ingressClass.name** | A class of NGINX Ingress Controller. An IngressClass resource with the name equal to the class must be deployed. Otherwise, the NGINX Ingress Controller will fail to start. the NGINX Ingress Controller only processes resources that belong to its class - i.e. have the "ingressClassName" field resource equal to the class. the NGINX Ingress Controller processes all the VirtualServer/VirtualServerRoute/TransportServer resources that do not have the "ingressClassName" field for all versions of Kubernetes. | nginx | | **controller.ingressClass.create** | Creates a new IngressClass object with the name `controller.ingressClass.name`. Set to `false` to use an existing ingressClass created using `kubectl` with the same name. If you use `helm upgrade`, do not change the values from the previous release as helm will delete IngressClass objects managed by helm. If you are upgrading from a release earlier than , do not set the value to false. | true | | **controller.ingressClass.setAsDefaultIngress** | New Ingresses without an `"ingressClassName"` field specified will be assigned the class specified in `controller.ingressClass.name`. Requires `controller.ingressClass.create`. | false | | **controller.watchNamespace** | Comma separated list of namespaces the NGINX Ingress Controller should watch for resources. By default the NGINX Ingress Controller watches all namespaces. Mutually exclusive with `controller.watchNamespaceLabel`. Please note that if configuring multiple namespaces using the Helm cli `--set` option, the string needs to wrapped in double quotes and the commas escaped using a backslash - e.g. `--set controller.watchNamespace="default\,nginx-ingress"`. | "" | | **controller.watchNamespaceLabel** | Configures NGINX Ingress Controller to watch only those namespaces with label foo=bar. By default the NGINX Ingress Controller watches all namespaces. Mutually exclusive with `controller.watchNamespace`. | "" | | **controller.watchSecretNamespace** | Comma separated list of namespaces the NGINX Ingress Controller should watch for resources of type Secret. If this arg is not configured, the NGINX Ingress Controller watches the same namespaces for all resources, see `controller.watchNamespace` and `controller.watchNamespaceLabel`. All namespaces included with this argument must be part of either `controller.watchNamespace` or `controller.watchNamespaceLabel`. Please note that if configuring multiple namespaces using the Helm cli `--set` option, the string needs to wrapped in double quotes and the commas escaped using a backslash - e.g. `--set controller.watchSecretNamespace="default\,nginx-ingress"`. | "" | | **controller.enableCustomResources** | Enable the custom resources. | true | | **controller.enableOIDC** | Enable OIDC policies. | false | | **controller.enableTLSPassthrough** | Enable TLS Passthrough on default port 443. Requires `controller.enableCustomResources`. | false | | **controller.tlsPassThroughPort** | Set the port for the TLS Passthrough. Requires `controller.enableCustomResources` and `controller.enableTLSPassthrough`. | 443 | | **controller.enableCertManager** | Enable x509 automated certificate management for VirtualServer resources using cert-manager (cert-manager.io). Requires `controller.enableCustomResources`. | false | | **controller.enableExternalDNS** | Enable integration with ExternalDNS for configuring public DNS entries for VirtualServer resources using [ExternalDNS](https://github.com/kubernetes-sigs/external-dns). Requires `controller.enableCustomResources`. | false | | **controller.globalConfiguration.create** | Creates the GlobalConfiguration custom resource. Requires `controller.enableCustomResources`. | false | | **controller.globalConfiguration.spec** | The spec of the GlobalConfiguration for defining the global configuration parameters of the NGINX Ingress Controller. | {} | | **controller.enableSnippets** | Enable custom NGINX configuration snippets in Ingress, VirtualServer, VirtualServerRoute and TransportServer resources. | false | | **controller.healthStatus** | Add a location "/nginx-health" to the default server. The location responds with the 200 status code for any request. Useful for external health-checking of the NGINX Ingress Controller. | false | | **controller.healthStatusURI** | Sets the URI of health status location in the default server. Requires `controller.healthStatus`. | "/nginx-health" | | **controller.nginxStatus.enable** | Enable the NGINX stub_status, or the NGINX Plus API. | true | | **controller.nginxStatus.port** | Set the port where the NGINX stub_status or the NGINX Plus API is exposed. | 8080 | | **controller.nginxStatus.allowCidrs** | Add IP/CIDR blocks to the allow list for NGINX stub_status or the NGINX Plus API. Separate multiple IP/CIDR by commas. | 127.0.0.1,::1 | | **controller.priorityClassName** | The PriorityClass of NGINX Ingress Controller pods. | None | | **controller.service.create** | Creates a service to expose NGINX Ingress Controller pods. | true | | **controller.service.type** | The type of service to create for NGINX Ingress Controller. | LoadBalancer | | **controller.service.externalTrafficPolicy** | The externalTrafficPolicy of the service. The value Local preserves the client source IP. | Local | | **controller.service.annotations** | The annotations of the NGINX Ingress Controller service. | {} | | **controller.service.extraLabels** | The extra labels of the service. | {} | | **controller.service.loadBalancerIP** | The static IP address for the load balancer. Requires `controller.service.type` set to `LoadBalancer`. The cloud provider must support this feature. | "" | | **controller.service.loadBalancerClass** | This property allows the user to define a specific load balancer implementation or configuration class for the service. Requires `controller.service.type` set to `LoadBalancer`. The cloud provider must support this feature. | "" | | **controller.service.externalIPs** | The list of external IPs for the NGINX Ingress Controller service. | [] | | **controller.service.clusterIP** | The clusterIP for the NGINX Ingress Controller service, autoassigned if not specified. | "" | | **controller.service.loadBalancerSourceRanges** | The IP ranges (CIDR) that are allowed to access the load balancer. Requires `controller.service.type` set to `LoadBalancer`. The cloud provider must support this feature. | [] | | **controller.service.name** | The name of the service. | Autogenerated | | **controller.service.customPorts** | A list of custom ports to expose through the NGINX Ingress Controller service. Follows the conventional Kubernetes yaml syntax for service ports. | [] | | **controller.service.httpPort.enable** | Enables the HTTP port for the NGINX Ingress Controller service. | true | | **controller.service.httpPort.port** | The HTTP port of the NGINX Ingress Controller service. | 80 | | **controller.service.httpPort.nodePort** | The custom NodePort for the HTTP port. Requires `controller.service.type` set to `NodePort`. | "" | | **controller.service.httpPort.targetPort** | The target port of the HTTP port of the NGINX Ingress Controller service. | 80 | | **controller.service.httpsPort.enable** | Enables the HTTPS port for the NGINX Ingress Controller service. | true | | **controller.service.httpsPort.port** | The HTTPS port of the NGINX Ingress Controller service. | 443 | | **controller.service.httpsPort.nodePort** | The custom NodePort for the HTTPS port. Requires `controller.service.type` set to `NodePort`. | "" | | **controller.service.httpsPort.targetPort** | The target port of the HTTPS port of the NGINX Ingress Controller service. | 443 | | **controller.service.sessionAffinity.enable** | Enable session affinity. | `false` | | **controller.service.sessionAffinity.type** | Session affinity type. Valid values: `None`, `ClientIP`. | `ClientIP` | | **controller.service.sessionAffinity.timeout** | Session affinity timeout in seconds. | 3600 | | **controller.serviceAccount.annotations** | The annotations of the NGINX Ingress Controller service account. | {} | | **controller.serviceAccount.name** | The name of the service account of the NGINX Ingress Controller pods. Used for RBAC. | Autogenerated | | **controller.serviceAccount.imagePullSecretName** | The name of the secret containing docker registry credentials. Secret must exist in the same namespace as the helm release. | "" | | **controller.serviceAccount.imagePullSecretsNames** | The list of secret names containing docker registry credentials. Secret must exist in the same namespace as the helm release. | [] | | **controller.reportIngressStatus.enable** | Updates the address field in the status of Ingress resources with an external address of the NGINX Ingress Controller. You must also specify the source of the external address either through an external service via `controller.reportIngressStatus.externalService`, `controller.reportIngressStatus.ingressLink` or the `external-status-address` entry in the ConfigMap via `controller.config.entries`. **Note:** `controller.config.entries.external-status-address` takes precedence over the others. | true | | **controller.reportIngressStatus.externalService** | Specifies the name of the service with the type LoadBalancer through which the NGINX Ingress Controller is exposed externally. The external address of the service is used when reporting the status of Ingress, VirtualServer and VirtualServerRoute resources. `controller.reportIngressStatus.enable` must be set to `true`. The default is autogenerated and enabled when `controller.service.create` is set to `true` and `controller.service.type` is set to `LoadBalancer`. | Autogenerated | | **controller.reportIngressStatus.ingressLink** | Specifies the name of the IngressLink resource, which exposes the NGINX Ingress Controller pods via a BIG-IP system. The IP of the BIG-IP system is used when reporting the status of Ingress, VirtualServer and VirtualServerRoute resources. `controller.reportIngressStatus.enable` must be set to `true`. | "" | | **controller.reportIngressStatus.enableLeaderElection** | Enable Leader election to avoid multiple replicas of the controller reporting the status of Ingress resources. `controller.reportIngressStatus.enable` must be set to `true`. | true | | **controller.reportIngressStatus.leaderElectionLockName** | Specifies the name of the ConfigMap, within the same namespace as the controller, used as the lock for leader election. controller.reportIngressStatus.enableLeaderElection must be set to true. | Autogenerated | | **controller.reportIngressStatus.annotations** | The annotations of the leader election configmap. | {} | | **controller.pod.annotations** | The annotations of the NGINX Ingress Controller pod. | {} | | **controller.pod.extraLabels** | The additional extra labels of the NGINX Ingress Controller pod. | {} | | **controller.appprotect.enable** | Enables the F5 WAF for NGINX module in the NGINX Ingress Controller. | false | | **controller.appprotect.v5** | Enables F5 WAF for NGINX v5. | false | | **controller.appprotect.plmStorage.url** | SeaweedFS S3 endpoint from which NGINX Ingress Controller fetches the policy and logconf bundle. Leave empty to turn off PLM support. Requires `controller.appprotect.v5` to be `true`. | "" | | **controller.appprotect.plmStorage.credentialsSecret** | Secret containing the SeaweedFS admin secret in the `seaweedfs_admin_secret` key. Format: `/`. | "" | | **controller.appprotect.plmStorage.caSecret** | Optional Secret containing the SeaweedFS admin secret under `seaweedfs_admin_secret`. Format: `/`. | "" | | **controller.appprotect.plmStorage.clientSSLSecret** | Optional Secret containing `tls.crt` and `tls.key` for SeaweedFS mTLS. Format: `/`. | "" | | **controller.appprotect.plmStorage.insecureSkipVerify** | Turns off SeaweedFS TLS verification. For development and testing only. | false | | **controller.appprotect.volumes** | Volumes for F5 WAF for NGINX v5. | [{"name": "app-protect-bd-config", "emptyDir": {}},{"name": "app-protect-config", "emptyDir": {}},{"name": "app-protect-bundles", "emptyDir": {}}] | | **controller.appprotect.enforcer.host** | Host that the F5 WAF for NGINX v5 Enforcer runs on. | "127.0.0.1" | | **controller.appprotect.enforcer.port** | Port that the F5 WAF for NGINX v5 Enforcer runs on. | 50000 | | **controller.appprotect.enforcer.image.repository** | The image repository of the F5 WAF for NGINX v5 Enforcer. | private-registry.nginx.com/nap/waf-enforcer | | **controller.appprotect.enforcer.image.tag** | The tag of the F5 WAF for NGINX v5 Enforcer. | "5.6.0" | | **controller.appprotect.enforcer.image.digest** | The digest of the F5 WAF for NGINX v5 Enforcer. Takes precedence over tag if set. | "" | | **controller.appprotect.enforcer.image.pullPolicy** | The pull policy for the F5 WAF for NGINX v5 Enforcer image. | IfNotPresent | | **controller.appprotect.enforcer.securityContext** | The security context for F5 WAF for NGINX v5 Enforcer container. | {} | | **controller.appprotect.configManager.image.repository** | The image repository of the F5 WAF for NGINX v5 Configuration Manager. | private-registry.nginx.com/nap/waf-config-mgr | | **controller.appprotect.configManager.image.tag** | The tag of the F5 WAF for NGINX v5 Configuration Manager. | "5.6.0" | | **controller.appprotect.configManager.image.digest** | The digest of the F5 WAF for NGINX v5 Configuration Manager. Takes precedence over tag if set. | "" | | **controller.appprotect.configManager.image.pullPolicy** | The pull policy for the F5 WAF for NGINX v5 Configuration Manager image. | IfNotPresent | | **controller.appprotect.configManager.securityContext** | The security context for F5 WAF for NGINX v5 Configuration Manager container. | {"allowPrivilegeEscalation":false,"runAsUser":101,"runAsNonRoot":true,"capabilities":{"drop":["all"]}} | | **controller.appprotect.logLevel** | Sets the log level for F5 WAF for NGINX | N/A | | **controller.appprotectdos.enable** | Enables the App Protect DoS module in the NGINX Ingress Controller. | false | | **controller.appprotectdos.debug** | Enable debugging for App Protect DoS. | false | | **controller.appprotectdos.maxDaemons** | Max number of ADMD instances. | 1 | | **controller.appprotectdos.maxWorkers** | Max number of nginx processes to support. | Number of CPU cores in the machine | | **controller.appprotectdos.memory** | RAM memory size to consume in MB. | 50% of free RAM in the container or 80MB, the smaller | | **controller.readyStatus.enable** | Enables the readiness endpoint `"/nginx-ready"`. The endpoint returns a success code when NGINX has loaded all the config after the startup. This also configures a readiness probe for the NGINX Ingress Controller pods that uses the readiness endpoint. | true | | **controller.readyStatus.port** | The HTTP port for the readiness endpoint. | 8081 | | **controller.readyStatus.initialDelaySeconds** | The number of seconds after the NGINX Ingress Controller pod has started before readiness probes are initiated. | 0 | | **controller.startupStatus.enable** | Enables the startup probe for the Ingress Controller. | false | | **controller.startupStatus.port** | The port where the startup endpoint is exposed. This is a required field if `controller.startupStatus.enable` is set to true. | N/A | | **controller.startupStatus.path** | The path to the startup endpoint. This is a required field if `controller.startupStatus.enable` is set to true. | N/A | | **controller.startupStatus.initialDelaySeconds** | The number of seconds after the Ingress Controller pod has started before startup probes are initiated. | N/A | | **controller.startupStatus.periodSeconds** | The number of seconds between each startup probe. | N/A | | **controller.startupStatus.timeoutSeconds** | The number of seconds after which the startup probe times out. | N/A | | **controller.startupStatus.successThreshold** | Minimum consecutive successes for the probe to be considered successful. | N/A | | **controller.startupStatus.failureThreshold** | When a probe fails, Kubernetes will try failureThreshold times before giving up. | N/A | | **controller.enableLatencyMetrics** | Enable collection of latency metrics for upstreams. Requires `prometheus.create`. | false | | **controller.minReadySeconds** | Specifies the minimum number of seconds for which a newly created Pod should be ready without any of its containers crashing, for it to be considered available. [docs](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#min-ready-seconds) | 0 | | **controller.autoscaling.enabled** | Enables HorizontalPodAutoscaling. | false | | **controller.autoscaling.create** | Create the HorizontalPodAutoscaler resource. This can be set to false to manage the HPA externally. | true | | **controller.autoscaling.annotations** | The annotations of the NGINX Ingress Controller HorizontalPodAutoscaler. | {} | | **controller.autoscaling.behavior** | Behavior configuration for the HPA. | {} | | **controller.autoscaling.minReplicas** | Minimum number of replicas for the HPA. | 1 | | **controller.autoscaling.maxReplicas** | Maximum number of replicas for the HPA. | 3 | | **controller.autoscaling.targetCPUUtilizationPercentage** | The target CPU utilization percentage. | 50 | | **controller.autoscaling.targetMemoryUtilizationPercentage** | The target memory utilization percentage. | 50 | | **controller.podDisruptionBudget.enabled** | Enables PodDisruptionBudget. | false | | **controller.podDisruptionBudget.annotations** | The annotations of the NGINX Ingress Controller pod disruption budget | {} | | **controller.podDisruptionBudget.minAvailable** | The number of Ingress Controller pods that should be available. This is a mutually exclusive setting with "maxUnavailable". | 0 | | **controller.podDisruptionBudget.maxUnavailable** | The number of Ingress Controller pods that can be unavailable. This is a mutually exclusive setting with "minAvailable". | 0 | | **controller.strategy** | Specifies the strategy used to replace old Pods with new ones. Docs for [Deployment update strategy](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#strategy), [Daemonset update strategy](https://kubernetes.io/docs/tasks/manage-daemon/update-daemon-set/#daemonset-update-strategy) and [StatefulSet update strategy](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#update-strategies) | {} | | **controller.statefulset.podManagementPolicy** | Controls how pods are created during initial scale up, when replacing pods on nodes, or when scaling down. Available options: `OrderedReady` or `Parallel`. Only applies when `controller.kind` is set to `statefulset`. | `OrderedReady` | | **controller.statefulset.persistentVolumeClaimRetentionPolicy.whenDeleted** | Controls the retention policy for PVCs when the StatefulSet is deleted. Available options: `Retain` or `Delete`. Only applies when `controller.kind` is set to `statefulset`. | `Retain` | | **controller.statefulset.persistentVolumeClaimRetentionPolicy.whenScaled** | Controls the retention policy for PVCs when the StatefulSet is scaled down. Available options: `Retain` or `Delete`. Only applies when `controller.kind` is set to `statefulset`. | `Retain` | | **controller.statefulset.nginxCachePVC.size** | The size of the persistent volume claim for NGINX cache storage. Only applies when `controller.kind` is set to `statefulset`. | `256Mi` | | **controller.statefulset.nginxCachePVC.storageClass** | The storage class for the persistent volume claim. Only applies when `controller.kind` is set to `statefulset`. | `""` | | **controller.statefulset.nginxCachePVC.accessModes** | The access modes for the persistent volume claim. Only applies when `controller.kind` is set to `statefulset`. | `["ReadWriteOnce"]` | | **controller.disableIPV6** | Disable IPV6 listeners explicitly for nodes that do not support the IPV6 stack. | false | | **controller.defaultHTTPListenerPort** | Sets the port for the HTTP `default_server` listener. When `controller.allowEmptyIngressHost` is enabled, also configures the HTTP listener port for Ingress resources without a `host`. | 80 | | **controller.defaultHTTPSListenerPort** | Sets the port for the HTTPS `default_server` listener. When `controller.allowEmptyIngressHost` is enabled, also configures the HTTPS listener port for Ingress resources without a `host`. | 443 | | **controller.allowEmptyIngressHost** | Allows Ingress resources to omit the `host` field. `spec.tls` is not supported on Ingress resources without a `host`; use `controller.defaultTLS.secret` instead. `nginx.org/listen-ports` and `nginx.org/listen-ports-ssl` annotations are not supported; use `controller.defaultHTTPListenerPort` and `controller.defaultHTTPSListenerPort` instead. | false | | **controller.readOnlyRootFilesystem** | Configure root filesystem as read-only and add volumes for temporary data. Three major releases after 3.5.x this argument will be moved permanently to the `controller.securityContext` section. | false | | **controller.enableSSLDynamicReload** | Enable lazy loading for SSL Certificates. | true | | **controller.telemetryReporting.enable** | Enable telemetry reporting. | true | | **controller.enableWeightChangesDynamicReload** | Enable weight changes without reloading the NGINX configuration. May require increasing `map_hash_bucket_size`, `map_hash_max_size`, `variable_hash_bucket_size`, and `variable_hash_max_size` in the [ConfigMap](/nic/configuration/global-configuration/configmap-resource.md) if there are many two-way splits. Requires `controller.nginxplus` | false | | **controller.networkPolicy.enabled** | Create a NetworkPolicy that targets the controller pods. If `enabled` is set to `true` and both ingress and egress are empty, the resulting NetworkPolicy denies all ingress by default. | false | | **controller.networkPolicy.ingress** | Configure ingress rules. When non-empty, the Ingress policy type is added automatically. | [] | | **controller.networkPolicy.egress** | Configure egress rules. When non-empty, the Egress policy type is added automatically. | [] | |**nginxAgent.enable** | Enable NGINX Agent 3.x to allow [connecting to NGINX One Console](/nginx-one-console/k8s/add-nic.md) or to integrate NGINX Agent 2.x for [Security Monitoring](/nic/tutorials/security-monitoring.md) . | false | |**nginxAgent.logLevel** | Log level for NGINX Agent. | "error" | |**nginxAgent.dataplaneKeySecretName** | Name of the Kubernetes Secret containing the Data Plane key used to authenticate to NGINX One Console. Learn more [here](/nginx-one-console/k8s/add-nic.md). Required when `nginxAgent.enable` is set to `true`. Requires NGINX Agent 3.x. | "" | |**nginxAgent.endpointHost** | Domain or IP address for the NGINX One Console. Requires NGINX Agent 3.x. | "agent.connect.nginx.com" | |**nginxAgent.endpointPort** | Port for the NGINX One Console endpoint. Requires NGINX Agent 3.x. | 443 | |**nginxAgent.tlsSkipVerify** | Skip TLS verification for the NGINX One Console endpoint. Requires NGINX Agent 3.x. | false | |**nginxAgent.instanceGroup** | Set a custom Instance Group name for the deployment, shown when connected to NGINX Instance Manager. `nginx-ingress.controller.fullname` will be used if not set. Requires NGINX Agent 2.x. | "" | |**nginxAgent.instanceManager.host** | FQDN or IP for connecting to NGINX Ingress Controller. Required when `nginxAgent.enable` is set to `true`. Requires NGINX Agent 2.x. | "" | |**nginxAgent.instanceManager.grpcPort** | Port for connecting to NGINX Ingress Controller. Requires NGINX Agent 2.x. | 443 | |**nginxAgent.instanceManager.sni** | Server Name Indication for Instance Manager. See the NGINX Agent [docs](/agent/configuration/encrypt-communication.md) for more details. Requires NGINX Agent 2.x. | "" | |**nginxAgent.instanceManager.tls.enable** | Enable TLS for Instance Manager connection. Requires NGINX Agent 2.x. | true | |**nginxAgent.instanceManager.tls.skipVerify** | Skip certification verification for Instance Manager connection. Requires NGINX Agent 2.x. | false | |**nginxAgent.instanceManager.tls.caSecret** | Name of `nginx.org/ca` secret used for verification of Instance Manager TLS. Requires NGINX Agent 2.x. | "" | |**nginxAgent.instanceManager.tls.secret** | Name of `kubernetes.io/tls` secret with a TLS certificate and key for using mTLS between NGINX Agent and Instance Manager. See the NGINX Instance Manager [docs](/nim/system-configuration/secure-traffic.md#mutual-client-certificate-authentication-setup-mtls) and the NGINX Agent [docs](/agent/configuration/encrypt-communication.md) for more details. Requires NGINX Agent 2.x. | "" | |**nginxAgent.syslog.host** | Address for NGINX Agent to run syslog listener. Requires NGINX Agent 2.x. | 127.0.0.1 | |**nginxAgent.syslog.port** | Port for NGINX Agent to run syslog listener. Requires NGINX Agent 2.x. | 1514 | |**nginxAgent.napMonitoring.collectorBufferSize** | Buffer size for collector. Will contain log lines and parsed log lines. Requires NGINX Agent 2.x. | 50000 | |**nginxAgent.napMonitoring.processorBufferSize** | Buffer size for processor. Will contain log lines and parsed log lines. Requires NGINX Agent 2.x. | 50000 | |**nginxAgent.customConfigMap** | The name of a custom ConfigMap to use instead of the one provided by default. Requires NGINX Agent 2.x.| "" | | **prometheus.create** | Expose NGINX or NGINX Plus metrics in the Prometheus format. | true | | **prometheus.port** | Configures the port to scrape the metrics. | 9113 | | **prometheus.scheme** | Configures the HTTP scheme to use for connections to the Prometheus endpoint. | http | | **prometheus.secret** | The namespace / name of a Kubernetes TLS Secret. If specified, this secret is used to secure the Prometheus endpoint with TLS connections. | "" | | **prometheus.service.create** | Create a Headless service to expose prometheus metrics. Requires `prometheus.create`. | false | | **prometheus.service.labels** | Kubernetes object labels to attach to the service object. | {service: "nginx-ingress-prometheus-service"} | | **prometheus.serviceMonitor.create** | Create a ServiceMonitor custom resource. Requires ServiceMonitor CRD to be installed. For the latest CRD, check the latest release on the [prometheus-operator](https://github.com/prometheus-operator/prometheus-operator) GitHub repo under `example/prometheus-operator-crd/monitoring.coreos.com_servicemonitors.yaml` | false | | **prometheus.serviceMonitor.labels** | Kubernetes object labels to attach to the serviceMonitor object. | {} | | **prometheus.serviceMonitor.selectorMatchLabels** | A set of labels to allow the selection of endpoints for the ServiceMonitor. | {service: "nginx-ingress-prometheus-service"} | | **prometheus.serviceMonitor.endpoints** | A list of endpoints allowed as part of this ServiceMonitor. | [port: prometheus] | | **rbac.create** | Configures RBAC. | true | | **rbac.clusterrole.create** | Configures creation of ClusterRole. Creation can be disabled when more fine-grained control over RBAC is required. For example when controller.watchNamespace is used. | true | | **serviceInsight.create** | Expose NGINX Plus Service Insight endpoint. | false | | **serviceInsight.port** | Configures the port to expose endpoints. | 9114 | | **serviceInsight.scheme** | Configures the HTTP scheme to use for connections to the Service Insight endpoint. | http | | **serviceInsight.secret** | The namespace / name of a Kubernetes TLS Secret. If specified, this secret is used to secure the Service Insight endpoint with TLS connections. | "" | | **serviceNameOverride** | Used to prevent cloud load balancers from being replaced due to service name change during helm upgrades. | "" | -------------------------------------------------------------------------------- title: "NGINX Ingress Controller LTS Helm chart parameters" url: https://docs.nginx.com/nginx-ingress-controller/lts/install/helm/parameters/index.md -------------------------------------------------------------------------------- This page describes the Helm chart parameters of F5 NGINX Ingress Controller LTSs. It is a reference for each parameter name, a description of the parameter, and any default values. ## NGINX Ingress Controller LTS Helm chart parameters **note:** The [values.yaml](https://github.com/nginx/kubernetes-ingress/blob/main/charts/nginx-ingress/values.yaml) file of the Helm chart is a single source of truth and includes parameter descriptions: the table may become outdated. The [values.schema.json](https://github.com/nginx/kubernetes-ingress/blob/main/charts/nginx-ingress/values.schema.json) file can be used to inspect how values are formatted. | Parameter | Description | Default | | ----------| ----------- | ------- | | **controller.name** | The name of the NGINX Ingress Controller LTS deployment, daemonset, or statefulset. | Autogenerated | | **controller.kind** | The kind of the NGINX Ingress Controller LTS installation - deployment, daemonset, or statefulset. | deployment | | **controller.annotations** | Allows for setting of `annotations` for deployment, daemonset, or statefulset. | {} | | **controller.nginxplus** | Deploys NGINX Ingress Controller LTS for NGINX Plus. | true | | **controller.mgmt.licenseTokenSecretName** | Configures the secret used in the [license_token](https://nginx.org/en/docs/ngx_mgmt_module.html#license_token) directive. This key assumes the secret is in the Namespace that NGINX Ingress Controller LTS is deployed in. The secret must be of type `nginx.com/license` with the base64 encoded JWT in the `license.jwt` key. | license-token | | **controller.mgmt.enforceInitialReport** | Configures the [enforce_initial_report](https://nginx.org/en/docs/ngx_mgmt_module.html#enforce_initial_report) directive, which enables or disables the 180-day grace period for sending the initial usage report. | false | | **controller.mgmt.usageReport.endpoint** | Configures the endpoint of the [usage_report](https://nginx.org/en/docs/ngx_mgmt_module.html#usage_report) directive. This is used to configure the endpoint NGINX uses to send usage reports to NGINX Instance Manager . | product.connect.nginx.com | | **controller.mgmt.usageReport.interval** | Configures the interval of the [usage_report](https://nginx.org/en/docs/ngx_mgmt_module.html#usage_report) directive. This specifies the frequency that usage reports are sent. Only seconds(s), minutes(m), and hours(h) are allowed and must be between 60s and 24h. | 1h | | **controller.mgmt.usageReport.proxyHost** | Configures the host name of the [proxy](https://nginx.org/en/docs/ngx_mgmt_module.html#proxy) directive with optional port. | N/A | | **controller.mgmt.usageReport.proxyCredentialsSecretName** | Configures the [proxy_username](https://nginx.org/en/docs/ngx_mgmt_module.html#proxy_username) directive as well as the [proxy_password](https://nginx.org/en/docs/ngx_mgmt_module.html#proxy_password) directive using a Kubernetes Opaque Secret. The Secret must contain `username` and `password` fields. | N/A | | **controller.mgmt.sslVerify** | Configures the [ssl_verify](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_verify) directive, which enables or disables verification of the usage reporting endpoint certificate. | true | | **controller.mgmt.resolver.ipv6** | Configures whether the mgmt block [resolver](https://nginx.org/en/docs/ngx_mgmt_module.html#resolver) directive will look up IPv6 addresses. | true | | **controller.mgmt.resolver.valid** | Configures an [NGINX time](https://nginx.org/en/docs/syntax.html) that the mgmt block [resolver](https://nginx.org/en/docs/ngx_mgmt_module.html#resolver) directive will override the TTL value of responses from nameservers with. | N/A | | **controller.mgmt.resolver.addresses** | Configures addresses used in the mgmt block [resolver](https://nginx.org/en/docs/ngx_mgmt_module.html#resolver) directive. This field takes a list of addresses. | N/A | | **controller.mgmt.sslCertificateSecretName** | Configures the secret used to create the `ssl_certificate` and `ssl_certificate_key` directives. This key assumes the secret is in the Namespace that NGINX Ingress Controller LTS is deployed in. The secret must be of type `kubernetes.io/tls` | N/A | | **controller.mgmt.sslTrustedCertificateSecretName** | Configures the secret used to create the file(s) referenced the in [ssl_trusted_certifcate](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_trusted_certificate), and [ssl_crl](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_crl) directives. This key assumes the secret is in the Namespace that NGINX Ingress Controller LTS is deployed in. The secret must be of type `nginx.org/ca`, where the `ca.crt` key contains a base64 encoded trusted cert, and the optional `ca.crl` key can contain a base64 encoded CRL. If the optional `ca.crl` key is supplied, it will configure the NGINX `ssl_crl` directive. | N/A | | **controller.mgmt.configMapName** | Allows changing the name of the MGMT config map. The name should not include a namespace| Autogenerated | | **controller.nginxReloadTimeout** | The timeout in milliseconds which the NGINX Ingress Controller LTS will wait for a successful NGINX reload after a change or at the initial start. | 60000 | | **controller.hostNetwork** | Enables NGINX Ingress Controller LTS pods to use the host's network namespace. | false | | **controller.dnsPolicy** | DNS policy for NGINX Ingress Controller LTS pods. | ClusterFirst | | **controller.nginxDebug** | Enables debugging for NGINX. Uses the `nginx-debug` binary. Requires `error-log-level: debug` in the ConfigMap via `controller.config.entries`. | false | | **controller.logLevel** | The log level of NGINX Ingress Controller LTS. | info | | **controller.logFormat** | The log format of NGINX Ingress Controller LTS. | glog | | **controller.directiveAutoAdjust** | Automatically adjusts NGINX buffer directives to prevent configuration errors. | false | | **controller.image.digest** | The image digest of NGINX Ingress Controller LTS. | None | | **controller.image.repository** | The image repository of NGINX Ingress Controller LTS. | private-registry.nginx.com/nginx-ic/nginx-plus-ingress | | **controller.image.tag** | The tag of the NGINX Ingress Controller LTS image. | | | **controller.image.pullPolicy** | The pull policy for the NGINX Ingress Controller LTS image. | IfNotPresent | | **controller.lifecycle** | The lifecycle of the NGINX Ingress Controller LTS pods. | {} | | **controller.customConfigMap** | The name of the custom ConfigMap used by the NGINX Ingress Controller LTS. If set, then the default config is ignored. | "" | | **controller.config.name** | The name of the ConfigMap used by the NGINX Ingress Controller LTS. | Autogenerated | | **controller.config.annotations** | The annotations of the NGINX Ingress Controller LTS configmap. | {} | | **controller.config.entries** | The entries of the ConfigMap for customizing NGINX configuration. See [ConfigMap resource docs](/nic/lts/configuration/global-configuration/configmap-resource.md) for the list of supported ConfigMap keys. | {} | | **controller.customPorts** | A list of custom ports to expose on the NGINX Ingress Controller LTS pod. Follows the conventional Kubernetes yaml syntax for container ports. | [] | | **controller.defaultTLS.cert** | The base64-encoded TLS certificate for the default HTTPS server. **Note:** It is recommended that you specify your own certificate. Alternatively, omitting the default server secret completely will configure NGINX to reject TLS connections to the default server. | | **controller.defaultTLS.key** | The base64-encoded TLS key for the default HTTPS server. **Note:** It is recommended that you specify your own key. Alternatively, omitting the default server secret completely will configure NGINX to reject TLS connections to the default server. | | **controller.defaultTLS.secret** | The secret with a TLS certificate and key for the default HTTPS server. The value must follow the following format: `/`. Used as an alternative to specifying a certificate and key using `controller.defaultTLS.cert` and `controller.defaultTLS.key` parameters. **Note:** Alternatively, omitting the default server secret completely will configure NGINX to reject TLS connections to the default server. | None | | **controller.wildcardTLS.cert** | The base64-encoded TLS certificate for every Ingress/VirtualServer host that has TLS enabled but no secret specified. If the parameter is not set, for such Ingress/VirtualServer hosts NGINX will break any attempt to establish a TLS connection. | None | | **controller.wildcardTLS.key** | The base64-encoded TLS key for every Ingress/VirtualServer host that has TLS enabled but no secret specified. If the parameter is not set, for such Ingress/VirtualServer hosts NGINX will break any attempt to establish a TLS connection. | None | | **controller.wildcardTLS.secret** | The secret with a TLS certificate and key for every Ingress/VirtualServer host that has TLS enabled but no secret specified. The value must follow the following format: `/`. Used as an alternative to specifying a certificate and key using `controller.wildcardTLS.cert` and `controller.wildcardTLS.key` parameters. | None | | **controller.nodeSelector** | The node selector for pod assignment for NGINX Ingress Controller LTS pods. | {} | | **controller.terminationGracePeriodSeconds** | The termination grace period of the NGINX Ingress Controller LTS pod. | 30 | | **controller.tolerations** | The tolerations of NGINX Ingress Controller LTS pods. | [] | | **controller.affinity** | The affinity of NGINX Ingress Controller LTS pods. | {} | | **controller.topologySpreadConstraints** | The topology spread constraints of the NGINX Ingress Controller LTS pods. | {} | | **controller.env** | The additional environment variables to be set on the NGINX Ingress Controller LTS pods. | [] | | **controller.volumes** | The volumes of NGINX Ingress Controller LTS pods. | [] | | **controller.volumeMounts** | The volumeMounts of NGINX Ingress Controller LTS pods. | [] | | **controller.initContainers** | InitContainers for NGINX Ingress Controller LTS pods. | [] | | **controller.extraContainers** | Extra (eg. sidecar) containers for NGINX Ingress Controller LTS pods. | [] | | **controller.podSecurityContext**| The SecurityContext for Ingress Controller pods. | "seccompProfile": {"type": "RuntimeDefault"} | | **controller.securityContext** | The SecurityContext for Ingress Controller container. | {} | | **controller.initContainerSecurityContext** | The SecurityContext for NGINX Ingress Controller LTS init container when `readOnlyRootFilesystem` is enabled by either setting `controller.securityContext.readOnlyRootFilesystem` or `controller.readOnlyRootFilesystem`to `true`. | {} | | **controller.resources** | The resources of NGINX Ingress Controller LTS pods. | requests: cpu=100m,memory=128Mi | | **controller.initContainerResources** | The resources of the init container which is used when `readOnlyRootFilesystem` is enabled by either setting `controller.securityContext.readOnlyRootFilesystem` or `controller.readOnlyRootFilesystem`to `true`. | requests: cpu=100m,memory=128Mi | | **controller.replicaCount** | The number of replicas of the NGINX Ingress Controller LTS deployment. | 1 | | **controller.ingressClass.name** | A class of NGINX Ingress Controller LTS. An IngressClass resource with the name equal to the class must be deployed. Otherwise, the NGINX Ingress Controller LTS will fail to start. the NGINX Ingress Controller LTS only processes resources that belong to its class - i.e. have the "ingressClassName" field resource equal to the class. the NGINX Ingress Controller LTS processes all the VirtualServer/VirtualServerRoute/TransportServer resources that do not have the "ingressClassName" field for all versions of Kubernetes. | nginx | | **controller.ingressClass.create** | Creates a new IngressClass object with the name `controller.ingressClass.name`. Set to `false` to use an existing ingressClass created using `kubectl` with the same name. If you use `helm upgrade`, do not change the values from the previous release as helm will delete IngressClass objects managed by helm. If you are upgrading from a release earlier than , do not set the value to false. | true | | **controller.ingressClass.setAsDefaultIngress** | New Ingresses without an `"ingressClassName"` field specified will be assigned the class specified in `controller.ingressClass.name`. Requires `controller.ingressClass.create`. | false | | **controller.watchNamespace** | Comma separated list of namespaces the NGINX Ingress Controller LTS should watch for resources. By default the NGINX Ingress Controller LTS watches all namespaces. Mutually exclusive with `controller.watchNamespaceLabel`. Please note that if configuring multiple namespaces using the Helm cli `--set` option, the string needs to wrapped in double quotes and the commas escaped using a backslash - e.g. `--set controller.watchNamespace="default\,nginx-ingress"`. | "" | | **controller.watchNamespaceLabel** | Configures NGINX Ingress Controller LTS to watch only those namespaces with label foo=bar. By default the NGINX Ingress Controller LTS watches all namespaces. Mutually exclusive with `controller.watchNamespace`. | "" | | **controller.watchSecretNamespace** | Comma separated list of namespaces the NGINX Ingress Controller LTS should watch for resources of type Secret. If this arg is not configured, the NGINX Ingress Controller LTS watches the same namespaces for all resources, see `controller.watchNamespace` and `controller.watchNamespaceLabel`. All namespaces included with this argument must be part of either `controller.watchNamespace` or `controller.watchNamespaceLabel`. Please note that if configuring multiple namespaces using the Helm cli `--set` option, the string needs to wrapped in double quotes and the commas escaped using a backslash - e.g. `--set controller.watchSecretNamespace="default\,nginx-ingress"`. | "" | | **controller.enableCustomResources** | Enable the custom resources. | true | | **controller.enableOIDC** | Enable OIDC policies. | false | | **controller.enableTLSPassthrough** | Enable TLS Passthrough on default port 443. Requires `controller.enableCustomResources`. | false | | **controller.tlsPassThroughPort** | Set the port for the TLS Passthrough. Requires `controller.enableCustomResources` and `controller.enableTLSPassthrough`. | 443 | | **controller.enableCertManager** | Enable x509 automated certificate management for VirtualServer resources using cert-manager (cert-manager.io). Requires `controller.enableCustomResources`. | false | | **controller.enableExternalDNS** | Enable integration with ExternalDNS for configuring public DNS entries for VirtualServer resources using [ExternalDNS](https://github.com/kubernetes-sigs/external-dns). Requires `controller.enableCustomResources`. | false | | **controller.globalConfiguration.create** | Creates the GlobalConfiguration custom resource. Requires `controller.enableCustomResources`. | false | | **controller.globalConfiguration.spec** | The spec of the GlobalConfiguration for defining the global configuration parameters of the NGINX Ingress Controller LTS. | {} | | **controller.enableSnippets** | Enable custom NGINX configuration snippets in Ingress, VirtualServer, VirtualServerRoute and TransportServer resources. | false | | **controller.healthStatus** | Add a location "/nginx-health" to the default server. The location responds with the 200 status code for any request. Useful for external health-checking of the NGINX Ingress Controller LTS. | false | | **controller.healthStatusURI** | Sets the URI of health status location in the default server. Requires `controller.healthStatus`. | "/nginx-health" | | **controller.nginxStatus.enable** | Enable the NGINX Plus status API. | true | | **controller.nginxStatus.port** | Set the port where the NGINX Plus API is exposed. | 8080 | | **controller.nginxStatus.allowCidrs** | Add IP/CIDR blocks to the allow list for the NGINX Plus API. Separate multiple IP/CIDR by commas. | 127.0.0.1,::1 | | **controller.priorityClassName** | The PriorityClass of NGINX Ingress Controller LTS pods. | None | | **controller.service.create** | Creates a service to expose NGINX Ingress Controller LTS pods. | true | | **controller.service.type** | The type of service to create for NGINX Ingress Controller LTS. | LoadBalancer | | **controller.service.externalTrafficPolicy** | The externalTrafficPolicy of the service. The value Local preserves the client source IP. | Local | | **controller.service.annotations** | The annotations of the NGINX Ingress Controller LTS service. | {} | | **controller.service.extraLabels** | The extra labels of the service. | {} | | **controller.service.loadBalancerIP** | The static IP address for the load balancer. Requires `controller.service.type` set to `LoadBalancer`. The cloud provider must support this feature. | "" | | **controller.service.loadBalancerClass** | This property allows the user to define a specific load balancer implementation or configuration class for the service. Requires `controller.service.type` set to `LoadBalancer`. The cloud provider must support this feature. | "" | | **controller.service.externalIPs** | The list of external IPs for the NGINX Ingress Controller LTS service. | [] | | **controller.service.clusterIP** | The clusterIP for the NGINX Ingress Controller LTS service, autoassigned if not specified. | "" | | **controller.service.loadBalancerSourceRanges** | The IP ranges (CIDR) that are allowed to access the load balancer. Requires `controller.service.type` set to `LoadBalancer`. The cloud provider must support this feature. | [] | | **controller.service.name** | The name of the service. | Autogenerated | | **controller.service.customPorts** | A list of custom ports to expose through the NGINX Ingress Controller LTS service. Follows the conventional Kubernetes yaml syntax for service ports. | [] | | **controller.service.httpPort.enable** | Enables the HTTP port for the NGINX Ingress Controller LTS service. | true | | **controller.service.httpPort.port** | The HTTP port of the NGINX Ingress Controller LTS service. | 80 | | **controller.service.httpPort.nodePort** | The custom NodePort for the HTTP port. Requires `controller.service.type` set to `NodePort`. | "" | | **controller.service.httpPort.targetPort** | The target port of the HTTP port of the NGINX Ingress Controller LTS service. | 80 | | **controller.service.httpsPort.enable** | Enables the HTTPS port for the NGINX Ingress Controller LTS service. | true | | **controller.service.httpsPort.port** | The HTTPS port of the NGINX Ingress Controller LTS service. | 443 | | **controller.service.httpsPort.nodePort** | The custom NodePort for the HTTPS port. Requires `controller.service.type` set to `NodePort`. | "" | | **controller.service.httpsPort.targetPort** | The target port of the HTTPS port of the NGINX Ingress Controller LTS service. | 443 | | **controller.service.sessionAffinity.enable** | Enable session affinity. | `false` | | **controller.service.sessionAffinity.type** | Session affinity type. Valid values: `None`, `ClientIP`. | `ClientIP` | | **controller.service.sessionAffinity.timeout** | Session affinity timeout in seconds. | 3600 | | **controller.serviceAccount.annotations** | The annotations of the NGINX Ingress Controller LTS service account. | {} | | **controller.serviceAccount.name** | The name of the service account of the NGINX Ingress Controller LTS pods. Used for RBAC. | Autogenerated | | **controller.serviceAccount.imagePullSecretName** | The name of the secret containing docker registry credentials. Secret must exist in the same namespace as the helm release. | "" | | **controller.serviceAccount.imagePullSecretsNames** | The list of secret names containing docker registry credentials. Secret must exist in the same namespace as the helm release. | [] | | **controller.reportIngressStatus.enable** | Updates the address field in the status of Ingress resources with an external address of the NGINX Ingress Controller LTS. You must also specify the source of the external address either through an external service via `controller.reportIngressStatus.externalService`, `controller.reportIngressStatus.ingressLink` or the `external-status-address` entry in the ConfigMap via `controller.config.entries`. **Note:** `controller.config.entries.external-status-address` takes precedence over the others. | true | | **controller.reportIngressStatus.externalService** | Specifies the name of the service with the type LoadBalancer through which the NGINX Ingress Controller LTS is exposed externally. The external address of the service is used when reporting the status of Ingress, VirtualServer and VirtualServerRoute resources. `controller.reportIngressStatus.enable` must be set to `true`. The default is autogenerated and enabled when `controller.service.create` is set to `true` and `controller.service.type` is set to `LoadBalancer`. | Autogenerated | | **controller.reportIngressStatus.ingressLink** | Specifies the name of the IngressLink resource, which exposes the NGINX Ingress Controller LTS pods via a BIG-IP system. The IP of the BIG-IP system is used when reporting the status of Ingress, VirtualServer and VirtualServerRoute resources. `controller.reportIngressStatus.enable` must be set to `true`. | "" | | **controller.reportIngressStatus.enableLeaderElection** | Enable Leader election to avoid multiple replicas of the controller reporting the status of Ingress resources. `controller.reportIngressStatus.enable` must be set to `true`. | true | | **controller.reportIngressStatus.leaderElectionLockName** | Specifies the name of the ConfigMap, within the same namespace as the controller, used as the lock for leader election. controller.reportIngressStatus.enableLeaderElection must be set to true. | Autogenerated | | **controller.reportIngressStatus.annotations** | The annotations of the leader election configmap. | {} | | **controller.pod.annotations** | The annotations of the NGINX Ingress Controller LTS pod. | {} | | **controller.pod.extraLabels** | The additional extra labels of the NGINX Ingress Controller LTS pod. | {} | | **controller.readyStatus.enable** | Enables the readiness endpoint `"/nginx-ready"`. The endpoint returns a success code when NGINX has loaded all the config after the startup. This also configures a readiness probe for the NGINX Ingress Controller LTS pods that uses the readiness endpoint. | true | | **controller.readyStatus.port** | The HTTP port for the readiness endpoint. | 8081 | | **controller.readyStatus.initialDelaySeconds** | The number of seconds after the NGINX Ingress Controller LTS pod has started before readiness probes are initiated. | 0 | | **controller.startupStatus.enable** | Enables the startup probe for the Ingress Controller. | false | | **controller.startupStatus.port** | The port where the startup endpoint is exposed. This is a required field if `controller.startupStatus.enable` is set to true. | N/A | | **controller.startupStatus.path** | The path to the startup endpoint. This is a required field if `controller.startupStatus.enable` is set to true. | N/A | | **controller.startupStatus.initialDelaySeconds** | The number of seconds after the Ingress Controller pod has started before startup probes are initiated. | N/A | | **controller.startupStatus.periodSeconds** | The number of seconds between each startup probe. | N/A | | **controller.startupStatus.timeoutSeconds** | The number of seconds after which the startup probe times out. | N/A | | **controller.startupStatus.successThreshold** | Minimum consecutive successes for the probe to be considered successful. | N/A | | **controller.startupStatus.failureThreshold** | When a probe fails, Kubernetes will try failureThreshold times before giving up. | N/A | | **controller.enableLatencyMetrics** | Enable collection of latency metrics for upstreams. Requires `prometheus.create`. | false | | **controller.minReadySeconds** | Specifies the minimum number of seconds for which a newly created Pod should be ready without any of its containers crashing, for it to be considered available. [docs](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#min-ready-seconds) | 0 | | **controller.autoscaling.enabled** | Enables HorizontalPodAutoscaling. | false | | **controller.autoscaling.create** | Create the HorizontalPodAutoscaler resource. This can be set to false to manage the HPA externally. | true | | **controller.autoscaling.annotations** | The annotations of the NGINX Ingress Controller LTS HorizontalPodAutoscaler. | {} | | **controller.autoscaling.behavior** | Behavior configuration for the HPA. | {} | | **controller.autoscaling.minReplicas** | Minimum number of replicas for the HPA. | 1 | | **controller.autoscaling.maxReplicas** | Maximum number of replicas for the HPA. | 3 | | **controller.autoscaling.targetCPUUtilizationPercentage** | The target CPU utilization percentage. | 50 | | **controller.autoscaling.targetMemoryUtilizationPercentage** | The target memory utilization percentage. | 50 | | **controller.podDisruptionBudget.enabled** | Enables PodDisruptionBudget. | false | | **controller.podDisruptionBudget.annotations** | The annotations of the NGINX Ingress Controller LTS pod disruption budget | {} | | **controller.podDisruptionBudget.minAvailable** | The number of Ingress Controller pods that should be available. This is a mutually exclusive setting with "maxUnavailable". | 0 | | **controller.podDisruptionBudget.maxUnavailable** | The number of Ingress Controller pods that can be unavailable. This is a mutually exclusive setting with "minAvailable". | 0 | | **controller.strategy** | Specifies the strategy used to replace old Pods with new ones. Docs for [Deployment update strategy](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#strategy), [Daemonset update strategy](https://kubernetes.io/docs/tasks/manage-daemon/update-daemon-set/#daemonset-update-strategy) and [StatefulSet update strategy](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#update-strategies) | {} | | **controller.statefulset.podManagementPolicy** | Controls how pods are created during initial scale up, when replacing pods on nodes, or when scaling down. Available options: `OrderedReady` or `Parallel`. Only applies when `controller.kind` is set to `statefulset`. | `OrderedReady` | | **controller.statefulset.persistentVolumeClaimRetentionPolicy.whenDeleted** | Controls the retention policy for PVCs when the StatefulSet is deleted. Available options: `Retain` or `Delete`. Only applies when `controller.kind` is set to `statefulset`. | `Retain` | | **controller.statefulset.persistentVolumeClaimRetentionPolicy.whenScaled** | Controls the retention policy for PVCs when the StatefulSet is scaled down. Available options: `Retain` or `Delete`. Only applies when `controller.kind` is set to `statefulset`. | `Retain` | | **controller.statefulset.nginxCachePVC.size** | The size of the persistent volume claim for NGINX cache storage. Only applies when `controller.kind` is set to `statefulset`. | `256Mi` | | **controller.statefulset.nginxCachePVC.storageClass** | The storage class for the persistent volume claim. Only applies when `controller.kind` is set to `statefulset`. | `""` | | **controller.statefulset.nginxCachePVC.accessModes** | The access modes for the persistent volume claim. Only applies when `controller.kind` is set to `statefulset`. | `["ReadWriteOnce"]` | | **controller.disableIPV6** | Disable IPV6 listeners explicitly for nodes that do not support the IPV6 stack. | false | | **controller.defaultHTTPListenerPort** | Sets the port for the HTTP `default_server` listener. | 80 | | **controller.defaultHTTPSListenerPort** | Sets the port for the HTTPS `default_server` listener. | 443 | | **controller.readOnlyRootFilesystem** | Configure root filesystem as read-only and add volumes for temporary data. Three major releases after 3.5.x this argument will be moved permanently to the `controller.securityContext` section. | false | | **controller.enableSSLDynamicReload** | Enable lazy loading for SSL Certificates. | true | | **controller.telemetryReporting.enable** | Enable telemetry reporting. | true | | **controller.enableWeightChangesDynamicReload** | Enable weight changes without reloading the NGINX configuration. May require increasing `map_hash_bucket_size`, `map_hash_max_size`, `variable_hash_bucket_size`, and `variable_hash_max_size` in the [ConfigMap](/nic/lts/configuration/global-configuration/configmap-resource.md) if there are many two-way splits. Requires `controller.nginxplus` | false | |**nginxAgent.enable** | Enable NGINX Agent 3.x to allow [connecting to NGINX One Console](/nginx-one-console/k8s/add-nic.md) . | false | |**nginxAgent.logLevel** | Log level for NGINX Agent. | "error" | |**nginxAgent.dataplaneKeySecretName** | Name of the Kubernetes Secret containing the Data Plane key used to authenticate to NGINX One Console. Learn more [here](/nginx-one-console/k8s/add-nic.md). Required when `nginxAgent.enable` is set to `true`. Requires NGINX Agent 3.x. | "" | |**nginxAgent.endpointHost** | Domain or IP address for the NGINX One Console. Requires NGINX Agent 3.x. | "agent.connect.nginx.com" | |**nginxAgent.endpointPort** | Port for the NGINX One Console endpoint. Requires NGINX Agent 3.x. | 443 | |**nginxAgent.tlsSkipVerify** | Skip TLS verification for the NGINX One Console endpoint. Requires NGINX Agent 3.x. | false | |**nginxAgent.instanceGroup** | Set a custom Instance Group name for the deployment, shown when connected to NGINX Instance Manager. `nginx-ingress.controller.fullname` will be used if not set. Requires NGINX Agent 2.x. | "" | |**nginxAgent.instanceManager.host** | FQDN or IP for connecting to NGINX Ingress Controller LTS. Required when `nginxAgent.enable` is set to `true`. Requires NGINX Agent 2.x. | "" | |**nginxAgent.instanceManager.grpcPort** | Port for connecting to NGINX Ingress Controller LTS. Requires NGINX Agent 2.x. | 443 | |**nginxAgent.instanceManager.sni** | Server Name Indication for Instance Manager. See the NGINX Agent [docs](/agent/configuration/encrypt-communication.md) for more details. Requires NGINX Agent 2.x. | "" | |**nginxAgent.instanceManager.tls.enable** | Enable TLS for Instance Manager connection. Requires NGINX Agent 2.x. | true | |**nginxAgent.instanceManager.tls.skipVerify** | Skip certification verification for Instance Manager connection. Requires NGINX Agent 2.x. | false | |**nginxAgent.instanceManager.tls.caSecret** | Name of `nginx.org/ca` secret used for verification of Instance Manager TLS. Requires NGINX Agent 2.x. | "" | |**nginxAgent.instanceManager.tls.secret** | Name of `kubernetes.io/tls` secret with a TLS certificate and key for using mTLS between NGINX Agent and Instance Manager. See the NGINX Instance Manager [docs](/nim/system-configuration/secure-traffic.md#mutual-client-certificate-authentication-setup-mtls) and the NGINX Agent [docs](/agent/configuration/encrypt-communication.md) for more details. Requires NGINX Agent 2.x. | "" | |**nginxAgent.syslog.host** | Address for NGINX Agent to run syslog listener. Requires NGINX Agent 2.x. | 127.0.0.1 | |**nginxAgent.syslog.port** | Port for NGINX Agent to run syslog listener. Requires NGINX Agent 2.x. | 1514 | |**nginxAgent.napMonitoring.collectorBufferSize** | Buffer size for collector. Will contain log lines and parsed log lines. Requires NGINX Agent 2.x. | 50000 | |**nginxAgent.napMonitoring.processorBufferSize** | Buffer size for processor. Will contain log lines and parsed log lines. Requires NGINX Agent 2.x. | 50000 | |**nginxAgent.customConfigMap** | The name of a custom ConfigMap to use instead of the one provided by default. Requires NGINX Agent 2.x.| "" | | **nginxServiceMesh.enable** | Enable integration with NGINX Service Mesh. See the NGINX Service Mesh docs for more details. Requires `controller.nginxplus`. | false | | **nginxServiceMesh.enableEgress** | Enable NGINX Service Mesh workloads to route egress traffic through the NGINX Ingress Controller LTS. See the NGINX Service Mesh docs for more details. Requires `nginxServiceMesh.enable`. | false | | **prometheus.create** | Expose NGINX Plus metrics in the Prometheus format. | true | | **prometheus.port** | Configures the port to scrape the metrics. | 9113 | | **prometheus.scheme** | Configures the HTTP scheme to use for connections to the Prometheus endpoint. | http | | **prometheus.secret** | The namespace / name of a Kubernetes TLS Secret. If specified, this secret is used to secure the Prometheus endpoint with TLS connections. | "" | | **prometheus.service.create** | Create a Headless service to expose prometheus metrics. Requires `prometheus.create`. | false | | **prometheus.service.labels** | Kubernetes object labels to attach to the service object. | {service: "nginx-ingress-prometheus-service"} | | **prometheus.serviceMonitor.create** | Create a ServiceMonitor custom resource. Requires ServiceMonitor CRD to be installed. For the latest CRD, check the latest release on the [prometheus-operator](https://github.com/prometheus-operator/prometheus-operator) GitHub repo under `example/prometheus-operator-crd/monitoring.coreos.com_servicemonitors.yaml` | false | | **prometheus.serviceMonitor.labels** | Kubernetes object labels to attach to the serviceMonitor object. | {} | | **prometheus.serviceMonitor.selectorMatchLabels** | A set of labels to allow the selection of endpoints for the ServiceMonitor. | {service: "nginx-ingress-prometheus-service"} | | **prometheus.serviceMonitor.endpoints** | A list of endpoints allowed as part of this ServiceMonitor. | [port: prometheus] | | **rbac.create** | Configures RBAC. | true | | **rbac.clusterrole.create** | Configures creation of ClusterRole. Creation can be disabled when more fine-grained control over RBAC is required. For example when controller.watchNamespace is used. | true | | **serviceInsight.create** | Expose NGINX Plus Service Insight endpoint. | false | | **serviceInsight.port** | Configures the port to expose endpoints. | 9114 | | **serviceInsight.scheme** | Configures the HTTP scheme to use for connections to the Service Insight endpoint. | http | | **serviceInsight.secret** | The namespace / name of a Kubernetes TLS Secret. If specified, this secret is used to secure the Service Insight endpoint with TLS connections. | "" | | **serviceNameOverride** | Used to prevent cloud load balancers from being replaced due to service name change during helm upgrades. | "" | -------------------------------------------------------------------------------- title: "Ingress-specific policy behavior" url: https://docs.nginx.com/nginx-ingress-controller/configuration/policy-resource/ingress-specific-behavior/index.md -------------------------------------------------------------------------------- Ingress resources support Policy resources, but with some differences from VirtualServer and VirtualServerRoute resources. Review the following behavior before you attach a policy to an Ingress. ## Ingress doesn't support every policy type Ingress policy support is narrower than VirtualServer support. If you need route-level control for features like JWT, OIDC (NJS), cache, or rate limiting, use VirtualServer and VirtualServerRoute instead. ## WAF on Ingress must use "nginx.com/policies" WAF is a Plus-only feature. When you use a WAF policy with Ingress, reference it through: ```yaml metadata: annotations: nginx.com/policies: waf-policy ``` Don't attach WAF to Ingress with `nginx.org/policies`. **important:** On Ingress, `nginx.org/policies` and `nginx.com/policies` aren't interchangeable. Reference WAF policies only through `nginx.com/policies`. ## Egress mTLS on Ingress only configures TLS parameters `egressMTLS` defines how NGINX authenticates to the upstream and verifies the upstream certificate. It doesn't switch the upstream transport from plain HTTP to HTTPS. For Ingress, you still need the upstream connection itself to use TLS. Common ways to do this include: - `nginx.org/ssl-services` for HTTPS upstreams - `nginx.org/grpc-services` for gRPC upstreams For example: ```yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: webapp-ingress annotations: nginx.org/policies: egress-mtls-policy nginx.org/ssl-services: "secure-app" spec: ingressClassName: nginx rules: - host: webapp.example.com http: paths: - path: / pathType: Prefix backend: service: name: secure-app port: number: 8443 ``` Without upstream TLS, the egress mTLS policy has TLS settings but no TLS connection to apply them to. NGINX then typically sends plain HTTP to an HTTPS upstream port. **Note:** For Ingress, `egressMTLS` configures how NGINX uses TLS when connecting to the upstream. It doesn't decide whether the upstream connection uses TLS. Configure that separately, for example with `nginx.org/ssl-services` or `nginx.org/grpc-services`. ## Mergeable Ingress behavior For mergeable Ingress: - Policies on the master apply to inherited minion configuration. - Policies on the minion override policies of the same type from the master. This matches the general expectation that a more specific resource overrides a broader one. For `egressMTLS`, there is one extra detail: - The master or standard Ingress policy applies at server scope. - The minion override applies at location scope, so the minion can replace the value set by the master. ## What's next Learn about [policy precedence and override rules](/nic/configuration/policy-resource/precedence-and-overrides.md). -------------------------------------------------------------------------------- title: "Troubleshoot F5 DoS for NGINX" url: https://docs.nginx.com/nginx-ingress-controller/integrations/app-protect-dos/troubleshoot-app-protect-dos/index.md -------------------------------------------------------------------------------- This document describes how to troubleshoot problems when using NGINX Ingress Controller and the App Protect DoS module. To troubleshoot other parts of NGINX Ingress Controller, check the [troubleshooting](/nic/troubleshooting/troubleshoot-common.md) section of the documentation. ## Potential problems The table below outlines potential problems with NGINX Ingress Controller when the App Protect DoS module is enabled. It suggests how to troubleshoot those problems with methods explained in the next section. |Problem area | Symptom | Troubleshooting method | Common cause | | ---| ---| ---| --- | |Start | NGINX Ingress Controller fails to start. | Check the NGINX Ingress Controller logs. | Misconfigured DosProtectedResource, APDosLogConf or APDosPolicy. | |DosProtectedResource, APDosLogConf, APDosPolicy or Ingress Resource. | The configuration is not applied. | Check the events of the DosProtectedResource, APDosLogConf, APDosPolicy and Ingress Resource, check the Ingress Controller logs. | DosProtectedResource, APDosLogConf or APDosPolicy is invalid. | ## Troubleshooting ethods ### Checking NGINX Ingress Controller and App Protect DoS logs App Protect DoS logs are part of the NGINX Ingress Controller logs when the module is enabled. To check the Ingress Controller logs, follow the steps of [Checking the Ingress Controller Logs](/nic/troubleshooting/troubleshoot-common#checking-nginx-ingress-controller-logss) of the Troubleshooting guide. For App Protect DoS specific logs, look for messages starting with `APP_PROTECT_DOS`, such as: ```shell 2021/06/14 08:17:50 [notice] 242#242: APP_PROTECT_DOS { "event": "shared_memory_connected", "worker_pid": 242, "mode": "operational", "mode_changed": true } ``` ### Checking Ingress Resource Events Follow the steps of [Troubleshooting Ingress Resources](/nic/troubleshooting/troubleshoot-ingress). ### Checking VirtualServer Resource Events Follow the steps of [Troubleshooting VirtualServer Resources](/nic/troubleshooting/troubleshoot-virtualserver). ### Checking for DoSProtectedResource Events After you create or update an DosProtectedResource, you can immediately check if the NGINX configuration was successfully applied by NGINX: ```shell kubectl describe dosprotectedresource dos-protected Name: dos-protected Namespace: default Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal AddedOrUpdated 2s nginx-ingress-controller Configuration for default/dos-protected was added or updated ``` Note that in the events section, we have a `Normal` event with the `AddedOrUpdated` reason, which informs us that the configuration was successfully applied. If the DosProtectedResource refers to a missing resource, you should see a message like the following: ```shell Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning Rejected 8s nginx-ingress-controller dos protected refers (default/dospolicy) to an invalid DosPolicy: DosPolicy default/dospolicy not found ``` This can be fixed by adding the missing resource. ### Checking for APDosLogConf Events After you create or update an APDosLogConf, you can immediately check if the NGINX configuration was successfully applied by NGINX: ```shell kubectl describe apdoslogconf logconf Name: logconf Namespace: default Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal AddedOrUpdated 11s nginx-ingress-controller AppProtectDosLogConfig default/logconf was added or updated ``` Note that in the events section, we have a `Normal` event with the `AddedOrUpdated` reason, which informs us that the configuration was successfully applied. ### Check events of APDosPolicy After you create or update an APDosPolicy, you can immediately check if the NGINX configuration was successfully applied by NGINX: ```shell kubectl describe apdospolicy dospolicy Name: dospolicy Namespace: default . . . Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal AddedOrUpdated 2m25s nginx-ingress-controller AppProtectDosPolicy default/dospolicy was added or updated ``` The events section has a *Normal* event with the *AddedOrUpdated reason*, indicating the policy was successfully accepted. ## Run App Protect DoS in Debug log Mode When you configure NGINX Ingress Controller to use debug log mode, the setting also applies to the App Protect DoS module. See [Enable debugging for NGINX Ingress Controller ](/nic/troubleshooting/troubleshoot-common.md#enable-debugging-for-nginx-ingress-controller) for instructions. You can enable debug log mode to App Protect DoS module only by setting the `app-protect-dos-debug` [configmap](/nic/configuration/global-configuration/configmap-resource.md#modules). -------------------------------------------------------------------------------- title: "Troubleshoot F5 WAF for NGINX" url: https://docs.nginx.com/nginx-ingress-controller/integrations/app-protect-waf-v5/troubleshoot-app-protect-waf/index.md -------------------------------------------------------------------------------- This document describes how to troubleshoot problems when using NGINX Ingress Controller and the F5 WAF for NGINX module version 5. For general troubleshooting of NGINX Ingress Controller, check the general [troubleshooting](/nic/troubleshooting/troubleshoot-common) documentation. ## Potential problems The table below categorizes some potential problems with NGINX Ingress Controller when F5 WAF for NGINX module is enabled. It suggests how to troubleshoot those problems, using one or more methods from the next section. |Problem area | Symptom | Troubleshooting method | Common cause | | ---| ---| ---| --- | |Start. | The Ingress Controller fails to start. | Check the logs. | Misconfigured policy bundle. | |Start | The configuration is not applied. | Check if a policy bundle is compiled using version of the compiler running in NGINX Ingress Controller. | Policy bundle is invalid. | |Start | The configuration is not applied. | Check if bundle is present in a volume. | Policy bundle is not present in the mounted volume. | |APLogConf, Policy or Ingress Resource. | The configuration is not applied. | Check the events of the APLogConf, Policy and Ingress Resource, check the logs, replace the policy bundle. | Policy bundle is invalid. | |[Bundle source](/nic/integrations/app-protect-waf-v5/bundle-sources.md). | Bundle not fetched, Warning event on Policy. | Check Policy events and status. Verify the bundle source URL and credentials are correct. | Invalid URL, authentication failure, or bundle not yet compiled on the management plane. | |[Bundle source](/nic/integrations/app-protect-waf-v5/bundle-sources.md). | VirtualServer or Ingress returns HTTP 500. | Check Policy status for bundle source errors. | Bundle source is unreachable or the bundle is not yet available. | |[Bundle source](/nic/integrations/app-protect-waf-v5/bundle-sources.md). | Policy not updating after bundle recompilation. | Verify ``enablePolling`` is ``true`` and ``pollInterval`` is at least ``1m``. | Polling is not enabled, or the poll interval is below the minimum. | ## Troubleshooting methods ### Check NGINX Ingress Controller and App Protect logs App Protect logs are part of NGINX Ingress Controller logs when the module is enabled. To check NGINX Ingress Controller logs, follow the steps of [Checking the Ingress Controller Logs](/nic/troubleshooting/troubleshoot-common#checking-the-ingress-controller-logs) of the Troubleshooting guide. For App Protect specific logs, look for messages starting with `APP_PROTECT`, for example: ```shell 2020/07/10 11:13:20 [notice] 17#17: APP_PROTECT { "event": "configuration_load_success", "software_version": "2.52.1", "completed_successfully":true,"attack_signatures_package":{"revision_datetime":"2020-06-18T10:11:32Z","version":"2020.06.18"}} ``` ### Check Ingress resource events Read the topic [Troubleshooting Ingress resources](/nic/troubleshooting/troubleshoot-ingress). ### Check APLogConf events After you create or update an APLogConf, you can immediately check if the NGINX configuration was successfully applied by NGINX: ```shell kubectl describe aplogconf logconf Name: logconf Namespace: default Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal AddedOrUpdated 11s nginx-ingress-controller AppProtectLogConfig default/logconf was added or updated ``` Note that in the events section, we have a `Normal` event with the `AddedOrUpdated` reason, which informs us that the configuration was successfully applied. ### Check APPolicy events After you create or update an APPolicy, you can immediately check if the NGINX configuration was successfully applied by NGINX: ```shell kubectl describe appolicy dataguard-alarm Name: dataguard-alarm Namespace: default Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal AddedOrUpdated 2m25s nginx-ingress-controller AppProtectPolicy default/dataguard-alarm was added or updated ``` The events section has a *Normal* event with the *AddedOrUpdated reason*, indicating the policy was successfully accepted. ### Replace the policy **Note:** This method only applies if using [external references](/nginx-app-protect/v4/configuration/#external-references) If items on the external reference change but the spec of the APPolicy remains unchanged (even when re-applying the policy), Kubernetes will not detect the update. In this case you can force-replace the resource. This will remove the resource and add it again, triggering a reload. For example: ```shell kubectl replace appolicy -f your-policy-manifest.yaml --force ``` ### Check the availability of APPolicy external references **Note:** This method only applies if you're using [external references](/nginx-app-protect/v4/configuration/#external-references) in F5 WAF for NGINX policies. To check which servers host the external references of a policy: ```shell kubectl get appolicy mypolicy -o jsonpath='{.items[*].spec.policy.*.link}' | tr ' ' '\n' http://192.168.100.100/resources/headersettings.txt ``` You can check the total time a http request takes, in multiple ways eg. using curl: ```shell curl -w '%{time_total}' http://192.168.100.100/resources/headersettings.txt ``` ## Run App Protect in debug mode When you set NGINX Ingress Controller to use debug mode, the setting also applies to the F5 WAF for NGINX module. See [Running NGINX in the Debug Mode](/nic/troubleshooting/troubleshoot-common.md#enable-debugging-for-nginx-ingress-controller) for instructions. ## Known issues When using NGINX Ingress Controller with the F5 WAF for NGINX module, the following issues have been reported. The occurrence of these issues is commonly related to a higher number of Ingress Resources with App Protect being enabled in a cluster. ### Subrequests compatibility Policies that rely on NGINX subrequests (such as `ExternalAuth`, `APIKey`, `JWT` with remote JWKS fetching, `OIDC`, or `Cache` with `cacheBackgroundUpdate`) and F5 WAF for NGINX may not function as expected and may cause issues when applied together on the same route. When you make a change that requires NGINX to apply a new configuration, NGINX Ingress Controller reloads NGINX automatically. Without the F5 WAF for NGINX module enabled, usual reload times are around 150ms. If F5 WAF for NGINX module is enabled and is being used by any number of Ingress Resources, these reloads might take a few seconds instead. ### NGINX configuration drift If you are running more than one instance of NGINX Ingress Controller, the extended reload time may cause the NGINX configuration of your instances to be out of sync. This can occur because there is no order imposed on how NGINX Ingress Controller processes the Kubernetes Resources. The configurations will be the same after all instances have completed the reload. In order to reduce these inconsistencies, we advise that you do not apply changes to multiple resources handled by NGINX Ingress Controller at the same time. ### NGINX fails to start or reload The first time NGINX Ingress Controller starts, or whenever there is a change that requires reloading NGINX, NGINX Ingress Controller will verify if the reload was successful. The timeout for this verification is normally 4 seconds. When App Protect is enabled, this timeout increases to 20 seconds. This timeout should be more than enough to verify configurations. However, when numerous Ingress resources with App Protect enabled are handled by NGINX Ingress Controller at the same time, you may find that you need to extend the timeout further. Examples of when this might be necessary include: - You need to apply a large amount of Ingress Resources at once. - You are running NGINX Ingress Controller for the first time in a cluster where the Ingress resources with App Protect enabled are already present. You can increase this timeout by setting the `nginx-reload-timeout` [cli-argument](/nic/configuration/global-configuration/command-line-arguments.md#cmdoption-nginx-reload-timeout). When using the User Defined Signature feature, an update to an `APUserSig` requires more reload time from NGINX Plus compared with the other AppProtect resources. As a consequence, we recommend increasing the `nginx-reload-timeout` to 30 seconds if you're planning to use this feature. If you are using external references in your F5 WAF for NGINX policies, verify if the servers hosting the referenced resources are available and that their response time is as short as possible (see the Check the Availability of APPolicy External References section). If the references are not available during NGINX Ingress Controller startup, the pod will fail to start. In case the resources are not available during a reload, the reload will fail, and NGINX Plus will use the previous correct configuration. -------------------------------------------------------------------------------- title: "Use Helm to uninstall NGINX Ingress Controller" url: https://docs.nginx.com/nginx-ingress-controller/install/helm/uninstall/index.md -------------------------------------------------------------------------------- This page describes how to use Helm to uninstall F5 NGINX Ingress Controller. It explains how to remove the chart, then remove the custom resource definitions (CRDs). By following these instructions, you will remove NGINX Ingress Controller from your Kubernetes cluster. ## Remove the Helm chart To uninstall NGINX Ingress Controller, you must first remove the chart. To remove a release named **\**, use the following command: ```shell helm uninstall ``` The command removes all the Kubernetes components associated with the release, then deletes the release itself. ## Remove the CRDs After removing the release, pull the chart sources: ```shell helm pull oci://ghcr.io/nginx/charts/nginx-ingress --untar --version ``` **Note:** The next command will delete all corresponding custom resources in your cluster across all namespaces. Before using it, check there are no custom resources that you want to keep, and that there are no other NGINX Ingress Controller instances running in the cluster. Then use _kubectl_ to delete the CRDs: ```shell kubectl delete -f crds/ ``` #### Example output ```text customresourcedefinition.apiextensions.k8s.io "aplogconfs.appprotect.f5.com" deleted customresourcedefinition.apiextensions.k8s.io "appolicies.appprotect.f5.com" deleted customresourcedefinition.apiextensions.k8s.io "apusersigs.appprotect.f5.com" deleted customresourcedefinition.apiextensions.k8s.io "apdoslogconfs.appprotectdos.f5.com" deleted customresourcedefinition.apiextensions.k8s.io "apdospolicies.appprotectdos.f5.com" deleted customresourcedefinition.apiextensions.k8s.io "dosprotectedresources.appprotectdos.f5.com" deleted customresourcedefinition.apiextensions.k8s.io "dnsendpoints.externaldns.nginx.org" deleted customresourcedefinition.apiextensions.k8s.io "globalconfigurations.k8s.nginx.org" deleted customresourcedefinition.apiextensions.k8s.io "policies.k8s.nginx.org" deleted customresourcedefinition.apiextensions.k8s.io "transportservers.k8s.nginx.org" deleted customresourcedefinition.apiextensions.k8s.io "virtualserverroutes.k8s.nginx.org" deleted customresourcedefinition.apiextensions.k8s.io "virtualservers.k8s.nginx.org" deleted ``` **Note:** If you run multiple NGINX Ingress Controller releases in your cluster with custom resources enabled, every release will share a single version of the CRDs. When uninstalling a release, ensure that you don’t remove the CRDs until there are no other NGINX Ingress Controller releases running in the cluster. The [Run multiple NGINX Ingress Controllers](/nic/install/multiple-controllers.md) topic has more details. ## Remove secrets If your deployment used NGINX Plus, you should also remove the secrets created for your license and the F5 registry. ```shell kubectl delete secret nplus-license ``` #### Example output ```text secret "nplus-license" deleted ``` ```shell kubectl delete secret regcred ``` #### Example output ```text secret "regcred" deleted ``` -------------------------------------------------------------------------------- title: "Use Helm to uninstall NGINX Ingress Controller LTS" url: https://docs.nginx.com/nginx-ingress-controller/lts/install/helm/uninstall/index.md -------------------------------------------------------------------------------- This page describes how to use Helm to uninstall F5 NGINX Ingress Controller LTS. It explains how to remove the chart, then remove the custom resource definitions (CRDs). By following these instructions, you will remove NGINX Ingress Controller LTS from your Kubernetes cluster. ## Remove the Helm chart To uninstall NGINX Ingress Controller LTS, you must first remove the chart. To remove a release named **\**, use the following command: ```shell helm uninstall ``` The command removes all the Kubernetes components associated with the release, then deletes the release itself. ## Remove the CRDs After removing the release, pull the chart sources: ```shell helm pull oci://ghcr.io/nginx/charts/nginx-ingress --untar --version ``` **warning:** The next command will delete all corresponding custom resources in your cluster across all namespaces. Before using it, check there are no custom resources that you want to keep, and that there are no other NGINX Ingress Controller LTS instances running in the cluster. Then use _kubectl_ to delete the CRDs: ```shell kubectl delete -f crds/ ``` #### Example output ```text customresourcedefinition.apiextensions.k8s.io "aplogconfs.appprotect.f5.com" deleted customresourcedefinition.apiextensions.k8s.io "appolicies.appprotect.f5.com" deleted customresourcedefinition.apiextensions.k8s.io "apusersigs.appprotect.f5.com" deleted customresourcedefinition.apiextensions.k8s.io "apdoslogconfs.appprotectdos.f5.com" deleted customresourcedefinition.apiextensions.k8s.io "apdospolicies.appprotectdos.f5.com" deleted customresourcedefinition.apiextensions.k8s.io "dosprotectedresources.appprotectdos.f5.com" deleted customresourcedefinition.apiextensions.k8s.io "dnsendpoints.externaldns.nginx.org" deleted customresourcedefinition.apiextensions.k8s.io "globalconfigurations.k8s.nginx.org" deleted customresourcedefinition.apiextensions.k8s.io "policies.k8s.nginx.org" deleted customresourcedefinition.apiextensions.k8s.io "transportservers.k8s.nginx.org" deleted customresourcedefinition.apiextensions.k8s.io "virtualserverroutes.k8s.nginx.org" deleted customresourcedefinition.apiextensions.k8s.io "virtualservers.k8s.nginx.org" deleted ``` **caution:** If you run multiple NGINX Ingress Controller LTS releases in your cluster with custom resources enabled, every release will share a single version of the CRDs. When uninstalling a release, ensure that you don’t remove the CRDs until there are no other NGINX Ingress Controller LTS releases running in the cluster. The [Run multiple NGINX Ingress Controller LTSs](/nic/lts/install/multiple-controllers.md) topic has more details. ## Remove secrets If your deployment used NGINX Plus, you should also remove the secrets created for your license and the F5 registry. ```shell kubectl delete secret nplus-license ``` #### Example output ```text secret "nplus-license" deleted ``` ```shell kubectl delete secret regcred ``` #### Example output ```text secret "regcred" deleted ``` -------------------------------------------------------------------------------- title: "Cross-namespace configuration" url: https://docs.nginx.com/nginx-ingress-controller/configuration/ingress-resources/cross-namespace-configuration/index.md -------------------------------------------------------------------------------- This topic explains how to spread Ingress configuration across different namespaces in F5 NGINX Ingress Controller. You can spread the Ingress configuration for a common host across multiple Ingress resources using Mergeable Ingress resources. Such resources can belong to the *same* or *different* namespaces. This enables easier management when using a large number of paths. See the [Mergeable Ingress Resources](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/mergeable-ingress-types) example in our GitHub repo. When [`-allow-empty-ingress-host`](/nic/configuration/global-configuration/command-line-arguments.md#cmdoption-allow-empty-ingress-host) is enabled, this also applies to Ingress resources without a `host`. As an alternative to Mergeable Ingress resources, you can use [VirtualServer and VirtualServerRoute resources](/nic//configuration/virtualserver-and-virtualserverroute-resources.md) for cross-namespace configuration. See the [Cross-Namespace Configuration](https://github.com/nginx/kubernetes-ingress/tree/v/examples/custom-resources/cross-namespace-configuration) example in our GitHub repo. -------------------------------------------------------------------------------- title: "Cross-namespace configuration" url: https://docs.nginx.com/nginx-ingress-controller/lts/configuration/ingress-resources/cross-namespace-configuration/index.md -------------------------------------------------------------------------------- This topic explains how to spread Ingress configuration across different namespaces in F5 NGINX Ingress Controller LTS. You can spread the Ingress configuration for a common host across multiple Ingress resources using Mergeable Ingress resources. Such resources can belong to the *same* or *different* namespaces. This enables easier management when using a large number of paths. See the [Mergeable Ingress Resources](https://github.com/nginx/kubernetes-ingress/tree/release-2026-lts/examples/ingress-resources/mergeable-ingress-types) example in our GitHub repo. As an alternative to Mergeable Ingress resources, you can use [VirtualServer and VirtualServerRoute resources](/nic/lts/configuration/virtualserver-and-virtualserverroute-resources.md) for cross-namespace configuration. See the [Cross-Namespace Configuration](https://github.com/nginx/kubernetes-ingress/tree/release-2026-lts/examples/custom-resources/cross-namespace-configuration) example in our GitHub repo. -------------------------------------------------------------------------------- title: "Custom templates" url: https://docs.nginx.com/nginx-ingress-controller/configuration/global-configuration/custom-templates/index.md -------------------------------------------------------------------------------- F5 NGINX Ingress Controller uses templates to generate NGINX configuration for Ingress resources, VirtualServer resources and the main NGINX configuration file. You can customize the templates and apply them via the ConfigMap. See the [corresponding example](https://github.com/nginx/kubernetes-ingress/tree/v/examples/shared-examples/custom-templates). -------------------------------------------------------------------------------- title: "Custom templates" url: https://docs.nginx.com/nginx-ingress-controller/lts/configuration/global-configuration/custom-templates/index.md -------------------------------------------------------------------------------- F5 NGINX Ingress Controller LTS uses templates to generate NGINX configuration for Ingress resources, VirtualServer resources and the main NGINX configuration file. You can customize the templates and apply them via the ConfigMap. See the [corresponding example](https://github.com/nginx/kubernetes-ingress/tree/release-2026-lts/examples/shared-examples/custom-templates). -------------------------------------------------------------------------------- title: "Install NGINX Ingress Controller LTS with Manifests" url: https://docs.nginx.com/nginx-ingress-controller/lts/install/manifests/index.md -------------------------------------------------------------------------------- This guide explains how to use Manifests to install F5 NGINX Ingress Controller LTS, then create both common and custom resources and set up role-based access control. ## Before you begin If you are using NGINX Plus, get the NGINX Ingress Controller LTS JWT and [create a license secret](/nic/lts/install/license-secret.md). ### Get the NGINX Controller Image **note:** We recommend installing the latest LTS patch release of NGINX Ingress Controller LTS, available on the GitHub repository's [releases page](https://github.com/nginx/kubernetes-ingress/releases). Choose one of the following methods to get the NGINX Ingress Controller LTS image: - **NGINX Plus Ingress Controller**: You have two options for this, both requiring an NGINX Ingress Controller LTS subscription. - - [Download NGINX Ingress Controller LTS from the F5 Registry](/nic/lts/install/images/registry-download.md) topic. - - [Add an NGINX Ingress Controller LTS image to your cluster](/nic/lts/install/images/add-image-to-cluster.md) ### Clone the repository Clone the NGINX Ingress Controller LTS repository using the command shown below, and replace `` with the specific release you want to use. ```shell git clone https://github.com/nginx/kubernetes-ingress.git --branch ``` For example, if you want to use version , the command would be: ```shell git clone https://github.com/nginx/kubernetes-ingress.git --branch v ``` This guide assumes you are using the latest release. Change the active directory. ```shell cd kubernetes-ingress ``` ## Set up role-based access control (RBAC) {#configure-rbac} **Note:** To complete these steps you need admin access to your cluster. Refer to your Kubernetes platform's documentation to set up admin access. For Google Kubernetes Engine (GKE), you can refer to their [Role-Based Access Control guide](https://cloud.google.com/kubernetes-engine/docs/how-to/role-based-access-control). 1. Create a namespace and a service account: ```shell kubectl apply -f deployments/common/ns-and-sa.yaml ``` 2. Create a cluster role and binding for the service account: ```shell kubectl apply -f deployments/rbac/rbac.yaml ```
If you're planning to use F5 WAF for NGINX or F5 DoS for NGINX, additional roles and bindings are needed. 1. (F5 WAF for NGINX only) Create the *App Protect* role and binding: ```shell kubectl apply -f deployments/rbac/ap-rbac.yaml ``` 2. (F5 DoS for NGINX only) Create the *App Protect DoS* role and binding: ```shell kubectl apply -f deployments/rbac/apdos-rbac.yaml ``` ## Create common resources {#create-common-resources} In this section, you'll create resources that most NGINX Ingress Controller installations require: 1. (Optional) Create a secret for the default NGINX server's TLS certificate and key. Complete this step only if you're using the [default server TLS secret](/nic/configuration/global-configuration/command-line-arguments#cmdoption-default-server-tls-secret.md) command-line argument. If you're not, feel free to skip this step. By default, the server returns a _404 Not Found_ page for all requests when no ingress rules are set up. Although we provide a self-signed certificate and key for testing purposes, we recommend using your own certificate. ```shell make secrets kubectl apply -f examples/shared-examples/default-server-secret/default-server-secret.yaml ``` 2. Create a ConfigMap to customize your NGINX settings: ```shell kubectl apply -f deployments/common/nginx-config.yaml ``` 3. Create an `IngressClass` resource. NGINX Ingress Controller won't start without an `IngressClass` resource. ```shell kubectl apply -f deployments/common/ingress-class.yaml ``` If you want to make this NGINX Ingress Controller instance your cluster's default, uncomment the `ingressclass.kubernetes.io/is-default-class` annotation. This action will auto-assign `IngressClass` to new ingresses that don't specify an `ingressClassName`. ## Deploy NGINX Ingress Controller LTS {#deploy-ingress-controller} You have three options for deploying NGINX Ingress Controller LTS: - **Deployment**. Choose this method for the flexibility to dynamically change the number of NGINX Ingress Controller LTS replicas. - **DaemonSet**. Choose this method if you want NGINX Ingress Controller LTS to run on all nodes or a subset of nodes. - **StatefulSet**. Choose this method when you need stable, persistent storage and ordered deployment/scaling for your NGINX Ingress Controller LTS pods. Before you start, update the [command-line arguments](/nic/lts/configuration/global-configuration/command-line-arguments.md) for the NGINX Ingress Controller LTS container in the relevant manifest file to meet your specific requirements. ### Using a Deployment For additional context on managing containers using Kubernetes Deployments, refer to the official Kubernetes [Deployments](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) documentation. When you deploy NGINX Ingress Controller as a Deployment, Kubernetes automatically sets up a single NGINX Ingress Controller pod. - For NGINX, run: ```shell kubectl apply -f deployments/deployment/nginx-ingress.yaml ``` - For NGINX Plus, run: ```shell kubectl apply -f deployments/deployment/nginx-plus-ingress.yaml ``` Update the `nginx-plus-ingress.yaml` file to include your chosen image from the F5 Container registry or your custom container image. ### Using a DaemonSet For additional context on managing containers using Kubernetes DaemonSets, refer to the official Kubernetes [DaemonSets](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) documentation. When you deploy NGINX Ingress Controller as a DaemonSet, Kubernetes creates an Ingress Controller pod on every node in the cluster. - For NGINX, run: ```shell kubectl apply -f deployments/daemon-set/nginx-ingress.yaml ``` - For NGINX Plus, run: ```shell kubectl apply -f deployments/daemon-set/nginx-plus-ingress.yaml ``` Update the `nginx-plus-ingress.yaml` file to include your chosen image from the F5 Container registry or your custom container image. ### Using a StatefulSet For additional context on managing containers using Kubernetes StatefulSets, refer to the official Kubernetes [StatefulSets](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/) documentation. When you deploy NGINX Ingress Controller as a StatefulSet, Kubernetes creates pods with stable network identities and persistent storage. - For NGINX, run: ```shell kubectl apply -f deployments/stateful-set/nginx-ingress.yaml ``` - For NGINX Plus, run: ```shell kubectl apply -f deployments/stateful-set/nginx-plus-ingress.yaml ``` Update the `nginx-plus-ingress.yaml` file to include your chosen image from the F5 Container registry or your custom container image. **Note:** StatefulSets include persistent volume claims for nginx cache storage via `volumeClaimTemplates`. You may need to configure a StorageClass in your cluster or modify the volumeClaimTemplates section in the manifest to match your storage requirements. Other volumes (like those needed for App Protect modules) are configured in the regular `volumes` section, not in volumeClaimTemplates. ## Confirm NGINX Ingress Controller LTS is running To confirm the NGINX Ingress Controller pods are operational, run: ```shell kubectl get pods --namespace=nginx-ingress ``` ## How to access NGINX Ingress Controller LTS ### Using a Deployment or StatefulSet For Deployments and StatefulSets, you have two options for accessing NGINX Ingress Controller LTS pods. #### Option 1: Create a NodePort service For more information about the _NodePort_ service, refer to the [Kubernetes documentation](https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport). 1. To create a service of type *NodePort*, run: ```shell kubectl create -f deployments/service/nodeport.yaml ``` Kubernetes automatically allocates two ports on every node in the cluster. You can access NGINX Ingress Controller LTS by combining any node's IP address with these ports. #### Option 2: Create a LoadBalancer service For more information about the _LoadBalancer_ service, refer to the [Kubernetes documentation](https://kubernetes.io/docs/concepts/services-networking/service/#type-loadbalancer). 1. To set up a _LoadBalancer_ service, run one of the following commands based on your cloud provider: - GCP or Azure: ```shell kubectl apply -f deployments/service/loadbalancer.yaml ``` - AWS: ```shell kubectl apply -f deployments/service/loadbalancer-aws-elb.yaml ``` For more details see service guide [here](https://kubernetes-sigs.github.io/aws-load-balancer-controller/latest/guide/service/annotations/#annotations) 2. AWS users: Follow these additional steps to work with ELB in TCP mode. - Add the following keys to the `nginx-config.yaml` ConfigMap file, which you created in the [Create common resources](#create-common-resources) section. ```yaml proxy-protocol: "True" real-ip-header: "proxy_protocol" set-real-ip-from: "0.0.0.0/0" ``` - Update the ConfigMap: ```shell kubectl apply -f deployments/common/nginx-config.yaml ``` **note:** AWS users have more customization options for their load balancers. These include choosing the load balancer type and configuring SSL termination. Refer to the [Kubernetes documentation](https://kubernetes.io/docs/concepts/services-networking/service/#type-loadbalancer) to learn more. 3. To access NGINX Ingress Controller LTS, get the public IP of your load balancer. - For GCP or Azure, run: ```shell kubectl get svc nginx-ingress --namespace=nginx-ingress ``` - For AWS find the DNS name: ```shell kubectl describe svc nginx-ingress --namespace=nginx-ingress ``` Resolve the DNS name into an IP address using `nslookup`: ```shell nslookup ``` You can also find more details about the public IP in the status section of an ingress resource. For more details, refer to the [Reporting Resources Status doc](/nic/lts/configuration/global-configuration/reporting-resources-status.md). ### Using a DaemonSet Connect to ports 80 and 443 using the IP address of any node in the cluster where NGINX Ingress Controller LTS is running. ## Uninstall NGINX Ingress Controller LTS **warning:** Proceed with caution when performing these steps, as they will remove NGINX Ingress Controller LTS and all related resources, potentially affecting your running services. 1. **Delete the nginx-ingress namespace**: To remove NGINX Ingress Controller LTS and all auxiliary resources, run: ```shell kubectl delete namespace nginx-ingress ``` 1. **Remove the cluster role and cluster role binding**: ```shell kubectl delete clusterrole nginx-ingress kubectl delete clusterrolebinding nginx-ingress ``` 1. **Delete the Custom Resource Definitions**: #### Deleting CRDs from single YAML Delete core custom resource definitions: ```shell kubectl delete -f https://raw.githubusercontent.com/nginx/kubernetes-ingress/v/deploy/crds.yaml ``` #### Deleting CRDs after cloning the repo Delete core custom resource definitions: ```shell kubectl delete -f config/crd/bases/crds.yaml ``` -------------------------------------------------------------------------------- title: "Install NGINX Ingress Controller with Manifests" url: https://docs.nginx.com/nginx-ingress-controller/install/manifests/index.md -------------------------------------------------------------------------------- This guide explains how to use Manifests to install F5 NGINX Ingress Controller, then create both common and custom resources and set up role-based access control. ## Before you begin If you are using NGINX Plus, get the NGINX Ingress Controller JWT and [create a license secret](/nic/install/license-secret.md). ### Get the NGINX Controller Image **Note:** We recommend the most recent stable version of NGINX Ingress Controller, available on the GitHub repository's [releases page](https://github.com/nginx/kubernetes-ingress/releases). Choose one of the following methods to get the NGINX Ingress Controller image: - **NGINX Ingress Controller**: Download the image `nginx/nginx-ingress` from [DockerHub](https://hub.docker.com/r/nginx/nginx-ingress). - **NGINX Plus Ingress Controller**: You have two options for this, both requiring an NGINX Ingress Controller subscription. - - [Download NGINX Ingress Controller from the F5 Registry](/nic/install/images/registry-download.md) topic. - - [Add an NGINX Ingress Controller image to your cluster](/nic/install/images/add-image-to-cluster.md) - **Build your own image**: To build your own image, follow the [Build NGINX Ingress Controller](/nic/install/build.md) topic. ### Clone the repository Clone the NGINX Ingress Controller repository using the command shown below, and replace `` with the specific release you want to use. ```shell git clone https://github.com/nginx/kubernetes-ingress.git --branch ``` For example, if you want to use version , the command would be: ```shell git clone https://github.com/nginx/kubernetes-ingress.git --branch v ``` This guide assumes you are using the latest release. Change the active directory. ```shell cd kubernetes-ingress ``` ### App Protect DoS To use App Protect DoS, install the App Protect DoS Arbitrator using the provided manifests in the same namespace as the NGINX Ingress Controller. If you install multiple NGINX Ingress Controllers in the same namespace, they will need to share the same Arbitrator because there can only be one Arbitrator in a single namespace. ## Set up role-based access control (RBAC) {#configure-rbac} **Note:** To complete these steps you need admin access to your cluster. Refer to your Kubernetes platform's documentation to set up admin access. For Google Kubernetes Engine (GKE), you can refer to their [Role-Based Access Control guide](https://cloud.google.com/kubernetes-engine/docs/how-to/role-based-access-control). 1. Create a namespace and a service account: ```shell kubectl apply -f deployments/common/ns-and-sa.yaml ``` 2. Create a cluster role and binding for the service account: ```shell kubectl apply -f deployments/rbac/rbac.yaml ```
If you're planning to use F5 WAF for NGINX or F5 DoS for NGINX, additional roles and bindings are needed. 1. (F5 WAF for NGINX only) Create the *App Protect* role and binding: ```shell kubectl apply -f deployments/rbac/ap-rbac.yaml ``` 2. (F5 DoS for NGINX only) Create the *App Protect DoS* role and binding: ```shell kubectl apply -f deployments/rbac/apdos-rbac.yaml ``` ## Create common resources {#create-common-resources} In this section, you'll create resources that most NGINX Ingress Controller installations require: 1. (Optional) Create a secret for the default NGINX server's TLS certificate and key. Complete this step only if you're using the [default server TLS secret](/nic/configuration/global-configuration/command-line-arguments#cmdoption-default-server-tls-secret.md) command-line argument. If you're not, feel free to skip this step. By default, the server returns a _404 Not Found_ page for all requests when no ingress rules are set up. Although we provide a self-signed certificate and key for testing purposes, we recommend using your own certificate. ```shell make secrets kubectl apply -f examples/shared-examples/default-server-secret/default-server-secret.yaml ``` 2. Create a ConfigMap to customize your NGINX settings: ```shell kubectl apply -f deployments/common/nginx-config.yaml ``` 3. Create an `IngressClass` resource. NGINX Ingress Controller won't start without an `IngressClass` resource. ```shell kubectl apply -f deployments/common/ingress-class.yaml ``` If you want to make this NGINX Ingress Controller instance your cluster's default, uncomment the `ingressclass.kubernetes.io/is-default-class` annotation. This action will auto-assign `IngressClass` to new ingresses that don't specify an `ingressClassName`. ## Create core custom resources {#create-custom-resources} To make sure your NGINX Ingress Controller pods reach the `Ready` state, you'll need to create custom resource definitions (CRDs) for various components. Alternatively, you can disable this requirement by setting the `-enable-custom-resources` command-line argument to `false`. There are two ways you can install the custom resource definitions: 1. Using a URL to apply a single CRD yaml file, which we recommend. 1. Applying your local copy of the CRD yaml files, which requires you to clone the repository. The core custom CRDs are the following: - [VirtualServer and VirtualServerRoute](/nic/configuration/virtualserver-and-virtualserverroute-resources.md) - [TransportServer](/nic/configuration/transportserver-resource.md) - [Policy](/nic/configuration/policy-resource/before-you-begin.md) - [GlobalConfiguration](/nic/configuration/global-configuration/globalconfiguration-resource.md) #### Install CRDs from single YAML ```shell kubectl apply -f https://raw.githubusercontent.com/nginx/kubernetes-ingress/v/deploy/crds.yaml ``` #### Install CRDs after cloning the repo **Note:** Read the steps outlined in [Upgrade from 3.x to 4.x](/nic/install/upgrade.md#upgrade-from-3x-to-4x) before running the CRD upgrade and perform the steps if applicable. ```shell kubectl apply -f config/crd/bases/k8s.nginx.org_virtualservers.yaml kubectl apply -f config/crd/bases/k8s.nginx.org_virtualserverroutes.yaml kubectl apply -f config/crd/bases/k8s.nginx.org_transportservers.yaml kubectl apply -f config/crd/bases/k8s.nginx.org_policies.yaml kubectl apply -f config/crd/bases/k8s.nginx.org_globalconfigurations.yaml ``` ### Create optional custom resources There are optional CRDs that are necessary if you want to use F5 WAF for NGINX or F5 DoS for NGINX. _F5 WAF for NGINX_: - `APPolicy` - `APLogConf` - `APUserSig` _F5 DoS for NGINX_: - `APDosPolicy` - `APDosLogConf` - `DosProtectedResource` #### Install CRDs from single YAML _F5 WAF for NGINX_: **Note:** This step can be skipped if you are using F5 WAF for NGINX module with policy bundles. ```shell kubectl apply -f https://raw.githubusercontent.com/nginx/kubernetes-ingress/v/deploy/crds-nap-waf.yaml ``` _F5 DoS for NGINX_: ```shell kubectl apply -f https://raw.githubusercontent.com/nginx/kubernetes-ingress/v/deploy/crds-nap-dos.yaml ``` #### Install CRDs after cloning the repo _F5 WAF for NGINX_: **Note:** This step can be skipped if you are using F5 WAF for NGINX module with policy bundles. ```shell kubectl apply -f config/crd/bases/appprotect.f5.com_aplogconfs.yaml kubectl apply -f config/crd/bases/appprotect.f5.com_appolicies.yaml kubectl apply -f config/crd/bases/appprotect.f5.com_apusersigs.yaml ``` _F5 DoS for NGINX_: ```shell kubectl apply -f config/crd/bases/appprotectdos.f5.com_apdoslogconfs.yaml kubectl apply -f config/crd/bases/appprotectdos.f5.com_apdospolicy.yaml kubectl apply -f config/crd/bases/appprotectdos.f5.com_dosprotectedresources.yaml ``` ## Deploy NGINX Ingress Controller {#deploy-ingress-controller} You have three options for deploying NGINX Ingress Controller: - **Deployment**. Choose this method for the flexibility to dynamically change the number of NGINX Ingress Controller replicas. - **DaemonSet**. Choose this method if you want NGINX Ingress Controller to run on all nodes or a subset of nodes. - **StatefulSet**. Choose this method when you need stable, persistent storage and ordered deployment/scaling for your NGINX Ingress Controller pods. Before you start, update the [command-line arguments](/nic/configuration/global-configuration/command-line-arguments.md) for the NGINX Ingress Controller container in the relevant manifest file to meet your specific requirements. ### Using a Deployment For additional context on managing containers using Kubernetes Deployments, refer to the official Kubernetes [Deployments](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) documentation. When you deploy NGINX Ingress Controller as a Deployment, Kubernetes automatically sets up a single NGINX Ingress Controller pod. - For NGINX, run: ```shell kubectl apply -f deployments/deployment/nginx-ingress.yaml ``` - For NGINX Plus, run: ```shell kubectl apply -f deployments/deployment/nginx-plus-ingress.yaml ``` Update the `nginx-plus-ingress.yaml` file to include your chosen image from the F5 Container registry or your custom container image. ### Using a DaemonSet For additional context on managing containers using Kubernetes DaemonSets, refer to the official Kubernetes [DaemonSets](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) documentation. When you deploy NGINX Ingress Controller as a DaemonSet, Kubernetes creates an Ingress Controller pod on every node in the cluster. - For NGINX, run: ```shell kubectl apply -f deployments/daemon-set/nginx-ingress.yaml ``` - For NGINX Plus, run: ```shell kubectl apply -f deployments/daemon-set/nginx-plus-ingress.yaml ``` Update the `nginx-plus-ingress.yaml` file to include your chosen image from the F5 Container registry or your custom container image. ### Using a StatefulSet For additional context on managing containers using Kubernetes StatefulSets, refer to the official Kubernetes [StatefulSets](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/) documentation. When you deploy NGINX Ingress Controller as a StatefulSet, Kubernetes creates pods with stable network identities and persistent storage. - For NGINX, run: ```shell kubectl apply -f deployments/stateful-set/nginx-ingress.yaml ``` - For NGINX Plus, run: ```shell kubectl apply -f deployments/stateful-set/nginx-plus-ingress.yaml ``` Update the `nginx-plus-ingress.yaml` file to include your chosen image from the F5 Container registry or your custom container image. **Note:** StatefulSets include persistent volume claims for nginx cache storage via `volumeClaimTemplates`. You may need to configure a StorageClass in your cluster or modify the volumeClaimTemplates section in the manifest to match your storage requirements. Other volumes (like those needed for App Protect modules) are configured in the regular `volumes` section, not in volumeClaimTemplates. ## Confirm NGINX Ingress Controller is running To confirm the NGINX Ingress Controller pods are operational, run: ```shell kubectl get pods --namespace=nginx-ingress ``` ## How to access NGINX Ingress Controller ### Using a Deployment or StatefulSet For Deployments and StatefulSets, you have two options for accessing NGINX Ingress Controller pods. #### Option 1: Create a NodePort service For more information about the _NodePort_ service, refer to the [Kubernetes documentation](https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport). 1. To create a service of type *NodePort*, run: ```shell kubectl create -f deployments/service/nodeport.yaml ``` Kubernetes automatically allocates two ports on every node in the cluster. You can access NGINX Ingress Controller by combining any node's IP address with these ports. #### Option 2: Create a LoadBalancer service For more information about the _LoadBalancer_ service, refer to the [Kubernetes documentation](https://kubernetes.io/docs/concepts/services-networking/service/#type-loadbalancer). 1. To set up a _LoadBalancer_ service, run one of the following commands based on your cloud provider: - GCP or Azure: ```shell kubectl apply -f deployments/service/loadbalancer.yaml ``` - AWS: ```shell kubectl apply -f deployments/service/loadbalancer-aws-elb.yaml ``` For more details see service guide [here](https://kubernetes-sigs.github.io/aws-load-balancer-controller/latest/guide/service/annotations/#annotations) 2. AWS users: Follow these additional steps to work with ELB in TCP mode. - Add the following keys to the `nginx-config.yaml` ConfigMap file, which you created in the [Create common resources](#create-common-resources) section. ```yaml proxy-protocol: "True" real-ip-header: "proxy_protocol" set-real-ip-from: "0.0.0.0/0" ``` - Update the ConfigMap: ```shell kubectl apply -f deployments/common/nginx-config.yaml ``` **Note:** AWS users have more customization options for their load balancers. These include choosing the load balancer type and configuring SSL termination. Refer to the [Kubernetes documentation](https://kubernetes.io/docs/concepts/services-networking/service/#type-loadbalancer) to learn more. 3. To access NGINX Ingress Controller, get the public IP of your load balancer. - For GCP or Azure, run: ```shell kubectl get svc nginx-ingress --namespace=nginx-ingress ``` - For AWS find the DNS name: ```shell kubectl describe svc nginx-ingress --namespace=nginx-ingress ``` Resolve the DNS name into an IP address using `nslookup`: ```shell nslookup ``` You can also find more details about the public IP in the status section of an ingress resource. For more details, refer to the [Reporting Resources Status doc](/nic/configuration/global-configuration/reporting-resources-status.md). ### Using a DaemonSet Connect to ports 80 and 443 using the IP address of any node in the cluster where NGINX Ingress Controller is running. ## Uninstall NGINX Ingress Controller **Note:** Proceed with caution when performing these steps, as they will remove NGINX Ingress Controller and all related resources, potentially affecting your running services. 1. **Delete the nginx-ingress namespace**: To remove NGINX Ingress Controller and all auxiliary resources, run: ```shell kubectl delete namespace nginx-ingress ``` 1. **Remove the cluster role and cluster role binding**: ```shell kubectl delete clusterrole nginx-ingress kubectl delete clusterrolebinding nginx-ingress ``` 1. **Delete the Custom Resource Definitions**: #### Deleting CRDs from single YAML Delete core custom resource definitions: ```shell kubectl delete -f https://raw.githubusercontent.com/nginx/kubernetes-ingress/v/deploy/crds.yaml ``` Delete custom resource definitions for the F5 WAF for NGINX module: ```shell kubectl delete -f https://raw.githubusercontent.com/nginx/kubernetes-ingress/v/deploy/crds-nap-waf.yaml ``` Delete custom resource definitions for the F5 DoS for NGINX module: ```shell kubectl delete -f https://raw.githubusercontent.com/nginx/kubernetes-ingress/v/deploy/crds-nap-dos.yaml ``` #### Deleting CRDs after cloning the repo Delete core custom resource definitions: ```shell kubectl delete -f config/crd/bases/crds.yaml ``` Delete custom resource definitions for the F5 WAF for NGINX module: ```shell kubectl delete -f config/crd/bases/crds-nap-waf.yaml ``` Delete custom resource definitions for the F5 DoS for NGINX module: ```shell kubectl delete -f config/crd/bases/crds-nap-dos.yaml ``` -------------------------------------------------------------------------------- title: "Policy precedence and override rules" url: https://docs.nginx.com/nginx-ingress-controller/configuration/policy-resource/precedence-and-overrides/index.md -------------------------------------------------------------------------------- When more than one policy of the same type applies to a resource, NGINX Ingress Controller uses precedence rules to decide which policy takes effect. ## VirtualServer and VirtualServerRoute Policy precedence goes from broader scope to narrower scope: - `VirtualServer.spec.policies` - `VirtualServer.route.policies` - `VirtualServerRoute.subroute.policies` If the same policy type appears at multiple levels, the more specific level wins. For example: - Route-level `accessControl` overrides spec-level `accessControl`. - Subroute-level `cors` overrides route-level `cors`. ## Ingress and mergeable Ingress For Ingress: - Policies apply to the whole Ingress. - With mergeable Ingress, minion policies override master policies of the same type. ## What's next See the [policy type reference](/nic/configuration/policy-resource/policy-reference.md) for the fields and merging behavior of each policy type. -------------------------------------------------------------------------------- title: "Product telemetry" url: https://docs.nginx.com/nginx-ingress-controller/lts/overview/product-telemetry/index.md -------------------------------------------------------------------------------- Learn why, what and how F5 NGINX Ingress Controller LTS collects telemetry. --- ## Overview NGINX Ingress Controller LTS collects product telemetry data to allow its developers to understand how it's deployed and configured by users. This data is used to triage development work, prioritizing features and functionality that will benefit the most people. Product telemetry is enabled by default, collected once every 24 hours. It's then sent over HTTPS to a service managed by F5 at `oss.edge.df.f5.com`. **note:** If you would prefer not to send any telemetry data, you can [opt-out](#opt-out) when installing NGINX Ingress Controller LTS. --- ## Data collected These are the data points collected and reported by NGINX Ingress Controller LTS: - **Project Name** The name of the software, which will be labeled `NIC`. - **Project Version** NGINX Ingress Controller LTS version. - **Project Architecture** The architecture of the kubernetes environment. (e.g. amd64, arm64, etc...) - **Cluster ID** A unique identifier of the kubernetes cluster that NGINX Ingress Controller LTS is deployed to. - **Cluster Version** The version of the Kubernetes cluster. - **Cluster Platform** The platform that the kubernetes cluster is operating on. (e.g. eks, aks, etc...) - **Cluster Node Count** The number of nodes in the cluster that NGINX Ingress Controller LTS is deployed to. - **Installation ID** Used to identify a unique installation of NGINX Ingress Controller LTS. - **VirtualServers** The number of VirtualServer resources managed by NGINX Ingress Controller LTS. - **VirtualServerRoutes** The number of VirtualServerRoute resources managed by NGINX Ingress Controller LTS. - **TransportServers** The number of TransportServer resources managed by NGINX Ingress Controller LTS. - **Replicas** Number of Deployment or StatefulSet replicas, or DaemonSet instances. - **Secrets** Number of Secret resources managed by NGINX Ingress Controller LTS. - **ClusterIPServices** Number of ClusterIP Services managed by NGINX Ingress Controller LTS. - **NodePortServices** Number of NodePort Services managed by NGINX Ingress Controller LTS. - **LoadBalancerServices** Number of LoadBalancer Services managed by NGINX Ingress Controller LTS. - **ExternalNameServices** Number of ExternalName Services managed by NGINX Ingress Controller LTS. - **RegularIngressCount** The number of Regular Ingress resources managed by NGINX Ingress Controller LTS. - **MasterIngressCount** The number of Master Ingress resources managed by NGINX Ingress Controller LTS. - **MinionIngressCount** The number of Minion Ingress resources managed by NGINX Ingress Controller LTS. - **IngressClasses** Number of Ingress Classes in the cluster. - **IngressAnnotations** List of Ingress annotations managed by NGINX Ingress Controller LTS - **AccessControlPolicies** Number of AccessControl policies. - **RateLimitPolicies** Number of RateLimit policies. - **APIKeyPolicies** Number of API Key Auth policies. - **JWTAuthPolicies** Number of JWTAuth policies. - **BasicAuthPolicies** Number of BasicAuth policies. - **IngressMTLSPolicies** Number of IngressMTLS policies. - **EgressMTLSPolicies** Number of EgressMTLS policies. - **OIDCPolicies** Number of OIDC policies. - **WAFPolicies** Number of WAF policies. - **CachePolicies** Number of Cache policies. - **CORSPolicies** Number of CORS policies. - **GlobalConfiguration** Represents the use of a GlobalConfiguration resource. - **AppProtectVersion** The AppProtect version - **IsPlus** Represents whether NGINX is Plus or OSS - **InstallationFlags** List of command line arguments configured for NGINX Ingress Controller LTS - **BuildOS** The base operating system image in which NGINX Ingress Controller LTS is running on. - **ConfigMapKeys** List of ConfigMap keys from the main ConfigMap configured for NGINX Ingress Controller LTS. - **MGMTConfigMapKeys** List of MGMTConfigMap keys from the mgmt ConfigMap configured for NGINX Ingress Controller LTS. --- ## Opt out Product telemetry can be disabled when installing NGINX Ingress Controller LTS. ### Helm When installing or upgrading NGINX Ingress Controller LTS with Helm, set the `controller.telemetryReporting.enable` option to `false`. This can be set directly in the `values.yaml` file, or using the `--set` option ```shell helm upgrade --install ... --set controller.telemetryReporting.enable=false ``` --- ### Manifests When installing NGINX Ingress Controller LTS with Manifests, set the `-enable-telemetry-reporting` flag to `false` -------------------------------------------------------------------------------- title: "Product telemetry" url: https://docs.nginx.com/nginx-ingress-controller/overview/product-telemetry/index.md -------------------------------------------------------------------------------- Learn why, what and how F5 NGINX Ingress Controller collects telemetry. --- ## Overview NGINX Ingress Controller collects product telemetry data to allow its developers to understand how it's deployed and configured by users. This data is used to triage development work, prioritizing features and functionality that will benefit the most people. Product telemetry is enabled by default, collected once every 24 hours. It's then sent over HTTPS to a service managed by F5 at `oss.edge.df.f5.com`. **Note:** If you would prefer not to send any telemetry data, you can [opt-out](#opt-out) when installing NGINX Ingress Controller. --- ## Data collected These are the data points collected and reported by NGINX Ingress Controller: - **Project Name** The name of the software, which will be labeled `NIC`. - **Project Version** NGINX Ingress Controller version. - **Project Architecture** The architecture of the kubernetes environment. (e.g. amd64, arm64, etc...) - **Cluster ID** A unique identifier of the kubernetes cluster that NGINX Ingress Controller is deployed to. - **Cluster Version** The version of the Kubernetes cluster. - **Cluster Platform** The platform that the kubernetes cluster is operating on. (e.g. eks, aks, etc...) - **Cluster Node Count** The number of nodes in the cluster that NGINX Ingress Controller is deployed to. - **Installation ID** Used to identify a unique installation of NGINX Ingress Controller. - **VirtualServers** The number of VirtualServer resources managed by NGINX Ingress Controller. - **VirtualServerRoutes** The number of VirtualServerRoute resources managed by NGINX Ingress Controller. - **TransportServers** The number of TransportServer resources managed by NGINX Ingress Controller. - **Replicas** Number of Deployment or StatefulSet replicas, or DaemonSet instances. - **Secrets** Number of Secret resources managed by NGINX Ingress Controller. - **ClusterIPServices** Number of ClusterIP Services managed by NGINX Ingress Controller. - **NodePortServices** Number of NodePort Services managed by NGINX Ingress Controller. - **LoadBalancerServices** Number of LoadBalancer Services managed by NGINX Ingress Controller. - **ExternalNameServices** Number of ExternalName Services managed by NGINX Ingress Controller. - **RegularIngressCount** The number of Regular Ingress resources managed by NGINX Ingress Controller. - **MasterIngressCount** The number of Master Ingress resources managed by NGINX Ingress Controller. - **MinionIngressCount** The number of Minion Ingress resources managed by NGINX Ingress Controller. - **IngressClasses** Number of Ingress Classes in the cluster. - **IngressAnnotations** List of Ingress annotations managed by NGINX Ingress Controller - **AccessControlPolicies** Number of AccessControl policies. - **RateLimitPolicies** Number of RateLimit policies. - **APIKeyPolicies** Number of API Key Auth policies. - **JWTAuthPolicies** Number of JWTAuth policies. - **BasicAuthPolicies** Number of BasicAuth policies. - **IngressMTLSPolicies** Number of IngressMTLS policies. - **EgressMTLSPolicies** Number of EgressMTLS policies. - **OIDCPolicies** Number of OIDC policies. - **WAFPolicies** Number of WAF policies. - **CachePolicies** Number of Cache policies. - **CORSPolicies** Number of CORS policies. - **GlobalConfiguration** Represents the use of a GlobalConfiguration resource. - **AppProtectVersion** The AppProtect version - **IsPlus** Represents whether NGINX is Plus or OSS - **InstallationFlags** List of command line arguments configured for NGINX Ingress Controller - **BuildOS** The base operating system image in which NGINX Ingress Controller is running on. - **ConfigMapKeys** List of ConfigMap keys from the main ConfigMap configured for NGINX Ingress Controller. - **MGMTConfigMapKeys** List of MGMTConfigMap keys from the mgmt ConfigMap configured for NGINX Ingress Controller. --- ## Opt out Product telemetry can be disabled when installing NGINX Ingress Controller. ### Helm When installing or upgrading NGINX Ingress Controller with Helm, set the `controller.telemetryReporting.enable` option to `false`. This can be set directly in the `values.yaml` file, or using the `--set` option ```shell helm upgrade --install ... --set controller.telemetryReporting.enable=false ``` --- ### Manifests When installing NGINX Ingress Controller with Manifests, set the `-enable-telemetry-reporting` flag to `false` -------------------------------------------------------------------------------- title: "Enable Service Insight" url: https://docs.nginx.com/nginx-ingress-controller/logging-and-monitoring/service-insight/index.md -------------------------------------------------------------------------------- The Service Insight feature is available only for F5 NGINX Plus. The F5 NGINX Ingress Controller exposes an endpoint which provides host statistics for services exposed using the VirtualServer (VS) and TransportServer (TS) resources. It exposes data in JSON format and returns HTTP status codes. The response body holds information about the total, down and the unhealthy number of upstream pods associated with the configured hostname. Returned HTTP codes indicate the health of the service. The service is indicated as not healthy (HTTP response code different than 200 OK) if all upstreams (pods) are determined unhealthy by NGINX Plus. The service is healthy if at least one upstream pod is healthy as determined by NGINX Plus. In this case, the endpoint returns HTTP code 200 OK. NGINX Plus determination of healthy can be tuned using advanced health checks, and also dynamically relate to pods responses and responsiveness. See [Upstream Healthcheck](/nic/configuration/virtualserver-and-virtualserverroute-resources.md#upstream) ## Enabling Service Insight Endpoint If you're using *Kubernetes manifests* (Deployment, DaemonSet, or StatefulSet) to install the Ingress Controller, to enable the Service Insight endpoint: 1. Run the Ingress Controller with the `-enable-service-insight` [command-line argument](/nic/configuration/global-configuration/command-line-arguments.md). This will expose the Ingress Controller endpoint via paths `/probe/{hostname}` for Virtual Servers, and `/probe/ts/{service_name}` for Transport Servers on port `9114` (customizable with the `-service-insight-listen-port` command-line argument). The `service_name` parameter refers to the name of the deployed service (the service specified under `upstreams` in the transport server). 1. To enable TLS for the Service Insight endpoint, configure the `-service-insight-tls-secret` cli argument with the namespace and name of a TLS Secret. 1. Add the Service Insight port to the list of the ports of the Ingress Controller container in the template of the Ingress Controller pod: ```yaml - name: service-insight containerPort: 9114 ``` If you're using *Helm* to install the Ingress Controller, to enable Service Insight endpoint, configure the `serviceInsight.*` parameters of the Helm chart. See the [Installation with Helm](/nic/install/helm.md) doc. ## Available Statistics and HTTP Response Codes The Service Insight provides the following statistics: - Total number of VS or TS pods - Number of VS or TS pods in 'Up' state - Number of VS or TS pods in 'Unhealthy' state These statistics are returned as JSON: ```json { "Total": , "Up": , "Unhealthy": } ``` Response codes: - HTTP 200 OK - Service is healthy - HTTP 404 Not Found - No upstreams/VS/TS found for the requested hostname/name - HTTP 418 I'm a teapot - The service is down (All upstreams/VS/TS are "Unhealthy") **Note**: wildcards in hostnames are not supported at the moment. -------------------------------------------------------------------------------- title: "Enable Service Insight" url: https://docs.nginx.com/nginx-ingress-controller/lts/logging-and-monitoring/service-insight/index.md -------------------------------------------------------------------------------- The F5 NGINX Ingress Controller LTS exposes an endpoint which provides host statistics for services exposed using the VirtualServer (VS) and TransportServer (TS) resources. It exposes data in JSON format and returns HTTP status codes. The response body holds information about the total, down and the unhealthy number of upstream pods associated with the configured hostname. Returned HTTP codes indicate the health of the service. The service is indicated as not healthy (HTTP response code different than 200 OK) if all upstreams (pods) are determined unhealthy by NGINX Plus. The service is healthy if at least one upstream pod is healthy as determined by NGINX Plus. In this case, the endpoint returns HTTP code 200 OK. NGINX Plus determination of healthy can be tuned using advanced health checks, and also dynamically relate to pods responses and responsiveness. See [Upstream Healthcheck](/nic/lts/configuration/virtualserver-and-virtualserverroute-resources.md#upstream) ## Enabling Service Insight Endpoint If you're using *Kubernetes manifests* (Deployment, DaemonSet, or StatefulSet) to install the Ingress Controller, to enable the Service Insight endpoint: 1. Run the Ingress Controller with the `-enable-service-insight` [command-line argument](/nic/lts/configuration/global-configuration/command-line-arguments.md). This will expose the Ingress Controller endpoint via paths `/probe/{hostname}` for Virtual Servers, and `/probe/ts/{service_name}` for Transport Servers on port `9114` (customizable with the `-service-insight-listen-port` command-line argument). The `service_name` parameter refers to the name of the deployed service (the service specified under `upstreams` in the transport server). 1. To enable TLS for the Service Insight endpoint, configure the `-service-insight-tls-secret` cli argument with the namespace and name of a TLS Secret. 1. Add the Service Insight port to the list of the ports of the Ingress Controller container in the template of the Ingress Controller pod: ```yaml - name: service-insight containerPort: 9114 ``` If you're using *Helm* to install the Ingress Controller, to enable Service Insight endpoint, configure the `serviceInsight.*` parameters of the Helm chart. See the [Installation with Helm](/nic/lts/install/helm.md) doc. ## Available Statistics and HTTP Response Codes The Service Insight provides the following statistics: - Total number of VS or TS pods - Number of VS or TS pods in 'Up' state - Number of VS or TS pods in 'Unhealthy' state These statistics are returned as JSON: ```json { "Total": , "Up": , "Unhealthy": } ``` Response codes: - HTTP 200 OK - Service is healthy - HTTP 404 Not Found - No upstreams/VS/TS found for the requested hostname/name - HTTP 418 I'm a teapot - The service is down (All upstreams/VS/TS are "Unhealthy") **Note**: wildcards in hostnames are not supported at the moment. -------------------------------------------------------------------------------- title: "Install NGINX Ingress Controller with NGINX Ingress Operator" url: https://docs.nginx.com/nginx-ingress-controller/install/operator/index.md -------------------------------------------------------------------------------- This document explains how to install F5 NGINX Ingress Controller using NGINX Ingress Operator. ## Before you begin If you're using NGINX Plus, get the NGINX Ingress Controller JWT and [create a license secret](/nic/install/license-secret.md). **Note:** We recommend the most recent stable version of NGINX Ingress Controller, available on the GitHub repository's [releases page](https://github.com/nginx/kubernetes-ingress/releases). Make sure you have access to the NGINX Ingress Controller image: - For NGINX Ingress Controller, use the image `nginx/nginx-ingress` from [DockerHub](https://hub.docker.com/r/nginx/nginx-ingress). - For NGINX Plus Ingress Controller, view the [Download NGINX Ingress Controller from the F5 Registry](/nic/install/images/registry-download) topic for details on how to pull the image from the F5 Docker registry. - The [Add an NGINX Ingress Controller image to your cluster](/nic/install/images/add-image-to-cluster.md) topic describes how to use your subscription JWT token to get the image. - The [Build NGINX Ingress Controller](/nic/install/build.md) topic explains how to push an image to a private Docker registry. Install the NGINX Ingress Operator following the [instructions](https://github.com/nginx/nginx-ingress-helm-operator/blob/main/docs/installation.md). Create the SecurityContextConstraint as outlined in the ["Getting Started" instructions](https://github.com/nginx/nginx-ingress-helm-operator/blob/main/README.md#getting-started). **Note:** If you're upgrading your operator installation to a later release, navigate [here](https://github.com/nginx/nginx-ingress-helm-operator/blob/main/helm-charts/nginx-ingress) and run `kubectl apply -f crds/` or `oc apply -f crds/` as a prerequisite ## Create the NGINX Ingress Controller manifest Create a manifest `nginx-ingress-controller.yaml` with the following content: ```yaml apiVersion: charts.nginx.org/v1alpha1 kind: NginxIngress metadata: name: nginxingress-sample namespace: nginx-ingress spec: controller: image: pullPolicy: IfNotPresent repository: nginx/nginx-ingress tag: -ubi ingressClass: name: nginx kind: deployment nginxplus: false replicaCount: 1 serviceAccount: imagePullSecretName: "" ``` **Note:** For NGINX Plus, change the `image.repository` and `image.tag` values and change `nginxPlus` to `True`. If required, set the `serviceAccount.imagePullSecretName` or `serviceAccount.imagePullSecretsNames` to the name of the pre-created docker config secret that should be associated with the ServiceAccount. ## Deploy NGINX Ingress Controller ```shell kubectl apply -f nginx-ingress-controller.yaml ``` A new instance of NGINX Ingress Controller will be deployed by the NGINX Ingress Operator in the `default` namespace with default parameters. To configure other parameters of the NginxIngressController resource, check the [documentation](https://github.com/nginx/nginx-ingress-helm-operator/blob/main/docs/nginx-ingress-controller.md). ## Troubleshooting If you experience an `OOMkilled` error when deploying the NGINX Ingress Operator in a large cluster, it's likely because the Helm operator is caching all Kubernetes objects and using up too much memory. If you encounter this issue, try the following solutions: - Set the operator to only watch one namespace. - If monitoring multiple namespaces is required, consider manually increasing the memory limit for the operator. Keep in mind that this value might be overwritten after a release update. We are working with the OpenShift team to resolve this issue. -------------------------------------------------------------------------------- title: "Policy type reference" url: https://docs.nginx.com/nginx-ingress-controller/configuration/policy-resource/policy-reference/index.md -------------------------------------------------------------------------------- This reference describes the fields and merging behavior for each Policy type. ## AccessControl The access control policy configures NGINX to deny or allow requests from clients with the specified IP addresses or subnets. For example, the following policy allows access for clients from the subnet `10.0.0.0/8` and denies access for any other clients: ```yaml accessControl: allow: - 10.0.0.0/8 ``` In contrast, the following policy does the opposite. It denies access for clients from `10.0.0.0/8` and allows access for any other clients: ```yaml accessControl: deny: - 10.0.0.0/8 ``` **Note:** This feature uses the NGINX [ngx_http_access_module](https://nginx.org/en/docs/http/ngx_http_access_module.html). The NGINX Ingress Controller access control policy supports either allow rules or deny rules, but not both, unlike the module itself. |Field | Description | Type | Required | | ---| ---| ---| --- | |``allow`` | Allows access for the specified networks or addresses. For example, ``192.168.1.1`` or ``10.1.1.0/16``. | ``[]string`` | No | |``deny`` | Denies access for the specified networks or addresses. For example, ``192.168.1.1`` or ``10.1.1.0/16``. | ``[]string`` | No | \* an accessControl must include either `allow` or `deny`. | ### AccessControl merging behavior A VirtualServer or VirtualServerRoute can reference multiple access control policies. For example, this configuration references two policies, each with a configured allow list: ```yaml policies: - name: allow-policy-one - name: allow-policy-two ``` When a resource references more than one access control policy, NGINX Ingress Controller merges the contents into a single allow list or a single deny list. NGINX Ingress Controller doesn't support referencing both allow and deny policies together, as shown in the following example. If a resource references both allow and deny lists, NGINX Ingress Controller uses only the allow list policies. ```yaml policies: - name: deny-policy - name: allow-policy-one - name: allow-policy-two ``` ## RateLimit The rate limit policy configures NGINX to limit the processing rate of requests. For example, the following policy limits all subsequent requests from a single IP address once the rate exceeds 10 requests per second: ```yaml rateLimit: rate: 10r/s zoneSize: 10M key: ${binary_remote_addr} ``` **Note:** This feature uses the NGINX [ngx_http_limit_req_module](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html). **Note:** When you turn on the [zone sync feature](/nic/configuration/global-configuration/configmap-resource.md#zone-sync) with NGINX Plus, NGINX Ingress Controller synchronizes the rate limiting zone across all replicas in the cluster. This means all replicas know about requests that other replicas in the cluster have already rate limited. |Field | Description | Type | Required | | ---| ---| ---| --- | |``rate`` | The rate of requests permitted. The rate is specified in requests per second (r/s) or requests per minute (r/m). | ``string`` | Yes | |``key`` | The key to which the rate limit is applied. Can contain text, variables, or a combination of them. Variables must be surrounded by ``${}``. For example: ``${binary_remote_addr}``. Accepted variables are ``$binary_remote_addr``, ``$request_uri``,``$request_method``, ``$url``, ``$http_``, ``$args``, ``$arg_``, ``$cookie_``, ``$jwt_claim_``. | ``string`` | Yes | |``zoneSize`` | Size of the shared memory zone. Only positive values are allowed. Allowed suffixes are ``k`` or ``m``, if none are present ``k`` is assumed. | ``string`` | Yes | |``delay`` | The delay parameter specifies a limit at which excessive requests become delayed. If not set all excessive requests are delayed. | ``int`` | No | |``noDelay`` | Disables the delaying of excessive requests while requests are being limited. Overrides ``delay`` if both are set. | ``bool`` | No | |``burst`` | Excessive requests are delayed until their number exceeds the ``burst`` size, in which case the request is terminated with an error. | ``int`` | No | |``dryRun`` | Turns on dry run mode. In this mode, NGINX Ingress Controller doesn't apply the rate limit, but it accounts for the number of excessive requests as usual in the shared memory zone. | ``bool`` | No | |``logLevel`` | Sets the desired logging level for cases when the server refuses to process requests due to rate exceeding, or delays request processing. Allowed values are ``info``, ``notice``, ``warn`` or ``error``. Default is ``error``. | ``string`` | No | |``rejectCode`` | Sets the status code to return in response to rejected requests. Must fall into the range ``400..599``. Default is ``503``. | ``int`` | No | |``scale`` | Keeps the rate limit constant by dividing the configured rate by the number of NGINX Ingress Controller pods currently serving traffic. This adjustment keeps the rate limit consistent, even as the number of pods fluctuates due to autoscaling. **This doesn't work correctly if requests from a client aren't distributed evenly across all Ingress Controller pods** (for example, with sticky sessions or long-lived TCP connections with many requests). In these cases, [zone sync](/nic/configuration/global-configuration/configmap-resource.md#zone-sync) gives better results. Turning on `zone-sync` suppresses this setting. | ``bool`` | No | |``condition`` | Add a condition to a rate-limit policy. | [ratelimit.condition](#ratelimitcondition) | No | **Note:** For each policy referenced in a VirtualServer or its VirtualServerRoutes, NGINX Ingress Controller generates a single rate limiting zone defined by the [`limit_req_zone`](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_zone) directive. If two VirtualServer resources reference the same policy, NGINX Ingress Controller generates two different rate limiting zones, one zone per VirtualServer. ### RateLimit merging behavior A VirtualServer or VirtualServerRoute can reference multiple rate limit policies. For example, this configuration references two policies: ```yaml policies: - name: rate-limit-policy-one - name: rate-limit-policy-two ``` When a resource references more than one rate limit policy, NGINX Ingress Controller configures NGINX to use all referenced rate limits. When you define multiple policies, each additional policy inherits the `dryRun`, `logLevel`, and `rejectCode` parameters from the first policy referenced (`rate-limit-policy-one`, in the example above). ### RateLimit.Condition RateLimit.Condition defines a condition for a rate limit policy. For example: ```yaml condition: jwt: claim: user_details.level match: premium default: true ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``jwt`` | defines a JWT condition to rate limit against. | [ratelimit.condition.jwt](#ratelimitconditionjwt) | No | |``variables`` | defines a Variable condition to rate limit against. | [ratelimit.condition.variables](#ratelimitconditionvariables) | No | |``default`` | sets the rate limit in this policy to be the default if no conditions are met. In a group of policies with the same condition, only one policy can be the default. | ``bool`` | No | **Note:** Conditions (`jwt` or `variables`) are optional, but each policy can only have one. If conditions are used and a request doesn't match any of them, NGINX Ingress Controller applies the `default` policy, if one is defined. Otherwise, if no `default` is set, the request isn't rate limited. Combine the rate limit policy with condition with one or more rate limit policies. For example, you can combine multiple rate limit policies that use [RateLimit.Condition.JWT](#ratelimitconditionjwt) to apply different tiers of rate limit based on the value of a JWT claim. For a practical example of tiered rate limiting by the value of a JWT claim, see the example in the [GitHub repository](https://github.com/nginx/kubernetes-ingress/tree/v/examples/custom-resources/rate-limit-tiered-jwt-claim/README.md). ### RateLimit.Condition.JWT **Note:** This feature is only available with NGINX Plus. RateLimit.Condition.JWT defines a condition for a rate limit by JWT claim. For example, the following condition applies a rate limit policy only to requests with a JWT claim `user_details.level` with a value `premium`: ```yaml jwt: claim: user_details.level match: premium ``` The rate limit policy applies only to requests that contain a JWT with the specified claim and value. For example, the following JWT payload matches the JWT condition: ```json { "user_details": { "level": "premium" }, "sub": "client1" } ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``claim`` | Claim is the JWT claim on which to apply the rate limit. Nested claims should be separated by ".". | ``string`` | Yes | |``match`` | the value of the claim to match against. | ``string`` | Yes | ### RateLimit.Condition.Variables RateLimit.Condition.Variables defines a condition for a rate limit by NGINX variable. The following example defines a condition for a rate limit policy that applies only to requests with the request method with a value `GET`: ```yaml variables: - name: $request_method match: GET ``` **Note:** NGINX Ingress Controller currently supports only one variable at a time. |Field | Description | Type | Required | | ---| ---| ---| --- | |``name`` | the name of the NGINX variable on which to apply the rate limit. | ``string`` | Yes | |``match`` | the value of the NGINX variable to match against. Values prefixed with the `~` character denote the following is a [regular expression](https://nginx.org/en/docs/http/ngx_http_map_module.html#map). | ``string`` | Yes | ## APIKey The API Key auth policy configures NGINX to authorize client requests based on the presence of a valid API Key in a header or query parameter specified in the policy. **Note:** This feature uses the NGINX [ngx_http_auth_request_module](https://nginx.org/en/docs/http/ngx_http_auth_request_module.html) and [NGINX JavaScript (NJS)](https://nginx.org/en/docs/njs/). Subrequests may not function as expected and may cause issues when the `APIKey` policy and a `WAF` policy are applied together on the same route. The policy stores API keys securely using SHA-256 hashing. When a client sends an API Key, NJS hashes it and compares it to the hashed API Key in the NGINX configuration. If the hashed keys match, the NJS subrequest issues a 204 No Content response to the `auth_request` directive, indicating successful authorization. If the client doesn't provide an API Key in the specified header or query parameter, NGINX returns a 401 Unauthorized response. If the client presents an invalid key in the expected header or query parameter, NGINX returns a 403 Forbidden response and denies access. You can use the [errorPages](/nic/configuration/virtualserver-and-virtualserverroute-resources.md#errorpage) property on a route to change the default behavior for 401 or 403 errors. The policy requires at least one header or query parameter. The policy below configures NGINX Ingress Controller to require the API Key `password` in the header "my-header". ```yaml apiKey: suppliedIn: header: - "my-header" clientSecret: api-key-secret ``` ```yaml apiVersion: v1 kind: Secret metadata: name: api-key-secret type: nginx.org/apikey data: client1: cGFzc3dvcmQ= # password ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``suppliedIn`` | `header` or `query`. | | Yes | |``suppliedIn.header`` | An array of headers that the API Key may appear in. | ``string[]`` | No | |``suppliedIn.query`` | An array of query params that the API Key may appear in. | ``string[]`` | No | |``clientSecret`` | The name of the Kubernetes secret that stores the API Key(s). It must be in the same namespace as the Policy resource. The secret must be of the type ``nginx.org/apikey``, and the API Key(s) must be stored in a key: val format where each key is a unique clientID and each value is a unique base64 encoded API Key | ``string`` | Yes | **Note:** An APIKey policy must include at least one of the `suppliedIn.header` or `suppliedIn.query` parameters. You can also include both. ### APIKey merging behavior A VirtualServer or VirtualServerRoute can be associated with only one API Key policy per route or subroute. You can replace an API Key policy from a higher level with a different policy defined on a more specific route. For example, a VirtualServer can implement different API Key policies at various levels. In the following configuration, the server-wide `api-key-policy-server` applies to `/backend1` for authorization, because that route has no more specific policy. `/backend2` uses `api-key-policy-route`, defined at the route level. ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServer metadata: name: virtual-server spec: host: virtual-server.example.com policies: - name: api-key-policy-server upstreams: - name: backend2 service: backend2-svc port: 80 - name: backend1 service: backend1-svc port: 80 routes: - path: /backend1 action: pass: backend1 - path: /backend2 action: pass: backend2 policies: - name: api-key-policy-route ``` ## BasicAuth The basic auth policy configures NGINX to authenticate client requests using the [HTTP Basic authentication scheme](https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication). For example, the following policy rejects all requests that don't include a valid username and password combination in the HTTP header `Authentication`: ```yaml basicAuth: secret: htpasswd-secret realm: "My API" ``` **Note:** This feature uses the NGINX [ngx_http_auth_basic_module](https://nginx.org/en/docs/http/ngx_http_auth_basic_module.html). |Field | Description | Type | Required | | ---| ---| ---| --- | |``secret`` | The name of the Kubernetes secret that stores the Htpasswd configuration. It must be in the same namespace as the Policy resource. The secret must be of the type ``nginx.org/htpasswd``, and the config must be stored in the secret under the key ``htpasswd``. Otherwise, NGINX Ingress Controller rejects the secret as invalid. | ``string`` | Yes | |``realm`` | The realm for the basic authentication. | ``string`` | No | ### BasicAuth merging behavior A VirtualServer or VirtualServerRoute can reference multiple basic auth policies, but NGINX Ingress Controller applies only the first one. It ignores every subsequent reference. For example, this configuration references two policies: ```yaml policies: - name: basic-auth-policy-one - name: basic-auth-policy-two ``` In this example, NGINX Ingress Controller uses the configuration from the first policy reference, `basic-auth-policy-one`, and ignores `basic-auth-policy-two`. ## JWT using a local Kubernetes secret **Note:** This feature is only available with NGINX Plus. The JWT policy configures NGINX Plus to authenticate client requests using JSON Web Tokens. The following example policy rejects all requests that don't include a valid JWT in the HTTP header `token`: ```yaml jwt: secret: jwk-secret realm: "My API" token: $http_token ``` You can pass the JWT claims and JOSE headers to the upstream servers. For example: ```yaml action: proxy: upstream: webapp requestHeaders: set: - name: user value: ${jwt_claim_user} - name: alg value: ${jwt_header_alg} ``` This example uses the `requestHeaders` of [Action.Proxy](/nic/configuration/virtualserver-and-virtualserverroute-resources.md#actionproxy) to set the values of two headers that NGINX passes to the upstream servers. The value of the `${jwt_claim_user}` variable is the `user` claim of a JWT. For other claims, use `${jwt_claim_name}`, where `name` is the name of the claim. Nested claims and claims that include a period (`.`) aren't supported. Similarly, use `${jwt_header_name}`, where `name` is the name of a header. This example uses the `alg` header. **Note:** This feature uses the NGINX Plus [ngx_http_auth_jwt_module](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html). |Field | Description | Type | Required | | ---| ---| ---| --- | |``secret`` | The name of the Kubernetes secret that stores the JWK. It must be in the same namespace as the Policy resource. The secret must be of the type ``nginx.org/jwk``, and the JWK must be stored in the secret under the key ``jwk``. Otherwise, NGINX Ingress Controller rejects the secret as invalid. | ``string`` | Yes | |``realm`` | The realm of the JWT. | ``string`` | Yes | |``token`` | The token specifies a variable that contains the JSON Web Token. By default the JWT is passed in the ``Authorization`` header as a Bearer Token. JWT may be also passed as a cookie or a part of a query string, for example: ``$cookie_auth_token``. Accepted variables are ``$http_``, ``$arg_``, ``$cookie_``. | ``string`` | No | ### JWT merging behavior A VirtualServer or VirtualServerRoute can reference multiple JWT policies, but NGINX Ingress Controller applies only the first one. It ignores every subsequent reference. For example, this configuration references two policies: ```yaml policies: - name: jwt-policy-one - name: jwt-policy-two ``` In this example, NGINX Ingress Controller uses the configuration from the first policy reference, `jwt-policy-one`, and ignores `jwt-policy-two`. ## JWT using JWKS from a remote location **Note:** This feature is only available with NGINX Plus. The JWT policy configures NGINX Plus to authenticate client requests using JSON Web Tokens. You can import the JWKS keys for a JWT policy from a URL, such as a remote server or an identity provider, so you don't have to copy and update them on the Ingress Controller pod. The following example policy rejects all requests that don't include a valid JWT in the HTTP header fetched from the identity provider: ```yaml jwt: realm: MyProductAPI token: $http_token jwksURI: keyCache: 1h ``` **Note:** This feature uses the NGINX Plus directive [auth_jwt_key_request](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_request), part of [ngx_http_auth_jwt_module](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html). Subrequests may not function as expected and may cause issues when fetching JWKs from a remote URI (`jwksURI`) in a `JWT` policy and a `WAF` policy are applied together on the same route. |Field | Description | Type | Required | Default | | ---| ---| ---| --- | --- | |``jwksURI`` | The remote URI where NGINX Ingress Controller sends the request to retrieve the JSON Web Key set.| ``string`` | Yes | -- | |``keyCache`` | Turns on in-memory caching of JWKS (JSON Web Key Sets) obtained from the ``jwksURI`` and sets a valid time for expiration. | ``string`` | Yes | -- | |``realm`` | The realm of the JWT. | ``string`` | Yes | -- | |``token`` | The token specifies a variable that contains the JSON Web Token. By default the JWT is passed in the ``Authorization`` header as a Bearer Token. JWT may be also passed as a cookie or a part of a query string, for example: ``$cookie_auth_token``. Accepted variables are ``$http_``, ``$arg_``, ``$cookie_``. | ``string`` | No | -- | |``sniEnabled`` | Turns on SNI (Server Name Indication) for the JWT policy. Use this when the remote server requires SNI to serve the correct certificate. | ``bool`` | No | `false` | |``sniName`` | The SNI name to use when connecting to the remote server. If not set, NGINX Ingress Controller uses the hostname from the ``jwksURI``. | ``string`` | No | -- | |``sslVerify`` | Turns on verification of the JWKS server SSL certificate. | ``bool`` | No | `false` | |``sslVerifyDepth`` | Sets the verification depth in the JWKS server certificates chain. | ``int`` | No | `1` | |``trustedCertSecret`` | The name of the Kubernetes secret that stores the CA certificate for JWKS server verification. It must be in the same namespace as the Policy resource. The secret must be of the type ``nginx.org/ca``, and the certificate must be stored in the secret under the key ``ca.crt``. | ``string`` | No | -- | **Note:** NGINX Ingress Controller turns on content caching by default for each JWT policy, with a default time of 12 hours. This improves resiliency by letting NGINX Ingress Controller retrieve the JWKS (JSON Web Key Set) from the cache even after it expires. ### JWT using JWKS merging behavior This behavior is similar to using a local Kubernetes secret. A VirtualServer or VirtualServerRoute can reference multiple JWT policies, but NGINX Ingress Controller applies only the first one. It ignores every subsequent reference. For example, this configuration references two policies: ```yaml policies: - name: jwt-policy-one - name: jwt-policy-two ``` In this example, NGINX Ingress Controller uses the configuration from the first policy reference, `jwt-policy-one`, and ignores `jwt-policy-two`. ## IngressMTLS The IngressMTLS policy configures client certificate verification. For example, the following policy verifies a client certificate using the CA certificate specified in `ingress-mtls-secret`: ```yaml ingressMTLS: clientCertSecret: ingress-mtls-secret verifyClient: "on" verifyDepth: 1 ``` Below is an example of `ingress-mtls-secret` using the secret type `nginx.org/ca`: ```yaml kind: Secret metadata: name: ingress-mtls-secret apiVersion: v1 type: nginx.org/ca data: ca.crt: ``` A VirtualServer that references an IngressMTLS policy must: - Turn on [TLS termination](/nic/configuration/virtualserver-and-virtualserverroute-resources.md#virtualservertls). - Reference the policy in the VirtualServer [`spec`](/nic/configuration/virtualserver-and-virtualserverroute-resources.md#virtualserver-specification). You can't reference an IngressMTLS policy in a [`route`](/nic/configuration/virtualserver-and-virtualserverroute-resources.md#virtualserverroute) or in a VirtualServerRoute [`subroute`](/nic/configuration/virtualserver-and-virtualserverroute-resources.md#virtualserverroutesubroute). A Kubernetes Ingress that references an IngressMTLS policy must: - Turn on [TLS termination](/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md#auth-and-ssltls). - Reference the policy on the Ingress. For [mergeable Ingresses](/nic/configuration/ingress-resources/custom-annotations.md#custom-annotations-with-mergeable-ingress-resources), reference the policy on the master Ingress only. You can't reference an IngressMTLS policy on a minion Ingress. If a resource doesn't meet these conditions, NGINX sends the `500` status code to clients. You can pass the client certificate details, including the certificate, to the upstream servers. For example: ```yaml action: proxy: upstream: webapp requestHeaders: set: - name: client-cert-subj-dn value: ${ssl_client_s_dn} # subject DN - name: client-cert value: ${ssl_client_escaped_cert} # client certificate in the PEM format (urlencoded) ``` This example uses the `requestHeaders` of [Action.Proxy](/nic/configuration/virtualserver-and-virtualserverroute-resources.md#actionproxy) to set the values of the two headers that NGINX passes to the upstream servers. See the [list of embedded variables](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#variables) that `ngx_http_ssl_module` supports, which you can use to pass the client certificate details. **Note:** This feature uses the NGINX [ngx_http_ssl_module](https://nginx.org/en/docs/http/ngx_http_ssl_module.html). ### Configure a certificate revocation list The IngressMTLS policy supports configuring a certificate revocation list (CRL) for your policy, in one of two ways. **Note:** You can use only one of these configuration options at a time. 1. Add the `ca.crl` field to the `nginx.org/ca` secret type, which accepts a base64 encoded certificate revocation list. Example: ```yaml kind: Secret metadata: name: ingress-mtls-secret apiVersion: v1 type: nginx.org/ca data: ca.crt: ca.crl: ``` 2. Add the `crlFileName` field to your IngressMTLS policy spec with the name of the CRL file. **Note:** Use this configuration option only when your CRL is larger than 1 MiB. Otherwise, use the `nginx.org/ca` secret type to manage your CRL. Example: ```yaml apiVersion: k8s.nginx.org/v1 kind: Policy metadata: name: ingress-mtls-policy spec: ingressMTLS: clientCertSecret: ingress-mtls-secret crlFileName: webapp.crl verifyClient: "on" verifyDepth: 1 ``` **Note:** When you configure a CRL with the `ingressMTLS.crlFileName` field, keep this additional context in mind: - NGINX Ingress Controller expects the CRL, in this case `webapp.crl`, to be in `/etc/nginx/secrets`. Add a volume mount to the NGINX Ingress Controller deployment to add your CRL to `/etc/nginx/secrets`. - When you update the content of your CRL, for example after you revoke a new certificate, NGINX needs to reload to pick up the latest changes. Depending on your environment, this may require you to update the name of your CRL and apply this update to your `ingress-mtls.yaml` policy so NGINX picks up the latest CRL. See the Kubernetes documentation on [volumes](https://kubernetes.io/docs/concepts/storage/volumes/) to find the best implementation for your environment. |Field | Description | Type | Required | | ---| ---| ---| --- | |``clientCertSecret`` | The name of the Kubernetes secret that stores the CA certificate. It must be in the same namespace as the Policy resource. The secret must be of the type ``nginx.org/ca``, and the certificate must be stored in the secret under the key ``ca.crt``. Otherwise, NGINX Ingress Controller rejects the secret as invalid. | ``string`` | Yes | |``verifyClient`` | Verification for the client. Possible values are ``"on"``, ``"off"``, ``"optional"``, ``"optional_no_ca"``. The default is ``"on"``. | ``string`` | No | |``verifyDepth`` | Sets the verification depth in the client certificates chain. The default is ``1``. | ``int`` | No | |``crlFileName`` | The file name of the Certificate Revocation List. NGINX Ingress Controller looks for this file in `/etc/nginx/secrets`. | ``string`` | No | ### IngressMTLS merging behavior A VirtualServer or an Ingress can reference only a single IngressMTLS policy, and NGINX Ingress Controller ignores every subsequent reference. For example, this configuration references two policies: ```yaml policies: - name: ingress-mtls-policy-one - name: ingress-mtls-policy-two ``` In this example, NGINX Ingress Controller uses the configuration from the first policy reference, `ingress-mtls-policy-one`, and ignores `ingress-mtls-policy-two`. ## EgressMTLS The EgressMTLS policy configures upstream authentication and certificate verification. For example, the following policy uses `egress-mtls-secret` to authenticate with the upstream application and `egress-trusted-ca-secret` to verify the certificate of the application: ```yaml egressMTLS: tlsSecret: egress-mtls-secret trustedCertSecret: egress-trusted-ca-secret verifyServer: on verifyDepth: 2 ``` **Note:** This feature uses the NGINX [ngx_http_proxy_module](https://nginx.org/en/docs/http/ngx_http_proxy_module.html). |Field | Description | Type | Required | | ---| ---| ---| --- | |``tlsSecret`` | The name of the Kubernetes secret that stores the TLS certificate and key. It must be in the same namespace as the Policy resource. The secret must be of the type ``kubernetes.io/tls``, the certificate must be stored in the secret under the key ``tls.crt``, and the key must be stored under the key ``tls.key``. Otherwise, NGINX Ingress Controller rejects the secret as invalid. | ``string`` | No | |``trustedCertSecret`` | The name of the Kubernetes secret that stores the CA certificate. It must be in the same namespace as the Policy resource. The secret must be of the type ``nginx.org/ca``, and the certificate must be stored in the secret under the key ``ca.crt``. Otherwise, NGINX Ingress Controller rejects the secret as invalid. | ``string`` | No | |``verifyServer`` | Turns on verification of the upstream HTTPS server certificate. | ``bool`` | No | |``verifyDepth`` | Sets the verification depth in the proxied HTTPS server certificates chain. The default is ``1``. | ``int`` | No | |``sessionReuse`` | Turns on reuse of SSL sessions to the upstreams. The default is ``true``. | ``bool`` | No | |``serverName`` | Turns on passing of the server name through the ``Server Name Indication`` extension. | ``bool`` | No | |``sslName`` | Lets you override the server name used to verify the certificate of the upstream HTTPS server. | ``string`` | No | |``ciphers`` | Specifies the enabled ciphers for requests to an upstream HTTPS server. The default is ``DEFAULT``. | ``string`` | No | |``protocols`` | Specifies the protocols for requests to an upstream HTTPS server. Accepted values are ``SSLv2``, ``SSLv3``, ``TLSv1``, ``TLSv1.1``, ``TLSv1.2``, and ``TLSv1.3`` (case-insensitive). The default is ``TLSv1 TLSv1.1 TLSv1.2``. | ``string`` | No | ### EgressMTLS merging behavior A VirtualServer or VirtualServerRoute can reference multiple EgressMTLS policies, but NGINX Ingress Controller applies only the first one. It ignores every subsequent reference. For example, this configuration references two policies: ```yaml policies: - name: egress-mtls-policy-one - name: egress-mtls-policy-two ``` In this example, NGINX Ingress Controller uses the configuration from the first policy reference, `egress-mtls-policy-one`, and ignores `egress-mtls-policy-two`. ## ExternalAuth **Note:** Subrequests may not function as expected and may cause issues when the `ExternalAuth` policy and a `WAF` policy are applied together on the same route. The ExternalAuth policy configures NGINX to authenticate client requests using an external authentication server. You can use this policy with services such as [oauth2-proxy](https://oauth2-proxy.github.io/oauth2-proxy/) or any custom authentication service that supports the `auth_request` pattern. When a client sends a request, NGINX makes an internal subrequest to the external authentication service. If the service returns a `2xx` response, NGINX forwards the original request to the upstream. If it returns `401` or `403`, NGINX denies access. If you configure `authSigninURI`, NGINX redirects unauthenticated clients to a sign-in page. For example, the following policy configures external authentication using an HTTP Basic Auth backend service: ```yaml externalAuth: authURI: "/auth" authServiceName: "default/basic-auth-svc" ``` The following policy uses OAuth2 Proxy with a sign-in redirect: ```yaml externalAuth: authURI: "/oauth2/auth" authSigninURI: "/oauth2/signin" authServiceName: "default/oauth2-proxy-svc" sslEnabled: true sslVerify: true sslVerifyDepth: 2 sniName: "external-auth-tls" trustedCertSecret: "external-auth-ca-secret" ``` An example ExternalAuth policy for VirtualServer resources is available in the GitHub repository for [basic auth](https://github.com/nginx/kubernetes-ingress/blob/v/examples/custom-resources/external-auth) and [OAuth2 with basic auth](https://github.com/nginx/kubernetes-ingress/blob/v/examples/custom-resources/external-auth-oauth2). Examples for Ingress resources are also available for [basic auth](https://github.com/nginx/kubernetes-ingress/blob/v/examples/ingress-resources/external-auth) and [OAuth2 with basic auth using mergeable Ingresses](https://github.com/nginx/kubernetes-ingress/blob/v/examples/ingress-resources/external-auth-mergeable). |Field | Description | Type | Required | | ---| ---| ---| --- | |``authURI`` | The URI of the external authentication server. NGINX sends an internal subrequest to this URI to verify the client. Must start with ``/``. For example, ``/auth`` or ``/oauth2/auth``. | ``string`` | Yes | |``authServiceName`` | The name of the Kubernetes service for the external authentication server. Can include an optional namespace prefix in the format ``/``. For example, ``basic-auth-svc`` or ``auth-namespace/auth-service``. If no namespace is specified, the namespace of the Policy resource is used. | ``string`` | Yes | |``authServicePorts`` | The ports of the Kubernetes service to which authentication requests are sent. If not specified, the first port from the service definition is used. | ``[]int`` | No | |``authSigninURI`` | The URI to redirect unauthenticated clients to for sign-in. Used when the external authentication server requires redirection, such as with OAuth2 Proxy. Must start with ``/``. For example, ``/oauth2/signin``. | ``string`` | No | |``authSnippets`` | Custom NGINX configuration snippets to add to the external authentication location block. For example, you can add extra headers or parameters for the ``auth_request`` module. The content must be valid NGINX configuration. Requires the [``-enable-snippets``](/nic/configuration/global-configuration/command-line-arguments.md#cmdoption-enable-snippets) command-line argument. | ``string`` | No | |``authSigninRedirectBasePath`` | The base path for the NGINX location block that handles sign-in redirect requests from the external authentication server. For example, OAuth2 Proxy expects ``/oauth2``. Defaults to ``/oauth2`` if not specified. | ``string`` | No | |``sslEnabled`` | Turns on HTTPS when proxying requests to the external authentication server. The default is ``false``. | ``bool`` | No | |``sslVerify`` | Turns on verification of the external authentication server's SSL certificate. The default is ``false``. | ``bool`` | No | |``sslVerifyDepth`` | Sets the verification depth in the external authentication server certificates chain. The default is ``1``. | ``int`` | No | |``trustedCertSecret`` | The name of the Kubernetes secret that stores the CA certificate for external authentication server certificate verification. Can include an optional namespace prefix as ``/``. The secret must be of the type ``nginx.org/ca``, and the certificate must be stored under the key ``ca.crt``. | ``string`` | No | |``sniName`` | The server name used for SNI and certificate verification when connecting to the external authentication server over TLS. If not specified, defaults to ``..svc`` derived from ``authServiceName``. | ``string`` | No | ### ExternalAuth merging behavior A VirtualServer, VirtualServerRoute, Ingress, or mergeable Ingress can reference only one ExternalAuth policy per route, and NGINX Ingress Controller ignores every subsequent reference. This means you can't combine different types of external authentication on the same route. For example, you can't apply both an OAuth2 policy and a basic auth policy to the same route. ```yaml policies: - name: external-auth-policy-one - name: external-auth-policy-two ``` NGINX Ingress Controller uses the configuration from the first policy reference, `external-auth-policy-one`, and ignores `external-auth-policy-two`. To use different authentication methods on different routes, apply each ExternalAuth policy to its own route. ### OAuth2 sign-in redirect location When you configure `authSigninURI`, NGINX Ingress Controller generates an internal location block to handle sign-in redirects, based on `authSigninRedirectBasePath`, which defaults to `/oauth2`. Because NGINX location blocks are defined at the server level, only one sign-in redirect location can exist per host. If multiple routes on the same VirtualServer, VirtualServerRoute, Ingress, or mergeable Ingress host reference ExternalAuth policies with `authSigninURI`, NGINX Ingress Controller uses the sign-in redirect configuration from the first policy it processes. All routes that use `authSigninURI` share that single redirect location. This means all routes on the same host that require OAuth2 sign-in must use the same OAuth2 Proxy backend for the redirect flow. If you need different OAuth2 providers for different routes, use separate hosts. ## OIDC **Note:** This feature is turned off by default. To turn it on, set the [enable-oidc](/nic/configuration/global-configuration/command-line-arguments.md#cmdoption-enable-oidc) command-line argument of NGINX Ingress Controller. **Note:** Subrequests may not function as expected and may cause issues when the `OIDC` policy and a `WAF` policy are applied together on the same route. The OIDC policy configures NGINX Plus as a relying party for OpenID Connect authentication. For example, the following policy uses the client ID `nginx-plus` and the client secret `oidc-secret` to authenticate with the OpenID Connect provider `https://idp.example.com`: ```yaml spec: oidc: clientID: nginx-plus clientSecret: oidc-secret authEndpoint: https://idp.example.com/openid-connect/auth tokenEndpoint: https://idp.example.com/openid-connect/token jwksURI: https://idp.example.com/openid-connect/certs endSessionEndpoint: https://idp.example.com/openid-connect/logout postLogoutRedirectURI: / accessTokenEnable: true pkceEnable: false ``` NGINX Plus passes the ID of an authenticated user to the backend in the HTTP header `username`. **Note:** This feature uses the [reference implementation](https://github.com/nginxinc/nginx-openid-connect/) of NGINX Plus as a relying party for OpenID Connect authentication. ### Prerequisites To use OIDC, turn on [zone synchronization](/nginx/admin-guide/high-availability/zone_sync.md). If you don't set up zone synchronization, NGINX Plus fails to reload. You also need to configure a resolver, which NGINX Plus uses to resolve the IDP authorization endpoint. You can find an example configuration [in the GitHub repository](https://github.com/nginx/kubernetes-ingress/blob/v/examples/custom-resources/oidc#step-7---configure-nginx-plus-zone-synchronization-and-resolver). **Note:** The configuration in the example doesn't turn on TLS, so synchronization between replicas happens in clear text. This can expose tokens. ### Limitations The OIDC policy defines a few internal locations that you can't customize: `/_jwks_uri`, `/_token`, `/_refresh`, `/_id_token_validation`, `/logout`. In addition, `/_codexch` is the default value for the redirect URI, and `/_logout` is the default value for the post logout redirect URI. You can customize both. Specifying one of these locations as a route in the VirtualServer or VirtualServerRoute causes a collision, and NGINX Plus fails to reload. |Field | Description | Type | Required | | ---| ---| ---| --- | |``clientID`` | The client ID provided by your OpenID Connect provider. | ``string`` | Yes | |``clientSecret`` | The name of the Kubernetes secret that stores the client secret provided by your OpenID Connect provider. It must be in the same namespace as the Policy resource. The secret must be of the type ``nginx.org/oidc``, and the secret stored under the key ``client-secret``. Otherwise, NGINX Ingress Controller rejects the secret as invalid. If you enable PKCE, don't configure this field. | ``string`` | Yes | |``authEndpoint`` | URL for the authorization endpoint provided by your OpenID Connect provider. | ``string`` | Yes | |``authExtraArgs`` | A list of extra URL arguments to pass to the authorization endpoint provided by your OpenID Connect provider. Arguments must be URL encoded, multiple arguments may be included in the list, for example ``[ arg1=value1, arg2=value2 ]`` | ``string[]`` | No | |``tokenEndpoint`` | URL for the token endpoint provided by your OpenID Connect provider. | ``string`` | Yes | |``endSessionEndpoint`` | URL provided by your OpenID Connect provider to request the end user be logged out. | ``string`` | No | |``jwksURI`` | URL for the JSON Web Key Set (JWK) document provided by your OpenID Connect provider. | ``string`` | Yes | |``scope`` | List of OpenID Connect scopes. The scope ``openid`` always needs to be present and others can be added concatenating them with a ``+`` sign, for example ``openid+profile+email``, ``openid+email+userDefinedScope``. The default is ``openid``. | ``string`` | No | |``redirectURI`` | Lets you override the default redirect URI. The default is ``/_codexch``. | ``string`` | No | |``postLogoutRedirectURI`` | URI to redirect to after the logout has been performed. Requires ``endSessionEndpoint``. The default is ``/_logout``. | ``string`` | No | |``zoneSyncLeeway`` | Specifies the maximum timeout in milliseconds for synchronizing ID/access tokens and shared values between Ingress Controller pods. The default is ``200``. | ``int`` | No | |``accessTokenEnable`` | Option of whether Bearer token is used to authorize NGINX to access protected backend. | ``boolean`` | No | |``pkceEnable`` | Turns on Proof Key for Code Exchange. The OpenID client needs to be in public mode. `clientSecret` is not used in this mode. | ``boolean`` | No | |``sslVerify`` | Use this option to turn on TLS verification when calls are made to the IDP endpoints. | ``boolean`` | No | |``verifyDepth`` | Sets the verification depth in the proxied HTTPS server certificates chain. The default is ``1``. | ``int`` | No | |``trustedCertSecret`` | The name of the Kubernetes secret that stores the CA certificate. It must be in the same namespace as the Policy resource. The secret must be of the type ``nginx.org/ca``, and the certificate must be stored in the secret under the key ``ca.crt``. Otherwise, NGINX Ingress Controller rejects the secret as invalid. | ``string`` | No | **Note:** Only one OIDC policy can be referenced in a VirtualServer and its VirtualServerRoutes. However, you can still apply the same policy to different routes in the VirtualServer and VirtualServerRoutes. ### OIDC merging behavior A VirtualServer or VirtualServerRoute can reference only a single OIDC policy, and NGINX Ingress Controller ignores every subsequent reference. For example, this configuration references two policies: ```yaml policies: - name: oidc-policy-one - name: oidc-policy-two ``` In this example, NGINX Ingress Controller uses the configuration from the first policy reference, `oidc-policy-one`, and ignores `oidc-policy-two`. ## OIDCNative **Note:** This feature is only available with NGINX Plus and requires the [enable-oidc](/nic/configuration/global-configuration/command-line-arguments.md#cmdoption-enable-oidc) command-line argument. **Note:** Subrequests may not function as expected and may cause issues when the `OIDCNative` policy and a `WAF` policy are applied together on the same route. The OIDCNative policy configures NGINX Plus as a relying party for OpenID Connect authentication using the built-in `ngx_http_oidc_module`. Unlike the NJS-based [`oidc`](#oidc) policy, the native implementation handles the entire OIDC flow within the NGINX core, including token exchange, session management, and front-channel logout. For example, the following policy authenticates users against a Keycloak identity provider: ```yaml spec: oidcNative: issuer: https://idp.example.com/realms/master clientID: nginx-plus clientSecret: oidc-secret scope: openid profile email logoutURI: /logout postLogoutRedirectURI: /_logout ``` **Note:** This feature uses the NGINX [ngx_http_oidc_module](https://nginx.org/en/docs/http/ngx_http_oidc_module.html). ### Prerequisites Configure a resolver so NGINX Plus can resolve the identity provider's hostname for discovery and token exchange. Add the following to your ConfigMap: ```yaml data: resolver-addresses: "kube-dns.kube-system.svc.cluster.local" ``` When you use the [zone synchronization](/nic/configuration/global-configuration/configmap-resource.md#zone-sync) feature, NGINX Ingress Controller synchronizes OIDC session data across all replicas. Put the ConfigMap in its final state, with or without zone sync turned on, before you create the OIDCNative policy and its referencing VirtualServer or Ingress. NGINX declares the native module's session zone with sync only when zone sync is turned on, and NGINX can't change the sync flag of an existing shared memory zone across a reload. ### Differences from the NJS-based OIDC policy | Aspect | OIDC (NJS) | OIDCNative (native module) | | --- | --- | --- | | Implementation| NGINX JavaScript reference implementation | Built-in `ngx_http_oidc_module` | | Discovery | Manual endpoint configuration (`authEndpoint`, `tokenEndpoint`, `jwksURI`) | Automatic through OpenID Connect Discovery (`issuer` or `configURL`) | | Ingress Support | No | Yes, through `nginx.com/policies` annotation | | Front-channel logout | Through NJS handler | Through `frontChannelLogoutURI` directive | | PKCE | `pkceEnable` boolean | `pkce` enum (on/off), auto-detected from provider metadata by default | | Multiple providers per host | One per VirtualServer | Multiple (unique provider per policy and resource combination) | **Note:** `oidc` and `oidcNative` are mutually exclusive. A Policy resource must define only one of them. If you set both, NGINX Ingress Controller marks the Policy as Invalid. On a VirtualServer, if you apply both an `oidc` policy and an `oidcNative` policy to the same route, that route gets a Warning status and returns a static 500 response. | Field | Description | Type | Required | Default | | --- | --- | --- | --- | --- | | `issuer` | The Issuer Identifier URL of the OpenID Provider. Must use the `https` scheme and exactly match the value of `issuer` in the OpenID Provider metadata. | `string` | Yes | -- | | `clientID` | The client ID provided by your OpenID Connect provider. | `string` | Yes | -- | | `clientSecret` | The name of the Kubernetes secret that stores the client secret. Must be of type `nginx.org/oidc` with the secret stored under the key `client-secret` and must be in the same namespace as the Policy resource. Not required when PKCE is enabled with a public client. | `string` | No | -- | | `configURL` | The URL of the OpenID Provider Configuration Information (discovery endpoint). Must include a path and use the `http` or `https` scheme. If not set, defaults to `/.well-known/openid-configuration`. | `string` | No | `/.well-known/openid-configuration` | | `scope` | Space-separated list of OpenID Connect scopes. Must contain `openid`. Example: `"openid profile email"`. | `string` | No | `openid` | | `redirectURI` | Overrides the default redirect URI path used for the authorization callback. | `string` | No | `/oidc_callback_` | | `cookieName` | Sets the name of the session cookie. Must contain only letters, digits, and underscores. | `string` | No | `NGX_OIDC_` | | `extraAuthArgs` | Additional query arguments appended to the authorization request URL. Example: `"display=page&prompt=login"`. | `string` | No | -- | | `pkce` | Explicitly turns PKCE on or off. By default, NGINX Ingress Controller turns on PKCE automatically based on OpenID Provider metadata. | `string (on/off)` | No | -- | | `logoutURI` | URI path for initiating session logout. Session logout is unavailable until this field is set. Use a path the attached route can serve, for example `/logout` for a policy on the `/` route, or `/tea/logout` for a policy on the `/tea` route. A path outside the attached route isn't matched by that route's location, so logout doesn't trigger. | `string` | No | -- | | `postLogoutRedirectURI` | Path the user is returned to after logout completes. Must be a path on the same host, absolute URLs aren't supported. NGINX Ingress Controller generates an unauthenticated location at this path serving a plain-text confirmation, shared if multiple providers use the same path. | `string` | No | -- | | `frontChannelLogoutURI` | URI path for OIDC front-channel logout. When set, the identity provider calls this URI in a hidden iframe during global logout, letting NGINX end the local session. | `string` | No | -- | | `logoutTokenHint` | Adds the `id_token_hint` argument to the provider's logout endpoint when redirecting the user during logout. Required by some providers. | `bool` | No | `false` | | `sessionTimeout` | Duration after which the session expires unless refreshed. Example: `"8h"`, `"30m"`. | `string` | No | `8h` | | `userInfoEnable` | Turns on downloading of the UserInfo data and makes UserInfo claims available through `$oidc_claim_` variables. | `bool` | No | `false` | | `trustedCertSecret` | The name of the Kubernetes secret that stores the CA certificate for verifying the provider's TLS certificate. Must be of type `nginx.org/ca` with the certificate stored under key `ca.crt`. | `string` | No | -- | | `sslVerify` | Turns on verification of the OpenID Provider's TLS certificate. Set to false to skip verification (dev/test only). | `bool` | No | `true` | | `sslName` | Overrides the TLS SNI name and Host header used when connecting to the OpenID Provider. Must be a valid DNS name and can't include a port. When unset, the hostname of the endpoint being called is used, taken from the provider's discovery metadata. | `string` | No | -- | | `sslVerifyDepth` | Verification depth in the OpenID Provider TLS certificate chain. | `int` | No | `1` | | `proxyBufferSize` | Buffer size used when proxying requests to the OpenID Provider. Applies to proxy_buffer_size and each buffer in proxy_buffers. | `string` | No | `32k` | ### OIDCNative on Ingress When using OIDCNative with Ingress resources, reference the policy through the `nginx.com/policies` annotation: ```yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: webapp annotations: nginx.com/policies: "oidc-native-policy" spec: ingressClassName: nginx tls: - hosts: - webapp.example.com secretName: tls-secret rules: - host: webapp.example.com http: paths: - path: / pathType: Prefix backend: service: name: webapp-svc port: number: 80 ``` For mergeable Ingress, you can apply the policy at the master level (server-wide) or on individual minions (location-level): ```yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: webapp-master annotations: nginx.org/mergeable-ingress-type: "master" nginx.com/policies: "oidc-native-policy" spec: ingressClassName: nginx tls: - hosts: - webapp.example.com secretName: tls-secret rules: - host: webapp.example.com ``` **Note:** On Ingress, reference OIDCNative policies through `nginx.com/policies`, not `nginx.org/policies`. Ingress resources don't support the NJS-based `oidc` policy. ### OIDCNative merging behavior A VirtualServer, VirtualServerRoute, or Ingress can reference only a single OIDCNative policy per context, and NGINX Ingress Controller ignores every subsequent reference. For example, this configuration references two policies: ```yaml policies: - name: oidc-policy-one - name: oidc-policy-two ``` In this example, NGINX Ingress Controller uses the configuration from the first policy reference, `oidc-policy-one`, and ignores `oidc-policy-two`. Multiple OIDCNative policies can coexist on the same host when applied to different routes (VirtualServer) or different mergeable Ingress minions. Each generates a unique `oidc_provider` block with its own session store. If you override those paths and two policies on the same host claim the same one, for example by setting `redirectURI` to the same value, NGINX Ingress Controller rejects the conflicting policy and that route or minion returns a 500 response. `postLogoutRedirectURI` is the exception: providers may share it, and only one location is generated. ## Cache The cache policy configures proxy caching, which improves performance by storing and serving cached responses to clients instead of proxying every request to upstream servers. **Note:** Subrequests may not function as expected and may cause issues when `cacheBackgroundUpdate` in a `Cache` policy and a `WAF` policy are applied together on the same route. For example, the following policy creates a cache zone named "mycache" with 10 MB of memory allocated, and caches all GET response codes for 30 seconds: ```yaml cache: cacheZoneName: "mycache" cacheZoneSize: "10m" allowedCodes: ["any"] allowedMethods: ["GET"] time: "30s" ``` Here's an example with more specific configuration: ```yaml cache: cacheZoneName: "mycache" cacheZoneSize: "100m" allowedCodes: [200, 301, 302] allowedMethods: ["GET", "POST"] time: "5m" levels: "1:2" overrideUpstreamCache: true inactive: "60m" useTempPath: false maxSize: "10g" minFree: "1g" manager: files: 100 sleep: "50ms" threshold: "200ms" cacheKey: "$scheme$host$request_uri" cacheUseStale: [ "error", "timeout", "updating", "http_500" ] cacheRevalidate: true cacheBackgroundUpdate: true cacheMinUses: 1 lock: enable: true timeout: "5s" age: "30s" conditions: noCache: [ "$cookie_nocache", "$arg_nocache" ] bypass: [ "$http_authorization" ] ``` **Note:** This feature uses the NGINX [ngx_http_proxy_module](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_path) `proxy_cache_path` and related directives. |Field | Description | Type | Required | | --- | ---| ---| --- | |``cacheZoneName`` | CacheZoneName defines the name of the cache zone. Must start with a lowercase letter, followed by alphanumeric characters or underscores, and end with an alphanumeric character. Single lowercase letters are also allowed. Examples: "cache", "my_cache", "cache1". | ``string`` | Yes | |``cacheZoneSize`` | CacheZoneSize defines the size of the cache zone. Must be a number followed by a size unit: 'k' for kilobytes, 'm' for megabytes, or 'g' for gigabytes. Examples: "10m", "1g", "512k". | ``string`` | Yes | |``allowedCodes`` | AllowedCodes defines which HTTP response codes should be cached. Accepts either: - The string "any" to cache all response codes (must be the only element) - A list of HTTP status codes as integers (100-599) Examples: ["any"], [200, 301, 404], [200]. Invalid: ["any", 200] (cannot mix "any" with specific codes). | ``[]IntOrString`` | No | |``time`` | The default cache time for responses. Required when allowedCodes is specified. Must be a number followed by a time unit: 's' for seconds, 'm' for minutes, 'h' for hours, 'd' for days. Examples: "30s", "5m", "1h", "2d". | ``string`` | No | |``allowedMethods`` | AllowedMethods defines which HTTP methods should be cached. Only "GET", "HEAD", and "POST" are supported by the NGINX `proxy_cache_methods` directive. GET and HEAD are always cached by default even if not specified. Maximum of 3 items allowed. Examples: ["GET"], ["GET", "HEAD", "POST"]. Invalid methods: PUT, DELETE, PATCH, and so on. | ``[]string`` | No | |``levels`` | Levels defines the cache directory hierarchy levels for storing cached files. Must be in format "X:Y" or "X:Y:Z" where X, Y, Z are either 1 or 2. This controls the number of subdirectory levels and their name lengths. Examples: "1:2", "2:2", "1:2:2". Invalid: "3:1", "1:3", "1:2:3". | ``string`` | No | |``overrideUpstreamCache`` | OverrideUpstreamCache controls whether to override upstream cache headers (using the `proxy_ignore_headers` directive). When true, NGINX ignores cache-related headers from upstream servers like Cache-Control, Expires, and so on. Default: false. | ``bool`` | No | |``cachePurgeAllow`` | CachePurgeAllow defines IP addresses or CIDR blocks allowed to purge cache. This feature is only available in NGINX Plus. Examples: ["192.168.1.100", "10.0.0.0/8", "::1"]. | ``[]string`` | No | |``cacheKey`` | CacheKey defines a key for caching (`proxy_cache_key`). By default, "$scheme$proxy_host$uri". Must not contain command execution patterns: $(, `, ;, &&, || | ``string`` | No | |``cacheUseStale`` | CacheUseStale determines in which cases a stale cached response can be used (`proxy_cache_use_stale`). Valid parameters: error, timeout, invalid_header, updating, http_500, http_502, http_503, http_504, http_403, http_404, http_429, off. | ``[]string`` | No | |``cacheRevalidate`` | CacheRevalidate turns on revalidation of expired cache items using conditional requests (`proxy_cache_revalidate`). Uses "If-Modified-Since" and "If-None-Match" header fields. | ``bool`` | No | |``cacheBackgroundUpdate`` | CacheBackgroundUpdate lets NGINX start a background subrequest to update an expired cache item (`proxy_cache_background_update`). NGINX returns a stale cached response to the client while it updates the cache. | ``bool`` | No | |``cacheMinUses`` | CacheMinUses sets the number of requests after which NGINX Ingress Controller caches the response (`proxy_cache_min_uses`). | ``integer`` | No | |``inactive`` | Inactive sets the time after which cached data that are not accessed get removed from the cache (inactive parameter). By default, inactive is set to 10 minutes. | ``string`` | No | |``maxSize`` | MaxSize sets the maximum cache size (max_size parameter). When the size is exceeded, the cache manager removes the least recently used data. | ``string`` | No | |``minFree`` | MinFree sets the minimum amount of free space required on the file system with cache (min_free parameter). When there is not enough free space, the cache manager removes the least recently used data. | ``string`` | No | |``useTempPath`` | UseTempPath controls whether temporary files and the cache are put on different file systems (use_temp_path parameter). If set to false, NGINX puts temporary files directly in the cache directory (use_temp_path=off). Default: false (use_temp_path=off, which puts temp files directly in the cache directory for better performance). | ``bool`` | No | |``manager`` | Manager configures the cache manager process parameters (manager_files, manager_sleep, manager_threshold). | ``object`` | No | |``manager.files`` | Files sets the maximum number of files that the cache manager deletes in one iteration. During one iteration, the cache manager deletes no more than manager_files items (by default, 100). | ``integer`` | No | |``manager.sleep`` | Sleep sets the pause between cache manager iterations. Between iterations, a pause configured by manager_sleep (by default, 50 milliseconds) is made. | ``string`` | No | |``manager.threshold`` | Threshold sets the maximum duration of one cache manager iteration. The duration of one iteration is limited by manager_threshold (by default, 200 milliseconds). | ``string`` | No | |``lock`` | Lock configures cache locking to prevent multiple identical requests from populating the same cache element simultaneously. | ``object`` | No | |``lock.enable`` | Enable sets whether cache locking is turned on (`proxy_cache_lock`). When on, only one request at a time can populate a new cache element according to the `proxy_cache_key`. | ``bool`` | No | |``lock.timeout`` | Timeout sets a timeout for `proxy_cache_lock`. When the time expires, NGINX passes the request to the proxied server, but it doesn't cache the response. | ``string`` | No | |``lock.age`` | Age sets the maximum time a cache lock can be held (`proxy_cache_lock_age`). If the last request passed to the proxied server for populating a new cache element hasn't completed within the specified time, NGINX may pass one more request. | ``string`` | No | |``conditions`` | Conditions defines when responses should not be cached or taken from cache. | ``object`` | No | |``conditions.noCache`` | NoCache defines conditions under which the response won't be saved to a cache (`proxy_no_cache`). If at least one value of the string parameters isn't empty and isn't equal to "0", NGINX doesn't save the response. | ``[]string`` | No | |``conditions.bypass`` | Bypass defines conditions under which the response won't be taken from a cache (`proxy_cache_bypass`). If at least one value of the string parameters isn't empty and isn't equal to "0", NGINX doesn't take the response from the cache. | ``[]string`` | No | ### Cache merging behavior A VirtualServer or VirtualServerRoute can reference multiple cache policies, but NGINX Ingress Controller applies only the first one. It ignores every subsequent reference. ## CORS The CORS policy configures Cross-Origin Resource Sharing (CORS) headers. **Note:** This feature uses the NGINX `add_header` directive. Below is an example of a CORS policy configuring all the available options: ```yaml apiVersion: k8s.nginx.org/v1 kind: Policy metadata: name: cors-policy spec: cors: allowOrigin: - "https://test.example.com" - "https://app.example.com" - "https://admin.example.com" allowMethods: - "GET" - "POST" - "PUT" allowHeaders: - "Content-Type" - "Authorization" - "X-Requested-With" - "X-API-Key" allowCredentials: true exposeHeaders: - "X-Total-Count" - "X-Page-Size" - "X-RateLimit-Remaining" - "X-RateLimit-Reset" maxAge: 3600 ``` |Field | Description | Type | Required | | --- | ---| ---| --- | |``allowOrigin`` | AllowOrigin defines the origins that are allowed to make cross-origin requests. Can be exact domains, single wildcards, or `*` for all origins. Examples: ["https://example.com", "https://*.mydomain.com", "*"] Security: When allowCredentials is true, wildcard "*" is not allowed. The server must specify explicit origins for credentialed requests. |``array[string]`` | Yes | |``allowMethods`` | AllowMethods defines the HTTP methods that are allowed for cross-origin requests. | ``array[string]`` | No | |``allowHeaders`` | AllowHeaders defines the headers that are allowed in cross-origin requests. Common safe headers: ["Accept", "Accept-Language", "Content-Language", "Content-Type"] Custom headers: ["Authorization", "X-Requested-With", "X-Custom-Header"] | ``array[string]`` | No | |``allowCredentials`` | AllowCredentials indicates whether the response to the request can be exposed when the credentials flag is true. When used as part of a response to a preflight request, this indicates whether the actual request can be made using credentials. | ``boolean`` | No | |``exposeHeaders`` | ExposeHeaders defines the headers that browsers are allowed to access. Use this field to expose additional custom headers to the browser. Example: ["X-Total-Count", "X-Page-Size", "X-RateLimit-Remaining"] Note: Set-Cookie headers cannot be exposed through CORS per official MDN specification. | ``array[string]`` | No | |``maxAge`` | MaxAge defines how long (in seconds) the results of a preflight request can be cached. Default: 86400 (24 hours). | ``integer`` | No | **Note:** If CORS is currently configured in deployments using `snippets` or `responseHeaders.add`, migrate those settings to the CORS policy and remove the duplicate configuration. ### CORS merging behavior A VirtualServer or VirtualServerRoute can reference multiple CORS policies, but NGINX Ingress Controller applies only the first one. It ignores every subsequent reference. ## WAF **Note:** This feature uses the NGINX Plus [F5 WAF for NGINX module](/waf/). **Note:** Policies that rely on NGINX subrequests (such as `ExternalAuth`, `APIKey`, `JWT` with remote JWKS fetching, `OIDC`, or `Cache` with `cacheBackgroundUpdate`) and a `WAF` policy may not function as expected and may cause issues when applied together on the same route. The WAF policy configures NGINX Plus to secure client requests using F5 WAF for NGINX policies. For example, the following policy turns on the referenced APPolicy. You can configure multiple APLogConfs with log destinations: ```yaml waf: enable: true apPolicy: "default/dataguard-alarm" securityLogs: - enable: true apLogConf: "default/logconf" logDest: "syslog:server=syslog-svc.default:514" - enable: true apLogConf: "default/logconf" logDest: "syslog:server=syslog-svc-secondary.default:514" ``` **Note:** The field `waf.securityLog` is deprecated and will be removed in a future release. NGINX Ingress Controller ignores it if `waf.securityLogs` is populated. |Field | Description | Type | Required | | ---| ---| ---| --- | |``enable`` | Turns on F5 WAF for NGINX. | ``bool`` | Yes | |``apPolicy`` | The [F5 WAF for NGINX policy](/nic/integrations/app-protect-waf/configuration.md#waf-policies) of the WAF. References an APPolicy CR by `[/]`. When you start the Ingress Controller with `-plm-storage-url`, PLM must have compiled the referenced APPolicy (status.bundle.state == ready). Mutually exclusive with `apBundle`. | `string` | No | |``apBundle`` | The [F5 WAF for NGINX policy bundle](/nic/integrations/app-protect-waf/configuration.md#waf-bundles). Mutually exclusive with ``apPolicy`` and ``apBundleSource``. | ``string`` | No | |``apBundleSource`` | [Remote source](/nic/integrations/app-protect-waf-v5/bundle-sources.md) for fetching the WAF policy bundle. Mutually exclusive with ``apBundle`` and ``apPolicy``. | [waf.apBundleSource](#wafapbundlesource) | No | |``securityLog.enable`` | **Deprecated:** Turns on the security log. | ``bool`` | No | |``securityLog.apLogConf`` | **Deprecated:** The [F5 WAF for NGINX log conf](/nic/integrations/app-protect-waf/configuration.md#waf-logs) resource. Accepts an optional namespace. Only works with ``apPolicy``. | ``string`` | No | |``securityLog.apLogBundle`` | **Deprecated:** The [F5 WAF for NGINX log bundle](/nic/integrations/app-protect-waf/configuration.md#waf-bundles) resource. Only works with ``apBundle``. | ``string`` | No | |``securityLog.logDest`` | **Deprecated:** The log destination for the security log. Only accepted variables are ``syslog:server=; localhost; :``, ``stderr``, ````. | ``string`` | No | |``securityLogs`` | Config for security log destinations. | [waf.securityLogs](#wafsecuritylogs) | No | ### WAF.SecurityLogs |Field | Description | Type | Required | | ---| ---| ---| --- | |``enable`` | Turns on the security log. | ``bool`` | No | |``apLogConf`` | The [App Protect WAF log conf](/nic/integrations/app-protect-waf/configuration.md#waf-logs) resource. `apLogConf` references an APLogconf CR by `[/]`. When you start the Ingress Controller with `-plm-storage-url`, PLM must have compiled the referenced APLogConf (`status.bundle.state == ready`). Only works with `apPolicy`. | ``string`` | No | |``apLogBundle`` | The [App Protect WAF log bundle](/nic/integrations/app-protect-waf/configuration.md#waf-bundles) resource. Only works with ``apBundle``. Mutually exclusive with ``apLogBundleSource``. | ``string`` | No | |``apLogBundleSource`` | [Remote source](/nic/integrations/app-protect-waf-v5/bundle-sources.md) for fetching the log profile bundle. Mutually exclusive with ``apLogBundle``. | [waf.apBundleSource](#wafapbundlesource) | No | |``logDest`` | The log destination for the security log. Only accepted variables are ``syslog:server=; localhost; :``, ``stderr``, ````. | ``string`` | No | ### WAF.ApBundleSource The `apBundleSource` object configures how NGINX Ingress Controller fetches a pre-compiled WAF bundle from a remote source. `waf.securityLogs[].apLogBundleSource` uses the same fields. NGINX Ingress Controller supports three source types: - **N1C (NGINX One Console)** -- fetch policies compiled and managed through NGINX One Console. See [policy docs](/nginx-one-console/waf-integration/policy/_index.md). - **NIM (NGINX Instance Manager)** -- fetch policies compiled and managed through NGINX Instance Manager. See [bundle docs](/nim/waf-integration/policies-and-logs/bundles/create-bundle.md). - **HTTPS** -- fetch compiled `.tgz` bundles from any HTTPS server or endpoint. For details and examples, see [Connect F5 WAF for NGINX to bundle sources](/nic/integrations/app-protect-waf-v5/bundle-sources.md). |Field | Description | Type | Required | | ---| ---| ---| --- | |``type`` | Source backend: ``N1C`` (NGINX One Console), ``NIM`` (NGINX Instance Manager), or ``HTTPS``. Defaults to ``HTTPS``. | ``string`` | No | |``url`` | Tenant URL for ``N1C``/``NIM``, or full ``.tgz`` bundle URL for ``HTTPS``. Must use ``https://``. | ``string`` | Yes | |``name`` | Management-plane policy name for ``N1C``/``NIM``. For ``apLogBundleSource``, set this to the log profile name. Ignored for ``HTTPS``. | ``string`` | No | |``namespace`` | Management-plane namespace or tenant. Required for ``N1C``. Not used for ``NIM`` or ``HTTPS``. | ``string`` | No | |``enablePolling`` | Optional. Defaults to ``false``. When ``false`` or unset, NGINX Ingress Controller fetches the bundle once at policy creation or update. When ``true``, NGINX Ingress Controller re-fetches the bundle at ``pollInterval``. | ``bool`` | No | |``pollInterval`` | How often to re-fetch when ``enablePolling`` is ``true``. Minimum ``1m``, default ``5m``. | ``string`` | No | |``secret`` | Secret in the same namespace as the Policy. For ``N1C``/``NIM``, use ``nginx.com/waf-bundle`` (token or username/password). For ``HTTPS``, use ``kubernetes.io/tls`` for client mTLS (``tls.crt`` and ``tls.key``). | ``string`` | No | |``trustedCertSecret`` | Name of an ``nginx.org/ca`` Secret containing a custom CA certificate (``ca.crt``) for verifying the server TLS certificate. Must be in the same namespace as the Policy. | ``string`` | No | |``insecureSkipVerify`` | Turns off TLS certificate verification. Not recommended for production. | ``bool`` | No | |``verifyChecksum`` | Turns on SHA-256 verification of the downloaded bundle. HTTPS only. | ``bool`` | No | |``timeout`` | Time limit for a single bundle fetch request. Default ``60s``. | ``string`` | No | |``retryAttempts`` | Number of additional fetch attempts after a temporary fetch error (for example, a timeout or HTTP 5xx). Valid range is ``1``-``10``. | ``int`` | No | For example, see the snippets below: for NGINX Instance Manager: ```yaml spec: waf: enable: true apBundleSource: type: NIM url: "https://" name: "" secret: "nim-credentials" enablePolling: true pollInterval: "5m" securityLogs: - enable: true apLogBundleSource: type: NIM url: "https://" name: "" secret: "nim-credentials" enablePolling: true pollInterval: "5m" logDest: "stderr" ``` for NGINX One Console: ```yaml spec: waf: enable: true apBundleSource: type: N1C url: "https://.console.ves.volterra.io" name: "" namespace: "default" secret: "n1c-credentials" enablePolling: true pollInterval: "5m" securityLogs: - enable: true apLogBundleSource: type: N1C url: "https://.console.ves.volterra.io" name: "secops_dashboard" namespace: "default" secret: "n1c-credentials" enablePolling: true pollInterval: "5m" logDest: "stderr" ``` For HTTPS: ```yaml spec: waf: enable: true apBundleSource: url: "https://bundle-server.default.svc.cluster.local/bundles/attack-signatures-blocking.tgz" secret: "bundle-client-tls" trustedCertSecret: "bundle-server-ca" enablePolling: true pollInterval: "5m" securityLogs: - enable: true apLogBundleSource: url: "https://bundle-server.default.svc.cluster.local/bundles/log-default.tgz" secret: "bundle-client-tls" trustedCertSecret: "bundle-server-ca" enablePolling: true pollInterval: "5m" logDest: "stderr" ``` ### WAF merging behavior A VirtualServer or VirtualServerRoute can reference multiple WAF policies, but NGINX Ingress Controller applies only the first one. It ignores every subsequent reference. For example, this configuration references two policies: ```yaml policies: - name: waf-policy-one - name: waf-policy-two ``` In this example, NGINX Ingress Controller uses the configuration from the first policy reference, `waf-policy-one`, and ignores `waf-policy-two`. ## HSTS The HSTS policy sets up [HTTP Strict Transport Security](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Strict-Transport-Security). It instructs browsers to enforce HTTPS connections to the host for a specified duration. For example, the following policy sets an HSTS duration of 30 days and extends the policy to all host subdomains: ```yaml hsts: maxAge: 2592000 includeSubDomains: true ``` When you deploy NGINX Ingress Controller behind a proxy or load balancer that terminates TLS upstream, set `behindProxy` to `true`. In this mode, NGINX uses the `X-Forwarded-Proto` request header to determine whether the connection is HTTPS, instead of checking the `$https` variable directly: ```yaml hsts: maxAge: 2592000 behindProxy: true ``` A VirtualServer that references an HSTS policy must: - Use [TLS termination](/nic/configuration/virtualserver-and-virtualserverroute-resources.md#virtualservertls), or set `behindProxy` to `true` if TLS is terminated upstream. - Reference the policy in the VirtualServer [spec](/nic/configuration/virtualserver-and-virtualserverroute-resources.md#virtualserver-specification). You can't reference an HSTS policy in a [route](/nic/configuration/virtualserver-and-virtualserverroute-resources.md#virtualserverroute) or in a VirtualServerRoute [subroute](/nic/configuration/virtualserver-and-virtualserverroute-resources.md#virtualserverroutesubroute). If a resource doesn't meet these conditions, NGINX sends status code `500` to clients. **Note:** This feature uses the NGINX `add_header` directive and the [ngx_http_ssl_module](https://nginx.org/en/docs/http/ngx_http_ssl_module.html) `$https` variable. |Field | Description | Type | Required | Default | | ---| ---| ---| --- | --- | |`maxAge` | Sets the duration in seconds that the browser should cache and enforce the HSTS policy. | `int` | Yes | -- | |`includeSubDomains` | Extends the HSTS policy to all subdomains of the host. | `bool` | No | `false` | |`behindProxy` | Sets the HSTS header based on the `X-Forwarded-Proto` request header rather than the `$https` variable. Set this to `true` when you deploy NGINX Ingress Controller behind a proxy or load balancer that terminates TLS upstream. | `bool` | No | `false` | |`preload` | Adds the domain to browsers' [HSTS preload lists](https://hstspreload.org/). Requires `includeSubDomains` to be set to `true` and `maxAge` to be at least 31536000 (one year). | `bool` | No | `false` | **Note:** HSTS instructs browsers to enforce HTTPS for the duration of `maxAge`. Deleting the policy doesn't clear the browser's cached directive, so users may be unable to access the application over HTTP until the cached policy expires. To remove HSTS safely, first set `maxAge` to `0` and apply the updated policy. This instructs browsers to expire the cached directive immediately. Once applied, remove the policy reference from the VirtualServer and delete the policy resource. See the MDN documentation on [HSTS expiration](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Strict-Transport-Security#expiration) for more details. ### HSTS merging behavior A VirtualServer can reference only a single HSTS policy, and NGINX Ingress Controller ignores every subsequent reference. For example, this configuration references two policies: ```yaml policies: - name: hsts-policy-one - name: hsts-policy-two ``` In this example, NGINX Ingress Controller uses the configuration from the first policy reference, `hsts-policy-one`, and ignores `hsts-policy-two`. ## What's next Learn how to [manage Policy resources with kubectl](/nic/configuration/policy-resource/using-policy.md). -------------------------------------------------------------------------------- title: "Reporting resource status" url: https://docs.nginx.com/nginx-ingress-controller/configuration/global-configuration/reporting-resources-status/index.md -------------------------------------------------------------------------------- This page describes how to view the status of resources managed by F5 NGINX Ingress Controller. ## Ingress resources An Ingress resource status includes the address (an IP address or a DNS name), through which the hosts of that Ingress resource are publicly accessible. You can see the address in the output of the `kubectl get ingress` command, in the ADDRESS column, as shown below: ```shell kubectl get ingresses ``` ```text NAME HOSTS ADDRESS PORTS AGE cafe-ingress cafe.example.com 12.13.23.123 80, 443 2m ``` NGINX Ingress Controller must be configured to report an Ingress status: 1. Use the command-line flag `-report-ingress-status`. 1. Define a source for an external address. This can be either of: 1. A user defined address, specified in the `external-status-address` ConfigMap key. 1. A Service of the type LoadBalancer configured with an external IP or address and specified by the `-external-service` command-line flag. View the [ConfigMap keys](/nic/configuration/global-configuration/configmap-resource.md) and [Command-line arguments](/nic/configuration/global-configuration/command-line-arguments.md) topics for more information. **Note:** NGINX Ingress Controller does not clear the status of Ingress resources when it is being shut down. ## VirtualServer and VirtualServerRoute resources A VirtualServer or VirtualServerRoute resource includes the status field with information about the state of the resource and the IP address, through which the hosts of that resource are publicly accessible. You can see the status in the output of the `kubectl get virtualservers` or `kubectl get virtualserverroutes` commands as shown below: ```shell kubectl get virtualservers ``` ```text NAME STATE HOST IP PORTS AGE cafe Valid cafe.example.com 12.13.23.123 [80,443] 34s ``` To see an external hostname address associated with a VirtualServer resource, use the `-o wide` option: ```shell kubectl get virtualservers -o wide ``` ```text NAME STATE HOST IP EXTERNALHOSTNAME PORTS AGE cafe Valid cafe.example.com ae430f41a1a0042908655abcdefghijkl-12345678.eu-west-2.elb.amazonaws.com [80,443] 106s ``` **Note:** If there are multiple addresses, only the first one is shown. In order to see additional addresses or extra information about the `Status` of the resource, use the following command: ```shell kubectl describe virtualserver ``` ```text ... Status: External Endpoints: Ip: 12.13.23.123 Ports: [80,443] Message: Configuration for cafe/cafe was added or updated Reason: AddedOrUpdated State: Valid ``` ### Status specification The following fields are reported in both VirtualServer and VirtualServerRoute status: |Field | Description | Type | | ---| ---| --- | |*State* | Current state of the resource. Can be ``Valid``, ``Warning`` an ``Invalid``. For more information, refer to the ``message`` field. | *string* | |*Reason* | The reason of the last update. | *string* | |*Message* | Additional information about the state. | *string* | |*ExternalEndpoints* | A list of external endpoints for which the hosts of the resource are publicly accessible. | *[externalEndpoint](#externalendpoint)* | The *ReferencedBy* field is reported for the VirtualServerRoute status only: |Field | Description | Type | | ---| ---| --- | | *ReferencedBy* | The VirtualServer that references this VirtualServerRoute. Format as ``namespace/name`` | *string* | ### externalEndpoint |Field | Description | Type | | ---| ---| --- | |``IP`` | The external IP address. | ``string`` | |``Hostname`` | The external LoadBalancer Hostname address. | ``string`` | |``Ports`` | A list of external ports. | ``string`` | NGINX Ingress Controller must be configured to report a VirtualServer or VirtualServerRoute status: 1. If you want NGINX Ingress Controller to report the `externalEndpoints`, define a source for an external address (The rest of the fields will be reported without the external address configured). This can be: 1. A user defined address, specified in the `external-status-address` ConfigMap key. 1. A Service of the type LoadBalancer configured with an external IP or address and specified by the `-external-service` command-line flag. View the [ConfigMap keys](/nic/configuration/global-configuration/configmap-resource.md) and [Command-line arguments](/nic/configuration/global-configuration/command-line-arguments.md) topics for more information. **Note:** NGINX Ingress Controller does not clear the status of VirtualServer and VirtualServerRoute resources when it is being shut down. ## Policy resources A Policy resource includes the status field with information about the state of the resource. You can see the status in the output of the `kubectl get policy` command as shown below: ```shell kubectl get policy ``` ```text NAME STATE AGE webapp-policy Valid 30s ``` In order to see additional addresses or extra information about the `Status` of the resource, use the following command: ```shell kubectl describe policy ``` ```text ... Status: Message: Configuration for default/webapp-policy was added or updated Reason: AddedOrUpdated State: Valid ``` ### Status specification The following fields are reported in Policy status: |Field | Description | Type | | ---| ---| --- | |``State`` | Current state of the resource. Can be ``Valid`` or ``Invalid``. For more information, refer to the ``message`` field. | ``string`` | |``Reason`` | The reason of the last update. | ``string`` | |``Message`` | Additional information about the state. | ``string`` | ## TransportServer resources A TransportServer resource includes the status field with information about the state of the resource. You can see the status in the output of the `kubectl get transportserver` command as shown below: ```shell kubectl get transportserver ``` ```text NAME STATE REASON AGE dns-tcp Valid AddedOrUpdated 47m ``` To see additional addresses or extra information about the `Status` of the resource, use the following command: ```shell kubectl describe transportserver ``` ```text Status: Message: Configuration for default/dns-tcp was added or updated Reason: AddedOrUpdated State: Valid ``` ### Status specification The following fields are reported in TransportServer status: |Field | Description | Type | | ---| ---| --- | | *State* | Current state of the resource. Can be ``Valid``, ``Warning`` or ``Invalid``. For more information, refer to the ``message`` field. | *string* | | *Reason* | The reason of the last update. | *string* | | *Message* | Additional information about the state. | *string* | -------------------------------------------------------------------------------- title: "Reporting resource status" url: https://docs.nginx.com/nginx-ingress-controller/lts/configuration/global-configuration/reporting-resources-status/index.md -------------------------------------------------------------------------------- This page describes how to view the status of resources managed by F5 NGINX Ingress Controller LTS. ## Ingress resources An Ingress resource status includes the address (an IP address or a DNS name), through which the hosts of that Ingress resource are publicly accessible. You can see the address in the output of the `kubectl get ingress` command, in the ADDRESS column, as shown below: ```shell kubectl get ingresses ``` ```text NAME HOSTS ADDRESS PORTS AGE cafe-ingress cafe.example.com 12.13.23.123 80, 443 2m ``` NGINX Ingress Controller LTS must be configured to report an Ingress status: 1. Use the command-line flag `-report-ingress-status`. 1. Define a source for an external address. This can be either of: 1. A user defined address, specified in the `external-status-address` ConfigMap key. 1. A Service of the type LoadBalancer configured with an external IP or address and specified by the `-external-service` command-line flag. View the [ConfigMap keys](/nic/lts/configuration/global-configuration/configmap-resource.md) and [Command-line arguments](/nic/lts/configuration/global-configuration/command-line-arguments.md) topics for more information. **note:** NGINX Ingress Controller LTS does not clear the status of Ingress resources when it is being shut down. ## VirtualServer and VirtualServerRoute resources A VirtualServer or VirtualServerRoute resource includes the status field with information about the state of the resource and the IP address, through which the hosts of that resource are publicly accessible. You can see the status in the output of the `kubectl get virtualservers` or `kubectl get virtualserverroutes` commands as shown below: ```shell kubectl get virtualservers ``` ```text NAME STATE HOST IP PORTS AGE cafe Valid cafe.example.com 12.13.23.123 [80,443] 34s ``` To see an external hostname address associated with a VirtualServer resource, use the `-o wide` option: ```shell kubectl get virtualservers -o wide ``` ```text NAME STATE HOST IP EXTERNALHOSTNAME PORTS AGE cafe Valid cafe.example.com ae430f41a1a0042908655abcdefghijkl-12345678.eu-west-2.elb.amazonaws.com [80,443] 106s ``` **note:** If there are multiple addresses, only the first one is shown. In order to see additional addresses or extra information about the `Status` of the resource, use the following command: ```shell kubectl describe virtualserver ``` ```text ... Status: External Endpoints: Ip: 12.13.23.123 Ports: [80,443] Message: Configuration for cafe/cafe was added or updated Reason: AddedOrUpdated State: Valid ``` ### Status specification The following fields are reported in both VirtualServer and VirtualServerRoute status: |Field | Description | Type | | ---| ---| --- | |*State* | Current state of the resource. Can be ``Valid``, ``Warning`` an ``Invalid``. For more information, refer to the ``message`` field. | *string* | |*Reason* | The reason of the last update. | *string* | |*Message* | Additional information about the state. | *string* | |*ExternalEndpoints* | A list of external endpoints for which the hosts of the resource are publicly accessible. | *[externalEndpoint](#externalendpoint)* | The *ReferencedBy* field is reported for the VirtualServerRoute status only: |Field | Description | Type | | ---| ---| --- | | *ReferencedBy* | The VirtualServer that references this VirtualServerRoute. Format as ``namespace/name`` | *string* | ### externalEndpoint |Field | Description | Type | | ---| ---| --- | |``IP`` | The external IP address. | ``string`` | |``Hostname`` | The external LoadBalancer Hostname address. | ``string`` | |``Ports`` | A list of external ports. | ``string`` | NGINX Ingress Controller LTS must be configured to report a VirtualServer or VirtualServerRoute status: 1. If you want NGINX Ingress Controller LTS to report the `externalEndpoints`, define a source for an external address (The rest of the fields will be reported without the external address configured). This can be: 1. A user defined address, specified in the `external-status-address` ConfigMap key. 1. A Service of the type LoadBalancer configured with an external IP or address and specified by the `-external-service` command-line flag. View the [ConfigMap keys](/nic/lts/configuration/global-configuration/configmap-resource.md) and [Command-line arguments](/nic/lts/configuration/global-configuration/command-line-arguments.md) topics for more information. **note:** NGINX Ingress Controller LTS does not clear the status of VirtualServer and VirtualServerRoute resources when it is being shut down. ## Policy resources A Policy resource includes the status field with information about the state of the resource. You can see the status in the output of the `kubectl get policy` command as shown below: ```shell kubectl get policy ``` ```text NAME STATE AGE webapp-policy Valid 30s ``` In order to see additional addresses or extra information about the `Status` of the resource, use the following command: ```shell kubectl describe policy ``` ```text ... Status: Message: Configuration for default/webapp-policy was added or updated Reason: AddedOrUpdated State: Valid ``` ### Status specification The following fields are reported in Policy status: |Field | Description | Type | | ---| ---| --- | |``State`` | Current state of the resource. Can be ``Valid`` or ``Invalid``. For more information, refer to the ``message`` field. | ``string`` | |``Reason`` | The reason of the last update. | ``string`` | |``Message`` | Additional information about the state. | ``string`` | ## TransportServer resources A TransportServer resource includes the status field with information about the state of the resource. You can see the status in the output of the `kubectl get transportserver` command as shown below: ```shell kubectl get transportserver ``` ```text NAME STATE REASON AGE dns-tcp Valid AddedOrUpdated 47m ``` To see additional addresses or extra information about the `Status` of the resource, use the following command: ```shell kubectl describe transportserver ``` ```text Status: Message: Configuration for default/dns-tcp was added or updated Reason: AddedOrUpdated State: Valid ``` ### Status specification The following fields are reported in TransportServer status: |Field | Description | Type | | ---| ---| --- | | *State* | Current state of the resource. Can be ``Valid``, ``Warning`` or ``Invalid``. For more information, refer to the ``message`` field. | *string* | | *Reason* | The reason of the last update. | *string* | | *Message* | Additional information about the state. | *string* | -------------------------------------------------------------------------------- title: "TransportServer resources" url: https://docs.nginx.com/nginx-ingress-controller/configuration/transportserver-resource/index.md -------------------------------------------------------------------------------- This document is reference material for the TransportServer resource used by F5 NGINX Ingress Controller. The TransportServer resource allows you to configure TCP, UDP, and TLS Passthrough load balancing. The resource is implemented as a [Custom Resource](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/). The GitHub repository has [examples of the resources](https://github.com/nginx/kubernetes-ingress/tree/v/examples/custom-resources) for specific use cases. ## Prerequisites - For TCP and UDP, the TransportServer resource must be used in conjunction with the [GlobalConfiguration resource](/nic/configuration/global-configuration/globalconfiguration-resource.md), which must be created separately. - For TLS Passthrough, make sure to enable the [`-enable-tls-passthrough`](/nic/configuration/global-configuration/command-line-arguments#cmdoption-enable-tls-passthrough.md) command-line argument of NGINX Ingress Controller. ## TransportServer Specification The TransportServer resource defines load balancing configuration for TCP, UDP, or TLS Passthrough traffic. Below are a few examples: - TCP load balancing: ```yaml apiVersion: k8s.nginx.org/v1 kind: TransportServer metadata: name: dns-tcp spec: host: dns.example.com listener: name: dns-tcp protocol: TCP tls: secret: cafe-secret upstreams: - name: dns-app service: dns-service port: 5353 action: pass: dns-app ``` - UDP load balancing: ```yaml apiVersion: k8s.nginx.org/v1 kind: TransportServer metadata: name: dns-udp spec: listener: name: dns-udp protocol: UDP upstreams: - name: dns-app service: dns-service port: 5353 upstreamParameters: udpRequests: 1 udpResponses: 1 action: pass: dns-app ``` - TLS passthrough load balancing: ```yaml apiVersion: k8s.nginx.org/v1 kind: TransportServer metadata: name: secure-app spec: listener: name: tls-passthrough protocol: TLS_PASSTHROUGH host: app.example.com upstreams: - name: secure-app service: secure-app port: 8443 action: pass: secure-app ``` | Field | Description | Type | Required | | ---| ---| ---| --- | |``listener`` | The listener on NGINX that will accept incoming connections/datagrams. | [listener](#listener) | Yes | |``host`` | The host (domain name) of the server. Must be a valid subdomain as defined in RFC 1123, such as ``my-app`` or ``hello.example.com``. Wildcard domains like ``*.example.com`` are not allowed. When specified, NGINX will use this host for SNI-based routing. For TLS Passthrough, this field is required. For TCP with TLS termination, specifying the host enables SNI routing and requires specifying a TLS secret.| ``string`` | No | |``tls`` | The TLS termination configuration. Not supported for TLS Passthrough load balancing. | [tls](#tls) | No | |``upstreams`` | A list of upstreams. | [[]upstream](#upstream) | Yes | |``upstreamParameters`` | The upstream parameters. | [upstreamParameters](#upstreamparameters) | No | |``action`` | The action to perform for a client connection/datagram. | [action](#action) | Yes | |``ingressClassName`` | Specifies which Ingress Controller must handle the TransportServer resource. | ``string`` | No | |``streamSnippets`` | Sets a custom snippet in the ``stream`` context. | ``string`` | No | |``serverSnippets`` | Sets a custom snippet in the ``server`` context. | ``string`` | No | \* -- Required for TLS Passthrough load balancing. ### Listener The listener field references a listener that NGINX will use to accept incoming traffic for the TransportServer. For TCP and UDP, the listener must be defined in the [GlobalConfiguration resource](/nic/configuration/global-configuration/globalconfiguration-resource.md). When referencing a listener, both the name and the protocol must match. For TLS Passthrough, use the built-in listener with the name `tls-passthrough` and the protocol `TLS_PASSTHROUGH`. The combination of ``spec.listener.name`` and ``spec.host`` must be unique among all TransportServers. If two TransportServers specify the same combination of ``spec.listener.name`` and ``spec.host``, one of them will be rejected to prevent conflicts. In the case where no host is specified, it is considered an empty string. An example: ```yaml listener: name: dns-udp protocol: UDP ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``name`` | The name of the listener. | ``string`` | Yes | |``protocol`` | The protocol of the listener. | ``string`` | Yes | ### TLS The tls field defines TLS configuration for a TransportServer. When using TLS termination (not TLS Passthrough), you can specify the host field to enable SNI-based routing, allowing multiple applications to share the same listener port and be distinguished by the TLS SNI hostname. Each application can use its own TLS certificate and key specified via the secret field. ```yaml secret: cafe-secret ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``secret`` | The name of a secret with a TLS certificate and key. The secret must belong to the same namespace as the TransportServer. The secret must be of the type ``kubernetes.io/tls`` and contain keys named ``tls.crt`` and ``tls.key`` that contain the certificate and private key as described [here](https://kubernetes.io/docs/concepts/services-networking/ingress/#tls). | ``string`` | No | ### Upstream The upstream defines a destination for the TransportServer. For example: ```yaml name: secure-app service: secure-app port: 8443 maxFails: 3 maxConns: 100 failTimeout: 30s loadBalancingMethod: least_conn ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``name`` | The name of the upstream. Must be a valid DNS label as defined in RFC 1035. For example, ``hello`` and ``upstream-123`` are valid. The name must be unique among all upstreams of the resource. | ``string`` | Yes | |``service`` | The name of a [service](https://kubernetes.io/docs/concepts/services-networking/service/). The service must belong to the same namespace as the resource. If the service doesn't exist, NGINX will assume the service has zero endpoints and close client connections/ignore datagrams. | ``string`` | Yes | |``port`` | The port of the service. If the service doesn't define that port, NGINX will assume the service has zero endpoints and close client connections/ignore datagrams. The port must fall into the range ``1..65535``. | ``int`` | Yes | |``maxFails`` | Sets the [number](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#max_fails) of unsuccessful attempts to communicate with the server that should happen in the duration set by the failTimeout parameter to consider the server unavailable. The default ``1``. | ``int`` | No | |``maxConns`` | Sets the [number](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#max_conns) of maximum connections to the proxied server. Default value is zero, meaning there is no limit. The default is ``0``. | ``int`` | No | |``failTimeout`` | Sets the [time](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#fail_timeout) during which the specified number of unsuccessful attempts to communicate with the server should happen to consider the server unavailable and the period of time the server will be considered unavailable. The default is ``10s``. | ``string`` | No | |``healthCheck`` | The health check configuration for the Upstream. See the [health_check](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#health_check) directive. Note: this feature is supported only in NGINX Plus. | [healthcheck](#upstreamhealthcheck) | No | |``loadBalancingMethod`` | The method used to load balance the upstream servers. By default, connections are distributed between the servers using a weighted round-robin balancing method. See the [upstream](http://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#upstream) section for available methods and their details. | ``string`` | No | |``backup`` | The name of the backup service of type [ExternalName](https://kubernetes.io/docs/concepts/services-networking/service/#externalname). This will be used when the primary servers are unavailable. Note: The parameter cannot be used along with the ``random`` , ``hash`` or ``ip_hash`` load balancing methods. | ``string`` | No | |``backupPort`` | The port of the backup service. The backup port is required if the backup service name is provided. The port must fall into the range ``1..65535``. | ``uint16`` | No | ### Upstream.Healthcheck The Healthcheck defines an [active health check](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html?#health_check). In the example below we enable a health check for an upstream and configure all the available parameters: ```yaml name: secure-app service: secure-app port: 8443 healthCheck: enable: true interval: 20s timeout: 30s jitter: 3s fails: 5 passes: 5 port: 8080 ``` **Note:** This feature is only supported with NGINX Plus. |Field | Description | Type | Required | | ---| ---| ---| --- | |``enable`` | Enables a health check for an upstream server. The default is ``false``. | ``boolean`` | No | |``interval`` | The interval between two consecutive health checks. The default is ``5s``. | ``string`` | No | |``timeout`` | This overrides the timeout set by [proxy_timeout](http://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_timeout) which is set in `SessionParameters` for health checks. The default value is ``5s``. | ``string`` | No | |``jitter`` | The time within which each health check will be randomly delayed. By default, there is no delay. | ``string`` | No | |``fails`` | The number of consecutive failed health checks of a particular upstream server after which this server will be considered unhealthy. The default is ``1``. | ``integer`` | No | |``passes`` | The number of consecutive passed health checks of a particular upstream server after which the server will be considered healthy. The default is ``1``. | ``integer`` | No | |``port`` | The port used for health check requests. By default, the [server port is used](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#health_check_port). Note: in contrast with the port of the upstream, this port is not a service port, but a port of a pod. | ``integer`` | No | |``match`` | Controls the data to send and the response to expect for the healthcheck. | [match](#upstreamhealthcheckmatch) | No | ### Upstream.Healthcheck.Match The match controls the data to send and the response to expect for the healthcheck: ```yaml match: send: 'GET / HTTP/1.0\r\nHost: localhost\r\n\r\n' expect: "~200 OK" ``` Both `send` and `expect` fields can contain hexadecimal literals with the prefix `\x` followed by two hex digits, for example, `\x80`. See the [match](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#match) directive for details. |Field | Description | Type | Required | | ---| ---| ---| --- | |``send`` | A string to send to an upstream server. | ``string`` | No | |``expect`` | A literal string or a regular expression that the data obtained from the server should match. The regular expression is specified with the preceding ``~*`` modifier (for case-insensitive matching), or the ``~`` modifier (for case-sensitive matching). NGINX Ingress Controller validates a regular expression using the RE2 syntax. | ``string`` | No | ### UpstreamParameters The upstream parameters define various parameters for the upstreams: ```yaml upstreamParameters: udpRequests: 1 udpResponses: 1 connectTimeout: 60s nextUpstream: true nextUpstreamTimeout: 50s nextUpstreamTries: 1 ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``udpRequests`` | The number of datagrams, after receiving which, the next datagram from the same client starts a new session. See the [proxy_requests](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_requests) directive. The default is ``0``. | ``int`` | No | |``udpResponses`` | The number of datagrams expected from the proxied server in response to a client datagram. See the [proxy_responses](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_responses) directive. By default, the number of datagrams is not limited. | ``int`` | No | |``connectTimeout`` | The timeout for establishing a connection with a proxied server. See the [proxy_connect_timeout](http://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_connect_timeout) directive. The default is ``60s``. | ``string`` | No | |``nextUpstream`` | If a connection to the proxied server cannot be established, determines whether a client connection will be passed to the next server. See the [proxy_next_upstream](http://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_next_upstream) directive. The default is ``true``. | bool | No | |``nextUpstreamTries`` | The number of tries for passing a connection to the next server. See the [proxy_next_upstream_tries](http://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_next_upstream_tries) directive. The default is ``0``. | ``int`` | No | |``nextUpstreamTimeout`` | The time allowed to pass a connection to the next server. See the [proxy_next_upstream_timeout](http://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_next_upstream_timeout) directive. The default us ``0``. | ``string`` | No | ### SessionParameters The session parameters define various parameters for TCP connections and UDP sessions. ```yaml sessionParameters: timeout: 50s ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``timeout`` | The timeout between two successive read or write operations on client or proxied server connections. See [proxy_timeout](http://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_timeout) directive. The default is ``10m``. | ``string`` | No | ### Action The action defines an action to perform for a client connection/datagram. In the example below, client connections/datagrams are passed to an upstream `dns-app`: ```yaml action: pass: dns-app ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``pass`` | Passes connections/datagrams to an upstream. The upstream with that name must be defined in the resource. | ``string`` | Yes | ## Using TransportServer You can use the usual `kubectl` commands to work with TransportServer resources, similar to Ingress resources. For example, the following command creates a TransportServer resource defined in `transport-server-passthrough.yaml` with the name `secure-app`: ```shell kubectl apply -f transport-server-passthrough.yaml ``` ```text transportserver.k8s.nginx.org/secure-app created ``` You can get the resource by running: ```shell kubectl get transportserver secure-app ``` ```text NAME AGE secure-app 46sm ``` In the kubectl get and similar commands, you can also use the short name `ts` instead of `transportserver`. ### Using Snippets Snippets allow you to insert raw NGINX config into different contexts of NGINX configuration. In the example below, we use snippets to configure [access control](http://nginx.org/en/docs/stream/ngx_stream_access_module.html) in a TransportServer: ```yaml apiVersion: k8s.nginx.org/v1 kind: TransportServer metadata: name: cafe spec: host: cafe.example.com serverSnippets: | deny 192.168.1.1; allow 192.168.1.0/24; upstreams: - name: tea service: tea-svc port: 80 ``` Snippets can also be specified for a stream. In the example below, we use snippets to [limit the number of connections](https://nginx.org/en/docs/stream/ngx_stream_limit_conn_module.html): ```yaml apiVersion: k8s.nginx.org/v1 kind: TransportServer metadata: name: cafe spec: host: cafe.example.com streamSnippets: limit_conn_zone $binary_remote_addr zone=addr:10m; serverSnippets: limit_conn addr 1; upstreams: - name: tea service: tea-svc port: 80 ``` **Note:** To configure snippets in the `stream` context, use `stream-snippets` ConfigMap key. For additional information, view the [Advanced configuration with Snippets](/nic/configuration/ingress-resources/advanced-configuration-with-snippets.md) topic. ### Validation Two types of validation are available for the TransportServer resource: - *Structural validation* by the `kubectl` and Kubernetes API server. - *Comprehensive validation* by NGINX Ingress Controller. #### Structural Validation The custom resource definition for the TransportServer includes structural OpenAPI schema which describes the type of every field of the resource. If you try to create (or update) a resource that violates the structural schema (for example, you use a string value for the port field of an upstream), `kubectl` and Kubernetes API server will reject such a resource: - Example of `kubectl` validation: ```shell kubectl apply -f transport-server-passthrough.yaml ``` ```text error: error validating "transport-server-passthrough.yaml": error validating data: ValidationError(TransportServer.spec.upstreams[0].port): invalid type for org.nginx.k8s.v1.TransportServer.spec.upstreams.port: got "string", expected "integer"; if you choose to ignore these errors, turn validation off with --validate=false ``` - Example of Kubernetes API server validation: ```shell kubectl apply -f transport-server-passthrough.yaml --validate=false ``` ```text The TransportServer "secure-app" is invalid: []: Invalid value: map[string]interface {}{ ... }: validation failure list: spec.upstreams.port in body must be of type integer: "string" ``` If a resource is not rejected (it doesn't violate the structural schema), NGINX Ingress Controller will validate it further. #### Comprehensive Validation NGINX Ingress Controller validates the fields of a TransportServer resource. If a resource is invalid, NGINX Ingress Controller will reject it: the resource will continue to exist in the cluster, but NGINX Ingress Controller will ignore it. You can check if NGINX Ingress Controller successfully applied the configuration for a TransportServer. For our example `secure-app` TransportServer, we can run: ```shell kubectl describe ts secure-app ``` ```text ... Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal AddedOrUpdated 3s nginx-ingress-controller Configuration for default/secure-app was added or updated ``` Note how the events section includes a Normal event with the AddedOrUpdated reason that informs us that the configuration was successfully applied. If you create an invalid resource, NGINX Ingress Controller will reject it and emit a Rejected event. For example, if you create a TransportServer `secure-app` with a pass action that references a non-existing upstream, you will get : ```shell kubectl describe ts secure-app ``` ```text ... Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning Rejected 2s nginx-ingress-controller TransportServer default/secure-app is invalid and was rejected: spec.action.pass: Not found: "some-app" ``` Note how the events section includes a Warning event with the Rejected reason. **Note**: If you make an existing resource invalid, NGINX Ingress Controller will reject it and remove the corresponding configuration from NGINX. ## Customization via ConfigMap The [ConfigMap](/nic/configuration/global-configuration/configmap-resource.md) keys (except for `stream-snippets`, `stream-log-format`, `resolver-addresses`, `resolver-ipv6`, `resolver-valid` and `resolver-timeout`) do not affect TransportServer resources. -------------------------------------------------------------------------------- title: "TransportServer resources" url: https://docs.nginx.com/nginx-ingress-controller/lts/configuration/transportserver-resource/index.md -------------------------------------------------------------------------------- This document is reference material for the TransportServer resource used by F5 NGINX Ingress Controller LTS. The TransportServer resource allows you to configure TCP, UDP, and TLS Passthrough load balancing. The resource is implemented as a [Custom Resource](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/). The GitHub repository has [examples of the resources](https://github.com/nginx/kubernetes-ingress/tree/v/examples/custom-resources) for specific use cases. ## Prerequisites - For TCP and UDP, the TransportServer resource must be used in conjunction with the [GlobalConfiguration resource](/nic/lts/configuration/global-configuration/globalconfiguration-resource.md), which must be created separately. - For TLS Passthrough, make sure to enable the [`-enable-tls-passthrough`](/nic/lts/configuration/global-configuration/command-line-arguments#cmdoption-enable-tls-passthrough.md) command-line argument of NGINX Ingress Controller LTS. ## TransportServer Specification The TransportServer resource defines load balancing configuration for TCP, UDP, or TLS Passthrough traffic. Below are a few examples: - TCP load balancing: ```yaml apiVersion: k8s.nginx.org/v1 kind: TransportServer metadata: name: dns-tcp spec: host: dns.example.com listener: name: dns-tcp protocol: TCP tls: secret: cafe-secret upstreams: - name: dns-app service: dns-service port: 5353 action: pass: dns-app ``` - UDP load balancing: ```yaml apiVersion: k8s.nginx.org/v1 kind: TransportServer metadata: name: dns-udp spec: listener: name: dns-udp protocol: UDP upstreams: - name: dns-app service: dns-service port: 5353 upstreamParameters: udpRequests: 1 udpResponses: 1 action: pass: dns-app ``` - TLS passthrough load balancing: ```yaml apiVersion: k8s.nginx.org/v1 kind: TransportServer metadata: name: secure-app spec: listener: name: tls-passthrough protocol: TLS_PASSTHROUGH host: app.example.com upstreams: - name: secure-app service: secure-app port: 8443 action: pass: secure-app ``` | Field | Description | Type | Required | | ---| ---| ---| --- | |``listener`` | The listener on NGINX that will accept incoming connections/datagrams. | [listener](#listener) | Yes | |``host`` | The host (domain name) of the server. Must be a valid subdomain as defined in RFC 1123, such as ``my-app`` or ``hello.example.com``. Wildcard domains like ``*.example.com`` are not allowed. When specified, NGINX will use this host for SNI-based routing. For TLS Passthrough, this field is required. For TCP with TLS termination, specifying the host enables SNI routing and requires specifying a TLS secret.| ``string`` | No | |``tls`` | The TLS termination configuration. Not supported for TLS Passthrough load balancing. | [tls](#tls) | No | |``upstreams`` | A list of upstreams. | [[]upstream](#upstream) | Yes | |``upstreamParameters`` | The upstream parameters. | [upstreamParameters](#upstreamparameters) | No | |``action`` | The action to perform for a client connection/datagram. | [action](#action) | Yes | |``ingressClassName`` | Specifies which Ingress Controller must handle the TransportServer resource. | ``string`` | No | |``streamSnippets`` | Sets a custom snippet in the ``stream`` context. | ``string`` | No | |``serverSnippets`` | Sets a custom snippet in the ``server`` context. | ``string`` | No | \* -- Required for TLS Passthrough load balancing. ### Listener The listener field references a listener that NGINX will use to accept incoming traffic for the TransportServer. For TCP and UDP, the listener must be defined in the [GlobalConfiguration resource](/nic/lts/configuration/global-configuration/globalconfiguration-resource.md). When referencing a listener, both the name and the protocol must match. For TLS Passthrough, use the built-in listener with the name `tls-passthrough` and the protocol `TLS_PASSTHROUGH`. The combination of ``spec.listener.name`` and ``spec.host`` must be unique among all TransportServers. If two TransportServers specify the same combination of ``spec.listener.name`` and ``spec.host``, one of them will be rejected to prevent conflicts. In the case where no host is specified, it is considered an empty string. An example: ```yaml listener: name: dns-udp protocol: UDP ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``name`` | The name of the listener. | ``string`` | Yes | |``protocol`` | The protocol of the listener. | ``string`` | Yes | ### TLS The tls field defines TLS configuration for a TransportServer. When using TLS termination (not TLS Passthrough), you can specify the host field to enable SNI-based routing, allowing multiple applications to share the same listener port and be distinguished by the TLS SNI hostname. Each application can use its own TLS certificate and key specified via the secret field. ```yaml secret: cafe-secret ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``secret`` | The name of a secret with a TLS certificate and key. The secret must belong to the same namespace as the TransportServer. The secret must be of the type ``kubernetes.io/tls`` and contain keys named ``tls.crt`` and ``tls.key`` that contain the certificate and private key as described [here](https://kubernetes.io/docs/concepts/services-networking/ingress/#tls). | ``string`` | No | ### Upstream The upstream defines a destination for the TransportServer. For example: ```yaml name: secure-app service: secure-app port: 8443 maxFails: 3 maxConns: 100 failTimeout: 30s loadBalancingMethod: least_conn ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``name`` | The name of the upstream. Must be a valid DNS label as defined in RFC 1035. For example, ``hello`` and ``upstream-123`` are valid. The name must be unique among all upstreams of the resource. | ``string`` | Yes | |``service`` | The name of a [service](https://kubernetes.io/docs/concepts/services-networking/service/). The service must belong to the same namespace as the resource. If the service doesn't exist, NGINX will assume the service has zero endpoints and close client connections/ignore datagrams. | ``string`` | Yes | |``port`` | The port of the service. If the service doesn't define that port, NGINX will assume the service has zero endpoints and close client connections/ignore datagrams. The port must fall into the range ``1..65535``. | ``int`` | Yes | |``maxFails`` | Sets the [number](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#max_fails) of unsuccessful attempts to communicate with the server that should happen in the duration set by the failTimeout parameter to consider the server unavailable. The default ``1``. | ``int`` | No | |``maxConns`` | Sets the [number](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#max_conns) of maximum connections to the proxied server. Default value is zero, meaning there is no limit. The default is ``0``. | ``int`` | No | |``failTimeout`` | Sets the [time](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#fail_timeout) during which the specified number of unsuccessful attempts to communicate with the server should happen to consider the server unavailable and the period of time the server will be considered unavailable. The default is ``10s``. | ``string`` | No | |``healthCheck`` | The health check configuration for the Upstream. See the [health_check](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#health_check) directive. | [healthcheck](#upstreamhealthcheck) | No | |``loadBalancingMethod`` | The method used to load balance the upstream servers. By default, connections are distributed between the servers using a weighted round-robin balancing method. See the [upstream](http://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#upstream) section for available methods and their details. | ``string`` | No | |``backup`` | The name of the backup service of type [ExternalName](https://kubernetes.io/docs/concepts/services-networking/service/#externalname). This will be used when the primary servers are unavailable. Note: The parameter cannot be used along with the ``random`` , ``hash`` or ``ip_hash`` load balancing methods. | ``string`` | No | |``backupPort`` | The port of the backup service. The backup port is required if the backup service name is provided. The port must fall into the range ``1..65535``. | ``uint16`` | No | ### Upstream.Healthcheck The Healthcheck defines an [active health check](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html?#health_check). In the example below we enable a health check for an upstream and configure all the available parameters: ```yaml name: secure-app service: secure-app port: 8443 healthCheck: enable: true interval: 20s timeout: 30s jitter: 3s fails: 5 passes: 5 port: 8080 ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``enable`` | Enables a health check for an upstream server. The default is ``false``. | ``boolean`` | No | |``interval`` | The interval between two consecutive health checks. The default is ``5s``. | ``string`` | No | |``timeout`` | This overrides the timeout set by [proxy_timeout](http://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_timeout) which is set in `SessionParameters` for health checks. The default value is ``5s``. | ``string`` | No | |``jitter`` | The time within which each health check will be randomly delayed. By default, there is no delay. | ``string`` | No | |``fails`` | The number of consecutive failed health checks of a particular upstream server after which this server will be considered unhealthy. The default is ``1``. | ``integer`` | No | |``passes`` | The number of consecutive passed health checks of a particular upstream server after which the server will be considered healthy. The default is ``1``. | ``integer`` | No | |``port`` | The port used for health check requests. By default, the [server port is used](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#health_check_port). Note: in contrast with the port of the upstream, this port is not a service port, but a port of a pod. | ``integer`` | No | |``match`` | Controls the data to send and the response to expect for the healthcheck. | [match](#upstreamhealthcheckmatch) | No | ### Upstream.Healthcheck.Match The match controls the data to send and the response to expect for the healthcheck: ```yaml match: send: 'GET / HTTP/1.0\r\nHost: localhost\r\n\r\n' expect: "~200 OK" ``` Both `send` and `expect` fields can contain hexadecimal literals with the prefix `\x` followed by two hex digits, for example, `\x80`. See the [match](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#match) directive for details. |Field | Description | Type | Required | | ---| ---| ---| --- | |``send`` | A string to send to an upstream server. | ``string`` | No | |``expect`` | A literal string or a regular expression that the data obtained from the server should match. The regular expression is specified with the preceding ``~*`` modifier (for case-insensitive matching), or the ``~`` modifier (for case-sensitive matching). NGINX Ingress Controller LTS validates a regular expression using the RE2 syntax. | ``string`` | No | ### UpstreamParameters The upstream parameters define various parameters for the upstreams: ```yaml upstreamParameters: udpRequests: 1 udpResponses: 1 connectTimeout: 60s nextUpstream: true nextUpstreamTimeout: 50s nextUpstreamTries: 1 ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``udpRequests`` | The number of datagrams, after receiving which, the next datagram from the same client starts a new session. See the [proxy_requests](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_requests) directive. The default is ``0``. | ``int`` | No | |``udpResponses`` | The number of datagrams expected from the proxied server in response to a client datagram. See the [proxy_responses](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_responses) directive. By default, the number of datagrams is not limited. | ``int`` | No | |``connectTimeout`` | The timeout for establishing a connection with a proxied server. See the [proxy_connect_timeout](http://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_connect_timeout) directive. The default is ``60s``. | ``string`` | No | |``nextUpstream`` | If a connection to the proxied server cannot be established, determines whether a client connection will be passed to the next server. See the [proxy_next_upstream](http://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_next_upstream) directive. The default is ``true``. | bool | No | |``nextUpstreamTries`` | The number of tries for passing a connection to the next server. See the [proxy_next_upstream_tries](http://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_next_upstream_tries) directive. The default is ``0``. | ``int`` | No | |``nextUpstreamTimeout`` | The time allowed to pass a connection to the next server. See the [proxy_next_upstream_timeout](http://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_next_upstream_timeout) directive. The default us ``0``. | ``string`` | No | ### SessionParameters The session parameters define various parameters for TCP connections and UDP sessions. ```yaml sessionParameters: timeout: 50s ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``timeout`` | The timeout between two successive read or write operations on client or proxied server connections. See [proxy_timeout](http://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_timeout) directive. The default is ``10m``. | ``string`` | No | ### Action The action defines an action to perform for a client connection/datagram. In the example below, client connections/datagrams are passed to an upstream `dns-app`: ```yaml action: pass: dns-app ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``pass`` | Passes connections/datagrams to an upstream. The upstream with that name must be defined in the resource. | ``string`` | Yes | ## Using TransportServer You can use the usual `kubectl` commands to work with TransportServer resources, similar to Ingress resources. For example, the following command creates a TransportServer resource defined in `transport-server-passthrough.yaml` with the name `secure-app`: ```shell kubectl apply -f transport-server-passthrough.yaml ``` ```text transportserver.k8s.nginx.org/secure-app created ``` You can get the resource by running: ```shell kubectl get transportserver secure-app ``` ```text NAME AGE secure-app 46sm ``` In the kubectl get and similar commands, you can also use the short name `ts` instead of `transportserver`. ### Using Snippets Snippets allow you to insert raw NGINX config into different contexts of NGINX configuration. In the example below, we use snippets to configure [access control](http://nginx.org/en/docs/stream/ngx_stream_access_module.html) in a TransportServer: ```yaml apiVersion: k8s.nginx.org/v1 kind: TransportServer metadata: name: cafe spec: host: cafe.example.com serverSnippets: | deny 192.168.1.1; allow 192.168.1.0/24; upstreams: - name: tea service: tea-svc port: 80 ``` Snippets can also be specified for a stream. In the example below, we use snippets to [limit the number of connections](https://nginx.org/en/docs/stream/ngx_stream_limit_conn_module.html): ```yaml apiVersion: k8s.nginx.org/v1 kind: TransportServer metadata: name: cafe spec: host: cafe.example.com streamSnippets: limit_conn_zone $binary_remote_addr zone=addr:10m; serverSnippets: limit_conn addr 1; upstreams: - name: tea service: tea-svc port: 80 ``` **note:** To configure snippets in the `stream` context, use `stream-snippets` ConfigMap key. For additional information, view the [Advanced configuration with Snippets](/nic/lts/configuration/ingress-resources/advanced-configuration-with-snippets.md) topic. ### Validation Two types of validation are available for the TransportServer resource: - *Structural validation* by the `kubectl` and Kubernetes API server. - *Comprehensive validation* by NGINX Ingress Controller LTS. #### Structural Validation The custom resource definition for the TransportServer includes structural OpenAPI schema which describes the type of every field of the resource. If you try to create (or update) a resource that violates the structural schema (for example, you use a string value for the port field of an upstream), `kubectl` and Kubernetes API server will reject such a resource: - Example of `kubectl` validation: ```shell kubectl apply -f transport-server-passthrough.yaml ``` ```text error: error validating "transport-server-passthrough.yaml": error validating data: ValidationError(TransportServer.spec.upstreams[0].port): invalid type for org.nginx.k8s.v1.TransportServer.spec.upstreams.port: got "string", expected "integer"; if you choose to ignore these errors, turn validation off with --validate=false ``` - Example of Kubernetes API server validation: ```shell kubectl apply -f transport-server-passthrough.yaml --validate=false ``` ```text The TransportServer "secure-app" is invalid: []: Invalid value: map[string]interface {}{ ... }: validation failure list: spec.upstreams.port in body must be of type integer: "string" ``` If a resource is not rejected (it doesn't violate the structural schema), NGINX Ingress Controller LTS will validate it further. #### Comprehensive Validation NGINX Ingress Controller LTS validates the fields of a TransportServer resource. If a resource is invalid, NGINX Ingress Controller LTS will reject it: the resource will continue to exist in the cluster, but NGINX Ingress Controller LTS will ignore it. You can check if NGINX Ingress Controller LTS successfully applied the configuration for a TransportServer. For our example `secure-app` TransportServer, we can run: ```shell kubectl describe ts secure-app ``` ```text ... Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal AddedOrUpdated 3s nginx-ingress-controller Configuration for default/secure-app was added or updated ``` Note how the events section includes a Normal event with the AddedOrUpdated reason that informs us that the configuration was successfully applied. If you create an invalid resource, NGINX Ingress Controller LTS will reject it and emit a Rejected event. For example, if you create a TransportServer `secure-app` with a pass action that references a non-existing upstream, you will get : ```shell kubectl describe ts secure-app ``` ```text ... Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning Rejected 2s nginx-ingress-controller TransportServer default/secure-app is invalid and was rejected: spec.action.pass: Not found: "some-app" ``` Note how the events section includes a Warning event with the Rejected reason. **Note**: If you make an existing resource invalid, NGINX Ingress Controller LTS will reject it and remove the corresponding configuration from NGINX. ## Customization via ConfigMap The [ConfigMap](/nic/lts/configuration/global-configuration/configmap-resource.md) keys (except for `stream-snippets`, `stream-log-format`, `resolver-addresses`, `resolver-ipv6`, `resolver-valid` and `resolver-timeout`) do not affect TransportServer resources. -------------------------------------------------------------------------------- title: "Troubleshooting VirtualServer resources" url: https://docs.nginx.com/nginx-ingress-controller/lts/troubleshooting/troubleshoot-virtualserver/index.md -------------------------------------------------------------------------------- This page describes how to troubleshoot VirtualServer and VirtualServer resource events. ## Inspecting VirtualServer and VirtualServerRoute resource events After creating or updating a VirtualServer resource, you can immediately check if the NGINX configuration for that resource was successfully by using `kubectl describe vs `: ```shell kubectl describe vs cafe ``` ```shell Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal AddedOrUpdated 16s nginx-ingress-controller Configuration for default/cafe was added or updated ``` In the above example, we have a `Normal` event with the `AddedOrUpdate` reason, which informs us that the configuration was successfully applied. Checking the events of a VirtualServerRoute is similar: ```shell kubectl describe vsr coffee ``` ```shell Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal AddedOrUpdated 1m nginx-ingress-controller Configuration for default/coffee was added or updated ``` -------------------------------------------------------------------------------- title: "Troubleshooting VirtualServer resources" url: https://docs.nginx.com/nginx-ingress-controller/troubleshooting/troubleshoot-virtualserver/index.md -------------------------------------------------------------------------------- This page describes how to troubleshoot VirtualServer and VirtualServer resource events. ## Inspecting VirtualServer and VirtualServerRoute resource events After creating or updating a VirtualServer resource, you can immediately check if the NGINX configuration for that resource was successfully by using `kubectl describe vs `: ```shell kubectl describe vs cafe ``` ```shell Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal AddedOrUpdated 16s nginx-ingress-controller Configuration for default/cafe was added or updated ``` In the above example, we have a `Normal` event with the `AddedOrUpdate` reason, which informs us that the configuration was successfully applied. Checking the events of a VirtualServerRoute is similar: ```shell kubectl describe vsr coffee ``` ```shell Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal AddedOrUpdated 1m nginx-ingress-controller Configuration for default/coffee was added or updated ``` -------------------------------------------------------------------------------- title: "Build NGINX Ingress Controller" url: https://docs.nginx.com/nginx-ingress-controller/install/build/index.md -------------------------------------------------------------------------------- This document describes how to build an F5 NGINX Ingress Controller image from source code and upload it to a private Docker registry. It also includes information on the Makefile targets and variables. **Note:** If you do not need to build a custom image, see the [pre-built image options](#pre-built-images) at the end of this guide. ## Before you begin To get started, you need the following software installed on your machine: - [Docker v19.03 or higher](https://docs.docker.com/engine/release-notes/19.03/) - [GNU Make](https://www.gnu.org/software/make/) - [git](https://git-scm.com/) - [OpenSSL](https://www.openssl.org/), optionally, if you would like to generate a self-signed certificate and a key for the default server. - For NGINX Plus users, download the certificate (_nginx-repo.crt_) and key (_nginx-repo.key_) from [MyF5](https://my.f5.com). Although NGINX Ingress Controller is written in Golang, you don't need to have Golang installed. You can download the precompiled binary file or build NGINX Ingress Controller in a Docker container. ## Prepare the environment {#prepare-environment} Get your system ready for building and pushing the NGINX Ingress Controller image. 1. Sign in to your private registry. Replace `` with the path to your own private registry. ```shell docker login ``` 2. Clone the NGINX Ingress Controller GitHub repository. Replace `` with the version of NGINX Ingress Controller you want. ```shell git clone https://github.com/nginx/kubernetes-ingress.git --branch cd kubernetes-ingress ``` For instance if you want to clone version v, the commands to run would be: ```shell git clone https://github.com/nginx/kubernetes-ingress.git --branch v cd kubernetes-ingress ``` ## Build the NGINX Ingress Controller image {#build-image} After setting up your environment, follow these steps to build the NGINX Ingress Controller image. **Note:** If you have a local Golang environment and want to build the binary yourself, remove `TARGET=download` from the make commands. If you don't have Golang but still want to build the binary, use `TARGET=container`. ### For NGINX Build the image. Replace `` with your private registry's path. - For a Debian-based image: ```shell make debian-image PREFIX=/nginx-ingress TARGET=download ``` - For an Alpine-based image: ```shell make alpine-image PREFIX=/nginx-ingress TARGET=download ``` **What to expect**: The image is built and tagged with a version number, which is derived from the `VERSION` variable in the [_Makefile_](#makefile-details). This version number is used for tracking and deployment purposes. ### For NGINX Plus Place your NGINX Plus license files (_nginx-repo.crt_ and _nginx-repo.key_) in the project's root folder. To verify they're in place, run: ```shell ls nginx-repo.* ``` You should see: ```text nginx-repo.crt nginx-repo.key ``` Build the image. Replace `` with your private registry's path. ```shell make debian-image-plus PREFIX=/nginx-plus-ingress TARGET=download ``` **What to expect**: The image is built and tagged with a version number, which is derived from the `VERSION` variable in the [_Makefile_](#makefile-details). This version number is used for tracking and deployment purposes. **Note:** If a patch for NGINX Plus is released, make sure to rebuild your image to get the latest version. If your system is caching the Docker layers and not updating the packages, add `DOCKER_BUILD_OPTIONS="--pull --no-cache"` to the make command. ## Push the image to your private registry {#push-image} Once you've successfully built the NGINX or NGINX Plus Ingress Controller image, the next step is to upload it to your private Docker registry. This makes the image available for deployment to your Kubernetes cluster. ### For NGINX Upload the NGINX image. If you're using a custom tag, append `TAG=your-tag` to the command. Replace `` with your private registry's path. ```shell make push PREFIX=/nginx-ingress ``` ### For NGINX Plus Upload the NGINX Plus image. Like with the NGINX image, if you're using a custom tag, add `TAG=your-tag` to the end of the command. Replace `` with your private registry's path. ```shell make push PREFIX=/nginx-plus-ingress ``` ## Makefile details {#makefile-details} This section provides comprehensive information on the targets and variables available in the _Makefile_. These targets and variables allow you to customize how you build, tag, and push your NGINX or NGINX Plus images. ### Key Makefile targets {#key-makefile-targets} **Note:** To view available _Makefile_ targets, run `make` with no target or type `make help`. Key targets include: | Target | Description | |---------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | _build_ | Creates the NGINX Ingress Controller binary with your local Go environment. | | _alpine-image_ | Builds an Alpine-based image with NGINX. | | _alpine-image-plus_ | Builds an Alpine-based image with NGINX Plus. | | _alpine-image-plus-fips_ | Builds an Alpine-based image with NGINX Plus and FIPS. | | _alpine-image-nap-v5-plus-fips_ | Builds an Alpine-based image with NGINX Plus, the [F5 WAF for NGINX v5](/nginx-app-protect/) module and FIPS. | | _debian-image_ | Builds a Debian-based image with NGINX. | | _debian-image-plus_ | Builds a Debian-based image with NGINX Plus. | | _debian-image-nap-plus_ | Builds a Debian-based image with NGINX Plus and the [F5 WAF for NGINX](/nginx-app-protect/) module. | | _debian-image-nap-v5-plus_ | Builds a Debian-based image with NGINX Plus and the [F5 WAF for NGINX v5](/nginx-app-protect/) module. | | _debian-image-dos-plus_ | Builds a Debian-based image with NGINX Plus and the [F5 DoS for NGINX](/nginx-app-protect-dos/) module. | | _debian-image-nap-dos-plus_ | Builds a Debian-based image with NGINX Plus, [F5 WAF for NGINX](/nginx-app-protect/) and [F5 DoS for NGINX](/nginx-app-protect-dos/) modules. | | _ubi-image_ | Builds a UBI-based image with NGINX for [OpenShift](https://www.openshift.com/) clusters. | | _ubi-image-plus_ | Builds a UBI-based image with NGINX Plus for [OpenShift](https://www.openshift.com/) clusters. | | _ubi-image-nap-plus_ | Builds a UBI-based image with NGINX Plus and the [F5 WAF for NGINX](/nginx-app-protect/) module for [OpenShift](https://www.openshift.com/) clusters. | | _ubi-image-nap-v5-plus_ | Builds a UBI-based image with NGINX Plus and the [F5 WAF for NGINX v5](/nginx-app-protect/) module for [OpenShift](https://www.openshift.com/) clusters. | | _ubi-image-dos-plus_ | Builds a UBI-based image with NGINX Plus and the [F5 DoS for NGINX](/nginx-app-protect-dos/) module for [OpenShift](https://www.openshift.com/) clusters. | | _ubi-image-nap-dos-plus_ |

Builds a UBI-based image with NGINX Plus, [F5 WAF for NGINX](/nginx-app-protect/) and the [F5 DoS for NGINX](/nginx-app-protect-dos/) module for [OpenShift](https://www.openshift.com/) clusters.| **Note:** For RHEL, save your organization and activation keys in a file named _rhel_license_ at the project root. Ensure they are on separate lines, such as: - RHEL_ORGANIZATION=1111111 - RHEL_ACTIVATION_KEY=your-key ### Additional useful targets {#other-makefile-targets} A few other useful targets: | Target | Description | |---------------------------------------|---------------| | _push_ | Pushes the built image to the Docker registry. Configures with `PREFIX` and `TAG`. | | _all_ | Runs `test`, `lint`, `verify-codegen`, `update-crds`, and `debian-image`. Stops and reports an error if any of these targets fail. | | _test_ | Runs unit tests. | ### Makefile variables you can customize {#makefile-variables} The _Makefile_ includes several key variables. You have the option to either modify these variables directly in the _Makefile_ or override them when you run the `make` command. | Variable | Description | |-----------------------------------------|---------------| | _ARCH_ | Defines the architecture for the image and binary. The default is `amd64`, but you can also use `arm64`. | | _PREFIX_ | Gives the image its name. The default is `nginx/nginx-ingress`. | | _TAG_ | Adds a tag to the image. This is often the version of NGINX Ingress Controller. | | _DOCKER\_BUILD\_OPTIONS_ | Allows for additional [options](https://docs.docker.com/engine/reference/commandline/build/#options) during the `docker build` process, like `--pull`. | | _TARGET_ |

Determines the build environment. NGINX Ingress Controller compiles locally in a Golang environment by default. Ensure the NGINX Ingress Controller repo resides in your `$GOPATH` if you select this option.

Alternatively, you can set `TARGET=container` to build using a Docker [Golang](https://hub.docker.com/_/golang/) container. To skip compiling the binary if you're on a specific tag or the latest `main` branch commit, set `TARGET=download`.

| ## Alternatives to building your own image {#pre-built-images} If you prefer not to build your own NGINX Ingress Controller image, you can use pre-built images. Here are your options: **NGINX Ingress Controller**: Download the image `nginx/nginx-ingress` from [DockerHub](https://hub.docker.com/r/nginx/nginx-ingress) or [GitHub](https://github.com/nginx/kubernetes-ingress/pkgs/container/kubernetes-ingress). **NGINX Plus Ingress Controller**: You have two options for this: - Download the image using your NGINX Ingress Controller subscription certificate and key. View the [Download NGINX Ingress Controller from the F5 Registry](/nic/install/images/registry-download.md) topic. - Use your NGINX Ingress Controller subscription JWT token to get the image. View the [Add an NGINX Ingress Controller image to your cluster](/nic/install/images/add-image-to-cluster.md) topic. -------------------------------------------------------------------------------- title: "Enable OpenTracing (Removed in v5.0.0)" url: https://docs.nginx.com/nginx-ingress-controller/logging-and-monitoring/opentracing/index.md -------------------------------------------------------------------------------- This topic describes how to OpenTracing with F5 NGINX Ingress Controller. NGINX Ingress Controller supports [OpenTracing](https://opentracing.io/) with the third-party module [opentracing-contrib/nginx-opentracing](https://github.com/opentracing-contrib/nginx-opentracing). **Note:** OpenTracing support has been removed from v5.0.0 of NGINX Ingress Controller. From v5.1.0 onwards, you should follow the guidance in [Configure OpenTelemetry](/nic/logging-and-monitoring/opentelemetry.md). ## Prerequisites 1. Use a NGINX Ingress Controller image that contains OpenTracing. - You can find the images that include OpenTracing listed [in the technical specs doc](/nic/technical-specifications.md#supported-docker-images). - Alternatively, you follow [Build NGINX Ingress Controller](/nic/install/build.md) using `debian-image` (or `alpine-image`) for NGINX or `debian-image-plus` (or `alpine-image-plus`) for NGINX Plus. - [Jaeger](https://github.com/jaegertracing/jaeger-client-cpp), [Zipkin](https://github.com/rnburn/zipkin-cpp-opentracing) and [Datadog](https://github.com/DataDog/dd-opentracing-cpp/) tracers are installed by default. 1. Enable snippets annotations by setting the [`enable-snippets`](/nic/configuration/global-configuration/command-line-arguments#cmdoption-enable-snippets) command-line argument to true. 1. Load the OpenTracing module. You need to load the module with the configuration for the chosen tracer using the following ConfigMap keys: - `opentracing-tracer`: sets the path to the vendor tracer binary plugin. This is the path you used in the COPY line of step *ii* above. - `opentracing-tracer-config`: sets the tracer configuration in JSON format. The following example shows how to use these two keys to load the module with Jaeger tracer: ```yaml opentracing-tracer: "/usr/local/lib/libjaegertracing_plugin.so" opentracing-tracer-config: | { "service_name": "nginx-ingress", "propagation_format": "w3c", "sampler": { "type": "const", "param": 1 }, "reporter": { "localAgentHostPort": "jaeger-agent.default.svc.cluster.local:6831" } } ``` ## Enable OpenTracing globally To enable OpenTracing globally (for all Ingress, VirtualServer and VirtualServerRoute resources), set the `opentracing` ConfigMap key to `True`: ```yaml opentracing: True ``` ## Enable or disable OpenTracing per Ingress resource You can use annotations to enable or disable OpenTracing for a specific Ingress resource. As mentioned in the prerequisites section, both `opentracing-tracer` and `opentracing-tracer-config` must be configured. Consider the following two cases: ### OpenTracing is globally disabled 1. To enable OpenTracing for a specific Ingress resource, use the server snippet annotation: ```yaml nginx.org/server-snippets: | opentracing on; ``` 1. To enable OpenTracing for specific paths: - You need to use [Mergeable Ingress resources](/nic/configuration/ingress-resources/cross-namespace-configuration) - You need to use the location snippets annotation to enable OpenTracing for the paths of a specific Minion Ingress resource: ```yaml nginx.org/location-snippets: | opentracing on; ``` ### OpenTracing is globally enabled 1. To disable OpenTracing for a specific Ingress resource, use the server snippet annotation: ```yaml nginx.org/server-snippets: | opentracing off; ``` 1. To disable OpenTracing for specific paths: - You need to use [Mergeable Ingress resources](/nic/configuration/ingress-resources/cross-namespace-configuration) - You need to use the location snippets annotation to disable OpenTracing for the paths of a specific Minion Ingress resource: ```yaml nginx.org/location-snippets: | opentracing off; ``` ## Customize OpenTracing You can customize OpenTracing through the supported [OpenTracing module directives](https://github.com/opentracing-contrib/nginx-opentracing/blob/master/doc/Reference.md). Use the location-snippets ConfigMap keys or annotations to insert those directives into the generated NGINX configuration. For example, to propagate the active span context for upstream requests, you need to set the `opentracing_propagate_context` directive, which you can add to an Ingress resource using the location snippets annotation: ```yaml nginx.org/location-snippets: | opentracing_propagate_context; ``` **Note:** The `opentracing_propagate_context` and `opentracing_grpc_propagate_context` directives can be used in `http`, `server` or `location` contexts according to the [module documentation](https://github.com/opentracing-contrib/nginx-opentracing/blob/master/doc/Reference.md#opentracing_propagate_context). However, because of the way the module works and how NGINX Ingress Controller generates the NGINX configuration, it is only possible to use the directive in the `location` context. -------------------------------------------------------------------------------- title: "Manage Policy resources with kubectl" url: https://docs.nginx.com/nginx-ingress-controller/configuration/policy-resource/using-policy/index.md -------------------------------------------------------------------------------- Use standard `kubectl` commands to work with Policy resources, just as you would with built-in Kubernetes resources. For example, the following command creates a Policy resource defined in `access-control-policy-allow.yaml` with the name `webapp-policy`: ```shell kubectl apply -f access-control-policy-allow.yaml policy.k8s.nginx.org/webapp-policy configured ``` Get the resource by running: ```shell kubectl get policy webapp-policy NAME AGE webapp-policy 27m ``` For `kubectl get` and similar commands, you can also use the short name `pol` instead of `policy`. ## Attach policies to a resource You can apply policies to VirtualServer, VirtualServerRoute, and Ingress resources. For example: - VirtualServer: ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServer metadata: name: cafe namespace: cafe spec: host: cafe.example.com tls: secret: cafe-secret policies: # spec policies - name: policy1 upstreams: - name: coffee service: coffee-svc port: 80 routes: - path: /tea policies: # route policies - name: policy2 namespace: cafe route: tea/tea - path: /coffee policies: # route policies - name: policy3 namespace: cafe action: pass: coffee ``` For VirtualServer, you can apply a policy: * to all routes (spec policies) * to a specific route (route policies) Route policies of the same type override spec policies. In the example above, if `policy-1` and `policy-3` are both `accessControl` policies, NGINX applies `policy-3` to requests for `cafe.example.com/coffee`. NGINX enforces this override: the spec policies apply in the `server` context of the configuration, and the route policies apply in the `location` context. As a result, the route policies of the same type take precedence. - VirtualServerRoute, referenced by the VirtualServer above: ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServerRoute metadata: name: tea namespace: tea spec: host: cafe.example.com upstreams: - name: tea service: tea-svc port: 80 subroutes: # subroute policies - path: /tea policies: - name: policy4 namespace: tea action: pass: tea ``` For VirtualServerRoute, you can apply a policy to a subroute (subroute policies). Subroute policies of the same type override spec policies. In the example above, if `policy-1` (in the VirtualServer) and `policy-4` are both `accessControl` policies, NGINX applies `policy-4` to requests for `cafe.example.com/tea`. As with the VirtualServer, NGINX enforces this override. Subroute policies always override route policies, regardless of type. For example, NGINX Ingress Controller ignores `policy-2` from the VirtualServer route for the `/tea` subroute, because the subroute has its own policies, `policy4` in this case. If the subroute had no policies, NGINX Ingress Controller would apply `policy-2` instead. NGINX Ingress Controller enforces this override: the `location` context for the subroute has either route policies or subroute policies, but never both. - Ingress: ```yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: cafe-ingress annotations: nginx.org/policies: "webapp-policy" spec: ingressClassName: nginx tls: - hosts: - cafe.example.com secretName: tls-secret rules: - host: cafe.example.com http: paths: - path: /tea pathType: Prefix backend: service: name: tea-svc port: number: 80 - path: /coffee pathType: Prefix backend: service: name: coffee-svc port: number: 80 ``` For Ingress, you can apply policies: * to a single Ingress * to a master Ingress, where minion Ingresses inherit the policies * to minion Ingresses, where minion policies override master policies ## Invalid policies NGINX treats a policy as invalid if any of the following conditions is true: - The policy doesn't pass [comprehensive validation](#comprehensive-validation). - The policy isn't present in the cluster. - The policy doesn't meet its type-specific requirements. For example, an `ingressMTLS` policy requires TLS termination turned on in the VirtualServer. For an invalid policy, NGINX returns the 500 status code for client requests, following these rules: - If a policy is referenced in a VirtualServer `route` or a VirtualServerRoute `subroute`, NGINX returns the 500 status code for requests to the URIs of that route or subroute. - If a policy is referenced in the VirtualServer `spec`, NGINX returns the 500 status code for requests to all URIs of that VirtualServer. If a policy is invalid, the VirtualServer or VirtualServerRoute gets the [status](/nic/configuration/global-configuration/reporting-resources-status.md#virtualserver-and-virtualserverroute-resources) state `Warning`, with a message that explains why the policy is invalid. ## Validation Two types of validation are available for the Policy resource: - *Structural validation*, done by `kubectl` and the Kubernetes API server. - *Comprehensive validation*, done by NGINX Ingress Controller. ### Structural validation The custom resource definition for the Policy includes a structural OpenAPI schema, which describes the type of every field of the resource. If you try to create or update a resource that violates the structural schema, for example, if the resource uses a string value instead of an array of strings in the `allow` field, `kubectl` and the Kubernetes API server reject the resource. - Example of `kubectl` validation: ```shell kubectl apply -f access-control-policy-allow.yaml error: error validating "access-control-policy-allow.yaml": error validating data: ValidationError(Policy.spec.accessControl.allow): invalid type for org.nginx.k8s.v1.Policy.spec.accessControl.allow: got "string", expected "array"; if you choose to ignore these errors, turn validation off with --validate=false ``` - Example of Kubernetes API server validation: ```shell kubectl apply -f access-control-policy-allow.yaml --validate=false The Policy "webapp-policy" is invalid: spec.accessControl.allow: Invalid value: "string": spec.accessControl.allow in body must be of type array: "string" ``` If a resource passes structural validation, NGINX Ingress Controller's comprehensive validation runs next. ### Comprehensive validation NGINX Ingress Controller validates the fields of a Policy resource. If a resource is invalid, NGINX Ingress Controller rejects it. The resource continues to exist in the cluster, but NGINX Ingress Controller ignores it. Use `kubectl` to check whether NGINX Ingress Controller successfully applied a Policy configuration. For the example `webapp-policy` Policy, run: ```shell kubectl describe pol webapp-policy . . . Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal AddedOrUpdated 11s nginx-ingress-controller Policy default/webapp-policy was added or updated ``` The events section includes a Normal event with the AddedOrUpdated reason, which tells you the configuration applied successfully. If you create an invalid resource, NGINX Ingress Controller rejects it and emits a Rejected event. For example, if you create a Policy `webapp-policy` with an invalid IP `10.0.0.` in the `allow` field, you get: ```shell kubectl describe policy webapp-policy . . . Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning Rejected 7s nginx-ingress-controller Policy default/webapp-policy is invalid and was rejected: spec.accessControl.allow[0]: Invalid value: "10.0.0.": must be a CIDR or IP ``` The events section includes a Warning event with the Rejected reason. This information is also available in the `status` field of the Policy resource. Note the Status section of the Policy: ```shell kubectl describe pol webapp-policy . . . Status: Message: Policy default/webapp-policy is invalid and was rejected: spec.accessControl.allow[0]: Invalid value: "10.0.0.": must be a CIDR or IP Reason: Rejected State: Invalid ``` **Note:** If you make an existing resource invalid, NGINX Ingress Controller rejects it. -------------------------------------------------------------------------------- title: "VirtualServer and VirtualServerRoute resources" url: https://docs.nginx.com/nginx-ingress-controller/configuration/virtualserver-and-virtualserverroute-resources/index.md -------------------------------------------------------------------------------- This document is reference material for the VirtualServer and VirtualServerRoute resources used by F5 NGINX Ingress Controller. VirtualServer and VirtualServerRoute resources are load balancing configurations recommended as an alternative to the Ingress resource. They enable use cases not supported with the Ingress resource, such as traffic splitting and advanced content-based routing. The resources are implemented as [Custom Resources](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/). The GitHub repository has [examples of the resources](https://github.com/nginx/kubernetes-ingress/tree/v/examples/custom-resources) for specific use cases. --- ## VirtualServer specification The VirtualServer resource defines load balancing configuration for a domain name, such as `example.com`. The example below shows a VirtualServer with both traditional static routes and a dynamic route using `routeSelector`. The route with path `/dynamic` will automatically include any VirtualServerRoute resources that have the label `app: cafe`: ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServer metadata: name: cafe spec: host: cafe.example.com listener: http: http-8083 https: https-8443 tls: secret: cafe-secret gunzip: on upstreams: - name: tea service: tea-svc port: 80 - name: coffee service: coffee-svc port: 80 routes: - path: /tea action: pass: tea - path: /coffee action: pass: coffee - path: ~ ^/decaf/.*\\.jpg$ action: pass: coffee - path: =/green/tea action: pass: tea - path: /dynamic routeSelector: matchLabels: app: cafe ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``host`` | The host (domain name) of the server. Must be a valid subdomain as defined in RFC 1123, such as ``my-app`` or ``hello.example.com``. When using a wildcard domain like ``*.example.com`` the domain must be contained in double quotes. The ``host`` value needs to be unique among all Ingress and VirtualServer resources. See also [Handling Host and Listener Collisions](/nic/configuration/host-and-listener-collisions.md). | ``string`` | Yes | |``listener`` | Sets a custom HTTP and/or HTTPS listener. Valid fields are `listener.http` and `listener.https`. Each field must reference the name of a valid listener defined in a GlobalConfiguration resource | [listener](#virtualserverlistener) | No | |``tls`` | The TLS termination configuration. | [tls](#virtualservertls) | No | |``gunzip`` | Enables or disables [decompression](/nginx/admin-guide/web-server/compression.md) of gzipped responses for clients. Allowed values “on”/“off”, “true”/“false” or “yes”/“no”. If the ``gunzip`` value is not set, it defaults to ``off``. | ``boolean`` | No | |``externalDNS`` | The externalDNS configuration for a VirtualServer. | [externalDNS](#virtualserverexternaldns) | No | |``dos`` | A reference to a DosProtectedResource, setting this enables DOS protection of the VirtualServer. | ``string`` | No | |``policies`` | A list of policies. | [[]policy](#virtualserverpolicy) | No | |``upstreams`` | A list of upstreams. | [[]upstream](#upstream) | No | |``routes`` | A list of routes. | [[]route](#virtualserverroute) | No | |``ingressClassName`` | Specifies which Ingress Controller must handle the VirtualServer resource. | ``string`` | No | |``http-snippets`` | Sets a custom snippet in the http context. | ``string`` | No | |``server-snippets`` | Sets a custom snippet in server context. Overrides the ``server-snippets`` ConfigMap key. | ``string`` | No | |``add-header-inherit`` | Controls header inheritance behavior at the server level. Allowed values are ``on``, ``off``, and ``merge``. | ``string`` | No | ### VirtualServer.TLS The tls field defines TLS configuration for a VirtualServer. For example: ```yaml secret: cafe-secret redirect: enable: true ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``secret`` | The name of a secret with a TLS certificate and key. The secret must belong to the same namespace as the VirtualServer. The secret must be of the type ``kubernetes.io/tls`` and contain keys named ``tls.crt`` and ``tls.key`` that contain the certificate and private key as described [here](https://kubernetes.io/docs/concepts/services-networking/ingress/#tls). If the secret doesn't exist or is invalid, NGINX will break any attempt to establish a TLS connection to the host of the VirtualServer. If the secret is not specified but [wildcard TLS secret](/nic/configuration/global-configuration/command-line-arguments.md#cmdoption-wildcard-tls-secret) is configured, NGINX will use the wildcard secret for TLS termination. | ``string`` | No | |``redirect`` | The redirect configuration of the TLS for a VirtualServer. | [tls.redirect](#virtualservertlsredirect) | No | ### VirtualServer.TLS.Redirect | |``cert-manager`` | The cert-manager configuration of the TLS for a VirtualServer. | [tls.cert-manager](#virtualservertlscertmanager) | No | ### VirtualServer.TLS.CertManager | ### VirtualServer.TLS.Redirect The redirect field configures a TLS redirect for a VirtualServer: ```yaml enable: true code: 301 basedOn: scheme ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``enable`` | Enables a TLS redirect for a VirtualServer. The default is ``False``. | ``boolean`` | No | |``code`` | The status code of a redirect. The allowed values are: ``301`` , ``302`` , ``307`` , ``308``. The default is ``301``. | ``int`` | No | |``basedOn`` | The attribute of a request that NGINX will evaluate to send a redirect. The allowed values are ``scheme`` (the scheme of the request) or ``x-forwarded-proto`` (the ``X-Forwarded-Proto`` header of the request). The default is ``scheme``. | ``string`` | No | ### VirtualServer.Policy | ### VirtualServer.TLS.CertManager The cert-manager field configures x509 automated Certificate management for VirtualServer resources using cert-manager (cert-manager.io). Please see the [cert-manager configuration documentation](https://cert-manager.io/docs/configuration/) for more information on deploying and configuring Issuers. Example: ```yaml cert-manager: cluster-issuer: "my-issuer-name" ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``issuer`` | the name of an Issuer. An Issuer is a cert-manager resource which describes the certificate authority capable of signing certificates. The Issuer must be in the same namespace as the VirtualServer resource. Please note that one of `issuer` and `cluster-issuer` are required, but they are mutually exclusive - one and only one must be defined. | ``string`` | No | |``cluster-issuer`` | the name of a ClusterIssuer. A ClusterIssuer is a cert-manager resource which describes the certificate authority capable of signing certificates. It does not matter which namespace your VirtualServer resides, as ClusterIssuers are non-namespaced resources. Please note that one of `issuer` and `cluster-issuer` are required, but they are mutually exclusive - one and only one must be defined. | ``string`` | No | |``issuer-kind`` | The kind of the external issuer resource, for example AWSPCAIssuer. This is only necessary for out-of-tree issuers. This cannot be defined if `cluster-issuer` is also defined. | ``string`` | No | |``issuer-group`` | The API group of the external issuer controller, for example awspca.cert-manager.io. This is only necessary for out-of-tree issuers. This cannot be defined if `cluster-issuer` is also defined. | ``string`` | No | |``common-name`` | This field allows you to configure spec.commonName for the Certificate to be generated. This configuration adds a CN to the x509 certificate. | ``string`` | No | |``duration`` | This field allows you to configure spec.duration field for the Certificate to be generated. Must be specified using a [Go time.Duration](https://pkg.go.dev/time#ParseDuration) string format, which does not allow the d (days) suffix. You must specify these values using s, m, and h suffixes instead. | ``string`` | No | |``renew-before`` | this annotation allows you to configure spec.renewBefore field for the Certificate to be generated. Must be specified using a [Go time.Duration](https://pkg.go.dev/time#ParseDuration) string format, which does not allow the d (days) suffix. You must specify these values using s, m, and h suffixes instead. | ``string`` | No | |``usages`` | This field allows you to configure spec.usages field for the Certificate to be generated. Pass a string with comma-separated values i.e. ``key agreement,digital signature, server auth``. An exhaustive list of supported key usages can be found in the [the cert-manager api documentation](https://cert-manager.io/docs/reference/api-docs/#cert-manager.io/v1.KeyUsage). | ``string`` | No | |``issue-temp-cert`` | When ``true``, ask cert-manager for a [temporary self-signed certificate](https://cert-manager.io/docs/usage/certificate/#temporary-certificates-while-issuing) pending the issuance of the Certificate. This allows HTTPS-only servers to use ACME HTTP01 challenges when the TLS secret does not exist yet. | ``boolean`` | No | ### VirtualServer.Listener The listener field defines a custom HTTP and/or HTTPS listener. The respective listeners used must reference the name of a listener defined using a [GlobalConfiguration](/nic/configuration/global-configuration/globalconfiguration-resource.md) resource. For example: ```yaml http: http-8083 https: https-8443 ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``http`` | The name of am HTTP listener defined in a [GlobalConfiguration](/nic/configuration/global-configuration/globalconfiguration-resource.md) resource. | ``string`` | No | |``https`` | The name of an HTTPS listener defined in a [GlobalConfiguration](/nic/configuration/global-configuration/globalconfiguration-resource.md) resource. | ``string`` | No | ### VirtualServer.ExternalDNS The externalDNS field configures controlling DNS records dynamically for VirtualServer resources using [ExternalDNS](https://github.com/kubernetes-sigs/external-dns). Please see the [ExternalDNS configuration documentation](https://kubernetes-sigs.github.io/external-dns/) for more information on deploying and configuring ExternalDNS and Providers. Example: ```yaml enable: true ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``enable`` | Enables ExternalDNS integration for a VirtualServer resource. The default is ``false``. | ``string`` | No | |``labels`` | Configure labels to be applied to the Endpoint resources that will be consumed by ExternalDNS. | ``map[string]string`` | No | |``providerSpecific`` | Configure provider specific properties which holds the name and value of a configuration which is specific to individual DNS providers. | [[]ProviderSpecific](#virtualserverexternaldnsproviderspecific) | No | |``recordTTL`` | TTL for the DNS record. This defaults to 0 if not defined. See [the ExternalDNS TTL documentation for provider-specific defaults](https://kubernetes-sigs.github.io/external-dns/v0.14.2/ttl/#providers) | ``int64`` | No | |``recordType`` | The record Type that should be created, e.g. "A", "AAAA", "CNAME". This is automatically computed based on the external endpoints if not defined. | ``string`` | No | ### VirtualServer.ExternalDNS.ProviderSpecific The providerSpecific field of the externalDNS block allows the specification of provider specific properties which is a list of key value pairs of configurations which are specific to individual DNS providers. Example: ```yaml - name: my-name value: my-value - name: my-name2 value: my-value2 ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``name`` | The name of the key value pair. | ``string`` | Yes | |``value`` | The value of the key value pair. | ``string`` | Yes | ### VirtualServer.Policy The policy field references a [Policy resource](/nic/configuration/policy-resource/before-you-begin.md) by its name and optional namespace. For example: ```yaml name: access-control ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``name`` | The name of a policy. If the policy doesn't exist or invalid, NGINX will respond with an error response with the `500` status code. | ``string`` | Yes | |``namespace`` | The namespace of a policy. If not specified, the namespace of the VirtualServer resource is used. | ``string`` | No | ### VirtualServer.Route The route defines rules for matching client requests to actions like passing a request to an upstream. For example: ```yaml path: /tea action: pass: tea ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``path`` | The path of the route. NGINX will match it against the URI of a request. Possible values are: a prefix ( ``/`` , ``/path`` ), a longest prefix match ( ``^~/images/`` ) that prevents regex evaluation, an exact match ( ``=/exact/match`` ), a case insensitive regular expression ( ``~*^/Bar.*\.jpg`` ) or a case sensitive regular expression ( ``~^/foo.*\.jpg`` ). In the case of a prefix (must start with ``/`` ), a longest prefix match (must start with ``^~`` ) or an exact match (must start with ``=`` ), the path must not include any whitespace characters, ``{`` , ``}`` or ``;``. In the case of the regex matches, all double quotes ``"`` must be escaped and the match can't end in an unescaped backslash ``\``. The path must be unique among the paths of all routes of the VirtualServer. See [Path matching](/nic/configuration/path-matching.md) for details on matching types and priority. Check the [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) directive for more information. | ``string`` | Yes | |``policies`` | A list of policies. The policies override the policies of the same type defined in the ``spec`` of the VirtualServer. See [Applying policies](/nic/configuration/policy-resource/applying-policies.md) for more details. | [[]policy](#virtualserverpolicy) | No | |``action`` | The default action to perform for a request. | [action](#action) | No | |``dos`` | A reference to a DosProtectedResource, setting this enables DOS protection of the VirtualServer route. | ``string`` | No | |``splits`` | The default splits configuration for traffic splitting. Must include at least 2 splits. | [[]split](#split) | No | |``matches`` | The matching rules for advanced content-based routing. Requires the default ``action`` or ``splits``. Unmatched requests will be handled by the default ``action`` or ``splits``. | [matches](#match) | No | |``route`` | The name of a VirtualServerRoute resource that defines this route. If the VirtualServerRoute belongs to a different namespace than the VirtualServer, you need to include the namespace. For example, ``tea-namespace/tea``. | ``string`` | No | |``routeSelector`` | The RouteSelector allows selecting VirtualServerRoute resources using label selectors instead of specifying them by name and namespace. | [routeSelector](#virtualserverrouterouteselector) | No | |``errorPages`` | The custom responses for error codes. NGINX will use those responses instead of returning the error responses from the upstream servers or the default responses generated by NGINX. A custom response can be a redirect or a canned response. For example, a redirect to another URL if an upstream server responded with a 404 status code. | [[]errorPage](#errorpage) | No | |``location-snippets`` | Sets a custom snippet in the location context. Overrides the ``location-snippets`` ConfigMap key. | ``string`` | No | |``add-header-inherit`` | Controls header inheritance behavior at the location level. Allowed values are ``on``, ``off``, and ``merge``. | ``string`` | No | \* -- a route must include exactly one of the following: `action`, `splits`, `route` or `routeSelector` ### VirtualServer.Route.RouteSelector The RouteSelector field allows you to dynamically select VirtualServerRoute resources using label selectors instead of specifying them by name and namespace. This provides a more flexible and scalable way to associate VirtualServerRoutes with a VirtualServer route. See the [VirtualServerRoute specification](#virtualserverroute-specification) section below for examples of how to use RouteSelector with VirtualServerRoute resources. |Field | Description | Type | Required | | ---| ---| ---| --- | |``matchLabels`` | A map of key-value pairs. Each key-value pair in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed. | ``map[string]string`` | Yes | |``matchExpressions`` | A list of label selector requirements. The requirements are ANDed. For more information on label selector requirements, see the [Kubernetes documentation](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#resources-that-support-set-based-requirements). | N/A | No | ## VirtualServerRoute specification The VirtualServerRoute resource defines a route for a VirtualServer. It can consist of one or multiple subroutes. The VirtualServerRoute is an alternative to [Mergeable Ingress types](/nic/configuration/ingress-resources/cross-namespace-configuration.md). VirtualServer routes can reference VirtualServerRoute resources in two ways: by name using the `route` field, or dynamically using the `routeSelector` field with label selectors. The `routeSelector` approach allows you to add new VirtualServerRoute resources without modifying the VirtualServer configuration. #### Standard In this example, the VirtualServer `cafe` from the namespace `cafe-ns` defines a route that references a specific VirtualServerRoute by name. VirtualServer: ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServer metadata: name: cafe namespace: cafe-ns spec: host: cafe.example.com upstreams: - name: tea service: tea-svc port: 80 routes: - path: /tea action: pass: tea - path: /coffee route: coffee-ns/coffee ``` VirtualServerRoute: ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServerRoute metadata: name: coffee namespace: coffee-ns spec: host: cafe.example.com upstreams: - name: latte service: latte-svc port: 80 - name: espresso service: espresso-svc port: 80 subroutes: - path: /coffee/latte action: pass: latte - path: /coffee/espresso action: pass: espresso ``` #### RouteSelector In this example, the VirtualServer `cafe` from the namespace `cafe-ns` uses `routeSelector` to dynamically select any VirtualServerRoute with the label `app: coffee`. VirtualServer: ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServer metadata: name: cafe namespace: cafe-ns spec: host: cafe.example.com upstreams: - name: tea service: tea-svc port: 80 routes: - path: /tea action: pass: tea - path: /decaf routeSelector: matchLabels: app: coffee ``` VirtualServerRoute: ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServerRoute metadata: name: decaf namespace: coffee-ns labels: app: coffee spec: host: cafe.example.com upstreams: - name: latte service: latte-svc port: 80 - name: espresso service: espresso-svc port: 80 subroutes: - path: /decaf/latte action: pass: latte - path: /decaf/espresso action: pass: espresso ``` Note that each subroute must have a `path` that starts with the same prefix (here `/coffee`), which is defined in the route of the VirtualServer. Additionally, the `host` in the VirtualServerRoute must be the same as the `host` of the VirtualServer. |Field | Description | Type | Required | | ---| ---| ---| --- | |``host`` | The host (domain name) of the server. Must be a valid subdomain as defined in RFC 1123, such as ``my-app`` or ``hello.example.com``. When using a wildcard domain like ``*.example.com`` the domain must be contained in double quotes. Must be the same as the ``host`` of the VirtualServer that references this resource. | ``string`` | Yes | |``upstreams`` | A list of upstreams. | [[]upstream](#upstream) | No | |``subroutes`` | A list of subroutes. | [[]subroute](#virtualserverroutesubroute) | No | |``ingressClassName`` | Specifies which Ingress Controller must handle the VirtualServerRoute resource. Must be the same as the ``ingressClassName`` of the VirtualServer that references this resource. | ``string``_ | No | ### VirtualServerRoute.Subroute The subroute defines rules for matching client requests to actions like passing a request to an upstream. For example: ```yaml path: /coffee action: pass: coffee ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``path`` | The path of the subroute. NGINX will match it against the URI of a request. Possible values are: a prefix ( ``/`` , ``/path`` ), a longest prefix match ( ``^~/images/`` ) that prevents regex evaluation, an exact match ( ``=/exact/match`` ), a case insensitive regular expression ( ``~*^/Bar.*\.jpg`` ) or a case sensitive regular expression ( ``~^/foo.*\.jpg`` ). In the case of a prefix, the path must start with the same path as the path of the route of the VirtualServer that references this resource. In the case of a longest prefix match, the path must start with the same path as the path of the route of the VirtualServer that references this resource, including the ``^~`` modifier. In the case of an exact or regex match, the path must be the same as the path of the route of the VirtualServer that references this resource. In the case of regex matches, multiple VirtualServer regex routes (``~`` or ``~*``) can reference the same VirtualServerRoute; the subroutes must form an exact set match with those VirtualServer paths. See [Multiple regex routes in a VirtualServerRoute](#multiple-regex-routes-in-a-virtualserverroute) for details. A matching path of the route of the VirtualServer but in different type is not accepted, e.g. a regex path (`~/match`) cannot be used with a prefix path in VirtualServer (`/match`). In the case of a prefix, longest prefix match or an exact match, the path must not include any whitespace characters, ``{`` , ``}`` or ``;``. In the case of the regex matches, all double quotes ``"`` must be escaped and the match can't end in an unescaped backslash ``\``. Whitespace between the regex modifier and the path is stripped before comparison, so ``~/api`` and ``~ /api`` are treated as the same path. The path must be unique among the paths of all subroutes of the VirtualServerRoute after normalization. See [Path matching](/nic/configuration/path-matching.md) for details on matching types, priority, and subroute constraints. | ``string`` | Yes | |``policies`` | A list of policies. The policies override *all* policies defined in the route of the VirtualServer that references this resource. The policies also override the policies of the same type defined in the ``spec`` of the VirtualServer. See [Applying policies](/nic/configuration/policy-resource/applying-policies.md) for more details. | [[]policy](#virtualserverpolicy) | No | |``action`` | The default action to perform for a request. | [action](#action) | No | |``dos`` | A reference to a DosProtectedResource, setting this enables DOS protection of the VirtualServerRoute subroute. | ``string`` | No | |``splits`` | The default splits configuration for traffic splitting. Must include at least 2 splits. | [[]split](#split) | No | |``matches`` | The matching rules for advanced content-based routing. Requires the default ``action`` or ``splits``. Unmatched requests will be handled by the default ``action`` or ``splits``. | [matches](#match) | No | |``errorPages`` | The custom responses for error codes. NGINX will use those responses instead of returning the error responses from the upstream servers or the default responses generated by NGINX. A custom response can be a redirect or a canned response. For example, a redirect to another URL if an upstream server responded with a 404 status code. | [[]errorPage](#errorpage) | No | |``location-snippets`` | Sets a custom snippet in the location context. Overrides the ``location-snippets`` of the VirtualServer (if set) or the ``location-snippets`` ConfigMap key. | ``string`` | No | |``add-header-inherit`` | Controls header inheritance behavior at the location level. Allowed values are ``on``, ``off``, and ``merge``. | ``string`` | No | \* -- a subroute must include exactly one of the following: `action` or `splits`. ## Common VirtualServer and VirtualServerRoute specifications ### Upstream The upstream defines a destination for the routing configuration. For example: ```yaml name: tea service: tea-svc subselector: version: canary port: 80 lb-method: round_robin fail-timeout: 10s max-fails: 1 max-conns: 32 keepalive: 32 connect-timeout: 30s read-timeout: 30s send-timeout: 30s next-upstream: "error timeout non_idempotent" next-upstream-timeout: 5s next-upstream-tries: 10 client-max-body-size: 2m tls: enable: true ``` **Note**: The WebSocket protocol is supported without any additional configuration. |Field | Description | Type | Required | | ---| ---| ---| --- | |``name`` | The name of the upstream. Must be a valid DNS label as defined in RFC 1035. For example, ``hello`` and ``upstream-123`` are valid. The name must be unique among all upstreams of the resource. | ``string`` | Yes | |``service`` | The name of a [service](https://kubernetes.io/docs/concepts/services-networking/service/). If the Service belongs to a different namespace than the VirtualServer or VirtualServerRoute, you need to include the namespace. For example, ``tea-namespace/tea``. If the service doesn't exist, NGINX will assume the service has zero endpoints and return a ``502`` response for requests for this upstream. For NGINX Plus only, services of type [ExternalName](https://kubernetes.io/docs/concepts/services-networking/service/#externalname) are also supported in the same namespace (check the [prerequisites](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/externalname-services#prerequisites) ). | ``string`` | Yes | |``subselector`` | Selects the pods within the service using label keys and values. By default, all pods of the service are selected. Note: the specified labels are expected to be present in the pods when they are created. If the pod labels are updated, NGINX Ingress Controller will not see that change until the number of the pods is changed. | ``map[string]string`` | No | |``use-cluster-ip`` | Enables using the Cluster IP and port of the service instead of the default behavior of using the IP and port of the pods. When this field is enabled, the fields that configure NGINX behavior related to multiple upstream servers (like ``lb-method`` and ``next-upstream``) will have no effect, as NGINX Ingress Controller will configure NGINX with only one upstream server that will match the service Cluster IP. | ``boolean`` | No | |``port`` | The port of the service. If the service doesn't define that port, NGINX will assume the service has zero endpoints and return a ``502`` response for requests for this upstream. The port must fall into the range ``1..65535``. | ``uint16`` | Yes | |``lb-method`` | The load [balancing method](/nginx/admin-guide/load-balancer/http-load-balancer.md#choosing-a-load-balancing-method). To use the round-robin method, specify ``round_robin``. The default is specified in the ``lb-method`` ConfigMap key. | ``string`` | No | |``fail-timeout`` | The time during which the specified number of unsuccessful attempts to communicate with an upstream server should happen to consider the server unavailable. See the [fail_timeout](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#fail_timeout) parameter of the server directive. The default is set in the ``fail-timeout`` ConfigMap key. | ``string`` | No | |``max-fails`` | The number of unsuccessful attempts to communicate with an upstream server that should happen in the duration set by the ``fail-timeout`` to consider the server unavailable. See the [max_fails](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#max_fails) parameter of the server directive. The default is set in the ``max-fails`` ConfigMap key. | ``int`` | No | |``max-conns`` | The maximum number of simultaneous active connections to an upstream server. See the [max_conns](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#max_conns) parameter of the server directive. By default there is no limit. Note: if keepalive connections are enabled, the total number of active and idle keepalive connections to an upstream server may exceed the ``max_conns`` value. | ``int`` | No | |``keepalive`` | Configures the cache for connections to upstream servers. The value ``0`` disables the cache. See the [keepalive](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive) directive. The default is set in the ``keepalive`` ConfigMap key. | ``int`` | No | |``connect-timeout`` | The timeout for establishing a connection with an upstream server. See the [proxy_connect_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_connect_timeout) directive. The default is specified in the ``proxy-connect-timeout`` ConfigMap key. | ``string`` | No | |``read-timeout`` | The timeout for reading a response from an upstream server. See the [proxy_read_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_read_timeout) directive. The default is specified in the ``proxy-read-timeout`` ConfigMap key. | ``string`` | No | |``send-timeout`` | The timeout for transmitting a request to an upstream server. See the [proxy_send_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_send_timeout) directive. The default is specified in the ``proxy-send-timeout`` ConfigMap key. | ``string`` | No | |``next-upstream`` | Specifies in which cases a request should be passed to the next upstream server. See the [proxy_next_upstream](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream) directive. The default is ``error timeout``. | ``string`` | No | |``next-upstream-timeout`` | The time during which a request can be passed to the next upstream server. See the [proxy_next_upstream_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream_timeout) directive. The ``0`` value turns off the time limit. The default is ``0``. | ``string`` | No | |``next-upstream-tries`` | The number of possible tries for passing a request to the next upstream server. See the [proxy_next_upstream_tries](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream_tries) directive. The ``0`` value turns off this limit. The default is ``0``. | ``int`` | No | |``client-max-body-size`` | Sets the maximum allowed size of the client request body. See the [client_max_body_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size) directive. The default is set in the ``client-max-body-size`` ConfigMap key. | ``string`` | No | |``client-body-buffer-size`` | Sets the value of the [client_body_buffer_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_buffer_size) directive. The default is set in the ``client-body-buffer-size`` ConfigMap key. | ``string`` | No | |``tls`` | The TLS configuration for the Upstream. | [tls](#upstreamtls) | No | |``healthCheck`` | The health check configuration for the Upstream. See the [health_check](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#health_check) directive. Note: this feature is supported only in NGINX Plus. | [healthcheck](#upstreamhealthcheck) | No | |``slow-start`` | The slow start allows an upstream server to gradually recover its weight from 0 to its nominal value after it has been recovered or became available or when the server becomes available after a period of time it was considered unavailable. By default, the slow start is disabled. See the [slow_start](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#slow_start) parameter of the server directive. Note: The parameter cannot be used along with the ``random`` , ``hash`` or ``ip_hash`` load balancing methods and will be ignored. | ``string`` | No | |``queue`` | Configures a queue for an upstream. A client request will be placed into the queue if an upstream server cannot be selected immediately while processing the request. By default, no queue is configured. Note: this feature is supported only in NGINX Plus. | [queue](#upstreamqueue) | No | |``buffering`` | Enables buffering of responses from the upstream server. See the [proxy_buffering](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffering) directive. The default is set in the ``proxy-buffering`` ConfigMap key. | ``boolean`` | No | |``buffers`` | Configures the buffers used for reading a response from the upstream server for a single connection. | [buffers](#upstreambuffers) | No | |``buffer-size`` | Sets the size of the buffer used for reading the first part of a response received from the upstream server. See the [proxy_buffer_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffer_size) directive. The default is set in the ``proxy-buffer-size`` ConfigMap key. | ``string`` | No | |``busy-buffers-size`` | Sets the size of the buffer used for reading a response from the upstream server when the response is larger than the ``buffer-size``. See the [proxy_busy_buffers_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_busy_buffers_size) directive. The default is set in the ``proxy-busy-buffers-size`` ConfigMap key. | ``string`` | No | |``ntlm`` | Allows proxying requests with NTLM Authentication. See the [ntlm](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#ntlm) directive. In order for NTLM authentication to work, it is necessary to enable keepalive connections to upstream servers using the ``keepalive`` field. Note: this feature is supported only in NGINX Plus.| ``boolean`` | No | |``type`` |The type of the upstream. Supported values are ``http`` and ``grpc``. The default is ``http``. For gRPC, it is necessary to enable HTTP/2 in the [ConfigMap](/nic/configuration/global-configuration/configmap-resource.md#listeners) and configure TLS termination in the VirtualServer. | ``string`` | No | |``backup`` | The name of the backup service of type [ExternalName](https://kubernetes.io/docs/concepts/services-networking/service/#externalname). This will be used when the primary servers are unavailable. Note: The parameter cannot be used along with the ``random`` , ``hash`` or ``ip_hash`` load balancing methods. | ``string`` | No | |``backupPort`` | The port of the backup service. The backup port is required if the backup service name is provided. The port must fall into the range ``1..65535``. | ``uint16`` | No | ### Upstream.Buffers The buffers field configures the buffers used for reading a response from the upstream server for a single connection: ```yaml number: 4 size: 8K ``` See the [proxy_buffers](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffers) directive for additional information. |Field | Description | Type | Required | | ---| ---| ---| --- | |``number`` | Configures the number of buffers. The default is set in the ``proxy-buffers`` ConfigMap key. | ``int`` | Yes | |``size`` | Configures the size of a buffer. The default is set in the ``proxy-buffers`` ConfigMap key. | ``string`` | Yes | ### Upstream.TLS |Field | Description | Type | Required | | ---| ---| ---| --- | |``enable`` | Enables HTTPS for requests to upstream servers. The default is ``False`` , meaning that HTTP will be used. Note: by default, NGINX will not verify the upstream server certificate. To enable the verification, configure an [EgressMTLS Policy](/nic/configuration/policy-resource/policy-reference.md#egressmtls). | ``boolean`` | No | ### Upstream.Queue The queue field configures a queue. A client request will be placed into the queue if an upstream server cannot be selected immediately while processing the request: ```yaml size: 10 timeout: 60s ``` See [`queue`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#queue) directive for additional information. Note: This feature is supported only in NGINX Plus. |Field | Description | Type | Required | | ---| ---| ---| --- | |``size`` | The size of the queue. | ``int`` | Yes | |``timeout`` | The timeout of the queue. A request cannot be queued for a period longer than the timeout. The default is ``60s``. | ``string`` | No | ### Upstream.Healthcheck The Healthcheck defines an [active health check](/nginx/admin-guide/load-balancer.md#http-health-check). In the example below we enable a health check for an upstream and configure all the available parameters, including the `slow-start` parameter combined with [`mandatory` and `persistent`](/nginx/admin-guide/load-balancer/http-health-check.md#mandatory-health-checks): ```yaml name: tea service: tea-svc port: 80 slow-start: 30s healthCheck: enable: true path: /healthz interval: 20s jitter: 3s fails: 5 passes: 5 port: 8080 tls: enable: true connect-timeout: 10s read-timeout: 10s send-timeout: 10s headers: - name: Host value: my.service statusMatch: "! 500" mandatory: true persistent: true keepalive-time: 60s ``` **Note:** This feature is supported only in NGINX Plus. |Field | Description | Type | Required | | ---| ---| ---| --- | |``enable`` | Enables a health check for an upstream server. The default is ``false``. | ``boolean`` | No | |``path`` | The path used for health check requests. The default is ``/``. This not configurable for gRPC type upstreams. | ``string`` | No | |``interval`` | The interval between two consecutive health checks. The default is ``5s``. | ``string`` | No | |``jitter`` | The time within which each health check will be randomly delayed. By default, there is no delay. | ``string`` | No | |``fails`` | The number of consecutive failed health checks of a particular upstream server after which this server will be considered unhealthy. The default is ``1``. | ``integer`` | No | |``passes`` | The number of consecutive passed health checks of a particular upstream server after which the server will be considered healthy. The default is ``1``. | ``integer`` | No | |``port`` | The port used for health check requests. By default, the [server port is used](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#health_check_port). Note: in contrast with the port of the upstream, this port is not a service port, but a port of a pod. | ``integer`` | No | |``tls`` | The TLS configuration used for health check requests. By default, the ``tls`` field of the upstream is used. | [upstream.tls](#upstreamtls) | No | |``connect-timeout`` | The timeout for establishing a connection with an upstream server. By default, the ``connect-timeout`` of the upstream is used. | ``string`` | No | |``read-timeout`` | The timeout for reading a response from an upstream server. By default, the ``read-timeout`` of the upstream is used. | ``string`` | No | |``send-timeout`` | The timeout for transmitting a request to an upstream server. By default, the ``send-timeout`` of the upstream is used. | ``string`` | No | |``headers`` | The request headers used for health check requests. NGINX Plus always sets the ``Host`` , ``User-Agent`` and ``Connection`` headers for health check requests. | [[]header](#header) | No | |``statusMatch`` | The expected response status codes of a health check. By default, the response should have status code 2xx or 3xx. Examples: ``"200"`` , ``"! 500"`` , ``"301-303 307"``. See the documentation of the [match](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html?#match) directive. This not supported for gRPC type upstreams. | ``string`` | No | |``grpcStatus`` | The expected [gRPC status code](https://github.com/grpc/grpc/blob/master/doc/statuscodes.md#status-codes-and-their-use-in-grpc) of the upstream server response to the [Check method](https://github.com/grpc/grpc/blob/master/doc/health-checking.md). Configure this field only if your gRPC services do not implement the gRPC health checking protocol. For example, configure ``12`` if the upstream server responds with `12 (UNIMPLEMENTED)` status code. Only valid on gRPC type upstreams. | ``int`` | No | |``grpcService`` | The gRPC service to be monitored on the upstream server. Only valid on gRPC type upstreams. | ``string`` | No | |``mandatory`` | Require every newly added server to pass all configured health checks before NGINX Plus sends traffic to it. If this is not specified, or is set to false, the server will be initially considered healthy. When combined with [slow-start](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#slow_start), it gives a new server more time to connect to databases and “warm up” before being asked to handle their full share of traffic. | ``bool`` | No | |``persistent`` | Set the initial “up” state for a server after reload if the server was considered healthy before reload. Enabling persistent requires that the mandatory parameter is also set to `true`. | ``bool`` | No | |``keepalive-time`` | Enables [keepalive](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive) connections for health checks and specifies the time during which requests can be processed through one keepalive connection. The default is ``60s``. | ``string`` | No | ### Upstream.SessionCookie The SessionCookie field configures session persistence which allows requests from the same client to be passed to the same upstream server. The information about the designated upstream server is passed in a session cookie generated by NGINX. In the example below, we configure session persistence with a session cookie for an upstream and configure all the available parameters: ```yaml name: tea service: tea-svc port: 80 sessionCookie: enable: true name: srv_id path: / expires: 1h domain: .example.com httpOnly: false secure: true samesite: strict ``` See the [`sticky`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html?#sticky) directive for additional information. The session cookie corresponds to the `sticky cookie` method. **Note:** Session persistence via sticky cookie is available in both NGINX (since version 1.29.6) and NGINX Plus. |Field | Description | Type | Required | | ---| ---| ---| --- | |``enable`` | Enables session persistence with a session cookie for an upstream server. The default is ``false``. | ``boolean`` | No | |``name`` | The name of the cookie. | ``string`` | Yes | |``path`` | The path for which the cookie is set. | ``string`` | No | |``expires`` | The time for which a browser should keep the cookie. Can be set to the special value ``max`` , which will cause the cookie to expire on ``31 Dec 2037 23:55:55 GMT``. | ``string`` | No | |``domain`` | The domain for which the cookie is set. | ``string`` | No | |``httpOnly`` | Adds the ``HttpOnly`` attribute to the cookie. | ``boolean`` | No | |``secure`` | Adds the ``Secure`` attribute to the cookie. | ``boolean`` | No | |``samesite`` | Adds the ``SameSite`` attribute to the cookie. The allowed values are: ``strict``, ``lax``, ``none`` | ``string`` | No | ### Action The action defines an action to perform for a request. In the example below, client requests are passed to an upstream `coffee`: ```yaml path: /coffee action: pass: coffee ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``pass`` | Passes requests to an upstream. The upstream with that name must be defined in the resource. | ``string`` | No | |``redirect`` | Redirects requests to a provided URL. | [action.redirect](#actionredirect) | No | |``return`` | Returns a preconfigured response. | [action.return](#actionreturn) | No | |``proxy`` | Passes requests to an upstream with the ability to modify the request/response (for example, rewrite the URI or modify the headers). | [action.proxy](#actionproxy) | No | \* -- an action must include exactly one of the following: `pass`, `redirect`, `return` or `proxy`. ### Action.Redirect The redirect action defines a redirect to return for a request. In the example below, client requests are passed to a url `http://www.nginx.com`: ```yaml redirect: url: http://www.nginx.com code: 301 ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``url`` | The URL to redirect the request to. Supported NGINX variables: ``$scheme`` , ``$http_x_forwarded_proto`` , ``$request_uri`` , ``$host``. Variables must be enclosed in curly braces. For example: ``${host}${request_uri}``. | ``string`` | Yes | |``code`` | The status code of a redirect. The allowed values are: ``301`` , ``302`` , ``307`` , ``308``. The default is ``301``. | ``int`` | No | ### Action.Return The return action defines a preconfigured response for a request. In the example below, NGINX will respond with the preconfigured response for every request: ```yaml return: code: 200 type: text/plain body: "Hello World\n" headers: - name: x-coffee value: espresso ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``code`` | The status code of the response. The allowed values are: ``2XX``, ``4XX`` or ``5XX``. The default is ``200``. | ``int`` | No | |``type`` | The MIME type of the response. The default is ``text/plain``. | ``string`` | No | |``body`` | The body of the response. Supports NGINX variables*. Variables must be enclosed in curly brackets. For example: ``Request is ${request_uri}\n``. | ``string`` | Yes | |``headers`` | The custom headers of the response. | [[]Action.Return.Header](#actionreturnheader) | No | \* -- Supported NGINX variables: `$request_uri`, `$request_method`, `$request_body`, `$scheme`, `$http_`, `$args`, `$arg_`, `$cookie_`, `$host`, `$request_time`, `$request_length`, `$nginx_version`, `$pid`, `$connection`, `$remote_addr`, `$remote_port`, `$time_iso8601`, `$time_local`, `$server_addr`, `$server_port`, `$server_name`, `$server_protocol`, `$connections_active`, `$connections_reading`, `$connections_writing` and `$connections_waiting`. ### Action.Return.Header The header defines an HTTP Header for a canned response in an actionReturn: ```yaml name: x-coffee value: espresso ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``name`` | The name of the header. | ``string`` | Yes | |``value`` | The value of the header. | ``string`` | Yes | ### Action.Proxy The proxy action passes requests to an upstream with the ability to modify the request/response (for example, rewrite the URI or modify the headers). In the example below, the request URI is rewritten to `/`, and the request and the response headers are modified: ```yaml proxy: upstream: coffee requestHeaders: pass: true set: - name: My-Header value: Value - name: Client-Cert value: ${ssl_client_escaped_cert} responseHeaders: add: - name: My-Header value: Value - name: IC-Nginx-Version value: ${nginx_version} always: true hide: - x-internal-version ignore: - Expires - Set-Cookie pass: - Server rewritePath: / ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``upstream`` | The name of the upstream which the requests will be proxied to. The upstream with that name must be defined in the resource. | ``string`` | Yes | |``requestHeaders`` | The request headers modifications. | [action.Proxy.RequestHeaders](#actionproxyrequestheaders) | No | |``responseHeaders`` | The response headers modifications. | [action.Proxy.ResponseHeaders](#actionproxyresponseheaders) | No | |``rewritePath`` | The rewritten URI. If the route path is a regular expression -- starts with `~` -- the `rewritePath` can include capture groups with ``$1-9``. For example `$1` for the first group, and so on. For more information, check the [rewrite](https://github.com/nginx/kubernetes-ingress/tree/v/examples/custom-resources/rewrites) example. | ``string`` | No | ### Action.Proxy.RequestHeaders The RequestHeaders field modifies the headers of the request to the proxied upstream server. |Field | Description | Type | Required | | ---| ---| ---| --- | |``pass`` | Passes the original request headers to the proxied upstream server. See the [proxy_pass_request_header](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass_request_headers) directive for more information. Default is true. | ``bool`` | No | |``set`` | Allows redefining or appending fields to present request headers passed to the proxied upstream servers. See the [proxy_set_header](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_header) directive for more information. | [[]header](#actionproxyrequestheaderssetheader) | No | ### Action.Proxy.RequestHeaders.Set.Header The header defines an HTTP Header: ```yaml name: My-Header value: My-Value ``` It is possible to override the default value of the `Host` header, which NGINX Ingress Controller sets to [`$host`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_host): ```yaml name: Host value: example.com ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``name`` | The name of the header. | ``string`` | Yes | |``value`` | The value of the header. Supports NGINX variables*. Variables must be enclosed in curly brackets. For example: ``${scheme}``. | ``string`` | No | \* -- Supported NGINX variables: `$request_uri`, `$request_method`, `$request_body`, `$scheme`, `$http_`, `$args`, `$arg_`, `$cookie_`, `$host`, `$request_time`, `$request_length`, `$nginx_version`, `$pid`, `$connection`, `$remote_addr`, `$remote_port`, `$time_iso8601`, `$time_local`, `$server_addr`, `$server_port`, `$server_name`, `$server_protocol`, `$connections_active`, `$connections_reading`, `$connections_writing`, `$connections_waiting`, `$ssl_cipher`, `$ssl_ciphers`, `$ssl_client_cert`, `$ssl_client_escaped_cert`, `$ssl_client_fingerprint`, `$ssl_client_i_dn`, `$ssl_client_i_dn_legacy`, `$ssl_client_raw_cert`, `$ssl_client_s_dn`, `$ssl_client_s_dn_legacy`, `$ssl_client_serial`, `$ssl_client_v_end`, `$ssl_client_v_remain`, `$ssl_client_v_start`, `$ssl_client_verify`, `$ssl_curves`, `$ssl_early_data`, `$ssl_protocol`, `$ssl_server_name`, `$ssl_session_id`, `$ssl_session_reused`, `$jwt_claim_` (NGINX Plus only) and `$jwt_header_` (NGINX Plus only). ### Action.Proxy.ResponseHeaders The ResponseHeaders field modifies the headers of the response to the client. |Field | Description | Type | Required | | ---| ---| ---| --- | |``hide`` | The headers that will not be passed* in the response to the client from a proxied upstream server. See the [proxy_hide_header](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_hide_header) directive for more information. | ``[]string`` | No | |``pass`` | Allows passing the hidden header fields* to the client from a proxied upstream server. See the [proxy_pass_header](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass_header) directive for more information. | ``[]string`` | No | |``ignore`` | Disables processing of certain headers** to the client from a proxied upstream server. See the [proxy_ignore_headers](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ignore_headers) directive for more information. | ``[]string`` | No | |``add`` | Adds headers to the response to the client. | [[]addHeader](#addheader) | No | \* -- Default hidden headers are: `Date`, `Server`, `X-Pad` and `X-Accel-...`. \** -- The following fields can be ignored: `X-Accel-Redirect`, `X-Accel-Expires`, `X-Accel-Limit-Rate`, `X-Accel-Buffering`, `X-Accel-Charset`, `Expires`, `Cache-Control`, `Set-Cookie` and `Vary`. ### AddHeader The addHeader defines an HTTP Header with an optional `always` field: ```yaml name: My-Header value: My-Value always: true ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``name`` | The name of the header. | ``string`` | Yes | |``value`` | The value of the header. Supports NGINX variables*. Variables must be enclosed in curly brackets. For example: ``${scheme}``. | ``string`` | No | |``always`` | If set to true, add the header regardless of the response status code**. Default is false. See the [add_header](http://nginx.org/en/docs/http/ngx_http_headers_module.html#add_header) directive for more information. | ``bool`` | No | \* -- Supported NGINX variables: `$request_uri`, `$request_method`, `$request_body`, `$scheme`, `$http_`, `$args`, `$arg_`, `$cookie_`, `$host`, `$request_time`, `$request_length`, `$nginx_version`, `$pid`, `$connection`, `$remote_addr`, `$remote_port`, `$time_iso8601`, `$time_local`, `$server_addr`, `$server_port`, `$server_name`, `$server_protocol`, `$connections_active`, `$connections_reading`, `$connections_writing`, `$connections_waiting`, `$ssl_cipher`, `$ssl_ciphers`, `$ssl_client_cert`, `$ssl_client_escaped_cert`, `$ssl_client_fingerprint`, `$ssl_client_i_dn`, `$ssl_client_i_dn_legacy`, `$ssl_client_raw_cert`, `$ssl_client_s_dn`, `$ssl_client_s_dn_legacy`, `$ssl_client_serial`, `$ssl_client_v_end`, `$ssl_client_v_remain`, `$ssl_client_v_start`, `$ssl_client_verify`, `$ssl_curves`, `$ssl_early_data`, `$ssl_protocol`, `$ssl_server_name`, `$ssl_session_id`, `$ssl_session_reused`, `$jwt_claim_` (NGINX Plus only) and `$jwt_header_` (NGINX Plus only). **Note:** If `always` is false, the response header is added only if the response status code is any of `200`, `201`, `204`, `206`, `301`, `302`, `303`, `304`, `307` or `308`. ### Split The split defines a weight for an action as part of the splits configuration. In the example below NGINX passes 80% of requests to the upstream `coffee-v1` and the remaining 20% to `coffee-v2`: ```yaml splits: - weight: 80 action: pass: coffee-v1 - weight: 20 action: pass: coffee-v2 ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``weight`` | The weight of an action. Must fall into the range ``0..100``. The sum of the weights of all splits must be equal to ``100``. | ``int`` | Yes | |``action`` | The action to perform for a request. | [action](#action) | Yes | ### Match The match defines a match between conditions and an action or splits. In the example below, NGINX routes requests with the path `/coffee` to different upstreams based on the value of the cookie `user`: - `user=john` -> `coffee-future` - `user=bob` -> `coffee-deprecated` - If the cookie is not set or not equal to either `john` or `bob`, NGINX routes to `coffee-stable` ```yaml path: /coffee matches: - conditions: - cookie: user value: john action: pass: coffee-future - conditions: - cookie: user value: bob action: pass: coffee-deprecated action: pass: coffee-stable ``` In the next example, NGINX routes requests based on the value of the built-in [`$request_method` variable](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_request_method), which represents the HTTP method of a request: - all POST requests -> `coffee-post` - all non-POST requests -> `coffee` ```yaml path: /coffee matches: - conditions: - variable: $request_method value: POST action: pass: coffee-post action: pass: coffee ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``conditions`` | A list of conditions. Must include at least 1 condition. | [[]condition](#condition) | Yes | |``action`` | The action to perform for a request. | [action](#action) | No | |``splits`` | The splits configuration for traffic splitting. Must include at least 2 splits. | [[]split](#split) | No | **Note:** A match must include exactly one of the following: `action` or `splits`. ### Condition The condition defines a condition in a match. |Field | Description | Type | Required | | ---| ---| ---| --- | |``header`` | The name of a header. Must consist of alphanumeric characters or ``-``. | ``string`` | No | |``cookie`` | The name of a cookie. Must consist of alphanumeric characters or ``_``. | ``string`` | No | |``argument`` | The name of an argument. Must consist of alphanumeric characters or ``_``. | ``string`` | No | |``variable`` | The name of an NGINX variable. Must start with ``$``. See the list of the supported variables below the table. | ``string`` | No | |``value`` | The value to match the condition against. How to define a value is shown below the table. | ``string`` | Yes | **Note:** a condition must include exactly one of the following: `header`, `cookie`, `argument` or `variable`. Supported NGINX variables: `$args`, `$http2`, `$https`, `$remote_addr`, `$remote_port`, `$query_string`, `$request`, `$request_body`, `$request_uri`, `$request_method`, `$scheme`. Find the documentation for each variable [here](https://nginx.org/en/docs/varindex.html). The value supports two kinds of matching: - *Case-insensitive string comparison*. For example: - `john` -- case-insensitive matching that succeeds for strings, such as `john`, `John`, `JOHN`. - `!john` -- negation of the case-insensitive matching for john that succeeds for strings, such as `bob`, `anything`, `''` (empty string). - *Matching with a regular expression*. Note that NGINX supports regular expressions compatible with those used by the Perl programming language (PCRE). For example: - `~^yes` -- a case-sensitive regular expression that matches any string that starts with `yes`. For example: `yes`, `yes123`. - `!~^yes` -- negation of the previous regular expression that succeeds for strings like `YES`, `Yes123`, `noyes`. (The negation mechanism is not part of the PCRE syntax). - `~*no$` -- a case-insensitive regular expression that matches any string that ends with `no`. For example: `no`, `123no`, `123NO`. **Note:** A value must not include any unescaped double quotes (`"`) and must not end with an unescaped backslash (`\`). For example, the following are invalid values: `some"value`, `somevalue\`. ### ErrorPage The errorPage defines a custom response for a route for the case when either an upstream server responds with (or NGINX generates) an error status code. The custom response can be a redirect or a canned response. See the [error_page](https://nginx.org/en/docs/http/ngx_http_core_module.html#error_page) directive for more information. ```yaml path: /coffee errorPages: - codes: [502, 503] redirect: code: 301 url: https://nginx.org - codes: [404] return: code: 200 body: "Original resource not found, but success!" ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``codes`` | A list of error status codes. | ``[]int`` | Yes | |``redirect`` | The redirect action for the given status codes. | [errorPage.Redirect](#errorpageredirect) | No | |``return`` | The canned response action for the given status codes. | [errorPage.Return](#errorpagereturn) | No | **Note:** An errorPage must include exactly one of the following: `return` or `redirect`. ### ErrorPage.Redirect The redirect defines a redirect for an errorPage. In the example below, NGINX responds with a redirect when a response from an upstream server has a 404 status code. ```yaml codes: [404] redirect: code: 301 url: ${scheme}://cafe.example.com/error.html ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``code`` | The status code of a redirect. The allowed values are: ``301`` , ``302`` , ``307`` , ``308``. The default is ``301``. | ``int`` | No | |``url`` | The URL to redirect the request to. Supported NGINX variables: ``$scheme`` and ``$http_x_forwarded_proto``. Variables must be enclosed in curly braces. For example: ``${scheme}``. | ``string`` | Yes | ### ErrorPage.Return The return defines a canned response for an errorPage. In the example below, NGINX responds with a canned response when a response from an upstream server has either 401 or 403 status code. ```yaml codes: [401, 403] return: code: 200 type: application/json body: | {\"msg\": \"You don't have permission to do this\"} headers: - name: x-debug-original-statuses value: ${upstream_status} ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``code`` | The status code of the response. The default is the status code of the original response. | ``int`` | No | |``type`` | The MIME type of the response. The default is ``text/html``. | ``string`` | No | |``body`` | The body of the response. Supported NGINX variable: ``$upstream_status`` . Variables must be enclosed in curly braces. For example: ``${upstream_status}``. | ``string`` | Yes | |``headers`` | The custom headers of the response. | [[]errorPage.Return.Header](#errorpagereturnheader) | No | ### ErrorPage.Return.Header The header defines an HTTP Header for a canned response in an errorPage: ```yaml name: x-debug-original-statuses value: ${upstream_status} ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``name`` | The name of the header. | ``string`` | Yes | |``value`` | The value of the header. Supported NGINX variable: ``$upstream_status`` . Variables must be enclosed in curly braces. For example: ``${upstream_status}``. | ``string`` | No | ## Using VirtualServer and VirtualServerRoute You can use the usual `kubectl` commands to work with VirtualServer and VirtualServerRoute resources, similar to Ingress resources. For example, the following command creates a VirtualServer resource defined in `cafe-virtual-server.yaml` with the name `cafe`: ```shell kubectl apply -f cafe-virtual-server.yaml ``` ```text virtualserver.k8s.nginx.org "cafe" created ``` You can get the resource by running: ```shell kubectl get virtualserver cafe ``` ```text NAME STATE HOST IP PORTS AGE cafe Valid cafe.example.com 12.13.23.123 [80,443] 3m ``` In `kubectl get` and similar commands, you can use the short name `vs` instead of `virtualserver`. Similarly, for VirtualServerRoute you can use `virtualserverroute` or the short name `vsr`. ### Using Snippets Snippets allow you to insert raw NGINX config into different contexts of NGINX configuration. In the example below, we use snippets to configure several NGINX features in a VirtualServer: ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServer metadata: name: cafe namespace: cafe spec: http-snippets: | limit_req_zone $binary_remote_addr zone=mylimit:10m rate=1r/s; proxy_cache_path /tmp keys_zone=one:10m; host: cafe.example.com tls: secret: cafe-secret server-snippets: | limit_req zone=mylimit burst=20; upstreams: - name: tea service: tea-svc port: 80 - name: coffee service: coffee-svc port: 80 routes: - path: /tea location-snippets: | proxy_cache one; proxy_cache_valid 200 10m; action: pass: tea - path: /coffee action: pass: coffee ``` For additional information, view the [Advanced configuration with Snippets](/nic/configuration/ingress-resources/advanced-configuration-with-snippets.md) topic. ### Validation Two types of validation are available for VirtualServer and VirtualServerRoute resources: - *Structural validation* by the `kubectl` and Kubernetes API server. - *Comprehensive validation* by NGINX Ingress Controller. #### Structural Validation The custom resource definitions for VirtualServer and VirtualServerRoute include structural OpenAPI schema which describes the type of every field of those resources. If you try to create (or update) a resource that violates the structural schema (for example, you use a string value for the port field of an upstream), `kubectl` and Kubernetes API server will reject such a resource: - Example of `kubectl` validation: ```shell kubectl apply -f cafe-virtual-server.yaml ``` ```text error: error validating "cafe-virtual-server.yaml": error validating data: ValidationError(VirtualServer.spec.upstreams[0].port): invalid type for org.nginx.k8s.v1.VirtualServer.spec.upstreams.port: got "string", expected "integer"; if you choose to ignore these errors, turn validation off with --validate=false ``` - Example of Kubernetes API server validation: ```shell kubectl apply -f cafe-virtual-server.yaml --validate=false ``` ```text The VirtualServer "cafe" is invalid: []: Invalid value: map[string]interface {}{ ... }: validation failure list: spec.upstreams.port in body must be of type integer: "string" ``` If a resource is not rejected (it doesn't violate the structural schema), NGINX Ingress Controller will validate it further. #### Comprehensive Validation NGINX Ingress Controller validates the fields of the VirtualServer and VirtualServerRoute resources. If a resource is invalid, NGINX Ingress Controller will reject it: the resource will continue to exist in the cluster, but NGINX Ingress Controller will ignore it. You can check if NGINX Ingress Controller successfully applied the configuration for a VirtualServer. For our example `cafe` VirtualServer, we can run: ```shell kubectl describe vs cafe ``` ```text ... Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal AddedOrUpdated 16s nginx-ingress-controller Configuration for default/cafe was added or updated ``` Note how the events section includes a Normal event with the AddedOrUpdated reason that informs us that the configuration was successfully applied. If you create an invalid resource, NGINX Ingress Controller will reject it and emit a Rejected event. For example, if you create a VirtualServer `cafe` with two upstream with the same name `tea`, you will get: ```shell kubectl describe vs cafe ``` ```text ... Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning Rejected 12s nginx-ingress-controller VirtualServer default/cafe is invalid and was rejected: spec.upstreams[1].name: Duplicate value: "tea" ``` Note how the events section includes a Warning event with the Rejected reason. Additionally, this information is also available in the `status` field of the VirtualServer resource. Note the Status section of the VirtualServer: ```shell kubectl describe vs cafe ``` ```text ... Status: External Endpoints: Ip: 12.13.23.123 Ports: [80,443] Message: VirtualServer default/cafe is invalid and was rejected: spec.upstreams[1].name: Duplicate value: "tea" Reason: Rejected State: Invalid ``` NGINX Ingress Controller validates VirtualServerRoute resources in a similar way. **Note**: If you make an existing resource invalid, NGINX Ingress Controller will reject it and remove the corresponding configuration from NGINX. ## Multiple regex routes in a VirtualServerRoute NGINX Ingress Controller lets you reference the same VirtualServerRoute from multiple regex routes in a VirtualServer. This means you can group related regex paths by concern — for example, all `/api` paths go to one team's VirtualServerRoute and all `/images` paths go to another. ### How it works When NGINX Ingress Controller processes a VirtualServer, it collects every regex route (`~` or `~*`) that references the same VirtualServerRoute. It then validates the collected VirtualServer paths against the VirtualServerRoute's subroutes as a single set. Each subroute produces a separate NGINX `location` block in the generated configuration. ### Set match requirement The VirtualServer paths and VirtualServerRoute subroutes must form a **bidirectional set match**: - Every VirtualServer regex path that references the VirtualServerRoute must have a corresponding subroute. - Every subroute in the VirtualServerRoute must be referenced by a VirtualServer regex path. If a VirtualServer has a path that the referenced VirtualServerRoute does not, or if a VirtualServerRoute has a subpath that isn't present in the VirtualServer, NGINX Ingress Controller rejects the VirtualServerRoute and the VirtualServer enters a warning state. | Direction | Rule | Error if violated | | --- | --- | --- | | VirtualServer → VirtualServerRoute | Every VirtualServer regex path must appear as a subroute | "subroute with path '...' is missing" | | VirtualServerRoute → VirtualServer | Every subroute must be referenced by a VirtualServer regex path | "subroute path '...' is not referenced by any VS route" | ### Configure multiple regex routes The following example shows a VirtualServer with five routes. Two case-sensitive regex routes delegate to `vsr-api`, and two case-insensitive regex routes delegate to `vsr-media`. The `/health` route is handled directly by the VirtualServer. #### VirtualServer ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServer metadata: name: api-gateway spec: host: api-gateway.example.com routes: - path: /health action: return: code: 200 type: text/plain body: "OK" - path: "~/api/v1" route: vsr-api - path: "~/api/v2" route: vsr-api - path: "~*/images/jpg" route: vsr-media - path: "~*/images/png" route: vsr-media ``` #### VirtualServerRoute for API routes ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServerRoute metadata: name: vsr-api spec: host: api-gateway.example.com subroutes: - path: "~/api/v1" action: pass: api-v1 - path: "~ /api/v2" action: pass: api-v2 ``` Notice that the second subroute uses `"~ /api/v2"` (with a space) while the VirtualServer declares `"~/api/v2"` (without a space). NGINX Ingress Controller normalizes both to the same value, so the configuration is valid. See [Path normalization](#path-normalization) for details. #### VirtualServerRoute for media routes ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServerRoute metadata: name: vsr-media spec: host: api-gateway.example.com subroutes: - path: "~*/images/jpg" action: pass: images-jpg - path: "~*/images/png" action: pass: images-png ``` #### Generated NGINX configuration Each subroute produces a separate `location` block. The generated NGINX configuration for the four regex subroutes looks like this: ```nginx location ~ "/api/v1" { ... } location ~ "/api/v2" { ... } location ~* "/images/jpg" { ... } location ~* "/images/png" { ... } ``` #### Request routing | Request URI | Matching location | VirtualServerRoute | | --- | --- | --- | | `/health` | `/health` (prefix, direct VirtualServer action) | None | | `/api/v1` | `~ "/api/v1"` | `vsr-api` | | `/api/v2` | `~ "/api/v2"` | `vsr-api` | | `/images/jpg` | `~* "/images/jpg"` | `vsr-media` | | `/images/png` | `~* "/images/png"` | `vsr-media` | ### Path normalization NGINX Ingress Controller strips whitespace between the regex modifier (`~` or `~*`) and the path before comparing values. This means `"~/api/v1"` and `"~ /api/v1"` normalize to the same value. Normalization applies to all path comparisons: - **Set match validation**: A VirtualServer path `"~/api/v2"` matches a VirtualServerRoute subroute `"~ /api/v2"` because both normalize to `"~/api/v2"`. - **Duplicate detection**: If a single VirtualServerRoute contains both `"~*/images/jpg"` and `"~* /images/jpg"` as subroutes, they normalize to the same value and NGINX Ingress Controller rejects the VirtualServerRoute with a duplicate path error. **important:** Using `"~ /api/v1"` in a VirtualServerRoute to match `"~/api/v1"` in a VirtualServer is valid — the paths are in different resources and represent the same logical route. Having both `"~/api/v1"` and `"~ /api/v1"` as separate subroutes in the **same** VirtualServerRoute is invalid — they resolve to the same normalized path and are treated as duplicates. ### Validation rules NGINX Ingress Controller enforces the following rules for regex route delegation: - Every VirtualServer regex path referencing a VirtualServerRoute must have a matching subroute in that VirtualServerRoute. - Every subroute in the VirtualServerRoute must be referenced by a VirtualServer regex path. - Subroute paths must be unique after normalization. Paths that differ only in whitespace between the modifier and the URI are treated as duplicates. - All VirtualServer routes referencing the same VirtualServerRoute must use the same modifier category. You can't have a regex route (`~`) and a prefix route (`/`) both referencing the same VirtualServerRoute. - Case-sensitive (`~`) and case-insensitive (`~*`) modifiers can reference the same VirtualServerRoute because both are in the regex category. - Exact match routes (`=`) still require exactly one subroute. Only regex routes support multiple subroutes per VirtualServerRoute. ### Validation errors When NGINX Ingress Controller detects a validation failure, it rejects the VirtualServerRoute and the VirtualServer enters a warning state. Other VirtualServerRoutes and direct VirtualServer routes continue to serve traffic normally. #### Missing subroute This error occurs when the VirtualServer references a regex path through a VirtualServerRoute, but the VirtualServerRoute doesn't have a subroute for that path. For example, the VirtualServer references both `~/api/v1` and `~/api/v2` through `vsr-api`, but the VirtualServerRoute only defines `~/api/v1`: ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServerRoute metadata: name: vsr-api spec: host: api-gateway.example.com subroutes: - path: "~/api/v1" action: pass: api-v1 ``` NGINX Ingress Controller rejects the VirtualServerRoute with the following error: ```text VirtualServerRoute default/vsr-api is invalid: spec.subroutes: Invalid value: "subroutes": subroute with path '~/api/v2' is missing; all VS route paths must be covered by VSR subroutes ``` To fix this, add the missing subroute (`~/api/v2`) to the VirtualServerRoute, or remove the VirtualServer route that references it. #### Extra subroute This error occurs when the VirtualServerRoute has a subroute that no VirtualServer route references. For example, the VirtualServerRoute defines `~/api/v1`, `~/api/v2`, and `~/api/v3`, but the VirtualServer only references `~/api/v1` and `~/api/v2`: ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServerRoute metadata: name: vsr-api spec: host: api-gateway.example.com subroutes: - path: "~/api/v1" action: pass: api-v1 - path: "~/api/v2" action: pass: api-v2 - path: "~/api/v3" action: pass: api-v3 ``` NGINX Ingress Controller rejects the VirtualServerRoute with the following error: ```text VirtualServerRoute default/vsr-api is invalid: spec.subroutes[2].path: Invalid value: "~/api/v3": subroute path '~/api/v3' is not referenced by any VS route; all VSR subroutes must be referenced ``` To fix this, remove the extra subroute from the VirtualServerRoute, or add a corresponding regex route in the VirtualServer. #### Duplicate paths after normalization This error occurs when two subroutes in the same VirtualServerRoute resolve to the same path after normalization. Because NGINX Ingress Controller strips whitespace between the modifier and the URI, paths that appear different in YAML can be identical after normalization. For example, the following VirtualServerRoute has `"~/api/v1"` and `"~ /api/v1"` as separate subroutes: ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServerRoute metadata: name: vsr-api spec: host: api-gateway.example.com subroutes: - path: "~/api/v1" action: pass: api-v1 - path: "~ /api/v1" action: pass: api-v1-alt ``` Both paths normalize to `"~/api/v1"`. NGINX Ingress Controller rejects the VirtualServerRoute: ```text VirtualServerRoute default/vsr-api is invalid: spec.subroutes[0].path: Duplicate value: "~/api/v1", spec.subroutes[1].path: Duplicate value: "~ /api/v1" ``` To fix this, remove the duplicate subroute. If you need to match the same pattern, use a single subroute. ### See also - [VirtualServer and VirtualServerRoute resources](/nic/configuration/virtualserver-and-virtualserverroute-resources.md) for the full field reference. - NGINX [location directive documentation](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) for how NGINX evaluates regex locations. - [Multiple regex routes examples](https://github.com/nginx/kubernetes-ingress/tree/v/examples/custom-resources/vsr-multiple-regex-routes) on GitHub for runnable manifests and additional error case examples. ## Customization using ConfigMap You can customize the NGINX configuration for VirtualServer and VirtualServerRoutes resources using the [ConfigMap](/nic/configuration/global-configuration/configmap-resource.md). Most of the ConfigMap keys are supported, with the following exceptions: - `proxy-hide-headers` - `proxy-pass-headers` - `hsts` - `hsts-max-age` - `hsts-include-subdomains` - `hsts-behind-proxy` - `redirect-to-https` - `ssl-redirect` - `http-redirect-code` -------------------------------------------------------------------------------- title: "VirtualServer and VirtualServerRoute resources" url: https://docs.nginx.com/nginx-ingress-controller/lts/configuration/virtualserver-and-virtualserverroute-resources/index.md -------------------------------------------------------------------------------- This document is reference material for the VirtualServer and VirtualServerRoute resources used by F5 NGINX Ingress Controller LTS. VirtualServer and VirtualServerRoute resources are load balancing configurations recommended as an alternative to the Ingress resource. They enable use cases not supported with the Ingress resource, such as traffic splitting and advanced content-based routing. The resources are implemented as [Custom Resources](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/). The GitHub repository has [examples of the resources](https://github.com/nginx/kubernetes-ingress/tree/v/examples/custom-resources) for specific use cases. --- ## VirtualServer specification The VirtualServer resource defines load balancing configuration for a domain name, such as `example.com`. The example below shows a VirtualServer with both traditional static routes and a dynamic route using `routeSelector`. The route with path `/dynamic` will automatically include any VirtualServerRoute resources that have the label `app: cafe`: ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServer metadata: name: cafe spec: host: cafe.example.com listener: http: http-8083 https: https-8443 tls: secret: cafe-secret gunzip: on upstreams: - name: tea service: tea-svc port: 80 - name: coffee service: coffee-svc port: 80 routes: - path: /tea action: pass: tea - path: /coffee action: pass: coffee - path: ~ ^/decaf/.*\\.jpg$ action: pass: coffee - path: =/green/tea action: pass: tea - path: /dynamic routeSelector: matchLabels: app: cafe ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``host`` | The host (domain name) of the server. Must be a valid subdomain as defined in RFC 1123, such as ``my-app`` or ``hello.example.com``. When using a wildcard domain like ``*.example.com`` the domain must be contained in double quotes. The ``host`` value needs to be unique among all Ingress and VirtualServer resources. See also [Handling Host and Listener Collisions](/nic/lts/configuration/host-and-listener-collisions.md). | ``string`` | Yes | |``listener`` | Sets a custom HTTP and/or HTTPS listener. Valid fields are `listener.http` and `listener.https`. Each field must reference the name of a valid listener defined in a GlobalConfiguration resource | [listener](#virtualserverlistener) | No | |``tls`` | The TLS termination configuration. | [tls](#virtualservertls) | No | |``gunzip`` | Enables or disables [decompression](/nginx/admin-guide/web-server/compression.md) of gzipped responses for clients. Allowed values “on”/“off”, “true”/“false” or “yes”/“no”. If the ``gunzip`` value is not set, it defaults to ``off``. | ``boolean`` | No | |``externalDNS`` | The externalDNS configuration for a VirtualServer. | [externalDNS](#virtualserverexternaldns) | No | |``dos`` | A reference to a DosProtectedResource, setting this enables DOS protection of the VirtualServer. | ``string`` | No | |``policies`` | A list of policies. | [[]policy](#virtualserverpolicy) | No | |``upstreams`` | A list of upstreams. | [[]upstream](#upstream) | No | |``routes`` | A list of routes. | [[]route](#virtualserverroute) | No | |``ingressClassName`` | Specifies which Ingress Controller must handle the VirtualServer resource. | ``string`` | No | |``internalRoute`` | Specifies if the VirtualServer resource is an internal route or not. | ``boolean`` | No | |``http-snippets`` | Sets a custom snippet in the http context. | ``string`` | No | |``server-snippets`` | Sets a custom snippet in server context. Overrides the ``server-snippets`` ConfigMap key. | ``string`` | No | ### VirtualServer.TLS The tls field defines TLS configuration for a VirtualServer. For example: ```yaml secret: cafe-secret redirect: enable: true ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``secret`` | The name of a secret with a TLS certificate and key. The secret must belong to the same namespace as the VirtualServer. The secret must be of the type ``kubernetes.io/tls`` and contain keys named ``tls.crt`` and ``tls.key`` that contain the certificate and private key as described [here](https://kubernetes.io/docs/concepts/services-networking/ingress/#tls). If the secret doesn't exist or is invalid, NGINX will break any attempt to establish a TLS connection to the host of the VirtualServer. If the secret is not specified but [wildcard TLS secret](/nic/lts/configuration/global-configuration/command-line-arguments.md#cmdoption-wildcard-tls-secret) is configured, NGINX will use the wildcard secret for TLS termination. | ``string`` | No | |``redirect`` | The redirect configuration of the TLS for a VirtualServer. | [tls.redirect](#virtualservertlsredirect) | No | ### VirtualServer.TLS.Redirect | |``cert-manager`` | The cert-manager configuration of the TLS for a VirtualServer. | [tls.cert-manager](#virtualservertlscertmanager) | No | ### VirtualServer.TLS.CertManager | ### VirtualServer.TLS.Redirect The redirect field configures a TLS redirect for a VirtualServer: ```yaml enable: true code: 301 basedOn: scheme ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``enable`` | Enables a TLS redirect for a VirtualServer. The default is ``False``. | ``boolean`` | No | |``code`` | The status code of a redirect. The allowed values are: ``301`` , ``302`` , ``307`` , ``308``. The default is ``301``. | ``int`` | No | |``basedOn`` | The attribute of a request that NGINX will evaluate to send a redirect. The allowed values are ``scheme`` (the scheme of the request) or ``x-forwarded-proto`` (the ``X-Forwarded-Proto`` header of the request). The default is ``scheme``. | ``string`` | No | ### VirtualServer.Policy | ### VirtualServer.TLS.CertManager The cert-manager field configures x509 automated Certificate management for VirtualServer resources using cert-manager (cert-manager.io). Please see the [cert-manager configuration documentation](https://cert-manager.io/docs/configuration/) for more information on deploying and configuring Issuers. Example: ```yaml cert-manager: cluster-issuer: "my-issuer-name" ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``issuer`` | the name of an Issuer. An Issuer is a cert-manager resource which describes the certificate authority capable of signing certificates. The Issuer must be in the same namespace as the VirtualServer resource. Please note that one of `issuer` and `cluster-issuer` are required, but they are mutually exclusive - one and only one must be defined. | ``string`` | No | |``cluster-issuer`` | the name of a ClusterIssuer. A ClusterIssuer is a cert-manager resource which describes the certificate authority capable of signing certificates. It does not matter which namespace your VirtualServer resides, as ClusterIssuers are non-namespaced resources. Please note that one of `issuer` and `cluster-issuer` are required, but they are mutually exclusive - one and only one must be defined. | ``string`` | No | |``issuer-kind`` | The kind of the external issuer resource, for example AWSPCAIssuer. This is only necessary for out-of-tree issuers. This cannot be defined if `cluster-issuer` is also defined. | ``string`` | No | |``issuer-group`` | The API group of the external issuer controller, for example awspca.cert-manager.io. This is only necessary for out-of-tree issuers. This cannot be defined if `cluster-issuer` is also defined. | ``string`` | No | |``common-name`` | This field allows you to configure spec.commonName for the Certificate to be generated. This configuration adds a CN to the x509 certificate. | ``string`` | No | |``duration`` | This field allows you to configure spec.duration field for the Certificate to be generated. Must be specified using a [Go time.Duration](https://pkg.go.dev/time#ParseDuration) string format, which does not allow the d (days) suffix. You must specify these values using s, m, and h suffixes instead. | ``string`` | No | |``renew-before`` | this annotation allows you to configure spec.renewBefore field for the Certificate to be generated. Must be specified using a [Go time.Duration](https://pkg.go.dev/time#ParseDuration) string format, which does not allow the d (days) suffix. You must specify these values using s, m, and h suffixes instead. | ``string`` | No | |``usages`` | This field allows you to configure spec.usages field for the Certificate to be generated. Pass a string with comma-separated values i.e. ``key agreement,digital signature, server auth``. An exhaustive list of supported key usages can be found in the [the cert-manager api documentation](https://cert-manager.io/docs/reference/api-docs/#cert-manager.io/v1.KeyUsage). | ``string`` | No | |``issue-temp-cert`` | When ``true``, ask cert-manager for a [temporary self-signed certificate](https://cert-manager.io/docs/usage/certificate/#temporary-certificates-while-issuing) pending the issuance of the Certificate. This allows HTTPS-only servers to use ACME HTTP01 challenges when the TLS secret does not exist yet. | ``boolean`` | No | ### VirtualServer.Listener The listener field defines a custom HTTP and/or HTTPS listener. The respective listeners used must reference the name of a listener defined using a [GlobalConfiguration](/nic/lts/configuration/global-configuration/globalconfiguration-resource.md) resource. For example: ```yaml http: http-8083 https: https-8443 ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``http`` | The name of am HTTP listener defined in a [GlobalConfiguration](/nic/lts/configuration/global-configuration/globalconfiguration-resource.md) resource. | ``string`` | No | |``https`` | The name of an HTTPS listener defined in a [GlobalConfiguration](/nic/lts/configuration/global-configuration/globalconfiguration-resource.md) resource. | ``string`` | No | ### VirtualServer.ExternalDNS The externalDNS field configures controlling DNS records dynamically for VirtualServer resources using [ExternalDNS](https://github.com/kubernetes-sigs/external-dns). Please see the [ExternalDNS configuration documentation](https://kubernetes-sigs.github.io/external-dns/) for more information on deploying and configuring ExternalDNS and Providers. Example: ```yaml enable: true ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``enable`` | Enables ExternalDNS integration for a VirtualServer resource. The default is ``false``. | ``string`` | No | |``labels`` | Configure labels to be applied to the Endpoint resources that will be consumed by ExternalDNS. | ``map[string]string`` | No | |``providerSpecific`` | Configure provider specific properties which holds the name and value of a configuration which is specific to individual DNS providers. | [[]ProviderSpecific](#virtualserverexternaldnsproviderspecific) | No | |``recordTTL`` | TTL for the DNS record. This defaults to 0 if not defined. See [the ExternalDNS TTL documentation for provider-specific defaults](https://kubernetes-sigs.github.io/external-dns/v0.14.2/ttl/#providers) | ``int64`` | No | |``recordType`` | The record Type that should be created, e.g. "A", "AAAA", "CNAME". This is automatically computed based on the external endpoints if not defined. | ``string`` | No | ### VirtualServer.ExternalDNS.ProviderSpecific The providerSpecific field of the externalDNS block allows the specification of provider specific properties which is a list of key value pairs of configurations which are specific to individual DNS providers. Example: ```yaml - name: my-name value: my-value - name: my-name2 value: my-value2 ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``name`` | The name of the key value pair. | ``string`` | Yes | |``value`` | The value of the key value pair. | ``string`` | Yes | ### VirtualServer.Policy The policy field references a [Policy resource](/nic/lts/configuration/policy-resource/before-you-begin.md) by its name and optional namespace. For example: ```yaml name: access-control ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``name`` | The name of a policy. If the policy doesn't exist or invalid, NGINX will respond with an error response with the `500` status code. | ``string`` | Yes | |``namespace`` | The namespace of a policy. If not specified, the namespace of the VirtualServer resource is used. | ``string`` | No | ### VirtualServer.Route The route defines rules for matching client requests to actions like passing a request to an upstream. For example: ```yaml path: /tea action: pass: tea ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``path`` | The path of the route. NGINX will match it against the URI of a request. Possible values are: a prefix ( ``/`` , ``/path`` ), an exact match ( ``=/exact/match`` ), a case insensitive regular expression ( ``~*^/Bar.*\.jpg`` ) or a case sensitive regular expression ( ``~^/foo.*\.jpg`` ). In the case of a prefix (must start with ``/`` ) or an exact match (must start with ``=`` ), the path must not include any whitespace characters, ``{`` , ``}`` or ``;``. In the case of the regex matches, all double quotes ``"`` must be escaped and the match can't end in an unescaped backslash ``\``. The path must be unique among the paths of all routes of the VirtualServer. Check the [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) directive for more information. | ``string`` | Yes | |``policies`` | A list of policies. The policies override the policies of the same type defined in the ``spec`` of the VirtualServer. See [Applying Policies](/nic/lts/configuration/policy-resource/using-policy.md#attach-policies-to-a-resource) for more details. | [[]policy](#virtualserverpolicy) | No | |``action`` | The default action to perform for a request. | [action](#action) | No | |``dos`` | A reference to a DosProtectedResource, setting this enables DOS protection of the VirtualServer route. | ``string`` | No | |``splits`` | The default splits configuration for traffic splitting. Must include at least 2 splits. | [[]split](#split) | No | |``matches`` | The matching rules for advanced content-based routing. Requires the default ``action`` or ``splits``. Unmatched requests will be handled by the default ``action`` or ``splits``. | [matches](#match) | No | |``route`` | The name of a VirtualServerRoute resource that defines this route. If the VirtualServerRoute belongs to a different namespace than the VirtualServer, you need to include the namespace. For example, ``tea-namespace/tea``. | ``string`` | No | |``routeSelector`` | The RouteSelector allows selecting VirtualServerRoute resources using label selectors instead of specifying them by name and namespace. | [routeSelector](#virtualserverrouterouteselector) | No | |``errorPages`` | The custom responses for error codes. NGINX will use those responses instead of returning the error responses from the upstream servers or the default responses generated by NGINX. A custom response can be a redirect or a canned response. For example, a redirect to another URL if an upstream server responded with a 404 status code. | [[]errorPage](#errorpage) | No | |``location-snippets`` | Sets a custom snippet in the location context. Overrides the ``location-snippets`` ConfigMap key. | ``string`` | No | \* -- a route must include exactly one of the following: `action`, `splits`, `route` or `routeSelector` ### VirtualServer.Route.RouteSelector The RouteSelector field allows you to dynamically select VirtualServerRoute resources using label selectors instead of specifying them by name and namespace. This provides a more flexible and scalable way to associate VirtualServerRoutes with a VirtualServer route. See the [VirtualServerRoute specification](#virtualserverroute-specification) section below for examples of how to use RouteSelector with VirtualServerRoute resources. |Field | Description | Type | Required | | ---| ---| ---| --- | |``matchLabels`` | A map of key-value pairs. Each key-value pair in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed. | ``map[string]string`` | Yes | |``matchExpressions`` | A list of label selector requirements. The requirements are ANDed. For more information on label selector requirements, see the [Kubernetes documentation](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#resources-that-support-set-based-requirements). | N/A | No | ## VirtualServerRoute specification The VirtualServerRoute resource defines a route for a VirtualServer. It can consist of one or multiple subroutes. The VirtualServerRoute is an alternative to [Mergeable Ingress types](/nic/lts/configuration/ingress-resources/cross-namespace-configuration.md). VirtualServer routes can reference VirtualServerRoute resources in two ways: by name using the `route` field, or dynamically using the `routeSelector` field with label selectors. The `routeSelector` approach allows you to add new VirtualServerRoute resources without modifying the VirtualServer configuration. #### Standard In this example, the VirtualServer `cafe` from the namespace `cafe-ns` defines a route that references a specific VirtualServerRoute by name. VirtualServer: ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServer metadata: name: cafe namespace: cafe-ns spec: host: cafe.example.com upstreams: - name: tea service: tea-svc port: 80 routes: - path: /tea action: pass: tea - path: /coffee route: coffee-ns/coffee ``` VirtualServerRoute: ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServerRoute metadata: name: coffee namespace: coffee-ns spec: host: cafe.example.com upstreams: - name: latte service: latte-svc port: 80 - name: espresso service: espresso-svc port: 80 subroutes: - path: /coffee/latte action: pass: latte - path: /coffee/espresso action: pass: espresso ``` #### RouteSelector In this example, the VirtualServer `cafe` from the namespace `cafe-ns` uses `routeSelector` to dynamically select any VirtualServerRoute with the label `app: coffee`. VirtualServer: ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServer metadata: name: cafe namespace: cafe-ns spec: host: cafe.example.com upstreams: - name: tea service: tea-svc port: 80 routes: - path: /tea action: pass: tea - path: /decaf routeSelector: matchLabels: app: coffee ``` VirtualServerRoute: ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServerRoute metadata: name: decaf namespace: coffee-ns labels: app: coffee spec: host: cafe.example.com upstreams: - name: latte service: latte-svc port: 80 - name: espresso service: espresso-svc port: 80 subroutes: - path: /decaf/latte action: pass: latte - path: /decaf/espresso action: pass: espresso ``` Note that each subroute must have a `path` that starts with the same prefix (here `/coffee`), which is defined in the route of the VirtualServer. Additionally, the `host` in the VirtualServerRoute must be the same as the `host` of the VirtualServer. |Field | Description | Type | Required | | ---| ---| ---| --- | |``host`` | The host (domain name) of the server. Must be a valid subdomain as defined in RFC 1123, such as ``my-app`` or ``hello.example.com``. When using a wildcard domain like ``*.example.com`` the domain must be contained in double quotes. Must be the same as the ``host`` of the VirtualServer that references this resource. | ``string`` | Yes | |``upstreams`` | A list of upstreams. | [[]upstream](#upstream) | No | |``subroutes`` | A list of subroutes. | [[]subroute](#virtualserverroutesubroute) | No | |``ingressClassName`` | Specifies which Ingress Controller must handle the VirtualServerRoute resource. Must be the same as the ``ingressClassName`` of the VirtualServer that references this resource. | ``string``_ | No | ### VirtualServerRoute.Subroute The subroute defines rules for matching client requests to actions like passing a request to an upstream. For example: ```yaml path: /coffee action: pass: coffee ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``path`` | The path of the subroute. NGINX will match it against the URI of a request. Possible values are: a prefix ( ``/`` , ``/path`` ), an exact match ( ``=/exact/match`` ), a case insensitive regular expression ( ``~*^/Bar.*\.jpg`` ) or a case sensitive regular expression ( ``~^/foo.*\.jpg`` ). In the case of a prefix, the path must start with the same path as the path of the route of the VirtualServer that references this resource. In the case of an exact or regex match, the path must be the same as the path of the route of the VirtualServer that references this resource. A matching path of the route of the VirtualServer but in different type is not accepted, e.g. a regex path (`~/match`) cannot be used with a prefix path in VirtualServer (`/match`) In the case of a prefix or an exact match, the path must not include any whitespace characters, ``{`` , ``}`` or ``;``. In the case of the regex matches, all double quotes ``"`` must be escaped and the match can't end in an unescaped backslash ``\``. The path must be unique among the paths of all subroutes of the VirtualServerRoute. | ``string`` | Yes | |``policies`` | A list of policies. The policies override *all* policies defined in the route of the VirtualServer that references this resource. The policies also override the policies of the same type defined in the ``spec`` of the VirtualServer. See [Applying Policies](/nic/lts/configuration/policy-resource/using-policy.md#attach-policies-to-a-resource) for more details. | [[]policy](#virtualserverpolicy) | No | |``action`` | The default action to perform for a request. | [action](#action) | No | |``dos`` | A reference to a DosProtectedResource, setting this enables DOS protection of the VirtualServerRoute subroute. | ``string`` | No | |``splits`` | The default splits configuration for traffic splitting. Must include at least 2 splits. | [[]split](#split) | No | |``matches`` | The matching rules for advanced content-based routing. Requires the default ``action`` or ``splits``. Unmatched requests will be handled by the default ``action`` or ``splits``. | [matches](#match) | No | |``errorPages`` | The custom responses for error codes. NGINX will use those responses instead of returning the error responses from the upstream servers or the default responses generated by NGINX. A custom response can be a redirect or a canned response. For example, a redirect to another URL if an upstream server responded with a 404 status code. | [[]errorPage](#errorpage) | No | |``location-snippets`` | Sets a custom snippet in the location context. Overrides the ``location-snippets`` of the VirtualServer (if set) or the ``location-snippets`` ConfigMap key. | ``string`` | No | \* -- a subroute must include exactly one of the following: `action` or `splits`. ## Common VirtualServer and VirtualServerRoute specifications ### Upstream The upstream defines a destination for the routing configuration. For example: ```yaml name: tea service: tea-svc subselector: version: canary port: 80 lb-method: round_robin fail-timeout: 10s max-fails: 1 max-conns: 32 keepalive: 32 connect-timeout: 30s read-timeout: 30s send-timeout: 30s next-upstream: "error timeout non_idempotent" next-upstream-timeout: 5s next-upstream-tries: 10 client-max-body-size: 2m tls: enable: true ``` **Note**: The WebSocket protocol is supported without any additional configuration. |Field | Description | Type | Required | | ---| ---| ---| --- | |``name`` | The name of the upstream. Must be a valid DNS label as defined in RFC 1035. For example, ``hello`` and ``upstream-123`` are valid. The name must be unique among all upstreams of the resource. | ``string`` | Yes | |``service`` | The name of a [service](https://kubernetes.io/docs/concepts/services-networking/service/). If the Service belongs to a different namespace than the VirtualServer or VirtualServerRoute, you need to include the namespace. For example, ``tea-namespace/tea``. If the service doesn't exist, NGINX will assume the service has zero endpoints and return a ``502`` response for requests for this upstream. Services of type [ExternalName](https://kubernetes.io/docs/concepts/services-networking/service/#externalname) are also supported in the same namespace (check the [prerequisites](https://github.com/nginx/kubernetes-ingress/tree/v/examples/ingress-resources/externalname-services#prerequisites) ). | ``string`` | Yes | |``subselector`` | Selects the pods within the service using label keys and values. By default, all pods of the service are selected. Note: the specified labels are expected to be present in the pods when they are created. If the pod labels are updated, NGINX Ingress Controller LTS will not see that change until the number of the pods is changed. | ``map[string]string`` | No | |``use-cluster-ip`` | Enables using the Cluster IP and port of the service instead of the default behavior of using the IP and port of the pods. When this field is enabled, the fields that configure NGINX behavior related to multiple upstream servers (like ``lb-method`` and ``next-upstream``) will have no effect, as NGINX Ingress Controller LTS will configure NGINX with only one upstream server that will match the service Cluster IP. | ``boolean`` | No | |``port`` | The port of the service. If the service doesn't define that port, NGINX will assume the service has zero endpoints and return a ``502`` response for requests for this upstream. The port must fall into the range ``1..65535``. | ``uint16`` | Yes | |``lb-method`` | The load [balancing method](/nginx/admin-guide/load-balancer/http-load-balancer.md#choosing-a-load-balancing-method). To use the round-robin method, specify ``round_robin``. The default is specified in the ``lb-method`` ConfigMap key. | ``string`` | No | |``fail-timeout`` | The time during which the specified number of unsuccessful attempts to communicate with an upstream server should happen to consider the server unavailable. See the [fail_timeout](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#fail_timeout) parameter of the server directive. The default is set in the ``fail-timeout`` ConfigMap key. | ``string`` | No | |``max-fails`` | The number of unsuccessful attempts to communicate with an upstream server that should happen in the duration set by the ``fail-timeout`` to consider the server unavailable. See the [max_fails](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#max_fails) parameter of the server directive. The default is set in the ``max-fails`` ConfigMap key. | ``int`` | No | |``max-conns`` | The maximum number of simultaneous active connections to an upstream server. See the [max_conns](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#max_conns) parameter of the server directive. By default there is no limit. Note: if keepalive connections are enabled, the total number of active and idle keepalive connections to an upstream server may exceed the ``max_conns`` value. | ``int`` | No | |``keepalive`` | Configures the cache for connections to upstream servers. The value ``0`` disables the cache. See the [keepalive](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive) directive. The default is set in the ``keepalive`` ConfigMap key. | ``int`` | No | |``connect-timeout`` | The timeout for establishing a connection with an upstream server. See the [proxy_connect_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_connect_timeout) directive. The default is specified in the ``proxy-connect-timeout`` ConfigMap key. | ``string`` | No | |``read-timeout`` | The timeout for reading a response from an upstream server. See the [proxy_read_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_read_timeout) directive. The default is specified in the ``proxy-read-timeout`` ConfigMap key. | ``string`` | No | |``send-timeout`` | The timeout for transmitting a request to an upstream server. See the [proxy_send_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_send_timeout) directive. The default is specified in the ``proxy-send-timeout`` ConfigMap key. | ``string`` | No | |``next-upstream`` | Specifies in which cases a request should be passed to the next upstream server. See the [proxy_next_upstream](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream) directive. The default is ``error timeout``. | ``string`` | No | |``next-upstream-timeout`` | The time during which a request can be passed to the next upstream server. See the [proxy_next_upstream_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream_timeout) directive. The ``0`` value turns off the time limit. The default is ``0``. | ``string`` | No | |``next-upstream-tries`` | The number of possible tries for passing a request to the next upstream server. See the [proxy_next_upstream_tries](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream_tries) directive. The ``0`` value turns off this limit. The default is ``0``. | ``int`` | No | |``client-max-body-size`` | Sets the maximum allowed size of the client request body. See the [client_max_body_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size) directive. The default is set in the ``client-max-body-size`` ConfigMap key. | ``string`` | No | |``client-body-buffer-size`` | Sets the value of the [client_body_buffer_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_buffer_size) directive. The default is set in the ``client-body-buffer-size`` ConfigMap key. | ``string`` | No | |``tls`` | The TLS configuration for the Upstream. | [tls](#upstreamtls) | No | |``healthCheck`` | The health check configuration for the Upstream. See the [health_check](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#health_check) directive. | [healthcheck](#upstreamhealthcheck) | No | |``slow-start`` | The slow start allows an upstream server to gradually recover its weight from 0 to its nominal value after it has been recovered or became available or when the server becomes available after a period of time it was considered unavailable. By default, the slow start is disabled. See the [slow_start](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#slow_start) parameter of the server directive. Note: The parameter cannot be used along with the ``random`` , ``hash`` or ``ip_hash`` load balancing methods and will be ignored. | ``string`` | No | |``queue`` | Configures a queue for an upstream. A client request will be placed into the queue if an upstream server cannot be selected immediately while processing the request. By default, no queue is configured. | [queue](#upstreamqueue) | No | |``buffering`` | Enables buffering of responses from the upstream server. See the [proxy_buffering](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffering) directive. The default is set in the ``proxy-buffering`` ConfigMap key. | ``boolean`` | No | |``buffers`` | Configures the buffers used for reading a response from the upstream server for a single connection. | [buffers](#upstreambuffers) | No | |``buffer-size`` | Sets the size of the buffer used for reading the first part of a response received from the upstream server. See the [proxy_buffer_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffer_size) directive. The default is set in the ``proxy-buffer-size`` ConfigMap key. | ``string`` | No | |``busy-buffers-size`` | Sets the size of the buffer used for reading a response from the upstream server when the response is larger than the ``buffer-size``. See the [proxy_busy_buffers_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_busy_buffers_size) directive. The default is set in the ``proxy-busy-buffers-size`` ConfigMap key. | ``string`` | No | |``ntlm`` | Allows proxying requests with NTLM Authentication. See the [ntlm](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#ntlm) directive. In order for NTLM authentication to work, it is necessary to enable keepalive connections to upstream servers using the ``keepalive`` field. | ``boolean`` | No | |``type`` |The type of the upstream. Supported values are ``http`` and ``grpc``. The default is ``http``. For gRPC, it is necessary to enable HTTP/2 in the [ConfigMap](/nic/lts/configuration/global-configuration/configmap-resource.md#listeners) and configure TLS termination in the VirtualServer. | ``string`` | No | |``backup`` | The name of the backup service of type [ExternalName](https://kubernetes.io/docs/concepts/services-networking/service/#externalname). This will be used when the primary servers are unavailable. Note: The parameter cannot be used along with the ``random`` , ``hash`` or ``ip_hash`` load balancing methods. | ``string`` | No | |``backupPort`` | The port of the backup service. The backup port is required if the backup service name is provided. The port must fall into the range ``1..65535``. | ``uint16`` | No | ### Upstream.Buffers The buffers field configures the buffers used for reading a response from the upstream server for a single connection: ```yaml number: 4 size: 8K ``` See the [proxy_buffers](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffers) directive for additional information. |Field | Description | Type | Required | | ---| ---| ---| --- | |``number`` | Configures the number of buffers. The default is set in the ``proxy-buffers`` ConfigMap key. | ``int`` | Yes | |``size`` | Configures the size of a buffer. The default is set in the ``proxy-buffers`` ConfigMap key. | ``string`` | Yes | ### Upstream.TLS |Field | Description | Type | Required | | ---| ---| ---| --- | |``enable`` | Enables HTTPS for requests to upstream servers. The default is ``False`` , meaning that HTTP will be used. Note: by default, NGINX will not verify the upstream server certificate. To enable the verification, configure an [EgressMTLS Policy](/nic/lts/configuration/policy-resource/policy-specification.md#egressmtls). | ``boolean`` | No | ### Upstream.Queue The queue field configures a queue. A client request will be placed into the queue if an upstream server cannot be selected immediately while processing the request: ```yaml size: 10 timeout: 60s ``` See [`queue`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#queue) directive for additional information. |Field | Description | Type | Required | | ---| ---| ---| --- | |``size`` | The size of the queue. | ``int`` | Yes | |``timeout`` | The timeout of the queue. A request cannot be queued for a period longer than the timeout. The default is ``60s``. | ``string`` | No | ### Upstream.Healthcheck The Healthcheck defines an [active health check](/nginx/admin-guide/load-balancer.md#http-health-check). In the example below we enable a health check for an upstream and configure all the available parameters, including the `slow-start` parameter combined with [`mandatory` and `persistent`](/nginx/admin-guide/load-balancer/http-health-check.md#mandatory-health-checks): ```yaml name: tea service: tea-svc port: 80 slow-start: 30s healthCheck: enable: true path: /healthz interval: 20s jitter: 3s fails: 5 passes: 5 port: 8080 tls: enable: true connect-timeout: 10s read-timeout: 10s send-timeout: 10s headers: - name: Host value: my.service statusMatch: "! 500" mandatory: true persistent: true keepalive-time: 60s ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``enable`` | Enables a health check for an upstream server. The default is ``false``. | ``boolean`` | No | |``path`` | The path used for health check requests. The default is ``/``. This not configurable for gRPC type upstreams. | ``string`` | No | |``interval`` | The interval between two consecutive health checks. The default is ``5s``. | ``string`` | No | |``jitter`` | The time within which each health check will be randomly delayed. By default, there is no delay. | ``string`` | No | |``fails`` | The number of consecutive failed health checks of a particular upstream server after which this server will be considered unhealthy. The default is ``1``. | ``integer`` | No | |``passes`` | The number of consecutive passed health checks of a particular upstream server after which the server will be considered healthy. The default is ``1``. | ``integer`` | No | |``port`` | The port used for health check requests. By default, the [server port is used](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#health_check_port). Note: in contrast with the port of the upstream, this port is not a service port, but a port of a pod. | ``integer`` | No | |``tls`` | The TLS configuration used for health check requests. By default, the ``tls`` field of the upstream is used. | [upstream.tls](#upstreamtls) | No | |``connect-timeout`` | The timeout for establishing a connection with an upstream server. By default, the ``connect-timeout`` of the upstream is used. | ``string`` | No | |``read-timeout`` | The timeout for reading a response from an upstream server. By default, the ``read-timeout`` of the upstream is used. | ``string`` | No | |``send-timeout`` | The timeout for transmitting a request to an upstream server. By default, the ``send-timeout`` of the upstream is used. | ``string`` | No | |``headers`` | The request headers used for health check requests. NGINX Plus always sets the ``Host`` , ``User-Agent`` and ``Connection`` headers for health check requests. | [[]header](#header) | No | |``statusMatch`` | The expected response status codes of a health check. By default, the response should have status code 2xx or 3xx. Examples: ``"200"`` , ``"! 500"`` , ``"301-303 307"``. See the documentation of the [match](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html?#match) directive. This not supported for gRPC type upstreams. | ``string`` | No | |``grpcStatus`` | The expected [gRPC status code](https://github.com/grpc/grpc/blob/master/doc/statuscodes.md#status-codes-and-their-use-in-grpc) of the upstream server response to the [Check method](https://github.com/grpc/grpc/blob/master/doc/health-checking.md). Configure this field only if your gRPC services do not implement the gRPC health checking protocol. For example, configure ``12`` if the upstream server responds with `12 (UNIMPLEMENTED)` status code. Only valid on gRPC type upstreams. | ``int`` | No | |``grpcService`` | The gRPC service to be monitored on the upstream server. Only valid on gRPC type upstreams. | ``string`` | No | |``mandatory`` | Require every newly added server to pass all configured health checks before NGINX Plus sends traffic to it. If this is not specified, or is set to false, the server will be initially considered healthy. When combined with [slow-start](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#slow_start), it gives a new server more time to connect to databases and “warm up” before being asked to handle their full share of traffic. | ``bool`` | No | |``persistent`` | Set the initial “up” state for a server after reload if the server was considered healthy before reload. Enabling persistent requires that the mandatory parameter is also set to `true`. | ``bool`` | No | |``keepalive-time`` | Enables [keepalive](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive) connections for health checks and specifies the time during which requests can be processed through one keepalive connection. The default is ``60s``. | ``string`` | No | ### Upstream.SessionCookie The SessionCookie field configures session persistence which allows requests from the same client to be passed to the same upstream server. The information about the designated upstream server is passed in a session cookie generated by NGINX. In the example below, we configure session persistence with a session cookie for an upstream and configure all the available parameters: ```yaml name: tea service: tea-svc port: 80 sessionCookie: enable: true name: srv_id path: / expires: 1h domain: .example.com httpOnly: false secure: true samesite: strict ``` See the [`sticky`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html?#sticky) directive for additional information. The session cookie corresponds to the `sticky cookie` method. |Field | Description | Type | Required | | ---| ---| ---| --- | |``enable`` | Enables session persistence with a session cookie for an upstream server. The default is ``false``. | ``boolean`` | No | |``name`` | The name of the cookie. | ``string`` | Yes | |``path`` | The path for which the cookie is set. | ``string`` | No | |``expires`` | The time for which a browser should keep the cookie. Can be set to the special value ``max`` , which will cause the cookie to expire on ``31 Dec 2037 23:55:55 GMT``. | ``string`` | No | |``domain`` | The domain for which the cookie is set. | ``string`` | No | |``httpOnly`` | Adds the ``HttpOnly`` attribute to the cookie. | ``boolean`` | No | |``secure`` | Adds the ``Secure`` attribute to the cookie. | ``boolean`` | No | |``samesite`` | Adds the ``SameSite`` attribute to the cookie. The allowed values are: ``strict``, ``lax``, ``none`` | ``string`` | No | ### Action The action defines an action to perform for a request. In the example below, client requests are passed to an upstream `coffee`: ```yaml path: /coffee action: pass: coffee ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``pass`` | Passes requests to an upstream. The upstream with that name must be defined in the resource. | ``string`` | No | |``redirect`` | Redirects requests to a provided URL. | [action.redirect](#actionredirect) | No | |``return`` | Returns a preconfigured response. | [action.return](#actionreturn) | No | |``proxy`` | Passes requests to an upstream with the ability to modify the request/response (for example, rewrite the URI or modify the headers). | [action.proxy](#actionproxy) | No | \* -- an action must include exactly one of the following: `pass`, `redirect`, `return` or `proxy`. ### Action.Redirect The redirect action defines a redirect to return for a request. In the example below, client requests are passed to a url `http://www.nginx.com`: ```yaml redirect: url: http://www.nginx.com code: 301 ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``url`` | The URL to redirect the request to. Supported NGINX variables: ``$scheme`` , ``$http_x_forwarded_proto`` , ``$request_uri`` , ``$host``. Variables must be enclosed in curly braces. For example: ``${host}${request_uri}``. | ``string`` | Yes | |``code`` | The status code of a redirect. The allowed values are: ``301`` , ``302`` , ``307`` , ``308``. The default is ``301``. | ``int`` | No | ### Action.Return The return action defines a preconfigured response for a request. In the example below, NGINX will respond with the preconfigured response for every request: ```yaml return: code: 200 type: text/plain body: "Hello World\n" headers: - name: x-coffee value: espresso ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``code`` | The status code of the response. The allowed values are: ``2XX``, ``4XX`` or ``5XX``. The default is ``200``. | ``int`` | No | |``type`` | The MIME type of the response. The default is ``text/plain``. | ``string`` | No | |``body`` | The body of the response. Supports NGINX variables*. Variables must be enclosed in curly brackets. For example: ``Request is ${request_uri}\n``. | ``string`` | Yes | |``headers`` | The custom headers of the response. | [[]Action.Return.Header](#actionreturnheader) | No | \* -- Supported NGINX variables: `$request_uri`, `$request_method`, `$request_body`, `$scheme`, `$http_`, `$args`, `$arg_`, `$cookie_`, `$host`, `$request_time`, `$request_length`, `$nginx_version`, `$pid`, `$connection`, `$remote_addr`, `$remote_port`, `$time_iso8601`, `$time_local`, `$server_addr`, `$server_port`, `$server_name`, `$server_protocol`, `$connections_active`, `$connections_reading`, `$connections_writing` and `$connections_waiting`. ### Action.Return.Header The header defines an HTTP Header for a canned response in an actionReturn: ```yaml name: x-coffee value: espresso ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``name`` | The name of the header. | ``string`` | Yes | |``value`` | The value of the header. | ``string`` | Yes | ### Action.Proxy The proxy action passes requests to an upstream with the ability to modify the request/response (for example, rewrite the URI or modify the headers). In the example below, the request URI is rewritten to `/`, and the request and the response headers are modified: ```yaml proxy: upstream: coffee requestHeaders: pass: true set: - name: My-Header value: Value - name: Client-Cert value: ${ssl_client_escaped_cert} responseHeaders: add: - name: My-Header value: Value - name: IC-Nginx-Version value: ${nginx_version} always: true hide: - x-internal-version ignore: - Expires - Set-Cookie pass: - Server rewritePath: / ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``upstream`` | The name of the upstream which the requests will be proxied to. The upstream with that name must be defined in the resource. | ``string`` | Yes | |``requestHeaders`` | The request headers modifications. | [action.Proxy.RequestHeaders](#actionproxyrequestheaders) | No | |``responseHeaders`` | The response headers modifications. | [action.Proxy.ResponseHeaders](#actionproxyresponseheaders) | No | |``rewritePath`` | The rewritten URI. If the route path is a regular expression -- starts with `~` -- the `rewritePath` can include capture groups with ``$1-9``. For example `$1` for the first group, and so on. For more information, check the [rewrite](https://github.com/nginx/kubernetes-ingress/tree/v/examples/custom-resources/rewrites) example. | ``string`` | No | ### Action.Proxy.RequestHeaders The RequestHeaders field modifies the headers of the request to the proxied upstream server. |Field | Description | Type | Required | | ---| ---| ---| --- | |``pass`` | Passes the original request headers to the proxied upstream server. See the [proxy_pass_request_header](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass_request_headers) directive for more information. Default is true. | ``bool`` | No | |``set`` | Allows redefining or appending fields to present request headers passed to the proxied upstream servers. See the [proxy_set_header](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_header) directive for more information. | [[]header](#actionproxyrequestheaderssetheader) | No | ### Action.Proxy.RequestHeaders.Set.Header The header defines an HTTP Header: ```yaml name: My-Header value: My-Value ``` It is possible to override the default value of the `Host` header, which NGINX Ingress Controller LTS sets to [`$host`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_host): ```yaml name: Host value: example.com ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``name`` | The name of the header. | ``string`` | Yes | |``value`` | The value of the header. Supports NGINX variables*. Variables must be enclosed in curly brackets. For example: ``${scheme}``. | ``string`` | No | \* -- Supported NGINX variables: `$request_uri`, `$request_method`, `$request_body`, `$scheme`, `$http_`, `$args`, `$arg_`, `$cookie_`, `$host`, `$request_time`, `$request_length`, `$nginx_version`, `$pid`, `$connection`, `$remote_addr`, `$remote_port`, `$time_iso8601`, `$time_local`, `$server_addr`, `$server_port`, `$server_name`, `$server_protocol`, `$connections_active`, `$connections_reading`, `$connections_writing`, `$connections_waiting`, `$ssl_cipher`, `$ssl_ciphers`, `$ssl_client_cert`, `$ssl_client_escaped_cert`, `$ssl_client_fingerprint`, `$ssl_client_i_dn`, `$ssl_client_i_dn_legacy`, `$ssl_client_raw_cert`, `$ssl_client_s_dn`, `$ssl_client_s_dn_legacy`, `$ssl_client_serial`, `$ssl_client_v_end`, `$ssl_client_v_remain`, `$ssl_client_v_start`, `$ssl_client_verify`, `$ssl_curves`, `$ssl_early_data`, `$ssl_protocol`, `$ssl_server_name`, `$ssl_session_id`, `$ssl_session_reused`, `$jwt_claim_` and `$jwt_header_`. ### Action.Proxy.ResponseHeaders The ResponseHeaders field modifies the headers of the response to the client. |Field | Description | Type | Required | | ---| ---| ---| --- | |``hide`` | The headers that will not be passed* in the response to the client from a proxied upstream server. See the [proxy_hide_header](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_hide_header) directive for more information. | ``[]string`` | No | |``pass`` | Allows passing the hidden header fields* to the client from a proxied upstream server. See the [proxy_pass_header](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass_header) directive for more information. | ``[]string`` | No | |``ignore`` | Disables processing of certain headers** to the client from a proxied upstream server. See the [proxy_ignore_headers](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ignore_headers) directive for more information. | ``[]string`` | No | |``add`` | Adds headers to the response to the client. | [[]addHeader](#addheader) | No | \* -- Default hidden headers are: `Date`, `Server`, `X-Pad` and `X-Accel-...`. \** -- The following fields can be ignored: `X-Accel-Redirect`, `X-Accel-Expires`, `X-Accel-Limit-Rate`, `X-Accel-Buffering`, `X-Accel-Charset`, `Expires`, `Cache-Control`, `Set-Cookie` and `Vary`. ### AddHeader The addHeader defines an HTTP Header with an optional `always` field: ```yaml name: My-Header value: My-Value always: true ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``name`` | The name of the header. | ``string`` | Yes | |``value`` | The value of the header. Supports NGINX variables*. Variables must be enclosed in curly brackets. For example: ``${scheme}``. | ``string`` | No | |``always`` | If set to true, add the header regardless of the response status code**. Default is false. See the [add_header](http://nginx.org/en/docs/http/ngx_http_headers_module.html#add_header) directive for more information. | ``bool`` | No | \* -- Supported NGINX variables: `$request_uri`, `$request_method`, `$request_body`, `$scheme`, `$http_`, `$args`, `$arg_`, `$cookie_`, `$host`, `$request_time`, `$request_length`, `$nginx_version`, `$pid`, `$connection`, `$remote_addr`, `$remote_port`, `$time_iso8601`, `$time_local`, `$server_addr`, `$server_port`, `$server_name`, `$server_protocol`, `$connections_active`, `$connections_reading`, `$connections_writing`, `$connections_waiting`, `$ssl_cipher`, `$ssl_ciphers`, `$ssl_client_cert`, `$ssl_client_escaped_cert`, `$ssl_client_fingerprint`, `$ssl_client_i_dn`, `$ssl_client_i_dn_legacy`, `$ssl_client_raw_cert`, `$ssl_client_s_dn`, `$ssl_client_s_dn_legacy`, `$ssl_client_serial`, `$ssl_client_v_end`, `$ssl_client_v_remain`, `$ssl_client_v_start`, `$ssl_client_verify`, `$ssl_curves`, `$ssl_early_data`, `$ssl_protocol`, `$ssl_server_name`, `$ssl_session_id`, `$ssl_session_reused`, `$jwt_claim_` and `$jwt_header_`. **note:** If `always` is false, the response header is added only if the response status code is any of `200`, `201`, `204`, `206`, `301`, `302`, `303`, `304`, `307` or `308`. ### Split The split defines a weight for an action as part of the splits configuration. In the example below NGINX passes 80% of requests to the upstream `coffee-v1` and the remaining 20% to `coffee-v2`: ```yaml splits: - weight: 80 action: pass: coffee-v1 - weight: 20 action: pass: coffee-v2 ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``weight`` | The weight of an action. Must fall into the range ``0..100``. The sum of the weights of all splits must be equal to ``100``. | ``int`` | Yes | |``action`` | The action to perform for a request. | [action](#action) | Yes | ### Match The match defines a match between conditions and an action or splits. In the example below, NGINX routes requests with the path `/coffee` to different upstreams based on the value of the cookie `user`: - `user=john` -> `coffee-future` - `user=bob` -> `coffee-deprecated` - If the cookie is not set or not equal to either `john` or `bob`, NGINX routes to `coffee-stable` ```yaml path: /coffee matches: - conditions: - cookie: user value: john action: pass: coffee-future - conditions: - cookie: user value: bob action: pass: coffee-deprecated action: pass: coffee-stable ``` In the next example, NGINX routes requests based on the value of the built-in [`$request_method` variable](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_request_method), which represents the HTTP method of a request: - all POST requests -> `coffee-post` - all non-POST requests -> `coffee` ```yaml path: /coffee matches: - conditions: - variable: $request_method value: POST action: pass: coffee-post action: pass: coffee ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``conditions`` | A list of conditions. Must include at least 1 condition. | [[]condition](#condition) | Yes | |``action`` | The action to perform for a request. | [action](#action) | No | |``splits`` | The splits configuration for traffic splitting. Must include at least 2 splits. | [[]split](#split) | No | **note:** A match must include exactly one of the following: `action` or `splits`. ### Condition The condition defines a condition in a match. |Field | Description | Type | Required | | ---| ---| ---| --- | |``header`` | The name of a header. Must consist of alphanumeric characters or ``-``. | ``string`` | No | |``cookie`` | The name of a cookie. Must consist of alphanumeric characters or ``_``. | ``string`` | No | |``argument`` | The name of an argument. Must consist of alphanumeric characters or ``_``. | ``string`` | No | |``variable`` | The name of an NGINX variable. Must start with ``$``. See the list of the supported variables below the table. | ``string`` | No | |``value`` | The value to match the condition against. How to define a value is shown below the table. | ``string`` | Yes | **note:** a condition must include exactly one of the following: `header`, `cookie`, `argument` or `variable`. Supported NGINX variables: `$args`, `$http2`, `$https`, `$remote_addr`, `$remote_port`, `$query_string`, `$request`, `$request_body`, `$request_uri`, `$request_method`, `$scheme`. Find the documentation for each variable [here](https://nginx.org/en/docs/varindex.html). The value supports two kinds of matching: - *Case-insensitive string comparison*. For example: - `john` -- case-insensitive matching that succeeds for strings, such as `john`, `John`, `JOHN`. - `!john` -- negation of the case-insensitive matching for john that succeeds for strings, such as `bob`, `anything`, `''` (empty string). - *Matching with a regular expression*. Note that NGINX supports regular expressions compatible with those used by the Perl programming language (PCRE). For example: - `~^yes` -- a case-sensitive regular expression that matches any string that starts with `yes`. For example: `yes`, `yes123`. - `!~^yes` -- negation of the previous regular expression that succeeds for strings like `YES`, `Yes123`, `noyes`. (The negation mechanism is not part of the PCRE syntax). - `~*no$` -- a case-insensitive regular expression that matches any string that ends with `no`. For example: `no`, `123no`, `123NO`. **note:** A value must not include any unescaped double quotes (`"`) and must not end with an unescaped backslash (`\`). For example, the following are invalid values: `some"value`, `somevalue\`. ### ErrorPage The errorPage defines a custom response for a route for the case when either an upstream server responds with (or NGINX generates) an error status code. The custom response can be a redirect or a canned response. See the [error_page](https://nginx.org/en/docs/http/ngx_http_core_module.html#error_page) directive for more information. ```yaml path: /coffee errorPages: - codes: [502, 503] redirect: code: 301 url: https://nginx.org - codes: [404] return: code: 200 body: "Original resource not found, but success!" ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``codes`` | A list of error status codes. | ``[]int`` | Yes | |``redirect`` | The redirect action for the given status codes. | [errorPage.Redirect](#errorpageredirect) | No | |``return`` | The canned response action for the given status codes. | [errorPage.Return](#errorpagereturn) | No | **note:** An errorPage must include exactly one of the following: `return` or `redirect`. ### ErrorPage.Redirect The redirect defines a redirect for an errorPage. In the example below, NGINX responds with a redirect when a response from an upstream server has a 404 status code. ```yaml codes: [404] redirect: code: 301 url: ${scheme}://cafe.example.com/error.html ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``code`` | The status code of a redirect. The allowed values are: ``301`` , ``302`` , ``307`` , ``308``. The default is ``301``. | ``int`` | No | |``url`` | The URL to redirect the request to. Supported NGINX variables: ``$scheme`` and ``$http_x_forwarded_proto``. Variables must be enclosed in curly braces. For example: ``${scheme}``. | ``string`` | Yes | ### ErrorPage.Return The return defines a canned response for an errorPage. In the example below, NGINX responds with a canned response when a response from an upstream server has either 401 or 403 status code. ```yaml codes: [401, 403] return: code: 200 type: application/json body: | {\"msg\": \"You don't have permission to do this\"} headers: - name: x-debug-original-statuses value: ${upstream_status} ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``code`` | The status code of the response. The default is the status code of the original response. | ``int`` | No | |``type`` | The MIME type of the response. The default is ``text/html``. | ``string`` | No | |``body`` | The body of the response. Supported NGINX variable: ``$upstream_status`` . Variables must be enclosed in curly braces. For example: ``${upstream_status}``. | ``string`` | Yes | |``headers`` | The custom headers of the response. | [[]errorPage.Return.Header](#errorpagereturnheader) | No | ### ErrorPage.Return.Header The header defines an HTTP Header for a canned response in an errorPage: ```yaml name: x-debug-original-statuses value: ${upstream_status} ``` |Field | Description | Type | Required | | ---| ---| ---| --- | |``name`` | The name of the header. | ``string`` | Yes | |``value`` | The value of the header. Supported NGINX variable: ``$upstream_status`` . Variables must be enclosed in curly braces. For example: ``${upstream_status}``. | ``string`` | No | ## Using VirtualServer and VirtualServerRoute You can use the usual `kubectl` commands to work with VirtualServer and VirtualServerRoute resources, similar to Ingress resources. For example, the following command creates a VirtualServer resource defined in `cafe-virtual-server.yaml` with the name `cafe`: ```shell kubectl apply -f cafe-virtual-server.yaml ``` ```text virtualserver.k8s.nginx.org "cafe" created ``` You can get the resource by running: ```shell kubectl get virtualserver cafe ``` ```text NAME STATE HOST IP PORTS AGE cafe Valid cafe.example.com 12.13.23.123 [80,443] 3m ``` In `kubectl get` and similar commands, you can use the short name `vs` instead of `virtualserver`. Similarly, for VirtualServerRoute you can use `virtualserverroute` or the short name `vsr`. ### Using Snippets Snippets allow you to insert raw NGINX config into different contexts of NGINX configuration. In the example below, we use snippets to configure several NGINX features in a VirtualServer: ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServer metadata: name: cafe namespace: cafe spec: http-snippets: | limit_req_zone $binary_remote_addr zone=mylimit:10m rate=1r/s; proxy_cache_path /tmp keys_zone=one:10m; host: cafe.example.com tls: secret: cafe-secret server-snippets: | limit_req zone=mylimit burst=20; upstreams: - name: tea service: tea-svc port: 80 - name: coffee service: coffee-svc port: 80 routes: - path: /tea location-snippets: | proxy_cache one; proxy_cache_valid 200 10m; action: pass: tea - path: /coffee action: pass: coffee ``` For additional information, view the [Advanced configuration with Snippets](/nic/lts/configuration/ingress-resources/advanced-configuration-with-snippets.md) topic. ### Validation Two types of validation are available for VirtualServer and VirtualServerRoute resources: - *Structural validation* by the `kubectl` and Kubernetes API server. - *Comprehensive validation* by NGINX Ingress Controller LTS. #### Structural Validation The custom resource definitions for VirtualServer and VirtualServerRoute include structural OpenAPI schema which describes the type of every field of those resources. If you try to create (or update) a resource that violates the structural schema (for example, you use a string value for the port field of an upstream), `kubectl` and Kubernetes API server will reject such a resource: - Example of `kubectl` validation: ```shell kubectl apply -f cafe-virtual-server.yaml ``` ```text error: error validating "cafe-virtual-server.yaml": error validating data: ValidationError(VirtualServer.spec.upstreams[0].port): invalid type for org.nginx.k8s.v1.VirtualServer.spec.upstreams.port: got "string", expected "integer"; if you choose to ignore these errors, turn validation off with --validate=false ``` - Example of Kubernetes API server validation: ```shell kubectl apply -f cafe-virtual-server.yaml --validate=false ``` ```text The VirtualServer "cafe" is invalid: []: Invalid value: map[string]interface {}{ ... }: validation failure list: spec.upstreams.port in body must be of type integer: "string" ``` If a resource is not rejected (it doesn't violate the structural schema), NGINX Ingress Controller LTS will validate it further. #### Comprehensive Validation NGINX Ingress Controller LTS validates the fields of the VirtualServer and VirtualServerRoute resources. If a resource is invalid, NGINX Ingress Controller LTS will reject it: the resource will continue to exist in the cluster, but NGINX Ingress Controller LTS will ignore it. You can check if NGINX Ingress Controller LTS successfully applied the configuration for a VirtualServer. For our example `cafe` VirtualServer, we can run: ```shell kubectl describe vs cafe ``` ```text ... Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal AddedOrUpdated 16s nginx-ingress-controller Configuration for default/cafe was added or updated ``` Note how the events section includes a Normal event with the AddedOrUpdated reason that informs us that the configuration was successfully applied. If you create an invalid resource, NGINX Ingress Controller LTS will reject it and emit a Rejected event. For example, if you create a VirtualServer `cafe` with two upstream with the same name `tea`, you will get: ```shell kubectl describe vs cafe ``` ```text ... Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning Rejected 12s nginx-ingress-controller VirtualServer default/cafe is invalid and was rejected: spec.upstreams[1].name: Duplicate value: "tea" ``` Note how the events section includes a Warning event with the Rejected reason. Additionally, this information is also available in the `status` field of the VirtualServer resource. Note the Status section of the VirtualServer: ```shell kubectl describe vs cafe ``` ```text ... Status: External Endpoints: Ip: 12.13.23.123 Ports: [80,443] Message: VirtualServer default/cafe is invalid and was rejected: spec.upstreams[1].name: Duplicate value: "tea" Reason: Rejected State: Invalid ``` NGINX Ingress Controller LTS validates VirtualServerRoute resources in a similar way. **Note**: If you make an existing resource invalid, NGINX Ingress Controller LTS will reject it and remove the corresponding configuration from NGINX. ## Customization using ConfigMap You can customize the NGINX configuration for VirtualServer and VirtualServerRoutes resources using the [ConfigMap](/nic/lts/configuration/global-configuration/configmap-resource.md). Most of the ConfigMap keys are supported, with the following exceptions: - `proxy-hide-headers` - `proxy-pass-headers` - `hsts` - `hsts-max-age` - `hsts-include-subdomains` - `hsts-behind-proxy` - `redirect-to-https` - `ssl-redirect` - `http-redirect-code` -------------------------------------------------------------------------------- title: "Path matching" url: https://docs.nginx.com/nginx-ingress-controller/configuration/path-matching/index.md -------------------------------------------------------------------------------- This document describes how F5 NGINX Ingress Controller translates VirtualServer and VirtualServerRoute route paths into NGINX [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) directives, with a focus on the longest prefix match (`^~`) modifier. ## Overview Each route in a VirtualServer or subroute in a VirtualServerRoute has a `path` field that maps directly to an NGINX `location` directive. NGINX Ingress Controller supports five path matching types, each with different syntax and priority behavior. Understanding these types, especially the longest prefix match, allows you to control exactly which location block handles a given request. ## Matching types and priority The following table lists all five supported path types in order of priority from highest to lowest: | Priority | Path type | Syntax | NGINX directive | Description | | --- | --- | --- | --- | --- | | 1 (highest) | Exact match | `=/path` | `location = /path` | Matches only the exact URI. Stops all further searching immediately. | | 2 | Longest prefix match | `^~/path` | `location ^~ /path` | Matches by prefix and stops NGINX from evaluating any regex locations. | | 3 | Regex (case-sensitive) | `~ pattern` | `location ~ "pattern"` | Matches the URI against a case-sensitive regular expression. | | 3 | Regex (case-insensitive) | `~* pattern` | `location ~* "pattern"` | Matches the URI against a case-insensitive regular expression. | | 4 (lowest) | Prefix match | `/path` | `location /path` | Matches by prefix, but can be overridden by a regex match. | When NGINX receives a request, it evaluates locations in the following order: 1. **Exact match (`=`)**: If the URI matches an exact location, NGINX uses it immediately and stops searching. 2. **Prefix scan**: NGINX scans all prefix locations (both `^~` and plain `/`) and remembers the longest match. 3. **Longest prefix match (`^~`)**: If the longest matching prefix location has the `^~` modifier, NGINX uses it immediately and skips all regex evaluation. 4. **Regex evaluation (`~` and `~*`)**: NGINX checks regex locations in the order they appear in the configuration. The first matching regex wins. 5. **Prefix fallback**: If no regex matches, NGINX uses the longest matching prefix location from step 2. ## Longest prefix match (`^~`) The `^~` modifier gives a prefix location higher priority than regex locations. When a request URI matches a `^~` location, NGINX uses that location immediately and does not evaluate any regular expression locations, even if a regex would also match. This is useful when you want to serve a specific path prefix with certainty, regardless of any regex patterns that might also match. ### When to use longest prefix match Use `^~` when: - You need to guarantee that a specific path prefix is always handled by a particular upstream or action, even if regex locations exist that could match the same URI. - You are serving static assets from a known path (such as `/images/static/` or `/assets/`) and want to prevent regex patterns (such as `~ \.jpg$`) from intercepting those requests. ### Path syntax rules A longest prefix match path must follow these rules: - The path must start with `^~` followed by a valid URI path starting with `/`. - Optional whitespace between `^~` and the path is allowed (for example, `^~ /images/` and `^~/images/` are equivalent). - The URI portion must not contain whitespace, `{`, `}`, `;`, or `\` characters. - The URI portion must not contain `..` path segments. ### How it differs from a regular prefix match | Behavior | Prefix (`/path`) | Longest prefix (`^~/path`) | | --- | --- | --- | | Matches by URI prefix | Yes | Yes | | Can be overridden by regex | Yes | No | | Supports `proxy.rewritePath` | Yes | Yes | | Supports subroute delegation | Yes | Yes | The only difference is that `^~` prevents regex locations from overriding the match. ## VirtualServer example The following VirtualServer defines routes using several path types, including the longest prefix match: ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServer metadata: name: path-matching spec: host: path-matching.example.com upstreams: - name: static-backend service: static-svc port: 80 - name: images-backend service: images-svc port: 80 - name: app-backend service: app-svc port: 80 routes: - path: "=/images/logo.jpg" action: pass: static-backend - path: "^~/images/static/" action: pass: static-backend - path: "~ \\.jpg$" action: pass: images-backend - path: /images/ action: pass: app-backend ``` This configuration produces the following NGINX location blocks: ```nginx location = /images/logo.jpg { # Exact match - highest priority ... } location ^~ /images/static/ { # Longest prefix match - blocks regex evaluation ... } location ~ "\.jpg$" { # Case-sensitive regex ... } location /images/ { # Regular prefix - lowest priority ... } ``` With this configuration: | Request URI | Matching location | Reason | | --- | --- | --- | | `/images/logo.jpg` | `= /images/logo.jpg` | Exact match has highest priority. | | `/images/static/photo.jpg` | `^~ /images/static/` | Longest prefix match blocks regex (`~ \.jpg$`) from being evaluated. | | `/images/photo.jpg` | `~ "\.jpg$"` | Regular prefix `/images/` matches, but regex takes priority over plain prefix. | | `/images/photo.gif` | `/images/` | No regex matches `.gif`, so the prefix fallback is used. | ## VirtualServerRoute subroutes with longest prefix match When a VirtualServer route delegates to a VirtualServerRoute using the `route` or `routeSelector` field, the parent route's path acts as a constraint on subroute paths. For a `^~` parent path, each subroute must also use the `^~` modifier. ### VirtualServer with delegation ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServer metadata: name: path-matching-vsr spec: host: path-matching-vsr.example.com routes: - path: "^~/static/" route: static-routes ``` ### VirtualServerRoute with `^~` subroutes ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServerRoute metadata: name: static-routes spec: host: path-matching-vsr.example.com subroutes: - path: "^~/static/css/" action: pass: css-backend - path: "^~/static/js/" action: pass: js-backend ``` This produces two NGINX location blocks: ```nginx location ^~ /static/css/ { ... } location ^~ /static/js/ { ... } ``` **important:** The parent route's path (`^~/static/`) does not create an NGINX location block. Only the subroute paths become NGINX locations. If a request matches the parent prefix but not any subroute (for example, `/static/fonts/bold.woff`), no location handles it and NGINX returns a 404 response. ## Subroute path constraints When a VirtualServer route delegates to a VirtualServerRoute, the subroute paths must follow constraints based on the parent route's path type: | Parent path type | Subroute requirement | Multiple subroutes | Example | | --- | --- | --- | --- | | Prefix (`/path`) | Each subroute must start with the parent path and use the prefix type. | Yes | Parent: `/images/`, subroutes: `/images/thumbnails/`, `/images/originals/` | | Longest prefix (`^~/path`) | Each subroute must start with the parent path string including the `^~` modifier. | Yes | Parent: `^~/static/`, subroutes: `^~/static/css/`, `^~/static/js/` | | Regex (`~/pattern` or `~*/pattern`) | The subroute must have the exact same path as the parent. | No (one only) | Parent: `~ \.jpg$`, subroute: `~ \.jpg$` | | Exact (`=/path`) | The subroute must have the exact same path as the parent. | No (one only) | Parent: `=/exact`, subroute: `=/exact` | For longest prefix parent paths, a plain prefix subroute like `/static/css/` is rejected because the validation requires that the subroute path string starts with `^~/static/`. The `^~` modifier must be present on every subroute. ## See also - [VirtualServer and VirtualServerRoute resources](/nic/configuration/virtualserver-and-virtualserverroute-resources.md) for the full CRD field reference. - NGINX [location directive documentation](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) for the upstream NGINX matching algorithm. - NGINX [location priority](/nginx/admin-guide/web-server/web-server.md#nginx-location-priority) for a detailed explanation of NGINX's location evaluation order. - [Ingress path matching using path-regex annotation](/nic/tutorials/ingress-path-regex-annotation.md) for configuring regex path matching with Ingress resources. - [Path matching examples](https://github.com/nginx/kubernetes-ingress/tree/v/examples/custom-resources/path-matching) on GitHub for runnable examples of all five path types. -------------------------------------------------------------------------------- title: "Host and Listener collisions" url: https://docs.nginx.com/nginx-ingress-controller/configuration/host-and-listener-collisions/index.md -------------------------------------------------------------------------------- This document explains how F5 NGINX Ingress Controller handles host and listener collisions between resources. --- ## Winner Selection Algorithm If multiple resources contend for the same host or listener, NGINX Ingress Controller will pick the winner based on the `creationTimestamp` of the resources: the oldest resource will win. In case there are more than one oldest resource (their `creationTimestamp` is the same), NGINX Ingress Controller will choose the resource with the lexicographically smallest `uid`. **Note:** The `creationTimestamp` and `uid` fields are part of the [ObjectMeta](https://kubernetes.io/docs/reference/kubernetes-api/common-definitions/object-meta/) resource. --- ## Host collisions A host collision occurs when multiple Ingress, VirtualServer, and TransportServer (configured for TLS Passthrough) resources configure the same `host`. When [`-allow-empty-ingress-host`](/nic/configuration/global-configuration/command-line-arguments.md#cmdoption-allow-empty-ingress-host) is enabled, multiple Ingress resources without a `host` also constitute a host collision for the default server slot. NGINX Ingress Controller has two strategies for handling host collisions: - Choosing a single "winner" resource to handle the host. - Merging the configuration of the conflicting resources. --- ### Choosing the winner Consider the following two resources: - `cafe-ingress` Ingress: ```yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: cafe-ingress spec: ingressClassName: nginx rules: - host: cafe.example.com . . . ``` - `cafe-virtual-server` VirtualServer: ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServer metadata: name: cafe-virtual-server spec: host: cafe.example.com . . . ``` If a user creates both resources in the cluster, a host collision will occur. NGINX Ingress Controller will pick the winner using the [winner selection algorithm](#winner-selection-algorithm). If `cafe-virtual-server` was created first, it will win the host `cafe.example.com` and NGINX Ingress Controller will reject `cafe-ingress`. This will be reflected in the events and in the resource's status field: ```shell kubectl describe vs cafe-virtual-server ``` ```text ... Status: ... Message: Configuration for default/cafe-virtual-server was added or updated Reason: AddedOrUpdated State: Valid Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal AddedOrUpdated 9s nginx-ingress-controller Configuration for default/cafe-virtual-server was added or updated ``` ```shell kubectl describe ingress cafe-ingress ``` ```text Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning Rejected 66s nginx-ingress-controller All hosts are taken by other resources ``` Similarly, if `cafe-ingress` was created first, it will win `cafe.example.com` and NGINX Ingress Controller will reject `cafe-virtual-server`. **Note:** You can configure multiple hosts for Ingress resources, and its possible that an Ingress resource can be the winner for some of its hosts and a loser for the others. For example, if `cafe-ingress` had an additional rule host rule for `pub.example.com`, NGINX Ingress Controller would not reject the Ingress. Instead, it would allow `cafe-ingress` to handle `pub.example.com`. --- ### Merging configuration for the same host It is possible to merge configuration for multiple Ingress resources for the same host. One common use case for this approach is distributing resources across multiple namespaces. The [Cross-namespace configuration](/nic/configuration/ingress-resources/cross-namespace-configuration.md) topic has more information. It is *not* possible to merge the configurations for multiple VirtualServer resources for the same host. However, you can split the VirtualServers into multiple VirtualServerRoute resources, which a single VirtualServer can then reference. See the [corresponding example](https://github.com/nginx/kubernetes-ingress/tree/v/examples/custom-resources/cross-namespace-configuration) on GitHub. It is *not* possible to merge configuration for multiple TransportServer resources. --- ## Listener/Host collisions Listener/Host collisions occur when multiple TransportServer resources (configured for TCP/UDP load balancing) specify the same combination of `spec.listener.name` and `spec.host`. The combination of `spec.listener.name` and `spec.host` must be unique among all TransportServer resources. If two TransportServer resources specify the same spec.listener.name and spec.host, one of them will be rejected to prevent conflicts. In the case where spec.host is not specified, it is considered an empty string. NGINX Ingress Controller will choose the winner, which will own that listener and host combination. --- ### Choosing the winner Consider the following two resources: - `tcp-1` TransportServer: ```yaml apiVersion: k8s.nginx.org/v1 kind: TransportServer metadata: name: tcp-1 spec: host: dns.example.com listener: name: dns-tcp protocol: TCP . . . ``` - `tcp-2` TransportServer: ```yaml apiVersion: k8s.nginx.org/v1 kind: TransportServer metadata: name: tcp-2 spec: host: dns.example.com listener: name: dns-tcp protocol: TCP . . . ``` If a user creates both resources in the cluster, a listener collision will occur. As a result, NGINX Ingress Controller will pick the winner using the [winner selection algorithm](#winner-selection-algorithm). In our example, if `tcp-1` was created first, it will win the listener `dns-tcp` and NGINX Ingress Controller will reject `tcp-2`. This will be reflected in the events and in the resource's status field: ```shell kubectl describe ts tcp-2 ``` ```text ... Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning Rejected 10s nginx-ingress-controller Listener dns-tcp is taken by another resource ``` Similarly, if `tcp-2` was created first, it will win `dns-tcp` and NGINX Ingress Controller will reject `tcp-1`. -------------------------------------------------------------------------------- title: "Host and Listener collisions" url: https://docs.nginx.com/nginx-ingress-controller/lts/configuration/host-and-listener-collisions/index.md -------------------------------------------------------------------------------- This document explains how F5 NGINX Ingress Controller LTS handles host and listener collisions between resources. --- ## Winner Selection Algorithm If multiple resources contend for the same host or listener, NGINX Ingress Controller LTS will pick the winner based on the `creationTimestamp` of the resources: the oldest resource will win. In case there are more than one oldest resource (their `creationTimestamp` is the same), NGINX Ingress Controller LTS will choose the resource with the lexicographically smallest `uid`. **note:** The `creationTimestamp` and `uid` fields are part of the [ObjectMeta](https://kubernetes.io/docs/reference/kubernetes-api/common-definitions/object-meta/) resource. --- ## Host collisions A host collision occurs when multiple Ingress, VirtualServer, and TransportServer (configured for TLS Passthrough) resources configure the same `host`. NGINX Ingress Controller LTS has two strategies for handling host collisions: - Choosing a single "winner" resource to handle the host. - Merging the configuration of the conflicting resources. --- ### Choosing the winner Consider the following two resources: - `cafe-ingress` Ingress: ```yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: cafe-ingress spec: ingressClassName: nginx rules: - host: cafe.example.com . . . ``` - `cafe-virtual-server` VirtualServer: ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServer metadata: name: cafe-virtual-server spec: host: cafe.example.com . . . ``` If a user creates both resources in the cluster, a host collision will occur. NGINX Ingress Controller LTS will pick the winner using the [winner selection algorithm](#winner-selection-algorithm). If `cafe-virtual-server` was created first, it will win the host `cafe.example.com` and NGINX Ingress Controller LTS will reject `cafe-ingress`. This will be reflected in the events and in the resource's status field: ```shell kubectl describe vs cafe-virtual-server ``` ```text ... Status: ... Message: Configuration for default/cafe-virtual-server was added or updated Reason: AddedOrUpdated State: Valid Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal AddedOrUpdated 9s nginx-ingress-controller Configuration for default/cafe-virtual-server was added or updated ``` ```shell kubectl describe ingress cafe-ingress ``` ```text Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning Rejected 66s nginx-ingress-controller All hosts are taken by other resources ``` Similarly, if `cafe-ingress` was created first, it will win `cafe.example.com` and NGINX Ingress Controller LTS will reject `cafe-virtual-server`. **note:** You can configure multiple hosts for Ingress resources, and its possible that an Ingress resource can be the winner for some of its hosts and a loser for the others. For example, if `cafe-ingress` had an additional rule host rule for `pub.example.com`, NGINX Ingress Controller LTS would not reject the Ingress. Instead, it would allow `cafe-ingress` to handle `pub.example.com`. --- ### Merging configuration for the same host It is possible to merge configuration for multiple Ingress resources for the same host. One common use case for this approach is distributing resources across multiple namespaces. The [Cross-namespace configuration](/nic/lts/configuration/ingress-resources/cross-namespace-configuration.md) topic has more information. It is *not* possible to merge the configurations for multiple VirtualServer resources for the same host. However, you can split the VirtualServers into multiple VirtualServerRoute resources, which a single VirtualServer can then reference. See the [corresponding example](https://github.com/nginx/kubernetes-ingress/tree/v/examples/custom-resources/cross-namespace-configuration) on GitHub. It is *not* possible to merge configuration for multiple TransportServer resources. --- ## Listener/Host collisions Listener/Host collisions occur when multiple TransportServer resources (configured for TCP/UDP load balancing) specify the same combination of `spec.listener.name` and `spec.host`. The combination of `spec.listener.name` and `spec.host` must be unique among all TransportServer resources. If two TransportServer resources specify the same spec.listener.name and spec.host, one of them will be rejected to prevent conflicts. In the case where spec.host is not specified, it is considered an empty string. NGINX Ingress Controller LTS will choose the winner, which will own that listener and host combination. --- ### Choosing the winner Consider the following two resources: - `tcp-1` TransportServer: ```yaml apiVersion: k8s.nginx.org/v1 kind: TransportServer metadata: name: tcp-1 spec: host: dns.example.com listener: name: dns-tcp protocol: TCP . . . ``` - `tcp-2` TransportServer: ```yaml apiVersion: k8s.nginx.org/v1 kind: TransportServer metadata: name: tcp-2 spec: host: dns.example.com listener: name: dns-tcp protocol: TCP . . . ``` If a user creates both resources in the cluster, a listener collision will occur. As a result, NGINX Ingress Controller LTS will pick the winner using the [winner selection algorithm](#winner-selection-algorithm). In our example, if `tcp-1` was created first, it will win the listener `dns-tcp` and NGINX Ingress Controller LTS will reject `tcp-2`. This will be reflected in the events and in the resource's status field: ```shell kubectl describe ts tcp-2 ``` ```text ... Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning Rejected 10s nginx-ingress-controller Listener dns-tcp is taken by another resource ``` Similarly, if `tcp-2` was created first, it will win `dns-tcp` and NGINX Ingress Controller LTS will reject `tcp-1`. -------------------------------------------------------------------------------- title: "Run multiple NGINX Ingress Controller LTSs" url: https://docs.nginx.com/nginx-ingress-controller/lts/install/multiple-controllers/index.md -------------------------------------------------------------------------------- This document describes how to run multiple F5 NGINX Ingress Controller LTS instances. It explains the following topics: - Ingress class concept. - How to run NGINX Ingress Controller LTS in the same cluster with another Ingress Controller and prevent conflicts between them - How to run multiple NGINX Ingress Controller LTSs. **note:** This document refers to [Ingress](/nic/lts/configuration/ingress-resources/basic-configuration.md), [VirtualServer](/nic/lts/configuration/virtualserver-and-virtualserverroute-resources.md#virtualserver-specification), [VirtualServerRoute](/nic/lts/configuration/virtualserver-and-virtualserverroute-resources.md#virtualserverroute-specification), and [TransportServer](/nic/lts/configuration/transportserver-resource.md) resources as "configuration resources". ## Ingress class The [IngressClass](https://kubernetes.io/docs/concepts/services-networking/ingress/#ingress-class) resource allows for multiple Ingress Controller to operate in the same cluster. It also allow developers to select which Ingress Controller implementation to use for their Ingress resource. The IngressClass has the following characteristics: - Every Ingress Controller must only handle Ingress resources for its particular class. - Ingress resources need to have the `ingressClassName` field set to the value of the class of the Ingress Controller the user wants to use. - VirtualServer, VirtualServerRoute, Policy, and TransportServer resources need to have the `ingressClassName` field set to the value of the class of the Ingress Controller the user wants to use. ### Configuring Ingress class The default Ingress class of NGINX Ingress Controller LTS is `nginx`, which means that it only handles configuration resources with the Ingress class set to `nginx`. You can customize the class through the `-ingress-class` command-line argument. **note:** If the class of an Ingress resource is not set, Kubernetes will set it to the class of the default Ingress Controller. To make the Ingress Controller the default one, the `ingressclass.kubernetes.io/is-default-class` property must be set on the IngressClass resource. To learn more, see Step 3 *Create an IngressClass resource* of the [Create Common Resources](/nic/lts/install/manifests.md#create-common-resources) section. For VirtualServer, VirtualServerRoute, Policy and TransportServer resources, NGINX Ingress Controller LTS will always handle resources with an empty class. ## Run NGINX Ingress Controller LTS and another Ingress Controller It is possible to run NGINX Ingress Controller LTS and an Ingress Controller for another load balancer in the same cluster. This is often the case if you create your cluster through a cloud provider's managed Kubernetes service that by default might include the Ingress Controller for the HTTP load balancer of the cloud provider, and you want to use NGINX Ingress Controller LTS. To make sure that NGINX Ingress Controller LTS handles specific configuration resources, update those resources with the class set to the value that is configured in NGINX Ingress Controller LTS. By default, this is `nginx`. ## Run multiple NGINX Ingress Controller LTSs When running NGINX Ingress Controller LTS, you have the following options with regards to which configuration resources it handles: - Cluster-wide Ingress Controller (default): NGINX Ingress Controller LTS handles configuration resources created in any namespace of the cluster. As NGINX is a high-performance load balancer capable of serving many applications at the same time, this option is used by default in our installation manifests and Helm chart. - Defined-namespace Ingress Controller: You can configure the Ingress Controller to handle configuration resources only from particular namespaces, which is controlled through the `-watch-namespace` command-line argument. This can be useful if you want to use different NGINX Ingress Controller LTSs for different applications, both in terms of isolation and/or operation. - Ingress Controller for Specific Ingress Class: This option works in conjunction with either of the options above. You can further customize which configuration resources are handled by the Ingress Controller by configuring the class of the Ingress Controller and using that class in your configuration resources. The [Configuring Ingress Class](#configuring-ingress-class) section above explains where. These options allow you to run multiple NGINX Ingress Controller LTSs, each handling a different set of configuration resources. **note:** There is a reference available for all [Command-line arguments](/nic/lts/configuration/global-configuration/command-line-arguments) Every command-line argument is also available as a parameter for the [Helm chart](/nic/lts/install/helm/parameters.md) -------------------------------------------------------------------------------- title: "Run multiple NGINX Ingress Controllers" url: https://docs.nginx.com/nginx-ingress-controller/install/multiple-controllers/index.md -------------------------------------------------------------------------------- This document describes how to run multiple F5 NGINX Ingress Controller instances. It explains the following topics: - Ingress class concept. - How to run NGINX Ingress Controller in the same cluster with another Ingress Controller and prevent conflicts between them - How to run multiple NGINX Ingress Controllers. **Note:** This document refers to [Ingress](/nic/configuration/ingress-resources/basic-configuration.md), [VirtualServer](/nic/configuration/virtualserver-and-virtualserverroute-resources.md#virtualserver-specification), [VirtualServerRoute](/nic/configuration/virtualserver-and-virtualserverroute-resources.md#virtualserverroute-specification), and [TransportServer](/nic/configuration/transportserver-resource.md) resources as "configuration resources". ## Ingress class The [IngressClass](https://kubernetes.io/docs/concepts/services-networking/ingress/#ingress-class) resource allows for multiple Ingress Controller to operate in the same cluster. It also allow developers to select which Ingress Controller implementation to use for their Ingress resource. The IngressClass has the following characteristics: - Every Ingress Controller must only handle Ingress resources for its particular class. - Ingress resources need to have the `ingressClassName` field set to the value of the class of the Ingress Controller the user wants to use. - VirtualServer, VirtualServerRoute, Policy, and TransportServer resources need to have the `ingressClassName` field set to the value of the class of the Ingress Controller the user wants to use. ### Configuring Ingress class The default Ingress class of NGINX Ingress Controller is `nginx`, which means that it only handles configuration resources with the Ingress class set to `nginx`. You can customize the class through the `-ingress-class` command-line argument. **Note:** If the class of an Ingress resource is not set, Kubernetes will set it to the class of the default Ingress Controller. To make the Ingress Controller the default one, the `ingressclass.kubernetes.io/is-default-class` property must be set on the IngressClass resource. To learn more, see Step 3 *Create an IngressClass resource* of the [Create Common Resources](/nic/install/manifests.md#create-common-resources) section. For VirtualServer, VirtualServerRoute, Policy and TransportServer resources, NGINX Ingress Controller will always handle resources with an empty class. ## Run NGINX Ingress Controller and another Ingress Controller It is possible to run NGINX Ingress Controller and an Ingress Controller for another load balancer in the same cluster. This is often the case if you create your cluster through a cloud provider's managed Kubernetes service that by default might include the Ingress Controller for the HTTP load balancer of the cloud provider, and you want to use NGINX Ingress Controller. To make sure that NGINX Ingress Controller handles specific configuration resources, update those resources with the class set to the value that is configured in NGINX Ingress Controller. By default, this is `nginx`. ## Run multiple NGINX Ingress Controllers When running NGINX Ingress Controller, you have the following options with regards to which configuration resources it handles: - Cluster-wide Ingress Controller (default): NGINX Ingress Controller handles configuration resources created in any namespace of the cluster. As NGINX is a high-performance load balancer capable of serving many applications at the same time, this option is used by default in our installation manifests and Helm chart. - Defined-namespace Ingress Controller: You can configure the Ingress Controller to handle configuration resources only from particular namespaces, which is controlled through the `-watch-namespace` command-line argument. This can be useful if you want to use different NGINX Ingress Controllers for different applications, both in terms of isolation and/or operation. - Ingress Controller for Specific Ingress Class: This option works in conjunction with either of the options above. You can further customize which configuration resources are handled by the Ingress Controller by configuring the class of the Ingress Controller and using that class in your configuration resources. The [Configuring Ingress Class](#configuring-ingress-class) section above explains where. These options allow you to run multiple NGINX Ingress Controllers, each handling a different set of configuration resources. **Note:** There is a reference available for all [Command-line arguments](/nic/configuration/global-configuration/command-line-arguments) Every command-line argument is also available as a parameter for the [Helm chart](/nic/install/helm/parameters.md) -------------------------------------------------------------------------------- title: "Proxy Buffer Configuration Auto-Adjustment" url: https://docs.nginx.com/nginx-ingress-controller/configuration/proxy-buffers-configuration/index.md -------------------------------------------------------------------------------- This document explains how the `--enable-directive-autoadjust` option prevents NGINX configuration errors by automatically adjusting some HTTP proxy buffer directives. --- ## What it does The `--enable-directive-autoadjust` feature automatically fixes common proxy buffer configuration mistakes that would otherwise cause NGINX to fail with errors like: ```text [emerg] "proxy_busy_buffers_size" must be less than the size of all "proxy_buffers" minus one buffer ``` **What gets fixed:** - If you don't specify `proxy_buffers`, it sets a sensible default of `8 4k` - If your `proxy_busy_buffers_size` is too large, it reduces it to a safe value - If the number of proxy buffers is outside the valid range (minimum 2, maximum 1024), it gets clamped to those limits - Empty or invalid buffer settings get corrected automatically **Works with:** - [ConfigMap settings](/nic/configuration/global-configuration/configmap-resource.md#general-customization) - [Ingress annotations](/nic/configuration/ingress-resources/advanced-configuration-with-annotations/#general-customization) - [VirtualServer upstream buffer configurations](/nic/configuration/virtualserver-and-virtualserverroute-resources/#upstream) --- ## How to enable auto-adjustment #### Manifests Add the flag to the controller container: ```yaml args: - --enable-directive-autoadjust=true ``` #### Helm Enable via the Helm chart values file: ```yaml controller: directiveAutoAdjust: "true" ``` --- ## Examples ### Example 1 **Input:** ```yaml data: proxy-buffer-size: "5m" proxy-buffers: "8 1m" ``` #### Before (Error) Before enabling `--enable-directive-autoadjust`, NGINX fails to start with configuration validation errors. ```shell stderr: "2025/08/26 14:29:49 [emerg] 196#196: "proxy_busy_buffers_size" must be less than the size of all "proxy_buffers" minus one buffer in /etc/nginx/nginx.conf:121" ``` #### After (Fixed) With `--enable-directive-autoadjust`, the configuration is automatically adjusted: ```nginx proxy_buffers 8 1m; proxy_buffer_size 5m; proxy_busy_buffers_size 5m; ``` Logs: ```text I20250826 14:31:54.515490 1 configmaps.go:380] Changes made to proxy values: adjusted proxy_busy_buffers_size from to 5m because it was too small ``` ### Example 2 **Input:** ```yaml data: proxy-buffers: "1000000 1m" # Extremely high buffer count proxy-buffer-size: "999m" # Very large buffer size proxy-busy-buffers-size: "500m" ``` #### Before (Error) ```shell stderr: "2025/08/26 14:34:46 [emerg] 47#47: "proxy_busy_buffers_size" must be equal to or greater than the maximum of the value of "proxy_buffer_size" and one of the "proxy_buffers" in /etc/nginx/nginx.conf:121\n" ``` #### After (Fixed) With `--enable-directive-autoadjust`, sensible defaults are applied: ```shell proxy_buffers 1024 1m; proxy_buffer_size 999m; proxy_busy_buffers_size 999m; ``` Logs: ```shell I20250826 14:36:47.864375 1 configmaps.go:380] Changes made to proxy values: adjusted proxy_buffers number from 1000000 to 1024 I20250826 14:36:47.864389 1 configmaps.go:380] Changes made to proxy values: adjusted proxy_busy_buffers_size from 500m to 999m because it was too small ``` --- ## Monitoring and logging The controller outputs a log message whenever any of the proxy buffer directives are changed. Examples: ```text I20250826 14:06:43.734757 1 annotations.go:341] Changes made to proxy values: adjusted proxy_buffer_size from 512k to 64k because it was too big for proxy_buffers (2 64k) I20250826 14:06:43.734842 1 annotations.go:341] Changes made to proxy values: adjusted proxy_busy_buffers_size from to 64k because it was too small ``` View adjustment logs: ```bash kubectl logs -n | grep "Changes made to proxy values" ``` -------------------------------------------------------------------------------- title: "Proxy Buffer Configuration Auto-Adjustment" url: https://docs.nginx.com/nginx-ingress-controller/lts/configuration/proxy-buffers-configuration/index.md -------------------------------------------------------------------------------- This document explains how the `--enable-directive-autoadjust` option prevents NGINX configuration errors by automatically adjusting some HTTP proxy buffer directives. --- ## What it does The `--enable-directive-autoadjust` feature automatically fixes common proxy buffer configuration mistakes that would otherwise cause NGINX to fail with errors like: ```text [emerg] "proxy_busy_buffers_size" must be less than the size of all "proxy_buffers" minus one buffer ``` **What gets fixed:** - If you don't specify `proxy_buffers`, it sets a sensible default of `8 4k` - If your `proxy_busy_buffers_size` is too large, it reduces it to a safe value - If the number of proxy buffers is outside the valid range (minimum 2, maximum 1024), it gets clamped to those limits - Empty or invalid buffer settings get corrected automatically **Works with:** - [ConfigMap settings](/nic/lts/configuration/global-configuration/configmap-resource.md#general-customization) - [Ingress annotations](/nic/lts/configuration/ingress-resources/advanced-configuration-with-annotations/#general-customization) - [VirtualServer upstream buffer configurations](/nic/lts/configuration/virtualserver-and-virtualserverroute-resources/#upstream) --- ## How to enable auto-adjustment #### Manifests Add the flag to the controller container: ```yaml args: - --enable-directive-autoadjust=true ``` #### Helm Enable via the Helm chart values file: ```yaml controller: directiveAutoAdjust: "true" ``` --- ## Examples ### Example 1 **Input:** ```yaml data: proxy-buffer-size: "5m" proxy-buffers: "8 1m" ``` #### Before (Error) Before enabling `--enable-directive-autoadjust`, NGINX fails to start with configuration validation errors. ```shell stderr: "2025/08/26 14:29:49 [emerg] 196#196: "proxy_busy_buffers_size" must be less than the size of all "proxy_buffers" minus one buffer in /etc/nginx/nginx.conf:121" ``` #### After (Fixed) With `--enable-directive-autoadjust`, the configuration is automatically adjusted: ```nginx proxy_buffers 8 1m; proxy_buffer_size 5m; proxy_busy_buffers_size 5m; ``` Logs: ```text I20250826 14:31:54.515490 1 configmaps.go:380] Changes made to proxy values: adjusted proxy_busy_buffers_size from to 5m because it was too small ``` ### Example 2 **Input:** ```yaml data: proxy-buffers: "1000000 1m" # Extremely high buffer count proxy-buffer-size: "999m" # Very large buffer size proxy-busy-buffers-size: "500m" ``` #### Before (Error) ```shell stderr: "2025/08/26 14:34:46 [emerg] 47#47: "proxy_busy_buffers_size" must be equal to or greater than the maximum of the value of "proxy_buffer_size" and one of the "proxy_buffers" in /etc/nginx/nginx.conf:121\n" ``` #### After (Fixed) With `--enable-directive-autoadjust`, sensible defaults are applied: ```shell proxy_buffers 1024 1m; proxy_buffer_size 999m; proxy_busy_buffers_size 999m; ``` Logs: ```shell I20250826 14:36:47.864375 1 configmaps.go:380] Changes made to proxy values: adjusted proxy_buffers number from 1000000 to 1024 I20250826 14:36:47.864389 1 configmaps.go:380] Changes made to proxy values: adjusted proxy_busy_buffers_size from 500m to 999m because it was too small ``` --- ## Monitoring and logging The controller outputs a log message whenever any of the proxy buffer directives are changed. Examples: ```text I20250826 14:06:43.734757 1 annotations.go:341] Changes made to proxy values: adjusted proxy_buffer_size from 512k to 64k because it was too big for proxy_buffers (2 64k) I20250826 14:06:43.734842 1 annotations.go:341] Changes made to proxy values: adjusted proxy_busy_buffers_size from to 64k because it was too small ``` View adjustment logs: ```bash kubectl logs -n | grep "Changes made to proxy values" ``` -------------------------------------------------------------------------------- title: "Deploy a Policy for access control" url: https://docs.nginx.com/nginx-ingress-controller/configuration/access-control/index.md -------------------------------------------------------------------------------- This topic describes how to use F5 NGINX Ingress Controller to apply and update a Policy for access control. You can use access control policies with [VirtualServer custom resources](/nic/configuration/virtualserver-and-virtualserverroute-resources.md) or with [Ingress resources](/nic/configuration/ingress-resources/basic-configuration.md) using the `nginx.org/policies` annotation. --- ## Before you begin You should have a [working NGINX Ingress Controller](/nic/install/helm.md) instance. For ease of use in shell commands, set the following shell variables: 1. The public IP address for your NGINX Ingress Controller instance. ```shell IC_IP= ``` 2. The HTTP port of the same instance. ```shell IC_HTTP_PORT= ``` 3. The HTTPS port of the same instance (used for the [Ingress resource example](#use-access-control-with-ingress-resources)). ```shell IC_HTTPS_PORT= ``` --- ## Use access control with VirtualServer resources ### Deploy the example application Create the file _webapp.yaml_ with the following contents: *[Code: https://raw.githubusercontent.com/nginx/kubernetes-ingress/refs/heads/main/examples/custom-resources/access-control/webapp.yaml]* Apply it using `kubectl`: ```shell kubectl apply -f webapp.yaml ``` --- ### Deploy a Policy to create a deny rule Create a file named _access-control-policy-deny.yaml_. The highlighted _deny_ field will be used by the example application, and should be changed to the subnet of your machine. *[Code: https://raw.githubusercontent.com/nginx/kubernetes-ingress/refs/heads/main/examples/custom-resources/access-control/access-control-policy-deny.yaml]* Apply the policy: ```shell kubectl apply -f access-control-policy-deny.yaml ``` --- ### Configure load balancing Create a file named _virtual-server.yaml_ for the VirtualServer resource. The _policies_ field references the access control Policy created in the previous section. *[Code: https://raw.githubusercontent.com/nginx/kubernetes-ingress/refs/heads/main/examples/custom-resources/access-control/virtual-server.yaml]* Apply the policy: ```shell kubectl apply -f virtual-server.yaml ``` --- ## Test the example application Use `curl` to attempt to access the application: ```shell curl --resolve webapp.example.com:$IC_HTTP_PORT:$IC_IP http://webapp.example.com:$IC_HTTP_PORT ``` ```text 403 Forbidden

403 Forbidden

``` The *403* response is expected, successfully blocking your machine. --- ### Update the Policy to create an allow rule Update the Policy with the file _access-control-policy-allow.yaml_, setting the _allow_ field to the subnet of your machine. *[Code: https://raw.githubusercontent.com/nginx/kubernetes-ingress/refs/heads/main/examples/custom-resources/access-control/access-control-policy-allow.yaml]* Apply the Policy: ```shell kubectl apply -f access-control-policy-allow.yaml ``` ---- ### Verify the Policy update Attempt to access the application again: ```shell curl --resolve webapp.example.com:$IC_HTTP_PORT:$IC_IP http://webapp.example.com:$IC_HTTP_PORT ``` ```text Server address: 10.64.0.13:8080 Server name: webapp-5cbbc7bd78-wf85w ``` The successful response demonstrates that the policy has been updated. --- ## Use access control with Ingress resources You can also apply access control policies to standard Kubernetes Ingress resources using the `nginx.org/policies` annotation. This section walks through a complete example. ### Deploy the cafe application Create the file _cafe.yaml_ with the following contents: *[Code: https://raw.githubusercontent.com/nginx/kubernetes-ingress/refs/heads/main/examples/ingress-resources/access-control/cafe.yaml]* Apply it using `kubectl`: ```shell kubectl apply -f cafe.yaml ``` ### Configure NGINX to use the X-Real-IP header Create the file _nginx-config.yaml_ to configure NGINX to trust the `X-Real-IP` header. This ensures the access control policy uses the client IP provided in that header. *[Code: https://raw.githubusercontent.com/nginx/kubernetes-ingress/refs/heads/main/examples/ingress-resources/access-control/nginx-config.yaml]* Apply the ConfigMap: ```shell kubectl apply -f nginx-config.yaml ``` ### Deploy a Policy to create an allow rule Create a file named _access-control-policy-allow.yaml_. The highlighted _allow_ field permits traffic from the `10.0.0.0/8` CIDR range and blocks all other addresses. *[Code: https://raw.githubusercontent.com/nginx/kubernetes-ingress/refs/heads/main/examples/ingress-resources/access-control/access-control-policy-allow.yaml]* Apply the policy: ```shell kubectl apply -f access-control-policy-allow.yaml ``` ### Create the Ingress resource Create a file named _cafe-ingress.yaml_ for the Ingress resource. The highlighted `nginx.org/policies` annotation references the access control Policy created in the previous step. *[Code: https://raw.githubusercontent.com/nginx/kubernetes-ingress/refs/heads/main/examples/ingress-resources/access-control/cafe-ingress.yaml]* Apply the Ingress: ```shell kubectl apply -f cafe-ingress.yaml ``` ### Test the allow policy 1. Send a request with an IP in the allowed `10.0.0.0/8` range using the `X-Real-IP` header: ```shell curl --resolve cafe.example.com:$IC_HTTPS_PORT:$IC_IP https://cafe.example.com:$IC_HTTPS_PORT/coffee --insecure -H "X-Real-IP: 10.0.0.1" ``` ```text Server address: 10.244.0.6:8080 Server name: coffee-7586895968-r26zn ... ``` The request succeeds because `10.0.0.1` is in the allowed range. 2. Send a request with an IP outside the allowed range: ```shell curl --resolve cafe.example.com:$IC_HTTPS_PORT:$IC_IP https://cafe.example.com:$IC_HTTPS_PORT/coffee --insecure -H "X-Real-IP: 192.168.1.1" ``` ```text 403 Forbidden

403 Forbidden

``` The *403* response confirms that NGINX blocks clients outside the allowed range. ### Update the Policy to create a deny rule Update the Policy with the file _access-control-policy-deny.yaml_, which denies traffic from the `10.0.0.0/8` CIDR range and allows all other addresses. *[Code: https://raw.githubusercontent.com/nginx/kubernetes-ingress/refs/heads/main/examples/ingress-resources/access-control/access-control-policy-deny.yaml]* Apply the updated Policy: ```shell kubectl apply -f access-control-policy-deny.yaml ``` The Ingress resource picks up the change automatically because the policy name (`webapp-policy`) stays the same. ### Verify the deny policy 1. Send a request with an IP in the now-denied `10.0.0.0/8` range: ```shell curl --resolve cafe.example.com:$IC_HTTPS_PORT:$IC_IP https://cafe.example.com:$IC_HTTPS_PORT/coffee --insecure -H "X-Real-IP: 10.0.0.1" ``` ```text 403 Forbidden

403 Forbidden

``` The same IP that was previously allowed is now rejected. 2. Send a request with an IP outside the denied range: ```shell curl --resolve cafe.example.com:$IC_HTTPS_PORT:$IC_IP https://cafe.example.com:$IC_HTTPS_PORT/coffee --insecure -H "X-Real-IP: 192.168.1.1" ``` ```text Server address: 10.244.0.6:8080 Server name: coffee-7586895968-r26zn ... ``` Clients outside the denied range are now allowed through. -------------------------------------------------------------------------------- title: "Deploy a Policy for access control" url: https://docs.nginx.com/nginx-ingress-controller/lts/configuration/access-control/index.md -------------------------------------------------------------------------------- This topic describes how to use F5 NGINX Ingress Controller LTS to apply and update a Policy for access control. You can use access control policies with [VirtualServer custom resources](/nic/lts/configuration/virtualserver-and-virtualserverroute-resources.md) or with [Ingress resources](/nic/lts/configuration/ingress-resources/basic-configuration.md) using the `nginx.org/policies` annotation. --- ## Before you begin You should have a [working NGINX Ingress Controller LTS](/nic/lts/install/helm.md) instance. For ease of use in shell commands, set the following shell variables: 1. The public IP address for your NGINX Ingress Controller LTS instance. ```shell IC_IP= ``` 2. The HTTP port of the same instance. ```shell IC_HTTP_PORT= ``` 3. The HTTPS port of the same instance (used for the [Ingress resource example](#use-access-control-with-ingress-resources)). ```shell IC_HTTPS_PORT= ``` --- ## Use access control with VirtualServer resources ### Deploy the example application Create the file _webapp.yaml_ with the following contents: *[Code: https://raw.githubusercontent.com/nginx/kubernetes-ingress/refs/heads/release-2026-lts/examples/custom-resources/access-control/webapp.yaml]* Apply it using `kubectl`: ```shell kubectl apply -f webapp.yaml ``` --- ### Deploy a Policy to create a deny rule Create a file named _access-control-policy-deny.yaml_. The highlighted _deny_ field will be used by the example application, and should be changed to the subnet of your machine. *[Code: https://raw.githubusercontent.com/nginx/kubernetes-ingress/refs/heads/release-2026-lts/examples/custom-resources/access-control/access-control-policy-deny.yaml]* Apply the policy: ```shell kubectl apply -f access-control-policy-deny.yaml ``` --- ### Configure load balancing Create a file named _virtual-server.yaml_ for the VirtualServer resource. The _policies_ field references the access control Policy created in the previous section. *[Code: https://raw.githubusercontent.com/nginx/kubernetes-ingress/refs/heads/release-2026-lts/examples/custom-resources/access-control/virtual-server.yaml]* Apply the policy: ```shell kubectl apply -f virtual-server.yaml ``` --- ## Test the example application Use `curl` to attempt to access the application: ```shell curl --resolve webapp.example.com:$IC_HTTP_PORT:$IC_IP http://webapp.example.com:$IC_HTTP_PORT ``` ```text 403 Forbidden

403 Forbidden

``` The *403* response is expected, successfully blocking your machine. --- ### Update the Policy to create an allow rule Update the Policy with the file _access-control-policy-allow.yaml_, setting the _allow_ field to the subnet of your machine. *[Code: https://raw.githubusercontent.com/nginx/kubernetes-ingress/refs/heads/release-2026-lts/examples/custom-resources/access-control/access-control-policy-allow.yaml]* Apply the Policy: ```shell kubectl apply -f access-control-policy-allow.yaml ``` ---- ### Verify the Policy update Attempt to access the application again: ```shell curl --resolve webapp.example.com:$IC_HTTP_PORT:$IC_IP http://webapp.example.com:$IC_HTTP_PORT ``` ```text Server address: 10.64.0.13:8080 Server name: webapp-5cbbc7bd78-wf85w ``` The successful response demonstrates that the policy has been updated. --- ## Use access control with Ingress resources You can also apply access control policies to standard Kubernetes Ingress resources using the `nginx.org/policies` annotation. This section walks through a complete example. ### Deploy the cafe application Create the file _cafe.yaml_ with the following contents: *[Code: https://raw.githubusercontent.com/nginx/kubernetes-ingress/refs/heads/release-2026-lts/examples/ingress-resources/access-control/cafe.yaml]* Apply it using `kubectl`: ```shell kubectl apply -f cafe.yaml ``` ### Configure NGINX to use the X-Real-IP header Create the file _nginx-config.yaml_ to configure NGINX to trust the `X-Real-IP` header. This ensures the access control policy uses the client IP provided in that header. *[Code: https://raw.githubusercontent.com/nginx/kubernetes-ingress/refs/heads/release-2026-lts/examples/ingress-resources/access-control/nginx-config.yaml]* Apply the ConfigMap: ```shell kubectl apply -f nginx-config.yaml ``` ### Deploy a Policy to create an allow rule Create a file named _access-control-policy-allow.yaml_. The highlighted _allow_ field permits traffic from the `10.0.0.0/8` CIDR range and blocks all other addresses. *[Code: https://raw.githubusercontent.com/nginx/kubernetes-ingress/refs/heads/release-2026-lts/examples/ingress-resources/access-control/access-control-policy-allow.yaml]* Apply the policy: ```shell kubectl apply -f access-control-policy-allow.yaml ``` ### Create the Ingress resource Create a file named _cafe-ingress.yaml_ for the Ingress resource. The highlighted `nginx.org/policies` annotation references the access control Policy created in the previous step. *[Code: https://raw.githubusercontent.com/nginx/kubernetes-ingress/refs/heads/release-2026-lts/examples/ingress-resources/access-control/cafe-ingress.yaml]* Apply the Ingress: ```shell kubectl apply -f cafe-ingress.yaml ``` ### Test the allow policy 1. Send a request with an IP in the allowed `10.0.0.0/8` range using the `X-Real-IP` header: ```shell curl --resolve cafe.example.com:$IC_HTTPS_PORT:$IC_IP https://cafe.example.com:$IC_HTTPS_PORT/coffee --insecure -H "X-Real-IP: 10.0.0.1" ``` ```text Server address: 10.244.0.6:8080 Server name: coffee-7586895968-r26zn ... ``` The request succeeds because `10.0.0.1` is in the allowed range. 2. Send a request with an IP outside the allowed range: ```shell curl --resolve cafe.example.com:$IC_HTTPS_PORT:$IC_IP https://cafe.example.com:$IC_HTTPS_PORT/coffee --insecure -H "X-Real-IP: 192.168.1.1" ``` ```text 403 Forbidden

403 Forbidden

``` The *403* response confirms that NGINX blocks clients outside the allowed range. ### Update the Policy to create a deny rule Update the Policy with the file _access-control-policy-deny.yaml_, which denies traffic from the `10.0.0.0/8` CIDR range and allows all other addresses. *[Code: https://raw.githubusercontent.com/nginx/kubernetes-ingress/refs/heads/release-2026-lts/examples/ingress-resources/access-control/access-control-policy-deny.yaml]* Apply the updated Policy: ```shell kubectl apply -f access-control-policy-deny.yaml ``` The Ingress resource picks up the change automatically because the policy name (`webapp-policy`) stays the same. ### Verify the deny policy 1. Send a request with an IP in the now-denied `10.0.0.0/8` range: ```shell curl --resolve cafe.example.com:$IC_HTTPS_PORT:$IC_IP https://cafe.example.com:$IC_HTTPS_PORT/coffee --insecure -H "X-Real-IP: 10.0.0.1" ``` ```text 403 Forbidden

403 Forbidden

``` The same IP that was previously allowed is now rejected. 2. Send a request with an IP outside the denied range: ```shell curl --resolve cafe.example.com:$IC_HTTPS_PORT:$IC_IP https://cafe.example.com:$IC_HTTPS_PORT/coffee --insecure -H "X-Real-IP: 192.168.1.1" ``` ```text Server address: 10.244.0.6:8080 Server name: coffee-7586895968-r26zn ... ``` Clients outside the denied range are now allowed through. -------------------------------------------------------------------------------- title: "Upgrade NGINX Ingress Controller" url: https://docs.nginx.com/nginx-ingress-controller/install/upgrade/index.md -------------------------------------------------------------------------------- This document describes how to upgrade F5 NGINX Ingress Controller when a new version releases. It covers the necessary steps for minor versions as well as major versions (Such as 3.x to 4.x). Many of the nuances in upgrade paths relate to how custom resource definitions (CRDs) are managed. ## Minor NGINX Ingress Controller upgrades ### Upgrade NGINX Ingress Controller CRDs **Note:** If you are running NGINX Ingress Controller v3.x, you should read [Upgrade from NGINX Ingress Controller v3.x to v4.0.0](/nic/install/upgrade.md#upgrade-from-3x-to-4x) before continuing. To upgrade the CRDs, pull the Helm chart source, then use _kubectl apply_: ```shell helm pull oci://ghcr.io/nginx/charts/nginx-ingress --untar --version kubectl apply -f crds/ ``` Alternatively, CRDs can be upgraded without pulling the chart by running: ```shell kubectl apply -f https://raw.githubusercontent.com/nginx/kubernetes-ingress/v/deploy/crds.yaml ``` In the above command, `v` represents the version of the NGINX Ingress Controller release rather than the Helm chart version. **Note:** The following warning is expected and can be ignored: `Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply`. Check the [release notes](https://www.github.com/nginx/kubernetes-ingress/releases) for a new release for any special upgrade procedures. ### Upgrade NGINX Ingress Controller charts Once the CRDs have been upgraded, you can then upgrade the release chart. **Note:** When upgrading to version 5.6.0 or later, if you configured NGINX Service Mesh previously, remove any related values from the Helm chart before upgrading. These settings only applied to previous NGINX Service Mesh deployments. Users who never deployed service mesh are unaffected. The command depends on if you installed the chart using the registry or from source. To upgrade a release named _my-release_, use the following command: #### OCI registry ```shell helm upgrade my-release oci://ghcr.io/nginx/charts/nginx-ingress --version ``` #### Source ```shell helm upgrade my-release . ``` ## Upgrade from 3.x to 4.x **Note:** The instructions in this section are intended only for users upgrading from NGINX Ingress Controller 3.x to 4.0.0. Internal changes meant that backwards compability was not possible, requiring extra steps to upgrade. This section provides step-by-step instructions for upgrading NGINX Ingress Controller from version v3.x to v4.0.0. There are two necessary steps required - Update the `apiVersion` value of custom resources - Configure structured logging. If you want to use NGINX Plus, you will also need to follow the [Create a license Secret](/nic/install/license-secret.md) topic. ### Update custom resource apiVersion If you're using Helm chart version `v2.x`, update your `GlobalConfiguration`, `Policy`, and `TransportServer` resources from `apiVersion: k8s.nginx.org/v1alpha1` to `apiVersion: k8s.nginx.org/v1` before upgrading to NGINX Ingress Controller 4.0.0. If the Helm chart you have been using is `v1.0.2` or earlier (NGINX Ingress Controller `v3.3.2`), upgrade to Helm chart `v1.4.2` (NGINX Ingress Controller `v3.7.2`) before updating your GlobalConfiguration, Policy, and TransportServer resources. The example below shows the change for a Policy resource: you must do the same for all GlobalConfiguration and TransportServer resources. #### Before ```yaml apiVersion: k8s.nginx.org/v1alpha1 kind: Policy metadata: name: rate-limit-policy spec: rateLimit: rate: 1r/s key: ${binary_remote_addr} zoneSize: 10M ``` #### After ```yaml apiVersion: k8s.nginx.org/v1 kind: Policy metadata: name: rate-limit-policy spec: rateLimit: rate: 1r/s key: ${binary_remote_addr} zoneSize: 10M ``` **Note:** If a *GlobalConfiguration*, *Policy* or *TransportServer* resource is deployed with `apiVersion: k8s.nginx.org/v1alpha1`, it will be **deleted** during the upgrade process. After you move the custom resources to `v1`, run the following `kubectl` commands before upgrading to v4.0.0 Custom Resource Definitions (CRDs) to avoid webhook errors caused by leftover `v1alpha1` resources. For details, see [GitHub issue #7010](https://github.com/nginx/kubernetes-ingress/issues/7010). ```shell kubectl patch customresourcedefinitions transportservers.k8s.nginx.org --subresource='status' --type='merge' -p '{"status":{"storedVersions": ["v1"]}}' ``` ```shell kubectl patch customresourcedefinitions globalconfigurations.k8s.nginx.org --subresource='status' --type='merge' -p '{"status":{"storedVersions": ["v1"]}}' ``` ### Configure structured logging To configure structured logging, you must update your log deployment arguments from an integer to a string. You can also choose different formats for the log output. **Note:** These options apply to NGINX Ingress Controller logs, and do not affect NGINX logs. | **Level arguments** | **Format arguments** | |---------------------|----------------------| | `trace` | `json` | | `debug` | `text` | | `info` | `glog` | | `warning` | | | `error` | | | `fatal` | | #### Helm The Helm value `controller.logLevel` is now a string instead of an integer. To change the rendering of the log format, use the `controller.logFormat` key. ```yaml controller: logLevel: info logFormat: json ``` #### Manifests The command line argument `-v` has been replaced with `-log-level`, and takes a string instead of an integer. The argument `-logtostderr` has also been deprecated. To change the rendering of the log format, use the `-log-format` argument. ```yaml args: - -log-level=info - -log-format=json ``` ### Create License secret If you're using [NGINX Plus](/nic/overview/nginx-plus.md) with NGINX Ingress Controller, you should read the [Create a license Secret](/nic/install/license-secret.md) topic to set up your NGINX Plus license. The topic also contains guidance for [sending reports to NGINX Instance Manager](/nic/install/license-secret.md#nim), which is necessary for air-gapped environments. Earlier versions required usage reporting through the cluster connector. This is no longer needed because it's now built into NGINX Plus. ## Upgrade a version older than v3.1.0 Starting in version 3.1.0, NGINX Ingress Controller uses updated Helm resource names, labels, and annotations to follow Helm best practices. [See the changes.](https://github.com/nginx/kubernetes-ingress/pull/3606) When you upgrade with Helm from a version earlier than 3.1.0, some resources such as `Deployment`, `DaemonSet`, and `Service` are recreated. This causes downtime. To reduce downtime, update all resources to use the new naming convention. The following steps help you do that. **Note:** The following steps apply to both 2.x and 3.0.x releases. The steps you should follow depend on your Helm release name: #### nginx-ingress Use `kubectl describe` on deployment/daemonset to get the `Selector` value: ```shell kubectl describe deployments -n ``` Copy the key=value under `Selector`, such as: ```shell Selector: app=nginx-ingress-nginx-ingress ``` Check out the latest available tag using `git checkout v` Go to `/kubernetes-ingress/charts/nginx-ingress` Update the `selectorLabels: {}` field in the `values.yaml` file located at `/kubernetes-ingress/charts/nginx-ingress` with the copied `Selector` value. ```shell selectorLabels: {app: nginx-ingress-nginx-ingress} ``` Run `helm upgrade` with following arguments set: ```shell --set serviceNameOverride="nginx-ingress-nginx-ingress" --set controller.name="" --set fullnameOverride="nginx-ingress-nginx-ingress" ``` It might look like this: ```shell helm upgrade nginx-ingress oci://ghcr.io/nginx/charts/nginx-ingress --version 0.19.0 --set controller.kind=deployment/daemonset --set controller.nginxplus=false/true --set controller.image.pullPolicy=Always --set serviceNameOverride="nginx-ingress-nginx-ingress" --set controller.name="" --set fullnameOverride="nginx-ingress-nginx-ingress" -f values.yaml ``` Once the upgrade process has finished, use `kubectl describe` on the deployment to verify the change by reviewing its events: ```text Type Reason Age From Message ---- ------ ---- ---- ------- Normal ScalingReplicaSet 9m11s deployment-controller Scaled up replica set nginx-ingress-nginx-ingress- to 1 Normal ScalingReplicaSet 101s deployment-controller Scaled up replica set nginx-ingress-nginx-ingress- to 1 Normal ScalingReplicaSet 98s deployment-controller Scaled down replica set nginx-ingress-nginx-ingress- to 0 from 1 ``` #### Other release names Use `kubectl describe` on deployment/daemonset to get the `Selector` value: ```shell kubectl describe deployment/daemonset -n ``` Copy the key=value under ```Selector```, such as: ```shell Selector: app=-nginx-ingress ``` Check out the latest available tag using `git checkout v` Go to `/kubernetes-ingress/charts/nginx-ingress`. Update the `selectorLabels: {}` field in the `values.yaml` file located at `/kubernetes-ingress/charts/nginx-ingress` with the copied `Selector` value. ```shell selectorLabels: {app: -nginx-ingress} ``` Run `helm upgrade` with following arguments set: ```shell --set serviceNameOverride="-nginx-ingress" --set controller.name="" ``` It might look like this: ```shell helm upgrade test-release oci://ghcr.io/nginx/charts/nginx-ingress --version 0.19.0 --set controller.kind=deployment/daemonset --set controller.nginxplus=false/true --set controller.image.pullPolicy=Always --set serviceNameOverride="test-release-nginx-ingress" --set controller.name="" -f values.yaml ``` Once the upgrade process has finished, use `kubectl describe` on the deployment to verify the change by reviewing its events: ```shell Type Reason Age From Message ---- ------ ---- ---- ------- Normal ScalingReplicaSet 9m11s deployment-controller Scaled up replica set test-release-nginx-ingress- to 1 Normal ScalingReplicaSet 101s deployment-controller Scaled up replica set test-release-nginx-ingress- to 1 Normal ScalingReplicaSet 98s deployment-controller Scaled down replica set test-release-nginx-ingress- to 0 from 1 ``` ## Custom NGINX templates If you use custom NGINX configuration templates, review and apply any upstream template changes before upgrading. For more information, see [Custom templates](/nic/configuration/global-configuration/custom-templates.md). **warning:** In version 5.5, the default server block was removed from `main-template` and is now generated by `ingress-template`. You must update both templates before upgrading. If you had customizations inside the default server block in your `main-template`, the patch will remove them. Migrate those changes to your `ingress-template` manually. 1. Clone the repository: ```shell git clone https://github.com/nginx/kubernetes-ingress.git ``` 2. Generate a diff between the version you are upgrading from and the version you are upgrading to. For `main-template` and `ingress-template`, use `version1`. For `virtualserver-template` and `transportserver-template`, use `version2`: ```shell git diff v..v -- internal/configs/version1/ > upstream.patch git diff v..v -- internal/configs/version2/ > upstream.patch ``` 3. Extract your custom template from the ConfigMap to a local file. The valid key names are `main-template`, `ingress-template`, `virtualserver-template`, and `transportserver-template`: ```shell kubectl get configmap -n nginx-ingress -o jsonpath='{.data.}' > my-template.tmpl ``` 4. Apply the patch to your custom template: ```shell patch my-template.tmpl < upstream.patch ``` If the patch fails to apply some changes, review the rejected changes in `my-template.tmpl.rej` and apply them by manually. 5. Update the template key with the patched content. If you manage your ConfigMap directly, update the relevant key in your manifest and run `kubectl apply -f `. If you use Helm, set the key under `controller.config.entries` in your values file and run `helm upgrade`. -------------------------------------------------------------------------------- title: "Upgrade NGINX Ingress Controller LTS" url: https://docs.nginx.com/nginx-ingress-controller/lts/install/upgrade/index.md -------------------------------------------------------------------------------- This document describes how to upgrade F5 NGINX Ingress Controller LTS to a new patch release. Many of the nuances in upgrade paths relate to how custom resource definitions (CRDs) are managed. ## Upgrading to a new LTS patch release ### Upgrade NGINX Ingress Controller LTS CRDs To upgrade the CRDs, pull the Helm chart source, then use _kubectl apply_: ```shell helm pull oci://ghcr.io/nginx/charts/nginx-ingress-lts --untar --version kubectl apply -f crds/ ``` Alternatively, CRDs can be upgraded without pulling the chart by running: ```shell kubectl apply -f https://raw.githubusercontent.com/nginx/kubernetes-ingress/v/deploy/crds.yaml ``` In the above command, `v` represents the version of the NGINX Ingress Controller LTS release rather than the Helm chart version. **note:** The following warning is expected and can be ignored: `Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply`. Check the [release notes](https://www.github.com/nginx/kubernetes-ingress/releases) for a new release for any special upgrade procedures. ### Upgrade NGINX Ingress Controller LTS charts Once the CRDs have been upgraded, you can then upgrade the release chart. The command depends on if you installed the chart using the registry or from source. To upgrade a release named _my-release_, use the following command: #### OCI registry ```shell helm upgrade my-release oci://ghcr.io/nginx/charts/nginx-ingress-lts --version ``` #### Source ```shell helm upgrade my-release . ``` -------------------------------------------------------------------------------- title: "Migrate from Ingress-NGINX Controller to NGINX Ingress Controller" url: https://docs.nginx.com/nginx-ingress-controller/install/migrate-ingress-nginx/index.md -------------------------------------------------------------------------------- This document describes how to migrate from the community-maintained Ingress-NGINX Controller to F5 NGINX Ingress Controller. ## Overview This page explains two different ways to migrate from the community-maintained [Ingress-NGINX Controller](https://github.com/kubernetes/ingress-nginx) project to NGINX Ingress Controller: using NGINX's Ingress Resources or with Kubernetes's built-in Ingress Resources. This is typically because of implementation differences, and to take advantage of features such as [NGINX Plus integration](/nic/overview/nginx-plus). **Note:** For more information, you may want to read the two-part blog series from the [NGINX Community Blog](https://blog.nginx.org/) on Migrating from ingress-nginx to NGINX Ingress Controller: - [Part one](https://blog.nginx.org/blog/migrating-ingress-controllers-part-one) - [Part two](https://blog.nginx.org/blog/migrating-ingress-controllers-part-2) ## Before you begin To complete the instructions in this guide, you need the following: - A working knowledge of [Ingress Controllers](/glossary/#k8s-ingress-controller). - An [NGINX Ingress Controller installation](/nic/install/) on the same host as an existing Ingress-NGINX Controller. There are two primary paths for migrating between the community Ingress-NGINX Controller to NGINX Ingress Controller: - Using NGINX Ingress Resources - Using Kubernetes Ingress Resources. ## Migration with NGINX Ingress resources This path uses Kubernetes Ingress Resources to set root permissions, then NGINX Ingress Resources for configuration using custom resource definitions (CRDs): * [VirtualServer and VirtualServerRoute](/nic/configuration/virtualserver-and-virtualserverroute-resources) * [TransportServer](/nic/configuration/transportserver-resource) * [GlobalConfiguration](/nic/configuration/global-configuration/globalconfiguration-resource) * [Policy](/nic/configuration/policy-resource) ### Configuring SSL termination and HTTP path-based routing The following two code examples correspond to a Kubernetes Ingress Resource and an [NGINX VirtualServer Resource](/nic/configuration/virtualserver-and-virtualserverroute-resources). Although the syntax and indentation is different, they accomplish the same basic Ingress functions, used for SSL termination and Layer 7 path-based routing. Kubernetes Ingress Resource: ```yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: nginx-test spec: tls: - hosts: - foo.bar.com secretName: tls-secret rules: - host: foo.bar.com http: paths: - path: /login backend: serviceName: login-svc servicePort: 80 - path: /billing serviceName: billing-svc servicePort: 80 ``` NGINX VirtualServer Resource: ```yaml apiVersion: networking.k8s.io/v1 kind: VirtualServer metadata: name: nginx-test spec: host: foo.bar.com tls: secret: tls-secret upstreams: - name: login service: login-svc port: 80 - name: billing service: billing-svc port: 80 routes: - path: /login action: pass: login - path: /billing action: pass: billing ``` ### Configuring TCP/UDP load balancing and TLS passthrough NGINX Ingress Controller exposes TCP and UDP services using [TransportServer](/nic/configuration/transportserver-resource) and [GlobalConfiguration](/nic/configuration/global-configuration/globalconfiguration-resource) resources. These resources provide a broad range of options for TCP/UDP and TLS Passthrough load balancing. By contrast, the community Ingress-NGINX Controller exposes TCP/UDP services by using a Kubernetes ConfigMap object. ### Convert Ingress-NGINX Controller annotations to NGINX Ingress resources Kubernetes deployments often need to extend basic Ingress rules for advanced use cases such as canary and blue-green deployments, traffic throttling, and ingress-egress traffic manipulation. The community Ingress-NGINX Controller implements many of these using Kubernetes annotations with custom Lua extensions. These custom Lua extensions are intended for specific NGINX Ingress resource definitions and may not be as granular as required for advanced use cases. The following examples show how to convert these annotations into NGINX Ingress Controller Resources. #### Canary deployments Canary and blue-green deployments allow you to push code changes to production environments without disrupting existing users. NGINX Ingress Controller runs them on the data plane: to migrate from the community Ingress-NGINX Controller, you must map the latter's annotations to [VirtualServer and VirtualServerRoute resources](/nic/configuration/virtualserver-and-virtualserverroute-resources). The Ingress-NGINX Controller evaluates canary annotations in the following order: 1. _nginx.ingress.kubernetes.io/canary-by-header_ 1. _nginx.ingress.kubernetes.io/canary-by-cookie_ 1. _nginx.ingress.kubernetes.io/canary-by-weight_ For NGINX Ingress Controller to evalute them the same way, they must appear in the same order in the VirtualServer or VirtualServerRoute Manifest. Ingress-NGINX Controller: ```yaml nginx.ingress.kubernetes.io/canary: "true" nginx.ingress.kubernetes.io/canary-by-header: "httpHeader" ``` NGINX Ingress Controller: ```yaml matches: - conditions: - header: httpHeader value: never action: pass: echo - header: httpHeader value: always action: pass: echo-canary action: pass: echo ``` Ingress-NGINX Controller: ```yaml nginx.ingress.kubernetes.io/canary: "true" nginx.ingress.kubernetes.io/canary-by-header: "httpHeader" nginx.ingress.kubernetes.io/canary-by-header-value: "my-value" ``` NGINX Ingress Controller: ```yaml matches: - conditions: - header: httpHeader value: my-value action: pass: echo-canary action: pass: echo ``` Ingress-NGINX Controller: ```yaml nginx.ingress.kubernetes.io/canary: "true" nginx.ingress.kubernetes.io/canary-by-cookie: "cookieName" ``` NGINX Ingress Controller: ```yaml matches: - conditions: - cookie: cookieName value: never action: pass: echo - cookie: cookieName value: always action: pass: echo-canary action: pass: echo ``` #### Traffic control Environments using microservices tend to use extensive traffic-control policies to manage ephemeral applications using circuit breaking and rate and connection limiting to prevent error conditions due to unhealthy states or abnormal behavior. The following examples map Ingress-NGINX Controller annotations to NGINX [VirtualServer and VirtualServerRoute resources](/nic/configuration/virtualserver-and-virtualserverroute-resources) for rate limiting, custom HTTP errors, custom default backend and URI rewriting. Ingress-NGINX Controller: ```yaml nginx.ingress.kubernetes.io/custom-http-errors: "code" nginx.ingress.kubernetes.io/default-backend: "default-svc" ``` NGINX Ingress Controller: ```yaml errorPages: - codes: [code] redirect: code: 301 url: default-svc ``` Ingress-NGINX Controller: ```yaml nginx.ingress.kubernetes.io/limit-connections: "number" ``` NGINX Ingress Controller: ```yaml http-snippets: | limit_conn_zone $binary_remote_addr zone=zone_name:size; routes: - path: /path location-snippets: | limit_conn zone_name number; ``` Ingress-NGINX Controller: ```yaml nginx.ingress.kubernetes.io/limit-rate: "number" nginx.ingress.kubernetes.io/limit-rate-after: "number" ``` NGINX Ingress Controller: ```yaml location-snippets: | limit_rate number; limit_rate_after number; ``` Ingress-NGINX Controller: ```yaml nginx.ingress.kubernetes.io/limit-rpm: "number" nginx.ingress.kubernetes.io/limit-burst-multiplier: "multiplier" ``` NGINX Ingress Controller: ```yaml rateLimit: rate: number/m burst: number * multiplier key: ${binary_remote_addr} zoneSize: size ``` Ingress-NGINX Controller: ```yaml nginx.ingress.kubernetes.io/limit-rps: "number" nginx.ingress.kubernetes.io/limit-burst-multiplier: "multiplier" ``` NGINX Ingress Controller: ```yaml rateLimit: rate: number/s burst: number * multiplier key: ${binary_remote_addr} zoneSize: size ``` Ingress-NGINX Controller: ```yaml nginx.ingress.kubernetes.io/limit-whitelist: "CIDR" ``` NGINX Ingress Controller: ```yaml http-snippets: | server-snippets: | ``` Ingress-NGINX Controller: ```yaml nginx.ingress.kubernetes.io/rewrite-target: "URI" ``` NGINX Ingress Controller: ```yaml rewritePath: "URI" ``` There are four Ingress-NGINX Controller annotations without NGINX Ingress resource fields yet: they must be handled using snippets. - _nginx.ingress.kubernetes.io/limit-connections_ - _nginx.ingress.kubernetes.io/limit-rate_ - _nginx.ingress.kubernetes.io/limit-rate-after_ - _nginx.ingress.kubernetes.io/limit-whitelist_ #### Header manipulation Manipulating HTTP headers is useful in many cases, as they contain information that is important and relevant to systems involved in HTTP transactions. The community Ingress-NGINX Controller supports enabling and configuring cross-origin resource sharing (CORS) headings used by AJAX applications, where front-end Javascript code interacts with backend applications or web servers. These code blocks show how the Ingress-NGINX annotations correspond to NGINX Ingress Controller [VirtualServer and VirtualServerRoute resources](/nic/configuration/virtualserver-and-virtualserverroute-resources). Ingress-NGINX Controller: ```yaml nginx.ingress.kubernetes.io/enable-cors: "true" nginx.ingress.kubernetes.io/cors-allow-credentials: "true" nginx.ingress.kubernetes.io/cors-allow-headers: "X-Forwarded-For" nginx.ingress.kubernetes.io/cors-allow-methods: "PUT, GET, POST, OPTIONS" nginx.ingress.kubernetes.io/cors-allow-origin: "*" nginx.ingress.kubernetes.io/cors-max-age: "seconds" ``` NGINX Ingress Controller: ```yaml responseHeaders: add: - name: Access-Control-Allow-Credentials value: "true" - name: Access-Control-Allow-Headers value: "X-Forwarded-For" - name: Access-Control-Allow-Methods value: "PUT, GET, POST, OPTIONS" - name: Access-Control-Allow-Origin value: "*" - name: Access-Control-Max-Age value: "seconds" ``` #### Proxying and load balancing NGINX Ingress Controller has multiple proxy and load balancing functionalities you may want to configure based on the use case, such as configuring the load balancing algorithm and the timeout and buffering settings for proxied connections. This table shows how Ingress-NGINX Controller annotations map to statements in the upstream field for [VirtualServer and VirtualServerRoute resources](/nic/configuration/virtualserver-and-virtualserverroute-resources), covering load balancing, proxy timeout, proxy buffering and connection routing for a services' ClusterIP address and port. | Ingress-NGINX Controller | NGINX Ingress Controller | | ------------------------ | ------------------------ | | _nginx.ingress.kubernetes.io/load-balance_ | _lb-method_ | | _nginx.ingress.kubernetes.io/proxy-buffering_ | _buffering_ | | _nginx.ingress.kubernetes.io/proxy-buffers-number_ | _buffers_ | | _nginx.ingress.kubernetes.io/proxy-buffer-size_ | _buffers_ | | _nginx.ingress.kubernetes.io/proxy-connect-timeout_ | _connect-timeout_ | | _nginx.ingress.kubernetes.io/proxy-next-upstream_ | _next-upstream_ | | _nginx.ingress.kubernetes.io/proxy-next-upstream-timeout_ | _next-upstream-timeout_ | | _nginx.ingress.kubernetes.io/proxy-read-timeout_ | _read-timeout_ | | _nginx.ingress.kubernetes.io/proxy-send-timeout_ | _send-timeout_ | | _nginx.ingress.kubernetes.io/service-upstream_ | _use-cluster-ip_ | | _nginx.ingress.kubernetes.io/affinity_ (cookie) | _sticky-cookie-services_ | #### mTLS authentication mTLS authentication is a way of enforcing mutual authentication on traffic entering and exiting a cluster (north-sourth traffic). This secure form of communication is common within a service mesh, commonly used in strict zero-trust environments. NGINX Ingress Controller layer can handle mTLS authentication for end systems through the presentation of valid certificates for external connections. It accomplishes this through [Policy](/nic/configuration/policy-resource) resources, which correspond to Ingress-NGINX Controller annotations for [client certificate authentication](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#client-certificate-authentication) and [backend certificate authentication](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#backend-certificate-authentication). Ingress-NGINX Controller: ```yaml nginx.ingress.kubernetes.io/auth-tls-secret: secretName nginx.ingress.kubernetes.io/auth-tls-verify-client: "on" nginx.ingress.kubernetes.io/auth-tls-verify-depth: "1" ``` NGINX Ingress Controller: ```yaml ingressMTLS: clientCertSecret: secretName verifyClient: "on" verifyDepth: 1 ``` Ingress-NGINX Controller: ```yaml nginx.ingress.kubernetes.io/proxy-ssl-secret: "secretName" nginx.ingress.kubernetes.io/proxy-ssl-verify: "on|off" nginx.ingress.kubernetes.io/proxy-ssl-verify-depth: "1" nginx.ingress.kubernetes.io/proxy-ssl-protocols: "TLSv1.2" nginx.ingress.kubernetes.io/proxy-ssl-ciphers: "DEFAULT" nginx.ingress.kubernetes.io/proxy-ssl-name: "server-name" nginx.ingress.kubernetes.io/proxy-ssl-server-name: "on|off" ``` NGINX Ingress Controller: ```yaml egressMTLS: tlsSecret: secretName verifyServer: true|false verifyDepth: 1 protocols: TLSv1.2 ciphers: DEFAULT sslName: server-name serverName: true|false ``` #### Session persistence You can use [Policy](/nic/configuration/policy-resource) resources or annotations for session persistence, which have corresponding annotations for the community Ingress-NGINX Controller. Session persistence via sticky cookie is available in both NGINX (since version 1.29.6) and NGINX Plus. Ingress-NGINX Controller: ```yaml nginx.ingress.kubernetes.io/affinity: "cookie" nginx.ingress.kubernetes.io/session-cookie-name: "cookieName" nginx.ingress.kubernetes.io/session-cookie-expires: "3600" nginx.ingress.kubernetes.io/session-cookie-path: "/route" nginx.ingress.kubernetes.io/session-cookie-secure: "true" ``` NGINX Ingress Controller: ```yaml sessionCookie: enable: true name: cookieName expires: 3h path: /route secure: true ``` ## Migration with Kubernetes Ingress resources The other option for migrating from the community Ingress-NGINX Controller to NGINX Ingress Controller is using only [annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) and [ConfigMaps](https://kubernetes.io/docs/concepts/configuration/configmap/) from standard Kubernetes resources, potentially relying on [mergeable Ingress types](https://github.com/nginx/kubernetes-ingress/tree/main/examples/ingress-resources/mergeable-ingress-types). This ensures that all configuration is kept in the Ingress object. **Note:** You should avoid altering the `spec` field of the Ingress resource when taking this option. Ingress-NGINX Controller and NGINX Ingress Controller differ slightly in their implementations: changing the Kubernetes Ingress can create incompatibility issues. ### Advanced configuration with annotations This table maps the Ingress-NGINX Controller annotations to NGINX Ingress Controller's equivalent annotations, and the respective NGINX Directive. | Ingress-NGINX Controller | NGINX Ingress Controller | NGINX Directive | | ------------------------ | ------------------------ | --------------- | | [_nginx.ingress.kubernetes.io/configuration-snippet_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#configuration-snippet) | [_nginx.org/location-snippets_](/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md#snippets-and-custom-templates) | N/A | | [_nginx.ingress.kubernetes.io/load-balance_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#custom-nginx-load-balancing) (1) | [_nginx.org/lb-method_](/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md#backend-services-upstreams) | [_random two least_conn_](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#random) | | [_nginx.ingress.kubernetes.io/proxy-buffering_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#proxy-buffering) | [_nginx.org/proxy-buffering_](/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md#general-customization) | [_proxy_buffering_](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffering) | | [_nginx.ingress.kubernetes.io/proxy-buffers-number_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#proxy-buffers-number) | [_nginx.org/proxy-buffers_](/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md#general-customization) | [_proxy_buffers_](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffers) | | [_nginx.ingress.kubernetes.io/proxy-buffer-size_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#proxy-buffer-size) | [_nginx.org/proxy-buffer-size_](/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md#general-customization) | [_proxy_buffer_size_](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffer_size) | | [_nginx.ingress.kubernetes.io/proxy-busy-buffers-size_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#proxy-busy-buffers-size) | [_nginx.org/proxy-busy-buffers-size_](/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md#general-customization) | [_proxy_busy_buffers_size_](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_busy_buffers_size) | | [_nginx.ingress.kubernetes.io/client-body-buffer-size_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#client-body-buffer-size) | [_nginx.org/client-body-buffer-size_](/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md#general-customization) | [_client_body_buffer_size_](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_buffer_size) | | [_nginx.ingress.kubernetes.io/proxy-connect-timeout_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#custom-timeouts) | [_nginx.org/proxy-connect-timeout_](/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md#general-customization) | [_proxy_connect_timeout_](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_connect_timeout) | | [_nginx.ingress.kubernetes.io/proxy-read-timeout_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#custom-timeouts) | [_nginx.org/proxy-read-timeout_](/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md#general-customization) | [_proxy_read_timeout_](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_read_timeout) | | [_nginx.ingress.kubernetes.io/proxy-next-upstream_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#custom-timeouts) | [_nginx.org/proxy-next-upstream_](/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md#general-customization) | [_proxy_next_upstream_](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream) | | [_nginx.ingress.kubernetes.io/proxy-next-upstream-timeout_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#custom-timeouts) | [_nginx.org/proxy-next-upstream-timeout_](/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md#general-customization) | [_proxy_next_upstream_timeout_](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream_timeout) | | [_nginx.ingress.kubernetes.io/proxy-next-upstream-tries_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#custom-timeouts) | [_nginx.org/proxy-next-upstream-tries_](/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md#general-customization) | [_proxy_next_upstream_tries_](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream_tries) | | [_nginx.ingress.kubernetes.io/proxy-send-timeout_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#custom-timeouts) | [_nginx.org/proxy-send-timeout_](/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md#general-customization) | [_proxy_send_timeout_](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_send_timeout) | | [_nginx.ingress.kubernetes.io/rewrite-target_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#rewrite) | [_nginx.org/rewrite-target_](/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md#request-uriheader-manipulation) | [_rewrite_](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#rewrite) | | [_nginx.ingress.kubernetes.io/app-root_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#rewrite) | [_nginx.org/app-root_](/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md#request-uriheader-manipulation) | N/A | | [_nginx.ingress.kubernetes.io/ssl-ciphers_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#ssl-ciphers) | [_nginx.org/ssl-ciphers_](/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md#auth-and-ssltls) | [_ssl_ciphers_](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_ciphers) | | [_nginx.ingress.kubernetes.io/ssl-prefer-server-ciphers_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#ssl-ciphers) | [_nginx.org/ssl-prefer-server-ciphers_](/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md#auth-and-ssltls) | [_ssl_prefer_server_ciphers_](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_prefer_server_ciphers) | | [_nginx.ingress.kubernetes.io/server-snippet_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#server-snippet)| [_nginx.org/server-snippets_](/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md#snippets-and-custom-templates) | N/A | | [_nginx.ingress.kubernetes.io/ssl-redirect_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#server-side-https-enforcement-through-redirect) (2) | [_nginx.org/ssl-redirect_](/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md#auth-and-ssltls) | N/A | | [_nginx.ingress.kubernetes.io/affinity_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#session-affinity)| [_nginx.org/sticky-cookie-services_](/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md#backend-services-upstreams) | [_sticky_](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#sticky_cookie)| 1. Ingress-NGINX Controller implements some of its load balancing algorithms with Lua, which may not have an equivalent in NGINX Ingress Controller. 1. To redirect HTTP (80) traffic to HTTPS (443), NGINX Ingress Controller uses built-in NGINX `if` conditions while Ingress-NGINX Controller uses Lua. For [_nginx.ingress.kubernetes.io/force-ssl-redirect_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#server-side-https-enforcement-through-redirect) behavior (which works when SSL is terminated at an external load balancer), use [_nginx.org/redirect-to-https_](/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md#auth-and-ssltls). The following two snippets outline Ingress-NGINX Controller annotations that correspond to annotations for NGINX Ingress Controller. Ingress-NGINX Controller: ```yaml nginx.ingress.kubernetes.io/affinity: "cookie" nginx.ingress.kubernetes.io/session-cookie-name: "cookie_name" nginx.ingress.kubernetes.io/session-cookie-expires: "seconds" nginx.ingress.kubernetes.io/session-cookie-path: "/route" ``` NGINX Ingress Controller: ```yaml nginx.org/sticky-cookie-services: "serviceName=example-svc cookie_name expires=time path=/route" ``` **Note:** NGINX Ingress Controller has additional annotations for features using NGINX Plus that have no Ingress-NGINX Controller equivalent, such as active health checks and authentication using JSON Web Tokens (JWTs). The following table outlines annotation conversions for enabling TLS and gRPC protocols to backend (upstream) services. | Ingress-NGINX Controller | NGINX Ingress Controller | | ------------------------ | ------------------------ | | [_nginx.ingress.kubernetes.io/backend-protocol_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#backend-protocol): "HTTPS" | [_nginx.org/ssl-services_](/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md#backend-services-upstreams): "ssl_service_name" | | [_nginx.ingress.kubernetes.io/backend-protocol_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#backend-protocol): "GRPC" | [_nginx.org/grpc-services_](/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md#backend-services-upstreams): "grpc_service_name" | | [_nginx.ingress.kubernetes.io/backend-protocol_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#backend-protocol): "GRPCS" | [_nginx.org/ssl-services_](/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md#backend-services-upstreams): "grpc_service_name"
[_nginx.org/grpc-services_](/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md#backend-services-upstreams): "grpc_service_name" | **Note:** GRPC services require Ingresses with TLS termination and HTTP/2 enabled (see [_http2 ConfigMap key_](/nic/configuration/global-configuration/configmap-resource.md#listeners)). ### Global configuration with ConfigMaps This table maps the Ingress-NGINX Controller ConfigMap keys to NGINX Ingress Controller's equivalent ConfigMap keys. | Ingress-NGINX Controller | NGINX Ingress Controller | | ------------------------ | ------------------------ | | [_disable-access-log_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#disable-access-log) | [_access-log-off_](/nic/configuration/global-configuration/configmap-resource.md#logging) | | [_error-log-level_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#error-log-level) | [_error-log-level_](/nic/configuration/global-configuration/configmap-resource.md#logging) | | [_hsts_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#hsts) | [_hsts_](/nic/configuration/global-configuration/configmap-resource.md#auth-and-ssltls) | | [_hsts-include-subdomains_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#hsts-include-subdomains) | [_hsts-include-subdomains_](/nic/configuration/global-configuration/configmap-resource.md#auth-and-ssltls) | | [_hsts-max-age_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#hsts-max-age) | [_hsts-max-age_](/nic/configuration/global-configuration/configmap-resource.md#auth-and-ssltls) | | [_http-snippet_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#http-snippet) | [_http-snippets_](/nic/configuration/global-configuration/configmap-resource.md#snippets-and-custom-templates) | | [_keep-alive_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#keep-alive) | [_keepalive-timeout_](/nic/configuration/global-configuration/configmap-resource.md#general-customization) | | [_keep-alive-requests_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#keep-alive-requests) | [_keepalive-requests_](/nic/configuration/global-configuration/configmap-resource.md#general-customization) | | [_load-balance_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#load-balance) | [_lb-method_](/nic/configuration/global-configuration/configmap-resource.md#backend-services-upstreams) | | [_location-snippet_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#location-snippet) | [_location-snippets_](/nic/configuration/global-configuration/configmap-resource.md#snippets-and-custom-templates) | | [_log-format-escape-json_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#log-format-escape-json) | [_log-format-escaping: "json"_](/nic/configuration/global-configuration/configmap-resource.md#logging) | | [_log-format-stream_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#log-format-stream) | [_stream-log-format_](/nic/configuration/global-configuration/configmap-resource.md#logging) | | [_log-format-upstream_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#log-format-upstream) | [_log-format_](/nic/configuration/global-configuration/configmap-resource.md#logging) | | [_main-snippet_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#main-snippet) | [_main-snippets_](/nic/configuration/global-configuration/configmap-resource.md#snippets-and-custom-templates) | | [_max-worker-connections_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#max-worker-connections) | [_worker-connections_](/nic/configuration/global-configuration/configmap-resource.md#general-customization) | | [_max-worker-open-files_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#max-worker-open-files) | [_worker-rlimit-nofile_](/nic/configuration/global-configuration/configmap-resource.md#general-customization) | | [_proxy-body-size_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#proxy-body-size) | [_client-max-body-size_](/nic/configuration/global-configuration/configmap-resource.md#general-customization) | | [_client-body-buffer-size_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#client-body-buffer-size) | [_client-body-buffer-size_](/nic/configuration/global-configuration/configmap-resource.md#general-customization) | | [_proxy-buffering_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#proxy-buffering) | [_proxy-buffering_](/nic/configuration/global-configuration/configmap-resource.md#general-customization) | | [_proxy-buffers-number_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#proxy-buffers-number) | [_proxy-buffers: number size_](/nic/configuration/global-configuration/configmap-resource.md#general-customization) | | [_proxy-buffer-size_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#proxy-buffer-size) | [_proxy-buffers: number size_](/nic/configuration/global-configuration/configmap-resource.md#general-customization) | | [_proxy-connect-timeout_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#proxy-connect-timeout) | [_proxy-connect-timeout_](/nic/configuration/global-configuration/configmap-resource.md#general-customization) | | [_proxy-read-timeout_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#proxy-read-timeout) | [_proxy-read-timeout_](/nic/configuration/global-configuration/configmap-resource.md#general-customization) | | [_proxy-send-timeout_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#proxy-send-timeout) | [_proxy-send-timeout_](/nic/configuration/global-configuration/configmap-resource.md#general-customization) | | [_server-name-hash-bucket-size_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#server-name-hash-bucket-size) | [_server-names-hash-bucket-size_](/nic/configuration/global-configuration/configmap-resource.md#general-customization) | | [_proxy-headers-hash-max-size_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#proxy-headers-hash-max-size) | [_server-names-hash-max-size_](/nic/configuration/global-configuration/configmap-resource.md#general-customization) | | [_server-snippet_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#server-snippet) | [_server-snippets_](/nic/configuration/global-configuration/configmap-resource.md#snippets-and-custom-templates) | | [_server-tokens_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#server-tokens) | [_server-tokens_](/nic/configuration/global-configuration/configmap-resource.md#general-customization) | | [_ssl-ciphers_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#ssl-ciphers) | [_ssl-ciphers_](/nic/configuration/global-configuration/configmap-resource.md#auth-and-ssltls) | | [_ssl-dh-param_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#ssl-dh-param) | [_ssl-dhparam-file_](/nic/configuration/global-configuration/configmap-resource.md#auth-and-ssltls) | | [_ssl-protocols_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#ssl-protocols) | [_ssl-protocols_](/nic/configuration/global-configuration/configmap-resource.md#auth-and-ssltls) | | [_ssl-redirect_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#ssl-redirect) | [_ssl-redirect_](/nic/configuration/global-configuration/configmap-resource.md#auth-and-ssltls) | | [_upstream-keepalive-connections_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#upstream-keepalive-connections) | [_keepalive_](/nic/configuration/global-configuration/configmap-resource.md#backend-services-upstreams) | | [_use-http2_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#use-http2) | [_http2_](/nic/configuration/global-configuration/configmap-resource.md#listeners) | | [_use-proxy-protocol_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#use-proxy-protocol) | [_proxy-protocol_](/nic/configuration/global-configuration/configmap-resource.md#listeners) | | [_variables-hash-bucket-size_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#variables-hash-bucket-size) | [_variables-hash-bucket-size_](/nic/configuration/global-configuration/configmap-resource.md#general-customization) | | [_worker-cpu-affinity_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#worker-cpu-affinity) | [_worker-cpu-affinity_](/nic/configuration/global-configuration/configmap-resource.md#general-customization) | | [_worker-processes_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#worker-processes) | [_worker-processes_](/nic/configuration/global-configuration/configmap-resource.md#general-customization) | | [_worker-shutdown-timeout_](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#worker-shutdown-timeout) | [_worker-shutdown-timeout_](/nic/configuration/global-configuration/configmap-resource.md#general-customization) | -------------------------------------------------------------------------------- title: "Configure remote F5 WAF for NGINX bundle sources" url: https://docs.nginx.com/nginx-ingress-controller/integrations/app-protect-waf-v5/bundle-sources/index.md description: Configure NGINX Ingress Controller to fetch F5 WAF for NGINX policy bundles from NGINX One Console, NGINX Instance Manager, or an HTTPS server. -------------------------------------------------------------------------------- ## Overview This guide explains how to configure NGINX Ingress Controller to fetch pre-compiled F5 WAF for NGINX policy bundles from a remote source for VirtualServer resources, instead of manually copying bundles on to the cluster. This guide focuses on source-specific configuration details and validation steps. You can fetch bundles from: - **[NGINX One Console](/nginx-one-console/)** — for policies compiled and managed through NGINX One Console - **[NGINX Instance Manager](/nim/)** — for policies compiled and managed through NGINX Instance Manager - **[HTTPS](#https)** — for compiled `.tgz` bundles hosted on any HTTPS server Complete end-to-end NGINX Ingress Controller with F5 WAF for NGINX bundle source examples are available on GitHub: [N1C and NIM examples](https://github.com/nginx/kubernetes-ingress/tree/v/examples/custom-resources/waf-management-plane) and [HTTPS bundle server files](https://github.com/nginx/kubernetes-ingress/tree/v/examples/shared-examples/waf-bundle-server). ## NGINX One Console ### Before you begin - NGINX Ingress Controller deployed with [F5 WAF for NGINX v5](/nic/integrations/app-protect-waf-v5/installation.md). You can also [install with Helm](/nic/install/waf-helm.md). - An [NGINX One Console](/nginx-one-console/) account with a published WAF policy. See [Manage policies](/nginx-one-console/waf-integration/policy/_index.md). - A VirtualServer resource to attach the WAF policy to. **Note:** NGINX Ingress Controller does not trigger compilation. Compilation happens when a policy is published in NGINX One Console. Ensure the policy has been published and a compiled bundle is available before continuing. ### Create a credentials Secret Create a Secret of type `nginx.com/waf-bundle` in the same namespace as the Policy. The Secret must contain a `token` key with your NGINX One Console API token: To create an API token, see [Authentication](/nginx-one-console/api/authentication.md). ```shell kubectl create secret generic n1c-credentials \ --type=nginx.com/waf-bundle \ --from-literal=token= ``` ### Create a WAF Policy Create a Policy resource using `apBundleSource` with `type: N1C`: ```yaml apiVersion: k8s.nginx.org/v1 kind: Policy metadata: name: waf-policy spec: waf: enable: true apBundleSource: type: N1C url: "https://.console.ves.volterra.io" policyName: "my-blocking-policy" policyNamespace: "default" secret: "n1c-credentials" enablePolling: true pollInterval: "5m" ``` Replace `` with your NGINX One Console tenant hostname, `policyName` with the name of your published policy, and `policyNamespace` with the NGINX One Console namespace where the policy resides. **Note:** The field name is `policyName` for both `apBundleSource` and `apLogBundleSource`. In `apBundleSource`, set it to the published WAF policy name. In `apLogBundleSource`, set it to the log profile name (for example, `secops_dashboard`). **Note:** To skip TLS verification for testing, add `insecureSkipVerify: true` to the bundle source. Do not use this in production. ### Apply the policy to a VirtualServer After `waf-policy` is created, apply a VirtualServer that references it in `spec.policies`. ```yaml apiVersion: k8s.nginx.org/v1 kind: VirtualServer metadata: name: webapp spec: host: webapp.example.com policies: - name: waf-policy upstreams: - name: webapp service: webapp-svc port: 80 routes: - path: / action: pass: webapp ``` For complete HTTPS setup manifests, see the [bundle server files](https://github.com/nginx/kubernetes-ingress/tree/v/examples/shared-examples/waf-bundle-server). ### Verify the bundle was fetched 1. Check the Policy events for a successful fetch: ```shell kubectl describe policy waf-policy ``` Look for a `Normal` event confirming the bundle was fetched. If you see a `Warning` event, check the message for the cause — common issues include an incorrect `policyName`, an invalid token, or a policy that has not been published yet. 1. Send a legitimate request to confirm traffic flows normally: ```shell curl --resolve webapp.example.com:$IC_HTTP_PORT:$IC_IP \ http://webapp.example.com:$IC_HTTP_PORT/ ``` 1. Send a malicious request to confirm WAF is blocking: ```shell curl --resolve webapp.example.com:$IC_HTTP_PORT:$IC_IP \ "http://webapp.example.com:$IC_HTTP_PORT/" # Path traversal test curl -G "https://app.example.com/" --data-urlencode "file=../../../../etc/passwd" # SQL injection test curl -G "https://app.example.com/" --data-urlencode "id=1' UNION SELECT 1,2,3--" ``` If your policy is in transparent mode, if signatures are staged, or if you heavily customized the default policy, these requests may appear as `alerted` instead of `blocked`. The dashboard still confirms that the security event pipeline is working. For details on how the dashboard is organized and how to read each widget, see the [security dashboard reference](/nginx-one-console/waf-integration/waf-security-dashboard/dashboard-metrics-reference.md). --- ## Troubleshooting ### Publish fails with a configuration validation error **Symptom**: When you select **Publish** in the deployment wizard, NGINX One Console reports a configuration validation error such as `unknown directive "app_protect_enable"`, `unknown directive "app_protect_security_log"`, or a parser error referencing the WAF directives. **Cause**: One or more of the F5 WAF for NGINX prerequisites is not in place on the instance — typically the `load_module` line is missing from `nginx.conf`, `app_protect_enable on;` is not set in the context where the security log directive was pasted, or `app_protect_policy_file` references a path the data plane cannot resolve. **Fix**: Re-check the items in [Before you begin](#before-you-begin): - Confirm `load_module modules/ngx_http_app_protect_module.so;` is present in the main context of `nginx.conf`. See [Update configuration files](/waf/install/virtual-environment.md#update-configuration-files). - Confirm `app_protect_enable on;` is set in the same `server`, `http`, or `location` context where you pasted the security log snippet. - Confirm an `app_protect_policy_file` directive references a policy already deployed to the instance. See [WAF policies](/nginx-one-console/waf-integration/policy/_index.md). Re-run the deployment wizard after fixing the configuration. ### Publish succeeded but no events appear in the dashboard **Symptom**: The publish toast confirmed success, the instance is online in NGINX One Console, but the **WAF > Security Dashboard** shows no events for your instance. **Cause**: The most common causes are that NGINX Agent on the instance is older than 3.9.0 and does not include the auto-configured security event pipeline, the `secops_dashboard` log profile is not deployed to that instance, the `app_protect_security_log` directive is in a context that does not handle traffic, or the instance has not yet processed any requests F5 WAF for NGINX would inspect. **Fix**: 1. Confirm NGINX Agent on the instance is **3.9.0 or later**. Earlier 3.x releases publish the configuration successfully but do not forward security events to NGINX One Console. See the [NGINX Agent install and upgrade guide](/nginx-one-console/agent/install-upgrade/_index.md) to upgrade. 2. Go to **WAF** > **Log Profiles** and confirm `secops_dashboard` is listed as deployed to the target instance under **Deployed To**. 3. Open the instance configuration and confirm the `app_protect_security_log` directive sits in a `server` or `location` block that actually handles request traffic — not in a context the data plane never enters. 4. Confirm the instance is receiving traffic. Until F5 WAF for NGINX inspects a request, the dashboard has nothing to display. 5. Apply a global filter on the dashboard to scope to your instance hostname or policy, in case events are present but hidden by an existing filter. If events still do not appear after a request is processed, contact F5 support with the instance hostname and the time window you tested. For local data plane checks of the embedded OpenTelemetry Collector, generated collector pipeline, and debug forwarding, see [Troubleshoot security monitoring on the local data plane](/nginx-one-console/waf-integration/waf-security-dashboard/local-dataplane-troubleshooting.md). --- ## References **Conceptual background** - [Security monitoring overview](/nginx-one-console/waf-integration/waf-security-dashboard/security-monitoring-overview.md) **Reference** - [secops_dashboard log profile](/nginx-one-console/waf-integration/waf-security-dashboard/default-log-profile.md) - [Dashboard metrics reference](/nginx-one-console/waf-integration/waf-security-dashboard/dashboard-metrics-reference.md) - [Troubleshoot security monitoring on the local data plane](/nginx-one-console/waf-integration/waf-security-dashboard/local-dataplane-troubleshooting.md) **Related how-to guides** - [Deploy log profiles](/nginx-one-console/waf-integration/log-profiles/deploy-log-profiles.md) - [Add an instance](/nginx-one-console/connect-instances/add-instance.md) - [Find a security event by Support ID](/nginx-one-console/waf-integration/waf-security-dashboard/find-event-by-support-id.md) - [Query security events through the API](/nginx-one-console/api/query-events-api.md) -------------------------------------------------------------------------------- title: "Use the API to manage your Staged Configurations" url: https://docs.nginx.com/nginx-one-console/nginx-configs/staged-configs/api-staged-config/index.md -------------------------------------------------------------------------------- You can use F5 NGINX One Console API to manage your Staged Configurations. With our API, you can: - [Create an NGINX Staged Configuration](/nginx-one-console/api/api-reference-guide/#operation/createStagedConfig) - The details allow you to add existing configuration files. - [Get a list of existing Staged Configurations](/nginx-one-console/api/api-reference-guide/#operation/listStagedConfigs) - Be sure to record the `object_id` of your target Staged Configuration for your analysis report. - [Get an analysis report for an existing Staged Configuration](/nginx-one-console/api/api-reference-guide/#operation/getStagedConfigReport) -------------------------------------------------------------------------------- title: "Review and configure features" url: https://docs.nginx.com/nginx-one-console/agent/configure-instances/configure-agent-features/index.md -------------------------------------------------------------------------------- ## Overview This guide describes the F5 NGINX Agent features, and how to enable and disable them using the NGINX Agent configuration file, CLI flags, environment variables, and gRPC updates. ## Before you begin Before you start, make sure that you have: - [NGINX Agent installed](/nginx-one-console/agent/install-upgrade/) in your system. - Access to the NGINX Agent configuration file, CLI, or container environment. ## Features The following table lists the NGINX Agent features: | Feature Name | Description | Default | | ------------------- | --------------------------------------------------------------------------- | ------- | | configuration | Full read/write management of configurations, controlled by DataPlaneConfig ConfigMode. | On | | certificates | Inclusion of public keys and other certificates in the configurations toggled by DataPlaneConfig CertMode | On | | file-watcher | Monitoring of file changes in the allowed directories list and references from product configs. | On | | metrics | Full metrics reporting. | On | logs-nap | F5 WAF for NGINX logs. | On | ## Configuration sources You can enable or disable features using several configuration sources: ### CLI parameters Enable features at launch: ```shell ./nginx-agent --features=configuration,metrics,file-watcher ``` ### Environment variables Use environment variables for containerized deployments: ```shell export NGINX_AGENT_FEATURES="configuration,metrics,file-watcher" ``` ### Configuration file Define features in the `nginx-agent.conf` file: ```yaml features: - configuration - metrics - file-watcher ``` ## Use cases ### Enable metrics only 1. Access the NGINX instance: Connect using SSH to the VM or server where NGINX Agent is running. `ssh user@your-nginx-instance` 1. Open the NGINX Agent configuration file in a text editor. `sudo vim /etc/nginx-agent/nginx-agent.conf` 1. Add the features section: Add the following to the end of the configuration file if it doesn't already exist. ```yaml features: - metrics ``` 1. Restart the NGINX Agent service to apply the changes. `sudo systemctl restart nginx-agent` Once the steps have been completed, users will be able to view metrics data being sent but will not have the capability to push NGINX configuration changes. -------------------------------------------------------------------------------- title: "Add a file to a Config Sync Group" url: https://docs.nginx.com/nginx-one-console/nginx-configs/config-sync-groups/add-file-csg/index.md -------------------------------------------------------------------------------- ## Overview This guide explains how to add files in the F5 NGINX One Console. While you can manage files in the CLI, the NGINX One Console supports editing in a UI that resembles an Integrated Development Environment (IDE), with recommendations. ## Before you start Before you add files in your configuration, ensure: - You have access to the NGINX One Console. - Config Sync Groups are properly registered with NGINX One Console ## Important considerations This page applies when you want to add a file to a Config Sync Group. Any changes you make here apply to all [Instances](/nginx-one-console/glossary.md) of that Config Sync Group. ## Add a file You can use the NGINX One Console to add a file to a specific Config Sync Group. To do so: 1. Select the Config Sync Group to manage. 1. Select the **Configuration** tab. **Note:** From this window, select the file of your choice. If you want to delete this file, Select **Edit Configuration** and select the Trash icon. If this was a mistake, a revert button appears. But do not wait. As noted in one of the UI messages, "This action cannot be undone once you publish the configuration." 1. Select **Edit Configuration**. 1. In the **Edit Configuration** window that appears, select **Add File**. You now have multiple options, described in the sections which follow. ### New Configuration File Enter the name of the desired configuration file, such as `abc.conf` and select **Add**. The configuration file appears in the **Edit Configuration** window. ### New SSL Certificate or CA Bundle First you can select the toggle to allow NGINX One Console to manage the new certificate or bundle. In the screen that appears, you can add a certificate name. If you don't add a name, NGINX One Console will add a name for you, based on the expiration date for the certificate. You can add certificates in the following formats: - **SSL Certificate and Key** - **CA Certificate Bundle** In each case, you can upload files directly, or enter the content of the certificates in a text box. Once you upload these certificates, you may need to scroll down. You'll see: - **Certificate Details**, with the Subject Name, start and end dates. - **Key Details**, with the encryption key size and algorithm, such as RSA Select **Save and Continue**. You're taken to another screen where you can specify the locations for your files, which may be: - **Certificate File Path** - Enter the full path to your certificate, such as - /etc/nginx/server.crt - /etc/nginx/server.pem - **Key File Path** - Enter the full path to your certificate key, such as - /etc/nginx/server.key With the **Add Item** button, you can add the file to additional directories. When complete, select **Add** to include the certificate files that you've configured to desired directories. **Note:** Make sure to specify the path to your certificate in your NGINX configuration, with the `ssl_certificate` and `ssl_certificate_key` directives. ### Existing SSL Certificate or CA Bundle With this option, you can incorporate [Managed certificates](/nginx-one-console/nginx-configs/certificates/manage-certificates.md#managed-and-unmanaged-certificates). In the **Choose Certificate** drop-down, select the managed certificate of your choice, and select **Add**. You can then: 1. Review details of the certificate. The next steps depend on whether the certificate is a CA bundle or a certificate / key pair. 1. Enter the **Certificate File Path**, such as `/etc/ssl/nginx/mycert.crt` or `/etc/ssl/nginx/mycert.pem`. 1. If you selected a key pair, you'll also enter the **Key File Path**, such as `/etc/ssl/nginx/mycert.key`. 1. If you select **Add Item**, you can add the same certificate or key to another directory. 1. Select **Add**. You should now be returned to the **Edit Configuration** window. You should now see the files you specified in the directory tree. 1. Select **Next** and then **Save and Publish**. You may see a message that suggests publication is in progress. 1. When publication is complete, you're taken back to the **Configuration** tab. You should see the updated configuration in the window. With this option, you can incorporate [Managed certificates](/nginx-one-console/nginx-configs/certificates/manage-certificates.md#managed-and-unmanaged-certificates). ### Existing Log Profile With this option, you can deploy an existing log profile that you created in NGINX One Console. In the **Select a Log Profile** drop-down menu, select the log profile of your choice. Then take the following steps: 1. In **Log Profile Destination**, specify the file path where the log profile bundle should be deployed, such as `/etc/app_protect/conf/log_default.tgz`. 1. Select **Add**. NGINX One Console displays a code snippet for using the log profile bundle in your NGINX configuration. 1. Paste the code snippet into your NGINX configuration. The snippet includes the required directives as described in the [WAF logging documentation](https://docs.nginx.com/waf/logging/security-logs/#directives-in-nginxconf): - `app_protect_security_log_enable on` - `app_protect_security_log` with the log profile bundle path and destination 1. Select **Next** and then **Save and Publish**. When publication is complete, you'll be returned to the **Configuration** tab where you can see the updated configuration. ## See also - [Create and manage data plane keys](/nginx-one-console/connect-instances/create-manage-data-plane-keys.md) - [Add an NGINX instance](/nginx-one-console/connect-instances/add-instance.md) - [Manage certificates](/nginx-one-console/nginx-configs/certificates/manage-certificates.md) -------------------------------------------------------------------------------- title: "Add cookies, parameters and URLs" url: https://docs.nginx.com/nginx-one-console/waf-integration/policy/cookies-params-urls/index.md -------------------------------------------------------------------------------- ## Add cookies Cookie protections can be configured and managed directly within the policy editor by selecting the **Cookies** option. ## Cookie properties and types Each cookie configuration includes: - `Cookie Type`: `Explicit` or `Wildcard`. For details on explicit and wildcard matching, see the [Matching Types: Explicit vs Wildcard](/nginx-one-console/waf-integration/policy/waf-policy-matching-types.md) section. - `Cookie Name`: The name of the cookie to be monitored or protected - `Enforcement Type`: - **Allow**: Specifies that this cookie may be changed by the client. The cookie is not protected from modification - **Enforce**: Specifies that this cookie may not be changed by the client - `Attack Signatures`: Indicates whether attack signatures and threat campaigns are enabled, disabled, or not applicable - `Mask value in logs`: When enabled, the cookie's value will be masked in the request log for enhanced security and privacy For a complete list of configurable cookie properties and options, see the [Cookie Configuration Parameters](/waf/policies/parameter-reference.md) documentation under the `cookies` section. ## Cookie violations Select **Edit Configuration** to configure cookie violations. The following violations can be configured for cookies: - `VIOL_COOKIE_EXPIRED`: Triggered when a cookie's timestamp is expired - `VIOL_COOKIE_LENGTH`: Triggered when cookie length exceeds the configured limit - `VIOL_COOKIE_MALFORMED`: Triggered when cookies are not RFC-compliant - `VIOL_COOKIE_MODIFIED`: Triggered when domain cookies have been tampered with For each violation type, you can: - Set the enforcement action - Toggle `Alarm`, `Alarm and Block`, or `Disabled` settings For more details about enforcement modes, see the [Glossary](/nginx-one-console/glossary.md#nginx-app-protect-waf-terminology), specifically the entry: **Enforcement mode**. See the [Supported Violations](/waf/policies/violations.md#supported-violations) for additional details. ## Add a cookie to your policy 1. Choose Cookie Type: - Select either `Explicit` for exact cookie matching or `Wildcard` for pattern-based matching 1. Configure Basic Properties: - Enter the `Cookie Name` - Choose whether to mask the cookie value in logs 1. Set Enforcement Type: - Choose either `Allow` or `Enforce` 1. Optional: Configure Attack Signatures - If enabled, you can overwrite attack signatures for this specific cookie - For details on signature configuration, refer to the documentation on [Add Signature Sets](/nginx-one-console/waf-integration/policy/add-signature-sets.md) 1. Select **Add Cookie** to save your configuration # Add parameters Parameter protections can be configured and managed directly within the policy editor by selecting the **Parameters** option. ## Parameter properties and types Each parameter configuration includes: - `Parameter Type`: `Explicit` or `Wildcard`. For details on explicit and wildcard matching, see the [Matching Types: Explicit vs Wildcard](/nginx-one-console/waf-integration/policy/waf-policy-matching-types.md) section. - `Parameter Name`: The name of the parameter - `Location`: Where the parameter is expected (URL query string, POST data, etc.) - `Value Type`: The expected type of the parameter value (e.g., alpha-numeric, integer, email) - `Attack Signatures`: Whether attack signature checking is enabled for this parameter - `Mask value in logs`: When enabled, the parameter's value will be masked in the request log for enhanced security and privacy. This sets `sensitiveParameter` property of the parameter item. For a complete list of configurable parameter properties and options, see the [Parameter Configuration Parameters](/waf/policies/parameter-reference.md) documentation under the `parameters` section. ## Parameter violations Select **Edit Configuration** to configure parameter violations. The following violations can be configured for parameters: - `VIOL_PARAMETER`: Triggered when an illegal parameter is detected - `VIOL_PARAMETER_ARRAY_VALUE`: Triggered when an array parameter value is illegal - `VIOL_PARAMETER_DATA_TYPE`: Triggered when parameter data type doesn't match configured security policy - `VIOL_PARAMETER_EMPTY_VALUE`: Triggered when a parameter value is empty but shouldn't be - `VIOL_PARAMETER_LOCATION`: Triggered when a parameter is found in wrong location - `VIOL_PARAMETER_MULTIPART_NULL_VALUE`: Triggered when the multi-part request has a parameter value that contains the NULL character (0x00) - `VIOL_PARAMETER_NAME_METACHAR`: Triggered when illegal meta characters are found in parameter name - `VIOL_PARAMETER_NUMERIC_VALUE`: Triggered when numeric parameter value is outside allowed range - `VIOL_PARAMETER_REPEATED`: Triggered when a parameter name is repeated illegally - `VIOL_PARAMETER_STATIC_VALUE`: Triggered when a static parameter value doesn't match configured security policy - `VIOL_PARAMETER_VALUE_BASE64`: Triggered when the value is not a valid Base64 string - `VIOL_PARAMETER_VALUE_LENGTH`: Triggered when parameter value length exceeds limits - `VIOL_PARAMETER_VALUE_METACHAR`: Triggered when illegal meta characters are found in parameter value - `VIOL_PARAMETER_VALUE_REGEXP`: Triggered when parameter value doesn't match required pattern For each violation type, you can: - Set the enforcement action - Toggle `Alarm`, `Alarm and Block`, or `Disabled` settings For more details about enforcement modes, see the [Glossary](/nginx-one-console/glossary.md#nginx-app-protect-waf-terminology), specifically the entry: **Enforcement mode**. See the [Supported Violations](/waf/policies/violations.md#supported-violations) for additional details. ## Add a parameter to your policy 1. Choose Parameter Type: - Select either `Explicit` for exact parameter matching or `Wildcard` for pattern-based matching 1. Configure Basic Properties: - Enter the parameter `Parameter Name` - Select the `Location` where the parameter is expected - Choose the `Value Type` (alpha-numeric, integer, email, etc.) - Set the `Data Type` if applicable 1. Set Security Options: - Choose whether to enable attack signatures **Note:** Attack Signatures are only applicable when the Value Type is `User Input` or `Array` **and** the Data Type is either `Alphanumeric` or `Binary` - Decide if parameter value should be masked in logs which sets `sensitiveParameter` in [Parameter Configuration Reference](/waf/policies/parameter-reference.md) 1. Optional: Configure Attack Signatures - If enabled, you can overwrite attack signatures for this specific parameter - For details on signature configuration, refer to the documentation on [Add Signature Sets](/nginx-one-console/waf-integration/policy/add-signature-sets.md) 1. Select **Add Parameter** to save your configuration ## Add URLs URL protections can be configured and managed directly within the policy editor by selecting the **URLs** option. ## URL properties and types Each URL configuration includes: - `URL Type`: `Explicit` or `Wildcard`. For details on explicit and wildcard matching, see the [Matching Types: Explicit vs Wildcard](/nginx-one-console/waf-integration/policy/waf-policy-matching-types.md) section. - `Method`: Specifies the HTTP method(s) for the URL (`GET`, `POST`, `PUT`, etc.) - `Protocol`: The protocol for the URL (`HTTP`/`HTTPS`) - `Enforcement Type`: - **Allow**: Permits access to the URL with optional attack signature checks - **Disallow**: Blocks access to the URL entirely - `Attack Signatures`: Indicates whether attack signatures and threat campaigns are enabled, disabled, or not applicable **Note:** **⚠️ Important:** Attack Signatures are automatically shown as "Not Applicable" when Enforcement Type is set to `Disallow` since the URL is explicitly blocked and signature checking is unnecessary. For a complete list of configurable URL properties and options, see the [URL Configuration Parameters](/waf/policies/parameter-reference.md) documentation under the `urls` section. ## URL violations Select **Edit Configuration** to configure URL violations. The following violations can be configured for URLs: - `VIOL_URL`: Triggered when an illegal URL is accessed - `VIOL_URL_CONTENT_TYPE`: Triggered when there's an illegal request content type - `VIOL_URL_LENGTH`: Triggered when URL length exceeds the configured limit - `VIOL_URL_METACHAR`: Triggered when illegal meta characters are found in the URL For each violation type, you can: - Set the enforcement action - Toggle `Alarm`, `Alarm and Block`, or `Disabled` settings For more details about enforcement modes, see the [Glossary](/nginx-one-console/glossary.md#nginx-app-protect-waf-terminology), specifically the entry: **Enforcement mode**. See the [Supported Violations](/waf/policies/violations.md#supported-violations) for additional details. ## Add a URL to your policy 1. Choose URL Type: - Select either `Explicit` for exact URL matching or `Wildcard` for pattern-based matching 1. Configure Basic Properties: - Enter the `URL` path (e.g., `/index.html`, `/api/data`) - The URL path must start with `/` - Select HTTP `Method(s)` (e.g., `GET`, `POST`, *) - Choose the `Protocol` (`HTTP`/`HTTPS`) 1. Set Enforcement: - Choose whether to allow or disallow the URL - If `Allow URL` is selected, you can optionally enable attack signatures **Note:** **⚠️ Important:** Attack signatures cannot be enabled for disallowed URLs. 1. **Optional**: Configure Attack Signatures - If enabled, you can overwrite attack signatures for this specific URL - For details on signature configuration, refer to the documentation on [Add Signature Sets](/nginx-one-console/waf-integration/policy/add-signature-sets.md) 1. Select **Add URL** to save your configuration -------------------------------------------------------------------------------- title: "Add service-specific locations" url: https://docs.nginx.com/nginx-one-console/nginx-configs/config-templates/add-multiple-services/index.md -------------------------------------------------------------------------------- # Overview This guide shows how to extend a working submission from the [Submit Templates Guide](submit-templates.md) by adding server augments for new services with dedicated location augments. **Note:** Submissions are persistent objects. Use the [Get a submission](submit-templates.md#get-a-submission) operation to retrieve the stored values from a previous submission before building an update. ## Import template If there aren't existing augment templates that fit your needs, you will need to create and import them. ### New server augment template Because there isn’t a server augment yet, you need to create one. This template creates dedicated server blocks for each service. `http-server.tmpl` ```text server { listen {{ .listen_port }}; server_name{{ range .server_names }} {{ . }}{{ end }}; {{ augment_includes "http/server/location" . }} } ``` `schema.yaml` ```text $schema: "http://json-schema.org/draft-07/schema#" type: object properties: listen_port: type: integer description: "Port for the server to listen on" minimum: 1 maximum: 65535 default: 80 server_names: type: array description: "Array of domain names for this server" items: type: string minItems: 1 required: - listen_port - server_names additionalProperties: false ``` #### Import parameters When [Importing](import-templates.md#ready-to-import) this template allows you to set up: - name: `http-server` - type: `augment` - allowed_in_contexts: `["http"]` ### New location augment template Create a location augment template to add location blocks within each server. **Note:** If you already have a "health-check" location augment from earlier steps, you can add it to the new servers. `location-proxy.tmpl` ```text location {{ .path }} { proxy_pass {{ .upstream_url }}; proxy_connect_timeout {{ .proxy_timeout }}; proxy_read_timeout {{ .proxy_read_timeout }}; } ``` `schema.yaml` ```text $schema: "http://json-schema.org/draft-07/schema#" type: object properties: path: type: string description: "Location path (for example, /api, /admin)" pattern: "^/.*$" upstream_url: type: string description: "Backend service URL" pattern: "^https?://[^\\s]+$" proxy_timeout: type: string description: "Proxy connection timeout" pattern: "^\\d+[smhd]?$" default: "30s" proxy_read_timeout: type: string description: "Proxy read timeout" pattern: "^\\d+[smhd]?$" default: "60s" required: - path - upstream_url additionalProperties: false ``` #### Import parameters When [Importing](import-templates.md#ready-to-import) this template allows you to: - name: `location-proxy` - type: `augment` - allowed_in_contexts: `["http/server/location"]` ## Submit to add multiple services ### Request structure Example API request: ```json { "conf_path": "/etc/nginx/nginx.conf", "base_template": { "object_id": "", "values": { "backend_url": "http://example.com:8080" } }, "augments": [ { "object_id": "", "target_context": "http/server/location", "values": { "cors_allowed_origins": "https://app.example.com", "cors_allowed_methods": "GET, POST, PUT, DELETE, OPTIONS" } }, { "object_id": "", "target_context": "http/server" }, { "object_id": "", "target_context": "http", "values": { "listen_port": 80, "server_names": ["admin.example.com"] }, "child_augments": [ { "object_id": "", "target_context": "http/server/location", "values": { "path": "/admin", "upstream_url": "http://admin-backend:8080" } }, { "object_id": "", "target_context": "http/server/location", "values": { "health_check_path": "/admin/health" } } ] } ] } ``` ### New config template contents - **Existing augments remain:** CORS headers and the health-check location still apply to the main server. - **New server augments** Adds an extra server block. - **Service-specific routing:** The new `admin.example.com` server name has its own location blocks and routing rules. ### Response format If the request succeeds, the response includes the following output and the rendered NGINX configuration: #### Successful response (200 OK) ```json { "config": { "aux": [], "conf_path": "/etc/nginx/nginx.conf", "config_version": "nuZ4+d1T159/0vVV9vKaajrEw7QXc6T3fAnxKcVkC6I=", "configs": [ { "files": [ { "contents": "", "mtime": "0001-01-01T00:00:00Z", "name": "nginx.conf", "size": 483 } ], "name": "/etc/nginx" }, { "files": [ { "contents": "", "mtime": "0001-01-01T00:00:00Z", "name": "cors-headers.tmpl.4aaf36d4a643.conf", "size": 159 }, { "contents": "", "mtime": "0001-01-01T00:00:00Z", "name": "health-check.tmpl.78346de4dae4.conf", "size": 109 }, { "contents": "", "mtime": "0001-01-01T00:00:00Z", "name": "http-server.tmpl.81761e94d463.conf", "size": 145 }, { "contents": "", "mtime": "0001-01-01T00:00:00Z", "name": "location-proxy.tmpl.66ebf3e1dfd9.conf", "size": 121 } ], "name": "/etc/nginx/conf.d/augments" } ] }, "errors": null } ``` #### Rendered NGINX configuration ```nginx # configuration file /etc/nginx/nginx.conf: user nginx; worker_processes auto; events { worker_connections 1048; } http { server { listen 80; server_name admin.example.com; location /admin { proxy_pass http://admin-backend:8080; proxy_connect_timeout 30s; proxy_read_timeout 60s; } } server { listen 80; server_name _; location /health { access_log off; return 200 "healthy\n"; add_header Content-Type text/plain; } location / { proxy_pass http://example.com:8080; add_header 'Access-Control-Allow-Origin' 'https://app.example.com' always; add_header 'Access-Control-Allow-Methods' 'GET, POST, PUT, DELETE, OPTIONS' always; } } } ``` ## Child augments Child augments let you nest one augment inside another, creating hierarchical configurations. This is often used when a parent augment creates a container (such as a server block) that requires specific sub-configurations (such as location blocks). **How child augments work** When an augment template includes an `{{ augment_includes "context_path" . }}` extension point, you can provide child augments that target that context path. The child augments render only within their parent augment’s output. **Key behaviors** - Child location augments apply only to their parent server. - Different servers can have different location configurations. - Each server runs independently with its own routing rules. For details on designing templates with extension points, see the [Template Authoring Guide](author-templates.md). -------------------------------------------------------------------------------- title: "Add users to nginx-agent group" url: https://docs.nginx.com/nginx-one-console/agent/configure-instances/configure-agent-group/index.md -------------------------------------------------------------------------------- This page describes how the F5 NGINX Agent process interacts with the NGINX user on a system, and how to add users to the NGINX Agent group. ## Overview During installation, NGINX Agent detects the NGINX user (typically `nginx`) for the master and worker processes and adds this user to a group called `nginx-agent`. If you change the NGINX username after installing the NGINX Agent, you'll need to add the new username to the `nginx-agent` group so that the NGINX socket has the proper permissions. A failure to update the `nginx-agent` group when the NGINX username changes may result in non-compliance errors for NGINX Plus. --- ## NGINX socket NGINX Agent creates a socket in the default location `/var/run/nginx-agent/nginx.sock`. You can customize this location by editing the `nginx-agent.conf` file and setting the path similar to the following example: ```nginx configuration nginx: ... socket: "unix:/var/run/nginx-agent/nginx.sock" ``` The socket server starts when the NGINX socket configuration is enabled; the socket configuration is enabled by default. --- ## Add NGINX Users to nginx-agent group To manually add NGINX users to the `nginx-agent` group, take the following steps: 1. Verify the `nginx-agent` group exists: ```shell sudo getent group | grep nginx-agent ``` The output looks similar to the following example: ```shell nginx-agent:x:1001:root,nginx ``` If the group doesn't exist, create it by running the following command: ```shell sudo groupadd nginx-agent ``` 2. Verify the ownership of `/var/run/nginx-agent` directory: ```shell ls -l /var/run/nginx-agent ``` The output looks similar to the following: ```shell total 0 srwxrwxr-x 1 root nginx-agent 0 Jun 13 10:51 nginx.sockvv ``` If the group ownership is not `nginx-agent`, change the ownership by running the following command: ```shell sudo chown :nginx-agent /var/run/nginx-agent ``` 3. To add NGINX user(s) to the `nginx-agent` group, run the following command: ```shell sudo usermod -a -G nginx-agent ``` For example to add the `nginx` user, take the following step: ```shell sudo usermod -a -G nginx-agent nginx ``` Repeat for all NGINX users. -------------------------------------------------------------------------------- title: "Connect NGINX Plus container images" url: https://docs.nginx.com/nginx-one-console/connect-instances/connect-nginx-plus-container-images-to-nginx-one/index.md -------------------------------------------------------------------------------- ## Overview This guide explains how to set up an F5 NGINX Plus Docker container with NGINX Agent and connect it to the NGINX One Console. --- ## Before you start Before you start, make sure you have: - A valid JSON Web Token (JWT) for your NGINX subscription. - [A data plane key from NGINX One](/nginx-one-console/connect-instances/create-manage-data-plane-keys.md). - Docker installed and running on your system. #### Download your JWT license from MyF5 1. Log in to [MyF5](https://my.f5.com/manage/s/). 1. Go to **My Products & Plans > Subscriptions** to see your active subscriptions. 1. Find your NGINX subscription, and select the **Subscription ID** for details. 1. Download the **JSON Web Token** file from the subscription page. --- ## Process for private registry ### Log in to the NGINX private registry Use your JWT to log in to the NGINX private registry. Replace `YOUR_JWT_HERE` with your JWT. ```sh sudo docker login private-registry.nginx.com --username=YOUR_JWT_HERE --password=none ``` **Note:** To protect your system and data, follow these security practices: 1. **JWTs:** Treat JSON Web Tokens (JWTs) as sensitive data. Store them securely and delete them after use to prevent unauthorized access. 2. **Shell history:** Commands with JWTs or passwords are saved in plain text in your shell history. After running such commands, clear the history to protect credentials. For example: - Edit your shell history file (such as ~/.bash_history or ~/.zsh_history) to remove specific commands. - Use `history -c` to clear all shell history in bash or zsh. ### Pull the NGINX Plus image Pull the NGINX Plus image from the private registry. Replace `` with the desired version, such as `alpine`, `debian`, or `ubi`. ```sh docker pull private-registry.nginx.com/nginx-plus/agentv3: ``` You must specify a version tag that matches your distribution. The `latest` tag is not supported. Learn more in the [Deploying NGINX and NGINX Plus on Docker](/nginx/admin-guide/installing-nginx/installing-nginx-docker.md#pull-the-image) guide.
**Note:** To pull the `debian` image: ```sh sudo docker pull private-registry.nginx.com/nginx-plus/agent:debian ``` ### Start the NGINX Plus container Start the Docker container to connect it to NGINX One. Replace `YOUR_NGINX_ONE_DATA_PLANE_KEY_HERE` with your data plane key and `version-tag` with the version tag you pulled. **For NGINX Plus R33 or later**: - Use the `NGINX_LICENSE_JWT` variable to pass your JWT license. Replace `YOUR_JWT_HERE` with your JWT. For more details, see [About subscription licenses](solutions/about-subscription-licenses.md). ```sh sudo docker run \ --env=NGINX_LICENSE_JWT= \ --env=NGINX_AGENT_COMMAND_SERVER_PORT=443 \ --env=NGINX_AGENT_COMMAND_SERVER_HOST=agent.connect.nginx.com \ --env=NGINX_AGENT_COMMAND_AUTH_TOKEN= \ --env=NGINX_AGENT_COMMAND_TLS_SKIP_VERIFY=false \ --restart=always \ --runtime=runc \ -d private-registry.nginx.com/nginx-plus/agentv3: ```
**Note:** To start the container with the `debian` image: ```sh sudo docker run \ --env=NGINX_LICENSE_JWT= \ --env=NGINX_AGENT_COMMAND_SERVER_PORT=443 \ --env=NGINX_AGENT_COMMAND_SERVER_HOST=agent.connect.nginx.com \ --env=NGINX_AGENT_COMMAND_AUTH_TOKEN= \ --env=NGINX_AGENT_COMMAND_TLS_SKIP_VERIFY=false \ --restart=always \ --runtime=runc \ -d private-registry.nginx.com/nginx-plus/agentv3:debian ``` --- ## References For more details, see: - [Deploying NGINX and NGINX Plus on Docker](/nginx/admin-guide/installing-nginx/installing-nginx-docker.md) - [Full List of Agent Environment Variables](/agent/configuration/configuration-overview.md#cli-flags-and-environment-variables) - [NGINX One Data Plane Keys](/nginx-one-console/connect-instances/create-manage-data-plane-keys.md) - [My F5 Knowledge Article](https://my.f5.com/manage/s/article/K000090257) -------------------------------------------------------------------------------- title: "Lab 4: Config Sync Groups" url: https://docs.nginx.com/nginx-one-console/workshops/lab4/config-sync-groups/index.md -------------------------------------------------------------------------------- ## Introduction In this lab, you'll create and manage Config Sync Groups in NGINX One Console. Config Sync Groups keep your NGINX instances in sync with a shared configuration. You'll create a group, add instances, apply a shared config, and fix sync errors. --- ## What you'll learn By the end of this lab, you can: - Create a Config Sync Group - Add instances to a Config Sync Group - Update the shared configuration for a Config Sync Group --- ## Before you begin Make sure you have: - All labs require an **F5 Distributed Cloud (XC) account** with NGINX One enabled. If you don’t have an account or need to verify access, follow the steps in [Lab 1: Before you begin](/nginx-one-console/workshops/lab1/getting-started-with-nginx-one-console.md#before-you-begin). - Completed [Lab 2: Run workshop components with Docker](/nginx-one-console/workshops/lab2/run-workshop-components-with-docker.md) - Docker and Docker Compose installed and running - Set these environment variables: - **TOKEN**: your data plane key, for example: ```shell export TOKEN="your-data-plane-key" ``` - **JWT**: your NGINX Plus license JWT. Save it as `nginx-repo.jwt`, then run: ```shell export JWT=$(cat path/to/nginx-repo.jwt) ``` - **NAME**: a unique ID for your workshop (for example, `s.jobs`): ```shell export NAME="s.jobs" ``` - Basic familiarity with Linux command line and NGINX concepts --- ## Exercise 1: Create a Config Sync Group A Config Sync Group lets you apply one configuration to multiple NGINX instances and keep them in sync. 1. In NGINX One Console, select **Manage > Config Sync Groups**. 2. In the **Config Sync Groups** pane, select **Add Config Sync Group**. 3. In the form, enter `$NAME-sync-group` in the **Name** field (for example, `s.jobs-sync-group`). 4. Select **Create**. The new group appears with **Details** and **Configuration** tabs. - The **Details** tab shows: - Object ID - Last publication status and config version ID - Config Sync Status (for example, Unknown) - Instance status counts (In Sync, Out of Sync, Offline, Unavailable) 5. Switch to the **Configuration** tab to view your group's configuration files. It's empty for now. You'll add one in Exercise 2. --- ## Exercise 2: Add instances to the Config Sync Group **Note:** You can mix NGINX Open Source and NGINX Plus instances in one group. Any config feature you use must work on every instance. If you need NGINX Plus-only features, create a separate group. When you create a Config Sync Group, it has no shared config. You can add one in two ways: - **Define config manually**: Select your group, go to the **Configuration** tab, then select **Edit Configuration**. Add or paste your NGINX config, select **Next**, review the diff, and select **Save and Publish**. - **Populate from first instance**: Add one NGINX instance. The console uses that instance's existing config as the group's shared config. ### Populate group config from first instance 1. Select **Manage > Config Sync Groups**. 2. Select your `$NAME-sync-group` (for example, `s.jobs-sync-group`). 3. On the **Details** tab, in the **Instances** pane, select **Add Instance to Config Sync Group**. 4. Select **Register a new instance with NGINX One then add to config sync group**, then select **Next**. 5. Select **Use existing key** and paste `$TOKEN` (or your actual data plane key) into the **Data Plane Key** box. 6. Select the **Docker Container** tab. The tab shows sample commands. Copy them and modify as follows. 7. **Log in to the private registry:** ```shell echo "$JWT" \ | docker login private-registry.nginx.com \ --username "$JWT" --password-stdin ``` 8. **Pull a Docker image** (replace version as needed). You can select a specific NGINX Plus version, OS type, and OS version. Here, use the R31 Alpine image: ```shell docker pull private-registry.nginx.com/nginx-plus/agent:nginx-plus-r31-alpine-3.19-20240522 ``` See [Pulling the image](nginx/admin-guide/installing-nginx/installing-nginx-docker.md#pull-the-image) for details. 9. **Run the container**. Copy the `docker run` command from the user interface and modify it: - Replace `YOUR_JWT_HERE` in `--env NGINX_LICENSE_JWT` with `$JWT` - Replace `YOUR_DATA_PLANE_KEY` in `--env NGINX_AGENT_SERVER_TOKEN` with `$TOKEN` - Add `--hostname "$NAME-one-manual"` and `--name "$NAME-one-manual"` - Ensure `--env NGINX_AGENT_INSTANCE_GROUP="$NAME-sync-group"` is set ```shell docker run \ --hostname "$NAME-one-manual" \ --name "$NAME-one-manual" \ --env NGINX_LICENSE_JWT="$JWT" \ --env NGINX_AGENT_SERVER_GRPCPORT=443 \ --env NGINX_AGENT_SERVER_HOST=agent.connect.nginx.com \ --env NGINX_AGENT_SERVER_TOKEN="$TOKEN" \ --env NGINX_AGENT_INSTANCE_GROUP="$NAME-sync-group" \ --env NGINX_AGENT_TLS_ENABLE=true \ --restart always \ --runtime runc \ -d private-registry.nginx.com/nginx-plus/agent:nginx-plus-r31-alpine-3.19-20240522 ``` 10. In the **Config Sync Groups** pane, select **Refresh**. The new instance appears and the shared config populates. The first instance added becomes the default config source. 11. Select the **Configuration** tab to view the shared config. ### Add instances using Docker Compose Instead of registering containers manually, you can set the sync group in your Compose file and restart all containers. 1. Stop the running containers: ```shell docker compose down ``` 2. Open `compose.yaml` in a text editor. 3. Uncomment the lines beginning with: ```yaml NGINX_AGENT_INSTANCE_GROUP: $NAME-sync-group ``` 4. Restart all containers: ```shell docker compose up --force-recreate -d ``` 5. In NGINX One Console, select **Refresh**. Instances with `NGINX_AGENT_INSTANCE_GROUP` set appear in the Config Sync Group. 6. Instances automatically sync the existing NGINX config. When sync finishes, the **Config Sync Status** shows `In Sync`. ![Table showing hostnames, NGINX versions, operating systems, availability status, and green In Sync indicators for each instance in the config sync group.](nginx-one-console/images/config-sync-status.png) --- ## Exercise 3: Edit the group config and sync changes Modify the shared configuration and apply the changes to all group members. 1. Select **Manage > Config Sync Groups**, then choose `$NAME-sync-group` (for example, `s.jobs-sync-group`). 2. Select the **Configuration** tab. 3. Select **Edit Configuration** (pencil icon). 4. In the file list, select `default.conf`. 5. In the editor pane, add these lines at 21–24: ```yaml location /test_header { add_header X-Test-App true; return 200 'HTTP/1.1 200 OK\nContent-Type: text/html\n\nWelcome to Lab 4 of the NGINX One Console Workshop!'; } ``` ![Editor showing modifications to default.conf with validator status 'NGINX Config OK'.](nginx-one-console/images/config-sync-edits.png) The file is marked **modified** and the validator shows **NGINX Config OK**. 6. Select **Next**, review the diff, then select **Save and Publish**. 7. Select the **Details** tab and confirm **Last Publication Status** shows **Succeeded**. 8. In the **Instances** table, confirm each host shows **Config Sync Status = In Sync**. 9. Test your change by curling any instance's HTTP endpoint. Replace `` and `` with the values from the Instances table (for example, `localhost:80`): ```shell curl http://localhost:80/test_header ``` You should see: ```text HTTP/1.1 200 OK Content-Type: text/html Welcome to Lab 4 of the NGINX One Console Workshop! ``` --- ## Next steps You're ready to install your NGINX Plus license (JWT) on each instance. This will let you upgrade them to NGINX R34. Go to [Lab 5: Upgrade NGINX Plus to the latest version](/nginx-one-console/workshops/lab5/upgrade-nginx-plus-to-latest-version.md). --- ## References - [NGINX One Console docs](/nginx-one-console/) -------------------------------------------------------------------------------- title: "Learn about Role-based access control" url: https://docs.nginx.com/nginx-one-console/rbac/overview/index.md -------------------------------------------------------------------------------- Role-based access control (RBAC) is a security system that governs access to resources within a software application. By assigning specific roles to users or groups, RBAC ensures that only authorized individuals have the ability to perform certain actions or access particular areas. The value of RBAC lies in its ability to provide clear and structured control over what users can see and do. This makes it easier to maintain security, streamline user management, and ensure compliance with internal policies or regulations. By giving users only the permissions they need to fulfill their roles, RBAC reduces the risk of unauthorized access and fosters a more efficient and secure operating environment. The NGINX One Console uses the **[F5 Distributed Cloud User Management](https://docs.cloud.f5.com/docs-v2/administration/how-tos/user-mgmt)** system for access controls and user permissions. General information can be found on the User Management documentation for **[F5 Distributed Cloud](https://docs.cloud.f5.com/docs-v2/administration/how-tos/user-mgmt)**. This document provides guidance and reference material for utilizing those features to grant and restrict access within the NGINX One Console. -------------------------------------------------------------------------------- title: "Security dashboard reference" url: https://docs.nginx.com/nginx-one-console/waf-integration/waf-security-dashboard/dashboard-metrics-reference/index.md description: Tabs, global controls, and widget-to-dimension mapping for the F5 WAF for NGINX security monitoring dashboard. -------------------------------------------------------------------------------- ## Overview Use this reference to look up how the F5 WAF for NGINX security monitoring dashboard is organized and which underlying dimension each widget reads from. Every widget in the dashboard has an in-product tooltip that explains what the displayed values mean. This article mainly covers the dashboard structure, the global controls that affect every widget, and the mapping you need when you want to reproduce a widget's view through the [analytics API](/nginx-one-console/api/query-events-api.md). --- ## Dashboard layout The security dashboard is organized into three tabs. All three tabs share the same global filter bar and time window control at the top of the page, so any filter or time change applies to every widget on every tab. | Tab | Purpose | |:--- |:--- | | **Main** | High-level summary of WAF activity in the selected window. Shows attack counts, threat intelligence, attack volume over time, and top-N breakdowns by geolocation, policy, IP, violation, and signature. This is the landing tab for triage. | | **Advanced** | Deeper analytics for tuning and investigation. Use this tab to drill into signature attributes (risk, accuracy, CVE), violation context, and other dimensions that are too detailed for the Main tab. | | **Event Logs** | A filterable list of individual security events. Open an event to see its support ID, full violation and signature detail, request context, and the raw matched request. This tab is the drill-down target when you want to inspect specific events behind a metric. | --- ## Global controls Two controls at the top of the page apply to every widget on every tab. ### Time window Selects the query window for all widgets. The picker offers preset windows from **Last 5 minutes** to **Last 60 days**. You can also select a custom range by highlighting an area of interest on any time-series chart. Time-series widgets bucket their data automatically based on the selected window — shorter windows produce finer buckets. **Note:** Security events are retained for **90 days**, but the dashboard time window picker tops out at the **last 60 days**. To query the full retention window, use the [analytics API](/nginx-one-console/api/query-events-api.md), which accepts any time range up to 90 days. ### Add Filter Applies one or more filter expressions to every widget on every tab. The dashboard supports the following filters: | Filter | Description | |:--- |:--- | | **Config Sync Group** | Scope events to instances belonging to a specific Config Sync Group. | | **Country** | Two-letter country code derived from the client IP. | | **Destination Hostname** | The HTTP `Host` header sent by the client. Use this to scope by the application being attacked. | | **Hostname** | The NGINX instance hostname (the data plane host). Use this to scope by the data plane producing the events. | | **Instance** | Scope events to a specific NGINX instance by object identity rather than hostname. | | **IP Address** | The originating client IP. | | **Outcome Reason** | The reason F5 WAF for NGINX produced its decision, such as a matched violation or a blocking rule. | | **Policy** | The F5 WAF for NGINX policy that produced the event. | | **Request Method** | The HTTP request method (`GET`, `POST`, and so on). | | **Response Code** | The HTTP response code returned for the request. | | **Signature Accuracy** | The accuracy level of a triggered signature: `low`, `medium`, or `high`. | | **Signature CVE** | A CVE identifier referenced by a triggered signature. | | **Signature ID** | The numeric F5 WAF for NGINX signature ID. | | **Signature Name** | The name of a triggered signature. | | **Signature Risk** | The risk level of a triggered signature: `low`, `medium`, or `high`. | | **Status** | The final WAF decision: `blocked`, `alerted`, or `passed`. | | **Subviolation** | The sub-violation name within a violation. | | **Support ID** | The unique identifier F5 WAF for NGINX assigns to each event. | | **Threat Campaign** | The name of a matched threat campaign. | | **URI** | The request URI path. | | **Violation** | The name of a triggered violation. | | **Violation Context** | Where in the request the violation occurred: `cookie`, `header`, `parameter`, `request`, or `URI`. | | **Violation Context Key** | The field name (for example, the parameter or header name) where the violation occurred. | | **Violation Context Value** | The field value where the violation occurred. | | **Violation Rating** | The numeric severity rating (0–5) F5 WAF for NGINX assigned to the violation. | Use filters to scope the dashboard to a specific policy, instance, hostname, country, IP, signature, violation, or any combination. To go from a metric on the dashboard to the underlying events, apply the relevant filter and switch to the **Event Logs** tab. Every dashboard filter is also available through the analytics API. To call the same filters from the API, see the [API reference guide](/nginx-one-console/api/api-reference-guide.md). --- ## Main tab widgets Each widget on the Main tab has a tooltip describing what it displays. The table below adds context the tooltips do not cover, such as how rows are counted and what distinct counts each Top-N table reports. | Widget | Notes | |:--- |:--- | | **All Web Attacks** | Total count of security events for the selected window and filters. | | **Threat Intelligence** | Unique counts of threat campaigns and signatures observed in the window. | | **Attack Requests Over Time** | Stacked time series of `blocked` and `alerted` events. Bucket size depends on the selected window. | | **Top Attack Geolocations** | Highest-volume client countries in the window. | | **Top WAF Policies** | Each row shows hits, distinct URIs, IPs, and violations for the policy. | | **Top Attack IP Addresses** | Each row shows hits, distinct URIs, violations, and policies for the client IP. | | **Top Violations** | Each row shows hits, distinct IPs, URIs, and policies for the violation. | | **Top Signatures** | Each row shows hits, distinct URIs, IPs, violations, and policies for the signature. | | **Top Subviolations** | Each row shows hits, distinct IPs, URIs, and policies for the sub-violation. | | **Top Attack URIs** | Each row shows hits, distinct IPs, violations, and policies for the URI. | | **Request Methods** | Donut chart showing the share of events for each HTTP method observed in the window. | | **Response Codes** | Donut chart showing the share of events for each response code observed in the window. | To reproduce these widgets through the analytics API, see [Query security events through the API](/nginx-one-console/api/query-events-api.md). --- ## Advanced tab widgets The Advanced tab exposes signature- and violation-level analytics for tuning and deeper investigation. Widgets on this tab read from the same data store as the Main tab and respect the same global filters and time window. | Widget | Notes | |:--- |:--- | | **Signatures** | Total signature hits and unique signature count, with distributions across signature accuracy and risk levels. | | **Signature Hits Request Status** | Donut chart of `blocked`, `alerted`, and `passed` shares of signature hits. | | **Violation Context** | Donut chart showing where in the request the violation occurred (`cookie`, `header`, `parameter`, `request`, `URI`). | | **Signature Hits Over Time** | Time series of signature hit volume. Bucket size depends on the selected window. | | **Top Signatures** | Same as the Top Signatures widget on the Main tab. Each row shows hits, distinct URIs, IPs, violations, and policies. | | **Top Signature CVEs** | Each row shows hits, distinct URIs, IPs, violations, and policies for signatures that reference the CVE. | | **Top Threat Campaigns** | Each row shows hits, distinct URIs, IPs, violations, and policies for the threat campaign. | | **Top Attacked Instances** | Each row shows hits, distinct URIs, IPs, violations, and policies, scoped to the NGINX instance hostname (not the HTTP `Host` header). | To reproduce these widgets through the analytics API, see [Query security events through the API](/nginx-one-console/api/query-events-api.md). --- ## Event Logs tab The Event Logs tab lists individual security events matching the global filters and time window. The tab has two parts: - A **Security Events** time-series chart at the top showing event volume by request status (`blocked` and `alerted`), the same view as the **Attack Requests Over Time** widget on the Main tab. - A paginated **events table** below the chart, where each row corresponds to one security event. ### Events table columns | Column | Description | |:--- |:--- | | **Status** | The final WAF decision: `blocked`, `alerted`, or `passed`. | | **URI** | The request URI that triggered the event. | | **Policy** | The F5 WAF for NGINX policy that produced the event. | | **Time** | When F5 WAF for NGINX produced the event. | | **Source Location** | Country derived from the client IP, when available. | | **Source IP** | The originating client IP. | | **Violation Rating** | Numeric severity rating (0–5) assigned by F5 WAF for NGINX. | | **Support ID** | The unique identifier F5 WAF for NGINX assigns to the event. Use this to correlate with raw F5 WAF for NGINX logs on the data plane. | To list events with the same columns through the analytics API, see [Query security events through the API](/nginx-one-console/api/query-events-api.md). ### Pagination The table is paginated. Use the controls in the bottom-right of the table to: - Change the page size (default `50` rows per page). - Navigate to a specific page or step through pages with the arrow controls. The total event count for the current filter and time window is shown in the bottom-left of the table. ### Event detail panel Selecting a row opens the event detail panel. The panel surfaces every field stored on the event, including: - **Triggered violations and signatures** — every violation and signature that fired on the event, with their full attributes. - **Request context** — method, URL, host, client IP, X-Forwarded-For chain, country, response code, and request status. - **Raw matched request** — the captured request payload, when available. --- ## References For more information, see: - [Security monitoring overview](/nginx-one-console/waf-integration/waf-security-dashboard/security-monitoring-overview.md) - [Set up security monitoring](/nginx-one-console/waf-integration/waf-security-dashboard/set-up-security-monitoring.md) - [Find a security event by Support ID](/nginx-one-console/waf-integration/waf-security-dashboard/find-event-by-support-id.md) - [Query security events through the API](/nginx-one-console/api/query-events-api.md) -------------------------------------------------------------------------------- title: "Upgrade NGINX Agent" url: https://docs.nginx.com/nginx-one-console/agent/install-upgrade/update/index.md -------------------------------------------------------------------------------- ## Overview **Note:** If you are using a version **older than NGINX Agent v2.31.0**, you must stop NGINX Agent before updating: - `sudo systemctl stop nginx-agent` And start it again after the update or upgrade: - `sudo systemctl start nginx-agent` Follow the steps below to update or upgrade NGINX Agent to the latest version. The same steps apply if you are **upgrading from NGINX Agent v2 to NGINX Agent v3**. 1. Open an SSH connection to the server where you've installed NGINX Agent. 1. Make a backup copy of the following locations to ensure that you can successfully recover if the upgrade does not complete successfully: - `/etc/nginx-agent` - Every configuration directory specfied in `/etc/nginx-agent/nginx-agent.conf` as a `config_dirs` value 1. Install the updated version of NGINX Agent: - CentOS, RHEL, RPM-Based ```shell sudo yum -y makecache sudo yum update -y nginx-agent ``` - Debian, Ubuntu, Deb-Based ```shell sudo apt-get update sudo apt-get install -y --only-upgrade nginx-agent -o Dpkg::Options::="--force-confold" ``` ## Migrate NGINX Agent running in containers To migrate NGINX Agent containers, we provide a script to convert NGINX Agent v2 config files to NGINX Agent v3 config files: [NGINX Agent Config Upgrade Script](https://github.com/nginx/agent/blob/v3/scripts/packages/upgrade-agent-config.sh) To upgrade the configuration, you can follow this example: ```shell wget https://raw.githubusercontent.com/nginx/agent/refs/heads/main/scripts/packages/upgrade-agent-config.sh ./upgrade-agent-config.sh --v2-config-file=./nginx-agent-v2.conf --v3-config-file=nginx-agent-v3.conf ``` If your NGINX Agent container was previously a member of a Config Sync Group, then your NGINX Agent config must be manually updated to add the Config Sync Group label. See [Add Config Sync Group](/nginx-one-console/nginx-configs/config-sync-groups/manage-config-sync-groups.md) for more information. ### Rolling back from NGINX Agent v3 to v2 If you need to roll back your environment to NGINX Agent v2, the upgrade process creates a backup of the NGINX Agent v2 config in the file `/etc/nginx-agent/nginx-agent-v2-backup.conf`. Replace the contents of `/etc/nginx-agent/nginx-agent.conf` with the contents of `/etc/nginx-agent/nginx-agent-v2-backup.conf` and then reinstall an older version of NGINX Agent. -------------------------------------------------------------------------------- title: "Add gRPC protection to a policy" url: https://docs.nginx.com/nginx-one-console/waf-integration/policy/grpc-protection-api/index.md description: Upload .proto files through the API so a WAF policy's grpc-profiles entry can validate gRPC messages. -------------------------------------------------------------------------------- F5 WAF for NGINX can protect gRPC APIs by validating messages against their Interface Definition Language (IDL) files (`.proto`). This is configured through the `grpc-profiles` property of a WAF policy. **Note:** This page covers only how to upload the `.proto` files a `grpc-profiles` entry references through the NGINX One Console API. For a full description of the gRPC protection feature itself (content profiles, defense attributes, URL association, streaming, violations, and logging), see [gRPC protection](/waf/policies/grpc-protection.md) and the `grpc-profiles` section of the [Policy parameter reference](/waf/policies/parameter-reference.md). **gRPC protection is currently supported through the API only.** There is no NGINX One Console UI for configuring `grpc-profiles` or uploading IDL files yet. **Note:** **Experimental API.** The file reference fields and request bodies described on this page (`file_references`, `file_references_archive`, and related schemas) are under active development. They may change in a future release. ## Why file references are needed A `grpc-profiles` entry's `idlFiles` point at the service's `.proto` file(s) using a `file://` URI, for example: ```json "idlFiles": [ { "idlFile": { "$ref": "file:///grpc_files/album.proto" }, "isPrimary": true } ] ``` The policy JSON only contains a *reference* to the file. It doesn't contain the file itself. To make the reference resolvable, you must separately upload the actual `.proto` file content in the same API request that creates or updates the policy. **Note:** **You must explicitly provide the file contents yourself.** NGINX One Console does not fetch, generate, or infer IDL files on your behalf — you own the `.proto` file(s) for your gRPC service, and you are responsible for uploading their exact content every time you create or update a policy version that references them. If a policy version's `file://` reference has no matching uploaded file, the policy will fail to compile. You can upload file references in either of two ways: - **Inline as base64 JSON**: good for a small number of files, or when you're already sending the policy as JSON. - **As a `.tar.gz` archive**: good for bundling many files (for example, a primary `.proto` plus its imports) without base64 overhead. Available only when *creating* a policy. ## Upload files as inline JSON (`file_references`) Send the policy as `application/json` to `POST /app-protect/policies` (create) or `PUT /app-protect/policies/{nap_policy_object_id}` (update a policy, creating a new version), with a `file_references` array alongside the base64-encoded `policy`: ```json { "policy": "", "file_references": [ { "type": "proto", "file_path": "/grpc_files/album.proto", "content": "" } ] } ``` Each entry in `file_references` requires: | Field | Description | |-------|--------------| | `type` | Currently only `proto` is supported. | | `file_path` | The absolute path matching the `file://` URI referenced in the policy JSON. Must start with `/`. | | `content` | The base64-encoded file content. | **Note:** On update (`PUT`), `file_references` **replaces the full set of files** for the new policy version. If you omit `file_references` on an update, the new version is created with **no** file references at all, even if a previous version had files uploaded. Always re-include every file reference the policy still needs when updating. ## Upload files as a .tar.gz archive (file_references_archive) When creating a policy (`POST /app-protect/policies` only, not supported on update), you can instead send `multipart/form-data` with a `policy` field and a `file_references_archive` field containing a gzip-compressed tarball of the referenced files: ```bash curl -X POST https://api.nginx-one.console.nginx.com/app-protect/policies \ -H "Authorization: Bearer " \ -F "policy=" \ -F "file_references_archive=@refs.tar.gz;type=application/gzip" ``` The archive's internal directory structure determines the absolute path each file resolves to. Build the archive from the **parent** of the referenced directory so the paths line up with your policy's `file://` URIs: ``` grpc_files/ ├── album.proto └── common/ └── messages.proto ``` ```bash # Correct: paths resolve to /grpc_files/album.proto, /grpc_files/common/messages.proto tar czf refs.tar.gz grpc_files/ # Incorrect: paths resolve to /album.proto (missing parent directory) cd grpc_files && tar czf refs.tar.gz *.proto ``` If your policy references `"$ref": "file:///grpc_files/album.proto"`, the archive must contain `grpc_files/album.proto` at that relative path. Only regular files are extracted. Symlinks, hardlinks, and directory-only entries are ignored. ## Path rules and limits `file_path` (JSON method) and archive entry paths (multipart method) must: - Be absolute (start with `/`) and contain no `..` traversal segments. - Target a directory outside the standard system paths (`/etc`, `/tmp`, `/usr`, `/var`, `/opt`, `/root`, and similar). Use a novel top-level directory such as `/grpc_files` for your referenced files. | Limit | Inline JSON (`file_references`) | Archive (`file_references_archive`) | |-------|----------------------------------|--------------------------------------| | Max size per file | 2 MB | 2 MB (uncompressed) | | Max total size | 5 MB | 10 MB (uncompressed) / 5 MB (compressed archive) | | Max number of files | 50 | 100 | ## Retrieving file reference information `GET` requests for a policy version return **metadata only** for each file reference: `type`, `file_path`, `size`, and a base64-encoded SHA-256 `hash`. They don't return the file's content. ```json "file_references": [ { "type": "proto", "file_path": "/grpc_files/album.proto", "size": 4096, "hash": "oxiWKPqR/soi4MQCgVnW8KHt8Jk68AqCeQcQ1sed4Dk=" } ] ``` There is currently no endpoint to download the original file content back from NGINX One Console. Keep a copy of your `.proto` files on your side. ## Full example The following creates an F5 WAF policy with a `grpc-profiles` entry that references `album.proto`, uploading the file inline as base64 JSON. This mirrors the [example](/waf/policies/grpc-protection.md#content-profiles) from the gRPC protection feature documentation. ```bash curl -X POST https://api.nginx-one.console.nginx.com/app-protect/policies \ -H "Authorization: Bearer " \ -H "Content-Type: application/json" \ -d '{ "policy": "", "file_references": [ { "type": "proto", "file_path": "/grpc_files/album.proto", "content": "" } ] }' ``` Decoded `policy` contents: ```json { "policy": { "name": "my-grpc-service-policy", "grpc-profiles": [ { "name": "photo_service_profile", "associateUrls": true, "defenseAttributes": { "maximumDataLength": 100000, "allowUnknownFields": false }, "attackSignaturesCheck": true, "idlFiles": [ { "idlFile": { "$ref": "file:///grpc_files/album.proto" }, "isPrimary": true } ] } ], "urls": [ { "name": "*", "type": "wildcard", "method": "*", "$action": "delete" } ] } } ``` ## See also - [Set security policies through the API](/nginx-one-console/waf-integration/policy/security-policy-api.md) - [API reference guide](/nginx-one-console/api/api-reference-guide.md) - [gRPC protection](/waf/policies/grpc-protection.md): full feature description - [Policy parameter reference](/waf/policies/parameter-reference.md): `grpc-profiles` field reference -------------------------------------------------------------------------------- title: "Explore configurations with Config Explorer" url: https://docs.nginx.com/nginx-one-console/nginx-configs/explore-configurations/index.md description: Use Config Explorer to browse and search your NGINX configurations as an interactive node graph. -------------------------------------------------------------------------------- ## Overview Config Explorer is an interactive view for any NGINX configuration in NGINX One Console. Instead of reading raw configuration files, you can navigate your entire NGINX configuration hierarchy as a visual node graph. Each node in the graph represents a directive context — such as `main`, `http`, `upstream`, `server`, or `location` — and you can select any node to inspect its properties and documentation. Config Explorer is available on the detail page of the following resources: - Staged Configurations - Instances - Config Sync Groups --- ## Before you begin Before you begin, ensure you have: - **An NGINX One Console account**: You must have access to NGINX One Console and permission to view the resource you want to explore. --- ## Open Config Explorer ### Open Config Explorer for a Staged Configuration 1. On the left menu, select **Staged Configurations**. 2. Select the Staged Configuration you want to explore. 3. Select the **Explorer** tab. ### Open Config Explorer for an Instance 1. On the left menu, select **Instances**. 2. Select the instance you want to explore. 3. Select the **Explorer** tab. ### Open Config Explorer for a Config Sync Group 1. On the left menu, select **Config Sync Groups**. 2. Select the Config Sync Group you want to explore. 3. Select the **Explorer** tab. --- ## Browse the configuration When Config Explorer opens, it displays your NGINX configuration as an interactive node graph on the canvas. 1. Use the **view filter** in the toolbar to narrow the graph to a specific directive type, such as Upstreams or HTTP. The directive count next to the filter updates to reflect the current view. 2. Select any node in the graph to open its **Properties** panel on the right. The Properties panel shows: - The breadcrumb path of the selected directive within the configuration hierarchy. - The source file and line reference. - The parsed field values for the directive, such as host, port, and flags. - **Children** — an expandable list of child directives. - **Docs** — inline NGINX documentation for the selected directive, including its syntax, default value, valid contexts, and a plain-English description. 3. Use the scroll and zoom controls in the lower-left corner of the canvas to pan and zoom the graph. 4. Review the **Directives** bar at the bottom of the canvas for a color-coded breakdown of all directive types and their counts in the current view. --- ## Search for directives 1. Select the **Search** bar in the toolbar. 2. Type any term to search across all directives in the configuration. Config Explorer searches the graph in real time and highlights matching nodes. 3. Matching results appear in a dropdown list showing the full breadcrumb path of each match. The counter in the search bar, for example `7/12`, shows your current position in the result set and the total number of matches. 4. Use the arrow keys or select a result from the dropdown to navigate between matches. Config Explorer moves focus to the matching node in the graph and updates the Properties panel. 5. Select the clear button in the search bar or press **Escape** to close the search results. --- ## References For more information, see: - [View and edit a Staged Configuration](/nginx-one-console/nginx-configs/staged-configs/edit-staged-config.md) - [Manage Config Sync Groups](/nginx-one-console/nginx-configs/config-sync-groups/manage-config-sync-groups.md) - [View and edit an NGINX instance](/nginx-one-console/nginx-configs/one-instance/view-edit-nginx-configurations.md) -------------------------------------------------------------------------------- title: "Troubleshoot security monitoring on the local data plane" url: https://docs.nginx.com/nginx-one-console/waf-integration/waf-security-dashboard/local-dataplane-troubleshooting/index.md description: Check the local NGINX Agent and OpenTelemetry Collector configuration when F5 WAF for NGINX security events do not appear in NGINX One Console. -------------------------------------------------------------------------------- ## Overview Use this guide when you completed [Set up security monitoring](/nginx-one-console/waf-integration/waf-security-dashboard/set-up-security-monitoring.md), sent test traffic, and still do not see F5 WAF for NGINX security events in the NGINX One Console dashboard. This guide focuses on the **local data plane**. It helps you verify four things: 1. Whether the embedded OpenTelemetry Collector is dropping security logs because the deployed log profile format is wrong. 2. Whether the NGINX Agent embedded OpenTelemetry Collector is the only process listening on port `1514`. 3. Whether the NGINX agent generated OpenTelemetry Collector config has the expected security log pipeline. 4. Whether debug logging shows the collector forwarding security logs to NGINX One Console. --- ## Before you begin Before you begin, ensure you have: - Access to the data plane host where NGINX Plus, F5 WAF for NGINX, and NGINX Agent are running. - Permission to read `/var/log/nginx-agent/` and `/etc/nginx-agent/`. - Security monitoring already configured by following [Set up security monitoring](/nginx-one-console/waf-integration/waf-security-dashboard/set-up-security-monitoring.md). --- ## 1. Check for invalid log profile errors Check whether NGINX Agent's embedded OpenTelemetry Collector is rejecting incoming F5 WAF for NGINX security logs. This happens when the logs don't match the format the security monitoring pipeline expects. Open the collector log on the data plane: ```shell sudo tail -f /var/log/nginx-agent/opentelemetry-collector-agent.log ``` Look for either of the following errors: ```text Security violation log body is not a string. All security violation logs will be dropped until the collector is restarted. ``` ```text Security violation log does not appear to be CSV format. Ensure the NAP logging profile uses the secops-dashboard-log format. All security violation logs will be dropped until the collector is restarted. ``` If you see either message, the data plane probably isn't using the NGINX One Console default `secops_dashboard` log profile for `app_protect_security_log`. Make sure every http, server, or location block that should log violations uses this profile. ### Fix 1. In NGINX One Console, redeploy the default [`secops_dashboard` log profile](/nginx-one-console/waf-integration/waf-security-dashboard/default-log-profile.md) to the affected instance. 2. Update the NGINX configuration so `app_protect_security_log` points to that deployed bundle. 3. Restart NGINX Agent to reset the embedded OpenTelemetry Collector after you correct the log profile. After the restart, send a new test request and check the dashboard again. --- ## 2. Check that port `1514` is reserved for the collector F5 WAF for NGINX sends security logs to `syslog:server=127.0.0.1:1514`. If another process is listening on port `1514`, the NGINX Agent embedded OpenTelemetry Collector may never receive the security logs. Run the following command on the data plane: ```shell sudo ss -ltnp | grep 1514 ``` Confirm that no unexpected process is listening on port `1514`. If another service is bound to that port, stop or reconfigure it. The embedded collector needs port `1514` free to receive F5 WAF for NGINX security logs. --- ## 3. Verify the generated OpenTelemetry log pipeline If the collector log does **not** show either invalid-log-profile error, verify that the generated OpenTelemetry Collector config still contains the security log pipeline. **Note:** NGINX Agent generates this security log pipeline only when at least one `http`, `server`, or `location` block is set up with `app_protect_security_log` pointing to `syslog:server=127.0.0.1:1514`. If no protected context uses that syslog destination, the pipeline isn't generated. Without it, no WAF security logs are forwarded to NGINX One Console. Open the generated collector config: ```shell sudo grep -A 12 "logs/default:" /etc/nginx-agent/opentelemetry-collector-agent.yaml ``` Confirm it includes the following pipeline: ```yaml logs/default: receivers: - tcp_log/nginx_app_protect processors: - securityviolationsfilter/default - batch/default_logs - resource/default exporters: - otlp_grpc/default ``` This pipeline accepts F5 WAF for NGINX security logs from `tcplog/nginx_app_protect`. It filters and batches the logs, then exports them to NGINX One Console through `otlp/default`. If this pipeline is missing or materially different, the collector isn't set up as expected for security monitoring. In that case, review any custom collector configuration merged through `nginx-agent.conf`, then restart NGINX Agent so it regenerates the collector config. --- ## 4. Enable debug logging for the collector pipeline If the collector log doesn't show the invalid-log-profile errors and the generated pipeline looks correct, turn on debug logging. This lets you confirm the embedded collector is processing and forwarding security logs. Add the following configuration to the end of `/etc/nginx-agent/nginx-agent.conf`: ```yaml collector: exporters: debug: {} pipelines: logs: default: receivers: - tcplog/nginx_app_protect processors: - securityviolationsfilter/default - batch/default_logs exporters: - otlp/default - debug ``` Restart NGINX Agent so the updated collector configuration is applied. The `debug` exporter causes the embedded OpenTelemetry Collector to write its processed log output to: ```text /var/log/nginx-agent/opentelemetry-collector-agent.log ``` Use this to confirm the collector is handling F5 WAF for NGINX security events locally. It continues forwarding them to NGINX One Console through `otlp/default`. **Note:** The debug exporter increases log volume. Remove it after troubleshooting so the collector log returns to its normal verbosity. --- ## What to do next After each fix: 1. Restart NGINX Agent. 2. Send a new test request through the protected application path. For example requests, see [Example test requests for a default blocking policy](/nginx-one-console/waf-integration/waf-security-dashboard/set-up-security-monitoring.md#example-test-requests-for-a-default-blocking-policy). 3. Check `/var/log/nginx-agent/opentelemetry-collector-agent.log`. 4. Recheck the **WAF** > **Security Dashboard** and **Event Logs** tab in NGINX One Console. If the collector is processing events locally but the dashboard remains empty, gather the following and provide them to F5 support: - NGINX Agent configuration: `/etc/nginx-agent/nginx-agent.conf` - Generated OpenTelemetry Collector configuration: `/etc/nginx-agent/opentelemetry-collector-agent.yaml` - NGINX Agent log: `/var/log/nginx-agent/agent.log` - Embedded OpenTelemetry Collector log: `/var/log/nginx-agent/opentelemetry-collector-agent.log` --- ## References For more information, see: - [Set up security monitoring](/nginx-one-console/waf-integration/waf-security-dashboard/set-up-security-monitoring.md) - [secops_dashboard log profile](/nginx-one-console/waf-integration/waf-security-dashboard/default-log-profile.md) - [Security dashboard reference](/nginx-one-console/waf-integration/waf-security-dashboard/dashboard-metrics-reference.md) - [Export NGINX instance metrics](/nginx-one-console/agent/configure-otel-metrics.md) -------------------------------------------------------------------------------- title: "Assign a display name to an instance" url: https://docs.nginx.com/nginx-one-console/agent/configure-instances/configure-instance-display-name/index.md description: Set a custom display name for an NGINX instance in NGINX One Console using the NGINX Agent display-name label. -------------------------------------------------------------------------------- ## Overview By default, NGINX One Console identifies each NGINX instance by its hostname. To assign a display name to an instance, set the reserved `display-name` NGINX Agent label. After NGINX Agent reports the label, NGINX One Console shows the display name alongside the hostname. You can then use the display name to filter and sort instances. ## Before you begin Before you start, make sure that you have: - [NGINX Agent installed](/nginx-one-console/agent/install-upgrade/) and connected to NGINX One Console. - Access to the NGINX Agent configuration file, CLI, or container environment. ## Set a display name Set the `display-name` label using any of the [NGINX Agent label](/nginx-one-console/agent/configure-instances/configuration-overview.md) configuration sources. ### Configuration file Add the label to the `labels` section of `/etc/nginx-agent/nginx-agent.conf`: ```yaml labels: display-name: prod-usw2-edge-03 ``` ### CLI parameters ```shell nginx-agent --labels=display-name=prod-usw2-edge-03 ``` ### Environment variables ```shell export NGINX_AGENT_LABELS="display-name=prod-usw2-edge-03" ``` After you save your changes, restart NGINX Agent to apply them: ```shell sudo systemctl restart nginx-agent ``` **Note:** Display names must be 256 characters or fewer. If a value exceeds this limit, the instance registers without a display name. Display names cannot include Unicode characters like emoji or other multi-byte characters. ## Where the display name appears Once NGINX Agent reports the `display-name` label, NGINX One Console uses it in the following places: - The **Instances** list and **Instance details** page, alongside the hostname. - Instance filters and sort options, so you can find instances by display name. - Data Plane Key details, Config Sync Group instance lists, and Staged Config publish flows. - Instance metrics dashboards and F5 WAF for NGINX security events, where available. If an instance doesn't have a display name set, NGINX One Console continues to show its hostname. -------------------------------------------------------------------------------- title: "Export NGINX instance metrics" url: https://docs.nginx.com/nginx-one-console/agent/configure-otel-metrics/index.md -------------------------------------------------------------------------------- ## Overview The F5 NGINX Agent now includes an embedded [OpenTelemetry](https://opentelemetry.io) collector, streamlining observability and metric collection for NGINX instances. By default, the NGINX Agent sends key metrics to the [NGINX One Console](/nginx-one-console/nginx-configs/metrics/review-metrics), providing quick visibility into server performance through pre-configured dashboards. With this feature, you can collect: - Metrics from NGINX Plus and NGINX Open Source - Host metrics such as CPU, memory, disk, and network activity from virtual machines (VMs) or containers For users requiring deeper integration with third-party observability tools, the NGINX Agent supports exporting additional metrics through the embedded OpenTelemetry collector. Tools such as Prometheus, Splunk, and other OpenTelemetry-compatible platforms can be configured to ingest these metrics, as detailed in the rest of this document. **Note:** The OpenTelemetry exporter is enabled by default. Once a valid connection to the management plane is established, the NGINX Agent will automatically begin exporting metrics to the NGINX One Console. ### Key benefits * Seamless Integration: No need to deploy an external OpenTelemetry Collector. All components are embedded within the Agent for streamlined observability. * Standardized Protocol: Support for OpenTelemetry standards ensures interoperability with a wide range of observability backends, including Prometheus, Splunk, and more. ### Verify that metrics are exported You can validate that metrics are successfully exported by using the methods below: - **NGINX One dashboard** - When an instance has connected to NGINX One Console [See: Connect to NGINX One Console](/nginx-one-console/connect-instances/add-instance.md), you should see metrics showing on the NGINX One Console Dashboard. - **Agent logs** Check the OpenTelemetry Collector logs for confirmation of successful metric processing: 1. Open the file: `/var/log/nginx-agent/opentelemetry-collector-agent.log` 2. Look for the following logs: ```text Everything is ready. Begin running and processing data. ``` ### Custom OTel Configuration NGINX Agent generates a default OpenTelemetry config (located at `/etc/nginx-agent/opentelemetry-collector-agent.yaml`) to send metrics to your management plane. An option is provided to bring your own [OpenTelemetry configs](https://opentelemetry.io/docs/collector/configuration/) which will be merged with the NGINX Agent default config. The **order of the OpenTelemetry config files matters**: the last config in the list will take priority over others listed, if they have the same value configured. #### VM 1. Edit the configuration file as root `vim /etc/nginx-agent/nginx-agent.conf` 1. Add the collector property 1. Save and restart the NGINX agent service `sudo systemctl restart nginx-agent` ```yaml collector: additional_config_paths: - "/etc/nginx-agent/my_config.yaml" ``` #### Container 1. Run the Docker container: Use the following command to run the NGINX Agent docker container. Replace the placeholder values (`YOUR_JWT_HERE`, `DPK`, `/path/to/my_config.yaml`, and ``) with the appropriate values for your environment: ```bash sudo docker run \ --env=NGINX_LICENSE_JWT= \ --env=NGINX_AGENT_COMMAND_SERVER_PORT=443 \ --env=NGINX_AGENT_COMMAND_SERVER_HOST=agent.connect.nginx.com \ --env=NGINX_AGENT_COMMAND_AUTH_TOKEN= \ --env=NGINX_AGENT_COMMAND_TLS_SKIP_VERIFY=false \ --env=NGINX_AGENT_COLLECTOR_ADDITIONAL_CONFIG_PATHS="/etc/nginx-agent/my_config.yaml" \ --volume=/path/to/my_config.yaml:/etc/nginx-agent/my_config.yaml:ro \ --restart=always \ --runtime=runc \ -d private-registry.nginx.com/nginx-plus/agentv3: ``` #### Example usage **Note:** NGINX Agent uses `/default` for naming its default processors, exporters and pipelines using the same naming in your own config might cause issues with sending metrics to your management plane #### Add Prometheus Exporter Configuration ```yaml exporters: prometheus: endpoint: "127.0.0.1:5643" resource_to_telemetry_conversion: enabled: true namespace: test-space service: pipelines: metrics/prometheus-example-pipeline: receivers: - # Use nginxplus for NGINX Plus or nginx for OSS processors: - resource/default exporters: - prometheus ``` #### Third-party OTel Collector ```yaml exporters: otlp_grpc/local-collector: endpoint: "my-local-collector.com:443" service: pipelines: metrics/otlp-example-pipeline: receivers: - nginxplus/nginx # Use nginxplus for NGINX Plus or nginx for OSS processors: - resource/default exporters: - otlp_grpc/local-collector ``` #### Add Debug Exporter ```yaml exporters: debug: verbosity: detailed sampling_initial: 5 sampling_thereafter: 200 service: pipelines: metrics/debug-example-pipeline: receivers: - nginxplus/nginx # Use nginxplus for NGINX Plus or nginx for OSS processors: - resource/default exporters: - debug ``` ### Troubleshooting To view the merged OpenTelemetry configuration, change the NGINX Agent log level to "debug" in `/etc/nginx-agent/nginx-agent.conf`: ```yaml log: level: debug -------------------------------------------------------------------------------- title: "Find a security event by Support ID" url: https://docs.nginx.com/nginx-one-console/waf-integration/waf-security-dashboard/find-event-by-support-id/index.md description: Look up an F5 WAF for NGINX security event in NGINX One Console using its Support ID. -------------------------------------------------------------------------------- ## Overview Use the Support ID Details page in NGINX One Console to look up a single F5 WAF for NGINX security event by its Support ID. F5 WAF for NGINX assigns a unique Support ID to every inspected request, and that ID travels through every system that touches the request: security logs, NGINX access logs, upstream application logs, and the security monitoring dashboard. Use this page when you already know the ID of the event you want to inspect — for example, from a customer support case, a raw F5 WAF for NGINX log line, an alert payload, or an upstream application that captured the ID from a request header. To explore events without a known ID, use the Event Logs tab on the [security monitoring dashboard](/nginx-one-console/waf-integration/waf-security-dashboard/dashboard-metrics-reference.md#event-logs-tab) instead. --- ## Before you begin Before you begin, ensure you have: - **A Support ID**: A numeric Support ID for the event you want to inspect. See [Where to find a Support ID](#where-to-find-a-support-id) below. - **Security monitoring set up**: F5 WAF for NGINX security events must already be flowing into NGINX One Console for the event to be available. See [Set up security monitoring](/nginx-one-console/waf-integration/waf-security-dashboard/set-up-security-monitoring.md). --- ## Look up an event by Support ID 1. In NGINX One Console, go to **WAF** > **Support ID Details**. 2. Paste the Support ID into the **Enter a Support ID** field. 3. The page displays the **Security Event** detail panel for the matching event, including the request, source, and raw request data. **Note:** Security events are retained for 90 days. Events older than 90 days are no longer available. ### Troubleshooting If no event is found, the most likely causes are: - The Support ID does not belong to a WAF-inspected request - The event is older than the 90-day retention window - The Support ID was mistyped --- ## What the detail panel shows The Security Event detail panel surfaces every field stored on the event: - **Request** — request method, URI, host, headers, and the **Raw Request** payload as captured by F5 WAF for NGINX. The raw request is the same payload that would appear in a NAP security log on the data plane. - **Source** — client IP, X-Forwarded-For chain, and a geolocation map showing the country derived from the client IP. - **Time of Request** — when F5 WAF for NGINX produced the event. - **Triggered violations and signatures** — every violation and signature that fired on the event, with their full attributes (name, accuracy, risk, CVE, context). - **Threat campaigns** — any threat campaigns matched by the event. - **Policy and outcome** — the F5 WAF for NGINX policy that produced the event, the request status (`blocked`, `alerted`, or `passed`), and the outcome reason. --- ## Where to find a Support ID A Support ID can come from any system that observed the request: - **NGINX One Console Event Logs tab** — every row in the Event Logs table on the [security dashboard](/nginx-one-console/waf-integration/waf-security-dashboard/dashboard-metrics-reference.md#event-logs-tab) includes the Support ID. Copy a Support ID from there to share in a ticket, support case, or follow-up message. - **F5 WAF for NGINX security logs on the data plane** — the Support ID is the first field in every log line emitted by the [`secops_dashboard` log profile](/nginx-one-console/waf-integration/waf-security-dashboard/default-log-profile.md). - **F5 WAF for NGINX rejection page** — when F5 WAF for NGINX blocks a request, the response page typically includes the Support ID so the user can quote it back to support. - **Customer support cases** — end users who hit a Request Rejected page can be asked to provide the Support ID from that page. --- ## Use cases ### Operator: triage a customer-reported block A customer reports that their request was rejected and provides the Support ID from the rejection page. The operator opens **WAF** > **Support ID Details**, pastes the ID, and sees the full event record, including which signature fired and which policy was in effect — enough to decide whether the block was a true positive or a candidate for policy tuning. ### Security engineer: correlate an upstream incident with WAF activity An upstream service surfaces an incident referencing a Support ID extracted from request headers. The engineer pastes the ID into the Support ID Details page to see whether the request was blocked, alerted, or passed by F5 WAF for NGINX, and which violations or signatures it triggered, before deciding whether the incident is WAF-related. --- ## Querying by Support ID through the API The same lookup is available through the analytics API. Use the [`getSecurityEvent`](/nginx-one-console/api/api-reference-guide/#operation/getSecurityEvent) operation to fetch a single event, or [`listSecurityEvents`](/nginx-one-console/api/api-reference-guide/#operation/listSecurityEvents) with `support_id` in `filter_fields` to look up by Support ID programmatically. For more on querying events through the API, see [Query security events through the API](/nginx-one-console/api/query-events-api.md). --- ## References For more information, see: - [Security dashboard reference](/nginx-one-console/waf-integration/waf-security-dashboard/dashboard-metrics-reference.md) - [Query security events through the API](/nginx-one-console/api/query-events-api.md) - [Set up security monitoring](/nginx-one-console/waf-integration/waf-security-dashboard/set-up-security-monitoring.md) -------------------------------------------------------------------------------- title: "Lab 5: Upgrade NGINX Plus to the latest version" url: https://docs.nginx.com/nginx-one-console/workshops/lab5/upgrade-nginx-plus-to-latest-version/index.md -------------------------------------------------------------------------------- ## Introduction In this lab, you'll upgrade NGINX Plus from R32 (or earlier) to the latest version. There are two scenarios: - **Docker**: Deploy a new container running the latest NGINX Plus image, add it to your Config Sync Group, then shift traffic and retire older containers. - **VM**: Push your JWT license to an existing VM instance, install the new NGINX Plus package, and restart the service. Pick the scenario that matches your setup. --- ## What you'll learn By the end of this lab, you can: - Deploy a Docker container running the latest NGINX Plus with NGINX Agent installed - Add a VM to a Config Sync Group and push your JWT license - Install or upgrade to the latest NGINX Plus on a VM - Check version and sync status in NGINX One Console - Clean up unavailable instances in NGINX One Console --- ## Before you begin Make sure you have: - All labs require an **F5 Distributed Cloud (XC) account** with NGINX One enabled. If you don’t have an account or need to verify access, follow the steps in [Lab 1: Before you begin](/nginx-one-console/workshops/lab1/getting-started-with-nginx-one-console.md#before-you-begin). - Completed [Lab 4: Config Sync Groups](/nginx-one-console/workshops/lab4/config-sync-groups.md) - Docker and Docker Compose installed and running (for Docker scenario) - A trial or paid NGINX One JWT license (saved as `nginx-repo.jwt`) from [MyF5](https://my.f5.com/manage/s/) - A VM with NGINX Plus R32 (or earlier), SSH access, and NGINX Agent installed (for VM scenario) - Set these environment variables: - **TOKEN**: your data plane key, for example: ```shell export TOKEN="your-data-plane-key" ``` - **JWT**: your NGINX Plus license JWT. Save it as `nginx-repo.jwt`, then run: ```shell export JWT=$(cat path/to/nginx-repo.jwt) ``` - **NAME**: a unique ID for your workshop (for example, `s.jobs`): ```shell export NAME="s.jobs" ``` - Basic familiarity with Linux and NGINX --- ## Scenario A: Upgrade NGINX Plus in Docker ### Exercise A1: Pull and run the latest NGINX Plus image 1. Log in to the private registry: ```shell echo "$JWT" | docker login private-registry.nginx.com \ --username "$JWT" --password-stdin ``` 2. Open `compose.yaml` in a text editor. Uncomment the **plus4** service block (lines 74–95). This block pulls the latest Debian NGINX Plus image with NGINX Agent, and sets your data plane key, JWT, and Config Sync Group. ```yaml plus4: # Debian latest NGINX Plus Web / Load Balancer environment: NGINX_AGENT_SERVER_HOST: 'agent.connect.nginx.com' NGINX_AGENT_SERVER_GRPCPORT: '443' NGINX_AGENT_TLS_ENABLE: 'true' NGINX_AGENT_SERVER_TOKEN: $TOKEN # Data plane key from NGINX One Console NGINX_LICENSE_JWT: $JWT NGINX_AGENT_INSTANCE_GROUP: $NAME-sync-group hostname: $NAME-plus4 container_name: $NAME-plus4 image: private-registry.nginx.com/nginx-plus/agent:debian volumes: - ./nginx-plus/etc/nginx/nginx.conf:/etc/nginx/nginx.conf - ./nginx-plus/etc/nginx/conf.d:/etc/nginx/conf.d - ./nginx-plus/etc/nginx/includes:/etc/nginx/includes - ./nginx-plus/usr/share/nginx/html:/usr/share/nginx/html ports: - '80' - '443' - '9000' - '9113' restart: always ``` **Note:** In VS Code, highlight lines 74–95 and press `Ctrl` + `/` to uncomment. 3. Restart your containers: ```shell docker compose down && docker compose up --force-recreate -d ``` 4. In NGINX One Console, go to **Instances**. 5. You should see your new instance (`$NAME-plus4`) in the list (for example, `s.jobs-plus4`). 6. Select the instance and confirm it runs the latest versions of NGINX Plus and NGINX Agent. 7. The `$NAME-plus4` container joins the `$NAME-sync-group` and inherits the shared config. **Note:** Because new containers in a sync group automatically pick up the shared config, you get a consistent setup across versions. This makes upgrades safer and avoids manual copy-paste steps. ### Exercise A2: Delete unavailable containers When you recreate containers, old entries remain in NGINX One Console. Clean them up: 1. In NGINX One Console, go to **Instances**. 2. Select **Add filter > Availability > Unavailable**. 3. Select the checkboxes for the unavailable hosts. 4. Select **Delete selected**, then confirm. 5. Remove the filter by selecting the **X** next to the filter tag. ![Table of three NGINX One Console instances filtered to 'Availability = Unavailable.' Shows hostnames, NGINX versions, gray Unavailable icons, and the Delete selected button.](nginx-one-console/images/unavailable-instances.png) --- ## Scenario B: Upgrade NGINX Plus on a VM with Config Sync Groups **Note:** These steps cover RHEL, Amazon Linux, CentOS, Oracle Linux, AlmaLinux, Rocky Linux, Debian, and Ubuntu only. ### Exercise B1: Create a Config Sync Group for VMs 1. In NGINX One Console, go to **Manage > Config Sync Groups**. 2. Select **Add Config Sync Group**. 3. In the **Name** field, enter `$NAME-sync-group-vm` (for example, `s.jobs-sync-group-vm`), then select **Create**. ### Exercise B2: Add your VM to the Config Sync Group 1. Select **Manage > Config Sync Groups**, then pick your group's name. 2. On the **Details** tab, in the **Instances** pane, select **Add Instance to Config Sync Group**. 3. Select **Register a new instance with NGINX One then add to config sync group**, then select **Next**. 4. Select **Use existing key**, paste `` into the **Data Plane Key** box. 5. Copy the pre-filled `curl` command and run it on your VM: **Example**: ```shell curl https://agent.connect.nginx.com/nginx-agent/install | \ DATA_PLANE_KEY="" \ sh -s -- -y -c "" ``` 6. Back in NGINX One Console, select **Refresh**. Your VM appears in the list with **Config Sync Status = In Sync**. ### Exercise B3: Enable the NGINX Plus API and dashboard Add a new configuration file (`/etc/nginx/conf.d/dashboard.conf`) in your group config to enable the NGINX Plus API and dashboard. 1. In the NGINX One Console, select **Manage > Config Sync Groups**, then pick your config sync group's name. 2. Select the **Configuration** tab, then select **Edit Configuration**. 3. Select **Add File**. 4. Select **New Configuration File**. 5. In the **File name** box, enter `/etc/nginx/conf.d/dashboard.conf`, then select **Add**. 6. Paste the following into the new file workspace: #### without SSL #### with SSL 7. Select **Next**, review the diff, then select **Save and Publish**. 8. Open your browser to `http://:9000/dashboard.html` (replace `` with the IP or hostname of one of your group members). You should see the NGINX Plus dashboard. ### Exercise B4: Add your JWT license file Each instance needs a JWT license before upgrading. Add it in your Config Sync Group so all members inherit it. 1. In NGINX One Console, select **Manage > Config Sync Groups**, then pick your group's name. 2. Select the **Configuration** tab, then **Edit Configuration**. 3. Select **Add File**, then **New Configuration File**. 4. In **File name**, enter `/etc/nginx/license.jwt`. 5. Select **Add**. 6. Paste the contents of your JWT file into the editor. 7. Select **Next**, review the diff, then select **Save and Publish**. See [About subscription licenses](solutions/about-subscription-licenses.md) for details. ### Exercise B5: Upgrade NGINX Plus on your VM 1. Upgrade the NGINX Plus package: - **RHEL, Amazon Linux, CentOS, Oracle Linux, AlmaLinux, Rocky Linux** ```shell sudo yum upgrade nginx-plus ``` - **Debian, Ubuntu** ```shell sudo apt update && sudo apt install nginx-plus ``` 2. In NGINX One Console, go to **Manage > Instances**. 3. Select your VM instance. 4. In the **Instance Details** pane, confirm the NGINX Plus version has been updated. 5. If the version doesn’t update right away, refresh the page after a few seconds. --- ## Next steps You have upgraded your instances to the latest NGINX Plus. Go to the [NGINX One documentation](/nginx-one-console/) for more advanced guides and use cases. --- ## References - [NGINX One Console docs](/nginx-one-console/) - [About subscription licenses](solutions/about-subscription-licenses.md) -------------------------------------------------------------------------------- title: "Review default roles" url: https://docs.nginx.com/nginx-one-console/rbac/roles/index.md -------------------------------------------------------------------------------- ## Default roles We provide three default **[roles](https://docs.cloud.f5.com/docs-v2/administration/how-tos/user-mgmt/roles)** that can be used for providing various access levels to the NGINX One Console. These roles will be automatically updated as new features are added to the NGINX One Console. Default roles can be scoped to specific namespaces. ### Admin The Admin role, identified as `f5xc-nginx-one-admin`, provides full read and write access to all endpoints and features within the NGINX One Console. It also supports Role-based access control for related XC services. ### User Our standard User role, listed as `f5xc-nginx-one-user` in the role list, provides read and write access to all endpoints and features, save for those considered to be administrator level. An example of an administrator level feature would be **[Instance Settings](https://docs.nginx.com/nginx-one/how-to/nginx-configs/clean-up-unavailable-instances/)** where unavailable instance clean up logic is set. ### Monitor Our read only or Monitor role, `f5xc-nginx-one-monitor`, grants read only access to all non-administrator features and endpoints within the NGINX One Console. -------------------------------------------------------------------------------- title: "Review policy" url: https://docs.nginx.com/nginx-one-console/waf-integration/policy/review-policy/index.md -------------------------------------------------------------------------------- Before you implement a policy on an NGINX instance or Config Sync Group, you may want to review it. F5 NGINX One Console creates a policy for your F5 WAF for NGINX system. ## Review F5 WAF for NGINX policies From NGINX One Console, select **WAF** > **Policies**. Select the name of the policy that you want to review. You'll see the following tabs: - Details, which includes: - Policy Details: Descriptions, status, enforcement type, latest version, and last deployed time. - Deployments: List of instances and Config Sync Groups where the F5 WAF for NGINX policy is deployed. - Policy JSON: The policy, in JSON format. With the **Edit** button, you can modify this policy. - Versions: Policy versions that you've written, labeled sequentially (`v1`, `v2`, `v3`, and so on). Each version shows its name, creation date, enforcement mode, and optional version comment. Select a version to view its details or update its comment. You can also apply an older policy version to your deployments. ## Modify existing policies From the NGINX One Console, you can also manage existing policies. In the Policies screen, identify a policy, and select **Actions**. From the menu that appears, you can: - **Edit** an existing policy. - **Save As** to save an existing policy with a new name. You can use an existing policy as a baseline for further customization. - **Deploy Latest Version** to apply the latest revision of an existing policy to the configured instances and Config Sync Groups. - **Export** the policy in JSON format. - **Delete** the policy. Once confirmed, you'll lose all work you've done on that policy. **Note:** If you use **Save As** to create a new policy, include the `app_protect_cookie_seed` [directive](/waf/policies/directives.md). -------------------------------------------------------------------------------- title: "Set up custom roles with API groups" url: https://docs.nginx.com/nginx-one-console/rbac/rbac-api/index.md -------------------------------------------------------------------------------- Beyond the [Default roles](/nginx-one-console/rbac/roles.md) for NGINX One Console access, you can create [custom roles](https://docs.cloud.f5.com/docs-v2/administration/how-tos/user-mgmt/roles#custom-roles) with more precisely defined access permissions. You can assign custom roles to users or service accounts. You can associate these roles with specific [namespaces](https://docs.cloud.f5.com/docs-v2/multi-cloud-app-connect/how-to/app-security/service-policy), to help facilitate the principle of least privilege across your tenant. For this use-case, we include a list of API groups that you can use to specify permissions for custom roles with more granular access controls to NGINX One Console APIs. ## F5 API groups for NGINX One The following table lists the available API groups that you can use to construct a Role. These are narrowly scoped API groups that align with all the features and functionality within the NGINX One Console. These groups can help you create custom roles tailored to your specific needs. **Note:** If you create custom roles using these API groups, users may not have access to all capabilities of the browser web portal. | API Group Name | Level of Access | Description | |-----------------------------------------|-----------------|-------------------------------------------------------------------------------------------------------------------------------| | f5xc-nginx-one-application-monitor | Read | View all features and data. | | f5xc-nginx-one-application-settings | Write | View and update settings. | | f5xc-nginx-one-application-write | Write | View and edit all features except settings. | | f5xc-nginx-one-custom-all-instances-metric-read | Read | View metrics for all Instances. Required to see the Overview dashboard. | | f5xc-nginx-one-custom-instance-list | Read | View list of all Instances. Also view summarized information such as certificate status and CVEs. | | f5xc-nginx-one-custom-all-instances-manage | Write | View and delete all Instances. | | f5xc-nginx-one-custom-instance-manage | Write | View and edit Instance details. | | f5xc-nginx-one-custom-instance-read | Read | View Instance and configuration details. | | f5xc-nginx-one-custom-certificate-manage | Write | View TLS/SSL certificate details. Create, update, and delete any managed certificates. | | f5xc-nginx-one-custom-certificate-read | Read | View TLS/SSL certificates. | | f5xc-nginx-one-custom-all-certificates-manage | Write | View all TLS/SSL certificates. Delete managed certificates. | | f5xc-nginx-one-custom-data-plane-key-manage | Write | View, create, update, and delete any Data Plane Keys. Note: The actual Data Plane Key is shown _only_ when created. | | f5xc-nginx-one-custom-data-plane-key-read | Read | View Data Plane Key Details. Note: The actual Data Plane Key is shown _only_ when created. | | f5xc-nginx-one-custom-all-data-plane-keys-manage | Write | View and delete Data Plane Keys. | | f5xc-nginx-one-custom-cve-read | Read | View NGINX CVEs. | | f5xc-nginx-one-custom-config-sync-group-manage | Write | View, create, update, and delete Config Sync Groups. | | f5xc-nginx-one-custom-config-sync-group-read | Read | View Config Sync Groups with details. | | f5xc-nginx-one-custom-all-config-sync-groups-manage | Write | View and delete Config Sync Groups. | | f5xc-nginx-one-custom-settings-manage | Write | View and update NGINX One Console Settings. | | f5xc-nginx-one-custom-settings-read | Read | View NGINX One Console Settings. | | f5xc-nginx-one-custom-event-read | Read | View NGINX One Events. | | f5xc-nginx-one-custom-ai-assistant | Write | Interact with the F5 AI Assistant. | | f5xc-nginx-one-custom-staged-config-manage | Write | View, create, update, and delete Staged Configs. | | f5xc-nginx-one-custom-staged-config-read | Read | View Staged Configs. | | f5xc-nginx-one-custom-templates-manage | Write | View, create, update, and delete templates. | | f5xc-nginx-one-custom-templates-read | Read | View templates. | | f5xc-nginx-one-custom-templates-submissions-contributor | Write | View and edit existing template submissions. Cannot create new submissions. | | f5xc-nginx-one-custom-templates-submissions-manage | Write | View, create, update, and delete template submissions. | | f5xc-nginx-one-custom-templates-submissions-owner | Write | View, create, and edit template submissions. | -------------------------------------------------------------------------------- title: "Set up security alerts" url: https://docs.nginx.com/nginx-one-console/secure-your-fleet/set-up-security-alerts/index.md -------------------------------------------------------------------------------- With this page, you'll learn how to set up alerts in F5 Distributed Cloud. Once configured, you'll see the CVEs and insecure configurations associated with your NGINX fleet. These instructions are intended for those responsible for keeping their NGINX infrastructure and application traffic secure. It assumes you know how to: - Install Linux programs or run Docker containers By the end of this tutorial, you'll be able to: - Access the NGINX One Console in F5 Distributed Cloud - Connect NGINX instances to the NGINX One Console - Review Security Risks associated with your NGINX fleet - Configure Alert Policies in F5 Distributed Cloud ## Background NGINX One Console is a service to monitor and manage NGINX. It's a part of the F5 Distributed Cloud and is included with all NGINX and F5 Distributed Cloud subscriptions. While NGINX is built to be secure and stable, critical vulnerabilities can occasionally emerge – and misconfigurations may leave your applications or APIs exposed to attacks. ## Before you begin If you already have accessed F5 Distributed Cloud and have NGINX instances available, you can skip these steps and start to connect instances to the NGINX One Console. ### Confirm access to the F5 Distributed Cloud Confirm that an F5 Distributed Cloud tenant has been provisioned for you. To do so: 1. Log in to [MyF5](https://my.f5.com/manage/s/) and review your subscriptions. 2. Look for **Distributed Cloud** in one of your subscriptions. This could appear under an NGINX subscription or a Distributed Cloud subscription. If you don’t see **Distributed Cloud** in any subscription, contact your F5 account team or Customer Success Manager. When you have a subscription, you or someone in your organization should have an email from no-reply@cloud.f5.com asking you to update your password. That email includes an account name, which is the TENANT_NAME. To access F5 Distributed Cloud, go to https://INSERT_YOUR_TENANT_NAME.console.ves.volterra.io/. If you have never logged in, select the **Forgot Password?** option in the log in screen. Alternatively, if someone within your organization has access, ask them to add you as a user within your tenant with access to NGINX One Console. If you do not know your TENANT_NAME, you can: - Email support@cloud.f5.com to get your username and TENANT_NAME. - If you have a dedicated F5 tenant administrator, ask them to: - Provide a username for your tenant - Ask them to reset your password ### Confirm access to NGINX One Console in the F5 Distributed Cloud Once you've logged in with your password, you should be able to see and select the NGINX One tile. 1. Select the **NGINX One** tile 1. Select **Visit Service** ### Install an instance of NGINX Ensure you have an instance of [NGINX Open Source or NGINX Plus](/nginx/admin-guide/installing-nginx/) installed and available. This guide provides instructions for connecting an instance installed in a Linux environment (VM or bare metal hardware) where you have command line access. Alternatively, we also have instructions for [Deploying NGINX and NGINX Plus with Docker](/nginx/admin-guide/installing-nginx/installing-nginx-docker.md) with NGINX and the NGINX Agent installed. That deployment can connect with environment variables. ## Connect at least one NGINX instance to the NGINX One Console If you already have connected instances to the NGINX One Console, you can start to [Configure an active alert policy](/nginx-one-console/secure-your-fleet/set-up-security-alerts.md#configure-an-active-alert-policy). Otherwise, you need to add an instance, generate a data plane key, and install NGINX Agent. We assume this is the first time you are connecting an instance. ### Add an instance You can add an instance to NGINX One Console in the following ways: - Directly, under **Instances** - Indirectly, by selecting a Config Sync Group, and selecting **Add Instance to Config Sync Group** In either case, NGINX One Console gives you a choice for data plane keys: - Create a new key - Use an existing key NGINX One Console takes the option you use, and adds the data plane key to a command that you'd use to register your target instance. You should see the command in the **Add Instance** screen in the console. Connect to the host where your NGINX instance is running. Run the provided command to [install NGINX Agent](/nginx-one-console/getting-started#install-nginx-agent) dependencies and packages on that host. ```bash curl https://agent.connect.nginx.com/nginx-agent/install | DATA_PLANE_KEY="" sh -s -- -y ``` Once the process is complete, you can configure that instance in your NGINX One Console. **Note:** To assign a display name that's easier to recognize than the hostname, set the `display-name` NGINX Agent label. For details, see [Assign a display name to an instance](/nginx-one-console/agent/configure-instances/configure-instance-display-name.md). ### Generate a data plane key A data plane key is a security token that ensures only trusted NGINX instances can register and communicate with NGINX One. To generate a data plane key, select **Manage > Instances > Add Instance**: - **For a new key:** In the **Add Instance** pane, select **Generate Data Plane Key**. - **To reuse an existing key:** If you already have a data plane key and want to use it again, select **Use existing key**. Then, enter the key's value in the **Data Plane Key** box. **Note:** Data plane keys are displayed only once and cannot be retrieved later. Be sure to copy and store this key securely. Data plane keys expire after one year. You can change this expiration date later by [editing the key](/nginx-one-console/connect-instances/create-manage-data-plane-keys.md#change-expiration-date). If you [revoke a data plane key](/nginx-one-console/connect-instances/create-manage-data-plane-keys.md#revoke-data-plane-key) you disconnect all instances registered with that key. For more options associated with data plane keys, see [Create and manage data plane keys](/nginx-one-console/connect-instances/create-manage-data-plane-keys.md). ### Install NGINX Agent After entering your data plane key, you'll see a `curl` command to install NGINX Agent, similar to the one below. Copy and run this command on each NGINX instance. Once installed, NGINX Agent typically registers with NGINX One within a few seconds. **Note:** Make sure your firewall rules for NGINX hosts allow traffic to port `443` from these IP address ranges: - `3.135.72.139/32` - `3.133.232.50/32` - `52.14.85.249/32` - `2600:1f16:19c8:d400::/62` NGINX Agent must be able to establish a connection to NGINX One Console's Agent endpoint (`agent.connect.nginx.com`). To install NGINX Agent on an NGINX instance: 1. **Check if NGINX is running and start it if it's not:** First, see if NGINX is running: ```shell sudo systemctl status nginx ``` If the status isn't `Active`, go ahead and start NGINX: ```shell sudo systemctl start nginx ``` 2. **Install NGINX Agent:** Next, use the `curl` command provided to you to install NGINX Agent: ``` shell curl https://agent.connect.nginx.com/nginx-agent/install | DATA_PLANE_KEY="YOUR_DATA_PLANE_KEY" sh -s -- -y ``` - Replace `YOUR_DATA_PLANE_KEY` with your actual data plane key. You can also install NGINX Agent from our repositories and configure it manually. Alternatively you can use our official NGINX Docker images, pre-configured with NGINX Agent. ## Configure an active alert policy The NGINX One Console monitors all connected NGINX instances for CVEs and insecure configurations. Using the F5 Distributed Cloud's Alert Policies, you can receive alerts for these risks in a manner of your choosing; for the purposes of this guide, we show you how to configure email alerts. The F5 Distributed Cloud generates alerts from all its services including NGINX One Console. You can configure rules to send those alerts to a receiver of your choice. These instructions walk you through how to configure an email notification when we see new CVEs or detect security issues with your NGINX instances. This page describes basic steps to set up an email alert. For authoritative documentation, see [Alerts - Email & SMS](https://docs.cloud.f5.com/docs-v2/shared-configuration/how-tos/alerting/alerts-email-sms). ## Configure alerts to be sent to your email To configure security-related alerts, follow these steps: 1. Go to the F5 Distributed Cloud Console at https://INSERT_YOUR_TENANT_NAME.console.ves.volterra.io. 1. Select **Audit Logs & Alerts** 1. Select **Alerts Management > Alert Receivers** 1. Select **Add Alert Receiver** 1. Enter the name of your choice. 1. (Optional) Specify a label and description. 1. Under **Receiver**, select **Email** and enter your email address. 1. Select **Add Alert Receiver** Your alert receiver should now appear on the list of Alert Receivers. 1. Select the **Actions** ellipsis (...) for your receiver. Select **Verify Email**. 1. Select **Send email** to confirm. 1. You should receive a verification code in the email provided. Copy that code. 1. Under the **Actions** column, select **Enter verification code**. 1. Paste the code and select **Verify receiver**. ## Configure Alert Policy Next, configure the policy that identifies when you'll get an alert. You'll need to reference available alerts in our [NGINX One Console Glossary](/nginx-one-console/glossary.md#nginx-alerts/). Relevant security alerts include: - SecurityRecommendationNGINX - HighCVENGINX - MediumCVENGINX - LowCVENGINX 1. Go to **Alerts Management > Alert Policies**. 1. Select **Add Alert Policy**. 1. Enter the name of your choice. You're limited to lower-case characters, numbers, and dashes. 1. (Optional) Specify a label and description. 1. Under **Alert Reciever Configuration > Alert Receivers,** select the **Alert Receiver** you just created. 1. Under **Policy Rules** select **Configure**. 1. In the **Policy Rules** screen that appears, select **Add Item**. 1. In the **Route** window that appears, review the **Select Alerts** drop-down. 1. Under **Select Alerts** select a filter. Now select **Matching Custom Criteria > Alertname > Configure**. In the screen that appears, use **Exact Match** and copy/paste an alert name from the [NGINX One Console Glossary](/nginx-one-console/glossary.md#nginx-alerts). 1. Select **Apply** to exit the **Alertname** window. 1. Select **Apply** to exit the **Route** window. 1. Select **Apply** to exit the **Policy Rules** window. 1. You can now select the **Add Alert policy** button. 1. Set the **Action as Send** and select **Apply**. ## Create more alert policies Repeat the process described in [Configure Alert Policy](#configure-alert-policy) section. Repeat again if and as needed for all of the alerts in the [NGINX One Console Glossary](/nginx-one-console/glossary.md#nginx-alerts/). ## Activate the alert policy Now to make sure your new policy works, add your new policies to the list of **Active Alert Policies**. To do so: 1. Select **Alerts Management > Active Alert Policies** 1. Select **Select Active Alert Policies**. 1. In the **Select Active Alert Policies** window, select **Add Item** 1. In the drop-down box that appears, select the Alert Policy that you created. 1. Select the **Add Select Active Alert Policies** button. 1. Select **Add Item** You've now set up F5 Distributed Cloud to send you alerts from NGINX One Console, to your email address. When the alert policy identifies an alert, it sends you an email from **alerts@cloud.f5.com**. ## Known issues When you set up an email alert for a problem, you'll see the alert in: - The F5 Distributed Cloud Console, under **Notifications > Alerts** in **Audit Logs & Alerts** - An email with a subject like ** Alert Requires Action** You may also get a follow-up email with the subject **Alert Resolved**. **Note:** Sometimes an **Alert Resolved** email is sent even though the issue is still active. To check the current status, go to the NGINX One Console. For CVEs, the trusted source is: - **NGINX One Console > Manage > Instances > `Instance hostname`** Open the instance dashboard to see the latest list of CVEs. Use the Console, not email, to confirm whether an issue is resolved. ## Summary In this tutorial, you learned how to: - Access the NGINX One Console - Connect an NGINX instance - Configure and activate an alert You will now receive an email any time the F5 Distributed Cloud sees one or more of the alerts that you configued. ## Next steps Now that you have NGINX instances connected to the NGINX One Console, consider reviewing our [use cases](/nginx-one-console/) to see how you can easily manage your NGINX instances, draft new configurations, and more. Additionally, you can review how to add additional Alert Receivers such as [SMS](https://docs.cloud.f5.com/docs-v2/shared-configuration/how-tos/alerting/alerts-email-sms), [Slack](https://docs.cloud.f5.com/docs-v2/shared-configuration/how-tos/alerting/alerts-slack), [PagerDuty](https://docs.cloud.f5.com/docs-v2/shared-configuration/how-tos/alerting/alerts-pagerduty), or with a [webhook](https://docs.cloud.f5.com/docs-v2/shared-configuration/how-tos/alerting/alerts-webhook). -------------------------------------------------------------------------------- title: "Uninstall NGINX Agent" url: https://docs.nginx.com/nginx-one-console/agent/install-upgrade/uninstall/index.md -------------------------------------------------------------------------------- ## Overview Follow the steps in this guide to remove F5 NGINX Agent from your NGINX instances. ## Before you begin The user following performing the uninstall steps needs to have `root` privilege. ## Uninstall NGINX Agent Complete the following steps on each host where you've installed NGINX Agent ### Uninstall NGINX Agent on Alpine Linux #### Expand instructions Complete the following steps on each host where you've installed NGINX agent: 1. Stop NGINX agent: ```shell sudo rc-service nginx-agent stop ``` 1. To uninstall NGINX agent, run the following command: ```shell sudo apk del nginx-agent ``` ### Uninstall NGINX Agent on Amazon Linux #### Expand instructions Complete the following steps on each host where you've installed NGINX agent: 1. Stop NGINX agent: ```shell sudo systemctl stop nginx-agent ``` 1. To uninstall NGINX agent, run the following command: ```shell sudo yum remove nginx-agent ``` ### Uninstall NGINX Agent on Debian #### Expand instructions Complete the following steps on each host where you've installed NGINX Agent: 1. Stop NGINX Agent: ```shell sudo systemctl stop nginx-agent ``` 1. To uninstall NGINX Agent, run the following command: ```shell sudo apt-get remove nginx-agent ``` **Note:** The `apt-get remove ` command will remove the package from your system, while keeping the associated configuration files for possible future use. If you want to completely remove the package and all of its configuration files, you should use `apt-get purge `. ### Uninstall NGINX Agent on RHEL, CentOS, Rocky Linux, AlmaLinux, and Oracle Linux #### Expand instructions Complete the following steps on each host where you've installed NGINX Agent: 1. Stop NGINX Agent: ```shell sudo systemctl stop nginx-agent ``` 1. To uninstall NGINX Agent, run the following command: ```shell sudo yum remove nginx-agent ``` ### Uninstall NGINX Agent on SLES #### Expand instructions Complete the following steps on each host where you've installed NGINX Agent: 1. Stop NGINX agent: ```shell sudo systemctl stop nginx-agent ``` 1. To uninstall NGINX agent, run the following command: ```shell sudo zypper remove nginx-agent ``` ### Uninstall NGINX Agent on Ubuntu #### Expand instructions Complete the following steps on each host where you've installed NGINX Agent: 1. Stop NGINX Agent: ```shell sudo systemctl stop nginx-agent ``` 1. To uninstall NGINX Agent, run the following command: ```shell sudo apt-get remove nginx-agent ``` **Note:** The `apt-get remove ` command will remove the package from your system, while keeping the associated configuration files for possible future use. If you want to completely remove the package and all of its configuration files, you should use `apt-get purge `. -------------------------------------------------------------------------------- title: "Community and contribution" url: https://docs.nginx.com/nginx-one-console/agent/community/index.md -------------------------------------------------------------------------------- Discover the various ways you can participate in the F5 NGINX Agent project: ## Community - Have questions or ideas to discuss? Join the conversation about NGINX Agent in the [NGINX Community Forum](https://community.nginx.org/). ## Contribute Get involved with the project by contributing! See our [contributing guide](https://github.com/nginx/agent/blob/main/CONTRIBUTING.md) for details. ## License [Apache License, Version 2.0](https://github.com/nginx/agent/blob/main/LICENSE) -------------------------------------------------------------------------------- title: "Configure SELinux" url: https://docs.nginx.com/nginx-one-console/agent/configure-instances/configure-selinux/index.md -------------------------------------------------------------------------------- ## Overview You can use the optional SELinux policy module included in the package to secure F5 NGINX Agent operations with flexible, mandatory access control that follows the principle of least privilege. **Note:** The SELinux policy module is optional. It is not loaded automatically during installation, even on SELinux-enabled systems. You must manually load the policy module using the steps below. ## Before you begin Take these preparatory steps before configuring SELinux: 1. Enable SELinux on your system. 2. Install the tools `load_policy`, `semodule`, and `restorecon`. 3. [Install NGINX Agent](/nginx-one-console/agent/install-upgrade/_index.md) with SELinux module files in place. **Note:** SELinux can use `permissive` mode, where policy violations are logged instead of enforced. Verify which mode your configuration uses. --- ## Enable SELinux for NGINX Agent {#selinux-agent} The following SELinux files are added when you install the NGINX Agent package: - `/usr/share/selinux/packages/nginx_agent.pp` - loadable binary policy module - `/usr/share/selinux/devel/include/contrib/nginx_agent.if` - interface definitions file - `/usr/share/man/man8/nginx_agent_selinux.8.gz` - policy man page To load the NGINX Agent policy, run the following commands as root: ```bash sudo semodule -n -i /usr/share/selinux/packages/nginx_agent.pp sudo /usr/sbin/load_policy sudo restorecon -R /usr/bin/nginx-agent sudo restorecon -R /var/log/nginx-agent sudo restorecon -R /etc/nginx-agent ``` ### Add ports to NGINX Agent SELinux context Make sure to add external ports to the firewall exception list. To allow external ports outside the HTTPD context, run: ```bash sudo setsebool -P httpd_can_network_connect 1 ``` **Note:** For more information, see [Using NGINX and NGINX Plus with SELinux](https://www.nginx.com/blog/using-nginx-plus-with-selinux/). --- ## Recommended Resources - - - - - - - -------------------------------------------------------------------------------- title: "Deploy policy" url: https://docs.nginx.com/nginx-one-console/waf-integration/policy/deploy-policy/index.md -------------------------------------------------------------------------------- After you've set up a policy, it won't do anything, until you deploy it to one or more instances and Config Sync Groups. This page assumes you've created a policy in NGINX One Console that you're ready to deploy. ## Deploy a policy To deploy a policy from NGINX One Console, take the following steps: 1. Select **WAF** > **Policies**. 1. Select the policy that you're ready to deploy. 1. Select the **Details** tab. 1. In the **Deploy Policy** window that appears, you can confirm the name of the current policy and the version to deploy. NGINX One Console defaults to the selected policy and latest version. 1. In the **Target** section, select Instance or Config Sync Group. 1. In the drop-down menu that appears, select the instance or Config Sync Group available in the current NGINX One Console. -------------------------------------------------------------------------------- title: "Configure log profiles" url: https://docs.nginx.com/nginx-one-console/waf-integration/log-profiles/configure-log-profiles/index.md -------------------------------------------------------------------------------- This document describes how to configure log profiles for F5 WAF for NGINX security logs in F5 NGINX One Console. Security logs contain information about HTTP requests and responses, how F5 WAF for NGINX processes them, and the final decision made based on your configured policy. Log profiles define which information is captured, the format of log messages, the destination for logs, and the filtering criteria for security events. ## Overview In NGINX One Console, you configure **Log Profiles** to control security logging for F5 WAF for NGINX. Security logs (also called Request logs or Traffic logs) provide visibility into what F5 WAF for NGINX detects and how it processes traffic according to your policies. F5 WAF for NGINX uses its own logging mechanism rather than NGINX's default access logging. With Log Profiles, you control: - **Filtering**: Which requests are logged (all requests, requests with violations, or blocked requests only) - **Format**: How log messages are structured (default, custom, Splunk, ArcSight, or BIG-IQ formats) - **Destination**: Where logs are sent (file or syslog server) - **Content**: What information is included in each log message (request details, violations, attack signatures, and more) - **Size limits**: Maximum sizes for log messages and request data For detailed information about security logging capabilities and available log attributes, see [Security Logs](/waf/logging/security-logs.md) and [Security logs examples](/waf/logging/security-logs.md#examples). ## Add a log profile From NGINX One Console, select **WAF** > **Log Profiles**. In the screen that appears, select **Add Log Profile**. This action opens a screen where you can: - In **Start From**, select an F5 WAF for NGINX [default logging profile](/waf/logging/logs-overview.md#default-logging-profile-bundles) as a starting point. - In **General Settings**, name and describe the log profile - Configure the filter settings to determine which requests are logged - Set the content format and options for how log messages are structured After you finish configuring all the settings, select **Add Profile** to save your log profile. ## Configure filter settings The **Request Type** filter determines which requests are logged based on what F5 WAF for NGINX detects: - **All**: Logs all requests, both legal and illegal - **Illegal**: Logs requests with violations (alerted or blocked) - **Blocked**: Logs requests with violations that were blocked Select the filter option that matches your monitoring and compliance needs. For production environments, you might start with **Blocked** to reduce log volume, then expand to **Illegal** or **All** as needed for troubleshooting. ## Configure content settings The content section specifies the format and structure of log messages. ### Format options Select one of the following log formats: - **Default**: Default format for F5 WAF for NGINX with comma-separated key-value pairs - **GRPC**: Variant of the default format suited for gRPC traffic - **User-defined**: Custom format that you define using a format string with placeholders - **Splunk**: Formatted for Splunk SIEM with F5 plugin - **ArcSight**: Formatted according to ArcSight Common Event Format (CEF) with custom fields adapted for F5 - **BIG-IQ**: Formatted for BIG-IQ, the F5 centralized management platform for BIG-IP ### Size limits Configure size restrictions for log messages: - **Max request size**: Limit in bytes for the `request` and `request_body_base64` fields (range: 1-10240 bytes, default: 2000 bytes). You can also set this to `any`, which is synonymous with 10240 bytes. - **Max message size**: Total size limit in KB for the entire log message (range: 1k-64k, default: 2k). Must not be smaller than `max_request_size`. ### Custom formatting If you select **User-defined** format, you can create a custom format string using placeholders for log attributes. For example: ``` Request ID %support_id%: %method% %uri% received on %date_time% from IP %ip_client% had the following violations: %violations% ``` Each attribute name is delimited by percent signs (for example, `%violation_rating%`). Available placeholders include attributes like `%ip_client%`, `%request%`, `%violations%`, `%attack_type%`, and many others. See the [Available security log attributes](/waf/logging/security-logs.md#available-security-log-attributes). ### Advanced options You can configure additional formatting options for how list values appear in your logs: - **List delimiter**: Character or string that separates list elements (default: comma) - **List prefix**: Character or string that starts a list (default: none) - **List suffix**: Character or string that ends a list (default: none) - **Escaping characters**: Replace specific characters in log values with alternative characters. Configure the `from` character to be replaced and the `to` result character. For detailed information about the JSON structure of security log configuration files (used in the Log Profile JSON section), see [Security log configuration file](/waf/logging/security-logs.md#security-log-configuration-file). ## Compile the log profile Before deploying a log profile, you can optionally compile the JSON configuration file into a bundle. If you don't compile manually, the deployment process will automatically compile the log profile. The compiled bundle is in compressed tar format (.tgz) and contains all the necessary configuration to enable security logging on your NGINX instances. ### Manage bundles for different compiler versions From the Log Profiles list, you can manage compiled bundles for your log profiles: 1. Go to **WAF** > **Log Profiles**. You'll see a list of all log profiles. 2. In the **Actions** column for a log profile, you can: - **Edit**: Open the log profile configuration editor to reconfigure settings - **Make a Copy**: Create a new log profile by copying the JSON content - **Deploy**: Deploy the log profile to instances or Config Sync Groups - **Download JSON**: Download the log profile JSON configuration - **Manage Bundles**: View and manage compiled bundles for different WAF compiler versions - **Delete**: Remove the log profile When you select **Manage Bundles**, you'll see all supported WAF compiler versions. For each version, you can see whether the log profile is compiled for that version. You can: - **Compile**: Compile the log profile into a bundle for a specific compiler version - **Download**: Download an existing compiled bundle for a specific compiler version This allows you to maintain compatibility with different versions of F5 WAF for NGINX across your infrastructure. ## Deploy the log profile After saving a log profile, deploy it to your NGINX instances to enable logging of WAF security events. See [Deploy log profiles](/nginx-one-console/waf-integration/log-profiles/deploy-log-profiles.md) for detailed deployment steps. The deployment process configures the required NGINX directives (`app_protect_security_log_enable` and `app_protect_security_log`) and ensures the log profile bundle is accessible to your instances. For detailed information about these directives and their configuration options, see [Security log directives](/waf/logging/security-logs.md#directives-in-nginxconf). For container-specific setup requirements, see the [Log profiles](/nginx-one-console/waf-integration/overview.md#log-profiles) configuration section in the overview. ## Review and manage log profiles From NGINX One Console, you can review the log profiles you've saved. For detailed information about reviewing and managing log profiles, see [Review log profiles](/nginx-one-console/waf-integration/log-profiles/review-log-profiles.md). -------------------------------------------------------------------------------- title: "Review log profiles" url: https://docs.nginx.com/nginx-one-console/waf-integration/log-profiles/review-log-profiles/index.md -------------------------------------------------------------------------------- Before you deploy a log profile to an NGINX instance or Config Sync Group, you may want to review it. F5 NGINX One Console creates a log profile for your F5 WAF for NGINX security logging. ## Review F5 WAF for NGINX log profiles From NGINX One Console, select **WAF** > **Log Profiles**. Select the name of the log profile that you want to review. You'll see the following tabs: - **Details**: Displays log profile information, including: - Last modified date and time (for example, 1/16/2026, 3:35:25 PM PST) - Total deployments - Log profile JSON configuration - **Deployments**: Shows deployment details for each instance or Config Sync Group, including: - Compiled version - Deployment status - Date deployed - Whether the latest log profile JSON was deployed - **Bundles**: Lists all WAF compiler versions with the following information: - Compiler version - Compilation status (Compiled, Not compiled, or Compiling) - Actions to compile or download the compiled bundle ## Modify existing log profiles From the NGINX One Console, you can also manage existing log profiles. In the Log Profiles screen, identify a log profile, and select **Actions**. From the menu that appears, you can: - **Edit**: Opens the log profile configuration editor where you can reconfigure settings. See [Configure log profiles](/nginx-one-console/waf-integration/log-profiles/configure-log-profiles.md) for details. - **Make a Copy**: Creates a new log profile by copying the JSON content into a new log profile object. You can use an existing log profile as a baseline for further customization. - **Deploy**: Applies the latest revision of the log profile to the configured instances and Config Sync Groups. - **Download JSON**: Downloads the log profile JSON configuration. - **Manage Bundles**: Opens a view to manage compiled bundles for different WAF compiler versions. - **Delete**: Removes the log profile. Once confirmed, you'll lose all work you've done on that log profile. -------------------------------------------------------------------------------- title: "Deploy log profiles" url: https://docs.nginx.com/nginx-one-console/waf-integration/log-profiles/deploy-log-profiles/index.md -------------------------------------------------------------------------------- After you've set up a log profile, it won't capture security events until you deploy it to instances or Config Sync Groups. ## Before you begin Before following the steps in this document, create a log profile in NGINX One Console that you're ready to deploy. ## Deploy a log profile To deploy a log profile from NGINX One Console, take the following steps: 1. Select **WAF** > **Log Profiles**. 1. Select the log profile that you're ready to deploy. 1. From **Actions**, select **Deploy**. 1. In the **Deploy Log Profile** window that appears, you can confirm the name of the current log profile. NGINX One Console defaults to the selected log profile. 1. In the **Target** section, select Instance or Config Sync Group. 1. In the drop-down menu that appears, select the instance or Config Sync Group. 1. Choose how to deploy the log profile: - **Add a new log profile path**: Specify a new file path where the log profile bundle should be deployed - **Update all log profiles**: Sync all log profiles on the target instance or Config Sync Group. This updates all existing log profiles by compiling their latest JSON contents into bundles and deploying them to all existing file paths If the log profile has not already been compiled for the WAF compiler version used by the target instance or Config Sync Group, NGINX One Console automatically compiles it into a bundle before deployment. ## Alternative: Deploy during configuration editing You can also deploy a log profile directly when editing the NGINX configuration for an instance or Config Sync Group. This method integrates log profile deployment into your regular configuration workflow. To deploy a log profile using the configuration editor: 1. Select **Instances** or **Config Sync Groups** and choose the target instance or Config Sync Group. 1. Select the **Configuration** tab and then **Edit Configuration**. 1. Select **Add File** and then choose **Existing Log Profile**. 1. In the **Select a Log Profile** drop-down menu, select the log profile you want to deploy. 1. In **Log Profile Destination**, specify the file path where the log profile bundle should be deployed, such as `/etc/nginx/app_protect/log_default.tgz`. 1. Select **Add**. NGINX One Console displays a code snippet with the required directives. 1. Paste the code snippet into your NGINX configuration. The snippet includes: - `app_protect_security_log_enable on` - `app_protect_security_log` with the log profile bundle path and destination For example: ``` app_protect_security_log_enable on; app_protect_security_log /etc/nginx/log-profile-bundle.tgz syslog:server=localhost:514; ``` 1. Select **Next** and then **Save and Publish**. For more information about adding files through the configuration editor, see [Add a file to a Config Sync Group](/nginx-one-console/nginx-configs/config-sync-groups/add-file-csg.md) or [Add a file to an instance](/nginx-one-console/nginx-configs/one-instance/add-file.md). ## Verify log profile deployment After deployment, verify that your log profile is active on the target instances or Config Sync Groups: 1. Confirm that the NGINX configuration includes the required directives: - `app_protect_security_log_enable on` - `app_protect_security_log` with the correct log profile bundle path and destination 2. Check that security logs are being generated at the configured destination (file path or syslog server). 3. Review the log entries to ensure they match the format and filter settings you configured in the log profile. To troubleshoot log profile deployment issues, see the [Container-related configuration requirements](/nginx-one-console/waf-integration/overview.md#container-related-configuration-requirements) section to ensure volumes and paths are correctly configured. -------------------------------------------------------------------------------- title: "Matching types: Explicit vs Wildcard" url: https://docs.nginx.com/nginx-one-console/waf-integration/policy/waf-policy-matching-types/index.md -------------------------------------------------------------------------------- In F5 WAF for NGINX (formerly known as NGINX App Protect WAF), matching can be defined in two ways: ## Explicit Matching Explicit matching refers to direct matches to specific names or paths in your application. For example: - URLs: `/index.html`, `/api/data` - Cookies: `sessionId`, `userPrefs` - Parameters: `username`, `email` Use explicit matching when you need to protect specific, known entities. ## Wildcard Matching Wildcard matching uses patterns to match multiple similar names or paths. For example: - URLs: `/test*` matches `/test`, `/test123`, `/testing` - Cookies: `test*` matches `test`, `test123`, `testing` - Parameters: `user*` matches `username`, `user_id`, `userEmail` Wildcard matching is useful when: - You need to protect multiple similar entities - You want to apply the same security controls to a group - The exact names or paths may vary or are dynamically generated Both explicit and wildcard matching allow you to configure additional properties, such as enforcement type, attack signatures, and more, depending on the entity being protected. -------------------------------------------------------------------------------- title: "Troubleshooting" url: https://docs.nginx.com/nginx-one-console/agent/support/troubleshooting/index.md -------------------------------------------------------------------------------- ## F5 NGINX Agent Troubleshooting **1. Container running but Agent is not connected to NGINX One Console?** - Check Agent logs ```bash docker logs ``` - If you are using NGINX Plus, a valid license will need to be passed into the container run command. - Ensure that the values sent with the container run command are correct. **2. Container running but instance is showing offline on NGINX One Console?** - Check Agent logs ```bash docker logs c5e1e3234900 | grep "nginx" ``` - Verify the following log message is shown: ```2025/02/17 19:02:58 [notice] 32#32: nginx/1.27.2 (nginx-plus-r33-p2 ``` - If not found, it could mean the container image is missing the NGINX service - Make sure NGINX is running ```ps -ef | grep "nginx"``` - Make sure NGINX is part of the image file. **3. NGINX Agent is installed on my Virtual Machine, but not showing up on NGINX One Console?** - Verify the agent is running ```sudo systemctl status nginx-agent``` - Check for any errors in the logs ```sudo tail -f /var/log/nginx-agent/agent.log``` - Check ```/etc/nginx-agent/nginx-agent.conf``` for any misconfigurations. -------------------------------------------------------------------------------- title: "Set security policies through the API" url: https://docs.nginx.com/nginx-one-console/waf-integration/policy/security-policy-api/index.md -------------------------------------------------------------------------------- You can use F5 NGINX One Console API to manage security policies. With our API, you can: - [List existing policies](/nginx-one-console/api/api-reference-guide/#operation/listNapPolicies) - You can set parameters to sort policies by type. - [Create a new policy](/nginx-one-console/api/api-reference-guide/#operation/createNapPolicy) - You need to translate the desired policy.json file to base64 format. - [Get policy details](/nginx-one-console/api/api-reference-guide/#operation/getNapPolicy) - Returns details of the policy you identified with the policy `object_id`. - [List F5 WAF for NGINX Deployments](/nginx-one-console/api/api-reference-guide/#operation/listNapPolicyDeployments) - The output includes: - Target of the deployment - Time of deployment - Enforcement mode - Policy version - Threat campaign - Attack signature - Bot signature -------------------------------------------------------------------------------- title: "Set log profiles through the API" url: https://docs.nginx.com/nginx-one-console/waf-integration/log-profiles/log-profile-api/index.md -------------------------------------------------------------------------------- You can use F5 NGINX One Console API to manage log profiles. With our API, you can: - [List log profiles](/nginx-one-console/api/api-reference-guide/#operation/listWafLogProfiles) - Returns WAF log profiles. - [Create a log profile](/nginx-one-console/api/api-reference-guide/#operation/createWafLogProfile) - Creates WAF log profile. - [Get log profile details](/nginx-one-console/api/api-reference-guide/#operation/getWafLogProfile) - Returns WAF log profile details, including the JSON configuration contents. - [Update log profile details](/nginx-one-console/api/api-reference-guide/#operation/updateWafLogProfile) - Updates WAF log profile details. - [Delete a log profile](/nginx-one-console/api/api-reference-guide/#operation/deleteWafLogProfile) - Deletes WAF log profile. - [List log profile deployments](/nginx-one-console/api/api-reference-guide/#operation/listWafLogProfileDeployments) - Returns WAF log profile deployments, providing details such as: - Target of the deployment, either an instance or CSG - Time of deployment - Deployment status - [Compile a log profile](/nginx-one-console/api/api-reference-guide/#operation/compileWafLogProfile) - Compiles the log profile into a bundle (.tgz) for a specific WAF compiler version. Use `download=true` to download the compiled bundle, or `download=false` to retrieve the compilation status (whether the bundle is already compiled or compilation is pending). -------------------------------------------------------------------------------- title: "Clean up unavailable NGINX instances" url: https://docs.nginx.com/nginx-one-console/nginx-configs/clean-up-unavailable-instances/index.md -------------------------------------------------------------------------------- ## Overview This guide explains how to set up automatic cleanup for NGINX instances in NGINX One. The cleanup process removes instances that have been unavailable for a specified duration. By default, this period is 24 hours from the time the NGINX instance was last updated. Administrators can change or disable the cleanup duration in **Settings > Instance Settings**. Events will be generated for NGINX instances that have been automatically cleaned up; you can see these events on the **Overview > Events** page. ## Before you start Before you set up automatic cleanup for NGINX instances, ensure: - You have [administrator access](/nginx-one-console/rbac/roles.md) to NGINX One Console. - You understand that this action will delete instances permanently after they are unavailable for the specified duration. ## Configure instance cleanup Follow these steps to set up automatic cleanup for NGINX instances in NGINX One: 1. On the left menu, select **Instance Settings**. 1. On the **Instance Settings** page, in the **Unavailable Instance Cleanup** section, select **Edit Duration**. 1. Choose the cleanup duration. - Select one of the predefined durations (None, 1 day, 7 days, 30 days) or set a custom duration. Selecting **None** disables automatic cleanup. - If you choose **Custom**, enter the duration in hours or days. 1. Select **Save** to apply the changes. ## Event log details When instances are cleaned up automatically, an event log entry is created. You can find these events on the **Overview > Events** page. The event log includes the following details: - **Impacted Object ID**: The unique identifier of the NGINX instance that was cleaned up. - **Type**: The type of event, which will be "Automated Object Cleanup". - **Timestamp**: The date and time when the instance was cleaned up. - **Message**: A description indicating that the instance was unavailable for the configured duration before being cleaned up. -------------------------------------------------------------------------------- title: "Glossary" url: https://docs.nginx.com/nginx-one-console/glossary/index.md -------------------------------------------------------------------------------- This glossary defines terms used in the F5 NGINX One Console and F5 Distributed Cloud. ## General terms | Term | Definition | |-------------|-------------| | **Config Sync Group** | A group of NGINX systems (or instances) with identical configurations. They may also share the same certificates. However, the instances in a Config Sync Group could belong to different systems and even different clusters. For more information, see this explanation of [Important considerations](/nginx-one-console/nginx-configs/config-sync-groups/manage-config-sync-groups.md#important-considerations) | | **Control Plane** | The control plane is the part of a network architecture that manages and controls the flow or data or traffic (the Data Plane). It is responsible for system-level tasks such as routing and traffic management. | | **Data Plane** | The data plane is the part of a network architecture that carries user traffic. It handles tasks like forwarding data packets between devices and managing network communication. In the context of NGINX, the data plane is responsible for tasks such as load balancing, caching, and serving web content. | | **Instance** | An instance is an individual system with NGINX installed. You can group the instances of your choice in a Config Sync Group. When you add an instance to NGINX One, you need to use a data plane key. | | **Namespace** | In F5 Distributed Cloud, a namespace groups a tenant’s configuration objects, similar to administrative domains. Every object in a namespace must have a unique name, and each namespace must be unique to its tenant. This setup ensures isolation, preventing cross-referencing of objects between namespaces. You'll see the namespace in the NGINX One Console URL as `/namespaces//`. To switch an instance between namespaces, you have to deregister an instance from an old namespace, and register it on the new namespace. | | **NGINX Agent** | A lightweight software component installed on NGINX instances to enable communication with the NGINX One console. | | **Staged Configurations** | Also known as **Staged Configs**. Allows you to save "work in progress." You can create it from scratch, an Instance, another Staged Config, or a Config Sync Group. It does _not_ have to be a working configuration until you publish it to an instance or a Config Sync Group. You can even manage your **Staged Configurations** through our [API](/nginx-one-console/api/api-reference-guide/#tag/StagedConfigs). | | **Tenant** | A tenant in F5 Distributed Cloud is an entity that owns a specific set of configuration and infrastructure. It is fundamental for isolation, meaning a tenant cannot access objects or infrastructure of other tenants. Tenants can be either individual or enterprise, with the latter allowing multiple users with role-based access control (RBAC). | ## F5 WAF for NGINX terminology [F5 WAF for NGINX Terminology](waf/fundamentals/terminology.md). ## NGINX Alerts To set up NGINX Alerts through the F5 Distributed Cloud, follow the procedure in [Set up security alerts](/nginx-one-console/secure-your-fleet/set-up-security-alerts/). You can configure a variety of NGINX alerts in the F5 Distributed Cloud. If you have access to the [F5 Distributed Cloud](/nginx-one-console/getting-started.md#confirm-access-to-the-f5-distributed-cloud), log in and select the **Audit Logs & Alerts** tile. Go to **Notifications > Alerts**. Select the gear icon and select **Alert Name > Active Alerts**. You may see one or more of the following alerts in the **Audit Logs & Alerts** Console. ### Alert Labels | **Alertname** | **Description** | **Alert Level** | **Action** | |--------------------------------|----------------------------------------------------------------------|-----------------|------------------------------------------------------------------------------------------------------------------| | HighCVENGINX | A high-severity CVE is impacting an NGINX instance | Critical | Review the CVE details in the NGINX One Console. Apply updates or change configurations to resolve the vulnerability. | | MediumCVENGINX | A medium-severity CVE is impacting an NGINX instance | Major | Review the CVE details in the NGINX One Console. Apply updates or configuration changes as needed. | | LowCVENGINX | A low-severity CVE is impacting an NGINX instance | Minor | Review the CVE details in the NGINX One Console. Consider updates or configuration changes to maintain security. | | SecurityRecommendationNGINX | A security recommendation has been found for an NGINX configuration | Critical | Review the configuration issue in the NGINX One Console. Follow the recommendations to secure the instance or Config Sync Group. | | OptimizationRecommendationNGINX| An optimization recommendation has been found for an NGINX configuration| Major | Review the optimization details in the NGINX One Console. Update the configuration to for the instance or Config Sync Group to enhance performance. | | BestPracticeRecommendationNGINX| A best practice recommendation has been found for an NGINX configuration | Minor | Review the best practice recommendation in the NGINX One Console. Update the configuration for the instance or Config Sync Group to align with industry standards. | | NGINXOffline | An NGINX instance is now offline | Major | Verify the host is online. Check the NGINX Agent's status on the instance and ensure it is connected to the NGINX One Console. | | NGINXUnavailable | An NGINX instance is now unavailable | Major | Ensure the NGINX Agent and host are active. Verify the NGINX Agent can connect to the NGINX One Console and resolve any network issues. | | NewNGINX | A new NGINX instance has connected to NGINX One | Minor | Review the instance details in the NGINX One Console. Confirm availability, CVEs, and recommendations to ensure the instance is operational. | | NGINXCertificateExpired | A certificate on an NGINX instance or Config Sync Group has expired | Critical | Get a new certificate from your certificate authority, then update the TLS/SSL certificate in the NGINX One Console. | | NGINXCertificateExpiring | A certificate on an NGINX instance or Config Sync Group will expire soon | Major | Renew the certificate through your certificate authority, then update the TLS/SSL certificate in the NGINX One Console. | ## Legal notice: Licensing agreements for NGINX products Using NGINX One is subject to our End User Service Agreement (EUSA). For [NGINX Plus](/nginx), usage is governed by the End User License Agreement (EULA). Open source projects, including [NGINX Agent](https://github.com/nginx/agent) and [NGINX Open Source](https://github.com/nginx/nginx), are covered under their respective licenses. For more details on these licenses, follow the provided links. ## References - [F5 Distributed Cloud: Core Concepts](https://docs.cloud.f5.com/docs/ves-concepts/core-concepts) -------------------------------------------------------------------------------- title: "Agent changelog" url: https://docs.nginx.com/nginx-one-console/agent/changelog/index.md -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- title: "API reference guide" url: https://docs.nginx.com/nginx-one-console/api/api-reference-guide/index.md -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- title: "Enable metrics" url: https://docs.nginx.com/nginx-one-console/nginx-configs/metrics/enable-metrics/index.md -------------------------------------------------------------------------------- The NGINX One Console dashboard and metrics views present system metrics and detailed NGINX metrics gathered through the NGINX Plus API or the Stub Status API and NGINX access log (for NGINX Open Source). ## Enable NGINX Plus Metrics ### Enable NGINX Plus API and dashboard #### without SSL To collect comprehensive metrics for NGINX Plus, including bytes streamed, information about upstream systems and caches, and counts of all HTTP status codes, add the following to your NGINX Plus configuration file, for example `/etc/nginx/nginx.conf` or an included file: **Note:** - By default, all clients can call the API. - To limit who can access the API, uncomment the `allow` and `deny` lines under `api write=on` and replace the example CIDR with your trusted network. - To restrict write methods (`POST`, `PATCH`, `DELETE`), uncomment and configure the `limit_except GET` block and set up [HTTP basic authentication](https://nginx.org/en/docs/http/ngx_http_auth_basic_module.html). **Note:** If there are issues with NGINX Agent discovering the NGINX Plus API, NGINX Agent can be manually configured with the address of the NGINX Plus API. - Add the following configuration to `/etc/nginx-agent/nginx-agent.conf`: ``` data_plane_config: nginx: api: url: "http://127.0.0.1:9000/api" ``` - Restart NGINX Agent for the configuration changes to take affect ``` sudo systemctl restart nginx-agent ``` - Run the following command ``` sudo journalctl -u nginx-agent | grep "Found NGINX Plus API" ``` - Ensure that the following log message is seen ``` Found NGINX Plus API ``` For more details, see the [NGINX Plus API module](https://nginx.org/en/docs/http/ngx_http_api_module.html) documentation and [Configuring the NGINX Plus API](/nginx/admin-guide/monitoring/live-activity-monitoring.md#configuring-the-api). #### with SSL If SSL is enabled on the NGINX Plus API with self-signed certificates like this example: ```nginx # This block enables the NGINX Plus API and dashboard with SSL # For configuration and security recommendations, see: # https://docs.nginx.com/nginx/admin-guide/monitoring/live-activity-monitoring/#configuring-the-api server { # Change the listen port if 9000 conflicts # (8080 is the conventional API port) listen 9000 ssl; ssl_certificate /etc/nginx/certs/nginx-selfsigned.crt; ssl_certificate_key /etc/nginx/certs/nginx-selfsigned.key; location /api/ { # To restrict write methods (POST, PATCH, DELETE), uncomment: # limit_except GET { # auth_basic "NGINX Plus API"; # auth_basic_user_file /path/to/passwd/file; # } # Enable API in write mode api write=on; # To restrict access by network, uncomment the following lines and set your network: # allow 192.0.2.0/24; # replace with your network # allow 127.0.0.1/32; # allow local NGINX Agent to call the NGINX Plus API to retrieve metrics # deny all; } # Serve the built-in dashboard at /dashboard.html location = /dashboard.html { root /usr/share/nginx/html; } } ``` **Note:** Make sure that the `server` and `location` blocks are in the same configuration file, and not split across multiple files using `include` directives. **Note:** To enable NGINX Agent to call the NGINX Plus API, follow the steps below: - Add the following configuration to `/etc/nginx-agent/nginx-agent.conf`: ``` data_plane_config: nginx: api: url: "https://127.0.0.1:9000/api" tls: ca: "/etc/nginx/certs/nginx-selfsigned.crt" ``` - Restart NGINX Agent for the configuration changes to take affect ``` sudo systemctl restart nginx-agent ``` - Run the following command ``` sudo journalctl -u nginx-agent | grep "Found NGINX Plus API" ``` - Ensure that the following log message is seen ``` Found NGINX Plus API ``` **Note:** Here is an example of how to generate self-signed certificates ``` openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/nginx/certs/nginx-selfsigned.key -out /etc/nginx/certs/nginx-selfsigned.crt -subj "/CN=localhost" -addext "subjectAltName=IP:127.0.0.1" ``` ### Enable NGINX Plus API and dashboard with Config Sync Groups To enable the NGINX Plus API and dashboard with [Config Sync Groups](/nginx-one-console/nginx-configs/config-sync-groups/manage-config-sync-groups.md), add a file named `/etc/nginx/conf.d/dashboard.conf` to your shared group config. Any instance you add to that group automatically uses those settings. 1. In the NGINX One Console, select **Manage > Config Sync Groups**, then pick your config sync group's name. 2. Select the **Configuration** tab, then select **Edit Configuration**. 3. Select **Add File**. 4. Select **New Configuration File**. 5. In the **File name** box, enter `/etc/nginx/conf.d/dashboard.conf`, then select **Add**. 6. Paste the following into the new file workspace: #### without SSL #### with SSL 7. Select **Next**, review the diff, then select **Save and Publish**. 8. Open your browser to `http://:9000/dashboard.html` (replace `` with the IP or hostname of one of your group members). You should see the NGINX Plus dashboard. ### Enable NGINX Plus Metric Collection To make NGINX Plus metrics available on the NGINX One Console, you must enable shared memory zones for the virtual servers being monitored. Shared memory zones store configuration and runtime state information shared across NGINX worker processes. To display [HTTP](nginx/admin-guide/load-balancer/http-load-balancer.md) and [TCP](nginx/admin-guide/load-balancer/tcp-udp-load-balancer.md) servers in NGINX Console, one or more status_zone directives must be defined. The same zone name can be reused across multiple server blocks. Since [R19](nginx/releases.md#r19), you can apply the status_zone directive to location blocks, allowing statistics to be aggregated separately for servers and locations. ```nginx server { # ... status_zone status_page; location / { proxy_pass http://backend; status_zone location_zone; } } ``` After saving the changes, reload NGINX to apply the new configuration: ```shell nginx -s reload ``` ## Enable NGINX Open Source Metrics To collect basic metrics about server activity for NGINX Open Source: 1. **Enable the stub status API** Add the following to your NGINX configuration file: ```nginx server { listen 127.0.0.1:8080; location /api { stub_status; allow 127.0.0.1; deny all; } } ``` **Note:** Make sure that the `server` and `location` blocks are in the same configuration file, and not split across multiple files using `include` directives. This configuration: - Enables the stub status API endpoint. - Allows requests only from `127.0.0.1` (localhost). - Blocks all other requests for security. For more details, see the [NGINX Stub Status module documentation](https://nginx.org/en/docs/http/ngx_http_stub_status_module.html). 2. **Configure access logging** Enable access logging in your NGINX configuration to collect detailed traffic metrics. Ensure that the following log format is used: ```nginx log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for" ' '"$bytes_sent" "$request_length" "$request_time" ' '"$gzip_ratio" $server_protocol '; access_log /var/log/nginx/access.log main; ``` This log format captures key metrics including request timing, response sizes, and client information. -------------------------------------------------------------------------------- title: "Review metrics on the NGINX One dashboard" url: https://docs.nginx.com/nginx-one-console/nginx-configs/metrics/review-metrics/index.md -------------------------------------------------------------------------------- After connecting your NGINX instances to NGINX One, you can monitor their performance and health. The NGINX One dashboard is designed for this purpose, offering an easy-to-use interface. ### Log in to NGINX One 1. Log in to [F5 Distributed Console](https://www.f5.com/cloud/products/distributed-cloud-console). 1. Select **NGINX One > Visit Service**. Navigating the dashboard: - **Drill down into specifics**: For in-depth information on a specific metric, like expiring certificates, click on the relevant link in the metric's card to go to a detailed overview page. - **Refine metric timeframe**: Metrics show the last hour's data by default. To view data from a different period, select the time interval you want from the drop-down menu. ![image](nginx-one-console/images/nginx-one-dashboard.png) **NGINX One Console dashboard metrics** | Metric | Description | Details | |---|---|---| | **Instance availability** | Understand the operational status of your NGINX instances. | - **Online**: The NGINX instance is actively connected and functioning properly.
- **Offline**: NGINX Agent is connected but the NGINX instance isn't running, isn't installed, or can't communicate with NGINX Agent.
- **Unavailable**: The connection between NGINX Agent and NGINX One Console has been lost or the instance has been decommissioned.
- **Unknown**: The current state can't be determined at the moment. | | **NGINX versions by instance** | See which NGINX versions are in use across your instances. | | | **Operating systems** | Find out which operating systems your instances are running on. | | | **Certificates** | Monitor the status of your SSL certificates to know which are expiring soon and which are still valid. | | | **Config recommendations** | Get configuration recommendations to optimize your instances' settings. | | | **CVEs (Common Vulnerabilities and Exposures)** | Evaluate the severity and number of potential security threats in your instances. | - **High**: Indicates a high-severity threat that needs immediate attention. NGINX CVSS score = 7.0-10.0
- **Medium**: Implies a moderate threat level. NGINX CVSS score = 4.0-6.9
- **Low**: Represent less critical issues that still require monitoring. NGINX CVSS score = 0.1-3.9.
- **None**: NGINX CVSS score = 0.0| | **CPU utilization** | Track CPU usage trends and pinpoint instances with high CPU demand. | | | **Memory utilization** | Watch memory usage patterns to identify instances using significant memory. | | | **Disk space utilization** | Monitor how much disk space your instances are using and identify those nearing capacity. | | | **Unsuccessful response codes** | Look for instances with a high number of HTTP server errors and investigate their error codes. | | | **Top network usage** | Review the network usage and bandwidth consumption of your instances. | | -------------------------------------------------------------------------------- title: "F5 NGINXaaS for Azure" url: https://docs.nginx.com/nginxaas-azure/index.md -------------------------------------------------------------------------------- ## About F5 NGINXaaS for Azure is a SaaS offering that is tightly integrated into Microsoft Azure and its ecosystem of services, making applications fast, efficient, and reliable bringing advanced traffic services enabled with the commercial version of NGINX, without any of the operational toil. ## Featured content ### [Getting started](/nginxaas-azure/getting-started/) Deploy NGINXaaS for Azure using the Azure portal, Azure CLI, or Terraform ### [Quickstart guides](/nginxaas-azure/quickstart/) Step-by-step guides for several common use cases, including scaling guidance, security controls, and more ### [Logging and monitoring](/nginxaas-azure/monitoring/) Collect, correlate, and analyze metrics for a thorough understanding of your application's health and behavior ### Billing ### [Marketplace billing](/nginxaas-azure/billing/overview/) See the pricing plans and learn about NGINX Capacity Units (NCUs) ### Certificates ### [Add certificates using the Azure portal](/nginxaas-azure/getting-started/ssl-tls-certificates/ssl-tls-certificates-portal/) Learn to manage SSL/TLS certificates using the Azure portal ### More information ### [Feature comparison](/nginxaas-azure/overview/feature-comparison/) Learn about the differences between NGINXaaS for Azure and NGINX Plus ### [Changelog](/nginxaas-azure/changelog/) See the latest updates: New features, improvements, and bug fixes -------------------------------------------------------------------------------- title: "Azure Resource Health" url: https://docs.nginx.com/nginxaas-azure/monitoring/azure-resource-health/index.md -------------------------------------------------------------------------------- ## Overview [Azure Resource Health](https://learn.microsoft.com/en-us/azure/service-health/resource-health-overview) is automatically enabled for all NGINXaaS deployments. It provides an up-to-date overview of deployment status and reveals ways in which NGINXaaS deployments may have become degraded over time. We encourage users to make full use of this functionality to maintain their deployments in a healthy state. ## Portal experience Users can explore resource health by locating their NGINXaaS deployment in the Azure portal, opening the "Help" section on the Service menu and selecting "Resource health". The **Resource Health** blade contains the health status of your NGINXaaS deployment as well a list of recent health events (if applicable). If your deployment is degraded, the Azure Resource Health status message will include actions you can take to repair your deployment. Users should follow Azure guidance to set up [alerting](https://learn.microsoft.com/en-us/azure/service-health/resource-health-alert-monitor-guide) for resource health events on their NGINXaaS deployments. ## Azure Resource Health API Users can also interact with the [Azure Resource Health API](https://learn.microsoft.com/en-us/rest/api/resourcehealth/operation-groups) to unlock the functionality of Azure Resource Health. -------------------------------------------------------------------------------- title: "Create NGINXaaS Dataplane API Key" url: https://docs.nginx.com/nginxaas-azure/quickstart/dataplane-api-key/index.md -------------------------------------------------------------------------------- ## Overview The NGINXaaS dataplane API key is used to authenticate with your NGINXaaS deployment's dataplane API. This key is required for various integrations including VMSS load balancing and Kubernetes load balancing. ## Requirements **Note:** The data plane API key has the following requirements: - The key should have an expiration date. The default expiration date is six months from the date of creation. The expiration date cannot be longer than two years from the date of creation. - The key should be at least 12 characters long. - The key requires three out of four of the following types of characters: - lowercase characters. - uppercase characters. - symbols. - numbers. A good example of an API key that will satisfy the requirements is UUIDv4. ## Create the API Key The data plane API key can be created using the Azure CLI or portal. ### Create an NGINXaaS data plane API key using the Azure portal 1. Go to your NGINXaaS for Azure deployment. 2. Select **NGINXaaS Loadbalancer for Kubernetes** on the left blade. 3. Select **New API Key**. 4. Provide a name for the new API key in the right panel, and select an expiration date. 5. Select the **Add API Key** button. 6. Copy the value of the new API key. **Note:** Make sure to write down the key value in a safe location after creation, as you cannot retrieve it again. If you lose the generated value, delete the existing key and create a new one. ### Create an NGINXaaS data plane API key using the Azure CLI Set shell variables about the name of the NGINXaaS you've already created: ```bash ## Customize this to provide the details about my already created NGINXaaS deployment nginxName=myNginx nginxGroup=myNginxGroup ``` Generate a new random data plane API key: ```bash # Generate a new random key or specify a value for it. keyName=myKey keyValue=$(uuidgen --random) ``` Create the key for your NGINXaaS deployment: ```bash az nginx deployment api-key create --name $keyName --secret-text $keyValue --deployment-name $nginxName --resource-group $nginxGroup ``` ## Get the Dataplane API Endpoint The data plane API endpoint can be retrieved using the Azure CLI or portal. ### View NGINXaaS data plane API endpoint using the Azure portal 1. Go to your NGINXaaS for Azure deployment. 2. Select **NGINXaaS Loadbalancer for Kubernetes** on the left blade. 3. The data plane API endpoint associated with the deployment is available at the top of the screen. ### View NGINXaaS data plane API endpoint using the Azure CLI ```bash dataplaneAPIEndpoint=$(az nginx deployment show -g "$nginxGroup" -n "$nginxName" --query properties.dataplaneApiEndpoint -o tsv) ``` ## Next Steps Once you have created the dataplane API key and obtained the endpoint, you can use them in various NGINXaaS integrations: - [Load Balancer for VMSS](/nginxaas-azure/loadbalancer-vmss/) - [Load Balancer for Kubernetes](/nginxaas-azure/loadbalancer-kubernetes/) -------------------------------------------------------------------------------- title: "Overview" url: https://docs.nginx.com/nginxaas-azure/getting-started/nginx-configuration/overview/index.md -------------------------------------------------------------------------------- This document provides details about using NGINX configuration files with your F5 NGINXaaS for Azure deployment, restrictions, and available directives. ## NGINX configuration common user workflows NGINX configurations can be uploaded to your NGINXaaS for Azure deployment using the Azure portal, Azure CLI, or Terraform. The following documents provide detailed steps on how to upload NGINX configurations: - [Upload using the Azure portal](/nginxaas-azure/getting-started/nginx-configuration/nginx-configuration-portal.md) - [Upload using the Azure CLI](/nginxaas-azure/getting-started/nginx-configuration/nginx-configuration-azure-cli) - [Upload using Terraform](/nginxaas-azure/getting-started/nginx-configuration/nginx-configurations-terraform.md) The topics below provide information on NGINX configuration restrictions and directives that are supported by NGINXaaS for Azure when using any of the above workflows. ## NGINX configuration automation workflows NGINX configurations stored in GitHub can be applied to existing NGINXaaS for Azure deployments using custom GitHub Action workflows. See [NGINXaaS for Azure Deployment Action](https://github.com/nginxinc/nginx-for-azure-deploy-action) for documentation and examples on how to incorporate these workflows in your GitHub Actions CI/CD pipelines. ## NGINX filesystem restrictions There are limits to where files, including NGINX configuration files, certificate files, and any other files uploaded to the deployment, can be placed on the filesystem. There are also limits on what directories NGINX can access during runtime. These limits help support the separation of roles, enforce the principle of least privilege, and ensure the smooth operation of the system. | Directory | Master Read | Master Write | Worker Read | Worker Write | Recommended Use | |-------------------|:-----------:|:------------:|:-----------:|:------------:|----------------------------------| | /etc/nginx/ | ✔️ | ✔️ | ❌ | ❌ | NGINX configuration, certificates, keys, application files (e.g. Lua or njs scripts) | | /opt/ | ✔️ | ✔️ | ✔️ | ❌ | Application files (e.g. Lua scripts) | | /srv/ | ✔️ | ✔️ | ✔️ | ❌ | Application files | | /var/www/ | ✔️ | ✔️ | ✔️ | ❌ | Static files (e.g. index.html) | | /tmp/ | ✔️ | ✔️ | ✔️ | ✔️ | Temporary files | | /var/cache/nginx/ | ✔️ | ✔️ | ✔️ | ✔️ | Cache data | For example, `/etc/nginx` is only readable by the NGINX master process, making it a secure location for certificate files that won't be accidentally served due to configuration errors. `/var/www` is a secure location for static content because the NGINX worker process can serve files from it but cannot modify them, ensuring content integrity. `/tmp` is a good choice for storing temporary files with `proxy_temp_path` or `client_body_temp_path` since it is writable by the NGINX worker process. Accessing files outside of the listed directories is unsupported. Such access may or may not function and is subject to change without notice. ## Disallowed configuration directives Some directives are not supported because of specific limitations. If you include one of these directives in your NGINX configuration, you'll get an error. | Disallowed Directive | Reason | |------------------ | ----------------- | | ssl_engine | No hardware SSL accelerator is available. | | debug_points | NGINXaaS does not provide access to NGINX processes for debugging. | | fastcgi_bind
fastcgi_bind_dynamic
grpc_bind
grpc_bind_dynamic
memcached_bind
memcached_bind_dynamic
proxy_bind
proxy_bind_dynamic
scgi_bind
scgi_bind_dynamic
tunnel_bind
tunnel_bind_dynamic
uwsgi_bind
uwsgi_bind_dynamic | Source IP specification for active-active deployments is not allowed. | | quic_bpf | QUIC connection migration is not currently supported for active-active deployments. | acme_issuer
uri
account_key
contact
ssl_trusted_certificate
ssl_verify
state_path
accept_terms_of_service
acme_shared_zone
acme_certificate
| ACME protocol support for active-active deployments is not supported. You may find that a few directives are not listed here as either allowed or disallowed. Our team is working on getting these directives supported soon. ## Directives that cannot be overridden Some directives cannot be overridden by the user provided configuration. | Persistent Directive | Value | Reason | |------------------ | ----------------------- | -----------------| | `user` | `nginx` | The `nginx` user has the correct permissions for accessing certificates, policy files and other auxfiles. | | `worker_processes` | `auto` | Set to `auto` to automatically set `worker_processes` to the number of CPU cores. | | `worker_connections` |
  • Standard V3 plan `30000`
  • basic plan `3000`
| To ensure reasonable performance of the NGINXaaS deployment for Standard V3 plan, the `worker_connections` is fixed at 3000/NCU; for basic plan this is set lower.

Changing the value in the config has no effect on the NGINX process in the deployment. | | `pid` | `/run/nginx/nginx.pid` | Set to this value to allow NGINXaaS to automatically manage the NGINX master process. | | `daemon` | `on` | Automatically set to `on` to allow NGINXaaS to manage the NGINX master process. | | `master_process` | `on` | This directive is intended for NGINX developers. | | `worker_cpu_affinity` | `auto` | The value `auto` allows binding worker processes automatically to available CPUs based on the current capacity of the deployment. | ## NGINX listen port restrictions - Due to port restrictions on Azure Load Balancer health probes, certain ports are not allowed for the `listen` directive in NGINX configuration. The following ports are blocked: - `19`, `21`, `70`, `119` - Azure health probe restricted ports - `49151`, `49153`, `5140`, `50000`, `54141`, `54779` - reserved ports to support other NGINXaaS features - The [Basic](/nginxaas-azure/billing/overview.md#basic-plan) plan (and the deprecated Standard (v1) plan) supports a maximum of 5 listen ports in the NGINX configuration. Configurations that specify over 5 unique ports are rejected. - The [Standard V3](/nginxaas-azure/billing/overview.md#standard-v3-plan) plan allow users to listen on more than 5 ports. The first five ports under this plan come at no extra cost and there are charges for each additional port utilized. ## Configuration directives list NGINXaaS for Azure supports a limited set of NGINX directives. #### Alphabetical index of directives [absolute_redirect](https://nginx.org/en/docs/http/ngx_http_core_module.html#absolute_redirect)\ [accept_mutex](https://nginx.org/en/docs/ngx_core_module.html#accept_mutex)\ [accept_mutex_delay](https://nginx.org/en/docs/ngx_core_module.html#accept_mutex_delay)\ [access_log (ngx_http_log_module)](https://nginx.org/en/docs/http/ngx_http_log_module.html#access_log)\ [access_log (ngx_stream_log_module)](https://nginx.org/en/docs/stream/ngx_stream_log_module.html#access_log)\ [add_after_body](https://nginx.org/en/docs/http/ngx_http_addition_module.html#add_after_body)\ [add_before_body](https://nginx.org/en/docs/http/ngx_http_addition_module.html#add_before_body)\ [add_header](https://nginx.org/en/docs/http/ngx_http_headers_module.html#add_header)\ [add_header_inherit](https://nginx.org/en/docs/http/ngx_http_headers_module.html#add_header_inherit)\ [add_trailer](https://nginx.org/en/docs/http/ngx_http_headers_module.html#add_trailer)\ [add_trailer_inherit](https://nginx.org/en/docs/http/ngx_http_headers_module.html#add_trailer_inherit)\ [addition_types](https://nginx.org/en/docs/http/ngx_http_addition_module.html#addition_types)\ [aio](https://nginx.org/en/docs/http/ngx_http_core_module.html#aio)\ [aio_write](https://nginx.org/en/docs/http/ngx_http_core_module.html#aio_write)\ [alias](https://nginx.org/en/docs/http/ngx_http_core_module.html#alias)\ [allow (ngx_http_access_module)](https://nginx.org/en/docs/http/ngx_http_access_module.html#allow)\ [allow (ngx_stream_access_module)](https://nginx.org/en/docs/stream/ngx_stream_access_module.html#allow)\ [ancient_browser](https://nginx.org/en/docs/http/ngx_http_browser_module.html#ancient_browser)\ [ancient_browser_value](https://nginx.org/en/docs/http/ngx_http_browser_module.html#ancient_browser_value)\ [auth_basic](https://nginx.org/en/docs/http/ngx_http_auth_basic_module.html#auth_basic)\ [auth_basic_user_file](https://nginx.org/en/docs/http/ngx_http_auth_basic_module.html#auth_basic_user_file)\ [auth_delay](https://nginx.org/en/docs/http/ngx_http_core_module.html#auth_delay)\ [auth_http](https://nginx.org/en/docs/mail/ngx_mail_auth_http_module.html#auth_http)\ [auth_http_header](https://nginx.org/en/docs/mail/ngx_mail_auth_http_module.html#auth_http_header)\ [auth_http_pass_client_cert](https://nginx.org/en/docs/mail/ngx_mail_auth_http_module.html#auth_http_pass_client_cert)\ [auth_http_timeout](https://nginx.org/en/docs/mail/ngx_mail_auth_http_module.html#auth_http_timeout)\ [auth_jwt](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt)\ [auth_jwt_claim_set](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_claim_set)\ [auth_jwt_header_set](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_header_set)\ [auth_jwt_key_cache](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_cache)\ [auth_jwt_key_file](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_file)\ [auth_jwt_key_request](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_request)\ [auth_jwt_leeway](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_leeway)\ [auth_jwt_require](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_require)\ [auth_jwt_type](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_type)\ [auth_oidc](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#auth_oidc)\ [auth_require](https://nginx.org/en/docs/http/ngx_http_auth_require_module.html#auth_require)\ [auth_request](https://nginx.org/en/docs/http/ngx_http_auth_request_module.html#auth_request)\ [auth_request_set](https://nginx.org/en/docs/http/ngx_http_auth_request_module.html#auth_request_set)\ [autoindex](https://nginx.org/en/docs/http/ngx_http_autoindex_module.html#autoindex)\ [autoindex_exact_size](https://nginx.org/en/docs/http/ngx_http_autoindex_module.html#autoindex_exact_size)\ [autoindex_format](https://nginx.org/en/docs/http/ngx_http_autoindex_module.html#autoindex_format)\ [autoindex_localtime](https://nginx.org/en/docs/http/ngx_http_autoindex_module.html#autoindex_localtime)\ [break](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#break)\ [connect_timeout](https://nginx.org/en/docs/ngx_mgmt_module.html#connect_timeout)\ [charset](https://nginx.org/en/docs/http/ngx_http_charset_module.html#charset)\ [charset_map](https://nginx.org/en/docs/http/ngx_http_charset_module.html#charset_map)\ [charset_types](https://nginx.org/en/docs/http/ngx_http_charset_module.html#charset_types)\ [chunked_transfer_encoding](https://nginx.org/en/docs/http/ngx_http_core_module.html#chunked_transfer_encoding)\ [client_body_buffer_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_buffer_size)\ [client_body_in_file_only](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_in_file_only)\ [client_body_in_single_buffer](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_in_single_buffer)\ [client_body_temp_path](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_temp_path)\ [client_body_timeout](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_timeout)\ [client_header_buffer_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_header_buffer_size)\ [client_header_timeout](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_header_timeout)\ [client_id](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_id)\ [client_max_body_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size)\ [client_secret](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_secret)\ [config_url](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#config_url)\ [connection_pool_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#connection_pool_size)\ [cookie_name](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#cookie_name)\ [create_full_put_path](https://nginx.org/en/docs/http/ngx_http_dav_module.html#create_full_put_path)\ [daemon](https://nginx.org/en/docs/ngx_core_module.html#daemon)\ [dav_access](https://nginx.org/en/docs/http/ngx_http_dav_module.html#dav_access)\ [dav_methods](https://nginx.org/en/docs/http/ngx_http_dav_module.html#dav_methods)\ [debug_connection](https://nginx.org/en/docs/ngx_core_module.html#debug_connection)\ [default_type](https://nginx.org/en/docs/http/ngx_http_core_module.html#default_type)\ [deny (ngx_http_access_module)](https://nginx.org/en/docs/http/ngx_http_access_module.html#deny)\ [deny (ngx_stream_access_module)](https://nginx.org/en/docs/stream/ngx_stream_access_module.html#deny)\ [directio](https://nginx.org/en/docs/http/ngx_http_core_module.html#directio)\ [directio_alignment](https://nginx.org/en/docs/http/ngx_http_core_module.html#directio_alignment)\ [disable_symlinks](https://nginx.org/en/docs/http/ngx_http_core_module.html#disable_symlinks)\ [early_hints](https://nginx.org/en/docs/http/ngx_http_core_module.html#early_hints)\ [empty_gif](https://nginx.org/en/docs/http/ngx_http_empty_gif_module.html#empty_gif)\ [enforce_initial_report](https://nginx.org/en/docs/ngx_mgmt_module.html#enforce_initial_report)\ [env](https://nginx.org/en/docs/ngx_core_module.html#env)\ [error_log](https://nginx.org/en/docs/ngx_core_module.html#error_log)\ [error_log_tag](https://nginx.org/en/docs/http/ngx_http_core_module.html#error_log_tag)\ [error_page](https://nginx.org/en/docs/http/ngx_http_core_module.html#error_page)\ [etag](https://nginx.org/en/docs/http/ngx_http_core_module.html#etag)\ [events](https://nginx.org/en/docs/ngx_core_module.html#events)\ [expires](https://nginx.org/en/docs/http/ngx_http_headers_module.html#expires)\ [extra_auth_args](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#extra_auth_args)\ [f4f](https://nginx.org/en/docs/http/ngx_http_f4f_module.html#f4f)\ [f4f_buffer_size](https://nginx.org/en/docs/http/ngx_http_f4f_module.html#f4f_buffer_size)\ [fastcgi_allow_upstream](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_allow_upstream)\ [fastcgi_bind_dynamic](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_bind_dynamic)\ [fastcgi_buffer_size](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_buffer_size)\ [fastcgi_buffering](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_buffering)\ [fastcgi_buffers](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_buffers)\ [fastcgi_busy_buffers_size](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_busy_buffers_size)\ [fastcgi_cache](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache)\ [fastcgi_cache_background_update](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_background_update)\ [fastcgi_cache_bypass](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_bypass)\ [fastcgi_cache_key](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_key)\ [fastcgi_cache_lock](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_lock)\ [fastcgi_cache_lock_age](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_lock_age)\ [fastcgi_cache_lock_timeout](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_lock_timeout)\ [fastcgi_cache_max_range_offset](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_max_range_offset)\ [fastcgi_cache_methods](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_methods)\ [fastcgi_cache_min_uses](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_min_uses)\ [fastcgi_cache_path](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_path)\ [fastcgi_cache_revalidate](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_revalidate)\ [fastcgi_cache_use_stale](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_use_stale)\ [fastcgi_cache_valid](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_valid)\ [fastcgi_catch_stderr](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_catch_stderr)\ [fastcgi_connect_timeout](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_connect_timeout)\ [fastcgi_force_ranges](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_force_ranges)\ [fastcgi_hide_header](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_hide_header)\ [fastcgi_ignore_client_abort](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_ignore_client_abort)\ [fastcgi_ignore_headers](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_ignore_headers)\ [fastcgi_index](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_index)\ [fastcgi_intercept_errors](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_intercept_errors)\ [fastcgi_keep_conn](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_keep_conn)\ [fastcgi_limit_rate](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_limit_rate)\ [fastcgi_max_temp_file_size](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_max_temp_file_size)\ [fastcgi_next_upstream](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_next_upstream)\ [fastcgi_next_upstream_timeout](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_next_upstream_timeout)\ [fastcgi_next_upstream_tries](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_next_upstream_tries)\ [fastcgi_no_cache](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_no_cache)\ [fastcgi_param](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_param)\ [fastcgi_pass](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_pass)\ [fastcgi_pass_header](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_pass_header)\ [fastcgi_pass_request_body](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_pass_request_body)\ [fastcgi_pass_request_headers](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_pass_request_headers)\ [fastcgi_read_timeout](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_read_timeout)\ [fastcgi_request_buffering](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_request_buffering)\ [fastcgi_request_dynamic](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_request_dynamic)\ [fastcgi_send_lowat](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_send_lowat)\ [fastcgi_send_timeout](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_send_timeout)\ [fastcgi_socket_keepalive](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_socket_keepalive)\ [fastcgi_split_path_info](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_split_path_info)\ [fastcgi_store](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_store)\ [fastcgi_store_access](http://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_store_access)\ [fastcgi_temp_file_write_size](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_temp_file_write_size)\ [fastcgi_temp_path](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_temp_path)\ [flv](https://nginx.org/en/docs/http/ngx_http_flv_module.html#flv)\ [frontchannel_logout_uri](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#frontchannel_logout_uri)\ [geo (ngx_http_geo_module)](https://nginx.org/en/docs/http/ngx_http_geo_module.html#geo)\ [geo (ngx_stream_geo_module)](https://nginx.org/en/docs/stream/ngx_stream_geo_module.html#geo)\ [grpc_allow_upstream](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_allow_upstream)\ [grpc_bind_dynamic](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_bind_dynamic)\ [grpc_buffer_size](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_buffer_size)\ [grpc_connect_timeout](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_connect_timeout)\ [grpc_hide_header](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_hide_header)\ [grpc_ignore_headers](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ignore_headers)\ [grpc_intercept_errors](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_intercept_errors)\ [grpc_next_upstream](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_next_upstream)\ [grpc_next_upstream_timeout](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_next_upstream_timeout)\ [grpc_next_upstream_tries](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_next_upstream_tries)\ [grpc_pass](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_pass)\ [grpc_pass_header](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_pass_header)\ [grpc_read_timeout](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_read_timeout)\ [grpc_request_dynamic](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_request_dynamic)\ [grpc_send_timeout](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_send_timeout)\ [grpc_set_header](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_set_header)\ [grpc_socket_keepalive](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_socket_keepalive)\ [grpc_ssl_certificate](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_certificate)\ [grpc_ssl_certificate_cache](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_certificate_cache)\ [grpc_ssl_certificate_key](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_certificate_key)\ [grpc_ssl_ciphers](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_ciphers)\ [grpc_ssl_conf_command](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_conf_command)\ [grpc_ssl_crl](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_crl)\ [grpc_ssl_name](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_name)\ [grpc_ssl_password_file](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_password_file)\ [grpc_ssl_protocols](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_protocols)\ [grpc_ssl_server_name](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_server_name)\ [grpc_ssl_session_reuse](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_session_reuse)\ [grpc_ssl_trusted_certificate](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_trusted_certificate)\ [grpc_ssl_verify](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_verify)\ [grpc_ssl_verify_depth](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_verify_depth)\ [gunzip](https://nginx.org/en/docs/http/ngx_http_gunzip_module.html#gunzip)\ [gunzip_buffers](https://nginx.org/en/docs/http/ngx_http_gunzip_module.html#gunzip_buffers)\ [gzip](https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip)\ [gzip_buffers](https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_buffers)\ [gzip_comp_level](https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_comp_level)\ [gzip_disable](https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_disable)\ [gzip_http_version](https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_http_version)\ [gzip_min_length](https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_min_length)\ [gzip_proxied](https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_proxied)\ [gzip_static](https://nginx.org/en/docs/http/ngx_http_gzip_static_module.html#gzip_static)\ [gzip_types](https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_types)\ [gzip_vary](https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_vary)\ [hash (ngx_http_upstream_module)](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#hash)\ [hash (ngx_stream_upstream_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#hash)\ [health_check (ngx_http_upstream_hc_module)](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#health_check)\ [health_check (ngx_stream_upstream_hc_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#health_check)\ [health_check_timeout](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#health_check_timeout)\ [hls](https://nginx.org/en/docs/http/ngx_http_hls_module.html#hls)\ [hls_buffers](https://nginx.org/en/docs/http/ngx_http_hls_module.html#hls_buffers)\ [hls_forward_args](https://nginx.org/en/docs/http/ngx_http_hls_module.html#hls_forward_args)\ [hls_fragment](https://nginx.org/en/docs/http/ngx_http_hls_module.html#hls_fragment)\ [hls_mp4_buffer_size](https://nginx.org/en/docs/http/ngx_http_hls_module.html#hls_mp4_buffer_size)\ [hls_mp4_max_buffer_size](https://nginx.org/en/docs/http/ngx_http_hls_module.html#hls_mp4_max_buffer_size)\ [http](https://nginx.org/en/docs/http/ngx_http_core_module.html#http)\ [http2_body_preread_size](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_body_preread_size)\ [http2_chunk_size](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_chunk_size)\ [http2_idle_timeout](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_idle_timeout)\ [http2_max_concurrent_pushes](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_max_concurrent_pushes)\ [http2_max_concurrent_streams](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_max_concurrent_streams)\ [http2_max_field_size](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_max_field_size)\ [http2_max_header_size](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_max_header_size)\ [http2_max_requests](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_max_requests)\ [http2_push](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_push)\ [http2_push_preload](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_push_preload)\ [http2_recv_buffer_size](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_recv_buffer_size)\ [http2_recv_timeout](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_recv_timeout)\ [http3](http://nginx.org/en/docs/http/ngx_http_v3_module.html#http3)\ [http3_hq](http://nginx.org/en/docs/http/ngx_http_v3_module.html#http3_hq)\ [http3_max_concurrent_streams](http://nginx.org/en/docs/http/ngx_http_v3_module.html#http3_max_concurrent_streams)\ [http3_stream_buffer_size](http://nginx.org/en/docs/http/ngx_http_v3_module.html#http3_stream_buffer_size)\ [if](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#if)\ [if_modified_since](https://nginx.org/en/docs/http/ngx_http_core_module.html#if_modified_since)\ [ignore_invalid_headers](https://nginx.org/en/docs/http/ngx_http_core_module.html#ignore_invalid_headers)\ [image_filter](http://nginx.org/en/docs/http/ngx_http_image_filter_module.html#image_filter)\ [image_filter_buffer](http://nginx.org/en/docs/http/ngx_http_image_filter_module.html#image_filter_buffer)\ [image_filter_interlace](http://nginx.org/en/docs/http/ngx_http_image_filter_module.html#image_filter_interlace)\ [image_filter_jpeg_quality](http://nginx.org/en/docs/http/ngx_http_image_filter_module.html#image_filter_jpeg_quality)\ [image_filter_sharpen](http://nginx.org/en/docs/http/ngx_http_image_filter_module.html#image_filter_sharpen)\ [image_filter_transparency](http://nginx.org/en/docs/http/ngx_http_image_filter_module.html#image_filter_transparency)\ [image_filter_webp_quality](http://nginx.org/en/docs/http/ngx_http_image_filter_module.html#image_filter_webp_quality)\ [imap_auth](https://nginx.org/en/docs/mail/ngx_mail_imap_module.html#imap_auth)\ [imap_capabilities](https://nginx.org/en/docs/mail/ngx_mail_imap_module.html#imap_capabilities)\ [imap_client_buffer](https://nginx.org/en/docs/mail/ngx_mail_imap_module.html#imap_client_buffer)\ [include](https://nginx.org/en/docs/ngx_core_module.html#include)\ [index](https://nginx.org/en/docs/http/ngx_http_index_module.html#index)\ [internal](https://nginx.org/en/docs/http/ngx_http_core_module.html#internal)\ [internal_redirect](http://nginx.org/en/docs/http/ngx_http_internal_redirect_module.html#internal_redirect)\ [ip_hash](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#ip_hash)\ [issuer](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#issuer)\ [js_access (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_access)\ [js_body_filter](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_body_filter)\ [js_content](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_content)\ [js_fetch_buffer_size (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_fetch_buffer_size)\ [js_fetch_buffer_size (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_fetch_buffer_size)\ [js_fetch_ciphers (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_fetch_ciphers)\ [js_fetch_ciphers (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_fetch_ciphers)\ [js_fetch_max_response_buffer_size (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_fetch_max_response_buffer_size)\ [js_fetch_max_response_buffer_size (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_fetch_max_response_buffer_size)\ [js_fetch_protocols (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_fetch_protocols)\ [js_fetch_protocols (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_fetch_protocols)\ [js_fetch_timeout (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_fetch_timeout)\ [js_fetch_timeout (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_fetch_timeout)\ [js_fetch_trusted_certificate (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_fetch_trusted_certificate)\ [js_fetch_trusted_certificate (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_fetch_trusted_certificate)\ [js_fetch_verify (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_fetch_verify)\ [js_fetch_verify (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_fetch_verify)\ [js_fetch_verify_depth (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_fetch_verify_depth)\ [js_fetch_verify_depth (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_fetch_verify_depth)\ [js_filter (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_filter)\ [js_header_filter](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_header_filter)\ [js_import (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_import)\ [js_import (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_import)\ [js_include (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_include)\ [js_include (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_include)\ [js_path (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_path)\ [js_path (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_path)\ [js_periodic (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_periodic)\ [js_periodic (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_periodic)\ [js_preload_object (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_preload_object)\ [js_preload_object (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_preload_object)\ [js_preread (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_preread)\ [js_set (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_set)\ [js_set (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_set)\ [js_var (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_var)\ [js_var (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_var)\ [js_shared_dict_zone (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_shared_dict_zone)\ [js_var (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_var)\ [js_var (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_var)\ [keepalive](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive)\ [keepalive_disable](https://nginx.org/en/docs/http/ngx_http_core_module.html#keepalive_disable)\ [keepalive_min_timeout](https://nginx.org/en/docs/http/ngx_http_core_module.html#keepalive_min_timeout)\ [keepalive_requests (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#keepalive_requests)\ [keepalive_time (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#keepalive_time)\ [keepalive_timeout (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#keepalive_timeout)\ [keyval (ngx_http_keyval_module)](https://nginx.org/en/docs/http/ngx_http_keyval_module.html#keyval)\ [keyval (ngx_stream_keyval_module)](https://nginx.org/en/docs/stream/ngx_stream_keyval_module.html#keyval)\ [keyval_zone (ngx_http_keyval_module)](https://nginx.org/en/docs/http/ngx_http_keyval_module.html#keyval_zone)\ [keyval_zone (ngx_stream_keyval_module)](https://nginx.org/en/docs/stream/ngx_stream_keyval_module.html#keyval_zone)\ [large_client_header_buffers](https://nginx.org/en/docs/http/ngx_http_core_module.html#large_client_header_buffers)\ [least_conn (ngx_http_upstream_module)](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#least_conn)\ [least_conn (ngx_stream_upstream_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#least_conn)\ [least_time (ngx_http_upstream_module)](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#least_time)\ [least_time (ngx_stream_upstream_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#least_time)\ [license_token](https://nginx.org/en/docs/ngx_mgmt_module.html#license_token)\ [limit_conn (ngx_http_limit_conn_module)](https://nginx.org/en/docs/http/ngx_http_limit_conn_module.html#limit_conn)\ [limit_conn (ngx_stream_limit_conn_module)](https://nginx.org/en/docs/stream/ngx_stream_limit_conn_module.html#limit_conn)\ [limit_conn_dry_run (ngx_http_limit_conn_module)](https://nginx.org/en/docs/http/ngx_http_limit_conn_module.html#limit_conn_dry_run)\ [limit_conn_dry_run (ngx_stream_limit_conn_module)](https://nginx.org/en/docs/stream/ngx_stream_limit_conn_module.html#limit_conn_dry_run)\ [limit_conn_log_level (ngx_http_limit_conn_module)](https://nginx.org/en/docs/http/ngx_http_limit_conn_module.html#limit_conn_log_level)\ [limit_conn_log_level (ngx_stream_limit_conn_module)](https://nginx.org/en/docs/stream/ngx_stream_limit_conn_module.html#limit_conn_log_level)\ [limit_conn_status](https://nginx.org/en/docs/http/ngx_http_limit_conn_module.html#limit_conn_status)\ [limit_conn_zone (ngx_http_limit_conn_module)](https://nginx.org/en/docs/http/ngx_http_limit_conn_module.html#limit_conn_zone)\ [limit_conn_zone (ngx_stream_limit_conn_module)](https://nginx.org/en/docs/stream/ngx_stream_limit_conn_module.html#limit_conn_zone)\ [limit_except](https://nginx.org/en/docs/http/ngx_http_core_module.html#limit_except)\ [limit_rate](https://nginx.org/en/docs/http/ngx_http_core_module.html#limit_rate)\ [limit_rate_after](https://nginx.org/en/docs/http/ngx_http_core_module.html#limit_rate_after)\ [limit_req](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req)\ [limit_req_dry_run](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_dry_run)\ [limit_req_log_level](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_log_level)\ [limit_req_status](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_status)\ [limit_req_zone](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_zone)\ [limit_zone](https://nginx.org/en/docs/http/ngx_http_limit_conn_module.html#limit_zone)\ [lingering_close](https://nginx.org/en/docs/http/ngx_http_core_module.html#lingering_close)\ [lingering_time](https://nginx.org/en/docs/http/ngx_http_core_module.html#lingering_time)\ [lingering_timeout](https://nginx.org/en/docs/http/ngx_http_core_module.html#lingering_timeout)\ [listen (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#listen)\ [listen (ngx_mail_core_module)](https://nginx.org/en/docs/mail/ngx_mail_core_module.html#listen)\ [load_module](https://nginx.org/en/docs/ngx_core_module.html#load_module)\ [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location)\ [lock_file](http://nginx.org/en/docs/ngx_core_module.html#lock_file)\ [log_format (ngx_http_log_module)](https://nginx.org/en/docs/http/ngx_http_log_module.html#log_format)\ [log_format (ngx_stream_log_module)](https://nginx.org/en/docs/stream/ngx_stream_log_module.html#log_format)\ [log_not_found](https://nginx.org/en/docs/http/ngx_http_core_module.html#log_not_found)\ [log_subrequest](https://nginx.org/en/docs/http/ngx_http_core_module.html#log_subrequest)\ [logout_uri](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#logout_uri)\ [logout_token_hint](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#logout_token_hint)\ [mail](https://nginx.org/en/docs/mail/ngx_mail_core_module.html#mail)\ [map (ngx_http_map_module)](https://nginx.org/en/docs/http/ngx_http_map_module.html#map)\ [map (ngx_stream_map_module)](https://nginx.org/en/docs/stream/ngx_stream_map_module.html#map)\ [map_hash_bucket_size (ngx_http_map_module)](https://nginx.org/en/docs/http/ngx_http_map_module.html#map_hash_bucket_size)\ [map_hash_bucket_size (ngx_stream_map_module)](https://nginx.org/en/docs/stream/ngx_stream_map_module.html#map_hash_bucket_size)\ [map_hash_max_size (ngx_http_map_module)](https://nginx.org/en/docs/http/ngx_http_map_module.html#map_hash_max_size)\ [map_hash_max_size (ngx_stream_map_module)](https://nginx.org/en/docs/stream/ngx_stream_map_module.html#map_hash_max_size)\ [master_process](https://nginx.org/en/docs/ngx_core_module.html#master_process)\ [match (ngx_http_upstream_hc_module)](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#match)\ [match (ngx_stream_upstream_hc_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#match)\ [max_errors](https://nginx.org/en/docs/mail/ngx_mail_core_module.html#max_errors)\ [max_headers](https://nginx.org/en/docs/http/ngx_http_core_module.html#max_headers)\ [max_ranges](https://nginx.org/en/docs/http/ngx_http_core_module.html#max_ranges)\ [memcached_allow_upstream](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_allow_upstream)\ [memcached_bind_dynamic](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_bind_dynamic)\ [memcached_buffer_size](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_buffer_size)\ [memcached_connect_timeout](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_connect_timeout)\ [memcached_gzip_flag](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_gzip_flag)\ [memcached_next_upstream](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_next_upstream)\ [memcached_next_upstream_timeout](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_next_upstream_timeout)\ [memcached_next_upstream_tries](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_next_upstream_tries)\ [memcached_pass](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_pass)\ [memcached_read_timeout](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_read_timeout)\ [memcached_send_timeout](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_send_timeout)\ [memcached_socket_keepalive](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_socket_keepalive)\ [merge_slashes](https://nginx.org/en/docs/http/ngx_http_core_module.html#merge_slashes)\ [mgmt](https://nginx.org/en/docs/ngx_mgmt_module.html#mgmt)\ [min_delete_depth](https://nginx.org/en/docs/http/ngx_http_dav_module.html#min_delete_depth)\ [mirror](https://nginx.org/en/docs/http/ngx_http_mirror_module.html#mirror)\ [mirror_request_body](https://nginx.org/en/docs/http/ngx_http_mirror_module.html#mirror_request_body)\ [modern_browser](https://nginx.org/en/docs/http/ngx_http_browser_module.html#modern_browser)\ [modern_browser_value](https://nginx.org/en/docs/http/ngx_http_browser_module.html#modern_browser_value)\ [more_clear_headers](https://github.com/openresty/headers-more-nginx-module?tab=readme-ov-file#more_clear_headers)\ [more_clear_input_headers](https://github.com/openresty/headers-more-nginx-module?tab=readme-ov-file#more_clear_input_headers)\ [more_set_headers](https://github.com/openresty/headers-more-nginx-module?tab=readme-ov-file#more_set_headers)\ [more_set_input_headers](https://github.com/openresty/headers-more-nginx-module?tab=readme-ov-file#more_set_input_headers)\ [mp4](https://nginx.org/en/docs/http/ngx_http_mp4_module.html#mp4)\ [mp4_buffer_size](https://nginx.org/en/docs/http/ngx_http_mp4_module.html#mp4_buffer_size)\ [mp4_limit_rate](https://nginx.org/en/docs/http/ngx_http_mp4_module.html#mp4_limit_rate)\ [mp4_limit_rate_after](https://nginx.org/en/docs/http/ngx_http_mp4_module.html#mp4_limit_rate_after)\ [mp4_max_buffer_size](https://nginx.org/en/docs/http/ngx_http_mp4_module.html#mp4_max_buffer_size)\ [mp4_start_key_frame](https://nginx.org/en/docs/http/ngx_http_mp4_module.html#mp4_start_key_frame)\ [mqtt](https://nginx.org/en/docs/stream/ngx_stream_mqtt_filter_module.html#mqtt)\ [mqtt_buffers](https://nginx.org/en/docs/stream/ngx_stream_mqtt_filter_module.html#mqtt_buffers)\ [mqtt_rewrite_buffer_size](https://nginx.org/en/docs/stream/ngx_stream_mqtt_filter_module.html#mqtt_rewrite_buffer_size)\ [mqtt_set_connect](https://nginx.org/en/docs/stream/ngx_stream_mqtt_filter_module.html#mqtt_set_connect)\ [msie_padding](https://nginx.org/en/docs/http/ngx_http_core_module.html#msie_padding)\ [msie_refresh](https://nginx.org/en/docs/http/ngx_http_core_module.html#msie_refresh)\ [multi_accept](https://nginx.org/en/docs/ngx_core_module.html#multi_accept)\ [ntlm](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#ntlm)\ [num_map](https://nginx.org/en/docs/http/ngx_http_num_map_module.html#num_map)\ [oidc_provider](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#oidc_provider)\ [open_file_cache](https://nginx.org/en/docs/http/ngx_http_core_module.html#open_file_cache)\ [open_file_cache_errors](https://nginx.org/en/docs/http/ngx_http_core_module.html#open_file_cache_errors)\ [open_file_cache_min_uses](https://nginx.org/en/docs/http/ngx_http_core_module.html#open_file_cache_min_uses)\ [open_file_cache_valid](https://nginx.org/en/docs/http/ngx_http_core_module.html#open_file_cache_valid)\ [open_log_file_cache (ngx_http_log_module)](https://nginx.org/en/docs/http/ngx_http_log_module.html#open_log_file_cache)\ [open_log_file_cache (ngx_stream_log_module)](https://nginx.org/en/docs/stream/ngx_stream_log_module.html#open_log_file_cache)\ [otel_exporter](https://nginx.org/en/docs/ngx_otel_module.html#otel_exporter)\ [otel_service_name](https://nginx.org/en/docs/ngx_otel_module.html#otel_service_name)\ [otel_trace](https://nginx.org/en/docs/ngx_otel_module.html#otel_trace)\ [otel_trace_context](https://nginx.org/en/docs/ngx_otel_module.html#otel_trace_context)\ [otel_span_name](https://nginx.org/en/docs/ngx_otel_module.html#otel_span_name)\ [otel_span_attr](https://nginx.org/en/docs/ngx_otel_module.html#otel_span_attr)\ [output_buffers](https://nginx.org/en/docs/http/ngx_http_core_module.html#output_buffers)\ [override_charset](https://nginx.org/en/docs/http/ngx_http_charset_module.html#override_charset)\ [pass](https://nginx.org/en/docs/stream/ngx_stream_pass_module.html#pass)\ [pid](https://nginx.org/en/docs/ngx_core_module.html#pid)\ [pkce](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#pkce)\ [pop3_auth](https://nginx.org/en/docs/mail/ngx_mail_pop3_module.html#pop3_auth)\ [pop3_capabilities](https://nginx.org/en/docs/mail/ngx_mail_pop3_module.html#pop3_capabilities)\ [port_in_redirect](https://nginx.org/en/docs/http/ngx_http_core_module.html#port_in_redirect)\ [post_logout_uri](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#post_logout_uri)\ [postpone_output](https://nginx.org/en/docs/http/ngx_http_core_module.html#postpone_output)\ [preread_buffer_size (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#preread_buffer_size)\ [preread_timeout (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#preread_timeout)\ [protocol](https://nginx.org/en/docs/mail/ngx_mail_core_module.html#protocol)\ [proxy_allow_upstream](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_allow_upstream)\ [proxy_bind_dynamic](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_bind_dynamic)\ [proxy_buffer](https://nginx.org/en/docs/mail/ngx_mail_proxy_module.html#proxy_buffer)\ [proxy_buffer_size (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffer_size)\ [proxy_buffer_size (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_buffer_size)\ [proxy_buffering](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffering)\ [proxy_buffers](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffers)\ [proxy_busy_buffers_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_busy_buffers_size)\ [proxy_cache](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache)\ [proxy_cache_background_update](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_background_update)\ [proxy_cache_bypass](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_bypass)\ [proxy_cache_convert_head](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_convert_head)\ [proxy_cache_key](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_key)\ [proxy_cache_lock](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_lock)\ [proxy_cache_lock_age](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_lock_age)\ [proxy_cache_lock_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_lock_timeout)\ [proxy_cache_max_range_offset](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_max_range_offset)\ [proxy_cache_methods](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_methods)\ [proxy_cache_min_uses](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_min_uses)\ [proxy_cache_path](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_path)\ [proxy_cache_purge](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_purge)\ [proxy_cache_revalidate](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_revalidate)\ [proxy_cache_use_stale](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_use_stale)\ [proxy_cache_valid](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_valid)\ [proxy_connect_timeout (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_connect_timeout)\ [proxy_connect_timeout (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_connect_timeout)\ [proxy_cookie_domain](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cookie_domain)\ [proxy_cookie_flags](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cookie_flags)\ [proxy_cookie_path](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cookie_path)\ [proxy_download_rate (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_download_rate)\ [proxy_force_ranges](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_force_ranges)\ [proxy_half_close (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_half_close)\ [proxy_headers_hash_bucket_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_headers_hash_bucket_size)\ [proxy_headers_hash_max_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_headers_hash_max_size)\ [proxy_hide_header](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_hide_header)\ [proxy_http_version](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_http_version)\ [proxy_ignore_client_abort](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ignore_client_abort)\ [proxy_ignore_headers](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ignore_headers)\ [proxy_intercept_errors](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_intercept_errors)\ [proxy_limit_rate](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_limit_rate)\ [proxy_max_temp_file_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_max_temp_file_size)\ [proxy_method](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_method)\ [proxy_next_upstream (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream)\ [proxy_next_upstream (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_next_upstream)\ [proxy_next_upstream_timeout (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream_timeout)\ [proxy_next_upstream_timeout (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_next_upstream_timeout)\ [proxy_next_upstream_tries (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream_tries)\ [proxy_next_upstream_tries (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_next_upstream_tries)\ [proxy_no_cache](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_no_cache)\ [proxy_pass (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass)\ [proxy_pass (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_pass)\ [proxy_pass_error_message](https://nginx.org/en/docs/mail/ngx_mail_proxy_module.html#proxy_pass_error_message)\ [proxy_pass_header](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass_header)\ [proxy_pass_request_body](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass_request_body)\ [proxy_pass_request_headers](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass_request_headers)\ [proxy_pass_trailers](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass_trailers)\ [proxy_protocol (ngx_mail_proxy_module)](https://nginx.org/en/docs/mail/ngx_mail_proxy_module.html#proxy_protocol)\ [proxy_protocol (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_protocol)\ [proxy_protocol_timeout (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#proxy_protocol_timeout)\ [proxy_read_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_read_timeout)\ [proxy_redirect](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_redirect)\ [proxy_requests (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_requests)\ [proxy_request_buffering](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_request_buffering)\ [proxy_request_dynamic](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_request_dynamic)\ [proxy_responses (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_responses)\ [proxy_send_lowat](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_send_lowat)\ [proxy_send_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_send_timeout)\ [proxy_session_drop (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_session_drop)\ [proxy_set_body](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_body)\ [proxy_set_header](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_header)\ [proxy_smtp_auth](https://nginx.org/en/docs/mail/ngx_mail_proxy_module.html#proxy_smtp_auth)\ [proxy_socket_keepalive (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_socket_keepalive)\ [proxy_socket_keepalive (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_socket_keepalive)\ [proxy_ssl (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl)\ [proxy_ssl_certificate (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_certificate)\ [proxy_ssl_certificate (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_certificate)\ [proxy_ssl_certificate_cache (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_certificate_cache)\ [proxy_ssl_certificate_cache (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_certificate_cache)\ [proxy_ssl_certificate_key (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_certificate_key)\ [proxy_ssl_certificate_key (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_certificate_key)\ [proxy_ssl_ciphers (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_ciphers)\ [proxy_ssl_ciphers (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_ciphers)\ [proxy_ssl_conf_command (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_conf_command)\ [proxy_ssl_conf_command (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_conf_command)\ [proxy_ssl_crl (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_crl)\ [proxy_ssl_crl (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_crl)\ [proxy_ssl_name (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_name)\ [proxy_ssl_name (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_name)\ [proxy_ssl_password_file (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_password_file)\ [proxy_ssl_password_file (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_password_file)\ [proxy_ssl_protocols (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_protocols)\ [proxy_ssl_protocols (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_protocols)\ [proxy_ssl_server_name (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_server_name)\ [proxy_ssl_server_name (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_server_name)\ [proxy_ssl_session_reuse (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_session_reuse)\ [proxy_ssl_session_reuse (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_session_reuse)\ [proxy_ssl_trusted_certificate (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_trusted_certificate)\ [proxy_ssl_trusted_certificate (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_trusted_certificate)\ [proxy_ssl_verify (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_verify)\ [proxy_ssl_verify (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_verify)\ [proxy_ssl_verify_depth (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_verify_depth)\ [proxy_ssl_verify_depth (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_verify_depth)\ [proxy_store](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_store)\ [proxy_store_access](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_store_access)\ [proxy_temp_file_write_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_temp_file_write_size)\ [proxy_temp_path](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_temp_path)\ [proxy_timeout (ngx_mail_proxy_module)](https://nginx.org/en/docs/mail/ngx_mail_proxy_module.html#proxy_timeout)\ [proxy_timeout (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_timeout)\ [proxy_upload_rate (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_upload_rate)\ [queue](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#queue)\ [quic_active_connection_id_limit](http://nginx.org/en/docs/http/ngx_http_v3_module.html#quic_active_connection_id_limit)\ [quic_gso](http://nginx.org/en/docs/http/ngx_http_v3_module.html#quic_gso)\ [quic_host_key](http://nginx.org/en/docs/http/ngx_http_v3_module.html#quic_host_key)\ [quic_retry](http://nginx.org/en/docs/http/ngx_http_v3_module.html#quic_retry)\ [random (ngx_http_upstream_module)](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#random)\ [random (ngx_stream_upstream_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#random)\ [random_index](https://nginx.org/en/docs/http/ngx_http_random_index_module.html#random_index)\ [read_ahead](https://nginx.org/en/docs/http/ngx_http_core_module.html#read_ahead)\ [read_timeout](https://nginx.org/en/docs/ngx_mgmt_module.html#read_timeout)\ [real_ip_header](https://nginx.org/en/docs/http/ngx_http_realip_module.html#real_ip_header)\ [real_ip_recursive](https://nginx.org/en/docs/http/ngx_http_realip_module.html#real_ip_recursive)\ [recursive_error_pages](https://nginx.org/en/docs/http/ngx_http_core_module.html#recursive_error_pages)\ [redirect_uri](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#redirect_uri)\ [referer_hash_bucket_size](https://nginx.org/en/docs/http/ngx_http_referer_module.html#referer_hash_bucket_size)\ [referer_hash_max_size](https://nginx.org/en/docs/http/ngx_http_referer_module.html#referer_hash_max_size)\ [request_pool_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#request_pool_size)\ [reset_timedout_connection](https://nginx.org/en/docs/http/ngx_http_core_module.html#reset_timedout_connection)\ [resolver (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver)\ [resolver (ngx_mail_core_module)](https://nginx.org/en/docs/mail/ngx_mail_core_module.html#resolver)\ [resolver (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#resolver)\ [resolver (ngx_stream_upstream_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#resolver)\ [resolver (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#resolver)\ [resolver_timeout (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver_timeout)\ [resolver_timeout (ngx_mail_core_module)](https://nginx.org/en/docs/mail/ngx_mail_core_module.html#resolver_timeout)\ [resolver_timeout (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#resolver_timeout)\ [resolver_timeout (ngx_stream_upstream_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#resolver_timeout)\ [resolver_timeout (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#resolver_timeout)\ [return (ngx_http_rewrite_module)](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#return)\ [return (ngx_stream_return_module)](https://nginx.org/en/docs/stream/ngx_stream_return_module.html#return)\ [rewrite](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#rewrite)\ [rewrite_log](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#rewrite_log)\ [root](https://nginx.org/en/docs/http/ngx_http_core_module.html#root)\ [satisfy](https://nginx.org/en/docs/http/ngx_http_core_module.html#satisfy)\ [scgi_allow_upstream](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_allow_upstream)\ [scgi_bind_dynamic](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_bind_dynamic)\ [scgi_buffer_size](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_buffer_size)\ [scgi_buffering](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_buffering)\ [scgi_buffers](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_buffers)\ [scgi_busy_buffers_size](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_busy_buffers_size)\ [scgi_cache](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache)\ [scgi_cache_background_update](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_background_update)\ [scgi_cache_bypass](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_bypass)\ [scgi_cache_key](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_key)\ [scgi_cache_lock](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_lock)\ [scgi_cache_lock_age](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_lock_age)\ [scgi_cache_lock_timeout](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_lock_timeout)\ [scgi_cache_max_range_offset](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_max_range_offset)\ [scgi_cache_methods](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_methods)\ [scgi_cache_min_uses](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_min_uses)\ [scgi_cache_path](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_path)\ [scgi_cache_purge](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_purge)\ [scgi_cache_revalidate](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_revalidate)\ [scgi_cache_use_stale](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_use_stale)\ [scgi_cache_valid](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_valid)\ [scgi_connect_timeout](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_connect_timeout)\ [scgi_force_ranges](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_force_ranges)\ [scgi_hide_header](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_hide_header)\ [scgi_ignore_client_abort](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_ignore_client_abort)\ [scgi_ignore_headers](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_ignore_headers)\ [scgi_intercept_errors](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_intercept_errors)\ [scgi_limit_rate](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_limit_rate)\ [scgi_max_temp_file_size](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_max_temp_file_size)\ [scgi_next_upstream](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_next_upstream)\ [scgi_next_upstream_timeout](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_next_upstream_timeout)\ [scgi_next_upstream_tries](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_next_upstream_tries)\ [scgi_no_cache](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_no_cache)\ [scgi_param](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_param)\ [scgi_pass](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_pass)\ [scgi_pass_header](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_pass_header)\ [scgi_pass_request_body](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_pass_request_body)\ [scgi_pass_request_headers](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_pass_request_headers)\ [scgi_read_timeout](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_read_timeout)\ [scgi_request_buffering](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_request_buffering)\ [scgi_request_dynamic](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_request_dynamic)\ [scgi_send_timeout](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_send_timeout)\ [scgi_socket_keepalive](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_socket_keepalive)\ [scgi_store](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_store)\ [scgi_store_access](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_store_access)\ [scgi_temp_file_write_size](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_temp_file_write_size)\ [scope](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#scope)\ [scgi_temp_path](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_temp_path)\ [secure_link](https://nginx.org/en/docs/http/ngx_http_secure_link_module.html#secure_link)\ [secure_link_md5](https://nginx.org/en/docs/http/ngx_http_secure_link_module.html#secure_link_md5)\ [secure_link_secret](https://nginx.org/en/docs/http/ngx_http_secure_link_module.html#secure_link_secret)\ [send_lowat](https://nginx.org/en/docs/http/ngx_http_core_module.html#send_lowat)\ [send_timeout (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#send_timeout)\ [send_timeout (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#send_timeout)\ [sendfile](https://nginx.org/en/docs/http/ngx_http_core_module.html#sendfile)\ [sendfile_max_chunk](https://nginx.org/en/docs/http/ngx_http_core_module.html#sendfile_max_chunk)\ [server (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#server)\ [server (ngx_mail_core_module)](https://nginx.org/en/docs/mail/ngx_mail_core_module.html#server)\ [server (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#server)\ [server (ngx_stream_upstream_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#server)\ [server_name (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#server_name)\ [server_name (ngx_mail_core_module)](https://nginx.org/en/docs/mail/ngx_mail_core_module.html#server_name)\ [server_name (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#server_name)\ [server_name_in_redirect](https://nginx.org/en/docs/http/ngx_http_core_module.html#server_name_in_redirect)\ [server_tokens](https://nginx.org/en/docs/http/ngx_http_core_module.html#server_tokens)\ [session_log](https://nginx.org/en/docs/http/ngx_http_session_log_module.html#session_log)\ [session_log_format](https://nginx.org/en/docs/http/ngx_http_session_log_module.html#session_log_format)\ [session_log_zone](https://nginx.org/en/docs/http/ngx_http_session_log_module.html#session_log_zone)\ [session_store](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#session_store)\ [session_timeout](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#session_timeout)\ [set (ngx_http_rewrite_module)](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#set)\ [set (ngx_stream_set_module)](https://nginx.org/en/docs/stream/ngx_stream_set_module.html#set)\ [set_real_ip_from (ngx_http_realip_module)](https://nginx.org/en/docs/http/ngx_http_realip_module.html#set_real_ip_from)\ [set_real_ip_from (ngx_mail_realip_module)](https://nginx.org/en/docs/mail/ngx_mail_realip_module.html#set_real_ip_from)\ [set_real_ip_from (ngx_stream_realip_module)](https://nginx.org/en/docs/stream/ngx_stream_realip_module.html#set_real_ip_from)\ [slice](https://nginx.org/en/docs/http/ngx_http_slice_module.html#slice)\ [smtp_auth](https://nginx.org/en/docs/mail/ngx_mail_smtp_module.html#smtp_auth)\ [smtp_capabilities](https://nginx.org/en/docs/mail/ngx_mail_smtp_module.html#smtp_capabilities)\ [smtp_client_buffer](https://nginx.org/en/docs/mail/ngx_mail_smtp_module.html#smtp_client_buffer)\ [smtp_greeting_delay](https://nginx.org/en/docs/mail/ngx_mail_smtp_module.html#smtp_greeting_delay)\ [source_charset](https://nginx.org/en/docs/http/ngx_http_charset_module.html#source_charset) [spdy_chunk_size](https://nginx.org/en/docs/http/ngx_http_spdy_module.html#spdy_chunk_size)\ [spdy_headers_comp](https://nginx.org/en/docs/http/ngx_http_spdy_module.html#spdy_headers_comp)\ [split_clients (ngx_http_split_clients_module)](https://nginx.org/en/docs/http/ngx_http_split_clients_module.html#split_clients)\ [split_clients (ngx_stream_split_clients_module)](https://nginx.org/en/docs/stream/ngx_stream_split_clients_module.html#split_clients)\ [ssi](https://nginx.org/en/docs/http/ngx_http_ssi_module.html#ssi)\ [ssi_last_modified](https://nginx.org/en/docs/http/ngx_http_ssi_module.html#ssi_last_modified)\ [ssi_min_file_chunk](https://nginx.org/en/docs/http/ngx_http_ssi_module.html#ssi_min_file_chunk)\ [ssi_silent_errors](https://nginx.org/en/docs/http/ngx_http_ssi_module.html#ssi_silent_errors)\ [ssi_types](https://nginx.org/en/docs/http/ngx_http_ssi_module.html#ssi_types)\ [ssi_value_length](https://nginx.org/en/docs/http/ngx_http_ssi_module.html#ssi_value_length)\ [ssl (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl)\ [ssl (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl)\ [ssl (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl)\ [ssl_buffer_size](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_buffer_size)\ [ssl_certificate (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_certificate)\ [ssl_certificate (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_certificate)\ [ssl_certificate (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_certificate)\ [ssl_certificate (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_certificate)\ [ssl_certificate_cache (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_certificate_cache)\ [ssl_certificate_cache (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_certificate_cache)\ [ssl_certificate_compression (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_certificate_compression)\ [ssl_certificate_compression (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_certificate_compression)\ [ssl_certificate_compression (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_certificate_compression)\ [ssl_certificate_key (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_certificate_key)\ [ssl_certificate_key (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_certificate_key)\ [ssl_certificate_key (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_certificate_key)\ [ssl_certificate_key (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_certificate_key)\ [ssl_ciphers (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_ciphers)\ [ssl_ciphers (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_ciphers)\ [ssl_ciphers (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_ciphers)\ [ssl_client_certificate (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_client_certificate)\ [ssl_client_certificate (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_client_certificate)\ [ssl_client_certificate (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_client_certificate)\ [ssl_conf_command (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_conf_command)\ [ssl_conf_command (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_conf_command)\ [ssl_conf_command (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_conf_command)\ [ssl_crl (ngx_http_oidc_module)](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#ssl_crl)\ [ssl_crl (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_crl)\ [ssl_crl (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_crl)\ [ssl_crl (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_crl)\ [ssl_dhparam (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_dhparam)\ [ssl_dhparam (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_dhparam)\ [ssl_early_data](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_early_data)\ [ssl_ecdh_curve (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_ecdh_curve)\ [ssl_ecdh_curve (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_ecdh_curve)\ [ssl_ecdh_curve (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_ecdh_curve)\ [ssl_ech_file](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_ech_file)\ [ssl_handshake_timeout](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_handshake_timeout)\ [ssl_name](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_name)\ [ssl_object_cache_inheritable](https://nginx.org/en/docs/ngx_core_module.html#ssl_object_cache_inheritable)\ [ssl_ocsp](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_ocsp)\ [ssl_ocsp_cache](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_ocsp_cache)\ [ssl_ocsp_responder](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_ocsp_responder)\ [ssl_password_file (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_password_file)\ [ssl_password_file (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_password_file)\ [ssl_password_file (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_password_file)\ [ssl_prefer_server_ciphers (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_prefer_server_ciphers)\ [ssl_prefer_server_ciphers (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_prefer_server_ciphers)\ [ssl_prefer_server_ciphers (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_prefer_server_ciphers)\ [ssl_preread (ngx_stream_ssl_preread_module)](http://nginx.org/en/docs/stream/ngx_stream_ssl_preread_module.html#var_ssl_preread_protocol)\ [ssl_protocols (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_protocols)\ [ssl_protocols (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_protocols)\ [ssl_protocols (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_protocols)\ [ssl_protocols (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_protocols)\ [ssl_reject_handshake](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_reject_handshake)\ [ssl_server_name](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_server_name)\ [ssl_session_cache (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_session_cache)\ [ssl_session_cache (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_session_cache)\ [ssl_session_cache (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_session_cache)\ [ssl_session_ticket_key (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_session_ticket_key)\ [ssl_session_ticket_key (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_session_ticket_key)\ [ssl_session_ticket_key (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_session_ticket_key)\ [ssl_session_tickets (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_session_tickets)\ [ssl_session_tickets (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_session_tickets)\ [ssl_session_tickets (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_session_tickets)\ [ssl_session_timeout (ngx_http_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_session_timeout)\ [ssl_session_timeout (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_session_timeout)\ [ssl_session_timeout (ngx_stream_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_session_timeout)\ [ssl_stapling](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_stapling)\ [ssl_stapling_file](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_stapling_file)\ [ssl_stapling_responder](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_stapling_responder)\ [ssl_stapling_verify](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_stapling_verify)\ [ssl_trusted_certificate (ngx_http_oidc_module)](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#ssl_trusted_certificate)\ [ssl_trusted_certificate (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_trusted_certificate)\ [ssl_trusted_certificate (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_trusted_certificate)\ [ssl_trusted_certificate (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_trusted_certificate)\ [ssl_trusted_certificate (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_trusted_certificate)\ [ssl_verify](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_verify)\ [ssl_verify_client (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_verify_client)\ [ssl_verify_client (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_verify_client)\ [ssl_verify_client (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_verify_client)\ [ssl_verify_depth (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_verify_depth)\ [ssl_verify_depth (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_verify_depth)\ [ssl_verify_depth (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_verify_depth)\ [stall_threshold](https://nginx.org/en/docs/ngx_core_module.html#stall_threshold)\ [ssl_verify_depth (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_verify_depth)\ [starttls](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#starttls)\ [state (ngx_http_upstream_module)](http://nginx.org/en/docs/http/ngx_http_upstream_module.html#state)\ [status_zone (ngx_http_api_module)](https://nginx.org/en/docs/http/ngx_http_api_module.html#status_zone)\ [sticky](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#sticky)\ [sticky_cookie_insert](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#sticky_cookie_insert)\ [stream (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#stream)\ [stub_status](https://nginx.org/en/docs/http/ngx_http_stub_status_module.html#stub_status)\ [sub_filter](https://nginx.org/en/docs/http/ngx_http_sub_module.html#sub_filter)\ [sub_filter_last_modified](https://nginx.org/en/docs/http/ngx_http_sub_module.html#sub_filter_last_modified)\ [sub_filter_once](https://nginx.org/en/docs/http/ngx_http_sub_module.html#sub_filter_once)\ [sub_filter_types](https://nginx.org/en/docs/http/ngx_http_sub_module.html#sub_filter_types)\ [subrequest_output_buffer_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#subrequest_output_buffer_size)\ [tcp_nodelay (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#tcp_nodelay)\ [tcp_nodelay (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#tcp_nodelay)\ [tcp_nopush](https://nginx.org/en/docs/http/ngx_http_core_module.html#tcp_nopush)\ [thread_pool](https://nginx.org/en/docs/ngx_core_module.html#thread_pool)\ [timeout](https://nginx.org/en/docs/mail/ngx_mail_core_module.html#timeout)\ [timer_resolution](https://nginx.org/en/docs/ngx_core_module.html#timer_resolution)\ [try_files](https://nginx.org/en/docs/http/ngx_http_core_module.html#try_files)\ [tunnel_allow_upstream](https://nginx.org/en/docs/http/ngx_http_tunnel_module.html#tunnel_allow_upstream)\ [tunnel_bind](https://nginx.org/en/docs/http/ngx_http_tunnel_module.html#tunnel_bind)\ [tunnel_bind_dynamic](https://nginx.org/en/docs/http/ngx_http_tunnel_module.html#tunnel_bind_dynamic)\ [tunnel_buffer_size](https://nginx.org/en/docs/http/ngx_http_tunnel_module.html#tunnel_buffer_size)\ [tunnel_connect_timeout](https://nginx.org/en/docs/http/ngx_http_tunnel_module.html#tunnel_connect_timeout)\ [tunnel_next_upstream](https://nginx.org/en/docs/http/ngx_http_tunnel_module.html#tunnel_next_upstream)\ [tunnel_next_upstream_timeout](https://nginx.org/en/docs/http/ngx_http_tunnel_module.html#tunnel_next_upstream_timeout)\ [tunnel_next_upstream_tries](https://nginx.org/en/docs/http/ngx_http_tunnel_module.html#tunnel_next_upstream_tries)\ [tunnel_pass](https://nginx.org/en/docs/http/ngx_http_tunnel_module.html#tunnel_pass)\ [tunnel_read_timeout](https://nginx.org/en/docs/http/ngx_http_tunnel_module.html#tunnel_read_timeout)\ [tunnel_send_lowat](https://nginx.org/en/docs/http/ngx_http_tunnel_module.html#tunnel_send_lowat)\ [tunnel_send_timeout](https://nginx.org/en/docs/http/ngx_http_tunnel_module.html#tunnel_send_timeout)\ [tunnel_socket_keepalive](https://nginx.org/en/docs/http/ngx_http_tunnel_module.html#tunnel_socket_keepalive)\ [types](https://nginx.org/en/docs/http/ngx_http_core_module.html#types)\ [types_hash_bucket_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#types_hash_bucket_size)\ [types_hash_max_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#types_hash_max_size)\ [underscores_in_headers](https://nginx.org/en/docs/http/ngx_http_core_module.html#underscores_in_headers)\ [uninitialized_variable_warn](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#uninitialized_variable_warn)\ [upstream (ngx_http_upstream_module)](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#upstream)\ [upstream (ngx_stream_upstream_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#upstream)\ [upstream_conf](https://nginx.org/en/docs/http/ngx_http_upstream_conf_module.html#upstream_conf)\ [usage_report](https://nginx.org/en/docs/ngx_mgmt_module.html#usage_report)\ [use](https://nginx.org/en/docs/ngx_core_module.html#use)\ [user](https://nginx.org/en/docs/ngx_core_module.html#user)\ [userid](https://nginx.org/en/docs/http/ngx_http_userid_module.html#userid)\ [userid_domain](https://nginx.org/en/docs/http/ngx_http_userid_module.html#userid_domain)\ [userid_expires](https://nginx.org/en/docs/http/ngx_http_userid_module.html#userid_expires)\ [userid_flags](https://nginx.org/en/docs/http/ngx_http_userid_module.html#userid_flags)\ [userid_mark](https://nginx.org/en/docs/http/ngx_http_userid_module.html#userid_mark)\ [userid_name](https://nginx.org/en/docs/http/ngx_http_userid_module.html#userid_name)\ [userid_p3p](https://nginx.org/en/docs/http/ngx_http_userid_module.html#userid_p3p)\ [userid_path](https://nginx.org/en/docs/http/ngx_http_userid_module.html#userid_path)\ [userid_service](https://nginx.org/en/docs/http/ngx_http_userid_module.html#userid_service)\ [userinfo](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#userinfo)\ [uuid_file](https://nginx.org/en/docs/ngx_mgmt_module.html#uuid_file)\ [uwsgi_allow_upstream](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_allow_upstream)\ [uwsgi_bind_dynamic](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_bind_dynamic)\ [uwsgi_buffer_size](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_buffer_size)\ [uwsgi_buffering](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_buffering)\ [uwsgi_buffers](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_buffers)\ [uwsgi_busy_buffers_size](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_busy_buffers_size)\ [uwsgi_cache](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache)\ [uwsgi_cache_background_update](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_background_update)\ [uwsgi_cache_bypass](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_bypass)\ [uwsgi_cache_key](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_key)\ [uwsgi_cache_lock](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_lock)\ [uwsgi_cache_lock_age](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_lock_age)\ [uwsgi_cache_lock_timeout](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_lock_timeout)\ [uwsgi_cache_max_range_offset](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_max_range_offset)\ [uwsgi_cache_methods](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_methods)\ [uwsgi_cache_min_uses](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_min_uses)\ [uwsgi_cache_path](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_path)\ [uwsgi_cache_purge](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_purge)\ [uwsgi_cache_revalidate](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_revalidate)\ [uwsgi_cache_use_stale](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_use_stale)\ [uwsgi_cache_valid](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_valid)\ [uwsgi_connect_timeout](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_connect_timeout)\ [uwsgi_force_ranges](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_force_ranges)\ [uwsgi_hide_header](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_hide_header)\ [uwsgi_ignore_client_abort](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ignore_client_abort)\ [uwsgi_ignore_headers](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ignore_headers)\ [uwsgi_intercept_errors](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_intercept_errors)\ [uwsgi_limit_rate](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_limit_rate)\ [uwsgi_max_temp_file_size](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_max_temp_file_size)\ [uwsgi_modifier1](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_modifier1)\ [uwsgi_modifier2](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_modifier2)\ [uwsgi_next_upstream](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_next_upstream)\ [uwsgi_next_upstream_timeout](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_next_upstream_timeout)\ [uwsgi_next_upstream_tries](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_next_upstream_tries)\ [uwsgi_no_cache](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_no_cache)\ [uwsgi_param](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_param)\ [uwsgi_pass](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_pass)\ [uwsgi_pass_header](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_pass_header)\ [uwsgi_pass_request_body](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_pass_request_body)\ [uwsgi_pass_request_headers](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_pass_request_headers)\ [uwsgi_read_timeout](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_read_timeout)\ [uwsgi_request_buffering](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_request_buffering)\ [uwsgi_request_dynamic](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_request_dynamic)\ [uwsgi_send_timeout](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_send_timeout)\ [uwsgi_socket_keepalive](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_socket_keepalive)\ [uwsgi_ssl_certificate](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_certificate)\ [uwsgi_ssl_certificate_cache](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_certificate_cache)\ [uwsgi_ssl_certificate_key](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_certificate_key)\ [uwsgi_ssl_conf_command](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_conf_command)\ [uwsgi_ssl_crl](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_crl)\ [uwsgi_ssl_name](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_name)\ [uwsgi_ssl_password_file](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_password_file)\ [uwsgi_ssl_protocols](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_protocols)\ [uwsgi_ssl_server_name](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_server_name)\ [uwsgi_ssl_session_reuse](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_session_reuse)\ [uwsgi_ssl_trusted_certificate](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_trusted_certificate)\ [uwsgi_ssl_verify](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_verify)\ [uwsgi_ssl_verify_depth](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_verify_depth)\ [uwsgi_store](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_store)\ [uwsgi_store_access](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_store_access)\ [uwsgi_temp_file_write_size](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_temp_file_write_size)\ [uwsgi_temp_path](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_temp_path)\ [valid_referers](https://nginx.org/en/docs/http/ngx_http_referer_module.html#valid_referers)\ [variables_hash_bucket_size (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#variables_hash_bucket_size)\ [variables_hash_bucket_size (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#variables_hash_bucket_size)\ [variables_hash_max_size (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#variables_hash_max_size)\ [variables_hash_max_size (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#variables_hash_max_size)\ [worker_aio_requests](https://nginx.org/en/docs/ngx_core_module.html#worker_aio_requests)\ [worker_connections](https://nginx.org/en/docs/ngx_core_module.html#worker_connections)\ [worker_cpu_affinity](https://nginx.org/en/docs/ngx_core_module.html#worker_cpu_affinity)\ [worker_priority](https://nginx.org/en/docs/ngx_core_module.html#worker_priority)\ [worker_processes](https://nginx.org/en/docs/ngx_core_module.html#worker_processes)\ [worker_rlimit_core](https://nginx.org/en/docs/ngx_core_module.html#worker_rlimit_core)\ [worker_rlimit_nofile](https://nginx.org/en/docs/ngx_core_module.html#worker_rlimit_nofile)\ [worker_shutdown_timeout](https://nginx.org/en/docs/ngx_core_module.html#worker_shutdown_timeout)\ [working_directory](https://nginx.org/en/docs/ngx_core_module.html#working_directory)\ [xclient](https://nginx.org/en/docs/mail/ngx_mail_proxy_module.html#xclient)\ [xml_entities](https://nginx.org/en/docs/http/ngx_http_xslt_module.html#xml_entities)\ [xslt_last_modified](https://nginx.org/en/docs/http/ngx_http_xslt_module.html#xslt_last_modified)\ [xslt_param](https://nginx.org/en/docs/http/ngx_http_xslt_module.html#xslt_param)\ [xslt_string_param](http://nginx.org/en/docs/http/ngx_http_xslt_module.html#xslt_string_param)\ [xslt_stylesheet](https://nginx.org/en/docs/http/ngx_http_xslt_module.html#xslt_stylesheet)\ [xslt_types](https://nginx.org/en/docs/http/ngx_http_xslt_module.html#xslt_types)\ [zone (ngx_http_upstream_module)](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#zone)\ [zone (ngx_stream_upstream_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#zone)\ [zone_sync](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync)\ [zone_sync_buffers](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_buffers)\ [zone_sync_connect_retry_interval](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_connect_retry_interval)\ [zone_sync_connect_timeout](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_connect_timeout)\ [zone_sync_interval](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_interval)\ [zone_sync_recv_buffer_size](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_recv_buffer_size)\ [zone_sync_server](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_server)\ [zone_sync_ssl](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_ssl)\ [zone_sync_ssl_certificate](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_ssl_certificate)\ [zone_sync_ssl_certificate_key](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_ssl_certificate_key)\ [zone_sync_ssl_ciphers](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_ssl_ciphers)\ [zone_sync_ssl_conf_command](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_ssl_conf_command)\ [zone_sync_ssl_crl](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_ssl_crl)\ [zone_sync_ssl_name](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_ssl_name)\ [zone_sync_ssl_password_file](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_ssl_password_file)\ [zone_sync_ssl_protocols](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_ssl_protocols)\ [zone_sync_ssl_server_name](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_ssl_server_name)\ [zone_sync_ssl_trusted_certificate](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_ssl_trusted_certificate)\ [zone_sync_ssl_verify](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_ssl_verify)\ [zone_sync_ssl_verify_depth](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_ssl_verify_depth)\ [zone_sync_timeout](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_timeout) #### Lua dynamic module directives [lua_load_resty_core](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#lua_load_resty_core)\ [lua_use_default_type](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#lua_use_default_type)\ [lua_malloc_trim](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#lua_malloc_trim)\ [lua_code_cache](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#lua_code_cache)\ [lua_thread_cache_max_entries](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#lua_thread_cache_max_entries)\ [lua_regex_cache_max_entries](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#lua_regex_cache_max_entries)\ [lua_regex_match_limit](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#lua_regex_match_limit)\ [lua_package_path](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#lua_package_path)\ [lua_package_cpath](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#lua_package_cpath)\ [init_by_lua](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#init_by_lua)\ [init_by_lua_block](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#init_by_lua_block)\ [init_by_lua_file](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#init_by_lua_file)\ [init_worker_by_lua](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#init_worker_by_lua)\ [init_worker_by_lua_block](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#init_worker_by_lua_block)\ [init_worker_by_lua_file](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#init_worker_by_lua_file)\ [exit_worker_by_lua_block](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#exit_worker_by_lua_block)\ [exit_worker_by_lua_file](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#exit_worker_by_lua_file)\ [set_by_lua](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#set_by_lua)\ [set_by_lua_block](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#set_by_lua_block)\ [set_by_lua_file](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#set_by_lua_file)\ [content_by_lua](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#content_by_lua)\ [content_by_lua_block](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#content_by_lua_block)\ [content_by_lua_file](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#content_by_lua_file)\ [server_rewrite_by_lua_block](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#server_rewrite_by_lua_block)\ [server_rewrite_by_lua_file](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#server_rewrite_by_lua_file)\ [rewrite_by_lua](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#rewrite_by_lua)\ [rewrite_by_lua_block](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#rewrite_by_lua_block)\ [rewrite_by_lua_file](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#rewrite_by_lua_file)\ [access_by_lua](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#access_by_lua)\ [access_by_lua_block](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#access_by_lua_block)\ [access_by_lua_file](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#access_by_lua_file)\ [header_filter_by_lua](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#header_filter_by_lua)\ [header_filter_by_lua_block](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#header_filter_by_lua_block)\ [header_filter_by_lua_file](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#header_filter_by_lua_file)\ [body_filter_by_lua](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#body_filter_by_lua)\ [body_filter_by_lua_block](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#body_filter_by_lua_block)\ [body_filter_by_lua_file](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#body_filter_by_lua_file)\ [log_by_lua](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#log_by_lua)\ [log_by_lua_block](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#log_by_lua_block)\ [log_by_lua_file](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#log_by_lua_file)\ [balancer_by_lua_block](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#balancer_by_lua_block)\ [balancer_by_lua_file](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#balancer_by_lua_file)\ [lua_need_request_body](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#lua_need_request_body)\ [ssl_client_hello_by_lua_block](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#ssl_client_hello_by_lua_block)\ [ssl_client_hello_by_lua_file](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#ssl_client_hello_by_lua_file)\ [ssl_certificate_by_lua_block](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#ssl_certificate_by_lua_block)\ [ssl_certificate_by_lua_file](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#ssl_certificate_by_lua_file)\ [ssl_session_fetch_by_lua_block](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#ssl_session_fetch_by_lua_block)\ [ssl_session_fetch_by_lua_file](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#ssl_session_fetch_by_lua_file)\ [ssl_session_store_by_lua_block](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#ssl_session_store_by_lua_block)\ [ssl_session_store_by_lua_file](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#ssl_session_store_by_lua_file)\ [lua_shared_dict](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#lua_shared_dict)\ [lua_socket_connect_timeout](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#lua_socket_connect_timeout)\ [lua_socket_send_timeout](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#lua_socket_send_timeout)\ [lua_socket_send_lowat](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#lua_socket_send_lowat)\ [lua_socket_read_timeout](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#lua_socket_read_timeout)\ [lua_socket_buffer_size](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#lua_socket_buffer_size)\ [lua_socket_pool_size](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#lua_socket_pool_size)\ [lua_socket_keepalive_timeout](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#lua_socket_keepalive_timeout)\ [lua_socket_log_errors](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#lua_socket_log_errors)\ [lua_ssl_ciphers](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#lua_ssl_ciphers)\ [lua_ssl_crl](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#lua_ssl_crl)\ [lua_ssl_protocols](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#lua_ssl_protocols)\ [lua_ssl_trusted_certificate](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#lua_ssl_trusted_certificate)\ [lua_ssl_verify_depth](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#lua_ssl_verify_depth)\ [lua_ssl_conf_command](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#lua_ssl_conf_command)\ [lua_http10_buffering](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#lua_http10_buffering)\ [rewrite_by_lua_no_postpone](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#rewrite_by_lua_no_postpone)\ [access_by_lua_no_postpone](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#access_by_lua_no_postpone)\ [lua_transform_underscores_in_response_headers](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#lua_transform_underscores_in_response_headers)\ [lua_check_client_abort](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#lua_check_client_abort)\ [lua_max_pending_timers](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#lua_max_pending_timers)\ [lua_max_running_timers](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#lua_max_running_timers)\ [lua_sa_restart](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#lua_sa_restart)\ [lua_worker_thread_vm_pool_size](https://github.com/openresty/lua-nginx-module?tab=readme-ov-file#lua_worker_thread_vm_pool_size) #### GeoIP2 dynamic module directives [geoip2 (ngx_http_geo2_module)](https://github.com/leev/ngx_http_geoip2_module#user-content-download-maxmind-geolite2-database-optional)\ [geoip2 (ngx_stream_geo2_module)](https://github.com/leev/ngx_http_geoip2_module#user-content-download-maxmind-geolite2-database-optional)\ [geoip2_proxy (ngx_http_geo2_module)](https://github.com/leev/ngx_http_geoip2_module#user-content-download-maxmind-geolite2-database-optional)\ [geoip2_proxy_recursive (ngx_http_geo2_module)](https://github.com/leev/ngx_http_geoip2_module#user-content-download-maxmind-geolite2-database-optional)\ -------------------------------------------------------------------------------- title: "Overview" url: https://docs.nginx.com/nginxaas-azure/getting-started/ssl-tls-certificates/overview/index.md -------------------------------------------------------------------------------- F5 NGINXaaS for Azure (NGINXaaS) enables customers to secure traffic by adding SSL/TLS certificates to a deployment. NGINXaaS can fetch certificates directly from Azure Key Vault, rotate certificates, and provide observability on the status of your certificates. This document provides details about using SSL/TLS certificates with your F5 NGINXaaS for Azure deployment. ## Supported certificate types and formats NGINXaaS supports **up to 100** TLS/SSL certificates of the following types: - Self-signed certificates, Domain Validated (DV) certificates, Organization Validated (OV) certificates, and Extended Validation (EV) certificates. NGINX supports the following certificate formats: - PEM and PKCS12 format certificates. ## Add SSL/TLS certificates Add a certificate from an Azure Key Vault to your NGINXaaS deployment using your preferred client tool: * [Add certificates using the Azure portal](/nginxaas-azure/getting-started/ssl-tls-certificates/ssl-tls-certificates-portal.md) * [Add certificates using the Azure CLI](/nginxaas-azure/getting-started/ssl-tls-certificates/ssl-tls-certificates-azure-cli.md) * [Add certificates using Terraform](/nginxaas-azure/getting-started/ssl-tls-certificates/ssl-tls-certificates-terraform.md) **Note:** - You can overwrite the NGINX default protocol to configure the desired TLS/SSL policy. Read more about the procedure in the [Module ngx_http_ssl_module](http://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_protocols) documentation. - You can associate multiple certificates for the same domain; the "ssl_certificate" directive can be specified multiple times to load certificates of different types. To learn more, see the [Module ngx_http_ssl_module](http://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_certificate) documentation. ### Add SSL/TLS certificates bundled with NGINXaaS configuration You can also add your certificate as a file to your NGINX configuration filesystem; refer to [Upload an NGINX configuration](/nginxaas-azure/getting-started/nginx-configuration/overview.md) to learn about the different options. Although this is a quick method for adding SSL/TLS certificates to your NGINXaaS deployment, we recommend adding certificates through Azure Key Vault (AKV) for enhanced security, certificate rotation, and monitoring. Once a certificate has been added, update your NGINX configuration to reference your SSL/TLS certificate and key file paths. ```nginx http { server { listen 443 ssl; ssl_certificate /etc/nginx/certs/mycert.cert; ssl_certificate_key /etc/nginx/certs/mycert.key; # ... } } ``` ## Certificate rotation NGINXaaS for Azure regularly polls the AKV to check if the certificate has been updated. If an updated certificate is found, it is automatically rotated on the deployment within 4 hours. Any change to the NGINX configuration will trigger all SSL/TLS certificates to be rotated immediately. When referencing a certificate in Azure Key Vault using a secret identifier, if the secret identifier specifies a version, NGINXaaS will not rotate the certificate. To enable certificate rotation, ensure the secret identifier does not contain a version, for example, `https://myvault.vault.azure.net/secrets/mysecret`. **Note:** If any of your SSL/TLS certificates or your NGINX configuration has issues, the certificates will not be rotated. ## Monitor certificates To view the status of your SSL/TLS certificates, [enable monitoring](/nginxaas-azure/monitoring/enable-metrics.md) for your NGINXaaS deployment and navigate to the **Metrics** tab in the Azure portal. View the `nginxaas.certificates` metric under the `nginxaas statistics` metric namespace. The `nginxaas.certificates` metric allows you to filter by certificate key vault URL and the status of the certificate. The status dimension reports the health of your certificates through the following values: | Status | Description | | ------------- | ------------- | | `active` | The certificate was successfully fetched from AKV. | | `unauthorized`| Azure returned a 401/403 error when fetching the certificate from AKV, which usually indicates an issue with the deployment's [Managed Identity](/nginxaas-azure/getting-started/managed-identity-portal.md). | | `not found` | Azure returned a 404 error when fetching the certificate from AKV. | | `incompatible`| An error occurred while fetching or processing the certificate from AKV.

The possible reasons include:

  • Error while downloading certificate and key
  • Missing content type in certificate
  • Missing content in certificate
  • Unrecognized content type, certificate not in PEM or PKCS12 format
| ![Interface screenshot showing the Azure metric nginxaas.certificates](nginxaas-azure/azure-metrics-nginxaas.certificates.png) ## Common certificate errors The following section describes common errors you might encounter while adding SSL/TLS certificates to your NGINXaaS deployment and how to resolve them. #### Common certificate errors #### Error code: `ForbiddenByRbac` **Description:** The [Managed Identity](/nginxaas-azure/getting-started/managed-identity-portal.md) associated with the NGINXaaS deployment does not have permissions to fetch certificates from key vault. This error is returned when the key vault's permission model is set to [Azure role-based access control](https://learn.microsoft.com/en-us/azure/role-based-access-control/overview?WT.mc_id=Portal-Microsoft_Azure_KeyVault). **Resolution:** Assign the [Key Vault Secrets User](https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles#key-vault-secrets-user) role to the managed identity associated with your NGINXaaS deployment. #### Create a role assignment using the CLI 1. Get the principal ID of the user or system assigned managed identity. - **User assigned managed identity** Set the following environment variables before copying the below Azure CLI command. - `MI_NAME`: the name of the managed identity - `MI_RESOURCE_GROUP`: the name of the resource group the managed identity is in ```shell mi_principal_id=$(az identity show --name $MI_NAME \ --resource-group $MI_RESOURCE_GROUP \ --query principalId --output tsv) ``` - **System assigned managed identity** Set the following environment variables before copying the below Azure CLI command. - `DEP_NAME`: the name of the NGINXaaS deployment - `DEP_RESOURCE_GROUP`: the name of the resource group the NGINXaaS deployment is in ```shell mi_principal_id=$(az nginx deployment show --name $DEP_NAME \ --resource-group $DEP_RESOURCE_GROUP \ --query identity.principalId --output tsv) ``` 1. Get the resource ID of the key vault. Set the following environment variables before copying the below Azure CLI command. - `KV_NAME`: the name of the key vault - `KV_RESOURCE_GROUP`: the name of the resource group the key vault is in ```shell key_vault_id=$(az keyvault show --name $KV_NAME \ --resource-group $KV_RESOURCE_GROUP \ --query id --output tsv) ``` 1. Create the role assignment. ```shell az role assignment create --assignee $mi_principal_id \ --role "Key Vault Secrets User" \ --scope $key_vault_id ``` #### Error code: `AccessDenied` **Description:** The [Managed Identity](/nginxaas-azure/getting-started/managed-identity-portal.md) associated with the NGINXaaS deployment has not been assigned to an access policy on the key vault. This error is returned when the key vault's permission model is set to [Vault access policy](https://learn.microsoft.com/en-us/azure/key-vault/general/assign-access-policy?WT.mc_id=Portal-Microsoft_Azure_KeyVault&tabs=azure-portal). **Resolution:** Assign an access policy to the managed identity associated with your NGINXaaS deployment with *Get secrets* permissions or higher. If you are using the Azure portal, assign an additional access policy to your user with *List certificates* permissions or higher. #### Create an access policy using the CLI 1. Get the principal ID of the user or system assigned managed identity. - **User assigned managed identity** Set the following environment variables before copying the below Azure CLI command. - `MI_NAME`: the name of the managed identity - `MI_RESOURCE_GROUP`: the name of the resource group the managed identity is in ```shell mi_principal_id=$(az identity show --name $MI_NAME \ --resource-group $MI_RESOURCE_GROUP \ --query principalId --output tsv) ``` - **System assigned managed identity** Set the following environment variables before copying the below Azure CLI command. - `DEP_NAME`: the name of the NGINXaaS deployment - `DEP_RESOURCE_GROUP`: the name of the resource group the NGINXaaS deployment is in ```shell mi_principal_id=$(az nginx deployment show --name $DEP_NAME \ --resource-group $DEP_RESOURCE_GROUP \ --query identity.principalId --output tsv) ``` 1. Create the access policy. Set the following environment variables before copying the below Azure CLI command. - `KV_NAME`: the name of the key vault - `KV_RESOURCE_GROUP`: the name of the resource group the key vault is in ```shell az keyvault set-policy --name $KV_NAME \ --resource-group $KV_RESOURCE_GROUP \ --object-id $mi_principal_id \ --secret-permissions get ``` #### Error code: `ForbiddenByFirewall` or `ForbiddenByConnection` **Description:** The key vault's firewall is enabled and NGINXaaS is not authorized to fetch certificates. **Resolution:** Allow NGINXaaS to access the key vault through one of these mechanisms: 1. [Configure Network Security Perimeter](/nginxaas-azure/quickstart/security-controls/certificates.md#configure-network-security-perimeter-nsp) to allow the subscription of the NGINXaaS deployment to access the key vault. #### Create a network security perimeter using the CLI 1. Create a network security perimeter. Set the following environment variables before copying the below Azure CLI command. - `NSP_NAME`: the name of the network security perimeter - `NSP_RESOURCE_GROUP`: the name of the resource group the network security perimeter will be in ```shell az network perimeter create --name $NSP_NAME --resource-group $NSP_RESOURCE_GROUP ``` 1. Create a profile for the network security perimeter. Set the following environment variable is set before copying the below Azure CLI command. - `PROFILE_NAME`: the name of the network security perimeter profile ```shell az network perimeter profile create --name $PROFILE_NAME \ --resource-group $NSP_RESOURCE_GROUP \ --perimeter-name $NSP_NAME ``` 1. Get the resource ID of the key vault. Set the following environment variables before copying the below Azure CLI command. - `KV_NAME`: the name of the key vault - `KV_RESOURCE_GROUP`: the name of the resource group the key vault is in ```shell key_vault_id=$(az keyvault show --name $KV_NAME \ --resource-group $KV_RESOURCE_GROUP \ --query id --output tsv) ``` 1. Get the resource ID of the network security profile. ```shell nsp_profile_id=$(az network perimeter profile show --name $PROFILE_NAME \ --resource-group $NSP_RESOURCE_GROUP \ --perimeter-name $NSP_NAME --query id --output tsv) ``` 1. Associate the key vault with the network security perimeter ```shell az network perimeter association create --name key-vault-association \ --perimeter-name $NSP_NAME \ --resource-group $NSP_RESOURCE_GROUP \ --private-link-resource "{id:$key_vault_id}" \ --profile "{id:$nsp_profile_id}" ``` 1. Add an inbound access rule to allow the NGINXaaS deployment's subscription. Set the following environment variables before copying the below Azure CLI command. - `RULE_NAME`: the name of the access rule - `DEP_SUBSCRIPTION_ID`: the subscription ID of the NGINXaaS deployment ```shell az network perimeter profile access-rule create --name $RULE_NAME \ --profile-name $PROFILE_NAME \ --perimeter-name $NSP_NAME \ --resource-group $NSP_RESOURCE_GROUP \ --subscriptions [0].id="/subscriptions/$DEP_SUBSCRIPTION_ID" ``` 2. Integrate with a Private Endpoint to allow NGINXaaS to fetch certificates via Azure Private Link. #### Create a private link using the CLI 1. Get the resource ID of the key vault. Set the following environment variables before copying the below Azure CLI command. - `KV_NAME`: the name of the key vault - `KV_RESOURCE_GROUP`: the name of tshe resource group the key vault is in ```shell key_vault_id=$(az keyvault show --name $KV_NAME \ --resource-group $KV_RESOURCE_GROUP \ --query id --output tsv) ``` 1. Create a private endpoint. Set the following environment variables before copying the below Azure CLI command. - `PE_NAME`: the name of the private endpoint - `PE_RESOURCE_GROUP`: the name of the resource group the private endpoint will be in - `VNET_NAME`: the name of the virtual network that is delegated to NGINXaaS - `VNET_RESOURCE_GROUP`: the name of the resource group the virtual network is in - `SUBNET_NAME`: the name of the subnet for private endpoints - `PE_CONNECTION_NAME`: the name of the private endpoint connection - `LOCATION`: the location of the virtual network ```shell az network private-endpoint create --name $PE_NAME \ --resource-group $PE_RESOURCE_GROUP \ --vnet-name $VNET_NAME \ --subnet $SUBNET_NAME \ --private-connection-resource-id $key_vault_id \ --group-id vault \ --connection-name $PE_CONNECTION_NAME \ --location $LOCATION ``` 1. Create a private DNS zone and link VNet. Set the following environment variables before copying the below Azure CLI command. - `ZONE_RESOURCE_GROUP`: the name of the resource group for the DNS zone - `ZONE_NAME`: the name of the DNS zone - `DNS_LINK_NAME`: the name of the DNS zone link ```shell vnet_id=$(az network vnet show --name $VNET_NAME \ --resource-group $VNET_RESOURCE_GROUP \ --query id --output tsv) ``` ```shell az network private-dns zone create --resource-group $ZONE_RESOURCE_GROUP \ --name $ZONE_NAME az network private-dns link vnet create --resource-group $ZONE_RESOURCE_GROUP \ --zone-name $ZONE_NAME \ --name $DNS_LINK_NAME \ --virtual-network $vnet_id \ --registration-enabled false ``` 1. Add DNS zone group to the private endpoint. Set the following environment variables before copying the below Azure CLI command. - `DNS_ZONE_GROUP_NAME`: the name of the resource group for the DNS zone ```shell az network private-endpoint dns-zone-group create \ --resource-group $PE_RESOURCE_GROUP \ --endpoint-name $PE_NAME \ --name $DNS_ZONE_GROUP_NAME \ --private-dns-zone $ZONE_NAME \ --zone-name $ZONE_NAME ``` 3. Allow access from Virtual Network delegated to NGINXaaS. #### Allow Virtual Network access using the CLI 1. Get the resource ID of the virtual network. Set the following environment variables before copying the below Azure CLI command. - `VNET_NAME`: the name of the virtual network that is delegated to NGINXaaS - `VNET_RESOURCE_GROUP`: the name of the resource group the virtual network is in ```shell vnet_id=$(az network vnet show --name $VNET_NAME \ --resource-group $VNET_RESOURCE_GROUP \ --query id --output tsv) ``` 1. Get the resource ID of the subnet. Set the following environment variable before copying the below Azure CLI command. - `SUBNET_NAME`: the name of the subnet that is delegated to NGINXaaS ```shell subnet_id=$(az network vnet subnet show --name $SUBNET_NAME \ --vnet-name $VNET_NAME \ --resource-group $VNET_RESOURCE_GROUP \ --query id --output tsv) ``` 1. Add the virtual network rule to the key vault. Set the following environment variables before copying the below Azure CLI command. - `KV_NAME`: the name of the key vault - `KV_RESOURCE_GROUP`: the name of the resource group the key vault is in ```shell az keyvault network-rule add --name $KV_NAME \ --resource-group $KV_RESOURCE_GROUP \ --subnet $subnet_id ``` **Note:** Ensure that the Network Security Group on the subnet delegated to the NGINXaaS deployment allows outbound traffic to the internet #### Error code: `AnotherOperationInProgress` **Description:** Another operation on this, or a dependent resource, is in progress. **Resolution:** Retry the operation after the current operation reaches a terminal state. #### Error code: `SecretNotFound` **Description:** The certificate's key vault secret ID was not found in the key vault. **Resolution:** Ensure the specified key vault secret ID exists and has the correct format, for example, `https://myvault.vault.azure.net/secrets/abcd/v1`. #### Error code: `CertificateInUse` **Description:** The certificate being deleted or modified is referenced in the NGINX configuration. The attempted modification would prevent the NGINX config from being applied. **Resolution:** Remove references to the certificate in the NGINX config, or add a new certificate resource to the NGINXaaS deployment with the modified certificate and key paths. #### Error code: `ForbiddenByPolicy` **Description:** The [Managed Identity](/nginxaas-azure/getting-started/managed-identity-portal.md) associated with the NGINXaaS deployment does not have permissions to fetch certificates from key vault. This error is returned when the key vault's permission model is set to [Vault access policy](https://learn.microsoft.com/en-us/azure/key-vault/general/assign-access-policy?WT.mc_id=Portal-Microsoft_Azure_KeyVault&tabs=azure-portal). **Resolution:** Assign an access policy to the managed identity associated with your NGINXaaS deployment with *Get secrets* permissions or higher. If you are using the Azure portal, assign an additional access policy to your user with *List certificates* permissions or higher. #### Create an access policy using the CLI 1. Get the principal ID of the user or system assigned managed identity. - **User assigned managed identity** Set the following environment variables before copying the below Azure CLI command. - `MI_NAME`: the name of the managed identity - `MI_RESOURCE_GROUP`: the name of the resource group the managed identity is in ```shell mi_principal_id=$(az identity show --name $MI_NAME \ --resource-group $MI_RESOURCE_GROUP \ --query principalId --output tsv) ``` - **System assigned managed identity** Set the following environment variables before copying the below Azure CLI command. - `DEP_NAME`: the name of the NGINXaaS deployment - `DEP_RESOURCE_GROUP`: the name of the resource group the NGINXaaS deployment is in ```shell mi_principal_id=$(az nginx deployment show --name $DEP_NAME \ --resource-group $DEP_RESOURCE_GROUP \ --query identity.principalId --output tsv) ``` 1. Create the access policy. Set the following environment variables before copying the below Azure CLI command. - `KV_NAME`: the name of the key vault - `KV_RESOURCE_GROUP`: the name of the resource group the key vault is in ```shell az keyvault set-policy --name $KV_NAME \ --resource-group $KV_RESOURCE_GROUP \ --object-id $mi_principal_id \ --secret-permissions get ``` #### Error code: `DuplicateFilePathError` **Description:** A file already exists on the instance's filesystem with the certificate or key's file path. **Resolution:** Rename the certificate or key path, so there are no collisions with existing certificate and NGINX config file paths. #### Error code: `SecretDisabled` **Description:** The certificate is set to disabled in the key vault. **Resolution:** Enable the certificate in the key vault. #### Enable a certificate in the key vault using the CLI 1. Get the resource ID of the certificate. Set the following environment variables before copying the below Azure CLI command. - `CERT_NAME`: the name of the certificate - `KV_NAME`: the name of the key vault ```shell certificate_id=$(az keyvault certificate show --name $CERT_NAME \ --vault-name $KV_NAME \ --query id --output tsv) ``` 1. Enable the certificate. ```shell az keyvault certificate set-attributes --enabled true --id $certificate_id ``` #### Error code: `NoCertificateContent` **Description:** No certificate was found when parsing the file. **Resolution:** Ensure the file is not empty and contains properly formatted PEM or PKCS12 certificate data. #### Error code: `MissingContentType` **Description:** The retrieved secret is missing the *contentType* field. **Resolution:** When creating an [Azure certificate](https://learn.microsoft.com/en-us/azure/key-vault/certificates/about-certificates) in key vault, the *contentType* field will be properly set to either *application/x-pem-file* or *application/x-pkcs12*. If a certificate is added as a generic [Azure secret](https://learn.microsoft.com/en-us/azure/key-vault/secrets/about-secrets) the *contentType* field must be manually set to help with interpreting the secret data when it is retrieved. We recommend creating a certificate object, instead of a secret object containing certificate data, to ensure proprer formatting and *contentType*. #### Error code: `UnrecognizedContentType` **Description:** The retrieved secret's content type, as interpreted from the *contentType* field, is of an unsupported type. **Resolution:** When creating an [Azure certificate](https://learn.microsoft.com/en-us/azure/key-vault/certificates/about-certificates) in key vault, the *contentType* field will be properly set to either *application/x-pem-file* or *application/x-pkcs12*. NGINXaaS does not support other content types added as generic [Azure secrets](https://learn.microsoft.com/en-us/azure/key-vault/secrets/about-secrets). Ensure the provided certificate is either a PEM or PKCS12 [Azure certificate](https://learn.microsoft.com/en-us/azure/key-vault/certificates/about-certificates). #### Error code: `PKCS12ParseFailure` **Description:** The PKCS12 certificate could not be parsed. **Resolution:** Ensure the file is not empty and contains properly formatted PKCS12 certificate data. #### Error code: `PEMParseFailure` **Description:** The PEM certificate could not be parsed. **Resolution:** Ensure the file is not empty and contains properly formatted PEM certificate data. -------------------------------------------------------------------------------- title: "Use a certificate from Azure Key Vault" url: https://docs.nginx.com/nginxaas-azure/quickstart/security-controls/certificates/index.md -------------------------------------------------------------------------------- ## Overview This tutorial walks through a complete example of using SSL/TLS certificates from Azure Key Vault in an F5 NGINXaaS for Azure (NGINXaaS) deployment to secure traffic. In this guide, you will create all necessary resources to add a certificate to an NGINXaaS deployment using the [Azure portal](https://portal.azure.com/). ## Create an Azure Key Vault (AKV) NGINXaaS enables customers to securely store SSL/TLS certificates in Azure Key Vault. If you do not have a key vault, follow these steps to create one: 1. From the Azure portal menu, or from the **Home** page, select **Create a resource**. 1. In the Search box, enter **Key Vault** and select the **Key Vault** service. 1. Select **Create**. 1. On the Create a key vault **Basics** tab, provide the following information: | Field | Description | |---------------------------- | ---------------------------- | | Subscription | Select the appropriate Azure subscription that you have access to. | | Resource group | Specify whether you want to create a new resource group or use an existing one.
For more information, see [Azure Resource Group overview](https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/overview). | | Key vault name | Provide a unique name for your key vault. For this tutorial, we use `nginxaas-kv`. | | Region | Select the region you want to deploy to. | For all other fields, you can leave them set to the default values. 1. Select **Review + Create** and then **Create**. ## Create an NGINXaaS deployment If you do not have an NGINXaaS deployment, follow the steps in [Deploy using the Azure portal](/nginxaas-azure/getting-started/create-deployment/deploy-azure-portal.md). **Note:** Your NGINXaaS deployment and your key vault must be in the same subscription. ## Add an SSL/TLS certificate to your key vault Next, you can add an SSL/TLS certificate to your key vault by following [Azure's documentation to import an existing certificate](https://learn.microsoft.com/en-us/azure/key-vault/certificates/tutorial-import-certificate?tabs=azure-portal), or you can generate a certificate. This tutorial will generate a self-signed certificate to quickly get started. 1. Go to your key vault, `nginxaas-kv`. 1. Select **Certificates** in the left menu. 1. Select [icon: plus]**Generate/Import** and provide the following information: | Field | Description | |---------------------------- | ---------------------------- | | Method of Certificate Creation | Select **Generate** | | Certificate Name | Provide a unique name for your certificate. For this tutorial, we use `nginxaas-cert`. | | Type of Certificate Authority (CA) | Select **Self-signed certificate**. | | CN | Provide the IP address of your NGINXaaS deployment as the CN. For example, `CN=135.237.74.224` | For all other fields, you can leave them set to the default values. 1. Select **Create**. ## Assign a managed identity to your NGINXaaS deployment In order for your NGINXaaS deployment to access your key vault, it must have an assigned managed identity with the `Key Vault Secrets User` role. For more information, see [Assign Managed Identities](/nginxaas-azure/getting-started/managed-identity-portal.md) and [Prerequisites for adding SSL/TLS certificates](/nginxaas-azure/getting-started/ssl-tls-certificates/ssl-tls-certificates-portal.md#prerequisites). 1. Go to your NGINXaaS deployment. 1. Select **Identity** in the left menu. 1. Under **System assigned**, ensure the status is set to "On". **Note:** When you create a deployment through the Azure portal, a system-assigned managed identity is automatically enabled for your deployment. 1. Under **System assigned**, select **Azure role assignments**. 1. Select [icon: plus]**Add role assignment** and provide the following information: | Field | Description | |---------------------------- | ---------------------------- | | Scope | Select **Key Vault**. | | Subscription | Select the Azure subscription your key vault is in. | | Resource | Select your key vault, `nginxaas-kv`. | | Role | Select **Key Vault Secrets User**. | 1. Select **Save**. ## Add your certificate to your NGINXaaS deployment Now, you can add your SSL/TLS certificate from your key vault to your NGINXaaS deployment. For more information, see [Add certificates using the Azure portal](/nginxaas-azure/getting-started/ssl-tls-certificates/ssl-tls-certificates-portal.md). 1. Go to your NGINXaaS deployment. 1. Select **NGINX certificates** in the left menu. 1. Select [icon: plus]**Add certificate** and provide the following information: | Field | Description | |---------------------------- | ---------------------------- | | Name | A unique name for the certificate. For this tutorial, we use `my-cert`. | | Certificate path | Set to `/etc/nginx/ssl/example.crt`. | | Key path | Set to `/etc/nginx/ssl/example.key`. | 1. Select **Select certificate** and provide the following information: | Field | Description | |----------------------- | ---------------------------- | | Key vault | Select `nginxaas-kv`. | | Certificate | Select `nginxaas-cert`. | 1. Select **Add certificate**. ## Reference your certificate in your NGINX configuration Once a certificate has been added to your NGINXaaS deployment, you can reference it in your NGINX configuration to secure traffic. Refer to [Upload an NGINX configuration](/nginxaas-azure/getting-started/nginx-configuration/overview.md) to add and update NGINX configuration files to your NGINXaaS deployment. The following NGINX configurations show examples of different certificate use cases. ### Use case 1: SSL/TLS termination NGINXaaS supports SSL/TLS termination by decrypting incoming encrypted traffic before forwarding it on to your upstream servers. ```nginx http { upstream backend { server backend1.example.com:8000; # replace with your backend server address and port } server { listen 443 ssl; ssl_certificate /etc/nginx/ssl/example.crt; # must match the Certificate path ssl_certificate_key /etc/nginx/ssl/example.key; # must match the Key path location / { proxy_pass http://backend; } } } ``` For more information on using NGINX for SSL/TLS termination, see [NGINX SSL Termination](https://docs.nginx.com/nginx/admin-guide/security-controls/terminating-ssl-http/). ### Use case 2: Secure traffic to upstream servers NGINXaaS supports backend encryption by encrypting traffic between your NGINXaaS deployment and your upstream servers. ```nginx http { upstream backend { server backend1.example.com:8443; # replace with your backend server address and port } server { listen 80; location / { proxy_pass https://backend; proxy_ssl_certificate /etc/nginx/ssl/client.crt; # must match the Certificate path proxy_ssl_certificate_key /etc/nginx/ssl/client.key; # must match the Key path } } } ``` For more information on using NGINX to secure traffic to upstream servers, refer to [Securing HTTP Traffic to Upstream Servers](https://docs.nginx.com/nginx/admin-guide/security-controls/securing-http-traffic-upstream/) and [Securing TCP Traffic to Upstream Servers](https://docs.nginx.com/nginx/admin-guide/security-controls/securing-tcp-traffic-upstream/). ## Restrict public access to Key Vault If you want to restrict public access to your key vault, you can configure: - a [Network Security Perimeter (NSP)](https://learn.microsoft.com/en-us/azure/private-link/network-security-perimeter-concepts). This will allow you to configure access rules to allow NGINXaaS to fetch certificates from your key vault while ensuring all other public access is denied. - Allow access from a Virtual Network. This will allow you to configure access from the Virtual Network that is delegated to NGINXaaS while ensuring all other public access is denied. - Integrate Azure Key Vault with [Azure Private Link](https://learn.microsoft.com/en-us/azure/private-link/private-link-overview). To enhance network security, you can configure your vault to only allow connections through private endpoints. Traffic between NGINXaaS and AKV traverses over the Microsoft backbone network. ### Configure Network Security Perimeter (NSP) 1. Follow [Azure's documentation on prerequisites](https://learn.microsoft.com/en-us/azure/private-link/create-network-security-perimeter-portal#prerequisites) to ensure you are registered to create an NSP. 1. In the Search box, enter **Network Security Perimeters** and select **Network Security Perimeters** from the search results. 1. Select [icon: plus]**Create**. 1. In the **Basics** tab, provide the following information: | Field | Description | |---------------------------- | ---------------------------- | | Subscription | Select the appropriate Azure subscription that you have access to. | | Resource group | Specify whether you want to create a new resource group or use an existing one.
For more information, see [Azure Resource Group overview](https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/overview). | | Name | Provide a unique name for your network security perimeter. For this tutorial, we use `nginxaas-nsp`. | | Region | Select the region you want to deploy to. | | Profile name | Leave the profile name as the default `defaultProfile`. | 1. In the **Resources** tab, select [icon: plus]**Add**. 1. Search for your key vault, `nginxaas-kv`, select it, and click **Select**. 1. In the **Inbound access rules** tab, select [icon: plus]**Add** and provide the following information: | Field | Description | |---------------------------- | ---------------------------- | | Rule Name | Set to `allow-nginxaas-deployment-sub`. | | Source Type | Select **Subscriptions**. | | Allowed sources | Select the subscription of your NGINXaaS deployment. | 1. Select **Review + Create** and then **Create**. By default, the key vault will be associated to the NSP in [Learning mode](https://learn.microsoft.com/en-us/azure/private-link/network-security-perimeter-concepts#access-modes-in-network-security-perimeter). This means traffic will be evaluated first based on the NSP's access rules. If no rules apply, evaluation will fall back to the key vault's firewall configuration. To fully secure public access, it is recommended to [transition to Enforced mode](https://learn.microsoft.com/en-us/azure/private-link/network-security-perimeter-transition#transition-to-enforced-mode-for-existing-resources). 1. Go to resource `nginxaas-nsp`. 1. Select **Associated resources** in the left menu. 1. Select the `nginxaas-kv` resource association. 1. Select **Change access mode**, set to **Enforced**, and select **Apply**. **Note:** If you are using the Azure portal to add certificates, you will also need to add an inbound access rule to allow your IP address, so the portal can list the certificates in your key vault. ### Integrate with Private Endpoint 1. Go to your key vault, `nginxaas-kv`. 1. Select **Settings** followed by **Networking** in the left menu. 1. Select the **Private endpoint connections** tab. 1. Select [icon: plus] **Create** 1. In the **Basics** tab, provide the following information: | Field | Description | |---------------------------- | ---------------------------- | | Subscription | Select the appropriate Azure subscription that you have access to. | | Resource group | Specify whether you want to create a new resource group or use an existing one.
For more information, see [Azure Resource Group overview](https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/overview). | | Name | Provide a unique name for your private link. For this tutorial, we use `nginxaas-pl`. | | Region | Select the region you want to deploy to. 1. In the **Resources** tab, select **Resource Type** as `Microsoft.KeyVault/vaults` and **Resource** as `nginxaas-kv` 1. In the **Virtual Network** tab, provide the following information | Field | Description | |---------------------------- | ---------------------------- | | Virtual network | Select the virtual network delegated to your NGINXaaS deployment. | | Subnet | Select a subnet from your virtual network that is not being used. 1. In the **DNS** tab, use the default settings to integrate your private endpoint with a private DNS zone. 1. Select **Review + Create** and then **Create**. Once a private link is configured and public access is disabled on Azure Key Vault, any certificates added to the NGINXaaS deployment will be fetched over the private link. ### Allow access from a Virtual Network 1. Go to your key vault, `nginxaas-kv`. 1. Select **Networking** in the left menu. 1. Select [icon: plus] **Add existing virtual network**. 1. Select the virtual network and subnet that is delegated to the NGINXaaS deployment. **Note:** Ensure that the Network Security Group on the subnet delegated to the NGINXaaS deployment allows outbound traffic to the internet -------------------------------------------------------------------------------- title: "Add certificates using the Azure portal" url: https://docs.nginx.com/nginxaas-azure/getting-started/ssl-tls-certificates/ssl-tls-certificates-portal/index.md -------------------------------------------------------------------------------- ## Overview You can manage SSL/TLS certificates for F5 NGINXaaS for Azure (NGINXaaS) using the Azure portal. ## Prerequisites NGINXaaS natively integrates with [Azure Key Vault (AKV)](https://azure.microsoft.com/en-us/products/key-vault), so you can bring your own certificates and manage them in a centralized location. You will need: - AKV to store certificates that you want to add to the deployment. - A user or system assigned identity associated with your NGINXaaS deployment. Ensure that your Managed Identity (MI) has read access to secrets stored in AKV: - If using Azure RBAC for AKV, ensure that your MI has [Key Vault Secrets User](https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles#key-vault-secrets-user) or higher permissions. - If using Access Policies for AKV, ensure that your MI has *GET secrets* or higher permissions. - Access to AKV through a public or private endpoint. If public access to AKV needs to be restricted, refer to [Restrict Public Access to Key Vault](/nginxaas-azure/quickstart/security-controls/certificates.md#restrict-public-access-to-key-vault). - In addition to the MI permissions, if using the Azure portal to manage certificates, ensure that you have read access to list certificates inside the Key Vault: - If using Azure RBAC for AKV, ensure that you have [Key Vault Reader](https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles#key-vault-reader) or higher permissions. - If using Access Policies for AKV, ensure that you have *LIST certificates* or higher permissions. - If public access is disabled on your key vault, add an inbound access rule to allow your client IP address. - If you're unfamiliar with Azure Key Vault, check out the [Azure Key Vault concepts](https://docs.microsoft.com/en-us/azure/key-vault/general/basic-concepts) documentation from Microsoft. ## Finding the Azure Key Vault Secret Identifier When adding a certificate using the Azure CLI, Terraform, or ARM/Bicep templates, you need to provide the **Secret Identifier** from Azure Key Vault. **Note:** **Use the Secret Identifier, not the Certificate Identifier.** NGINXaaS requires the **Secret Identifier** to access the certificate and its private key. To find the Secret Identifier: 1. Go to your Azure Key Vault in the Azure portal. 1. Select **Certificates** from the left menu. 1. Select the certificate you want to use. 1. Select the current version of the certificate. 1. Copy the **Secret Identifier** value (not the Certificate Identifier). The Secret Identifier format is: ```text https://{vault-name}.vault.azure.net/secrets/{certificate-name} ``` ### Adding an SSL/TLS certificate Before you begin, refer Azure documentation to [Import a certificate to your Key Vault](https://learn.microsoft.com/en-us/azure/key-vault/certificates/tutorial-import-certificate?tabs=azure-portal#import-a-certificate-to-your-key-vault). 1. Go to your NGINXaaS for Azure deployment. 1. Select **NGINX certificates** in the left menu. 1. Select [icon: plus]**Add certificate**. 1. Provide the required information: | Field | Description | |---------------------------- | ---------------------------- | | Name | A unique name for the certificate. | | Certificate path | This path can match one or more `ssl_certificate` directive file arguments in your NGINX configuration.
The certificate path must be unique within the same deployment. | | Key path | This path can match one or more `ssl_certificate_key` directive file arguments in your NGINX configuration.
The key path must be unique within the same deployment.
The key path and certificate path can be the same within the certificate. | - The **Select certificate** button will take you to a new screen where you will need to provide the following information: | Field | Description | |----------------------- | ---------------------------- | | Key vault | Select from the available key vaults. | | Certificate | Select the certificate you want to add from the previously selected key vault. | If you need to create a new key vault or certificate, you can do so by selecting **Create new key vault** or **Create new** under the **Key Vault** and **Certificate** fields, respectively. **Note:** If specifying an absolute file path as the `Certificate path` or `Key path`, see the [NGINX Filesystem Restrictions table](/nginxaas-azure/getting-started/nginx-configuration/overview/#nginx-filesystem-restrictions) for the allowed directories the file can be written to. **Note:** A certificate added to an NGINXaaS for Azure deployment using the Azure Portal refers to an unversioned Azure Key Vault (AKV) secret identifier. To add a certificate with a versioned AKV secret identifier, follow the documented steps with alternative [Client tools](/nginxaas-azure/client-tools/_index.md) for NGINXaaS for Azure. 1. Select **Add certificate**. 1. Repeat the same steps to add as many certificates as needed. 1. Now you can [provide an NGINX configuration](/nginxaas-azure/getting-started/nginx-configuration/nginx-configuration-portal.md) that references the certificate you just added by the **path** value. ### View certificate details 1. Go to your NGINXaaS for Azure deployment and select **NGINX certificates** in the left menu. 1. Select the name of the certificate from the list. 1. View the certificate details, including the certificate path, key path, thumbprint, and the certificate's status. This view will also show in a red box any errors that occurred during the certificate fetch process. ### Edit an SSL/TLS certificate 1. Go to your NGINXaaS for Azure deployment and select **NGINX certificates** in the left menu. 1. Select the checkbox next to the certificate you want to edit. 1. Select [icon: pencil] **Edit**. 1. Update the Name, Certificate path, Key path fields as needed. 1. Use the **Select certificate** option to update the Key vault, and Certificate fields as needed. 1. Select **Update**. ### Delete an SSL/TLS certificate 1. Go to your NGINXaaS for Azure deployment and select **NGINX certificates** in the left menu. 1. Select the checkbox next to the certificate you want to delete. 1. Select [icon: trash]**Delete**. 1. Confirm the delete action. **Note:** Deleting a TLS/SSL certificate currently in-use by the NGINXaaS for Azure deployment will cause an error. ## What's next [Upload an NGINX Configuration](/nginxaas-azure/getting-started/nginx-configuration/nginx-configuration-portal.md) -------------------------------------------------------------------------------- title: "Azure Resource Manager templates" url: https://docs.nginx.com/nginxaas-azure/client-tools/templates/index.md description: Learn how to use Azure Resource Manager (ARM) JSON and Bicep templates to manage NGINXaaS for Azure. -------------------------------------------------------------------------------- F5 NGINXaaS for Azure (NGINXaaS) deployments can be managed using the ARM API or the Azure CLI with ARM template deployments using JSON or Bicep formats. These deployments can be made locally or in a continuous integration pipeline. This document outlines common workflows using the ARM API. You can find example code to manage NGINXaaS deployments and related objects in the NGINXaaS GitHub repository, [NGINXaaS Snippets](https://github.com/nginxinc/nginxaas-for-azure-snippets). ## Prerequisites - [NGINXaaS Prerequisites](/nginxaas-azure/getting-started/prerequisites.md) - [Azure CLI Installation](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli) - You need to be logged in to your Azure account through the CLI if you are using that for template deployment, see [Azure CLI Authentication](https://learn.microsoft.com/en-us/cli/azure/authenticate-azure-cli) - See [NGINXaaS Snippets](https://github.com/nginxinc/nginxaas-for-azure-snippets/tree/main/arm-templates/deployments/prerequisites) for an example template to create the prerequisite resources. ## Workflows ### Create or update a deployment See [NGINXaaS Snippets](https://github.com/nginxinc/nginxaas-for-azure-snippets/tree/main/arm-templates/deployments/create-or-update) for an example template to create or update deployment resources. - [NGINXaaS Managed Identity Documentation](/nginxaas-azure/getting-started/managed-identity-portal.md) - [NGINXaaS Azure Monitor Documentation](/nginxaas-azure/monitoring/enable-metrics.md) ### Create or update a certificate Create or update a certificate under a deployment. This references an existing certificate in an Azure Key Vault and makes it available to the NGINX configuration. See [NGINXaaS Snippets](https://github.com/nginxinc/nginxaas-for-azure-snippets/tree/main/arm-templates/certificates/create-or-update) for an example template to create or update certificate resources. - [NGINXaaS Certificates Documentation](/nginxaas-azure/getting-started/ssl-tls-certificates/ssl-tls-certificates-portal.md) - [ARM Template Key Vault Documentation](https://learn.microsoft.com/en-us/azure/templates/microsoft.keyvault/vaults) ### Create or update a configuration Create or update the default configuration for a deployment using a gzipped archive based on the NGINXaaS documentation below. See [NGINXaaS Snippets](https://github.com/nginxinc/nginxaas-for-azure-snippets/tree/main/arm-templates/configuration) for an example template to create or update configuration resources. - [NGINXaaS GZIP Configuration Documentation](/nginxaas-azure/getting-started/nginx-configuration/nginx-configuration-portal.md#upload-gzip-nginx-configuration) ## Additional Docs If you are new to Azure Resource Manager templates, see: - [Azure JSON Templates Overview](https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/overview) - [Azure Bicep Templates Overview](https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/overview) -------------------------------------------------------------------------------- title: "Billing overview" url: https://docs.nginx.com/nginxaas-azure/billing/overview/index.md -------------------------------------------------------------------------------- NGINXaaS for Azure is deployed into your Azure subscription. Your NGINXaaS deployment resource is visible within your subscription, while the underlying infrastructure is managed by F5 and is abstracted away from you. ## Pricing plans NGINXaaS for Azure is billed monthly based on hourly consumption. F5 NGINXaaS for Azure (NGINXaaS) provides two pricing plans. ### Standard V3 plan The Standard V3 plan is an upgraded, purpose-built solution for modern enterprises looking to simplify application traffic management and scale workloads effortlessly. This improved plan offers a [99.95% uptime SLA](https://www.f5.com/pdf/customer-support/eusa-sla.pdf), high availability through active-active deployments, redundancy, lossless rolling upgrades, and dynamic autoscaling capabilities to optimize both performance and cost. The Standard V3 plan introduces dynamic autoscaling that ensures consumption pricing - customers pay only for what they use. Each NGINX Capacity Unit (NCU) delivers 2.2 Mbps bandwidth and 3,000 connections, providing unmatched flexibility and scalability to suit diverse workloads. The Standard V3 pricing model is designed to optimize efficiency and transparency: customers benefit from an affordable fixed price per deployment ($0.25/hour) that covers baseline overhead, while NCU usage ($0.008/hour/unit) and data processing ($0.005/GB) allow costs to scale precisely with demand. NGINXaaS is a consumption-based service, metered hourly, and billed monthly in NGINX Capacity Units (NCUs). The Standard V3 plan allows for optional Web Application Firewall (WAF) configuration and a higher number of listen ports, offering enhanced security and connectivity options for enterprise applications. The SKU for the Standard V3 pricing plan is `standardv3_Monthly`. ### Developer plan The Developer plan is ideal for those who are just starting out, as it's intended for early-stage trials, development work, and testing. Please note that it doesn't provide service level agreement (SLA) guarantees, and it lacks both redundancy options and the capability to scale resources as needed. When using the Developer plan, each NGINXaaS deployment is billed at the rate specified on the [Azure Marketplace Offer](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/f5-networks.f5-nginx-for-azure?tab=Overview). The SKU for the Developer pricing plan is `developer_Monthly`. **Note:** The costs for your plan will appear on the Azure Portal Cost Analysis page and the Azure Consumption APIs. There may be a 24h delay before usage is visible. ## NGINX Capacity Unit (NCU) An NGINX Capacity Unit (NCU) quantifies the capacity of an NGINX instance based on the underlying compute resources. This abstraction allows you to specify the desired minimum capacity in NCUs without having to consider the regional hardware differences. An NGINX Capacity Unit consists of the following parameters: * CPU: an NCU provides 20 [Azure Compute Units](https://learn.microsoft.com/en-us/azure/virtual-machines/acu) (ACUs) * Bandwidth: an NCU provides 2.2 Mbps of network throughput * Concurrent connections: an NCU provides 3000 concurrent connections. This performance is not guaranteed when F5 WAF for NGINX is used with NGINXaaS Each NCU provisioned (not consumed) is billed at the rate specified on the [Azure Marketplace Offer](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/f5-networks.f5-nginx-for-azure?tab=Overview). The minimum usage interval is 1 hour, and the maximum provisioned NCU size is billed for that hour. *Billing Example 1*: "I provisioned a 20 NCU NGINXaaS deployment in East US 2 at 9:04AM and then deleted it at 10:45AM. Assuming 1GB data processed" * The hourly fixed price per deployment is `$0.25/hour` * The hourly NCU rate in East US 2 is `$0.008/NCU/hour` * The data processing price is `$0.005/GB` * 9:00 hour: `20 NCU·hour` * 10:00 hour: `20 NCU·hour` * Total NCU·hours: `40 NCU·hour` * Fixed deployment cost: `2 hours * $0.25/hour = $0.50` * NCU usage cost: `40 NCU·hour * $0.008/NCU/hour = $0.32` * Data processing cost: `1 GB * $0.005/GB = $0.005` * Total: `$0.50 + $0.32 + $0.005 = $0.87` *Billing Example 2*: "I provisioned a 40 NCU NGINXaaS deployment in West Europe at 9:34AM. At 10:04AM I resized it to 20 NCUs. I then deleted it at 11:45AM. Assuming 2.5GB data processed" * The hourly fixed price per deployment is `$0.25/hour` * The hourly NCU rate in West Europe is `$0.008/NCU/hour` * The data processing price is `$0.005/GB` * 9:00 hour: `40 NCU·hour` * 10:00 hour: `40 NCU·hour` * 11:00 hour: `20 NCU·hour` * Total NCU·hours: `100 NCU·hour` * Fixed deployment cost: `3 hours * $0.25/hour = $0.75` * NCU usage cost: `100 NCU·hour * $0.008/NCU/hour = $0.80` * Data processing cost: `2.5 GB * $0.005/GB = $0.125` * Total: `$0.75 + $0.80 + $0.125 = $1.675` **Note:** Further guidance: * For how many NCUs should you provision and how to scale to match workload, see the [Scaling Guidance](/nginxaas-azure/quickstart/scaling.md) * To learn more about metrics related to NCUs, see the [NGINXaaS Statistics namespace](/nginxaas-azure/monitoring/metrics-catalog.md#nginxaas-statistics) ## Bandwidth The standard Azure [networking](https://azure.microsoft.com/en-us/pricing/details/virtual-network/) and [bandwidth](https://azure.microsoft.com/en-us/pricing/details/bandwidth/) charges apply to NGINX deployments. **Note:** The management traffic for NGINX instances is billed as a `Virtual Network Peering - Intra-Region Egress` charge. This charge includes the data for shipping metrics and logs. The cost for shipping metrics data is approximately $0.03/month. If you enable NGINX logging the cost increases by roughly $0.005 per GB of logs NGINX generates. To estimate this, multiply the number of requests by the average log line size of the access_log format you have configured. ## Review billing data NGINXaaS billing data is reported per deployment and can be viewed in the Azure Portal under "Cost Management + Billing".​ ### Standard V2 plan (deprecated) The Standard V2 plan is designed for production workloads offering a [99.95% uptime SLA](https://www.f5.com/pdf/customer-support/eusa-sla.pdf), high availability through active-active deployments, redundancy, autoscaling, lossless rolling upgrades, and more. Choosing the Standard V2 plan will result in billing based on metered consumption of NGINX Capacity Units (NCU). When using the Standard V2 plan, NGINXaaS is a consumption-based service, metered hourly, and billed monthly in NGINX Capacity Units (NCUs). The SKU for the Standard V2 pricing plan is `standardv2_Monthly`. The Standard V2 plan allows for configuration of F5 WAF for NGINX and a higher number of listen ports. -------------------------------------------------------------------------------- title: "Deploy using the Azure portal" url: https://docs.nginx.com/nginxaas-azure/getting-started/create-deployment/deploy-azure-portal/index.md -------------------------------------------------------------------------------- ## Overview This guide explains how to deploy F5 NGINXaaS for Azure (NGINXaaS) using [Microsoft Azure portal](https://azure.microsoft.com/en-us/get-started/azure-portal). The deployment process involves creating a new deployment, configuring the deployment, and testing the deployment. ## Find the NGINXaaS for Azure offer in the Azure portal You can start the NGINXaaS deployment process by visiting the [Create NGINXaaS](https://portal.azure.com/#create/f5-networks.f5-nginx-for-azure) page or finding the NGINXaaS service in the Azure portal: 1. [Sign in](https://portal.azure.com/) to the Azure portal with your Azure account. 1. Use the search field to find "NGINXaaS" in the Azure Portal. In the Services results, select **NGINXaaS**. 1. Select **+ Create** on the **NGINXaaS** page to start the deployment process. ## Create a deployment ### Basics tab 1. On the Create NGINXaaS Deployment **Basics** page, provide the following information: | Field | Description | |---------------------------- | ---------------------------- | | Subscription | Select the appropriate Azure subscription that you have access to.| | Resource group | Specify whether you want to create a new resource group or use an existing one.
For more information, see [Azure Resource Group overview](https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/overview). | | Name | Provide a unique name for your deployment. | | Region | Select the region you want to deploy to. | | Pricing Plan | Select the Standard V3 plan. For more information, see [Pricing Plans](/nginxaas-azure/billing/overview.md#pricing-plans) | | Scaling | Select Manual to set the capacity of your deployment in NCUs or select Autoscale to automatically adjust the capacity of your deployment. Learn more about NCUs in [Scaling Guidance](/nginxaas-azure/quickstart/scaling.md). | | Email | Provide an email address that can be notified about service alerts, maintenance data and activity reports. | | Upgrade Channel | Select the desired upgrade channel for your deployment. For more information, see [Upgrade Channels](/nginxaas-azure/quickstart/upgrade-channels.md). | 1. Next, select **Networking**. ### Networking tab 1. On the Create NGINXaaS Deployment **Networking** page, provide the following information: | Field | Description | |---------------------------- | ---------------------------- | | Virtual Network | A virtual network is required for communication between the resources you create.
You can create a new virtual network or use an existing one (for an existing one see note below).
Additionally, you can peer a new virtual network with existing ones (in any region) to create network access from NGINXaaS for Azure to your upstream servers. To peer the virtual network with another see [Create, change, or delete a virtual network peering](https://docs.microsoft.com/en-us/azure/virtual-network/virtual-network-manage-peering).| | Subnet | If you select an existing virtual network, you can select the existing subnet to be used. Before creating a deployment, the existing subnet needs to be delegated to `NGINX.NGINXPLUS/nginxDeployments`. To delegate a subnet to an Azure service, see [Delegate a subnet to an Azure service](https://learn.microsoft.com/en-us/azure/virtual-network/manage-subnet-delegation?source=recommendations#delegate-a-subnet-to-an-azure-service).

Otherwise, if you have chosen to create a new virtual network, a new subnet will be selected by default.

The minimum subnet size is `/27` and is sufficient for a single NGINXaaS deployment even at large scales. Multiple NGINXaaS deployments can be placed in a single delegated subnet, along with other resources. When doing so a larger subnet, e.g. a `/24`, is recommended. | | Allow NGINX access to Virtual Network | Confirm that you allow:
- Registration of the NGINX provider to your Azure subscription.
- Delegation of the subnet to the NGINX provider.| | IP address | Set the IP address (public or private) that the service listens to for requests:

If you select a public IP address:
- Create a new public IP or use an existing one (for an existing one see the note below).
- Set the resource name for your public IP address.
Newly created public IPs are [zone-redundant in supported regions](https://learn.microsoft.com/en-us/azure/virtual-network/ip-services/public-ip-addresses#availability-zone).

If you select a private IP address:
- Provide a static IP address from the same subnet range set previously. | | Inbound port rules | Select `None` to disallow inbound access on any port, or choose to allow traffic from one of these common http(s) ports.

**Note:** This option is only available when specifying a new virtual network as part of the create workflow. If you select an existing virtual network which is associated with a subnet and Network Security Group (NSG), you will need to edit the Inbound security rules to add access for the specific ports you want to allow (for example, ports 80 and 443).| | Apply default NGINX configuration | Confirm that you want your NGINXaaS deployment to be bootstrapped with a default NGINX configuration and a browsable splash page. | #### Notes on subnets: - The minimum subnet size for NGINXaaS is `/27` and is enough for a single NGINXaaS deployment even at large scales. - You can use an existing subnet; make sure that the subnet is delegated to `NGINX.NGINXPLUS/nginxDeployments` before creating a deployment in it. The subnet can contain other resources. - You can deploy more than one NGINXaaS in the same subnet. Every deployment in the subnet will share the subnet IP address space. A larger subnet, for example, a `/24` is recommended. - If you plan on using an IPv6 address on the frontend, make sure the subnet is dual-stack, i.e., the subnet has both IPv4 and IPv6 address spaces. Attempting to use a subnet that is not dual-stack will cause deployment creation to fail. - Changes to a virtual network's DNS settings will not be applied automatically to your NGINXaaS deployment. To ensure DNS settings are applied, you must add any custom DNS servers to the VNET's DNS settings before creating an NGINXaaS deployment. As a workaround for existing deployments, we recommend using the [`resolver` directive](https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver) to explicitly specify your name server(s) and the [`resolve` parameter](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#resolve) to automatically re-resolve the domain name of the server without restarting NGINX. #### Resolver directive example ```nginx resolver 10.0.0.2 valid=10s; upstream backends { zone backends 64k; server backends.example.com:8080 resolve; } server { location / { proxy_pass http://backends; } } ``` 1. Next, select **Tags**. ### Tags tab 1. Add custom tags for the new NGINXaaS Deployment. Each tag consists of a **name** and **value**. 1. After adding the tags, select **Next: Review+Create** ### Review + create tab 1. On the Review + create tab, your configuration is validated. You can review the selections made in the previous screens. 1. After validation has succeeded and you've reviewed the terms, select **Create** for Azure to start the deployment. 1. After the deployment finishes, select the NGINX deployment from the list (with "Type: NGINXaaS") to view information about the deployed resource, including the deployment IP address. ![Resource Deployment Completed page showing the available deployments and the new NGINXaaS type deployment in the Deployment details section.](nginxaas-azure/deployment-complete.png) ## Test your deployment 1. To test your deployment, you can go to the IP address noted on the overview page. The default NGINX welcome screen should load. **Note:** You will not see the default NGINX welcome screen if you unchecked "Apply default NGINX configuration" in the [Networking Tab screen](/nginxaas-azure/getting-started/create-deployment.md#networking-tab) above. You can proceed with providing your own NGINX configuration as outlined in the [NGINX configuration](/nginxaas-azure/getting-started/nginx-configuration.md#networking-tab) section. ![NGINXaaS Overview page showing the IP address of the deployment in the Essentials section.](nginxaas-azure/test-deployment.png) **Note:** It's not possible to manually stop or start NGINXaaS. If necessary, you have the option to delete the deployment and re-deploy at a future date. ## What's next [Assign Managed Identities](/nginxaas-azure/getting-started/managed-identity-portal.md) -------------------------------------------------------------------------------- title: "Enable NGINX logs using Azure Portal" url: https://docs.nginx.com/nginxaas-azure/monitoring/enable-logging/logging-using-portal/index.md -------------------------------------------------------------------------------- ## Overview F5 NGINXaaS for Azure (NGINXaaS) supports integrating Azure Diagnostic Settings to collect NGINX error and access logs. **Note:** Enabling logs using the **NGINX Logs** blade on your NGINXaaS deployment is now deprecated. This feature will be removed in an upcoming update. If you have issues accessing your NGINX logs using the deprecated method, please follow the steps in this guide to access your NGINX logs. ## Configuring NGINX logs collection using diagnostic settings ### Prerequisites - A valid NGINX configuration with log directives enabled. NGINX logs can be configured using [error_log](#setting-up-error-logs) and [access_log](#setting-up-access-logs) directives. - A system-assigned managed identity. **Note:** The system-assigned managed identity does not need any role assignments to enable the logging functionality described in this section. You will need to make sure that the managed identity has the appropriate role assignments to access other resources that it is attached to (for example, certificates stored in Azure Key Vault). - User must be an owner or user access administrator for the NGINX deployment resource. ### Adding diagnostic settings 1. Go to your NGINXaaS for Azure deployment. 1. Select **Diagnostic Settings** in the left menu. 1. Select **Add diagnostic setting**. 1. Choose the **NGINX Logs** option and complete the details on the form, including the **Diagnostic setting name**. **Note:** You will need to configure the system-assigned managed identity in order to see and select the **NGINX Logs** option. 1. Select preferred **Destination details**. ![Screenshot of the Diagnostic Settings configuration page](nginxaas-azure/diagnostic-settings.png) As NGINXaaS logs are stored in your storage, you can define the retention policy most appropriate for your needs. For more information about diagnostic settings destinations, please see the [Diagnostic Settings Destinations](https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/diagnostic-settings#destinations) documentation. **Note:** Due to limitations imposed by Azure, if the destination chosen is an Azure Storage account, the resource has to be in the same region as the NGINXaaS deployment resource. **Note:** If you are a Terraform user, please refer to [examples](https://github.com/nginxinc/nginxaas-for-azure-snippets/tree/main/terraform/deployments/with-diagnostic-setting-logging) provided to setup diagnostic settings for your NGINXaaS deployment ### Analyzing NGINX logs in Azure Storage If the diagnostic setting destination details included a storage account, logs show up in the storage container "insights-logs-nginxlogs" with the following format: `resourceID=//y=/m=/d=
/h=/PT1H.json` | **Attribute** | **Description** | |-----------------------------|-----------------| | `` | The resourceID of the NGINXaaS deployment in upper case.| | `` | The four-digit year when the log batch was generated.| | `` | The two-digit month when the log batch was generated.| | `
` | The two-digit day when the log batch was generated.| | `` | The two-digit hour value that indicates the starting hour for the log batch, in 24 hour UTC format| **Note:** It can take up to 90 minutes after adding diagnostic settings for logs to appear in the provided Azure Storage container. Each log event in the "PT1H.json" file is written in a new line delimited JSON text format. The properties that show up in each log line are described in the [Top Level Common Schema](https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/resource-logs-schema#top-level-common-schema) documentation. For instance, an access log event logging to a particular file path will have attributes similar to this example: ```yaml { "category": "NginxLogs", "location": "westcentralus", "operationName": "NGINX.NGINXPLUS/NGINXDEPLOYMENTS/LOG", "properties": { "message": "172.92.129.50 - \"-\" [18/Jan/2024:17:59:00 +0000] \"GET / HTTP/1.1\" 200 11232 \"-\" \"curl/8.4.0\" \"-\" \"20.69.58.179\" sn=\"localhost\" rt=0.000 ua=\"-\" us=\"-\" ut=\"-\" ul=\"-\" cs=\"-\" ", "filePath": "/var/log/nginx/access.log" }, "resourceId": "/SUBSCRIPTIONS/FFFFFFFF-FFFF-FFFF-FFFF-FFFFFFFFFFFF/RESOURCEGROUPS/RESOURCEGROUP1/PROVIDERS/NGINX.NGINXPLUS/NGINXDEPLOYMENTS/TEST1", "time": "2024-01-18T17:59:00.363956795Z" } ``` If [syslog-based](#logging-to-syslog) logs are used, the log event entry has different **properties** sub-fields: ```yaml #... "properties": { "message": "172.92.129.50 - - [16/Jan/2024:18:00:00 +0000] \"GET / HTTP/1.1\" 200 11232 \"-\" \"curl/8.4.0\"", "tag": "nginx", "severity": "info", "facility": "local7" }, #... ``` ### Analyzing NGINX logs in Azure Log Analytics workspaces If the diagnostic setting destination details included a Logs Analytics workspace, logs show up in the table "NGXOperationLogs" with the following non-standard attributes: | **Attribute** | **Description** | |-----------------------------|-----------------| | **Location** | The location of the NGINXaaS resource.| | **Message** | The generated NGINX log line. | | **FilePath** | The path to which NGINX logs were configured to be logged to if the nginx config used file-based logs. | | **Tag** | The tag with which NGINX logs were generated if syslog-based log configuration is used. By default this is nginx | | **Facility** | The syslog facility with which NGINX logs were generated if syslog-based log configuration is used. | | **Severity** | The syslog severity with which NGINX logs were generated if syslog-based log configuration is used. | Using a [KQL](https://learn.microsoft.com/en-us/azure/data-explorer/kusto/query/), a custom query can be run to view the logs: ``` NGXOperationLogs | where Location contains "eastus" ``` For more information on the standard attributes that appear in Logs Analytics,see the [Standard columns in Azure Monitor Logs](https://learn.microsoft.com/en-us/azure/azure-monitor/logs/log-standard-columns) documentation. For more information on using [KQL](https://learn.microsoft.com/en-us/azure/data-explorer/kusto/query/) see [Queries in Log Analytics](https://learn.microsoft.com/en-us/azure/azure-monitor/logs/queries?tabs=groupby). **Note:** It can take up to 90 minutes after adding diagnostic settings for logs to appear in the provided Logs Analytics Workspace. ### Disable NGINX logs collection 1. Go to your NGINXaaS for Azure deployment. 1. Select **Diagnostic Settings** in the left menu. 1. Edit the previously added Diagnostic Settings. 1. Select **Delete**. **Note:** It can take up to 90 minutes after removing the diagnostic settings for logs to stop publishing to the diagnostic destinations. ## Setting up error logs By default, NGINXaaS for Azure puts the error log at **/var/log/nginx/error.log**. It includes messages with severity **error** and above. While you should configure log files in the **/var/log/nginx** directory, you can change the filename and severity level. For example, the following line in the NGINX configuration sends errors to the `nginx-error.log` file, and limits messages to a severity level of **emerg**: ```nginx error_log /var/log/nginx/nginx-error.log emerg; ``` Alternatively, you can disable error logs completely with the following line: ```nginx error_log /dev/null; ``` To learn more about how to specify `error_log` in different configuration levels, see the documentation of the [error_log](https://nginx.org/en/docs/ngx_core_module.html?#error_log) directive. ## Setting up access logs NGINX access logs are disabled by default. You can enable access logs by adding **access_log** directives to your NGINX configuration to specify the location of the logs and formats. The log path should always be configured to be inside **/var/log/nginx**. ```nginx http { log_format myfmt '$remote_addr - $remote_user [$time_local] ' '"$request" $status $body_bytes_sent ' '"$http_referer" "$http_user_agent" "$gzip_ratio"'; access_log /var/log/nginx/nginx-access.log myfmt; # ... } ``` **Note:** The **$time_local** variable includes the date and time for each log. It helps with ordering logs after export. To explicitly disable access logs, apply the following config: ```nginx http { access_log off; } ``` or ```nginx http { access_log /dev/null; } ``` To learn more about how to specify `access__log` in different configuration levels and their effect, see [access_log](https://nginx.org/en/docs/http/ngx_http_log_module.html#access_log) **Note:** Unless you use **syslog**, keep NGINX logs in the **/var/log/nginx** directory. Otherwise, you may lose data from your logs. ## Limitations 1. File-based logs must be configured to use the path **/var/log/nginx**. 1. The **gzip** parameter for the **access_log** directive is not supported, and uploading a config with this parameter will cause an error. 1. Logging **error_log** to a cyclic memory buffer using the **memory:** prefix is not allowed and will cause a config upload error. 1. Egress Networking charges apply for traffic sent from the NGINX deployment to a syslog server present in a different VNet. -------------------------------------------------------------------------------- title: "Enable NGINX logs using CLI" url: https://docs.nginx.com/nginxaas-azure/monitoring/enable-logging/logging-using-cli/index.md -------------------------------------------------------------------------------- ## Overview F5 NGINXaaS for Azure (NGINXaaS) supports integrating Azure Diagnostic Settings to collect NGINX error and access logs. **Note:** Enabling logs using the **NGINX Logs** blade on your NGINXaaS deployment is now deprecated. This feature will be removed in an upcoming update. If you have issues accessing your NGINX logs using the deprecated method, please follow the steps in this guide to access your NGINX logs. ## Configuring NGINX logs collection using diagnostic settings ### Prerequisites - A valid NGINX configuration with log directives enabled. NGINX logs can be configured using [error_log](#setting-up-error-logs) and [access_log](#setting-up-access-logs) directives. - A system-assigned managed identity. **Note:** The system-assigned managed identity does not need any role assignments to enable the logging functionality described in this section. You will need to make sure that the managed identity has the appropriate role assignments to access other resources that it is attached to (for example, certificates stored in Azure Key Vault). - User must be an owner or user access administrator for the NGINX deployment resource. ### Adding diagnostic settings Diagnostic settings for the NGINXaaS deployment resource can be managed using the Azure monitor diagnostic settings [commands](https://learn.microsoft.com/en-us/cli/azure/monitor/diagnostic-settings?view=azure-cli-latest). To add diagnostic settings to export NGINX logs to a storage account for an NGINXaaS deployment, the following command can be used: ```shell az monitor diagnostic-settings create --resource --logs "[{category:NginxLogs,enabled:true,retention-policy:{enabled:false,days:0}}]" --name --storage-account ``` **Note:** Due to limitations imposed by Azure, if the destination chosen is an Azure Storage account, the resource has to be in the same region as the NGINXaaS deployment resource. To use a logs analytics workspace as the export destination, use the following command: ```shell az monitor diagnostic-settings create --resource --logs "[{category:NginxLogs,enabled:true,retention-policy:{enabled:false,days:0}}]" --name --workspace ``` To view the supported log categories for an NGINXaaS resource, use the following command: ```shell az monitor diagnostic-settings list --resource ``` As NGINXaaS logs are stored in your storage, you can define the retention policy most appropriate for your needs. ### Analyzing NGINX logs in Azure Storage If the diagnostic setting destination details included a storage account, logs show up in the storage container "insights-logs-nginxlogs" with the following format: `resourceID=//y=/m=/d=
/h=/PT1H.json` | **Attribute** | **Description** | |-----------------------------|-----------------| | `` | The resourceID of the NGINXaaS deployment in upper case.| | `` | The four-digit year when the log batch was generated.| | `` | The two-digit month when the log batch was generated.| | `
` | The two-digit day when the log batch was generated.| | `` | The two-digit hour value that indicates the starting hour for the log batch, in 24 hour UTC format| **Note:** It can take up to 90 minutes after adding diagnostic settings for logs to appear in the provided Azure Storage container. Each log event in the "PT1H.json" file is written in a new line delimited JSON text format. The properties that show up in each log line are described in the [Top Level Common Schema](https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/resource-logs-schema#top-level-common-schema) documentation. For instance, an access log event logging to a particular file path will have attributes similar to this example: ```yaml { "category": "NginxLogs", "location": "westcentralus", "operationName": "NGINX.NGINXPLUS/NGINXDEPLOYMENTS/LOG", "properties": { "message": "172.92.129.50 - \"-\" [18/Jan/2024:17:59:00 +0000] \"GET / HTTP/1.1\" 200 11232 \"-\" \"curl/8.4.0\" \"-\" \"20.69.58.179\" sn=\"localhost\" rt=0.000 ua=\"-\" us=\"-\" ut=\"-\" ul=\"-\" cs=\"-\" ", "filePath": "/var/log/nginx/access.log" }, "resourceId": "/SUBSCRIPTIONS/FFFFFFFF-FFFF-FFFF-FFFF-FFFFFFFFFFFF/RESOURCEGROUPS/RESOURCEGROUP1/PROVIDERS/NGINX.NGINXPLUS/NGINXDEPLOYMENTS/TEST1", "time": "2024-01-18T17:59:00.363956795Z" } ``` If [syslog-based](#logging-to-syslog) logs are used, the log event entry has different **properties** sub-fields: ```yaml #... "properties": { "message": "172.92.129.50 - - [16/Jan/2024:18:00:00 +0000] \"GET / HTTP/1.1\" 200 11232 \"-\" \"curl/8.4.0\"", "tag": "nginx", "severity": "info", "facility": "local7" }, #... ``` ### Analyzing NGINX logs in Azure Log Analytics workspaces If the diagnostic setting destination details included a Logs Analytics workspace, logs show up in the table "NGXOperationLogs" with the following non-standard attributes: | **Attribute** | **Description** | |-----------------------------|-----------------| | **Location** | The location of the NGINXaaS resource.| | **Message** | The generated NGINX log line. | | **FilePath** | The path to which NGINX logs were configured to be logged to if the nginx config used file-based logs. | | **Tag** | The tag with which NGINX logs were generated if syslog-based log configuration is used. By default this is nginx | | **Facility** | The syslog facility with which NGINX logs were generated if syslog-based log configuration is used. | | **Severity** | The syslog severity with which NGINX logs were generated if syslog-based log configuration is used. | Using a [KQL](https://learn.microsoft.com/en-us/azure/data-explorer/kusto/query/), a custom query can be run to view the logs: ``` NGXOperationLogs | where Location contains "eastus" ``` For more information on the standard attributes that appear in Logs Analytics,see the [Standard columns in Azure Monitor Logs](https://learn.microsoft.com/en-us/azure/azure-monitor/logs/log-standard-columns) documentation. For more information on using [KQL](https://learn.microsoft.com/en-us/azure/data-explorer/kusto/query/) see [Queries in Log Analytics](https://learn.microsoft.com/en-us/azure/azure-monitor/logs/queries?tabs=groupby). **Note:** It can take up to 90 minutes after adding diagnostic settings for logs to appear in the provided Logs Analytics Workspace. ## Setting up error logs By default, NGINXaaS for Azure puts the error log at **/var/log/nginx/error.log**. It includes messages with severity **error** and above. While you should configure log files in the **/var/log/nginx** directory, you can change the filename and severity level. For example, the following line in the NGINX configuration sends errors to the `nginx-error.log` file, and limits messages to a severity level of **emerg**: ```nginx error_log /var/log/nginx/nginx-error.log emerg; ``` Alternatively, you can disable error logs completely with the following line: ```nginx error_log /dev/null; ``` To learn more about how to specify `error_log` in different configuration levels, see the documentation of the [error_log](https://nginx.org/en/docs/ngx_core_module.html?#error_log) directive. ## Setting up access logs NGINX access logs are disabled by default. You can enable access logs by adding **access_log** directives to your NGINX configuration to specify the location of the logs and formats. The log path should always be configured to be inside **/var/log/nginx**. ```nginx http { log_format myfmt '$remote_addr - $remote_user [$time_local] ' '"$request" $status $body_bytes_sent ' '"$http_referer" "$http_user_agent" "$gzip_ratio"'; access_log /var/log/nginx/nginx-access.log myfmt; # ... } ``` **Note:** The **$time_local** variable includes the date and time for each log. It helps with ordering logs after export. To explicitly disable access logs, apply the following config: ```nginx http { access_log off; } ``` or ```nginx http { access_log /dev/null; } ``` To learn more about how to specify `access__log` in different configuration levels and their effect, see [access_log](https://nginx.org/en/docs/http/ngx_http_log_module.html#access_log) **Note:** Unless you use **syslog**, keep NGINX logs in the **/var/log/nginx** directory. Otherwise, you may lose data from your logs. ## Limitations 1. File-based logs must be configured to use the path **/var/log/nginx**. 1. The **gzip** parameter for the **access_log** directive is not supported, and uploading a config with this parameter will cause an error. 1. Logging **error_log** to a cyclic memory buffer using the **memory:** prefix is not allowed and will cause a config upload error. 1. Egress Networking charges apply for traffic sent from the NGINX deployment to a syslog server present in a different VNet. -------------------------------------------------------------------------------- title: "Enable NGINX logs using Terraform" url: https://docs.nginx.com/nginxaas-azure/monitoring/enable-logging/logging-using-terraform/index.md -------------------------------------------------------------------------------- ## Overview F5 NGINXaaS for Azure (NGINXaaS) supports integrating Azure Diagnostic Settings to collect NGINX error and access logs. **Note:** Enabling logs using the **NGINX Logs** blade on your NGINXaaS deployment is now deprecated. This feature will be removed in an upcoming update. If you have issues accessing your NGINX logs using the deprecated method, please follow the steps in this guide to access your NGINX logs. ## Configuring NGINX logs collection using diagnostic settings ### Prerequisites - A valid NGINX configuration with log directives enabled. NGINX logs can be configured using [error_log](#setting-up-error-logs) and [access_log](#setting-up-access-logs) directives. - A system-assigned managed identity. **Note:** The system-assigned managed identity does not need any role assignments to enable the logging functionality described in this section. You will need to make sure that the managed identity has the appropriate role assignments to access other resources that it is attached to (for example, certificates stored in Azure Key Vault). - User must be an owner or user access administrator for the NGINX deployment resource. ### Adding diagnostic settings To setup diagnostic settings to enable logging to a storage account for your NGINXaaS deployment, please refer to [examples](https://github.com/nginxinc/nginxaas-for-azure-snippets/tree/main/terraform/deployments/with-diagnostic-setting-logging) provided. Once the terraform configurations provided in the repository above are used, the following commands can be used to update the deployment. ```shell terraform init terraform plan -var="storage_account_resource_group=myresourcegroup" -var="storage_account_name=myaccountname" -out=plan.cache terraform apply plan.cache ``` As NGINXaaS logs are stored in your storage, you can define the retention policy most appropriate for your needs. **Note:** Due to limitations imposed by Azure, if the destination chosen is an Azure Storage account, the resource has to be in the same region as the NGINXaaS deployment resource. ### Analyzing NGINX logs in Azure Storage If the diagnostic setting destination details included a storage account, logs show up in the storage container "insights-logs-nginxlogs" with the following format: `resourceID=//y=/m=/d=
/h=/PT1H.json` | **Attribute** | **Description** | |-----------------------------|-----------------| | `` | The resourceID of the NGINXaaS deployment in upper case.| | `` | The four-digit year when the log batch was generated.| | `` | The two-digit month when the log batch was generated.| | `
` | The two-digit day when the log batch was generated.| | `` | The two-digit hour value that indicates the starting hour for the log batch, in 24 hour UTC format| **Note:** It can take up to 90 minutes after adding diagnostic settings for logs to appear in the provided Azure Storage container. Each log event in the "PT1H.json" file is written in a new line delimited JSON text format. The properties that show up in each log line are described in the [Top Level Common Schema](https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/resource-logs-schema#top-level-common-schema) documentation. For instance, an access log event logging to a particular file path will have attributes similar to this example: ```yaml { "category": "NginxLogs", "location": "westcentralus", "operationName": "NGINX.NGINXPLUS/NGINXDEPLOYMENTS/LOG", "properties": { "message": "172.92.129.50 - \"-\" [18/Jan/2024:17:59:00 +0000] \"GET / HTTP/1.1\" 200 11232 \"-\" \"curl/8.4.0\" \"-\" \"20.69.58.179\" sn=\"localhost\" rt=0.000 ua=\"-\" us=\"-\" ut=\"-\" ul=\"-\" cs=\"-\" ", "filePath": "/var/log/nginx/access.log" }, "resourceId": "/SUBSCRIPTIONS/FFFFFFFF-FFFF-FFFF-FFFF-FFFFFFFFFFFF/RESOURCEGROUPS/RESOURCEGROUP1/PROVIDERS/NGINX.NGINXPLUS/NGINXDEPLOYMENTS/TEST1", "time": "2024-01-18T17:59:00.363956795Z" } ``` If [syslog-based](#logging-to-syslog) logs are used, the log event entry has different **properties** sub-fields: ```yaml #... "properties": { "message": "172.92.129.50 - - [16/Jan/2024:18:00:00 +0000] \"GET / HTTP/1.1\" 200 11232 \"-\" \"curl/8.4.0\"", "tag": "nginx", "severity": "info", "facility": "local7" }, #... ``` ### Analyzing NGINX logs in Azure Log Analytics workspaces If the diagnostic setting destination details included a Logs Analytics workspace, logs show up in the table "NGXOperationLogs" with the following non-standard attributes: | **Attribute** | **Description** | |-----------------------------|-----------------| | **Location** | The location of the NGINXaaS resource.| | **Message** | The generated NGINX log line. | | **FilePath** | The path to which NGINX logs were configured to be logged to if the nginx config used file-based logs. | | **Tag** | The tag with which NGINX logs were generated if syslog-based log configuration is used. By default this is nginx | | **Facility** | The syslog facility with which NGINX logs were generated if syslog-based log configuration is used. | | **Severity** | The syslog severity with which NGINX logs were generated if syslog-based log configuration is used. | Using a [KQL](https://learn.microsoft.com/en-us/azure/data-explorer/kusto/query/), a custom query can be run to view the logs: ``` NGXOperationLogs | where Location contains "eastus" ``` For more information on the standard attributes that appear in Logs Analytics,see the [Standard columns in Azure Monitor Logs](https://learn.microsoft.com/en-us/azure/azure-monitor/logs/log-standard-columns) documentation. For more information on using [KQL](https://learn.microsoft.com/en-us/azure/data-explorer/kusto/query/) see [Queries in Log Analytics](https://learn.microsoft.com/en-us/azure/azure-monitor/logs/queries?tabs=groupby). **Note:** It can take up to 90 minutes after adding diagnostic settings for logs to appear in the provided Logs Analytics Workspace. ## Setting up error logs By default, NGINXaaS for Azure puts the error log at **/var/log/nginx/error.log**. It includes messages with severity **error** and above. While you should configure log files in the **/var/log/nginx** directory, you can change the filename and severity level. For example, the following line in the NGINX configuration sends errors to the `nginx-error.log` file, and limits messages to a severity level of **emerg**: ```nginx error_log /var/log/nginx/nginx-error.log emerg; ``` Alternatively, you can disable error logs completely with the following line: ```nginx error_log /dev/null; ``` To learn more about how to specify `error_log` in different configuration levels, see the documentation of the [error_log](https://nginx.org/en/docs/ngx_core_module.html?#error_log) directive. ## Setting up access logs NGINX access logs are disabled by default. You can enable access logs by adding **access_log** directives to your NGINX configuration to specify the location of the logs and formats. The log path should always be configured to be inside **/var/log/nginx**. ```nginx http { log_format myfmt '$remote_addr - $remote_user [$time_local] ' '"$request" $status $body_bytes_sent ' '"$http_referer" "$http_user_agent" "$gzip_ratio"'; access_log /var/log/nginx/nginx-access.log myfmt; # ... } ``` **Note:** The **$time_local** variable includes the date and time for each log. It helps with ordering logs after export. To explicitly disable access logs, apply the following config: ```nginx http { access_log off; } ``` or ```nginx http { access_log /dev/null; } ``` To learn more about how to specify `access__log` in different configuration levels and their effect, see [access_log](https://nginx.org/en/docs/http/ngx_http_log_module.html#access_log) **Note:** Unless you use **syslog**, keep NGINX logs in the **/var/log/nginx** directory. Otherwise, you may lose data from your logs. ## Limitations 1. File-based logs must be configured to use the path **/var/log/nginx**. 1. The **gzip** parameter for the **access_log** directive is not supported, and uploading a config with this parameter will cause an error. 1. Logging **error_log** to a cyclic memory buffer using the **memory:** prefix is not allowed and will cause a config upload error. 1. Egress Networking charges apply for traffic sent from the NGINX deployment to a syslog server present in a different VNet. -------------------------------------------------------------------------------- title: "Migrate to new plans" url: https://docs.nginx.com/nginxaas-azure/billing/change-plan/migrate-between-plans/index.md -------------------------------------------------------------------------------- ## Overview F5 NGINXaaS for Azure (NGINXaaS) supports in-place migration between certain plans without requiring redeployment. This allows you to upgrade your deployment to access new features and capabilities while maintaining your existing configuration and avoiding downtime. ## Supported migration paths The following in-place migrations are supported: - **Basic → Developer**: Upgrade from the Basic plan to the Developer plan to access advanced features - **Standard → Standard V3**: Upgrade from the legacy Standard plan to the modern Standard V3 plan - **Standard V2 → Standard V3**: Upgrade from Standard V2 to Standard V3 to access the latest features. **Note:** If you have a Standard or Standard V2 plan deployment, we encourage you to migrate to the Standard V3 plan as soon as possible. If you have a Basic plan deployment, we encourage you to migrate to the Developer plan as soon as possible. Legacy plans will stop receiving automatic updates that address critical security issues. ## Migration steps ### Use the Azure Portal 1. Go to the **Overview** page of the NGINXaaS deployment in the Azure portal. 2. Under **Essentials**, find the **Pricing Tier** and select **Click to Upgrade**. 3. Select the Standard V3 plan and select Submit. ### Use Terraform 1. Update the Terraform AzureRM provider to 4.6.0 or above. ``` terraform { required_version = "~> 1.3" required_providers { azurerm = { source = "hashicorp/azurerm" version = "~> 4.6.0" } } } ``` 2. Modify the SKU in the `azurerm_nginx_deployment` resource to your target plan: - For Developer plan: `developer_Monthly` - For Standard V3 plan: `standardv3_Monthly` 3. Run `terraform plan` and review the output to ensure your NGINXaaS deployment is being updated (not replaced). 4. Run `terraform apply` to complete the migration. ### Use the Azure CLI Before running the migration command, ensure you have the latest Azure CLI nginx extension installed: ```shell az extension update --name nginx --allow-preview true ``` Then run the command below to update your NGINXaaS deployment, replacing the SKU name with your target plan: ```shell az nginx deployment update --name myDeployment --resource-group myResourceGroup \ --sku name="" ``` Replace `` with one of the following: - `developer_n7ja87drquhy` for Developer plan - `standardv3_Monthly_n7ja87drquhy` for Standard V3 plan -------------------------------------------------------------------------------- title: "Overview and architecture" url: https://docs.nginx.com/nginxaas-azure/overview/overview/index.md -------------------------------------------------------------------------------- ## What Is F5 NGINXaaS for Azure? NGINXaaS for Azure is a service offering that is tightly integrated into Microsoft Azure public cloud and its ecosystem, making applications fast, efficient, and reliable with full lifecycle management of advanced NGINX traffic services. NGINXaaS for Azure is available in the Azure Marketplace. NGINXaaS for Azure is powered by [NGINX Plus](https://www.nginx.com/products/nginx/), which extends NGINX Open Source with advanced functionality and provides customers with a complete application delivery solution. Initial use cases covered by NGINXaaS include L4 TCP and L7 HTTP load balancing and reverse proxy which can be managed through various Azure management tools. NGINXaaS allows you to provision distinct deployments as per your business or technical requirements. NGINXaaS handles the NGINX Plus license management automatically. ## Capabilities The key capabilities of NGINXaaS for Azure are: - Simplifies onboarding by leveraging NGINX as a service. - Lowers operational overhead in running and optimizing NGINX. - Simplifies NGINX deployments with fewer moving parts (edge routing is built into the service). - Supports migration of existing NGINX configurations to the cloud with minimal effort. - Integrates with the Azure ecosystem (Microsoft Entra, Azure Key Vault, and Azure Monitor). - Addresses a wide range of deployment scenarios (HTTP reverse proxy, JWT authentication, etc). - Adopts a consumption-based pricing to align infrastructure costs to actual usage by billing transactions using Azure. - Supports end-to-end encryption from client to upstream server. - Supports the following protocols: HTTPS, HTTP, HTTP/2, HTTP/3, TCP, QUIC, IMAP, POP3, and SMTP. - Supports any type of message body for upstream and error status code responses, including text/plain, text/css, text/html, application/javascript, and application/json. ## Limitations - NGINXaaS supports at most one IPv4 and one IPv6 IP address at any given time. - NGINXaaS doesn't support a mix of public and private IPs at this time. - The IP address associated with an NGINXaaS deployment can't be changed from public to private, or from private to public. ## Supported regions NGINXaaS for Azure is supported in the following regions: | **North America** | **South America** | **Europe** | **Asia Pacific** | |----------------------------------------------------------|--------------------------------------------|--------------------------------------------|-------------------------| | West Central US
West US
East US 2
West US 2
West US 3
East US
Central US
North Central US
South Central US
Canada Central
Canada East | Brazil South | West Europe
North Europe
Sweden Central
Germany West Central
UK West
UK South | Australia East
Japan East
Korea Central
Southeast Asia
Central India
South India | ## NGINXaaS architecture ![The diagram illustrates the architecture of F5 NGINXaaS for Azure within a Microsoft Azure environment. It shows admins using Azure API/SDK, Azure Portal, Azure CLI, and Terraform to interact with the NGINX Plus component in the IaaS layer for edge routing. The diagram also depicts subnet delegation from the NGINX Plus component to a customer subscription, which includes Azure Key Vault, Azure Monitor, other Azure services, and multiple application servers (App Server 1, App Server 2, App Server N).](nginxaas-azure/n4a-architecture.png) - Azure management tools (API, CLI, portal, terraform) work with NGINXaaS to create, update, and delete deployments - Each NGINXaaS deployment has dedicated network and compute resources. There is no possibility of [noisy neighbor problems](https://learn.microsoft.com/en-us/azure/architecture/antipatterns/noisy-neighbor/noisy-neighbor) or data leakage between deployments - NGINXaaS can route traffic to upstreams even if the upstream servers are located in different geographies. See [Known Issues](/nginxaas-azure/known-issues.md) for any networking restrictions. - NGINXaaS supports request tracing. See the [Application Performance Management with NGINX Variables](https://www.f5.com/company/blog/nginx/application-tracing-nginx-plus) blog to learn more about tracing. - Supports HTTP to HTTPS, HTTPS to HTTP, and HTTP to HTTP redirects. NGINXaaS also provides the ability to create new rules for redirecting. See [How to Create NGINX Rewrite Rules | NGINX](https://www.nginx.com/blog/creating-nginx-rewrite-rules/) for more details. - NGINXaaS is deployed inside of your Azure network and can connect to your upstream application running in your ecosystem. Known networking limitations can be found in the [Known Issues](/nginxaas-azure/known-issues.md). ### Redundancy With the Standard V3 Plan, NGINXaaS uses the following redundancy features to keep your service available. - We run _at least_ two NGINX Plus instances for each deployment in an active-active pattern - NGINX Plus is constantly monitored for health. Any unhealthy instances are replaced with new ones - We use [Azure Availability Zones](https://learn.microsoft.com/en-us/azure/availability-zones/az-overview) to protect your deployment from local failures within an Azure region. We balance NGINX instances across the possible availability zones in [supported regions](https://learn.microsoft.com/en-us/azure/availability-zones/az-overview#azure-regions-with-availability-zones) **Note:** If you are creating a public IP for your deployment, be sure to make them [zone redundant](https://learn.microsoft.com/en-us/azure/virtual-network/ip-services/public-ip-addresses#availability-zone) to get the best uptime. ### Data plane traffic ![The diagram illustrates the architecture of F5 NGINXaaS for Azure, showing end users accessing a public IP that routes through a network security group within a customer's Azure subscription. This leads to a delegated subnet in a virtual network, which connects to a zone-redundant load balancer within the NGINXaaS subscription. The load balancer distributes traffic across NGINX Plus instances in multiple availability zones, ensuring scalability and redundancy.](nginxaas-azure/n4a-data-plane-architecture.svg) - NGINXaaS uses new Azure networking capabilities to keep end-user traffic private. A network security group ensures that the deployment is secured and inbound connections are allowed to the ports the NGINX service listens to. - You can restrict access to NGINXaaS by defining restriction rules at the Network Security Group level or using NGINX's access control list. To learn more, see the [NGINX module ngx_http_access_module](http://nginx.org/en/docs/http/ngx_http_access_module.html) documentation. - NGINXaaS deployment is automatically upgraded to receive security patches and the latest stable NGINX Plus version. ## What's next To get started, check the [NGINXaaS for Azure prerequisites](/nginxaas-azure/getting-started/prerequisites.md) -------------------------------------------------------------------------------- title: "Overview and prerequisites" url: https://docs.nginx.com/nginxaas-azure/getting-started/prerequisites/index.md -------------------------------------------------------------------------------- ## Before you begin Before you deploy NGINXaaS you need to meet the following prerequisites: - An Azure account with an active subscription (if you don’t have one, [create an account for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F)). - [Confirm that you have the appropriate access](https://docs.microsoft.com/en-us/azure/role-based-access-control/check-access) before starting the setup: - The simplest approach is to use Azure’s built-in [Owner](https://docs.microsoft.com/en-us/azure/role-based-access-control/built-in-roles#owner) role on either a specific resource group or the subscription. - It's possible to complete a limited setup with the built-in [Contributor](https://docs.microsoft.com/en-us/azure/role-based-access-control/built-in-roles#contributor) role. - The specific Azure permissions required to deploy NGINXaaS are: - microsoft.network/publicIPAddresses/join/action - nginx.nginxplus/nginxDeployments/Write - microsoft.network/virtualNetworks/subnets/join/action - nginx.nginxplus/nginxDeployments/configurations/Write - nginx.nginxplus/nginxDeployments/certificates/Write - Additionally, if you are creating the Virtual Network or IP address resources that NGINXaaS for Azure will be using, then you probably also want those permissions as well. - Note that assigning the managed identity permissions normally requires an "Owner" role. ## What's next [Create a Deployment](/nginxaas-azure/getting-started/create-deployment/) -------------------------------------------------------------------------------- title: "Restricting access with HTTP basic authentication" url: https://docs.nginx.com/nginxaas-azure/security-controls/auth-basic/index.md -------------------------------------------------------------------------------- You can restrict access to resources by implementing username/password authentication using the "HTTP Basic Authentication" protocol. For more information on configuring HTTP Basic Authentication please refer to the [NGINX Plus Restricting Access with HTTP Basic Authentication](https://docs.nginx.com/nginx/admin-guide/security-controls/configuring-http-basic-authentication/) documentation. ## Uploading a password file F5 NGINXaaS for Azure (NGINXaaS) accepts a file containing usernames and passwords using any of the password types specified in the [NGINX documentation](https://nginx.org/en/docs/http/ngx_http_auth_basic_module.html#auth_basic_user_file). The password file can be uploaded as a "protected file" when creating or updating your NGINX configuration to protect the file's contents from being read. The password file can alternatively be uploaded as a regular file. ![Screenshot of the Azure portal showing the password file upload](nginxaas-azure/auth-basic-htpasswd.png) ## Configuring NGINX Plus for HTTP basic authentication Inside the location or server you are protecting, specify the [`auth_basic`](https://nginx.org/en/docs/http/ngx_http_auth_basic_module.html#auth_basic) directive giving a name to the password-protected area. Specify the [`auth_basic_user_file`](https://nginx.org/en/docs/http/ngx_http_auth_basic_module.html#auth_basic_user_file) directive referencing the password file. ```nginx location /protected { auth_basic "Protected Area"; auth_basic_user_file /opt/.htpasswd; } ``` Submit the NGINX configuration to apply it. You should be prompted to log in when you access the protected location or server. **Note:** The NGINX worker processes will open the password file. You must place the password file in a [directory the worker processes are allowed to read](/nginxaas-azure/getting-started/nginx-configuration/nginx-configuration-portal.md#nginx-filesystem-restrictions) or else all authenticated requests will fail. - `/opt` - `/srv` - `/var/www` -------------------------------------------------------------------------------- title: "Runtime State Sharing" url: https://docs.nginx.com/nginxaas-azure/quickstart/runtime-state-sharing/index.md -------------------------------------------------------------------------------- F5 NGINXaaS for Azure (NGINXaaS) supports runtime state sharing using the [Zone Synchronization module](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html) to synchronize shared memory zones across NGINXaaS instances. With runtime state sharing, NGINXaaS instances can share some state data between them, including: - [Sticky‑learn session persistence](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#sticky_learn) - [Rate limiting](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_zone) - [Key‑value store](https://nginx.org/en/docs/http/ngx_http_keyval_module.html#keyval_zone) **Note:** It's not possible to add the `sync` parameter with a directive describing shared memory zones to an existing memory zone that was not configured to sync. It also cannot be removed from an existing memory zone configured to sync. To switch, remove the directive before reapplying it with the desired parameters. For information on enabling synchronization for rate limiting with NGINXaaS for Azure, please visit the [Rate Limiting](/nginxaas-azure/quickstart/rate-limiting.md) documentation. ## Configuring runtime state sharing among NGINXaaS for Azure deployment cluster instances To enable runtime state sharing, edit the NGINXaaS deployment's NGINX configuration to create a server with the `zone_sync` directive in the top-level `stream` block. The `stream` `server` block containing the `zone_sync` directive should use a local resolver at `127.0.0.1:49153`. It should also provide a `listen` directive with only a port for the TCP server. The chosen port should match the port used with `zone_sync_server` directive. NGINXaaS cluster instances should use `internal.nginxaas.nginx.com` as the domain name. They resolve using the `resolve` parameter of the `zone_sync_server` directive. ```nginx stream { resolver 127.0.0.1:49153 valid=20s; server { listen 9000; # should match the port specified with zone_sync_server zone_sync; zone_sync_server internal.nginxaas.nginx.com:9000 resolve; } } ``` **Note:** To enhance security, set up security rules for both incoming and outgoing traffic in the virtual network linked to the NSG of the subnet hosting NGINXaaS for Azure deployment. These rules should limit TCP traffic to the `zone_sync_server` port. ## Enable the SSL/TLS protocol for connections to another cluster instance of the NGINXaaS for Azure deployment To allow SSL connections between cluster instances, edit the NGINXaaS deployment's NGINX configuration to enable the `zone_sync_ssl` directive along with `zone_sync` directive in the top-level `stream` block. The `stream` `server` block containing the `zone_sync_ssl` directive should specify the `ssl` parameter with the `listen` directive for the TCP server. `ssl_certificate` and `ssl_certificate_key` directives can reference a Key Vault certificate attached to the deployment. ```nginx stream { resolver 127.0.0.1:49153 valid=20s; server { listen 9000 ssl; ssl_certificate /opt/ssl/server.crt; ssl_certificate_key /opt/ssl/server.key; zone_sync; zone_sync_server internal.nginxaas.nginx.com:9000 resolve; zone_sync_ssl on; } } ``` ## Enable verification of certificate of another cluster instance of the NGINXaaS for Azure deployment To enable verification of the cluster instance certificate edit the NGINXaaS deployment's NGINX configuration to enable the `zone_sync_ssl_verify` directive along with `zone_sync` directive in the top-level `stream` block and provide the `zone_sync_ssl_trusted_certificate` directive. `zone_sync_ssl_trusted_certificate` directive can reference a Key Vault certificate attached to the deployment. If the `zone_sync_ssl_name` directive is used, the `name` parameter should be set to `internal.nginxaas.nginx.com`. ```nginx stream { resolver 127.0.0.1:49153 valid=20s; server { listen 9000 ssl; ssl_certificate /opt/ssl/server.crt; ssl_certificate_key /opt/ssl/server.key; zone_sync; zone_sync_server internal.nginxaas.nginx.com:9000 resolve; zone_sync_ssl on; zone_sync_ssl_verify on; zone_sync_ssl_trusted_certificate /opt/ssl/server_ca.pem; } } ``` ## Set up certificate-based authentication across cluster instances of the NGINXaaS for Azure deployment To set up certificate-based authentication across the cluster instances edit the NGINXaaS deployment's NGINX configuration. Enable the `ssl_verify_client` directive and the `zone_sync` directive in the top-level `stream` block. In addition, provide the `ssl_client_certificate` directive. `zone_sync_ssl_certificate`, `zone_sync_ssl_certificate_key` and `ssl_client_certificate` directives can reference a Key Vault certificate attached to the deployment. ```nginx stream { resolver 127.0.0.1:49153 valid=20s; server { listen 9000 ssl; ssl_certificate /opt/ssl/zone_sync.crt; ssl_certificate_key /opt/ssl/zone_sync.key; ssl_verify_client on; ssl_client_certificate /opt/ssl/zone_sync_ca.pem; zone_sync; zone_sync_server internal.nginxaas.nginx.com:9000 resolve; zone_sync_ssl on; zone_sync_ssl_verify on; zone_sync_ssl_trusted_certificate /opt/ssl/zone_sync_ca.pem; zone_sync_ssl_certificate /opt/ssl/zone_sync.crt; zone_sync_ssl_certificate_key /opt/ssl/zone_sync.key; } } ``` Refer to [Runtime State Sharing](https://docs.nginx.com/nginx/admin-guide/high-availability/zone_sync/) for guidance on using other directives from the [Zone Synchronization module](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html) -------------------------------------------------------------------------------- title: "Scaling guidance" url: https://docs.nginx.com/nginxaas-azure/quickstart/scaling/index.md -------------------------------------------------------------------------------- F5 NGINXaaS for Azure (NGINXaaS) supports automatic scaling of your deployment to adapt to application traffic demands allowing you to control costs. When enabled, your NGINXaaS deployment will automatically scaled out to increase the capacity (and cost) or scaled in to decrease the capacity (and cost). Capacity is measured in [NGINX Capacity Units (NCU)](#nginx-capacity-unit-ncu). **Note:** This feature is only available for Standard plan(s). For the Developer plan, the capacity cannot be changed ## NGINX Capacity Unit (NCU) An NGINX Capacity Unit (NCU) quantifies the capacity of an NGINX instance based on the underlying compute resources. This abstraction allows you to specify the desired minimum capacity in NCUs without having to consider the regional hardware differences. An NGINX Capacity Unit consists of the following parameters: * CPU: an NCU provides 20 [Azure Compute Units](https://learn.microsoft.com/en-us/azure/virtual-machines/acu) (ACUs) * Bandwidth: an NCU provides 2.2 Mbps of network throughput * Concurrent connections: an NCU provides 3000 concurrent connections. This performance is not guaranteed when F5 WAF for NGINX is used with NGINXaaS ## Autoscaling With autoscaling enabled, the size of your NGINXaaS deployment will automatically adjust based on traffic requirements without the need to guess how many NCUs to provision. You must specify a minimum and maximum NCU count. NGINXaaS will maintain the size of the deployment ensuring the number of provisioned NCUs does not fall below the set minimum NCUs. Refer to the [Capacity Restrictions](#capacity-restrictions) when setting the minimum and maximum capacity. When creating a new NGINXaaS deployment with autoscaling enabled, the initial size of the deployment will match the minimum NCU count. To enable autoscaling using the Azure Portal, 1. Select **NGINXaaS scaling** in the left menu. 1. Select `Autoscale`. 1. Specify the minimum and maximum NCU count. 1. Select **Submit** to enable NGINXaaS deployment autoscaling. **Note:** For bandwidth-heavy workloads, NGINXaaS may automatically allocate NCUs beyond your configured autoscale maximum to satisfy demand, and you’ll be billed for any additional bandwidth consumed during the billing interval. For all other workloads, capacity is capped at the configured autoscale maximum ### Scaling rules NGINXaaS automatically adjusts the number of NCUs based on "scaling rules." A scaling rule defines when to scale, what direction to scale, and how much to scale. NGINXaaS will evaluate the following scaling rules, in order, based on the percentage capacity consumed metric and the provisioned NCU metric. - *Moderate Increase Rule*: Over the last 5 minutes, if the average capacity consumed is greater than or equal to 70% of the average provisioned NCUs, increase capacity by 20%. - *Urgent Increase Rule*: Over the last minute, if the capacity consumed is greater than or equal to 85% of the number of provisioned NCUs, increase capacity by 20%. - *Decrease Rule*: Over the last 10 minutes, if the average capacity consumed is less than or equal to 60% of the average provisioned NCUs, decrease capacity by 10%. To avoid creating a loop between scaling rules, NGINXaaS will not apply a scaling rule if it predicts that doing so would immediately trigger an opposing rule. For example, if the the "Urgent Increase Rule" is triggered due to a sudden spike in traffic, but the new capacity will cause the "Decrease Rule" to trigger immediately after, the autoscaler will not increase capacity. This prevents the deployment's capacity from increasing and decreasing erratically. ## Manual scaling We continue to support manual scaling for backwards compatibility with legacy plans. In manual mode, the NCU value you set will impact the capacity that we provision for your NGINXaaS deployment. If your workload exceeds this capacity, then we may or may not be able to support this additional load. We recommend using autoscaling to automatically adjust capacity based on traffic. To update the capacity of your deployment using the Azure Portal, 1. Select **NGINXaaS scaling** in the left menu. 1. Select `Manual`. 1. Set the desired number of NCUs. Scale increases in 10 NCU intervals (10, 20, 30, and so on). 1. Select **Submit** to update your deployment. **Note:** There's no downtime while an NGINXaaS deployment changes capacity. **Note:** When using manual scaling, consider setting up alerts on the `system.cpu` metric to be notified when scaling may be needed. See the [Metrics Catalog](/nginxaas-azure/monitoring/metrics-catalog.md) for more information on available metrics. **Note:** For bandwidth-intensive workloads, NGINXaaS may automatically allocate NCUs beyond your configured value to satisfy demand, and you’ll be billed for any additional bandwidth consumed during the billing interval. For all other workloads, capacity is capped at the configured value ## Capacity restrictions The following table outlines constraints on the specified capacity based on the chosen Marketplace plan, including the minimum capacity required for a deployment to be highly available, the maximum capacity, and what value the capacity must be a multiple of. By default, an NGINXaaS for Azure deployment will be created with the corresponding minimum capacity. | **Marketplace Plan** | **Minimum Capacity (NCUs)** | **Maximum Capacity (NCUs)** | **Multiple of** | |------------------------------|-----------------------------|-----------------------------|----------------------------| | Standard plan(s) | 10 | 500 | 10 | **Note:** If you need a higher maximum capacity, please [open a request](https://my.f5.com/manage/s/) and specify the Resource ID of your NGINXaaS deployment, the region, and the desired maximum capacity you wish to scale to. ## Connection processing methods restrictions - NGINXaaS only supports the `epoll` connection processing method when using the `use` directive, as NGINXaaS is based on Linux. ## Metrics NGINXaaS provides metrics for visibility of the current and historical capacity values. These metrics, in the `NGINXaaS Statistics` namespace, include: - NCUs Requested: `ncu.requested` -- how many NCUs have been requested using the API. This is the goal state of the system at that point in time. - NCUs Provisioned: `ncu.provisioned` -- how many NCUs have been successfully provisioned by the service. - This is the basis for [billing](/nginxaas-azure/billing/overview.md). - This may differ from `ncu.requested` temporarily during scale-out/scale-in events or during automatic remediation for a hardware failure. - Capacity Percentage: `nginxaas.capacity.percentage` -- the percentage of the current workload's total capacity that is being used. - If this is over 70%, consider scaling out; otherwise, requests may fail or take longer than expected. Alternatively, enable autoscaling, so your deployment can automatically scale based on the amount of capacity consumed. See the [Metrics Catalog](/nginxaas-azure/monitoring/metrics-catalog.md) for a reference of all metrics. **Note:** These metrics aren't visible unless enabled, see how to [Enable Metrics](/nginxaas-azure/monitoring/enable-metrics.md) for details. **Note:** The `ncu.consumed` metric is now deprecated and is on the path to retirement. Please change any alerting on this metric to use the new Capacity Percentage metric. ## Estimating how many NCUs to provision To calculate how many NCUs to provision, take the highest value across the parameters that make up an NCU: - CPU - Bandwidth - Concurrent connections Example 1: "I need to support 2,000 concurrent connections but only 4 Mbps of traffic. I need 52 ACUs." You would need `Max(52/20, 4/60, 2000/400)` = `Max(2.6, 0.07, 5)` = At least 5 NCUs. Example 2: "I don't know any of these yet!" Either start with the minimum and [adjust capacity](#adjusting-capacity) with the [iterative approach](#iterative-approach) described below, or [enable autoscaling](#autoscaling). In addition to the maximum capacity needed, we recommend adding a 10% to 20% buffer of additional capacity to account for unexpected spikes in traffic. Monitor the [Percentage Capacity Metric](#metrics) over time to determine your peak usage levels and adjust your requested capacity accordingly. ### Iterative approach 1. Make an estimate by either: - using the [Usage and Cost Estimator](/nginxaas-azure/billing/usage-and-cost-estimator.md) - compare to a [reference workload](#reference-workloads) 2. Observe the `nginxaas.capacity.percentage` [metric](#metrics) in Azure Monitor of your workload 3. Decide what headroom factor you wish to have 4. Multiply the headroom factor by the provisioned NCUs to get the target NCUs. 5. [Adjust capacity](#adjusting-capacity) to the target NCUs 6. repeat from step 2 -- it is always good to check back after making a change *Example*: 1. I am really unsure what size I needed so I just specified the default capacity, `20NCUs`. 2. I observe that my `nginxaas.capacity.percentage` is currently at `90%`. 3. This is early morning, traffic. I think midday traffic could be 3x what it is now. 4. `90% * 3 = 270%. 2.7 * 20 NCUs = 54 NCUs` 54 NCUs is my target capacity. 5. I can see that I need to scale by multiples of 10 so I'm going to scale out to `60NCUs`. 6. At midday I can see that I overestimated the traffic I would be getting and it was still a busy day. We peaked at `68%` of capacity, let me scale in to `50NCUs` to match the workload. ### Reference workloads These reference workloads were all measured with a simplistic NGINX config proxying requests to an upstream. Keepalive between NGINX and upstream is enabled. Minimal request matching or manipulation is done. | **TLS?** | **Conn/s** | **Req/s** | **Response Size** | **Throughput** | **NCU** | |:--------:|-----------:|----------:|------------------:|---------------:|--------:| | no | 12830 | 13430 | 0KB | 23Mbps | 18.8 | | no | 12080 | 13046 | 1KB | 125Mbps | 19 | | no | 12215 | 12215 | 10KB | 953Mbps | 21 | | no | 1960 | 1690 | 100KB | 1295Mbps | 23.6 | -------------------------------------------------------------------------------- title: "Support" url: https://docs.nginx.com/nginxaas-azure/get-help/support/index.md -------------------------------------------------------------------------------- ## Contact NGINX support To contact support about F5 NGINXaaS for Azure (NGINXaaS): 1. Go to your NGINXaaS deployment. In the left menu, select **Help**, then select **New support request** > **Raise a support ticket**. You will be redirected to **MyF5** to create a new case. Alternatively, you can directly go to the [MyF5 portal](https://my.f5.com) and log in with your F5 account. 1. Go to the **Support** section and select **Create a Case**. 1. Fill in the **Subject** and **Description** fields to describe the issue, and then select **Next**. Make sure the **Description** field is completed with the relevant information about your issue, bug report, or feedback. If you are contacting us to report an issue, please include the following information so that the support team can begin their investigation: - Location - Resource ID - Date and time of the issue The **Location** and **Resource ID** values are available in the **Properties** section of your deployment: ![Screenshot of the Azure portal showing the Properties section](nginxaas-azure/properties.png) If your deployment is configured to use F5 WAF for NGINX, please also collect the following information: - Package versions from the F5 WAF for NGINX page - Security policies in use and the content of all custom security policies 1. Select **NGINX Products** as the product family and **NGINXaaS** as the product module, and then select **Next**. 1. Enter the following required fields, at a minimum, and then select **Next**: - **Reason for contact** - **Subscription ID** (Select **No subscription available** or your F5 subscription ID if you have a private offer.) - **Cloud Provider** - **Assign a priority** 1. Complete the fields in the **Contact Information** section for your case and then select **Submit**. ## Update support contact information To provide or update the preferred support contact email: 1. Go to your NGINXaaS for Azure deployment. 2. Select **New Support request** in the left menu. 3. Select the **Edit** button next to the **Support Contact Email** field. 4. Provide a new email address in the field and select **Submit**. -------------------------------------------------------------------------------- title: "Upload using the Azure portal" url: https://docs.nginx.com/nginxaas-azure/getting-started/nginx-configuration/nginx-configuration-portal/index.md -------------------------------------------------------------------------------- An NGINX configuration can be applied to the deployment using the Azure portal in two different ways: - Create a new NGINX configuration from scratch or by pasting it in the Azure portal editor. - Upload a gzip compressed tar archive containing your NGINX configuration. As part of applying your NGINX configuration, the service validates the configuration for syntax and compatibility with F5 NGINXaaS for Azure (NGINXaaS). The use of certain directives and parameters is not allowed to ensure the NGINX configuration’s compatibility with IaaS deployment model in Azure. Validation errors are reported in the editor for you to correct. For more information, check the [NGINX Configuration Validation](/nginxaas-azure/getting-started/nginx-configuration.md#nginx-configuration-validation) section. **Note:** NGINXaaS supports Layer 7 HTTP loadbalancing. To configure .com and .net servers, refer to the server name in the server block within the HTTP context. To learn more, and see examples, follow the instructions in the [NGINX configuration validtion](/nginxaas-azure/getting-started/nginx-configuration/nginx-configuration-portal.md#nginx-configuration-validation) section. ## Prerequisites - If the NGINX configuration requires SSL/TLS certificates, then a managed identity and integration with Azure Key Vault is required. - A contributor role is required to apply the configuration to the deployment. ## Add an NGINX configuration 1. Go to your NGINXaaS for Azure deployment. 1. Select **NGINX configuration** in the left menu and you will see the default configuration that NGINXaaS provides. **Note:** If you don't see the default configuration, it's likely the deployment was created through a client tool other than the portal (For example, Terraform), or the "Apply default NGINX configuration" was unchecked during the deployment creation process in the portal. You can still proceed with the steps below to provide your own NGINX configuration for the deployment. 1. Select [icon: plus]**New File** to add a file path, then **Confirm**. | Property | Description | | -------- | ----------- | | File path | Each NGINX configuration file can be uniquely identified by a file path (for example, nginx.conf or /etc/nginx/nginx.conf) to align with the intended NGINX configuration file structure. | | Root file | The root file is the main NGINX configuration file.
  • The first file created will be the root file by default. You can designate a different root file if you have more than a single configuration file in your deployment.
  • The root file is designated with a icon on the portal.
| | Protected File | Indicates that the file may contain sensitive data such as passwords or represent an ssl/tls certificate.
  • To protect a file, enable the **Protected** [icon: toggle-right] toggle button.
  • You cannot access the file contents of a protected file saved to the NGINX configuration, but you can view its metadata, such as the SHA-256 hash of the file contents.
  • You can provide new contents for an existing protected file using the **Overwrite** link or resubmit it without having to provide the file contents again.
  • To modify the file path of a protected file or convert it to a regular file, delete the original file and create a new one.
  • A protected file is designated with a [icon: lock] icon on the portal.
| **Note:** If specifying an absolute file path, see the [NGINX Filesystem Restrictions table](/nginxaas-azure/getting-started/nginx-configuration/overview/#nginx-filesystem-restrictions) for the allowed directories the file can be written to. 1. Provide your NGINX configuration in the configuration file. 1. Files like SSL/TLS certificates can be added as well. However, we reccommend using Azure Key Vault to store your certificates. See [Add SSL/TLS certificates](/nginxaas-azure/getting-started/ssl-tls-certificates/overview.md#add-ssltls-certificates) for more information. 1. Select **Submit** to apply the new configuration. ### NGINX configuration validation NGINX configuration is validated real-time to check for syntax and compatibility with the service. Validation errors are reported in the editor for you to correct. For example, if you create/update an NGINX config with a particular directive that is not allowed, the service will analyze your NGINX config and provide real-time feedback. ![NGINX Configuration validation error](nginxaas-azure/validation-error.png) The editing experience consists of a single view for both editing and validation - If the config is invalid, then any errors are highlighted in-place in the config editor. Hover over the highlighted errors to learn more about them or check the problems section at the bottom. Corrections can be made in the same panel. - If the config is valid, then a green check mark appears next to NGINXaaS Analyzer at the bottom indicating that you can submit the config to deploy it. ## Upload a GZIP NGINX configuration Given the example directory structure, ```shell $ tree nginx nginx ├── nginx.conf ├── njs.js └── servers ├── server1.conf └── server2.conf 1 directory, 4 files ``` create a gzipped archive using the `-C` option to remove the top-level directory from the archive: ```shell $ tar -czf nginx.tar.gz -C nginx . $ tar -tzf nginx.tar.gz ./ ./nginx.conf ./njs.js ./servers/ ./servers/server1.conf ./servers/server2.conf ``` **Note:** Use the `-C` option with `tar` to change into the configuration directory before archiving. This strips the top-level directory from the archive paths and prevents path duplication errors during upload. `nginx.tar.gz` can be uploaded using the following portal workflow. Before continuing, ensure the file paths in the archive match the includes in the NGINX config. For example, ```nginx http { include servers/server1.conf; js_import njs.js; # ... } ``` 1. Go to your NGINXaaS for Azure deployment. 1. Select **NGINX configuration** from the left menu. 1. Select **Upload config package**. 1. Drag and drop or browse for the new gzip compressed archive file to upload. 1. Specify the root file. **Note:** Uploading a new file will replace all existing NGINX configuration files in your deployment. You must acknowledge this step before you proceed to upload. 1. Select **Upload**. ## Update an NGINX configuration 1. Go to your NGINXaaS for Azure deployment. 1. Select **NGINX configuration** in the left menu. 1. Select the configuration file you want to update from the File path list. 1. Make the necessary updates to the configuration. - You can also update the file path and/or assign the file as root. 1. (Optional) Select any other configuration files to make additional updates. 1. Submit your changes. ## Delete NGINX configuration Files 1. Go to your NGINXaaS for Azure deployment. 1. Select **NGINX configuration** in the left menu. 1. Select the configuration file you want to delete from the File path list. 1. Select the delete icon [icon: trash-2]. 1. Confirm your action to delete the configuration file. **Note:** Only non-root configuration files can be deleted. **Note:** See the [NGINX connfiguration overview](overview.md) topic to learn more about: - [NGINX configuration automation workflows](overview.md#nginx-configuration-automation-workflows) - [NGINX filesystem restrictions](overview.md#nginx-filesystem-restrictions) - [Disallowed configuration directives](overview.md#disallowed-configuration-directives) - [Directives that cannot be overridden](overview.md#directives-that-cannot-be-overridden) - [Configuration directives list](overview.md#configuration-directives-list) -------------------------------------------------------------------------------- title: "Upgrade channels" url: https://docs.nginx.com/nginxaas-azure/quickstart/upgrade-channels/index.md -------------------------------------------------------------------------------- ## Overview Maintaining the latest version NGINX Plus, operating system (OS), and other software dependencies is a key feature offered by F5 NGINXaaS for Azure (NGINXaaS). The **Upgrade Channel** is an upgrade path to which you can subscribe your NGINXaaS deployment to control the timing of software upgrades. The following channels are available: | Channel | Description | |-------------|---------------------------| | preview | Selecting this channel automatically upgrades your deployment to the latest supported version of NGINX Plus and its dependencies soon after they become available. We recommend using this setting to try out new capabilities in deployments running in your development, testing, and staging environments. Do not use the **Preview** channel in your production environment. | | stable | A deployment running on this channel will receive updates on NGINX Plus and its dependencies at a slower rate than the **Preview** channel. We recommend using this setting for production deployments where you might want stable features instead of the latest ones. This is the **default channel** if you do not specify one for your deployment. | **Note:** All channels will receive continuous updates related to OS patches, and security fixes. ## Availability of new features ### NGINX Plus and related modules | Channel | Availablity of NGINX Plus and related modules | |-------------|-----------------------------------------------| | preview | No sooner than 14 days of a new NGINX Plus [release](https://docs.nginx.com/nginx/releases/). | | stable | No sooner than 24 days of a new NGINX Plus [release](https://docs.nginx.com/nginx/releases/). | A new version of NGINX Plus and its related modules is first introduced to the **preview** channel, where it is goes through our acceptance testing. Once we have baked the software in the **preview** channel for a reasonable time, it is eventually graduated to the **stable** channel. The actual promotion timelines can vary, and you can view our [Changelog](/nginxaas-azure/changelog/changelog.md) for latest updates. ## Changing the upgrade channel To change the upgrade channel on your deployment using the Azure Portal: 1. Select **NGINX Upgrades** in the left menu. 1. Choose the desired **Upgrade Channel** from the dropdown menu. 1. Click **Submit**. -------------------------------------------------------------------------------- title: "Add certificates using the Azure CLI" url: https://docs.nginx.com/nginxaas-azure/getting-started/ssl-tls-certificates/ssl-tls-certificates-azure-cli/index.md -------------------------------------------------------------------------------- You can use Azure Key Vault (AKV) to store SSL/TLS certificates and keys to use in your F5 NGINXaaS for Azure (NGINXaaS) configuration. ### Prerequisites NGINXaaS natively integrates with [Azure Key Vault (AKV)](https://azure.microsoft.com/en-us/products/key-vault), so you can bring your own certificates and manage them in a centralized location. You will need: - AKV to store certificates that you want to add to the deployment. - A user or system assigned identity associated with your NGINXaaS deployment. Ensure that your Managed Identity (MI) has read access to secrets stored in AKV: - If using Azure RBAC for AKV, ensure that your MI has [Key Vault Secrets User](https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles#key-vault-secrets-user) or higher permissions. - If using Access Policies for AKV, ensure that your MI has *GET secrets* or higher permissions. - Access to AKV through a public or private endpoint. If public access to AKV needs to be restricted, refer to [Restrict Public Access to Key Vault](/nginxaas-azure/quickstart/security-controls/certificates.md#restrict-public-access-to-key-vault). - In addition to the MI permissions, if using the Azure portal to manage certificates, ensure that you have read access to list certificates inside the Key Vault: - If using Azure RBAC for AKV, ensure that you have [Key Vault Reader](https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles#key-vault-reader) or higher permissions. - If using Access Policies for AKV, ensure that you have *LIST certificates* or higher permissions. - If public access is disabled on your key vault, add an inbound access rule to allow your client IP address. - If you're unfamiliar with Azure Key Vault, check out the [Azure Key Vault concepts](https://docs.microsoft.com/en-us/azure/key-vault/general/basic-concepts) documentation from Microsoft. - Install [Azure CLI with NGINXaaS extension](/nginxaas-azure/client-tools/cli.md) ## Create a certificate Create a certificate under a deployment. This references an existing certificate in an Azure Key Vault and makes it available to NGINX configuration To create a certificate, use the `az nginx deployment certificate create` command: ```shell az nginx deployment certificate create --certificate-name --deployment-name --resource-group [--certificate-path] [--key-path] [--key-vault-secret-id] [--location] [--no-wait {0, 1, f, false, n, no, t, true, y, yes}] ``` ### Example - Create a certificate with a certificate path, key path, and key vault secret ID: ```shell az nginx deployment certificate create --certificate-name myCertificate \ --deployment-name myDeployment --resource-group myResourceGroup \ --certificate-path /etc/nginx/test.cert --key-path /etc/nginx/test.key \ --key-vault-secret-id keyVaultSecretId ``` **Note:** The `--key-vault-secret-id` must be the **Secret Identifier**, not the Certificate Identifier. To find the Secret Identifier, see [Finding the Azure Key Vault Secret Identifier](/nginxaas-azure/getting-started/ssl-tls-certificates/ssl-tls-certificates-portal/#finding-the-azure-key-vault-secret-identifier). See [Azure CLI Certificate Create Documentation](https://learn.microsoft.com/en-us/cli/azure/nginx/deployment/certificate#az-nginx-deployment-certificate-create) for more details on the available parameters. ## Update a certificate To update a certificate, use the `az nginx deployment certificate update` command: ```shell az nginx deployment certificate update [--add] [--certificate-name] [--certificate-path] [--deployment-name] [--force-string {0, 1, f, false, n, no, t, true, y, yes}] [--ids] [--key-path] [--key-vault-secret-id] [--location] [--no-wait {0, 1, f, false, n, no, t, true, y, yes}] [--remove] [--resource-group] [--set] [--subscription] ``` ### Example - Update the certificate virtual path, key virtual path and certificate: ```shell az nginx deployment certificate update --certificate-name myCertificate \ --deployment-name myDeployment --resource-group myResourceGroup \ --certificate-path /etc/nginx/testupdated.cert \ --key-path /etc/nginx/testupdated.key \ --key-vault-secret-id newKeyVaultSecretId ``` See [Azure CLI Certificate Create Documentation](https://learn.microsoft.com/en-us/cli/azure/nginx/deployment/certificate#az-nginx-deployment-certificate-update) for more details on the available parameters. ## Delete a certificate To delete a certificate, use the `az nginx deployment certificate delete` command: ```shell az nginx deployment certificate delete [--certificate-name] [--deployment-name] [--ids] [--no-wait {0, 1, f, false, n, no, t, true, y, yes}] [--resource-group] [--subscription] [--yes] ``` ### Example - Delete a certificate: ```shell az nginx deployment certificate delete --certificate-name myCertificate \ --deployment-name myDeployment --resource-group myResourceGroup ``` See [Azure CLI Certificate Delete Documentation](https://learn.microsoft.com/en-us/cli/azure/nginx/deployment/certificate#az-nginx-deployment-certificate-delete) for more details on the available parameters. -------------------------------------------------------------------------------- title: "Change from developer/basic plan to standard plan" url: https://docs.nginx.com/nginxaas-azure/change-plan/basic-to-standard/index.md -------------------------------------------------------------------------------- The developer/basic plan is designed for early-stage trials and testing and is not intended for production use. If you are ready to create a standard plan deployment and wish to preserve the configuration of an existing developer/basic plan deployment, you can [create a new deployment](/nginxaas-azure/getting-started/create-deployment.md), selecting the latest standard pricing plan, and manually reapply your NGINX configuration and certificates. You can also follow the instructions below to recreate your deployment using an Azure Resource Manager (ARM) template. ## Prerequisites - [Azure CLI Installation](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli) - You must be logged in to your Azure account through the CLI. See [Azure CLI Authentication](https://learn.microsoft.com/en-us/cli/azure/authenticate-azure-cli) ## Recreate deployment using ARM template To export an ARM template for an existing deployment: 1. Go to your existing NGINXaaS deployment. 1. Select **Export template** under **Automation** in the left menu. 1. Wait for the template to generate. 1. Select **Download**. 1. Decompress the template archive. 1. Open the `template.json` file and verify that the data in the template is correct. 1. In the `resources` section, change `sku.name` to `standardv3_Monthly`. This recreates the deployment as a standard plan deployment. 1. Delete the original developer/basic plan deployment. 1. On the command line, run: ```shell az deployment group create \ --subscription= \ --resource-group= \ --template-file= ``` ## Further reading For further details on recreating a deployment, see our [guide](/nginxaas-azure/quickstart/recreate.md). -------------------------------------------------------------------------------- title: "Connectivity test tool" url: https://docs.nginx.com/nginxaas-azure/get-help/connectivity/index.md -------------------------------------------------------------------------------- Use the connectivity test tool to determine whether a specific IP address is accessible from your deployment's dataplane. The connectivity test tool accepts an IP address and a port number. It uses [`netcat`](https://nc110.sourceforge.io/) to open a TCP connection with the given address, without sending any data to the address. The tool returns `netcat`'s output to the user. This is useful for debugging connectivity issues and determining if a problem is in NGINX configuration or Azure network configuration. To use the tool: - Retrieve your [data plane API endpoint](/nginxaas-azure/loadbalancer-kubernetes.md#nginxaas-data-plane-api-endpoint). - Create an [API key](/nginxaas-azure/loadbalancer-kubernetes.md#create-an-nginxaas-data-plane-api-key) if you do not already have one. - Append the `/connectivity` suffix to your deployment's data plane API endpoint, e.g. https://my-deployment.my-region.nginxaas.net/connectivity. Use a browser to navigate to this URL. - The browser will prompt you for a username and password. The username is optional. Please enter your API key in the password field. - You will then be able to use the connectivity tool through the browser. **Note:** The connectivity test tool will not accept loopback or multicast IP addresses. -------------------------------------------------------------------------------- title: "Deploy using the Azure CLI" url: https://docs.nginx.com/nginxaas-azure/getting-started/create-deployment/deploy-azure-cli/index.md -------------------------------------------------------------------------------- ## Overview The Azure CLI has an extension to be used for management of F5 NGINXaaS for Azure (NGINXaaS) deployments whether that be locally or in continuous integration pipelines. This document links you to information around basic NGINXaaS extension usage. ## Prerequisites - Install [Azure CLI with NGINXaaS extension](/nginxaas-azure/client-tools/cli.md) ## Create a deployment To create an NGINXaaS for Azure resource use the `az nginx deployment create` command: ```shell az nginx deployment create --deployment-name --resource-group [--auto-upgrade-profile] [--enable-diagnostics {0, 1, f, false, n, no, t, true, y, yes}] [--identity] [--location] [--logging] [--network-profile] [--no-wait {0, 1, f, false, n, no, t, true, y, yes}] [--scaling-properties] [--sku] [--tags] [--user-profile] ``` ### Examples - Create a deployment with public IP: ```shell az nginx deployment create --name myDeployment --resource-group \ myResourceGroup --location eastus2 --sku name="standardv3_Monthly" \ --identity type="SystemAssigned" \ --network-profile front-end-ip-configuration="{public-ip-addresses:[{id:/subscriptions/mySubscriptionID/resourceGroups/myResourceGroup/providers/Microsoft.Network/publicIPAddresses/myPublicIP}]}" \ network-interface-configuration="{subnet-id:/subscriptions/mySubscriptionID/resourceGroups/myResourceGroup/providers/Microsoft.Network/virtualNetworks/myVNet/subnets/mySubnet}" ``` - Create a deployment with private IP: ```shell az nginx deployment create --name myDeployment --resource-group \ myResourceGroup --location eastus2 --sku \ name="standardv3_Monthly" \ --identity type="SystemAssigned" \ --network-profile \ front-end-ip-configuration="{private-ip-addresses:[{private-ip-allocation-method:Static,subnet-id:/subscriptions/mySubscription/resourceGroups/myResourceGroup/providers/Microsoft.Network/virtualNetworks/myVNet/subnets/mySubnet,private-ip-address:10.0.0.2}]}" \ network-interface-configuration="{subnet-id:/subscriptions/mySubscriptionID/resourceGroups/myResourceGroup/providers/Microsoft.Network/virtualNetworks/myVNet/subnets/mySubnet}" ``` ```shell az nginx deployment create --name myDeployment --resource-group \ myResourceGroup --location eastus2 --sku \ name="standardv3_Monthly" \ --identity type="SystemAssigned" \ --network-profile \ front-end-ip-configuration="{private-ip-addresses:[{private-ip-allocation-method:Dynamic,subnet-id:/subscriptions/mySubscription/resourceGroups/myResourceGroup/providers/Microsoft.Network/virtualNetworks/myVNet/subnets/mySubnet,private-ip-address:10.0.0.2}]}" \ network-interface-configuration="{subnet-id:/subscriptions/mySubscriptionID/resourceGroups/myResourceGroup/providers/Microsoft.Network/virtualNetworks/myVNet/subnets/mySubnet}" ``` - Create a dual-stack (IPv4 + IPv6) NGINXaaS deployment with public IPs: ```shell az nginx deployment create --name myDeployment --resource-group \ myResourceGroup --location eastus2 --sku name="standardv3_Monthly" \ --identity type="SystemAssigned" \ --network-profile front-end-ip-configuration="{public-ip-addresses:[{id:/subscriptions/mySubscription/resourceGroups/myResourceGroup/providers/Microsoft.Network/publicIPAddresses/pubIPv4},{id:/subscriptions/mySubscription/resourceGroups/myResourceGroup/providers/Microsoft.Network/publicIPAddresses/pubIPv6}]}" \ network-interface-configuration="{subnet-id:/subscriptions/mySubscriptionID/resourceGroups/myResourceGroup/providers/Microsoft.Network/virtualNetworks/myVNet/subnets/mySubnet}" ``` - Create a dual-stack (IPv4 + IPv6) NGINXaaS deployment with private IPs: ```shell az nginx deployment create --name myDeployment --resource-group \ myResourceGroup --location eastus2 --sku \ name="standardv3_Monthly" \ --identity type="SystemAssigned" \ --network-profile \ front-end-ip-configuration="{private-ip-addresses:[{private-ip-allocation-method:Static,subnet-id:/subscriptions/mySubscription/resourceGroups/myResourceGroup/providers/Microsoft.Network/virtualNetworks/myVNet/subnets/mySubnet,private-ip-address:10.0.0.2},{private-ip-allocation-method:Static,subnet-id:/subscriptions/mySubscription/resourceGroups/myResourceGroup/providers/Microsoft.Network/virtualNetworks/myVNet/subnets/mySubnet,private-ip-address:2001:0db8:85a3:0000:0000:8a2e:0370:7334}]}" \ network-interface-configuration="{subnet-id:/subscriptions/mySubscriptionID/resourceGroups/myResourceGroup/providers/Microsoft.Network/virtualNetworks/myVNet/subnets/mySubnet}" ``` - Create a deployment with managed identity, storage account and scaling: ```shell az nginx deployment create --deployment-name myDeployment --resource-group \ myResourceGroup --location eastus2 --sku name=standardv3_Monthly \ --network-profile \ network-interface-configuration='{subnet-id:/subscriptions/subscriptionId/resourcegroups/myResourceGroup/providers/Microsoft.Network/virtualNetworks/vnet-azclitest/subnets/mySubnet}' \ front-end-ip-configuration='{public-ip-addresses:[{id:/subscriptions/subscriptionId/resourceGroups/myResourceGroup/providers/Microsoft.Network/publicIPAddresses/myPublicIP}]}' \ --identity '{"type":"SystemAssigned,UserAssigned","userAssignedIdentities":{"/subscriptions/subscriptionId/resourcegroups/myResourceGroup/providers/Microsoft.ManagedIdentity/userAssignedIdentities/myManagedIdentity":{}}}' \ --logging storage-account='{"account-name":"myStorageAccount","container-name":"myContainer"}' \ --scaling-properties capacity=10 ``` See the [Azure CLI Deployment Create Documentation](https://learn.microsoft.com/en-us/cli/azure/nginx/deployment#az-nginx-deployment-create) for more details on the required and optional parameters. ## Update a deployment To update an NGINXaaS for Azure resource use the `az nginx deployment update` command: ```shell az nginx deployment update [--add] [--auto-upgrade-profile] [--deployment-name] [--enable-diagnostics {0, 1, f, false, n, no, t, true, y, yes}] [--force-string {0, 1, f, false, n, no, t, true, y, yes}] [--identity] [--ids] [--location] [--logging] [--network-profile] [--no-wait {0, 1, f, false, n, no, t, true, y, yes}] [--remove] [--resource-group] [--scaling-properties] [--set] [--sku] [--subscription] [--tags] [--user-profile] ``` ### Example - Update tags and enable diagnostics support for a deployment: ```shell az nginx deployment update --name myDeployment --resource-group \ myResourceGroup --location eastus2 --tags tag1="value1" \ tag2="value2" --enable-diagnostics ``` Update an NGINXaaS deployment to a dual-stack (IPv4 + IPv6) network configuration with public IPs: ```shell az nginx deployment update --name myDeployment --resource-group myResourceGroup \ --network-profile front-end-ip-configuration="{public-ip-addresses:[{id:/subscriptions/mySubscriptionID/resourceGroups/myResourceGroup/providers/Microsoft.Network/publicIPAddresses/pubIPv4},{id:/subscriptions/mySubscriptionID/resourceGroups/myResourceGroup/providers/Microsoft.Network/publicIPAddresses/pubIPv6}]}" \ network-interface-configuration="{subnet-id:/subscriptions/mySubscriptionID/resourceGroups/myResourceGroup/providers/Microsoft.Network/virtualNetworks/myVNet/subnets/mySubnet}" ``` See the [Azure CLI Deployment Update Documentation](https://learn.microsoft.com/en-us/cli/azure/nginx/deployment#az-nginx-deployment-update) for more details on the required and optional parameters. ## Delete a deployment Use the `az nginx deployment delete` command to delete an NGINXaaS for Azure resource: ```shell az nginx deployment delete [--name] [--ids] [--no-wait {0, 1, f, false, n, no, t, true, y, yes}] [--resource-group] [--subscription] [--yes] ``` ### Example - Delete a deployment: ```shell az nginx deployment delete --name myDeployment \ --resource-group myResourceGroup ``` See the [Azure CLI Deployment Delete Documentation](https://learn.microsoft.com/en-us/cli/azure/nginx/deployment#az-nginx-deployment-delete) for more details on the required and optional parameters. ## Additional resources - [Azure CLI Public IP Documentation](https://learn.microsoft.com/en-us/cli/azure/network/public-ip) - [Azure CLI Storage Container Documentation](https://learn.microsoft.com/en-us/cli/azure/storage/container) -------------------------------------------------------------------------------- title: "Enable content caching" url: https://docs.nginx.com/nginxaas-azure/quickstart/basic-caching/index.md -------------------------------------------------------------------------------- F5 NGINXaaS for Azure (NGINXaaS) supports caching using the [ngx_http_proxy_module](https://nginx.org/en/docs/http/ngx_http_proxy_module.html) module, improving performance by allowing content to be served from cache without having to contact upstream servers. For more information on caching with NGINX, see [NGINX Content Caching](https://docs.nginx.com/nginx/admin-guide/content-cache/content-caching/). ## Configuring caching ```nginx http { # ... proxy_cache_path /var/cache/nginx keys_zone=mycache:10m; } ``` NGINXaaS for Azure only supports caching to `/var/cache/nginx`. This is because data at `/var/cache/nginx` will be stored in a separate [Temporary Disk](https://docs.microsoft.com/en-us/azure/virtual-machines/managed-disks-overview#temporary-disk). The size of the temporary disk is 4GB. ## Limitations Currently, `proxy_cache_purge` might not work as expected because NGINXaaS [deploys multiple instances of NGINX Plus](/nginxaas-azure/overview/overview.md#architecture) for high availability. The `PURGE` request will be routed to a single instance, and only the matched values on that instance will be purged. -------------------------------------------------------------------------------- title: "Enable F5 WAF for NGINX" url: https://docs.nginx.com/nginxaas-azure/app-protect/enable-waf/index.md -------------------------------------------------------------------------------- ## Overview This guide explains how to enable F5 WAF for NGINX on a F5 NGINX as a Service for Azure (NGINXaaS) deployment. [F5 WAF for NGINX](https://docs.nginx.com/nginx-app-protect-waf/v5) provides web application firewall (WAF) security protection for your web applications, including OWASP Top 10; response inspection; Meta characters check; HTTP protocol compliance; evasion techniques; disallowed file types; JSON & XML well-formedness; sensitive parameters & Data Guard. ## Before you start - F5 WAF for NGINX can only be enabled on NGINXaaS for Azure deployments with the **Standard v3** [plan](/nginxaas-azure/billing/overview.md/#standard-v3-plan) (and the deprecated **Standard v2** [plan](/nginxaas-azure/billing/overview.md/#standard-v2-plan-deprecated)) ## Enable F5 WAF for NGINX F5 WAF for NGINX is disabled by default and needs to be explicitly enabled on an NGINXaaS deployment. Follow these steps: ### Using the Microsoft Azure Portal Access the [Microsoft Azure portal](https://portal.azure.com) 1. Go to your NGINXaaS for Azure deployment. 2. Select F5 WAF for NGINX in the left menu. 3. Select **Enable F5 WAF for NGINX**. ## What's next [Configure F5 WAF for NGINX](/nginxaas-azure/app-protect/configure-waf.md) -------------------------------------------------------------------------------- title: "Enable metrics" url: https://docs.nginx.com/nginxaas-azure/monitoring/enable-metrics/index.md -------------------------------------------------------------------------------- Monitoring your application's performance is crucial for maintaining its reliability and efficiency. F5 NGINXaaS for Azure (NGINXaaS) seamlessly integrates with Azure Monitor, allowing you to collect, correlate, and analyze metrics for a thorough understanding of your application's health and behavior. Refer to the [Azure monitor overview](https://docs.microsoft.com/en-us/azure/azure-monitor/overview) documentation from Microsoft to learn more about Azure Monitor. ## Prerequisites - A system assigned managed identity. **Note:** The system-assigned managed identity does not need any role assignments to enable the metrics functionality described in this section. ## Collection Azure Monitor will collects metrics from the NGINXaaS deployment automatically if the prerequisites are met. No configuration is required. ## Exporting You can export Azure Monitor metrics to other destinations like Log Analytics workspace, Azure Storage Account, Azure Event Hubs or [Azure Monitor partner solutions](https://learn.microsoft.com/en-us/azure/partner-solutions/partners#observability-partners) using the diagnostic settings. For more information, see the [Metrics diagnostic settings](https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/diagnostic-settings) documentation from Microsoft. To configure diagnostic settings for a service, see [Create diagnostic settings in Azure Monitor](https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/create-diagnostic-settings). You can export metrics by selecting the `AllMetrics` category in diagnostic settings. **Note:** Not all metrics are exportable via diagnostic settings, for a list of exportable metrics, see [NGINXaaS exportable metrics](https://learn.microsoft.com/en-us/azure/azure-monitor/reference/supported-metrics/nginx-nginxplus-nginxdeployments-metrics). ## Cost and retention Azure Monitor platform metrics are ingested and stored free of charge, with a standard retention period of 93 days. Adding alerts, querying Azure Monitor using REST API or exporting metrics using Azure Monitor's diagnostic settings would incurs costs. For detailed pricing, you can refer to the [Azure Monitor pricing page](https://azure.microsoft.com/en-us/pricing/details/monitor/). ## Review metrics Once you have enabled monitoring, you can view the metrics using the Azure portal or the Azure Monitor API. **Note:** NGINX Agent periodically gathers connection and request statistics using an internal HTTP request. An Azure service health probe checks for status using a TCP connection for each listen port in the NGINX configuration, incrementing the connection count for each port. This contributes to minimal traffic and should not affect these metrics significantly, but you might see an unexpected number of connections and requests. **Note:** If some of your deployment's metrics are intermittently missing in Azure monitor, it may indicate that the underlying resources for your deployment are being exhausted. Monitor the `nginxaas.capacity.percentage` metric to see the deployment's resource utilization. If it's nearing 100%, consider increasing the deployment's NCU capacity. See the [Scaling Guidance](/nginxaas-azure/quickstart/scaling.md) documentation for more information. ### View metrics with Azure Monitor metrics explorer Access the [Microsoft Azure portal](https://portal.azure.com) 1. Go to your NGINXaaS for Azure deployment. 2. In the navigation pane under **Monitoring**, select the **Metrics** section to access the Azure Monitor metrics explorer for your NGINXaaS deployment. Refer to the [Azure Monitor metrics explorer](https://docs.microsoft.com/en-us/azure/azure-monitor/essentials/metrics-getting-started) documentation from Microsoft to learn how you can create queries. **Note:** Many of NGINX Plus's advanced statistics need to be enabled in the "nginx.conf" file before they will appear in the metrics explorer, for example "plus.http.request.bytes_*". Refer to [Gathering Data to Appear in Statistics](https://docs.nginx.com/nginx/admin-guide/monitoring/live-activity-monitoring/#gathering-data-to-appear-in-statistics) to learn more. ### Retrieve metrics through Azure Monitor API This section shows you how to effectively discover, gather and analyze NGINXaaS metrics through the Azure Monitor REST API. **Note:** Refer to [Authenticate Azure Monitor requests](https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/rest-api-walkthrough?tabs=portal#authenticate-azure-monitor-requests) for instructions on authenticating your API requests against the Azure Monitor API endpoint. 1. **Retrieve metric definitions:** Metrics definitions give you insights into the various metrics available for NGINXaaS within a namespace and what they represent. The following `curl` example shows how to retrieve all metrics definitions within the `nginx connections statistics` namespace for your NGINXaaS deployment: ```shell curl --request GET --header "Authorization: Bearer $TOKEN" "https://management.azure.com/subscriptions/12345678-abcd-98765432-abcdef012345/resourceGroups/my-nginx-rg/providers/NGINX.NGINXPLUS/nginxDeployments/my-nginx-dep/providers/microsoft.insights/metricDefinitions?api-version=2024-02-01" ``` The following JSON shows an example response body: ```json { "value": [ ... { "id": "/subscriptions/12345678-abcd-98765432-abcdef012345/resourceGroups/my-nginx-rg/providers/NGINX.NGINXPLUS/nginxDeployments/my-nginx-dep/providers/microsoft.insights/metricdefinitions/Nginx Connections Statistics/nginx.conn.current", "resourceId": "/subscriptions/12345678-abcd-98765432-abcdef012345/resourceGroups/my-nginx-rg/providers/NGINX.NGINXPLUS/nginxDeployments/my-nginx-deployment", "namespace": "NGINX.NGINXPLUS/nginxDeployments", "category":"nginx connections statistics", "name": { "value": "nginx.conn.current", "localizedValue": "Current connections" }, ... } ... ] } ``` 2. **Metric values:** You can obtain the actual metric values which represent real-time or historical data points that tell you how your NGINXaaS is performing. The following `curl` example shows how to retrieve the value of metric `nginx.conn.current` over a 10-minute time window averaged over 5 minute intervals: ```shell curl --request GET --header "Authorization: Bearer $TOKEN" "https://management.azure.com/subscriptions/12345678-abcd-98765432-abcdef012345/resourceGroups/my-nginx-rg/providers/NGINX.NGINXPLUS/nginxDeployments/my-nginx-dep/providers/microsoft.insights/metrics?metricnames=nginx.conn.current×pan=2025-03-27T20:00:00Z/2025-03-27T20:10:00Z&aggregation=Average&interval=PT5M&api-version=2024-02-01" ``` The following JSON shows an example response body: ```json { "cost": 9, "timespan": "2025-03-27T20:00:00Z/2025-03-27T20:10:00Z", "interval": "PT5M", "value": [ { "id": "/subscriptions/12345678-abcd-98765432-abcdef012345/resourceGroups/my-nginx-rg/providers/NGINX.NGINXPLUS/nginxDeployments/my-nginx-dep/providers/Microsoft.Insights/metrics/nginx.conn.current", "type": "Microsoft.Insights/metrics", "name": { "value": "nginx.conn.current", "localizedValue": "Current connections" }, "unit": "Count", "timeseries": [ { "metadatavalues": [], "data": [ { "timeStamp": "2025-03-27T20:00:00Z", "average": 4 }, { "timeStamp": "2025-03-27T20:05:00Z", "average": 4 } ] } ], "errorCode": "Success" } ], "namespace": "NGINX.NGINXPLUS/nginxDeployments", "resourceregion": "eastus2" } ``` **Note:** Refer to the [Metrics Catalog](/nginxaas-azure/monitoring/metrics-catalog.md) for a listing of available namespaces and metrics. -------------------------------------------------------------------------------- title: "Hosting static content locally" url: https://docs.nginx.com/nginxaas-azure/quickstart/hosting-static-content/index.md -------------------------------------------------------------------------------- F5 NGINXaaS for Azure (NGINXaaS) supports hosting static content locally on the deployment, which allows users to serve static websites directly from their deployment. ## Uploading static files as a tarball Follow the steps listed below to upload static content and relevant NGINX configuration using `tar`: 1. Create an `nginx.conf` to configure your deployment to serve static content. The following is an example NGINX configuration: ```nginx http { server { listen 80; location / { root /srv; index index.html; } } } ``` 2. Store your static files alongside the NGINX configuration. The following shows the structure of a directory containing an NGINX configuration and an `index.html` file that will be served from the deployment. ```shell test-static-files $ tree . . ├── nginx.conf └── srv └── index.html 2 directories, 2 files ``` **Note:** `index.html` is placed under the `srv` directory. When using `tar` to upload static content, the static content has to be placed under one of the allowed paths listed in the [NGINX Filesystem Restrictions table](/nginxaas-azure/getting-started/nginx-configuration/overview/#nginx-filesystem-restrictions). 3. Create the tarball. ```shell test-static-files $ tar -cvzf /test.tar.gz * ``` 4. Upload the tarball following instructions listed in the [NGINX configuration](/nginxaas-azure/getting-started/nginx-configuration/nginx-configuration-portal.md#upload-gzip-nginx-configuration) documentation. 5. After uploading the configuration, you should see the following files in your deployment: - `nginx.conf` - `srv/index.html` 6. Browse to the deployment IP address, and you will see `index.html` being served from the deployment. ## Uploading static files directly to the deployment You can also upload static files directly to the deployment. See [Adding NGINX Configuration](/nginxaas-azure/getting-started/nginx-configuration/nginx-configuration-portal.md#add-nginx-configuration) to upload individual files to your deployment. Refer to the [NGINX Filesystem Restrictions table](/nginxaas-azure/getting-started/nginx-configuration/overview/#nginx-filesystem-restrictions) to see where files can be written to and read from. ## Limitations NGINX Configuration payload larger than 3 MB is not supported. For hosting larger static content or to avoid the payload size limitation, consider [hosting static content in Azure Blob Storage](/nginxaas-azure/quickstart/hosting-static-content-blob-storage.md). -------------------------------------------------------------------------------- title: "Setting up JWT authentication" url: https://docs.nginx.com/nginxaas-azure/quickstart/security-controls/jwt/index.md -------------------------------------------------------------------------------- F5 NGINXaaS for Azure (NGINXaaS) provides the option to control access to your resources using JWT authentication. With JWT authentication, a client provides a JSON Web Token, and the token will be validated against a local key file or a remote service. This document will explain how to validate tokens using Microsoft Entra as the remote service. For more information on JWT authentication with NGINX Plus, please refer to [ngx_http_auth_jwt_module](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html) and [NGINX Plus Setting up JWT Authentication](https://docs.nginx.com/nginx/admin-guide/security-controls/configuring-jwt-authentication/). ## Prerequisites - Set up a tenant, see [Quickstart: Set up a tenant](https://learn.microsoft.com/en-us/entra/identity-platform/quickstart-create-new-tenant). - URL of the remote service (IdP). In this documentation, we are using Microsoft Entra Signing Keys URL: `https://login.microsoftonline.com/common/discovery/keys`. For more information, see [Signing key rollover in the Microsoft identity platform](https://learn.microsoft.com/en-us/entra/identity-platform/signing-key-rollover). ## Configuring NGINX for JWT authentication 1. To configure NGINX to use JWT for authentication, you will need to create a JWT that will be issued to a client. You can use your identity provider (IdP) or your own service to create JWTs. For testing purposes, you can create your own JWT, see [Get Microsoft Entra tokens for users by using MSAL](https://learn.microsoft.com/en-us/azure/databricks/dev-tools/auth/aad-token-manual#msal-method) for details. If you wish to use your own local JSON Web Key (JWK) file for authentication, please upload it along with the NGINX configuration. Remember to respect the instance's filesystem restrictions and specify a location for the key file within one of the allowed directories. For details on uploading the configuration and file system restrictions, see [Upload an NGINX Configuration](https://docs.nginx.com/nginxaas-azure/getting-started/nginx-configuration/). 2. Set up an NGINX `location` block that enables the JWT authentication and defines the authentication realm ("API" in the example) with the [auth_jwt](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt) directive. To verify the signature or decrypt the content of JWT, you will need to specify the JWT type using the [auth_jwt_type](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_type) directive, and provide the path to the corresponding JSON Web Key (JWK) file using the [auth_jwt_key_file](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_file) and/or [auth_jwt_key_request](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_request) directives. Specifying both directives simultaneously will allow you to specify more than one key source. If no directives are specified, JWS signature verification will be skipped. ```nginx server { listen 80; location / { auth_jwt "API"; auth_jwt_key_file /srv/key.jwk; auth_jwt_key_request /_jwks_uri; } location = /_jwks_uri { proxy_pass https://login.microsoftonline.com/common/discovery/keys; subrequest_output_buffer_size 12k; } } ``` **Note:** When using the common Microsoft Entra signing keys you will need to increase the size of the subrequest output buffer as the key file will not fit in the default buffer. If the buffer is not sized properly, requests will result in empty responses. If [error logging is enabled](/nginxaas-azure/monitoring/enable-logging/), you will see an error in the error log. Enabling JWT key caching is recommended to achieve optimal performance. This can be done with the [auth_jwt_key_cache](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_cache) directive. Note that caching of keys obtained from variables is not supported. If you are using Microsoft Entra as an identity provider for JWT authentication, please be aware that [keys are rotated frequently](https://learn.microsoft.com/en-us/entra/identity-platform/signing-key-rollover), and it is recommended to take that into consideration before using it as a static file or caching the response from the subrequest. The full example of getting JWT authentication from a subrequest: ```nginx http { upstream my_backend { server 10.0.0.1; server 10.0.0.2; } server { listen 80; location / { auth_jwt "API"; auth_jwt_key_file conf/key.jwk; auth_jwt_key_request /_jwks_uri; auth_jwt_key_cache 1h; proxy_pass http://my_backend; } location = /_jwks_uri { internal; proxy_method GET; proxy_pass https://login.microsoftonline.com/common/discovery/keys; subrequest_output_buffer_size 12k; } } } ``` To learn more about configuring JWT in more complex scenarios such as claims validation, see [Arbitrary JWT Claims Validation](https://docs.nginx.com/nginx/admin-guide/security-controls/configuring-jwt-authentication/#arbitrary-jwt-claims-validation). -------------------------------------------------------------------------------- title: "Upload using the Azure CLI" url: https://docs.nginx.com/nginxaas-azure/getting-started/nginx-configuration/nginx-configuration-azure-cli/index.md -------------------------------------------------------------------------------- ## Overview F5 NGINXaaS for Azure (NGINXaaS) configurations can be managed using the Azure CLI. This document outlines common Azure CLI workflows to validate, create, and update NGINX configurations. ## Prerequisites - Install [Azure CLI with NGINXaaS extension](/nginxaas-azure/client-tools/cli.md) - If the NGINX configuration requires SSL/TLS certificates, then a managed identity and integration with Azure Key Vault is required. - A contributor role is required to apply the configuration to the deployment. ## Create a configuration To create a new NGINX configuration, use the `az nginx deployment configuration create` command: There are two supported ways to upload your configuration: 1. **Plain file upload** — Upload one or more configuration files directly using the `--files` parameter. 1. **Tarball upload** — Package your configuration files into a `.tar.gz` archive and upload it using the `--package` parameter. Both methods are valid and can be used depending on how your configuration is structured. ```shell az nginx deployment configuration create --configuration-name --deployment-name --resource-group [--files] [--location] [--no-wait {0, 1, f, false, n, no, t, true, y, yes}] [--package] [--protected-files] [--root-file] ``` ### Validate your configuration You can use the `analyze` command to validate your configuration before submitting it to the deployment:
#### Validate plain files If you are uploading individual configuration files, you can pass them as base64-encoded content through the `--files` parameter: ```shell az nginx deployment configuration analyze --deployment-name $DEPLOYMENT_NAME \ --resource-group $RESOURCE_GROUP --root-file /etc/nginx/nginx.conf \ --name default --files "$FILES_CONTENT" ``` #### Validate a tarball package If you are using a `.tar.gz` archive, encode it and pass it through the `--package` parameter: ```shell TAR_DATA=$(base64 -i nginx.tar.gz) az nginx deployment configuration analyze --deployment-name myDeployment \ --resource-group myResourceGroup --root-file nginx.conf \ --name default --package data="$TAR_DATA" ```
### Upload a configuration #### Upload plain files The following examples show how to upload plain configuration files directly to your deployment: - Create a single file configuration: ```shell az nginx deployment configuration create --name default \ --deployment-name myDeployment --resource-group myResourceGroup \ --root-file /etc/nginx/nginx.conf \ --files "[{content:'aHR0cCB7CiAgICB1cHN0cmVhbSBhcHAgewogICAgICAgIHpvbmUgYXBw \ IDY0azsKICAgICAgICBsZWFzdF9jb25uOwogICAgICAgIHNlcnZlciAxMC4wLjEuNDo4 \ MDAwOwogICAgfQoKICAgIHNlcnZlciB7CiAgICAgICAgbGlzdGVuIDgwOwogICAgICAg \ IHNlcnZlcl9uYW1lICouZXhhbXBsZS5jb207CgogICAgICAgIGxvY2F0aW9uIC8gewog \ ICAgICAgICAgICBwcm94eV9zZXRfaGVhZGVyIEhvc3QgJGhvc3Q7CiAgICAgICAgICAg \ IHByb3h5X3NldF9oZWFkZXIgWC1SZWFsLUlQICRyZW1vdGVfYWRkcjsKICAgICAgICAg \ ICAgcHJveHlfc2V0X2hlYWRlciBYLVByb3h5LUFwcCBhcHA7CiAgICAgICAgICAgIHBy \ b3h5X3NldF9oZWFkZXIgR2l0aHViLVJ1bi1JZCAwMDAwMDA7CiAgICAgICAgICAgIHBy \ b3h5X2J1ZmZlcmluZyBvbjsKICAgICAgICAgICAgcHJveHlfYnVmZmVyX3NpemUgNGs7 \ CiAgICAgICAgICAgIHByb3h5X2J1ZmZlcnMgOCA4azsKICAgICAgICAgICAgcHJveHlf \ cmVhZF90aW1lb3V0IDYwczsKICAgICAgICAgICAgcHJveHlfcGFzcyBodHRwOi8vYXBw \ OwogICAgICAgICAgICBoZWFsdGhfY2hlY2s7CiAgICAgICAgfQogICAgICAgIAogICAg \ fQp9',virtual-path:'/etc/nginx/nginx.conf'}]" ``` - Create a multiple file configuration: ```shell az nginx deployment configuration create --name default \ --deployment-name myDeployment --resource-group myResourceGroup \ --root-file /etc/nginx/nginx.conf \ --files "[{'content':'aHR0cCB7CiAgICB1cHN0cmVhbSBhcHAgewogICAgICAgIHpvbmUg \ YXBwIDY0azsKICAgICAgICBsZWFzdF9jb25uOwogICAgICAgIHNlcnZlciAxMC4wLjEu \ NDo4MDAwOwogICAgfQoKICAgIHNlcnZlciB7CiAgICAgICAgbGlzdGVuIDgwOwogICAg \ ICAgIHNlcnZlcl9uYW1lICouZXhhbXBsZS5jb207CgogICAgICAgIGxvY2F0aW9uIC8g \ ewogICAgICAgICAgICBpbmNsdWRlIC9ldGMvbmdpbngvY29uZi5kL3Byb3h5LmNvbmY7 \ CiAgICAgICAgICAgIHByb3h5X3Bhc3MgaHR0cDovL2FwcDsKICAgICAgICAgICAgaGVh \ bHRoX2NoZWNrOwogICAgICAgIH0KICAgICAgICAKICAgIH0KfQ==', \ 'virtual-path':'/etc/nginx/nginx.conf'}, \ {'content':'cHJveHlfc2V0X2hlYWRlciBIb3N0ICRob3N0Owpwcm94eV9zZXRfaGVhZGVy \ IFgtUmVhbC1JUCAkcmVtb3RlX2FkZHI7CnByb3h5X3NldF9oZWFkZXIgWC1Qcm94eS1B \ cHAgYXBwOwpwcm94eV9zZXRfaGVhZGVyIEdpdGh1Yi1SdW4tSWQgMDAwMDAwOwpwcm94 \ eV9idWZmZXJpbmcgb247CnByb3h5X2J1ZmZlcl9zaXplIDRrOwpwcm94eV9idWZmZXJz \ IDggOGs7CnByb3h5X3JlYWRfdGltZW91dCA2MHM7', \ 'virtual-path':'/etc/nginx/conf.d/proxy.conf'}]" ``` - Multiple file configuration with protected files: ```shell az nginx deployment configuration create --name default \ --deployment-name myDeployment --resource-group myResourceGroup \ --root-file /etc/nginx/nginx.conf \ --files "[{'content':'aHR0cCB7CiAgICB1cHN0cmVhbSBhcHAgewogICAgICAgIHpvbmUg \ YXBwIDY0azsKICAgICAgICBsZWFzdF9jb25uOwogICAgICAgIHNlcnZlciAxMC4wLjEu \ NDo4MDAwOwogICAgfQoKICAgIHNlcnZlciB7CiAgICAgICAgbGlzdGVuIDgwOwogICAg \ ICAgIHNlcnZlcl9uYW1lICouZXhhbXBsZS5jb207CgogICAgICAgIGxvY2F0aW9uIC8g \ ewogICAgICAgICAgICBpbmNsdWRlIC9ldGMvbmdpbngvY29uZi5kL3Byb3h5LmNvbmY7 \ CiAgICAgICAgICAgIHByb3h5X3Bhc3MgaHR0cDovL2FwcDsKICAgICAgICAgICAgaGVh \ bHRoX2NoZWNrOwogICAgICAgIH0KICAgICAgICAKICAgIH0KfQ==', \ 'virtual-path':'/etc/nginx/nginx.conf'}, \ {'content':'cHJveHlfc2V0X2hlYWRlciBIb3N0ICRob3N0Owpwcm94eV9zZXRfaGVhZGVy \ IFgtUmVhbC1JUCAkcmVtb3RlX2FkZHI7CnByb3h5X3NldF9oZWFkZXIgWC1Qcm94eS1B \ cHAgYXBwOwpwcm94eV9zZXRfaGVhZGVyIEdpdGh1Yi1SdW4tSWQgMDAwMDAwOwpwcm94 \ eV9idWZmZXJpbmcgb247CnByb3h5X2J1ZmZlcl9zaXplIDRrOwpwcm94eV9idWZmZXJz \ IDggOGs7CnByb3h5X3JlYWRfdGltZW91dCA2MHM7', \ 'virtual-path':'/etc/nginx/conf.d/proxy.conf'}]" \ --protected-files "[{'content':'aHR0cCB7CiAgICB1cHN0cmVhbSBhcHAgewogICAgICAgIHpvbmUg \ YXBwIDY0azsKICAgICAgICBsZWFzdF9jb25uOwogICAgICAgIHNlcnZlciAxMC4wLjEu \ NDo4MDAwOwogICAgfQoKICAgIHNlcnZlciB7CiAgICAgICAgbGlzdGVuIDgwOwogICAg \ ICAgIHNlcnZlcl9uYW1lICouZXhhbXBsZS5jb207CgogICAgICAgIGxvY2F0aW9uIC8g \ ewogICAgICAgICAgICBpbmNsdWRlIC9ldGMvbmdpbngvY29uZi5kL3Byb3h5LmNvbmY7 \ CiAgICAgICAgICAgIHByb3h5X3Bhc3MgaHR0cDovL2FwcDsKICAgICAgICAgICAgaGVh \ bHRoX2NoZWNrOwogICAgICAgIH0KICAgICAgICAKICAgIH0KfQ==', \ 'virtual-path':'/etc/nginx/nginxprot.conf'}, \ {'content':'cHJveHlfc2V0X2hlYWRlciBIb3N0ICRob3N0Owpwcm94eV9zZXRfaGVhZGVy \ IFgtUmVhbC1JUCAkcmVtb3RlX2FkZHI7CnByb3h5X3NldF9oZWFkZXIgWC1Qcm94eS1B \ cHAgYXBwOwpwcm94eV9zZXRfaGVhZGVyIEdpdGh1Yi1SdW4tSWQgMDAwMDAwOwpwcm94 \ eV9idWZmZXJpbmcgb247CnByb3h5X2J1ZmZlcl9zaXplIDRrOwpwcm94eV9idWZmZXJz \ IDggOGs7CnByb3h5X3JlYWRfdGltZW91dCA2MHM7', \ 'virtual-path':'/etc/nginx/conf.d/proxyprot.conf'}]" ``` #### Upload a tarball package You can bundle your configuration files into a `.tar.gz` archive and upload it as a single package. - **Upload config using a package:** Create a gzip compress tarball package. **Note:** The package must start at the `etc` level. Your tarball must contain the `etc/nginx/` directory structure. ```shell $ tar -czf nginx.tar.gz etc $ tar -tzf nginx.tar.gz etc/ etc/nginx/ etc/nginx/nginx.conf etc/nginx/njs.js etc/nginx/servers/ etc/nginx/servers/server1.conf etc/nginx/servers/server2.conf ``` Where `etc` is a directory with the following structure: ```shell $ tree etc etc └── nginx ├── nginx.conf ├── njs.js └── servers ├── server1.conf └── server2.conf 2 directories, 4 files ``` Encode your tar.gz file and create your NGINXaaS configuration: ```shell TAR_DATA=$(base64 -i nginx.tar.gz) az nginx deployment configuration create --deployment-name myDeployment \ --resource-group myResourceGroup --root-file /etc/nginx/nginx.conf --name default \ --package data="$TAR_DATA" ``` - **Upload config using a package with protected files:** Create a gzip compress tarball package: ```shell $ tar -czf nginx-config.tgz etc $ tar -tzf nginx-config.tgz etc/ etc/nginx/ etc/nginx/nginx.conf etc/nginx/conf.d/ etc/nginx/conf.d/proxy.conf etc/nginx/conf.d/proxyprot.conf etc/nginx/conf.d/nginxprot.conf ``` Where `etc` is a directory with the following structure: ```shell $ tree etc etc └── nginx ├── conf.d │ ├── nginxprot.conf │ ├── proxy.conf │ └── proxyprot.conf └── nginx.conf 2 directories, 4 files ``` Encode your tarball file and create your NGINXaaS configuration with protected files: ```shell TAR_DATA=$(base64 -i nginx-config.tgz) az nginx deployment configuration create \ --deployment-name myDeployment \ --resource-group myResourceGroup \ --root-file /etc/nginx/nginx.conf \ --package data="${TAR_DATA}" protected-files="['/etc/nginx/conf.d/proxyprot.conf','/etc/nginx/conf.d/nginxprot.conf']" \ --name default ``` See the [Azure CLI Configuration Create Documentation](https://learn.microsoft.com/en-us/cli/azure/nginx/deployment/configuration?view=azure-cli-latest#az-nginx-deployment-configuration-create) for more details on the available parameters. ## Update a configuration Update a configuration for a deployment using a gzipped archive. Use the `az nginx deployment configuration update` command to update an existing NGINX configuration: ```shell az nginx deployment configuration update [--add] [--configuration-name] [--deployment-name] [--files] [--force-string {0, 1, f, false, n, no, t, true, y, yes}] [--ids] [--location] [--no-wait {0, 1, f, false, n, no, t, true, y, yes}] [--remove] [--resource-group] [--root-file] [--set] [--subscription] ``` ### Example - Update content of the first file in a configuration: ```shell az nginx deployment configuration update --name default \ --deployment-name myDeployment --resource-group myResourceGroup \ --files [0].content="aHR0cCB7CiAgICB1cHN0cmVhbSBhcHAgewogICAgICAgIHpvbmUg \ YXBwIDY0azsKICAgICAgICBsZWFzdF9jb25uOwogICAgICAgIHNlcnZlciAxMC4wLjEu \ NDo4MDAwOwogICAgfQoKICAgIHNlcnZlciB7CiAgICAgICAgbGlzdGVuIDgwOwogICAg \ ICAgIHNlcnZlcl9uYW1lICouZXhhbXBsZS5jb207CgogICAgICAgIGxvY2F0aW9uIC8g \ ewogICAgICAgICAgICBwcm94eV9zZXRfaGVhZGVyIEhvc3QgJGhvc3Q7CiAgICAgICAg \ ICAgIHByb3h5X3NldF9oZWFkZXIgWC1SZWFsLUlQICRyZW1vdGVfYWRkcjsKICAgICAg \ ICAgICAgcHJveHlfc2V0X2hlYWRlciBYLVByb3h5LUFwcCBhcHA7CiAgICAgICAgICAg \ IHByb3h5X3NldF9oZWFkZXIgR2l0aHViLVJ1bi1JZCAwMDAwMDA7CiAgICAgICAgICAg \ IHByb3h5X2J1ZmZlcmluZyBvbjsKICAgICAgICAgICAgcHJveHlfYnVmZmVyX3NpemUg \ NGs7CiAgICAgICAgICAgIHByb3h5X2J1ZmZlcnMgOCA4azsKICAgICAgICAgICAgcHJv \ eHlfcmVhZF90aW1lb3V0IDYwczsKICAgICAgICAgICAgcHJveHlfcGFzcyBodHRwOi8v \ YXBwOwogICAgICAgICAgICBoZWFsdGhfY2hlY2s7CiAgICAgICAgfQogICAgICAgIAog \ ICAgfQp9" ``` See the [Azure CLI Configuration Update Documentation](https://learn.microsoft.com/en-us/cli/azure/nginx/deployment/configuration?view=azure-cli-latest#az-nginx-deployment-configuration-update) for more details on the available parameters. **Note:** See the [NGINX connfiguration overview](overview.md) topic to learn more about: - [NGINX configuration automation workflows](overview.md#nginx-configuration-automation-workflows) - [NGINX filesystem restrictions](overview.md#nginx-filesystem-restrictions) - [Disallowed configuration directives](overview.md#disallowed-configuration-directives) - [Directives that cannot be overridden](overview.md#directives-that-cannot-be-overridden) - [Configuration directives list](overview.md#configuration-directives-list) -------------------------------------------------------------------------------- title: "Usage and cost estimator" url: https://docs.nginx.com/nginxaas-azure/billing/usage-and-cost-estimator/index.md --------------------------------------------------------------------------------

Cost Estimation for Standard V3 Plan

1. Estimate NCU Usage

NGINX Capacity Units Needed -- Sold in bundles of 10, with a minimum of 10
Show calculations
x new connections per second * y average connection duration seconds = z average concurrent connections 
Max(
    x concurrent connections /  Conns per NCU,
    y connections per second /  conns per second per NCU,
    z Mbps / Mbps per NCU
) =  NCUs
Usage needs at least x NCUs, rounded to the nearest 10, with a minimum of 10 = total NCUs

2. Estimate Monthly Cost

Total Monthly Payment --
The standard Azure networking and bandwidth charges apply to NGINX deployments.
Show calculations
Standard V3 Pricing Components:

hours × ( fixed deployment + NCUs × per NCU per hour + additional listen ports × 2 NCUs × ) + GB × per GB
Total:

Regional Pricing:
Region Tier NCU Cost/hr (No WAF) NCU Cost/hr (With WAF) Fixed Deployment/hr (No WAF) Fixed Deployment/hr (With WAF)

Note: Deployment fees and NCU costs vary by region tier and WAF usage. Data processing cost is $0.005/GB across all configurations.

-------------------------------------------------------------------------------- title: "Hosting static content in Azure Blob Storage" url: https://docs.nginx.com/nginxaas-azure/quickstart/hosting-static-content-blob-storage/index.md -------------------------------------------------------------------------------- F5 NGINXaaS for Azure (NGINXaaS) can serve static content stored in Azure Blob Storage using private endpoints, ensuring maximum security by keeping your storage account completely inaccessible from the public Internet. This approach also eliminates the configuration payload size limitations of local hosting. ## Before you begin - [An Azure Storage Account](https://learn.microsoft.com/en-us/azure/storage/common/storage-account-create) - [An NGINXaaS for Azure deployment](/nginxaas-azure/getting-started/create-deployment) - [A virtual network with available subnet space for private endpoints](https://learn.microsoft.com/en-us/azure/virtual-network/quick-create-portal) - Static content files to serve ## Configure Azure Blob Storage ### Upload static files to a container Upload your static files to a container in your storage account. In this example, we'll use a container named `content`. ### Disable public network access 1. In your storage account, in the **Security + networking** section, go to **Networking**. 1. Under **Public network access**, select **Disable**. 1. Select **Save**. ### Disable anonymous blob access 1. In your storage account, in the **Settings** section, go to **Configuration**. 1. Find the **Allow Blob anonymous access** setting and set it to **Disabled**. 1. Select **Save**. ### Set container access level to private 1. In the **Data management** section, go to **Containers**. 1. Select your container (for example, `content`). 1. Select **Change access level**. 1. Set **Anonymous access level** to **Private (no anonymous access)**. 1. Select **OK**. ### Create a new subnet for private endpoint NICs 1. Navigate to your virtual network where NGINXaaS is deployed. 1. Go to **Subnets** under **Settings**. 1. Select **+ Subnet**. 1. Create a new subnet which will be used to assign IP address to your Private Endpoint NIC. 1. Make a note of the subnet name; you will need it in the next step. ### Create a private endpoint 1. In your storage account, in the **Security + networking** section, go to **Networking**. 1. Go to the **Private endpoint connections** tab. 1. Select **+ Private endpoint**. 1. Configure the private endpoint: - **Name**: Provide a descriptive name for the private endpoint - **Network Interface Name**: Provide a name for the network interface - **Target sub-resource**: Select **blob** - **Virtual network**: Select the same virtual network as your NGINXaaS deployment - **Subnet**: Select the subnet created in the previous step - **Private DNS integration**: Enable this option to automatically create DNS records ### Generate a Shared Access Signature (SAS) token 1. In your storage account, in the **Security + networking** section, go to **Shared access signature**. 1. Configure the SAS token with minimal required permissions: - **Allowed services**: Check **Blob** - **Allowed resource types**: Check **Object** - **Allowed permissions**: Check **Read** only - **Start and expiry date/time**: Set appropriate validity period - **Allowed protocols**: Select **HTTPS only** 1. Select **Generate SAS and connection string**. 1. Copy the **SAS token** (the part starting with `?sv=`). **Note:** Store the SAS token securely and regenerate it regularly according to your security policies. Grant only the minimum permissions required for your use case. ## Configure NGINXaaS Create an NGINX configuration that uses the private endpoint and SAS token to access your Azure Blob Storage. The following NGINX config points to the `content` directory with `/static/` location and uses the SAS token from the previous step to authorize requests to blob storage. The resolver is set to 168.63.129.16 which is the Azure internal DNS IP. It doesn't change. It resolves the storage account endpoint to the private endpoint IP configured earlier. ```nginx user nginx; worker_processes auto; worker_rlimit_nofile 8192; pid /run/nginx/nginx.pid; error_log /var/log/nginx/error.log error; http { upstream storage_origin { server your-storage-account.blob.core.windows.net:443; keepalive 32; } resolver 168.63.129.16 valid=10s; server { listen 443 ssl; set $sas_token '?sv=YYYY-MM-DD&ss=b&srt=o&sp=r&se=YYYY-MM-DDTHH:MM:SSZ&st=YYYY-MM-DDTHH:MM:SSZ&spr=https&sig=YOUR_SAS_SIGNATURE_HERE'; ssl_certificate /etc/nginx/example.cert; ssl_certificate_key /etc/nginx/example.key; location /static/ { rewrite ^/static/(.*)$ /content/$1 break; proxy_pass https://storage_origin$uri$sas_token; proxy_set_header Host your-storage-account.blob.core.windows.net; proxy_http_version 1.1; proxy_set_header Connection ""; } } } ``` **Note:** Replace the following placeholders: - `your-storage-account` with your actual storage account name - `YOUR_SAS_SIGNATURE_HERE` with your actual SAS token signature - Update the SAS token parameters according to your generated token ### Configuration breakdown | Directive | Description | |------------|-------------| | **upstream storage_origin** | Defines the Azure Blob Storage endpoint as the backend server | | **resolver 168.63.129.16** | Uses Azure's internal DNS resolver to resolve the storage account to the private endpoint IP | | **set $sas_token** | Stores the SAS token for authorization | | **rewrite** | Maps the `/static/` path to the `/content/` container in blob storage | | **proxy_pass** | Forwards requests to the storage account with the SAS token appended | | **keepalive 32** | Maintains persistent connections for better performance | ## Upload the configuration Upload your NGINX configuration to your NGINXaaS deployment following the instructions in the [NGINX configuration](/nginxaas-azure/getting-started/nginx-configuration/nginx-configuration-portal.md) documentation. ## Test the configuration 1. Go to `https:///static/` to access your static content. - For example, if you have an `index.html` file in your `content` container, access it via `https:///static/index.html`. 1. Your content should be served from Azure Blob Storage through the private endpoint. ## Verify private endpoint connectivity You can verify that traffic flows through the private endpoint by checking that: 1. The storage account is completely inaccessible from the public Internet 1. DNS resolution of your storage account resolves to the private IP address of the private endpoint 1. Network traffic flows through your virtual network without traversing the public Internet ## Benefits of this approach - **Maximum security**: Storage account is completely private with no public Internet access - **No payload size limits**: Unlike local hosting, you're not limited by the 3 MB configuration payload size - **Scalable storage**: Azure Blob Storage can handle large amounts of static content - **Network isolation**: All traffic flows through your private virtual network - **Cost-effective**: Azure Blob Storage offers cost-effective storage for static content - **Controlled access**: SAS tokens provide fine-grained access control with expiration ## Security considerations - **SAS token management**: Regularly rotate SAS tokens and grant minimal required permissions - **Network isolation**: Ensure private endpoints are properly configured in isolated subnets - **Access monitoring**: Enable logging and monitoring for storage account access - **Principle of least privilege**: Grant only the minimum permissions necessary for your use case ## Limitations - Requires private endpoint configuration and additional subnet space - SAS tokens need regular rotation and management - Additional complexity compared to public access methods - Private endpoint incurs additional Azure networking costs -------------------------------------------------------------------------------- title: "NGINXaaS Load Balancer for Kubernetes" url: https://docs.nginx.com/nginxaas-azure/loadbalancer-kubernetes/index.md -------------------------------------------------------------------------------- ## Overview F5 NGINXaaS for Azure simplifies advanced Layer 4 and Layer 7 load balancing for Kubernetes clusters. With the NGINX Load Balancer for Kubernetes (NLK) feature, you can enable multi-cluster load balancing, failover, disaster recovery, and blue-green or canary deployments. ```mermaid flowchart TB Users[웃 Users] -.-> |GET '/tea' | NGINXaaS{NGINXaaS} NGINXaaS -.-> P1 NLK --> |Update upstream 'tea'| NGINXaaS subgraph AK[Azure Kubernetes Cluster] TeaSvc{Tea svc} -.-> P2(Pod) TeaSvc -.-> P1(Pod) k8sapi[K8s API] --> |watch| NLK(NLK controller) end style Users color:orange,stroke:orange,fill:#faefd9 linkStyle 0,1 color:orange,stroke:orange style NLK color:green,stroke:green,stroke-width:4px,fill:#d9fade style NGINXaaS color:green,stroke:green,stroke-width:4px,fill:#d9fade linkStyle 2 color:green,stroke:green style AK fill:#9bb1de,color:# style k8sapi color:#3075ff,stroke:#3075ff,stroke-width:4px linkStyle 5 color:#3075ff,stroke:#3075ff accDescr: A diagram showing users sending GET requests to NGINXaaS, which proxies traffic to a Kubernetes-based service named "TeaSvc" running multiple pods in an Azure Kubernetes Cluster, with upstream configurations dynamically managed via an NLK controller watching the Kubernetes API. ``` The NLK controller monitors [Kubernetes Services](https://kubernetes.io/docs/concepts/services-networking/service/) and updates an [NGINX Upstream](https://nginx.org/en/docs/http/ngx_http_upstream_module.html) dynamically. NGINXaaS applies these updates immediately and keeps them in sync during scaling or upgrades. ### Example use cases - You can use NGINXaaS for Azure to enforce rate limiting and application security with F5 WAF for NGINX, then forward all accepted traffic to your Kubernetes applications. - You can use NGINXaaS for Azure to receive traffic on `api.example.com` and route requests by URL path - for example, forwarding `/login` to a Kubernetes-based login service, `/graph` to a Kubernetes-hosted graph service, and `/process` to an application server on a standalone VM. ## Getting Started This guide explains how to integrate NGINXaaS with an Azure Kubernetes Service (AKS) cluster. See [Advanced Configuration](#advanced-configuration) for options to customize the installation.. Before following the steps in this guide, you must: - Create an AKS cluster. - Create an NGINXaaS deployment. See the [documentation](/nginxaas-azure/getting-started/create-deployment/deploy-azure-portal/) to deploy via the Azure portal. - Ensure network connectivity between the subnet delegated to the NGINXaaS deployment and the subnet where AKS is deployed. For example, the AKS cluster and NGINXaaS deployment can run on the same Azure VNET or on peered VNETs. ### Initial setup The steps in this section must be completed once for each new setup. We will install the NLK controller in the Kubernetes cluster and authorize that to send updates to the NGINXaaS deployment. 1. Create an NGINXaaS data plane API key. 1. Look up the NGINXaaS data plane API endpoint. 1. Install the NLK controller. #### Create an NGINXaaS data plane API key For detailed instructions on creating the dataplane API key and obtaining the API endpoint, see [Create NGINXaaS Dataplane API Key](/nginxaas-azure/quickstart/dataplane-api-key/). Copy the following to a secure location: - The API key value (you'll need this for the NLK configuration) - The dataplane API endpoint (you'll need this with the `/nplus` suffix) #### Install the NLK controller The NLK controller can be installed in your Kubernetes cluster using either Helm or the official [AKS Extension](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/f5-networks.f5-nginx-for-azure-aks-extension?tab=overview) available on the Azure Marketplace. ##### Install the NLK controller using Helm Install the NLK controller using `helm install`. Be sure your kubectl context is pointed at the desired cluster. ```shell helm install nlk oci://registry-1.docker.io/nginxcharts/nginxaas-loadbalancer-kubernetes --version 1.2.6 \ --set "nlk.dataplaneApiKey=${keyValue}" \ --set "nlk.config.nginxHosts=${dataplaneAPIEndpoint}nplus" \ --set "nlk.config.tls.mode=ca-tls" ``` ##### Install the AKS Extension using the Azure CLI Install the NLK controller using `az k8s-extension`. ```shell ## Customize this to provide the details about my already created AKS cluster aksName=myCluster aksGroup=myClusterGroup az k8s-extension create \ --name nlk \ --extension-type "nginxinc.nginxaas-aks-extension" \ --scope cluster \ --cluster-name ${aksName} \ --resource-group ${aksGroup} \ --cluster-type managedClusters \ --plan-name f5-nginx-for-azure-aks-extension \ --plan-product f5-nginx-for-azure-aks-extension \ --plan-publisher f5-networks \ --release-namespace nlk \ --config nlk.dataplaneApiKey=${keyValue} \ --config nlk.config.nginxHosts=${dataplaneAPIEndpoint}nplus \ --config nlk.config.tls.mode=ca-tls ``` ##### Install the AKS Extension using the Azure portal You can also install the NLK controller AKS extension by navigating to [F5 NGINXaaS Loadbalancer for Kubernetes](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/f5-networks.f5-nginx-for-azure-aks-extension) in the Azure Marketplace and following the installation steps. **Note:** If you are creating a new AKS cluster as part of this installation, note that we will enable the [Azure CNI Node Subnet plugin](https://learn.microsoft.com/en-us/azure/aks/concepts-network-cni-overview). This causes Cluster IP addresses to be exposed within your VNET. - Select **Get it now**. - Select **Continue** to proceed with the installation. - On the **Basics** tab, provide the following information: | Field | Description | |---------------------------- | ---------------------------- | | Subscription | Select the appropriate Azure subscription. | | Resource group | Select the AKS cluster's resource group. | - Select **Cluster Details**, and provide the AKS cluster name. You can select an existing AKS cluster or create a new one. - Select **Application Details**, and provide the following information: | Field | Description | |---------------------------- | ---------------------------- | | Cluster extension resource name | Provide a name for the NLK controller. | | Installation namespace | Provide the AKS namespace for the NLK controller. | | Allow minor version upgrades of extension | Select whether to allow the extension to be upgraded automatically to the latest minor version. | | NGINXaaS Dataplane API Key | Provide the previously generated data plane API key value: `{keyValue}` | | NGINXaaS Dataplane API Endpoint | Provide the previously retrieved data plane API endpoint value: `{dataplaneAPIEndpoint}nplus` | - Select **Review + Create** to continue. - Azure will validate the extension settings. This page will provide a summary of the provided information. Select **Create**. **Note:** The NGINXaaS data plane API that NLK uses is mounted at `${dataplaneAPIEndpoint}nplus`. For example, if the data plane API endpoint is `https://mynginx-75b3bf22a555.eastus2.nginxaas.net/` then the value for `nlk.config.nginxHosts` should be `https://mynginx-75b3bf22a555.eastus2.nginxaas.net/nplus`. ### Create an NGINX configuration with dynamic upstream You must define an [NGINX upstream](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#upstream) that satisfies the following requirements for it to be managed by the NLK controller: - The upstream cannot have any servers listed in it specified via the `server` directive. The controller will manage the servers dynamically. - The upstream must have a shared memory [zone](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#zone) defined. - The upstream must have a [state](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#state) file declared. The following is an example NGINX Configuration that can be used: ```nginx http { upstream my-service { # NOTE: There are no servers defined here as they will be managed dynamically by the controller. zone my-service 64K; # required state /tmp/my-service.state; # required } server { listen 80; location / { proxy_pass http://my-service; } } } ``` [Apply the NGINX configuration](/nginxaas-azure/getting-started/nginx-configuration/overview/) to your deployment after making the required changes. ### Create a Kubernetes Service Expose a Kubernetes `Service` to route traffic to your workload. The `Service` has the following requirements: - Add the annotation: `nginx.com/nginxaas: nginxaas` to mark the service to be monitored by NLK. - Choose one of the following `Service` types: - `NodePort`: To route external traffic into the cluster using a well defined port exposed on each AKS worker node. - `ClusterIP`: To route traffic to pods directly if you are running an Azure Container Networking Interface (CNI) that lets you expose the pods on the Azure VNET. If you created a new AKS cluster through the AKS Extenstion installation, this is configured automatically. - `LoadBalancer`: To route traffic to the cluster's external load balancer. The load balancer routes traffic into the cluster as normal. - The port name must be formatted as `{{NGINX Context}}-{{NGINX upstream name}}`. For example: - If the upstream is in the `http` context and named `my-service` then the name is `http-my-service` - If the upstream is in the `stream` context and named `jet` then the port name is `stream-jet` **Note:** **NGINX Ingress Controller users**: with v5.0.0 and upwards, if you wish to route traffic from your NGINXaaS deployment to your NGINX Ingress Controller service, please make the following changes to your helm chart values: - Add `"nginx.com/nginxaas": "nginxaas"` to the NGINX Ingress Controller service annotations. - Modify the `service.httpPort.name` or `service.httpsPort.name` values to provide the expected port name format, as above. The following example uses a service of type `NodePort`: ```yaml apiVersion: v1 kind: Service metadata: name: my-service annotations: # Let the controller know to pay attention to this service. # If you are connecting multiple controller the value can be used to distinguish them nginx.com/nginxaas: nginxaas spec: # expose a port on the nodes type: NodePort ports: - targetPort: http port: 80 protocol: TCP # The port name helps connect to NGINXaaS. It must be prefixed with either `http-` or `stream-` # and the rest of the name must match the name of an upstream in that context. name: http-my-service selector: app: awesome ``` ## Advanced Configuration ### Controller Configuration | Helm Value | Description | Value | |------------------------|----------------------------------------------------------|--------------------------------| | `nlk.config.logLevel` | How verbose should the NLK controller logs be. | Possible values are `debug`, `info`, `warn`, `error`. Default: `info`. | | `nlk.config.nginxHosts` | The NGINX Plus APIs to send upstream updates to. | Should be set to `{{dataplaneApiEndpoint}}nplus`.| | `nlk.config.serviceAnnotationMatch` | The value to match on a Service's `nginx.com/nginxaas` annotation. Useful when configuring multiple NLK controllers to update separate NGINXaaS deployemnts. | Default: `nginxaas`. | | `nlk.config.tls.mode` | The TLS mode to use when forming connections with the data plane API server. | Should be set to `ca-tls` to enforce TLS with the data plane API server.| | `nlk.dataplaneApiKey` | The NGINXaaS data plane API key that will authorize the controller to talk to your NGINXaaS deployment. | | ### Multiple AKS clusters A single NGINXaaS deployment can direct traffic to multiple AKS clusters. Each AKS cluster needs its own copy of NLK installed and connected to NGINXaaS. ```mermaid flowchart TB TeaUsers[웃 Users] -.-> |GET /tea | NGINXaaS{NGINXaaS} CoffeeUsers[웃 Users] -.-> |GET /coffee | NGINXaaS NGINXaaS -.-> |GET /tea| E H --> |Update upstream 'tea'| NGINXaaS NGINXaaS -.-> |GET /coffee| K M --> |Update upstream 'coffee'| NGINXaaS subgraph SG2[Azure Kubernetes Cluster 2] k8sapi2[K8s API] --> |watch| M(NLK controller) I{Coffee svc} -.-> J(Pod) I -.-> K(Pod) end subgraph SG1[Azure Kubernetes Cluster 1] k8sapi1[K8s API] --> |watch| H(NLK controller) D{Tea svc} -.-> E(Pod) D -.-> F(Pod) end style TeaUsers color:red,stroke:red,fill:#faefd9 linkStyle 0,2 color:red,stroke:red style CoffeeUsers color:orange,stroke:orange,fill:#faefd9 linkStyle 1,4 color:orange,stroke:orange style NGINXaaS color:green,stroke:green,stroke-width:4px,fill:#d9fade linkStyle 3,5 color:green,stroke:green style SG1 fill:#9bb1de,color:# style SG2 fill:#9bb1de,color:# style k8sapi1 color:#3075ff,stroke:#3075ff,stroke-width:4px style k8sapi2 color:#3075ff,stroke:#3075ff,stroke-width:4px linkStyle 6,9 color:#3075ff,stroke:#3075ff style H color:green,stroke:green,stroke-width:4px,fill:#d9fade style M color:green,stroke:green,stroke-width:4px,fill:#d9fade accDescr:A diagram showing NGINXaaS directing separate user GET requests for `/tea` and `/coffee` to respective Kubernetes-based services "TeaSvc" and "CoffeeSvc" that are running in separate Azure Kubernetes Clusters. An NLK controller in each cluster is independently updating the NGINXaaS with dynamic upstream configuration. ``` **Note:** - Configuring multiple NLK controllers to update the same upstream isn't supported and will result in unpredictable behavior. ### Multiple NGINXaaS deployments Multiple NLK controllers can be installed in the same AKS cluster to update separate NGINXaaS deployments. Each NLK needs a unique helm release name and needs a unique helm value for `nlk.config.serviceAnnotationMatch`. Each NLK will only watch services that have the matching annotation. **Note:** - Consider using `helm` to install multiple NLK controllers on an AKS cluster. Installing multiple copies of the controller on the same AKS cluster is not supported via the [AKS Extension](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/f5-networks.f5-nginx-for-azure-aks-extension?tab=overview). ## Troubleshooting NGINXaaS Loadbalancer for Kubernetes and NGINXaaS continually monitor and attempt to repair in case of error. However, if upstreams are not populated as expected, here are a few things you can look for. ### NLK controller logs The controller reports status information about the requests it is making to NGINXaaS. This is a good place to look to ensure that the controller has picked up your service and that it is communicating with NGINXaaS correctly. Run the following command to view the controller logs: `kubectl logs deployment/nlk-nginxaas-loadbalancer-kubernetes`. The logs can be made more verbose by setting the Helm value `nlk.config.logLevel` (see [Controller Configuration](#controller-configuration)). ### Enable NGINX Upstream Update Logs NGINXaaS supports exporting dynamic upstream update logs to an Azure Storage account or to a Log Analytics workspace. To set up logging: 1. Select **Diagnostic settings** under **Monitoring**. 1. Select **Add diagnostic setting**. 1. On the following panel, provide a **Diagnostic setting name**. 1. Enable the **NGINX Upstream Update Logs** category. 1. Select a destination. For more information on logging, see [Enable NGINX Logs](/nginxaas-azure/monitoring/enable-logging/). ### Metrics NGINXaaS has the following metrics that are useful to monitor upstream health: - `plus.http.upstream.peers.state.up` -- does the peer report being healthy. - `plus.http.upstream.peers.request.count` -- which peers are handling requests. See the [metrics catalog](/nginxaas-azure/monitoring/metrics-catalog.md) for the entire list of NGINXaaS metrics. -------------------------------------------------------------------------------- title: "NGINXaaS Load Balancer for VMSS" url: https://docs.nginx.com/nginxaas-azure/loadbalancer-vmss/index.md -------------------------------------------------------------------------------- ## Overview F5 NGINXaaS for Azure provides seamless integration with Azure Virtual Machine Scale Sets (VMSS) through dynamic upstream management. This feature enables automatic scaling of your application backend without requiring manual NGINX configuration updates when VMSS instances are added or removed. With NGINXaaS load balancing for VMSS, you can: - Automatically sync VMSS instance IP addresses to NGINX upstreams - Scale VMSS instances without NGINX configuration changes ## Architecture ```mermaid flowchart TB Agent[nginx-asg-sync Agent] --> |Update upstreams through API| NGINXaaS MI[Managed Identity] --> |Read permissions| VMSS Agent --> |Uses| MI style NGINXaaS fill:#e1f5fe style Agent fill:#f3e5f5 style VMSS fill:#fff3e0 ``` The `nginx-asg-sync` agent (an open-source NGINX agent) monitors your VMSS for scaling changes and automatically updates the NGINXaaS upstream configuration through the dataplane API. This ensures that traffic is distributed to all instances without manual intervention. ## Prerequisites Before setting up NGINXaaS load balancing for VMSS, ensure you have: - An active NGINXaaS for Azure deployment - Azure Virtual Machine Scale Sets (VMSS) - Network connectivity between NGINXaaS and VMSS instances - An Azure VM or container to run the nginx-asg-sync agent - Appropriate Azure permissions to assign managed identities ## Getting started ### Create NGINXaaS deployment with dynamic upstreams Dynamic upstreams are an NGINX abstraction that enables runtime management of backend servers through the [NGINX Plus API](https://docs.nginx.com/nginx/admin-guide/load-balancer/dynamic-configuration-api/). Unlike static upstreams where servers are explicitly defined in the NGINX configuration and remain unchanging at runtime, dynamic upstreams contain servers that are entirely managed by the NGINX Plus API and can be added, removed, or modified during runtime without configuration reloads. Create an NGINXaaS deployment. See the [documentation](/nginxaas-azure/getting-started/create-deployment/deploy-azure-portal/) to deploy using the Azure portal. Then, create an NGINX configuration with dynamic upstreams that will be managed by the nginx-asg-sync agent. The upstream must meet these requirements: - The upstream cannot have any static servers defined through the `server` directive - The upstream must have a shared memory [zone](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#zone) defined - The upstream must have a [state](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#state) file declared Example NGINX configuration: ```nginx http{ upstream backend-one { zone backend-one 64k; state /opt/nginx/state/backend-one.conf; } server { listen 80; status_zone backend; location /backend-one { proxy_set_header Host $host; proxy_pass http://backend-one; } location @hc-backend-one { internal; proxy_connect_timeout 1s; proxy_read_timeout 1s; proxy_send_timeout 1s; proxy_pass http://backend-one; health_check interval=1s mandatory; } } } ``` **Note:** **Health Checks Recommendation**: It is strongly recommended to add active health checks for your upstreams when using NGINXaaS load balancing for VMSS. This will prevent traffic from being routed to an unready VMSS instance and ensure better reliability during scaling operations. [Apply this NGINX configuration](/nginxaas-azure/getting-started/nginx-configuration/overview/) to your NGINXaaS deployment. ### Confirm network connectivity Verify the network connectivity between the subnet delegated to the NGINXaaS deployment and the existing Azure Virtual Machine Scale Sets (VMSS). For example, the VMSS and NGINXaaS deployment can run on the same Azure VNET or on peered VNETs. ### Create NGINXaaS dataplane API key Create a dataplane API key that the nginx-asg-sync agent will use to authenticate with your NGINXaaS deployment. For detailed instructions on creating the dataplane API key and obtaining the API endpoint, see [Create NGINXaaS Dataplane API Key](/nginxaas-azure/quickstart/dataplane-api-key/). Copy the following to a secure location: - The API key value (you'll need this for the nginx-asg-sync configuration) - The dataplane API endpoint (you'll need this with the `/nplus` suffix) ### Create VM for nginx-asg-sync agent Create an Azure VM that will run the nginx-asg-sync agent. For detailed instructions on creating VMs, see the [Azure documentation](https://learn.microsoft.com/en-us/azure/virtual-machines/linux/quick-create-portal). ### Assign managed identity permissions nginx-asg-sync uses the Azure API to read the IP addresses of the Virtual Machine Scale Set. To access the Azure API, nginx-asg-sync must run in an environment with appropriate permissions over the VMSS backend. This section configures a system-assigned managed identity with the minimum required permissions. You can assign managed identity permissions using: - **Azure Portal**: - For system-assigned identity: Navigate to your nginx-asg-sync VM resource → **Identity** - For role assignment: Navigate to your VMSS resource → **Access control (IAM)** → **Role assignments** - **Azure CLI**: Use Azure CLI commands for programmatic assignment For detailed instructions on managed identities, see the [Azure documentation](https://learn.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/overview). The following steps show how to assign managed identity permissions using the Azure CLI:
Show Azure CLI Steps #### Enable system-assigned managed identity on the VM Enable system-assigned managed identity on your nginx-asg-sync VM: ```bash # Set variables vmName="sync-agent-vm" vmResourceGroup="myResourceGroup" vmssResourceGroup="myVmssResourceGroup" vmssName="backend-one-vmss" subscriptionId=$(az account show --query id --output tsv) # Enable system-assigned managed identity on VM az vm identity assign \ --resource-group $vmResourceGroup \ --name $vmName ``` #### Get the VM's managed identity principal ID Retrieve the principal ID of the VM's system-assigned managed identity: ```bash # Get the identity's principal ID principalId=$(az vm show \ --resource-group $vmResourceGroup \ --name $vmName \ --query identity.principalId \ --output tsv) ``` #### Create the custom role Create a custom role with the necessary permissions for nginx-asg-sync. The exact permissions and the assignment scope depend on the [VMSS orchestration mode](https://learn.microsoft.com/en-us/azure/virtual-machine-scale-sets/virtual-machine-scale-sets-orchestration-modes): **Uniform VMSS:** Create a file called `vmss-network-read-role-uniform.json`: ```json { "Name": "VMSS-Network-Read-Role-Uniform", "Description": "Read VMSS and VMSS network interface information (Uniform mode)", "Actions": [ "Microsoft.Compute/virtualMachineScaleSets/read", "Microsoft.Compute/virtualMachineScaleSets/networkInterfaces/read" ], "NotActions": [], "AssignableScopes": [ "/subscriptions/" ] } ``` **Flexible VMSS:** Create a file called `vmss-network-read-role-flexible.json`: ```json { "Name": "VMSS-Network-Read-Role-Flexible", "Description": "Read VMSS, VM, and network interface information (Flexible mode)", "Actions": [ "Microsoft.Compute/virtualMachineScaleSets/read", "Microsoft.Compute/virtualMachineScaleSets/networkInterfaces/read", "Microsoft.Compute/virtualMachineScaleSets/virtualMachines/read", "Microsoft.Compute/virtualMachines/read", "Microsoft.Network/networkInterfaces/read" ], "NotActions": [], "AssignableScopes": [ "/subscriptions/" ] } ``` ```bash # Replace subscription ID in the JSON file (if using the template above) # Create the custom role for Uniform mode az role definition create \ --role-definition vmss-network-read-role-uniform.json # Create the custom role for Flexible mode az role definition create \ --role-definition vmss-network-read-role-flexible.json ``` #### Get the VMSS resource ID (scope) Get the full resource ID of your VMSS to use as the scope for role assignment: ```bash # Get VMSS resource ID vmssId=$(az vmss show \ --resource-group $vmssResourceGroup \ --name $vmssName \ --query id \ --output tsv) ``` #### Assign the custom role to the VM's managed identity Assign the custom role to the VM's system-assigned managed identity. The scope for assignment differs by VMSS mode: **Uniform VMSS:** Assign at the VMSS resource scope: ```bash # Set role name for assignment roleName="VMSS-Network-Read-Role-Uniform" # Assign the custom role to the VM's managed identity (Uniform) az role assignment create \ --assignee-object-id $principalId \ --assignee-principal-type ServicePrincipal \ --role $roleName \ --scope $vmssId ``` **Flexible VMSS:** Assign at the resource group scope: ```bash # Set role name for assignment roleName="VMSS-Network-Read-Role-Flexible" # Assign the custom role to the VM's managed identity (Flexible) az role assignment create \ --assignee-object-id $principalId \ --assignee-principal-type ServicePrincipal \ --role $roleName \ --scope /subscriptions//resourceGroups/ ``` #### Verify role assignment Verify that the role assignment was created successfully: ```bash # Verify role assignment az role assignment list \ --assignee $principalId \ --all \ --output table ``` **Note:** **Permission Propagation**: After creating the role assignment, it may take a few minutes for the permissions to take effect across Azure services.
### Install nginx-asg-sync agent The nginx-asg-sync agent can be installed on Azure VM, run as a Docker container, or deployed using Azure Container Instances (ACI). Download and install the agent after configuring managed identity permissions.
Option A: Install on Azure VM Install nginx-asg-sync agent on the VM you created in [Create VM for nginx-asg-sync agent](#create-vm-for-nginx-asg-sync-agent): ```bash # SSH into the VM and install # Get latest version and detect architecture VERSION=$(curl -sL https://api.github.com/repos/nginx/nginx-asg-sync/releases/latest | grep tag_name | cut -d '"' -f 4 | sed 's/^v//') if [ -z "$VERSION" ]; then echo "Failed to fetch latest version, using fallback version 1.0.4" VERSION="1.0.4" fi BASE_URL="https://github.com/nginxinc/nginx-asg-sync/releases/download/v${VERSION}" ARCH=$(uname -m) case "$ARCH" in x86_64) ARCH="amd64" ;; aarch64) ARCH="arm64" ;; *) echo "Unsupported architecture" && exit 1 ;; esac # Download and install nginx-asg-sync curl -LO "${BASE_URL}/nginx-asg-sync_${VERSION}_linux_${ARCH}.tar.gz" tar -xzf "nginx-asg-sync_${VERSION}_linux_${ARCH}.tar.gz" chmod +x nginx-asg-sync sudo mv nginx-asg-sync /usr/local/bin/ # Verify installation ls -la /usr/local/bin/nginx-asg-sync # Create configuration directory sudo mkdir -p /etc/nginx/ ``` **Next Steps**: After installation, proceed to [Configure nginx-asg-sync agent](#configure-nginx-asg-sync-agent) to create the configuration file and start the agent.
Option B: Run as Container Deploy nginx-asg-sync as a container using Docker. The host system (VM or other compute resource) running the container must have a managed identity with appropriate VMSS permissions configured as described in [Assign managed identity permissions](#assign-managed-identity-permissions): ```bash # Pull the Docker image docker pull docker-registry.nginx.com/nginx/asg-sync # Create the configuration file (config.yaml) in your current directory # (See Step 7 for the complete configuration file content) # Run nginx-asg-sync container docker run --rm -it \ -v $(pwd)/config.yaml:/etc/nginx/config.yaml \ -e CONFIG_PATH=/etc/nginx/config.yaml \ docker-registry.nginx.com/nginx/asg-sync:latest /nginx-asg-sync -config_path /etc/nginx/config.yaml ```
Option C: Run on Azure Container Instances (ACI) Deploy nginx-asg-sync as an Azure Container Instance with persistent configuration stored in Azure Files. [Azure Container Instances (ACI)](https://learn.microsoft.com/en-us/azure/container-instances/) provides a serverless way to run containerized applications without managing virtual machines. ACI is ideal for scenarios that need on-demand, burstable, and pay-per-execution container workloads: #### Create Azure storage account Create a storage account to store the nginx-asg-sync configuration file: ```bash # Set variables resourceGroup="myResourceGroup" storageAccountName="nginxasgsyncconfig" location="eastus" # Create storage account az storage account create \ --resource-group $resourceGroup \ --name $storageAccountName \ --location $location \ --sku Standard_LRS ``` ##### Create file share Create a file share within the storage account: ```bash # Create file share for configuration az storage share create \ --account-name $storageAccountName \ --name configshare ``` ##### Upload configuration file Upload your nginx-asg-sync configuration file to the file share. Create the `config.yaml` file using the content described in [Configuration file](#configuration-file): ```bash # Upload config.yaml to file share az storage file upload \ --account-name $storageAccountName \ --share-name configshare \ --source config.yaml ``` ##### Create user-asigned managed identity and assign VMSS permissions Create a user-assigned managed identity and assign the custom role for VMSS access. Use or create the same custom role as mentioned in [Create the Custom Role](#create-the-custom-role): ```bash # Create user-assigned managed identity az identity create \ --resource-group $resourceGroup \ --name nginx-asg-sync-identity \ --location $location # Get managed identity IDs IDENTITY_ID=$(az identity show \ --resource-group $resourceGroup \ --name nginx-asg-sync-identity \ --query id -o tsv) IDENTITY_CLIENT_ID=$(az identity show \ --resource-group $resourceGroup \ --name nginx-asg-sync-identity \ --query clientId -o tsv) IDENTITY_PRINCIPAL_ID=$(az identity show \ --resource-group $resourceGroup \ --name nginx-asg-sync-identity \ --query principalId -o tsv) # Set VMSS variables vmssResourceGroup="myVmssResourceGroup" vmssName="backend-one-vmss" roleName="VMSS-Network-Read-Role" # Get VMSS resource ID vmssId=$(az vmss show \ --resource-group $vmssResourceGroup \ --name $vmssName \ --query id \ --output tsv) ## Assign the custom role to user-assigned managed identity # For Uniform VMSS (assign at VMSS resource scope): az role assignment create \ --assignee-object-id $IDENTITY_PRINCIPAL_ID \ --assignee-principal-type ServicePrincipal \ --role $roleName \ --scope $vmssId # For Flexible VMSS (assign at resource group scope): # az role assignment create \ # --assignee-object-id $IDENTITY_PRINCIPAL_ID \ # --assignee-principal-type ServicePrincipal \ # --role $roleName \ # --scope /subscriptions//resourceGroups/ ``` ##### Create container instance Deploy the nginx-asg-sync container instance with the user-assigned managed identity that has proper VMSS permissions: ```bash # Set ACI variables aciName="nginx-asg-sync-aci" containerImage="docker-registry.nginx.com/nginx/asg-sync:latest" # Get storage account key storageKey=$(az storage account keys list \ --resource-group $resourceGroup \ --account-name $storageAccountName \ --query '[0].value' \ --output tsv) # Create container instance with user-assigned managed identity az container create \ --resource-group $resourceGroup \ --name $aciName \ --image $containerImage \ --location $location \ --os-type Linux \ --cpu 1 \ --memory 1.5 \ --restart-policy Always \ --assign-identity $IDENTITY_ID \ --environment-variables AZURE_CLIENT_ID=$IDENTITY_CLIENT_ID \ --command-line "/nginx-asg-sync -config_path /etc/nginx/config.yaml" \ --azure-file-volume-account-name $storageAccountName \ --azure-file-volume-account-key $storageKey \ --azure-file-volume-share-name configshare \ --azure-file-volume-mount-path /etc/nginx ``` ##### Verify ACI deployment Check that the container instance is running successfully: ```bash # Check container status az container show \ --resource-group $resourceGroup \ --name $aciName \ --query "{Status:instanceView.state}" \ --output table # View container logs az container logs \ --resource-group $resourceGroup \ --name $aciName # For streaming logs (real-time) az container attach \ --resource-group $resourceGroup \ --name $aciName ``` Example output when the container starts successfully: ``` 2025/12/31 10:25:30 nginx-asg-sync version v1.0.4 2025/12/31 10:25:30 Updated HTTP servers of backend-one for group backend-one-vmss ; Added: [172.19.0.6:80 172.19.0.7:80], Removed: [], Updated: [] ```
### Configure nginx-asg-sync agent Create the configuration file for nginx-asg-sync to connect to your NGINXaaS deployment and monitor VMSS instances. nginx-asg-sync is configured in **/etc/nginx/config.yaml**. #### Configuration file Create `/etc/nginx/config.yaml`: ```yaml # nginx-asg-sync configuration for NGINXaaS for Azure cloud_provider: Azure subscription_id: resource_group_name: your_vmss_resource_group # NGINXaaS dataplane API endpoint (note the /nplus suffix) # Example : /nplus api_endpoint: https://your-nginxaas-endpoint.region.nginxaas.net/nplus sync_interval: 5s # Authentication headers for NGINXaaS custom_headers: Content-Type: application/json Authorization: ApiKey your_base64_encoded_dataplane_api_key # VMSS upstreams configuration upstreams: - name: backend-one virtual_machine_scale_set: backend-one-vmss port: 80 kind: http max_conns: 0 max_fails: 1 fail_timeout: 10s slow_start: 0s ``` #### Configuration parameters
View Configuration Parameters | Parameter | Description | Required | |-----------|-------------|----------| | `cloud_provider` | Must be set to "Azure" | Yes | | `subscription_id` | Azure subscription ID containing the VMSS | Yes | | `resource_group_name` | Resource group name containing the VMSS | Yes | | `api_endpoint` | NGINXaaS dataplane API endpoint with `/nplus` suffix | Yes | | `sync_interval` | How often to check for VMSS changes | No (default: 5s) | | `custom_headers` | Authentication headers for NGINXaaS API | Yes | | `upstreams[].name` | Name of the NGINX upstream to manage | Yes | | `upstreams[].virtual_machine_scale_set` | Name of the VMSS to monitor | Yes | | `upstreams[].port` | Port number on VMSS instances | Yes | | `upstreams[].kind` | Traffic type: "http" or "stream" | Yes | | `upstreams[].max_conns` | Maximum connections per instance | No (default: 0) | | `upstreams[].max_fails` | Failed attempts before marking unhealthy | No (default: 1) | | `upstreams[].fail_timeout` | Time to consider instance failed | No (default: 10s) | | `upstreams[].slow_start` | Gradual weight increase time | No (default: 0s) |
#### Start the agent (VM installation) Start nginx-asg-sync directly using the command line on your VM: ```bash # Run nginx-asg-sync directly nginx-asg-sync config_path=/etc/nginx/config.yaml # Run with log file output nginx-asg-sync config_path=/etc/nginx/config.yaml -log_path=< path to log file > ``` Example output when the agent starts successfully: ``` 2026/01/08 15:44:12 nginx-asg-sync version 1.0.4 2026/01/08 15:44:13 Updated HTTP servers of backend-one for group backend-one-vmss ; Added: [172.19.0.6:80 172.19.0.7:80], Removed: [], Updated: [] 2026/01/08 16:08:07 Updated HTTP servers of backend-one for group backend-one-vmss ; Added: [172.19.0.8:80], Removed: [], Updated: [] ``` ## Monitoring and troubleshooting ### Verify upstream updates Check that upstreams are being updated in NGINXaaS: 1. **Using NGINXaaS API Dashboard:** - Navigate to your NGINXaaS deployment in Azure portal - Go to **Monitoring** → **Metrics** - Monitor the following metrics for upstream health: - `plus.http.upstream.peers.state.up` – shows if the peer reports being healthy - `plus.http.upstream.peers.request.count` – shows which peers are handling requests 1. **Using dataplane API:** ```bash # Check upstream status using the dataplane API curl -H "Authorization: ApiKey " \ -H "Content-Type: application/json" \ "/nplus/9/http/upstreams//servers" ``` Example output showing VMSS instances synchronized to the upstream: ```json [ { "max_conns": 0, "max_fails": 1, "server": "172.19.0.8:80", "fail_timeout": "10s", "slow_start": "0s", "id": 2115793169 }, { "max_conns": 0, "max_fails": 1, "server": "172.19.0.6:80", "fail_timeout": "10s", "slow_start": "0s", "id": 528011112 }, { "max_conns": 0, "max_fails": 1, "server": "172.19.0.7:80", "fail_timeout": "10s", "slow_start": "0s", "id": 1665424715 } ] ``` ### Common troubleshooting steps
View Troubleshooting Guide 1. **Agent can't authenticate with NGINXaaS:** - Verify the API key is correctly base64 encoded - Check that the API key hasn't expired - Ensure the dataplane API endpoint URL is correct with `/nplus` suffix 1. **Agent can't read VMSS information:** - Verify managed identity is assigned to the VM/container - Check that appropriate permissions are granted to the identity - Confirm subscription ID and resource group names are correct 1. **Upstreams not updating:** - Verify upstream names in config.yaml match NGINX configuration - Check that upstreams are properly defined with zone and state - Monitor agent logs for error messages
## References and Additional Resources - [Assign managed identities](/nginxaas-azure/getting-started/managed-identity-portal/) - [nginx-asg-sync examples](https://github.com/nginx/nginx-asg-sync/blob/main/examples/nginxforazure.md) - [Create an NGINXaaS deployment](/nginxaas-azure/getting-started/create-deployment/deploy-azure-portal/) - [VMSS Creation](https://learn.microsoft.com/en-us/azure/virtual-machine-scale-sets/flexible-virtual-machine-scale-sets-portal) - [NGINXaaS monitoring and metrics](/nginxaas-azure/monitoring/metrics-catalog.md) -------------------------------------------------------------------------------- title: "Add certificates using Terraform" url: https://docs.nginx.com/nginxaas-azure/getting-started/ssl-tls-certificates/ssl-tls-certificates-terraform/index.md -------------------------------------------------------------------------------- ## Overview You can manage SSL/TLS certificates for F5 NGINXaaS for Azure (NGINXaaS) using Terraform. ## Prerequisites - Confirm that you meet the [NGINXaaS Prerequisites](/nginxaas-azure/getting-started/prerequisites.md). - [Authenticate Terraform to Azure](https://learn.microsoft.com/en-us/azure/developer/terraform/authenticate-to-azure) - [Install Terraform](https://learn.hashicorp.com/tutorials/terraform/install) **Note:** The examples in the NGINXaaS for Azure Snippets GitHub repository use the prerequisites module [available in the same repository](https://github.com/nginxinc/nginxaas-for-azure-snippets/tree/main/terraform/prerequisites). ## Upload and manage a certificate You can find examples of Terraform configurations in the [NGINXaaS for Azure Snippets GitHub repository](https://github.com/nginxinc/nginxaas-for-azure-snippets/tree/main/terraform/certificates) To create a deployment, add a certificate, and use it in a configuration, run the following commands: ```shell terraform init terraform plan terraform apply --auto-approve ``` ## Delete a deployment Once the deployment is no longer needed, run the following to clean up the deployment and related resources: ```shell terraform destroy --auto-approve ``` ## Additional resources - [Terraform NGINX certificate documentation](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/nginx_certificate) - [NGINXaaS Managed Identity Documentation](/nginxaas-azure/getting-started/managed-identity-portal.md) - [NGINXaaS Azure Monitor Documentation](/nginxaas-azure/monitoring/enable-metrics.md) -------------------------------------------------------------------------------- title: "Assign Managed Identities" url: https://docs.nginx.com/nginxaas-azure/getting-started/managed-identity-portal/index.md -------------------------------------------------------------------------------- ## Overview F5 NGINXaaS for Azure (NGINXaaS) leverages managed identities for its integrations with Azure services. Managed identities are used for the following integrations: - Azure Key Vault (AKV): fetch SSL/TLS certificates from AKV to your NGINXaaS deployment, so that they can be referenced by your NGINX configuration. - Azure Monitor: publish metrics from your NGINX deployment to Azure Monitor. - Azure Storage: export logs from your NGINX deployment to Azure Blob Storage Container. ## Prerequisites - A user assigned managed identity (optional, for additional integrations). If you are unfamiliar with managed identities for Azure resources, refer to the [Managed Identity documentation](https://learn.microsoft.com/en-us/entra/identity/managed-identities-azure-resources/overview) from Microsoft. - Owner access on the resource group or subscription to assign the managed identity to the NGINX deployment. ## Adding a user assigned managed identity 1. Go to your NGINXaaS for Azure deployment. 2. Select **Identity** in the left menu, select the **User Assigned** tab, and select **Add**. 3. Select the appropriate **subscription** and **user assigned managed identity**, then select **Add**.
**Note:** NGINXaaS supports adding one user assigned managed identity in addition to the required system assigned managed identity. Adding more than one user assigned managed identity is not supported. 4. The added user assigned managed identity will show up in the main table. ## Removing a user assigned managed identity 1. Select the managed identity you want to remove from the list and then select **Remove**. 2. Confirm the operation by selecting **Yes** on the confirmation prompt. **Note:** Removing a user-assigned managed identity from an NGINX deployment will only cause issues if that specific user-assigned identity was being used to fetch certificates from Azure Key Vault. In such cases, updates to the deployment will fail until the configuration is updated to not use those certificates. ## System assigned managed identity The system-assigned managed identity is required for all NGINXaaS deployments. When creating deployments through the Azure Portal, this identity is automatically created. For deployments created using other methods (such as ARM templates, Bicep, or Terraform), you must explicitly create the system-assigned managed identity. Once created, it cannot be removed. ### Viewing and Managing the system assigned managed identity 1. Go to your NGINXaaS for Azure deployment. 2. Select **Identity** in the left menu, select the **System Assigned** tab to view the system-assigned managed identity details. 3. The system assigned managed identity will be shown as enabled with Status **On**. **Note:** The system-assigned managed identity cannot be disabled or removed. Attempting to toggle the status to "Off" will result in an error. ### Add a system-assigned managed identity **Note:** This section applies only to legacy deployments created before system-assigned managed identity became mandatory. New deployments already have this identity enabled. To add a system-assigned managed identity to an existing deployment: 1. Go to your NGINXaaS for Azure deployment. 2. Select **Identity** in the left menu, then select the **System Assigned** tab. 3. Toggle the **Status** to **On**. 4. Select **Save** to apply the changes. #### Managing role assignments To provide the role assignments necessary for the deployment: 1. Select **Azure Role Assignments** under Permissions on the System Assigned tab. 2. Select **Add Role Assignments** 3. On the **Add role assignment (Preview)** panel, select the appropriate **Scope** and **Role**. Then select **Save**. ## Legacy deployments without system assigned managed identity **Note:** **Legacy Deployments**: Deployments created before system-assigned managed identity became mandatory will continue to operate normally and can still be updated (including deployment properties and NGINX configurations). However, logging and monitoring features will not work. You can add a system-assigned managed identity to these deployments by navigating to the Identity page and enabling it under the System Assigned tab. ## Checking for deployments without system assigned managed identity Use the following Azure Resource Graph query to identify NGINXaaS deployments that do not have a system-assigned managed identity. #### Azure Resource Graph Query You can run this query in the Azure Portal by navigating to **Azure Resource Graph Explorer** or by using the search bar and typing "Resource Graph Explorer". ``` Resources | where type == "nginx.nginxplus/nginxdeployments" | where isnull(identity) or identity.type !has "SystemAssigned" | project name, location, type, id, identity ``` The query lists the name, location, resource type, resource ID, and current identity configuration of the NGINXaaS deployments that do not have a system assigned managed identity. ## What's next [Add SSL/TLS Certificates](/nginxaas-azure/getting-started/ssl-tls-certificates/ssl-tls-certificates-portal.md) -------------------------------------------------------------------------------- title: "Azure SDK" url: https://docs.nginx.com/nginxaas-azure/client-tools/sdk/index.md description: Learn how to use the Python Azure Management SDK to manage NGINXaaS for Azure deployments. -------------------------------------------------------------------------------- F5 NGINXaaS for Azure (NGINXaaS) deployments can be managed using the multi-language SDK. This document outlines common workflows using the Python SDK. You can find example code to manage NGINXaaS deployments and related objects in the NGINXaaS GitHub repository, [NGINXaaS Snippets](https://github.com/nginxinc/nginxaas-for-azure-snippets/tree/main/sdk/python/). ## Prerequisites - [NGINXaaS Prerequisites](/nginxaas-azure/getting-started/prerequisites.md) - Install Azure Identity package - [azure-identity](https://pypi.org/project/azure-identity/) - Install the NGINX SDK - [azure-mgmt-nginx](https://pypi.org/project/azure-mgmt-nginx/) - See [NGINXaaS Snippets](https://github.com/nginxinc/nginxaas-for-azure-snippets/tree/main/sdk/python/deployments/) for an example script to create prerequisite resources. ## Workflows - For a complete list of NGINXaaS SDK documentation, see the [Azure NGINXaaS SDK Documentation](https://learn.microsoft.com/en-us/python/api/overview/azure/mgmt-nginx-readme) - [Azure Authentication SDK Documentation](https://learn.microsoft.com/en-us/azure/developer/python/sdk/authentication-overview) ### Create or update a deployment For example scripts to create or update deployment resources, see [NGINXaaS Snippets](https://github.com/nginxinc/nginxaas-for-azure-snippets/tree/main/sdk/python/deployments/) - [Azure SDK Deployment Create or Update Documentation](https://learn.microsoft.com/en-us/python/api/azure-mgmt-nginx/azure.mgmt.nginx.operations.deploymentsoperations?view=azure-python#azure-mgmt-nginx-operations-deploymentsoperations-begin-create-or-update) - [Azure SDK Deployment Delete Documentation](https://learn.microsoft.com/en-us/python/api/azure-mgmt-nginx/azure.mgmt.nginx.operations.deploymentsoperations?view=azure-python#azure-mgmt-nginx-operations-deploymentsoperations-begin-delete) - [NGINXaaS Managed Identity Documentation](/nginxaas-azure/getting-started/managed-identity-portal.md) - [NGINXaaS Azure Monitor Documentation](/nginxaas-azure/monitoring/enable-metrics/) ### Create or update a certificate Create or update a certificate under a deployment. This references an existing certificate in an Azure Key Vault and makes it available to the NGINX configuration. See [NGINXaaS Snippets](https://github.com/nginxinc/nginxaas-for-azure-snippets/tree/main/sdk/python/certificates/) for example scripts to create or update deployment certificate resources. - [Azure SDK Certificate Create or Update Documentation](https://learn.microsoft.com/en-us/python/api/azure-mgmt-nginx/azure.mgmt.nginx.operations.certificatesoperations?view=azure-python#azure-mgmt-nginx-operations-certificatesoperations-begin-create-or-update) - [Azure SDK Certificate Delete Documentation](https://learn.microsoft.com/en-us/python/api/azure-mgmt-nginx/azure.mgmt.nginx.operations.configurationsoperations?view=azure-python#azure-mgmt-nginx-operations-configurationsoperations-begin-delete) - [NGINXaaS Certificates Documentation](/nginxaas-azure/getting-started/ssl-tls-certificates/ssl-tls-certificates-portal.md) - [Azure SDK Key Vault Documentation](https://learn.microsoft.com/en-us/python/api/overview/azure/key-vault) ### Create or update a configuration Create or update the default configuration for a deployment using a gzipped archive based on the NGINXaaS documentation below. See [NGINXaaS Snippets](https://github.com/nginxinc/nginxaas-for-azure-snippets/tree/main/sdk/python/configurations/) for example scripts to create or update deployment configuration resources. - [Azure SDK Configuration Create or Update Documentation](https://learn.microsoft.com/en-us/python/api/azure-mgmt-nginx/azure.mgmt.nginx.operations.configurationsoperations?view=azure-python#azure-mgmt-nginx-operations-configurationsoperations-begin-create-or-update) - [NGINXaaS GZIP Configuration Documentation](/nginxaas-azure/getting-started/nginx-configuration/nginx-configuration-portal.md#upload-gzip-nginx-configuration) ## Additional Docs - [Python Azure SDK Overview](https://learn.microsoft.com/en-us/python/api/overview/azure/nginx) - [Java Azure SDK Overview](https://learn.microsoft.com/en-us/java/api/overview/azure/nginx) - [JavaScript Azure SDK Overview](https://learn.microsoft.com/en-us/javascript/api/overview/azure/nginx) - [Go Azure SDK Documentation](https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/nginx/armnginx) - [.NET Azure SDK Overview (Preview)](https://learn.microsoft.com/en-us/dotnet/api/overview/azure/nginx?view=azure-dotnet-preview) -------------------------------------------------------------------------------- title: "Configure alerts" url: https://docs.nginx.com/nginxaas-azure/monitoring/configure-alerts/index.md -------------------------------------------------------------------------------- ## Overview **Note:** F5 NGINXaaS for Azure (NGINXaaS) publishes platform metrics to Azure Monitor. To learn more about how to create and manage metrics-based alert rules, refer to the [Alerts section in Azure Monitor](https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/alerts-create-new-alert-rule?tabs=metric) documentation from Microsoft. This guide explains how to create and configure metrics-based alerts for your NGINXaaS for Azure deployment using Azure Monitor. ## Prerequisites - Setup is complete for [NGINXaaS for Azure deployment](/nginxaas-azure/getting-started/create-deployment/). - To complete this setup, you must be an owner or user access administrator for the NGINX deployment resource. - To enable metrics, see [Enable Metrics](/nginxaas-azure/monitoring/enable-metrics.md). **Note:** See [Azure monitor overview](https://docs.microsoft.com/en-us/azure/azure-monitor/overview) documentation to familiarize with Azure Monitor. ## Create metrics-based alerts for proactive monitoring. 1. Go to your NGINXaaS for Azure deployment. 2. Select **Alerts** in the left menu. 3. In the **Create** menu, select **Alert rule**. 4. Select the **Scope** tab, and choose NGINX deployment as the scope of the alert. **Note:** The scope is auto-selected as NGINX deployment. 5. In the **Conditions** tab, select a **Signal name**, for example, "nginx.http.request.count". ![Screenshot of the Conditions tab showing how to select a Signal name from the list](nginxaas-azure/alert-select-signal.png) 6. Define the **alert logic** such as: - Set the threshold and average as per your requirements. - Set the frequency to evaluate alerts as per your requirements. ![Screenshot of the alert logic page showing how to set the threshold and frequency](nginxaas-azure/alert-logic.png) 7. Define the **actions**: - Create an **action group** for future reference. See the [Configure basic action group settings](https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/action-groups) section. - Define the **notification settings**: whom to notify when the alert is triggered. See the [Configure notifications](https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/action-groups) section. - (Optional) Define an action to be performed when the alert is triggered, such as a runbook or azure function. 8. Fill out the details of the alert: - Specify the **severity** of the alert, and the name of the rule. - In the **advanced options** tab, you can turn on "Enable alert upon creation" and "Automatically resolve alerts". **Note:** [Standard Azure alert charges will apply](https://azure.microsoft.com/en-us/pricing/details/monitor/). -------------------------------------------------------------------------------- title: "Configure F5 WAF for NGINX" url: https://docs.nginx.com/nginxaas-azure/app-protect/configure-waf/index.md -------------------------------------------------------------------------------- ## Overview This guide explains how to configure the F5 WAF for NGINX security features. ## Configure To use F5 WAF for NGINX apply the following changes to the NGINX config file. 1. Load the F5 WAF for NGINX module on the main context: ```nginx load_module modules/ngx_http_app_protect_module.so; ``` 2. Set the enforcer address: ```nginx app_protect_enforcer_address 127.0.0.1:50000; ``` **Note:** The app_protect_enforcer_address directive is a required directive for F5 WAF for NGINX to work and must match 127.0.0.1:50000 3. Enable F5 WAF for NGINX with the `app_protect_enable` directives in the appropriate scope. The `app_protect_enable` directive may be set in the `http`, `server`, and `location` contexts. It is recommended to have a basic policy enabled in the `http` or `server` context to process malicious requests in a more complete manner. ```nginx app_protect_enable on; ``` 4. Configure the path of the pre-compiled policy file to the `app_protect_policy_file` directive. You can find the list of supported policies and their paths under the [Precompiled Policies](#precompiled-policies) section. ```nginx app_protect_policy_file /etc/app_protect/conf/NginxDefaultPolicy.json; ``` Sample Config with F5 WAF for NGINX configured: ```nginx user nginx; worker_processes auto; worker_rlimit_nofile 8192; pid /run/nginx/nginx.pid; load_module modules/ngx_http_app_protect_module.so; events { worker_connections 4000; } error_log /var/log/nginx/error.log debug; http { access_log off; server_tokens ""; app_protect_enforcer_address 127.0.0.1:50000; server { listen 80 default_server; location / { app_protect_enable on; app_protect_policy_file /etc/app_protect/conf/NginxDefaultPolicy.json; proxy_pass http://127.0.0.1:80/proxy/$request_uri; } location /proxy { default_type text/html; return 200 "Hello World\n"; } } } ``` ## Precompiled Policies NGINXaaS for Azure ships with the two reference policies (Default and Strict) supported in F5 WAF for NGINX. These policies are supported in both the blocking and transparent enforcement modes. For more information on these policies refer the F5 WAF for NGINX [configuration guide](https://docs.nginx.com/nginx-app-protect-waf/v5/configuration-guide/configuration/). The following table shows the path to the precompiled policy file that needs to be used with the `app_protect_policy_file` directive: | Policy | Enforcement Mode | Path | |---------------------------- | ---------------------------- | -------------------------------------------- | | Default | Strict | /etc/app_protect/conf/NginxDefaultPolicy.json | | Default | Transparent | /etc/app_protect/conf/NginxDefaultPolicy_transparent.json | | Strict | Strict | /etc/app_protect/conf/NginxStrictPolicy.json | | Strict | Transparent | /etc/app_protect/conf/NginxStrictPolicy_transparent.json | To view the contents of the available security policies, navigate to the azure portal and select the **Security Policies** tab in the F5 WAF for NGINX section. ## Custom policies NGINXaas for Azure also supports custom security policies. You can create and modify custom security policies to deploy to F5 WAF for NGINX Instances using the API or Azure Portal. ### Manage custom policies To create a custom security policy in the Azure Portal: 1. Select your deployment 2. Select **F5 WAF for NGINX** from the menu on the left 3. Select **Custom Policies** 4. Select **Add Custom Security Policy** to open the policy editor In the policy editor: - Enter the policy **Name**, **File path**, your policy content, and then select **Save**. - The **File path** must start with the prefix "/etc/app_protect/conf/". - The **File path** extension must be ".json". After your policy has been saved, you can then reference it in your NGINX configuration. For more information on policy configuration and syntax, refer to the F5 WAF for NGINX [configuration guide](https://docs.nginx.com/nginx-app-protect-waf/v5/configuration-guide/configuration/). **Note:** The **name** field within the security policy must be unique among the policies referenced in your NGINX configuration. The **Custom Policies** tab shows the status of your custom policies (Compilation and Application Status). Custom policies are automatically compiled when created or modified. Policies that are applied to the NGINX configuration cannot be deleted until they are first removed from the configuration. It is highly recommended to use logging to monitor the performance of F5 WAF for NGINX and to help diagnose problems. See [Enable F5 WAF for NGINX Logs](/nginxaas-azure/app-protect/enable-logging.md) for directions to configure security and operational logs. ## What's next [Enable F5 WAF for NGINX Logs](/nginxaas-azure/app-protect/enable-logging.md) -------------------------------------------------------------------------------- title: "Deploy using Terraform" url: https://docs.nginx.com/nginxaas-azure/getting-started/create-deployment/deploy-terraform/index.md -------------------------------------------------------------------------------- ## Overview F5 NGINXaaS for Azure (NGINXaaS) deployments can be managed using Terraform. This document outlines common Terraform workflows for NGINXaaS. ## Prerequisites - Confirm that you meet the [NGINXaaS Prerequisites](/nginxaas-azure/getting-started/prerequisites.md). - [Authenticate Terraform to Azure](https://learn.microsoft.com/en-us/azure/developer/terraform/authenticate-to-azure) - [Install Terraform](https://learn.hashicorp.com/tutorials/terraform/install) **Note:** The examples in the NGINXaaS for Azure Snippets GitHub repository use the prerequisites module [available in the same repository](https://github.com/nginxinc/nginxaas-for-azure-snippets/tree/main/terraform/prerequisites). **Note:** **System-Assigned Managed Identity Required**: Ensure your Terraform configuration includes the required system-assigned managed identity by setting `identity.type = "SystemAssigned"` or `"SystemAssigned, UserAssigned"` for each deployment. ## Create a deployment You can find examples of Terraform configurations in the [NGINXaaS for Azure Snippets GitHub repository](https://github.com/nginxinc/nginxaas-for-azure-snippets/tree/main/terraform/deployments/create-or-update) To create a deployment, use the following Terraform commands: ```shell terraform init terraform plan terraform apply --auto-approve ``` ## Delete a deployment Once the deployment is no longer needed, run the following to clean up the deployment and related resources: ```shell terraform destroy --auto-approve ``` ## Additional resources - If you're just starting with Terraform, you can learn more on their [official website](https://www.terraform.io/). - [Terraform NGINX deployment documentation](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/nginx_deployment) - [NGINXaaS Managed Identity Documentation](/nginxaas-azure/getting-started/managed-identity-portal.md) - [NGINXaaS Azure Monitor Documentation](/nginxaas-azure/monitoring/enable-metrics.md) -------------------------------------------------------------------------------- title: "Enable F5 WAF for NGINX Logs" url: https://docs.nginx.com/nginxaas-azure/app-protect/enable-logging/index.md -------------------------------------------------------------------------------- ## Overview F5 NGINXaaS for Azure (NGINXaaS) supports exporting F5 WAF for NGINX logs to an Azure Storage account or to a Log Analytics workspace. ## Setting up operational logs F5 WAF for NGINX operational logs are sent to the NGINX error logs. See [Enable NGINX Logs](/nginxaas-azure/monitoring/enable-logging/) to configure error logs. ## Setting up security logs 1. Enable the NGINX Security Logs category in **Diagnostic Settings**. For more information on logging, see [Enable NGINX Logs](/nginxaas-azure/monitoring/enable-logging/). ![Screenshot of the NGINXaaS WAF security logs diagnostic settings](nginxaas-azure/security-diagnostic-setting.png) 2. Update your NGINX configuration to enable security logs in an http/server/location context. ```nginx app_protect_security_log_enable on; ``` 3. Configure the log configuration path and destination to the `app_protect_security_log` directive. More information on supported log configurations and destination can be found in the following sections. ```nginx app_protect_security_log "/etc/app_protect/conf/log_all.json" syslog:server=localhost:5140; ``` Sample NGINX config with security logs enabled: ```nginx user nginx; worker_processes auto; worker_rlimit_nofile 8192; pid /run/nginx/nginx.pid; load_module modules/ngx_http_app_protect_module.so; events { worker_connections 4000; } error_log /var/log/nginx/error.log debug; http { access_log off; server_tokens ""; app_protect_enforcer_address 127.0.0.1:50000; server { listen 80 default_server; location / { app_protect_enable on; app_protect_policy_file /etc/app_protect/conf/NginxDefaultPolicy.tgz; app_protect_security_log_enable on; app_protect_security_log "/etc/app_protect/conf/log_all.tgz" syslog:server=localhost:5140; proxy_pass http://127.0.0.1:80/proxy/$request_uri; } location /proxy { default_type text/html; return 200 "Hello World\n"; } } } ``` You can find more details on these directives in the [Security log](https://docs.nginx.com/nginx-app-protect-waf/v5/logging-overview/security-log/) documentation. ### Log Configuration NGINXaaS for Azure ships with several pre-compiled log configuration bundles. More details on these logging bundles can be found in the [Security log](https://docs.nginx.com/waf/logging/security-logs/) documentation. The following table shows the path to the log configuration file that needs to be used with the app_protect_security_log directive: | Profile | Path | |---------------------------- | -------------------------------------------- | | log_default | /etc/app_protect/conf/log_default.json | | log_all | /etc/app_protect/conf/log_all.json | | log_illegal | /etc/app_protect/conf/log_illegal.json | | log_blocked | /etc/app_protect/conf/log_blocked.json | | log_grpc_all | /etc/app_protect/conf/log_grpc_all.json | | log_grpc_illegal | /etc/app_protect/conf/log_grpc_illegal.json | | log_grpc_blocked | /etc/app_protect/conf/log_grpc_blocked.json | To view the contents of the available log configuration, navigate to the azure portal and select the Log Configurations tab in the F5 WAF for NGINX section. ### Logging Destinations 1. Logging to NGINXaaS syslog (Recommended) NGINXaaS for Azure supports a local syslog server running on port 5140. Syslogs forwarded to this destination are sent to the sink configured in the **Diagnostic Setting** section. ```nginx app_protect_security_log "/etc/app_protect/conf/log_all.json" syslog:server=localhost:5140; ``` **Note:** When using a NGINXaaS syslog destination, the syslog server destination needs to match localhost:5140. Configuring log directives to other syslog locations will result in an error in the NGINX config. 2. File Logging NGINXaaS for Azure supports logging to a file path. Any logs written under `/var/log/app_protect` will be sent to the sink configured in **Diagnostic Setting**. ```nginx app_protect_security_log "/etc/app_protect/conf/log_all.json" /var/log/app_protect/security.log; ``` **Note:** When using a file destination, the configured path for nginx security logs has to be within `/var/log/app_protect`. Configuring log directives to other file locations will result in an error in the NGINX config. ## Analyzing NGINX security logs in Azure Log Analytics workspaces. If the diagnostic setting destination details included a Logs Analytics workspace, logs appear in the "NGXSecurityLogs" table with the following columns: | **Attribute** | **Description** | |-----------------------------|-----------------| | **Location** | The location of the NGINXaaS resource.| | **Message** | The generated NGINX security log line. | | **FilePath** | The path to which NGINX security logs are configured to be logged to if the nginx config uses file-based logs. | | **Tag** | The tag with which NGINX security logs are generated if syslog-based log configuration is used. | | **Facility** | The syslog facility that generates the NGINX security logs if syslog-based log configuration is being used. | | **Severity** | The syslog severity with which NGINX security logs were generated if syslog-based log configuration is used. | To view the raw data in the NGINX security log, run the following KQL query: ``` NGXSecurityLogs | extend JSONLog = extract(@"json_log\s*=\s*""({.*?})""", 1, Message) | extend Log = parse_json(replace_string(JSONLog, '""', '"')) | project Log ``` ![Screenshot showing NGINX security logs in the Logs Analytics Workspace](nginxaas-azure/log-analytics-security.png) The following sample queries will help you get started with creating visualizations based on security logs. Blocked requests by IP ``` NGXSecurityLogs | extend JSONLog = extract(@"json_log\s*=\s*""({.*?})""", 1, Message) | extend Log = parse_json(replace_string(JSONLog, '""', '"')) | where Log.enforcementAction == "block" | project ClientIP = tostring(Log.clientIp), TimeGenerated | summarize count() by ClientIP, bin(TimeGenerated, 1m) | render timechart ``` Blocked requests by URL ``` NGXSecurityLogs | extend JSONLog = extract(@"json_log\s*=\s*""({.*?})""", 1, Message) | extend Log = parse_json(replace_string(JSONLog, '""', '"')) | where Log.enforcementAction == "block" | project URL = tostring(Log.url), TimeGenerated | summarize count() by URL, bin(TimeGenerated, 1m) | render timechart ``` Top matched rules ``` NGXSecurityLogs | extend JSONLog = extract(@"json_log\s*=\s*""({.*?})""", 1, Message) | extend Log = parse_json(replace_string(JSONLog, '""', '"')) | where Log.enforcementAction == "block" | project attackType = Log.enforcementState.attackType, TimeGenerated | mv-expand attackType | project attackName = tostring(attackType.name), TimeGenerated | summarize count() by attackName, bin(TimeGenerated, 1m) | render timechart ``` To add a visualization to a dashboard, select the **Pin to dashboard** icon in the top right of the log analytics workspace. **Note:** It can take up to 90 minutes after adding diagnostic settings for logs to appear in the provided Logs Analytics Workspace. -------------------------------------------------------------------------------- title: "Enable rate limiting" url: https://docs.nginx.com/nginxaas-azure/quickstart/rate-limiting/index.md -------------------------------------------------------------------------------- F5 NGINXaaS for Azure (NGINXaaS) supports rate limiting using the [ngx_http_limit_req_module](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html) module to limit the processing rate of requests. For more information on rate limiting with NGINX, see [NGINX Limiting Access to Proxied HTTP Resources](https://docs.nginx.com/nginx/admin-guide/security-controls/controlling-access-proxied-http/) and [Rate Limiting with NGINX and NGINX Plus](https://www.nginx.com/blog/rate-limiting-nginx/). ## Configuring basic rate limiting ```nginx http { #... limit_req_zone $binary_remote_addr zone=mylimit:10m rate=1r/s; server { #... location /login/ { limit_req zone=mylimit; } } ``` **Note:** As a prerequisite to using the `sync` parameter with `limit_req_zone` directive for rate limiting, enable [Runtime State Sharing with NGINXaaS for Azure](/nginxaas-azure/quickstart/runtime-state-sharing.md). -------------------------------------------------------------------------------- title: "Feature overview" url: https://docs.nginx.com/nginxaas-azure/overview/feature-comparison/index.md description: Compare NGINXaaS for Azure with other NGINX offerings. -------------------------------------------------------------------------------- NGINXaaS for Azure delivers the core capabilities of NGINX as a managed service, integrated with Microsoft Azure. It provides most of the features of NGINX Open Source and many from NGINX Plus, but some capabilities are not included. Below is a feature breakdown with notes on support and limitations. ## Load balancing - [HTTP and TCP/UDP load balancing](https://docs.nginx.com/nginx/admin-guide/load-balancer/http-load-balancer/) - [Layer 7 request routing](https://www.nginx.org/en/docs/http/ngx_http_core_module.html#location) - [Session persistence](https://docs.nginx.com/nginx/admin-guide/load-balancer/http-load-balancer/#enabling-session-persistence) - [Active health checks](https://docs.nginx.com/nginx/admin-guide/load-balancer/http-health-check/) - [DNS-based service discovery](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#service) --- ## Content caching - [Static and dynamic content caching](https://docs.nginx.com/nginx/admin-guide/content-cache/content-caching/) - MQTT protocol support for IoT devices **Limitation:** - [Cache purging API](https://docs.nginx.com/nginx/admin-guide/content-cache/content-caching/#purging-content-from-the-cache) is not available --- ## Web server and reverse proxy - Origin server for static content - Reverse proxy for [HTTP](https://nginx.org/en/docs/http/ngx_http_proxy_module.html), [FastCGI](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html), [memcached](https://nginx.org/en/docs/http/ngx_http_memcached_module.html), [SCGI](https://nginx.org/en/docs/http/ngx_http_scgi_module.html), and [uwsgi](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html) - [HTTP/2 gateway](https://www.nginx.org/en/docs/http/ngx_http_v2_module.html) - [gRPC proxy](https://nginx.org/en/docs/http/ngx_http_grpc_module.html) - [HTTP/2 server push](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_push) - [HTTP/3 over QUIC](https://nginx.org/en/docs/http/ngx_http_v3_module.html) --- ## Security - [HTTP basic authentication](https://www.nginx.org/en/docs/http/ngx_http_auth_basic_module.html) - [Authentication subrequests](https://nginx.org/en/docs/http/ngx_http_auth_request_module.html) (For external authentication systems) - [IP-based access controls](https://nginx.org/en/docs/http/ngx_http_access_module.html) - [Rate limiting](https://blog.nginx.org/blog/rate-limiting-nginx) - Dual-stack RSA/ECC SSL/TLS offload - TLS 1.3 support - [JWT authentication](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html) - OpenID Connect SSO - NGINX as a SAML Service Provider - [F5 WAF for NGINX](https://www.f5.com/products/nginx/nginx-app-protect) (Available at an extra cost) **Limitations:** - Internal redirect and F5 WAF for NGINX DoS are not available --- ## Monitoring - Export metrics directly into [Azure Monitor](https://learn.microsoft.com/en-us/azure/azure-monitor/overview) - Dashboards in [Azure Monitor](https://learn.microsoft.com/en-us/azure/azure-monitor/overview) and [Azure Portal](https://azure.microsoft.com/en-us/get-started/azure-portal) - [Extended status with 100+ metrics](https://docs.nginx.com/nginx/admin-guide/monitoring/live-activity-monitoring/) **Limitations:** - No built-in live dashboard like NGINX Plus; visibility is provided through Azure Monitor instead - Native OpenTelemetry tracing is not available --- ## High availability (HA) - [Active-active HA](https://docs.nginx.com/nginx/admin-guide/high-availability/) - [Configuration synchronization across the cluster](https://docs.nginx.com/nginx/admin-guide/high-availability/configuration-sharing/) - [State sharing](https://docs.nginx.com/nginx/admin-guide/high-availability/zone_sync/) for session persistence, rate limiting, and key-value store **Limitation:** - [Active-passive HA](https://docs.nginx.com/nginx/admin-guide/high-availability/) is not applicable in the managed service model --- ## Programmability - [NGINX JavaScript (njs) module](https://www.f5.com/company/blog/nginx/harnessing-power-convenience-of-javascript-for-each-request-with-nginx-javascript-module) - [Key-value store](https://nginx.org/en/docs/http/ngx_http_keyval_module.html) **Limitations:** - [NGINX Plus API for dynamic reconfiguration](https://docs.nginx.com/nginx/admin-guide/load-balancer/dynamic-configuration-api/) is not available --- ## Streaming media - Live streaming: RTMP, HLS, DASH - VOD: Flash (FLV), MP4 **Limitations:** - Adaptive bitrate streaming (HLS/HDS) and [MP4 bandwidth controls](https://nginx.org/en/docs/http/ngx_http_mp4_module.html) are not available --- ## Ecosystem and extensibility - Dynamic module support for: - [Image-Filter](https://nginx.org/en/docs/http/ngx_http_image_filter_module.html) - [njs](https://nginx.org/en/docs/njs/) - [OpenTelemetry](https://nginx.org/en/docs/ngx_otel_module.html) - [XSLT](https://nginx.org/en/docs/http/ngx_http_xslt_module.html) - Delivered as a managed service in Microsoft Azure - [Commercial support](https://my.f5.com/manage/s/article/K000140156/) from F5 **Limitations:** - [Ingress Controller](https://www.f5.com/products/nginx/nginx-ingress-controller) and OpenShift Router are not included - Dynamic module repository is limited compared to NGINX Plus --- **Note:** For a detailed comparison between NGINX Plus and NGINX Open Source, refer to [Differences between NGINX Open Source and NGINX Plus](https://www.f5.com/products/get-f5/nginx-open-source-vs-nginx-one-differences-in-features) on the F5 website. -------------------------------------------------------------------------------- title: "Securing upstream traffic" url: https://docs.nginx.com/nginxaas-azure/quickstart/security-controls/securing-upstream-traffic/index.md -------------------------------------------------------------------------------- Learn how to encrypt HTTP traffic between F5 NGINXaaS for Azure (NGINXaaS) and an upstream group or a proxied server. To secure TCP traffic to upstream servers, follow the [NGINX Plus guide](https://docs.nginx.com/nginx/admin-guide/security-controls/securing-tcp-traffic-upstream/). As with securing HTTP traffic, you will need to [add the SSL/TLS client certificate](/nginxaas-azure/getting-started/ssl-tls-certificates/ssl-tls-certificates-portal.md) to the NGINXaaS deployment. ### Prerequisites - [Add a SSL/TLS Certificate](/nginxaas-azure/getting-started/ssl-tls-certificates/ssl-tls-certificates-portal.md) to the NGINXaaS deployment. - Enable [njs module](/nginxaas-azure/quickstart/njs-support.md) if configuration uses njs directives. ### Configuring NGINX [Add the client certificate and the key](/nginxaas-azure/getting-started/ssl-tls-certificates/ssl-tls-certificates-portal.md) that will be used to authenticate NGINX to the NGINXaaS deployment. Make a note of the filepaths you assign to the `Certificate path` and `Key path`. Next, change the URL to an upstream group to support SSL connections. In the NGINX configuration file, specify the “https” protocol for the proxied server or an upstream group in the `proxy_pass` directive: ```nginx location /upstream { proxy_pass https://backend.example.com; } ``` Add the client certificate and key to the NGINX config to authenticate NGINX on each upstream server with `proxy_ssl_certificate` and `proxy_ssl_certificate_key` directives using the filepaths noted above. NGINXaaS for Azure expects the directive's file arguments to match the filepaths assigned to a certificate and key that have been added to the NGINXaaS Deployment. ```nginx location /upstream { proxy_pass https://backend.example.com; proxy_ssl_certificate /etc/nginx/client.pem; proxy_ssl_certificate_key /etc/nginx/client.key; } ``` If you use a self-signed certificate for an upstream or your own CA, you may include this file by adding it to the [NGINX configuration](/nginxaas-azure/getting-started/nginx-configuration/nginx-configuration-portal.md) and including the `proxy_ssl_trusted_certificate` directive. The file must be in the PEM format. Optionally, include the [`proxy_ssl_verify`](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_verify) and [`proxy_ssl_verify_depth`](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_verify_depth) directives to have NGINX check the validity of the security certificates: ```nginx location /upstream { # ... proxy_ssl_trusted_certificate /etc/nginx/trusted_ca_cert.crt; proxy_ssl_verify on; proxy_ssl_verify_depth 2; # ... } ``` If your configuration is using the [njs module](/nginxaas-azure/quickstart/njs-support.md), you can include the `js_fetch_trusted_certificate` directive to [verify](http://nginx.org/en/docs/njs/reference.html#fetch_verify) HTTPS certificates with the [Fetch API](http://nginx.org/en/docs/njs/reference.html#ngx_fetch). Toggle `yes` to include the CA file as proctectd file when using Azure Portal as show below: ![Screenshot of the Azure portal showing the toggle for protected files](nginxaas-azure/add-ca-as-protected-file.png) ### Configuring upstreams Each upstream server should be configured to accept HTTPS connections. For each upstream server, specify a path to the server certificate and the private key [added to the NGINXaaS Deployment](/nginxaas-azure/getting-started/ssl-tls-certificates/ssl-tls-certificates-portal.md) with `ssl_certificate` and `ssl_certificate_key` directives: ```nginx server { listen 443 ssl; server_name backend1.example.com; ssl_certificate /etc/ssl/certs/server.crt; ssl_certificate_key /etc/ssl/certs/server.key; #... location /upstream { proxy_pass http://url_to_app.com; # ... } } ``` Specify the path to a trusted client CA certificate added to the [NGINX configuration](/nginxaas-azure/getting-started/nginx-configuration/nginx-configuration-portal.md) with the `ssl_client_certificate` or `ssl_trusted_certificate` directives. The file should be in PEM format. ```nginx server { #... ssl_client_certificate /etc/ssl/certs/ca.crt; ssl_verify_client optional; #... } ``` Complete example to secure your traffic between NGINX and upstream servers is available [here](https://docs.nginx.com/nginx/admin-guide/security-controls/securing-http-traffic-upstream/#complete-example). ### Additional configuration If your keys specified in `proxy_ssl_certificate_key` use passphrase, then include the passphrases as file to the NGINX configuration and reference the file in `proxy_ssl_password_file`. It is recomended to use a protected file as an argument for this directive. ```nginx location /upstream { proxy_pass https://backend.example.com; proxy_ssl_certificate /etc/nginx/client.pem; proxy_ssl_certificate_key /etc/nginx/client.key; proxy_ssl_password_file pswd.txt; } ``` You can also configure NGINX with a list of revoked certificates using `proxy_ssl_crl` directive. Include this file in PEM format in your NGINX configuration. ```nginx location /upstream { # ... proxy_ssl_crl /etc/nginx/revoked.crt; # ... } ``` `ssl_session_ticket_key` directive specifies a file with the secret key used to encrypt and decrypt TLS session tickets. To use these directives in your config file, include a file to your [NGINX configuration](/nginxaas-azure/getting-started/nginx-configuration/nginx-configuration-portal.md) with 80 or 48 bytes of random data generated using `openssl` command, in your config bundle. For example, ```nginx http { server { ssl_certificate /etc/nginx/client.pem; ssl_certificate_key /etc/nginx/client.key; ssl_client_certificate /etc/nginx/ca.pem; ssl_session_ticket_key keys; } } ``` -------------------------------------------------------------------------------- title: "Set up OIDC authentication" url: https://docs.nginx.com/nginxaas-azure/quickstart/security-controls/oidc/index.md -------------------------------------------------------------------------------- ## Overview Learn how to configure F5 NGINXaaS for Azure with OpenID Connect (OIDC) authentication. There are currently two methods available for setting up OIDC authentication. 1. Using Native OIDC implementation (Introduced from NGINX Plus R34) 2. Using NJS based implementation ## Prerequisites These prerequisites are used for both methods of configuring NGINXaaS for Azure with IdP using Native OIDC and NJS. 1. Configure an NGINXaaS deployment with [SSL/TLS certificates](/nginxaas-azure/getting-started/ssl-tls-certificates/). 2. Enable [Runtime State Sharing](/nginxaas-azure/quickstart/runtime-state-sharing.md) on the NGINXaaS deployment. ## Configure NGINXaaS for Azure with IdP using Native OIDC This method applies to NGINX Plus Release 34 and later. In earlier versions, NGINX Plus relied on an njs-based solution, which required NGINX JavaScript files, key-value stores, and advanced OpenID Connect logic. In the latest NGINX Plus version, the new [OpenID Connect module](https://nginx.org/en/docs/http/ngx_http_oidc_module.html) simplifies this process to just a few directives. ### Prerequisites 1. Configure the IdP. For example, you can [register a Microsoft Entra Web application](/nginx/deployment-guides/single-sign-on/entra-id/#entra-setup) as the IdP. 1. A domain name pointing to your NGINXaaS deployment, for example, `demo.example.com`. This will be referred to as `` throughout this guide. With your IdP configured, you can enable OIDC on NGINXaaS for Azure. 1. Ensure that you have the values of the **Client ID**, **Client Secret**, and **Tenant ID** obtained during IdP configuration. 1. In your NGINX configuration file, add a public DNS resolver with the [`resolver`](https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver) directive in the [`http {}`](https://nginx.org/en/docs/http/ngx_http_core_module.html#http) context: ```nginx http { resolver 127.0.0.1:49153 ipv4=on valid=300s; # ... } ``` 1. In the [`http {}`](https://nginx.org/en/docs/http/ngx_http_core_module.html#http) context, define your IdP provider by specifying the [`oidc_provider {}`](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#oidc_provider) context. The `session_store` directive stores the session data and we need `keyval_zone` to sync this data in a clustered environment. Include the `state` parameter to persist session data across NGINX restarts. For example, for Microsoft Entra ID: ```nginx http { resolver 127.0.0.1:49153 ipv4=on valid=300s; keyval_zone zone=my_store:8M state=/opt/oidc_sessions.json timeout=1h sync; oidc_provider entra { issuer https://login.microsoftonline.com//v2.0; client_id ; client_secret ; session_store my_store; logout_uri /logout; post_logout_uri https:///post_logout/; logout_token_hint on; userinfo on; } # ... } ``` Where: - `` is your Microsoft Entra Tenant ID - `` is your Application (client) ID from Entra ID - `` is your client secret from Entra ID - `` is your NGINXaaS deployment FQDN **Note:** The `state=/opt/oidc_sessions.json` parameter enables persistence of OIDC session data across NGINX restarts. The state file path must be placed in a directory accessible to the NGINX worker processes, following [NGINX Filesystem Restrictions](/nginxaas-azure/getting-started/nginx-configuration/overview/#nginx-filesystem-restrictions). 1. Configure your server block with OIDC protection. The following example uses localhost as the upstream server: ```nginx server { listen 443 ssl; server_name ; ssl_certificate /etc/ssl/certs/fullchain.pem; ssl_certificate_key /etc/ssl/private/key.pem; location / { # Protect this location with OIDC auth_oidc entra; # Forward OIDC claims as headers proxy_set_header sub $oidc_claim_sub; proxy_set_header email $oidc_claim_email; proxy_set_header name $oidc_claim_name; proxy_pass http://127.0.0.1:8080; } location /post_logout/ { return 200 "You have been logged out.\n"; default_type text/plain; } } server { # Simple test upstream server listen 8080; location / { return 200 "Hello, $http_name!\nEmail: $http_email\nEntra ID sub: $http_sub\n"; default_type text/plain; } } ``` 1. Add the runtime state sharing configuration to your NGINX configuration as mentioned in the [Prerequisites](/nginxaas-azure/quickstart/security-controls/oidc.md#prerequisites). This enables synchronization of OIDC session data across NGINXaaS instances: ```nginx stream { resolver 127.0.0.1:49153 valid=20s; server { listen 9000; zone_sync; zone_sync_server internal.nginxaas.nginx.com:9000 resolve; } } ``` #### Configuration example with localhost as an upstream server ```nginx http { # Use a public DNS resolver for OIDC discovery resolver 127.0.0.1:49153 ipv4=on valid=300s; keyval_zone zone=my_store:8M state=/opt/oidc_sessions.json timeout=1h sync; # Define OIDC provider (Microsoft Entra ID example) oidc_provider entra { # The issuer is typically something like: # https://login.microsoftonline.com//v2.0 issuer https://login.microsoftonline.com//v2.0; # Replace with your actual Entra client_id and client_secret client_id ; client_secret ; session_store my_store; # RP‑initiated logout logout_uri /logout; post_logout_uri https:///post_logout/; logout_token_hint on; # Fetch userinfo claims userinfo on; } server { listen 443 ssl; server_name ; ssl_certificate /etc/ssl/certs/fullchain.pem; ssl_certificate_key /etc/ssl/private/key.pem; location / { # Protect this location with Entra OIDC auth_oidc entra; # Forward OIDC claims as headers if desired proxy_set_header sub $oidc_claim_sub; proxy_set_header email $oidc_claim_email; proxy_set_header name $oidc_claim_name; proxy_pass http://127.0.0.1:8080; } location /post_logout/ { return 200 "You have been logged out.\n"; default_type text/plain; } } server { # Simple test upstream server listen 8080; location / { return 200 "Hello, $http_name!\nEmail: $http_email\nEntra ID sub: $http_sub\n"; default_type text/plain; } } } stream { resolver 127.0.0.1:49153 valid=20s; server { listen 9000; zone_sync; zone_sync_server internal.nginxaas.nginx.com:9000 resolve; } } ``` 1. Upload the NGINX configurations. See [Upload an NGINX configuration](/nginxaas-azure/getting-started/nginx-configuration/) for more details. For more detailed steps on this OIDC configuration, please refer to: - [Single Sign-On with Microsoft Entra ID](/nginx/deployment-guides/single-sign-on/entra-id.md) - [Terraform snippets for Native OIDC use case](https://github.com/nginxinc/nginxaas-for-azure-snippets/tree/main/terraform/configurations/native-oidc) ### Testing 1. Open `https:///` in a browser. You will be automatically redirected to your IdP sign-in page. 1. Enter valid IdP credentials. Upon successful sign-in, you will be redirected back to NGINXaaS and see your protected application. Using the example configuration, you will see a message displaying the authenticated user's information in the browser: ```text Hello, [Name]! Email: [email] Entra ID sub: [subject_id] ``` 1. To test logout, navigate to `https:///logout`. NGINXaaS initiates an RP-initiated logout, and your IdP ends the session and redirects back to the post-logout page. ## Configure NGINXaaS for Azure with IdP using NJS ### Prerequisites 1. [Configure the IdP](https://github.com/nginxinc/nginx-openid-connect/blob/main/README.md#configuring-your-idp). For example, you can [register a Microsoft Entra Web application](https://learn.microsoft.com/en-us/entra/identity-platform/quickstart-register-app) as the IdP. Configuring NGINXaaS for Azure with OIDC is similar as [Configuring NGINX Plus](https://github.com/nginxinc/nginx-openid-connect/blob/main/README.md#configuring-nginx-plus) in [nginx-openid-connect](https://github.com/nginxinc/nginx-openid-connect) but it also has its own specific configurations that must be completed to work normally. 1. If your IdP supports OpenID Connect Discovery (usually at the URI /.well-known/openid-configuration), use the `configure.sh` script in [nginx-openid-connect](https://github.com/nginxinc/nginx-openid-connect) to complete the configuration. Otherwise, follow [Configuring NGINX Plus](https://github.com/nginxinc/nginx-openid-connect/blob/main/README.md#configuring-nginx-plus) to complete the configuration. 2. Configure NGINXaaS with specific configurations: - `openid_connect_configuration.conf`: a. Set a proper timeout value for `map $host $zone_sync_leeway`. ```nginx map $host $zone_sync_leeway { # Specifies the maximum timeout for synchronizing ID tokens between cluster # nodes when you use shared memory zone content sync. This option is only # recommended for scenarios where cluster nodes can randomly process # requests from user agents and there may be a situation where node "A" # successfully received a token, and node "B" receives the next request in # less than zone_sync_interval. default 2000; # Time in milliseconds, e.g. (zone_sync_interval * 2 * 1000) } ``` b. Set a proper path for `proxy_cache_path`, see [Enable content caching](/nginxaas-azure/quickstart/basic-caching.md). ```nginx proxy_cache_path /var/cache/nginx/jwt levels=1 keys_zone=jwk:64k max_size=1m; ``` c. Enable `sync` for the keyval memory zones and specify the state files to persist the current state across NGINX restarts. The state file paths are subject to [NGINX Filesystem Restrictions table](/nginxaas-azure/getting-started/nginx-configuration/overview/#nginx-filesystem-restrictions) and must be placed in a directory accessible to the NGINX worker processes. ```nginx keyval_zone zone=oidc_id_tokens:1M state=/opt/oidc_id_tokens.json timeout=1h sync; keyval_zone zone=oidc_access_tokens:1M state=/opt/oidc_access_tokens.json timeout=1h sync; keyval_zone zone=refresh_tokens:1M state=/opt/refresh_tokens.json timeout=8h sync; keyval_zone zone=oidc_pkce:128K timeout=90s sync; # Temporary storage for PKCE code verifier. ``` - `openid_connect.server_conf`: Remove the `location /api/` block, since NGINXaaS for Azure currently restricts access to the `api` directive. ```nginx location /api/ { api write=on; allow 127.0.0.1; # Only the NGINX host may call the NGINX Plus API deny all; access_log off; } ``` - Modify the root config file `nginx.conf` properly with `frontend.conf` content: a. Add `load_module modules/ngx_http_js_module.so;` near the top of the root config file, if it doesn't exist. b. Add `include conf.d/openid_connect_configuration.conf;` in the http block before the server block. #### Configuration example with localhost as an upstream server ```nginx load_module modules/ngx_http_js_module.so; http { # This is the backend application we are protecting with OpenID Connect upstream my_backend { zone my_backend 64k; # Reuse the localhost as a upstream server # Modify to the real upstream server address if you have server 127.0.0.1; } # A local server block representing the upstream server for testing only # Remove if you have the real upstream servers server { listen 80; default_type text/html; return 200 '

This is a site protected by OIDC!

\n'; } # Custom log format to include the 'sub' claim in the REMOTE_USER field log_format main_jwt '$remote_addr - $jwt_claim_sub [$time_local] "$request" $status ' '$body_bytes_sent "$http_referer" "$http_user_agent" "$http_x_forwarded_for"'; # The frontend server - reverse proxy with OpenID Connect authentication # include conf.d/openid_connect_configuration.conf; server { include conf.d/openid_connect.server_conf; # Authorization code flow and Relying Party processing error_log /var/log/nginx/error.log debug; # Reduce severity level as required listen 443 ssl; # Use SSL/TLS in production ssl_certificate /etc/nginx/ssl/my-cert.crt; ssl_certificate_key /etc/nginx/ssl/my-cert.key; location / { # This site is protected with OpenID Connect auth_jwt "" token=$session_jwt; error_page 401 = @do_oidc_flow; #auth_jwt_key_file $oidc_jwt_keyfile; # Enable when using filename auth_jwt_key_request /_jwks_uri; # Enable when using URL # Successfully authenticated users are proxied to the backend, # with 'sub' claim passed as HTTP header proxy_set_header username $jwt_claim_sub; # Bearer token is uses to authorize NGINX to access protected backend #proxy_set_header Authorization "Bearer $access_token"; # Intercept and redirect "401 Unauthorized" proxied responses to nginx # for processing with the error_page directive. Necessary if Access Token # can expire before ID Token. #proxy_intercept_errors on; proxy_pass http://my_backend; # The backend site/app access_log /var/log/nginx/access.log main_jwt; } } } stream { # Add localhost resolver for internal clustering hostname with resolver metrics collection resolver 127.0.0.1:49153 valid=20s status_zone=stream_resolver_zone1; server { listen 9000; zone_sync; zone_sync_server internal.nginxaas.nginx.com:9000 resolve; } } ``` 3. Upload the NGINX configurations. See [Upload an NGINX configuration](/nginxaas-azure/getting-started/nginx-configuration/) for more details. 4. In a web browser, open `https:///`. The browser will be redirected to the IdP server. After a successful login using the credentials of a user who has the authorization, the protected URI can be accessed. For example, using the `nginx.conf` in this guide, open `https:///` and complete the authentication. The browser will show: ```text This is a site protected by OIDC! ``` ## Limitations of Native OIDC vs NJS The Native OIDC implementation has the following limitations compared to the NJS-based implementation: - Proof Key for Code Exchange (PKCE) Support - Front-Channel Logout - Back-Channel Logout These features will be added in future releases. ## Troubleshooting [Enable NGINX logs](/nginxaas-azure/monitoring/enable-logging/) and [Troubleshooting](https://github.com/nginxinc/nginx-openid-connect/tree/main?tab=readme-ov-file#troubleshooting) the OIDC issues. ## Monitoring [Enable metrics](/nginxaas-azure/monitoring/enable-metrics.md), check [real time monitoring](https://github.com/nginxinc/nginx-openid-connect/blob/main/README.md#real-time-monitoring) to see how OIDC metrics are collected, and use "plus.http.*" metrics filtered with location_zone dimension in [NGINX requests and response statistics](/nginxaas-azure/monitoring/metrics-catalog.md#nginx-requests-and-response-statistics) to check the OIDC metrics. ## See Also - [Microsoft identity platform documentation](https://learn.microsoft.com/en-us/entra/identity-platform/) - [NGINX Plus Native OIDC Module Reference documentation](https://nginx.org/en/docs/http/ngx_http_oidc_module.html) - [Single Sign-On with Microsoft Entra ID](/nginx/deployment-guides/single-sign-on/entra-id.md) - [Single Sign-On with OpenID Connect and Identity Providers](nginx/admin-guide/security-controls/configuring-oidc.md) - [Terraform snippets for sample Native OIDC](https://github.com/nginxinc/nginxaas-for-azure-snippets/tree/main/terraform/configurations/native-oidc) -------------------------------------------------------------------------------- title: "Troubleshooting guide" url: https://docs.nginx.com/nginxaas-azure/get-help/troubleshooting/index.md -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- title: "Upload using Terraform" url: https://docs.nginx.com/nginxaas-azure/getting-started/nginx-configuration/nginx-configurations-terraform/index.md -------------------------------------------------------------------------------- ## Overview F5 NGINXaaS for Azure (NGINXaaS) configurations can be managed using Terraform. This document outlines common Terraform workflows for NGINXaaS. ## Prerequisites - Confirm that you meet the [NGINXaaS Prerequisites](/nginxaas-azure/getting-started/prerequisites.md). - [Authenticate Terraform to Azure](https://learn.microsoft.com/en-us/azure/developer/terraform/authenticate-to-azure) - [Install Terraform](https://learn.hashicorp.com/tutorials/terraform/install) **Note:** The examples in the NGINXaaS for Azure Snippets GitHub repository use the prerequisites module [available in the same repository](https://github.com/nginxinc/nginxaas-for-azure-snippets/tree/main/terraform/prerequisites). ## Upload an NGINX configuration You can find examples of Terraform configurations in the [NGINXaaS for Azure Snippets GitHub repository](https://github.com/nginxinc/nginxaas-for-azure-snippets/tree/main/terraform/configurations) To create a deployment and add a configuration, run the following commands: ```shell terraform init terraform plan terraform apply --auto-approve ``` ## Manage an NGINX configuration NGINX configuration files are uploaded and returned as base64 encoded data. We recommend using git or other version control systems to view human-readable differences between configuration files during `terraform plan`. Alternatively, you can decode the file contents to view the whole file. For example, ```shell $ terraform plan ... - config_file { - content = "aHR0cCB7CiAgICBzZXJ2ZXIgewogICAgICAgIGxvY2F0aW9uIC8gewogICAgICAgICAgICByZXR1cm4gMjAwICJIZWxsbyI7CiAgICAgICAgfQogICAgfQoK" -> null - virtual_path = "nginx.conf" -> null } + config_file { + content = "aHR0cCB7CiAgICBzZXJ2ZXIgewogICAgICAgIGxvY2F0aW9uIC8gewogICAgICAgICAgICByZXR1cm4gMjAwICJIZWxsbyBXb3JsZCEiOwogICAgICAgIH0KICAgIH0KfQoK" + virtual_path = "nginx.conf" } ... ``` ``` $ echo aHR0cCB7CiAgICBzZXJ2ZXIgewogICAgICAgIGxvY2F0aW9uIC8gewogICAgICAgICAgICByZXR1cm4gMjAwICJIZWxsbyBXb3JsZCEiOwogICAgICAgIH0KICAgIH0KfQoK | base64 --decode http { server { location / { return 200 "Hello World!"; } } } ``` ## Delete a deployment Once the deployment is no longer needed, run the following to clean up the deployment and related resources: ```shell terraform destroy --auto-approve ``` ## Additional resources - [Terraform NGINX configuration documentation](https://registry.terraform.io/providers/hashicorp/azurerm/3.97.0/docs/resources/nginx_configuration) - [NGINXaaS Managed Identity Documentation](/nginxaas-azure/getting-started/managed-identity-portal.md) - [NGINXaaS Azure Monitor Documentation](/nginxaas-azure/monitoring/enable-metrics.md) **Note:** See the [NGINX connfiguration overview](overview.md) topic to learn more about: - [NGINX configuration automation workflows](overview.md#nginx-configuration-automation-workflows) - [NGINX filesystem restrictions](overview.md#nginx-filesystem-restrictions) - [Disallowed configuration directives](overview.md#disallowed-configuration-directives) - [Directives that cannot be overridden](overview.md#directives-that-cannot-be-overridden) - [Configuration directives list](overview.md#configuration-directives-list) -------------------------------------------------------------------------------- title: "IP Intelligence" url: https://docs.nginx.com/nginxaas-azure/app-protect/ip-intelligence/index.md -------------------------------------------------------------------------------- ## Overview F5 WAF for NGINX includes an IP Intelligence feature that allows you to customize enforcement based on the source IP address of a request. Using IP Intelligence, you can block or log requests from IP addresses associated with known threat categories such as botnets, scanners, and phishing proxies. IP Intelligence is available on NGINXaaS for Azure deployments with the **Standard v3** [plan](/nginxaas-azure/billing/overview.md/#standard-v3-plan). **Note:** IP Intelligence does not require an additional license. F5's existing license with the BrightCloud threat intelligence provider covers all NGINXaaS deployments. The IP address database is automatically updated every 60 minutes. ## Threat categories IP Intelligence classifies IP addresses into the following threat categories. You can individually configure each category to block, alarm, or allow traffic. | Category | Description | |----------------------|-----------------------------------------------------------------------------| | Anonymous Proxy | IP addresses associated with anonymous proxy services | | BotNets | IP addresses that are part of known botnet networks | | Cloud-based Services | IP addresses from cloud providers often used for automated attacks | | Denial of Service | IP addresses associated with denial-of-service attacks | | Infected Sources | IP addresses from hosts known to be compromised | | Mobile Threats | IP addresses associated with malicious mobile applications | | Phishing Proxies | IP addresses used as proxies for phishing campaigns | | Scanners | IP addresses associated with network and vulnerability scanners | | Spam Sources | IP addresses identified as sources of spam | | Tor Proxies | IP addresses of known Tor exit nodes | | Web Attacks | IP addresses associated with web-based attacks | | Windows Exploits | IP addresses associated with Windows-specific exploits | Since the threat database is continuously updated, enforcement may change over time. IP addresses may be added, removed, or moved between categories based on their reported activity. ## Add IP Intelligence to a WAF policy To use IP Intelligence, you must add the `ip-intelligence` section to a [custom WAF policy](/nginxaas-azure/app-protect/configure-waf.md#custom-policies). No additional setup or enablement steps are required. Your WAF policy needs two additions: 1. The `VIOL_MALICIOUS_IP` violation in `blocking-settings`. 2. The `ip-intelligence` section with the desired threat categories. ### Example policy The following policy turns on IP Intelligence with all categories set to block and alarm. ```json { "policy": { "name": "ip_intelligence_policy", "template": { "name": "POLICY_TEMPLATE_NGINX_BASE" }, "applicationLanguage": "utf-8", "enforcementMode": "blocking", "blocking-settings": { "violations": [ { "name": "VIOL_MALICIOUS_IP", "alarm": true, "block": true } ] }, "ip-intelligence": { "enabled": true, "ipIntelligenceCategories": [ { "category": "Anonymous Proxy", "alarm": true, "block": true }, { "category": "BotNets", "alarm": true, "block": true }, { "category": "Cloud-based Services", "alarm": true, "block": true }, { "category": "Denial of Service", "alarm": true, "block": true }, { "category": "Infected Sources", "alarm": true, "block": true }, { "category": "Mobile Threats", "alarm": true, "block": true }, { "category": "Phishing Proxies", "alarm": true, "block": true }, { "category": "Scanners", "alarm": true, "block": true }, { "category": "Spam Sources", "alarm": true, "block": true }, { "category": "Tor Proxies", "alarm": true, "block": true }, { "category": "Web Attacks", "alarm": true, "block": true }, { "category": "Windows Exploits", "alarm": true, "block": true } ] } } } ``` - `"block": true` rejects requests from matching IP addresses. - `"alarm": true` logs matching requests in the security logs. You can customize each category independently — for example, blocking botnets while only alarming on scanners. You can reference this policy in your NGINX configuration using the `app_protect_policy_file` directive, as described in [Configure F5 WAF for NGINX](/nginxaas-azure/app-protect/configure-waf.md). For the full policy configuration reference, see the official [IP Intelligence documentation](https://docs.nginx.com/waf/policies/ip-intelligence/#configure-policies-for-ip-intelligence). -------------------------------------------------------------------------------- title: "OIDC with Microsoft Entra ID in Private Subnets" url: https://docs.nginx.com/nginxaas-azure/quickstart/security-controls/private-subnet-oidc-entra/index.md -------------------------------------------------------------------------------- ## Overview Learn how to configure F5 NGINXaaS for Azure with OpenID Connect (OIDC) authentication using Microsoft Entra ID when your NGINXaaS deployment is in a private subnet. This guide addresses the networking requirements to enable authentication traffic to reach Microsoft Entra ID endpoints while maintaining security controls. When NGINXaaS is deployed in a private subnet, authentication traffic must reach external Microsoft Entra ID endpoints at `login.microsoftonline.com`. This guide provides the following solutions to enable this connectivity while controlling outbound traffic: 1. **Azure NAT Gateway with NSG rules** - Lower cost, simpler configuration 1. **Azure Firewall** - Higher security, more granular control ## Before you begin To complete this guide, you need to set up the following: - [An NGINXaaS deployment](/nginxaas-azure/getting-started/create-deployment) with a private IP address - A configured Microsoft Entra ID application registration. See [Configure Entra ID](/nginx/deployment-guides/single-sign-on/entra-id/#entra-setup) for detailed setup instructions. - [SSL/TLS certificates](/nginxaas-azure/getting-started/ssl-tls-certificates/) configured for your NGINXaaS deployment - [Runtime State Sharing](/nginxaas-azure/quickstart/runtime-state-sharing.md) enabled on the NGINXaaS deployment ## Solution comparison Choose the networking solution that best fits your security and cost requirements: | Feature | Azure NAT Gateway | Azure Firewall | |---------|-------------------|----------------| | **Hourly cost** | Lower - see [NAT Gateway pricing](https://azure.microsoft.com/en-us/pricing/details/azure-nat-gateway/) | Higher - see [Firewall pricing](https://azure.microsoft.com/en-us/pricing/details/azure-firewall/) | | **Address space** | Less required | More required (2 additional /26 subnets) | | **Security** | Less secure - broader IP range filtering | More secure - precise FQDN-based filtering | | **Configuration** | Simple setup, less overhead | More complex configuration, more overhead | | **Public IPs** | 1 required | 2 required | | **Filtering precision** | Generic rules enabling broader filtering | Surgical precision for traffic filtering | ## Common configuration steps Both solutions require these initial steps: ### Configure OIDC in NGINXaaS 1. Follow the standard [OIDC authentication](/nginxaas-azure/quickstart/security-controls/oidc/) guide with these Microsoft Entra ID specific considerations. 1. Set the `oidc_jwt_keyfile` endpoint in your `openid_connect_configuration.conf`: ```nginx # Use the correct Microsoft Entra ID keys endpoint map $host $oidc_jwt_keyfile { default "https://login.microsoftonline.com//discovery/v2.0/keys"; } ``` **Note:** The `oidc_jwt_keyfile` endpoint is not listed in the Microsoft App Registration's endpoints pane but is required for proper OIDC configuration. 1. Configure DNS resolution appropriately: - For dual-stack subnets, ensure both IPv4 and IPv6 address spaces are configured - Use Azure DNS (127.0.0.1:49153) or configure firewall rules for your preferred DNS service ```nginx http { # For IPv4-only deployments resolver 127.0.0.1:49153 ipv4=on ipv6=off valid=300s; # For dual-stack deployments resolver 127.0.0.1:49153 ipv4=on valid=300s; } ``` **Note:** If you plan to use IPv6 addresses on the frontend, ensure your subnet is dual-stack with both IPv4 and IPv6 address spaces. For IPv4-only deployments, set `ipv6=off` in your resolver configuration. ## Configure connectivity using Azure NAT Gateway This solution uses Azure NAT Gateway with Network Security Group (NSG) rules to enable controlled outbound connectivity. ### Configure NSG rules When you create an NGINXaaS deployment, Azure automatically creates and attaches an NSG to the delegated subnet. You need to modify this NSG to allow Microsoft Entra ID connectivity. 1. Add Microsoft IP address ranges to NSG rules: Navigate to your NGINXaaS subnet's NSG and add inbound and outbound rules for the Microsoft IP addresses listed in the [Microsoft 365 URLs and IP address ranges documentation](https://learn.microsoft.com/en-us/microsoft-365/enterprise/urls-and-ip-address-ranges?view=o365-worldwide#microsoft-365-common-and-office-online) under section #56. 1. Create a custom outbound rule to deny general internet access: - **Priority**: Higher numerical value (lower priority) than the Microsoft IP rules - **Action**: Deny - **Destination**: Internet - **Purpose**: Override the default `AllowInternetOutBound` rule **Note:** The default `AllowInternetOutBound` rule cannot be edited, so you must create a higher-priority rule to deny general internet access while allowing the specific Microsoft IP ranges. The priority of the custom deny rule should be a higher numerical value (lower priority) than the Microsoft IP allow rules. ### Create and configure Azure NAT Gateway 1. Create an Azure NAT Gateway: ```bash # Create Azure NAT Gateway az network nat gateway create \ --resource-group \ --name \ --location \ --public-ip-addresses ``` 1. Associate the Azure NAT Gateway with your NGINXaaS subnet: ```bash # Associate Azure NAT Gateway with subnet az network vnet subnet update \ --resource-group \ --vnet-name \ --name \ --nat-gateway ``` This configuration allows NGINXaaS to reach Microsoft Entra ID endpoints while blocking general internet access. **Note:** Using Azure NAT Gateway with NSG rules still requires allowing broad IP address ranges. Based on Microsoft's documentation, you need to allow at least two /18 subnets and two /19 subnets for complete Microsoft Entra ID connectivity. For more precise filtering, consider using Azure Firewall instead. ## Configure connectivity using Azure Firewall This solution provides more granular control using Azure Firewall with DNS-based filtering. **Note:** Azure Firewall provides DNS-based filtering capabilities but comes at a significantly higher cost compared to Azure NAT Gateway (approximately 28x cost increase). However, it enables more precise firewall rules for better security. ### Create firewall subnets Create two new subnets in your virtual network: 1. **Azure Firewall subnet**: - Name: `AzureFirewallSubnet` (immutable) - Address space: /26 subnet - Purpose: Azure Firewall 1. **Firewall Management subnet**: - Name: `AzureFirewallManagementSubnet` (immutable) - Address space: /26 subnet - Purpose: Firewall Management ```bash # Create Azure Firewall subnet az network vnet subnet create \ --resource-group \ --vnet-name \ --name AzureFirewallSubnet \ --address-prefixes # Create Firewall Management subnet az network vnet subnet create \ --resource-group \ --vnet-name \ --name AzureFirewallManagementSubnet \ --address-prefixes ``` ### Create Azure Firewall 1. Create the firewall with Standard SKU (required for DNS proxy functionality): ```bash # Create public IPs for firewall az network public-ip create \ --name \ --resource-group \ --allocation-method Static \ --sku Standard az network public-ip create \ --name \ --resource-group \ --allocation-method Static \ --sku Standard # Create firewall with new policy az extension add --name azure-firewall az network firewall create \ --name \ --resource-group \ --vnet-name \ --public-ip \ --firewall-policy ``` **Note:** The Standard SKU is required at minimum because it allows Azure Firewall to be configured as a DNS proxy, which is necessary for FQDN-based filtering. During creation, choose to create a new Firewall Policy and use your existing virtual network that contains the NGINXaaS subnet. ### Configure firewall policy 1. Enable DNS proxy in the firewall policy: ```bash # Enable DNS proxy az network firewall policy update \ --name \ --resource-group \ --enable-dns-proxy true \ --dns-servers 168.63.129.16 ``` 1. Configure private IP ranges to avoid SNAT for internal traffic: Navigate to your firewall policy in the Azure portal and under **Private IP ranges**, select "Always" or specify your NGINXaaS subnet IP addresses. ### Create network rules Create a network rule collection to allow NGINXaaS subnet access to Microsoft Entra ID: ```bash # Create network rule collection az network firewall policy rule-collection-group create \ --name NetworkRuleCollectionGroup \ --policy-name \ --resource-group \ --priority 200 # Add network rule for Microsoft Entra ID az network firewall policy rule-collection-group collection add-filter-collection \ --name EntraIDAccess \ --policy-name \ --resource-group \ --collection-priority 100 \ --rule-collection-group-name NetworkRuleCollectionGroup \ --action Allow \ --rule-name AllowEntraID \ --rule-type NetworkRule \ --protocols TCP \ --source-addresses \ --destination-fqdns login.microsoftonline.com \ --destination-ports 443 ``` ### Configure route table Direct NGINXaaS subnet traffic through the firewall: 1. Note the private IP address of your Azure Firewall (found in the firewall's overview page). 1. Create a route table: ```bash # Create route table az network route-table create \ --name \ --resource-group \ --location # Add default route pointing to firewall az network route-table route create \ --route-table-name \ --resource-group \ --name DefaultRoute \ --address-prefix 0.0.0.0/0 \ --next-hop-type VirtualAppliance \ --next-hop-ip-address ``` 1. Associate the route table with the NGINXaaS subnet: ```bash # Associate route table with NGINXaaS subnet az network vnet subnet update \ --resource-group \ --vnet-name \ --name \ --route-table ``` ## Testing the configuration After implementing either solution, test the OIDC authentication: 1. Access your NGINXaaS deployment URL. 1. Verify you are redirected to Microsoft Entra ID for authentication. 1. Complete the login process and confirm successful authentication. 1. Check NGINXaaS logs for any connectivity issues. ### Troubleshooting If authentication fails, check the following: 1. **DNS Resolution**: Ensure your firewall rules allow DNS queries. 1. **Certificate Validation**: Verify that the firewall allows HTTPS traffic to Microsoft endpoints. 1. **Timeout Settings**: Increase timeout values if experiencing slow authentication responses. 1. **Route Configuration**: Confirm the route table is properly associated with the NGINXaaS subnet. ## To secure your systems, address the following: - **Principle of Least Privilege**: Both solutions limit outbound connectivity to only required Microsoft endpoints. - **Monitoring**: Implement logging and monitoring for authentication traffic. - **Regular Updates**: Keep Microsoft IP address ranges updated in your NSG rules. - **Network Segmentation**: Consider additional network segmentation for enhanced security. ## To optimize your systems, we recommend: - **Azure NAT Gateway**: More cost-effective for basic filtering needs. - **Azure Firewall**: Better ROI when you need advanced filtering capabilities. - **Public IP Management**: Minimize the number of public IP addresses to reduce costs. Both solutions enable the minimal connectivity required between NGINXaaS and Microsoft Entra ID for OIDC authentication while maintaining security controls appropriate to your requirements. ## See also - [Set up OIDC authentication](/nginxaas-azure/quickstart/security-controls/oidc/) - [Single Sign-On with Microsoft Entra ID](/nginx/deployment-guides/single-sign-on/entra-id.md) - [Private Link to Upstreams](/nginxaas-azure/quickstart/security-controls/private-link-to-upstreams.md) - [Microsoft 365 URLs and IP address ranges](https://learn.microsoft.com/en-us/microsoft-365/enterprise/urls-and-ip-address-ranges) -------------------------------------------------------------------------------- title: "Connect to upstreams with Azure Private Link" url: https://docs.nginx.com/nginxaas-azure/quickstart/security-controls/private-link-to-upstreams/index.md -------------------------------------------------------------------------------- [Azure Private Link](https://learn.microsoft.com/en-us/azure/private-link/private-link-overview) eliminates exposure to the public internet by handling traffic over Microsoft's backbone network. This is especially useful if your NGINXaaS deployment and your upstreams are in different virtual networks. **Note:** Depending on your use-case, we recommend using [virtual network peering](https://learn.microsoft.com/en-us/azure/virtual-network/virtual-network-peering-overview) instead of a Private Link service to maintain NGINX's load balancing capabilities. To set up a Private Link between your NGINXaaS deployment and your upstreams, you'll need two resources: - [Private Link service](https://learn.microsoft.com/en-us/azure/private-link/private-link-service-overview) - [private endpoint](https://learn.microsoft.com/en-us/azure/private-link/private-endpoint-overview) ## Create a Private Link service A Private Link service is an Azure resource that enables Private Link access to your application. If your upstream is an Azure PaaS service (for example, Azure Storage), then you do not need a Private Link service. To create a Private Link service, 1. Configure your upstream to run behind a Standard Load Balancer. 1. Add load balacing rules per upstream server port. 1. Create a Private Link service and attach it to the Standard Load Balancer. The following example demonstrates this process using an existing virtual machine as the upstream. #### Create a Private Link service with the CLI ### Prerequisites - Resource Group - Virtual Network - Subnet - Workload Virtual Machine Please ensure the following environment variables are exported before copying the below Azure CLI commands. | Name | Description | |------------------ | ----------------- | | APP_LOCATION | Location of the resource group | APP_RESOURCE_GROUP | Name of the resource group the virtual machine is in | | APP_VNET_NAME | Name of the virtual network the virtual machine is in | | APP_SUBNET_NAME | Name of the subnet the virtual machine is in | | APP_VM_NAME | Name of the workload virtual machine | | APP_NIC_NAME | Name of the network interface of the virtual machine | | APP_IP_CONFIG_NAME | Name of the IP configuration associated with the NIC | ### Create a load balancer ```shell $ az network lb create \ --resource-group $APP_RESOURCE_GROUP \ --name load-balancer \ --sku Standard \ --vnet-name $APP_VNET_NAME \ --subnet $APP_SUBNET_NAME \ --frontend-ip-name frontend \ --backend-pool-name backend-pool ``` ### Create health probes and load balancing rules Depending on your NGINX configuration, you will need to add a load balancing rule and health probe for each port your upstream servers are listening on. For example, given the following NGINX configuration snippet, ```nginx upstream { server 10.0.1.4:8000; } ``` Create a health probe monitoring on port `8000`: ```shell $ az network lb probe create \ --resource-group $APP_RESOURCE_GROUP \ --lb-name load-balancer \ --name 8000-probe \ --protocol tcp \ --port 8000 ``` Create a load balancing rule listening on port `8000`: ```shell $ az network lb rule create \ --resource-group $APP_RESOURCE_GROUP \ --lb-name load-balancer \ --name 8000-rule \ --protocol tcp \ --frontend-port 8000 \ --backend-port 8000 \ --frontend-ip-name frontend \ --backend-pool-name backend-pool \ --probe-name 8000-probe \ --idle-timeout 15 \ --enable-tcp-reset true ``` ### Configure the workload VM behind the load balancer ```shell $ az network nic ip-config address-pool add \ --address-pool backend-pool \ --ip-config-name $APP_IP_CONFIG_NAME \ --nic-name $APP_NIC_NAME \ --resource-group $APP_RESOURCE_GROUP \ --lb-name load-balancer ``` ### Disable network policy The `privateLinkServiceNetworkPolicies` setting must be disabled to add a private link service in a virtual network. ```shell $ az network vnet subnet update \ --name $APP_SUBNET_NAME \ --vnet-name $APP_VNET_NAME \ --resource-group $APP_RESOURCE_GROUP \ --disable-private-link-service-network-policies yes ``` ### Create a private link service ```shell $ az network private-link-service create \ --resource-group $APP_RESOURCE_GROUP \ --name private-link-service \ --vnet-name $APP_VNET_NAME \ --subnet $APP_SUBNET_NAME \ --lb-name load-balancer \ --lb-frontend-ip-configs frontend \ --location $APP_LOCATION ``` ## Create a private endpoint A private endpoint is a network interface that connects to a service powered by Azure Private Link. To connect your NGINXaaS to your upstreams using a private endpoint, 1. Add a new, non-delegated subnet in your NGINXaaS deployment's virtual network. 1. Create a private endpoint. 1. Update your NGINX configuration to reference the private endpoint. The following example demonstrates this process using an existing NGINXaaS deployment and a Private Link service. #### Create a private endpoint with the CLI ### Prerequisites - Resource Group - Virtual Network - NGINXaaS deployment - Private Link service Please ensure the following environment variables are exported before copying the below Azure CLI commands. | Name | Description | |------------------ | ----------------- | | DEP_RESOURCE_GROUP | Name of the resource group the NGINXaaS deployment is in | | DEP_VNET_NAME | Name of the virtual network the NGINXaaS deployment is in | | PRIVATE_ENDPOINT_SUBNET_ADDRESS_SPACE | Desired address space of the private endpoint's subnet | | PRIVATE_LINK_SERVICE_ID | Resource ID of the Private Link service | ### Create a new subnet You must create a new subnet for the private endpoint because the existing NGINXaaS deployment's subnet is already delegated. ```shell $ az network vnet subnet create \ --resource-group $DEP_RESOURCE_GROUP \ --vnet-name $DEP_VNET_NAME \ --name subnet-priv-endpoint \ --address-prefix $PRIVATE_ENDPOINT_SUBNET_ADDRESS_SPACE ``` ### Create a private endpoint ```shell $ az network private-endpoint create \ --connection-name connection-1 \ --name private-endpoint \ --private-connection-resource-id $PRIVATE_LINK_SERVICE_ID \ --resource-group $DEP_RESOURCE_GROUP \ --subnet subnet-priv-endpoint \ --manual-request false \ --vnet-name $DEP_VNET_NAME ``` ### Update your NGINXaaS configuration with the private endpoint's IP address First, get the IP address of the private endpoint: ```shell $ export nic_id=$(az network private-endpoint show \ --resource-group $DEP_RESOURCE_GROUP \ --name private-endpoint \ --query "networkInterfaces[0].id" \ --output tsv) $ az network nic show \ --ids $nic_id \ --query "ipConfigurations[0].privateIPAddress" \ --output tsv ``` Then, reference it in your NGINX configuration's upstream servers. For example: ```nginx upstream { server :8000; } ``` ## Additional Resources The following guides provide step-by-step instructions to create a Private Link service and a private endpoint with your preferred client tool: * [Azure portal](https://learn.microsoft.com/en-us/azure/private-link/create-private-link-service-portal?tabs=dynamic-ip) * [Azure CLI](https://learn.microsoft.com/en-us/azure/private-link/create-private-link-service-cli) * [ARM template](https://learn.microsoft.com/en-us/azure/private-link/create-private-link-service-template) -------------------------------------------------------------------------------- title: "Disable F5 WAF for NGINX" url: https://docs.nginx.com/nginxaas-azure/app-protect/disable-waf/index.md -------------------------------------------------------------------------------- ## Overview This guide explains how to disable F5 WAF for NGINX on an NGINX as a Service for Azure (NGINXaaS) deployment. ## Before you start You must remove the WAF directives from your NGINX config file before attempting to disable WAF. ## Disable F5 WAF for NGINX ### Using the Microsoft Azure Portal Access the [Microsoft Azure portal](https://portal.azure.com) 1. Go to your NGINXaaS for Azure deployment. 2. Select F5 WAF for NGINX in the left menu. 3. Select **Disable**. -------------------------------------------------------------------------------- title: "Metrics catalog" url: https://docs.nginx.com/nginxaas-azure/monitoring/metrics-catalog/index.md -------------------------------------------------------------------------------- F5 NGINXaaS for Azure (NGINXaaS) provides a rich set of metrics that you can use to monitor the health and performance of your NGINXaaS deployment. This document provides a catalog of the metrics that are available for monitoring NGINXaaS for Azure in Azure Monitor. ## Available metrics - [Available metrics](#available-metrics) - [Metrics](#metrics) - [NGINXaaS statistics](#nginxaas-statistics) - [NGINX connections statistics](#nginx-connections-statistics) - [NGINX requests and response statistics](#nginx-requests-and-response-statistics) - [NGINX SSL statistics](#nginx-ssl-statistics) - [NGINX cache statistics](#nginx-cache-statistics) - [NGINX worker statistics](#nginx-worker-statistics) - [NGINX upstream statistics](#nginx-upstream-statistics) - [NGINX system statistics](#nginx-system-statistics) - [NGINX stream statistics](#nginx-stream-statistics) - [NGINX resolver statistics](#nginx-resolver-statistics) ## Metrics The following metrics are reported on by NGINXaaS for Azure in Azure Monitor. The metrics are categorized by the namespace used in Azure Monitor. The dimensions allow you to filter or split your queries in Azure Monitor providing you with a granular view over the metrics reported ### NGINXaaS statistics | **Metric** | **Display Name** | **Dimensions** | **Type** | **Description** | **Roll-up per** | | --------------------- | --------------------------- | -------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------- | | ncu.provisioned | NCU provisioned | | count | The number of successfully provisioned NCUs during the aggregation interval. During scaling events, this may lag behind `ncu.requested` as the system works to achieve the request. Available for Standard plan(s) only. | deployment | | ncu.requested | NCU requested | | count | The requested number of NCUs during the aggregation interval. Describes the goal state of the system. Available for Standard plans(s) only. | deployment | | nginxaas.capacity.percentage | NGINXaaS capacity percentage | | count | The percentage of the deployment's total capacity being used. This can be used to guide scaling your workload. See [Scaling Guidance](/nginxaas-azure/quickstart/scaling.md#iterative-approach) for details. Available for Standard plan(s) only. | deployment | | system.worker_connections | Worker connections | pid process_name | count | The number of nginx worker connections used on the dataplane. This metric is one of the factors which determines the deployment's consumed NCU value. | deployment | | nginxaas.certificates | Certificates | name status | count | The number of certificates added to the NGINXaaS deployment dimensioned by the name of the certificate and its status. Refer to [Certificate Health](/nginxaas-azure/getting-started/ssl-tls-certificates/overview.md#monitor-certificates) to learn more about the status dimension. | deployment | | nginxaas.maxmind | Maxmind status | status | count | The status of any MaxMind license in use for downloading geoip2 databases. Refer to [License Health](/nginxaas-azure/quickstart/geoip2.md#monitoring) to learn more about the status dimension. | deployment | | waf.enabled | Web application firewall enabled | | count | Current status of Web Application Firewall on the deployment. | deployment | | ports.used | Ports used | | count | The number of listen ports used by the deployment during the aggregation interval. | deployment | | system.listener_backlog.max | Max listener backlog | listen_addr, file_desc | count | The fullness (expressed as a fraction) of the fullest backlog queue. | deployment | | system.listener_backlog.queue_limit | Listener backlog queue limit | listen_address, file_desc | count | The capacity of a specific backlog queue, labeled by listen address. | deployment | | system.listener_backlog.length | Listener backlog length | listen_address, file_desc | count | The number of items in a specific backlog queue, labeled by listen address. | deployment | **Note:** The `ncu.consumed` metric is now deprecated and is on the path to retirement. Please change any alerting on this metric to use the new Capacity Percentage metric. ### NGINX connections statistics | **Metric** | **Display Name** | **Dimensions** | **Type** | **Description** | **Roll-up per** | |------------------------------|------------------|----------------|----------|---------------------------------------------------------------------------------------------------------------|-----------------| | nginx.conn.accepted | Accepted connections | build version | count | Accepted Connections The total number of accepted client connections during the aggregation interval. | deployment | | nginx.conn.dropped | Dropped connections | build version | count | Dropped Connections The total number of dropped client connections during the aggregation interval. | deployment | | nginx.conn.active | Active connections | build version | count | Active Connections The average number of active client connections during the aggregation interval. | deployment | | nginx.conn.idle | Idle connections | build version | count | Idle Connections The average number of idle client connections during the aggregation interval. | deployment | | nginx.conn.current | Current connections | build version | count | Current Connections The average number of active and idle client connections during the aggregation interval. | deployment | ### NGINX requests and response statistics | **Metric** | **Display Name** | **Dimensions** | **Type** | **Description** | **Roll-up per** | |----------------------------------------|------------------|-----------------------------|-------|-----------------------------------------------------------------------------------------------------------------------------|---------------| | nginx.http.request.count | Total HTTP requests | build version | count | HTTP Requests The total number of HTTP requests during the aggregation interval. | deployment | | nginx.http.request.current | Current HTTP requests | build version | count | Current Requests The number of current requests during the aggregation interval. | deployment | | nginx.http.limit_conns.passed | HTTP limit conn passed | build version limit_conn_zone | count | Limit Conn Zone Passed HTTP Connections The total number of connections that were neither limited nor accounted as limited during the aggregation interval. | limit conn zone | | nginx.http.limit_conns.rejected | HTTP limit conn rejected | build version limit_conn_zone | count | Limit Conn Zone Rejected HTTP Connections The total number of connections that were rejected during the aggregation interval. | limit conn zone | | nginx.http.limit_conns.rejected_dry_run| HTTP limit conn rejected dry-run | build version limit_conn_zone | count | Limit Conn Zone Rejected HTTP Connections In The Dry Run Mode The total number of connections accounted as rejected in the dry run mode during the aggregation interval. | limit conn zone | | nginx.http.limit_reqs.passed | HTTP limit requests passed | build version limit_req_zone | count | Limit Req Zone Passed HTTP Requests Rate The total number of requests that were neither limited nor accounted as limited during the aggregation interval. | limit req zone | | nginx.http.limit_reqs.delayed | HTTP limit requests delayed | build version limit_req_zone | count | Limit Req Zone Delayed HTTP Requests Rate The total number of requests that were delayed during the aggregation interval. | limit req zone | | nginx.http.limit_reqs.rejected | HTTP limit requests rejected | build version limit_req_zone | count | Limit Req Zone Rejected HTTP Requests Rate The total number of requests that were rejected during the aggregation interval. | limit req zone | | nginx.http.limit_reqs.delayed_dry_run | HTTP limit requests delayed dry-run | build version limit_req_zone | count | Limit Req Zone Delayed HTTP Requests Rate In The Dry Run Mode The total number of requests accounted as delayed in the dry run mode during the aggregation interval. | limit req zone | | nginx.http.limit_reqs.rejected_dry_run | HTTP limit requests rejected dry-run | build version limit_req_zone | count | Limit Req Zone Rejected HTTP Requests Rate In The Dry Run Mode The total number of requests accounted as rejected in the dry run mode during the aggregation interval. | limit req zone | | plus.http.request.count | Server zone HTTP requests | build version server_zone | count | Server Zone HTTP Requests The total number of HTTP requests during the aggregation interval. | server zone | | plus.http.response.count | Server zone HTTP responses | build version server_zone | count | Server Zone HTTP Responses The total number of HTTP responses during the aggregation interval. | server zone | | plus.http.status.1xx | Server zone HTTP 1xx responses | build version server_zone | count | Server Zone HTTP 1xx Responses The total number of HTTP responses with a 1xx status code during the aggregation interval. | server zone | | plus.http.status.2xx | Server zone HTTP 2xx responses | build version server_zone | count | Server Zone HTTP 2xx Responses The total number of HTTP responses with a 2xx status code during the aggregation interval. | server zone | | plus.http.status.3xx | Server zone HTTP 3xx responses | build version server_zone | count | Server Zone HTTP 3xx Responses The total number of HTTP responses with a 3xx status code during the aggregation interval. | server zone | | plus.http.status.4xx | Server zone HTTP 4xx responses | build version server_zone | count | Server Zone HTTP 4xx Responses The total number of HTTP responses with a 4xx status code during the aggregation interval. | server zone | | plus.http.status.5xx | Server zone HTTP 5xx responses | build version server_zone | count | Server Zone HTTP 5xx Responses The total number of HTTP responses with a 5xx status code during the aggregation interval. | server zone | | plus.http.status.processing | Server zone HTTP status processing | build version server_zone | avg | Server Zone Status Processing The number of client requests that are currently being processed. | server zone | | plus.http.request.bytes_rcvd | Server zone HTTP bytes received | build version server_zone | count | Server Zone Bytes Received The total number of bytes received from clients during the aggregation interval. | server zone | | plus.http.request.bytes_sent | Server zone HTTP bytes sent | build version server_zone | count | Server Zone Bytes Sent The total number of bytes sent to clients during the aggregation interval. | server zone | | plus.http.request.location_zone.count | Location zone HTTP requests | build version location_zone | count | Location Zone HTTP Requests The total number of HTTP requests during the aggregation interval. | location zone | | plus.http.response.location_zone.count | Location zone HTTP responses | build version location_zone | count | Location Zone HTTP Responses The total number of HTTP responses in the aggregation interval. | location zone | | plus.http.status.location_zone.1xx | Location zone HTTP 1xx responses | build version location_zone | count | Location Zone HTTP 1xx Responses The total number of HTTP responses with a 1xx status code during the aggregation interval. | location zone | | plus.http.status.location_zone.2xx | Location zone HTTP 2xx responses | build version location_zone | count | Location Zone HTTP 2xx Responses The total number of HTTP responses with a 2xx status code during the aggregation interval. | location zone | | plus.http.status.location_zone.3xx | Location zone HTTP 3xx responses | build version location_zone | count | Location Zone HTTP 3xx Responses The total number of HTTP responses with a 3xx status code during the aggregation interval. | location zone | | plus.http.status.location_zone.4xx | Location zone HTTP 4xx responses | build version location_zone | count | Location Zone HTTP 4xx Responses The total number of HTTP responses with a 4xx status code during the aggregation interval. | location zone | | plus.http.status.location_zone.5xx | Location zone HTTP 5xx responses | build version location_zone | count | Location Zone HTTP 5xx Responses The total number of HTTP responses with a 5xx status code during the aggregation interval. | location zone | | plus.http.request.location_zone.bytes_rcvd | Location zone HTTP bytes received | build version location_zone | count | Location Zone Bytes Received The total number of bytes received from clients during the aggregation interval. | location zone | | plus.http.request.location_zone.bytes_sent | Location zone HTTP bytes sent | build version location_zone | count | Location Zone Bytes Sent The total number of bytes sent to clients during the aggregation interval. | location zone | ### NGINX SSL statistics | **Metric** | **Display Name** | **Dimensions** | **Type** | **Description** | **Roll-up per** | |----------------------------------------|------------------|-----------------------------|-------|-----------------------------------------------------------------------------------------------------------------------------|---------------| | plus.ssl.failed | Failed SSL handshakes | build version | count | The total number of failed SSL handshakes during the aggregation interval. | deployment | | plus.ssl.handshakes | Successful SSL handshakes | build version | count | The total number of successful SSL handshakes during the aggregation interval. | deployment | | plus.ssl.reuses | SSL session reuses | build version | count | The total number of session reuses during SSL handshakes in the aggregation interval. | deployment | | plus.ssl.no_common_protocol | Handshakes failed - no common protocol | build version | count | The number of SSL handshakes failed because of no common protocol during the aggregation interval. | deployment | | plus.ssl.no_common_cipher | Handshakes failed - no shared cipher | build version | count | The number of SSL handshakes failed because of no shared cipher during the aggregation interval. | deployment | | plus.ssl.handshake_timeout | Handshakes failed - timeout | build version | count | The number of SSL handshakes failed because of a timeout during the aggregation interval. | deployment | | plus.ssl.peer_rejected_cert | Handshakes failed - certificate rejected | build version | count | The number of failed SSL handshakes when nginx presented the certificate to the client but it was rejected with a corresponding alert message during the aggregation interval. | deployment | | plus.ssl.verify_failures.no_cert | Cert verify failures - no cert | build version | count | SSL certificate verification errors - a client did not provide the required certificate during the aggregation interval. | deployment | | plus.ssl.verify_failures.expired_cert | Cert verify failures - expired cert | build version | count | SSL certificate verification errors - an expired or not yet valid certificate was presented by a client during the aggregation interval. | deployment | | plus.ssl.verify_failures.revoked_cert | Cert verify failures - revoked cert | build version | count | SSL certificate verification errors - a revoked certificate was presented by a client during the aggregation interval. | deployment | | plus.ssl.verify_failures.hostname_mismatch | Cert verify failures - hostname mismatch | build version | count | SSL certificate verification errors - server's certificate doesn't match the hostname during the aggregation interval. | deployment | | plus.ssl.verify_failures.other | Cert verify failures - other | build version | count | SSL certificate verification errors - other SSL certificate verification errors during the aggregation interval. | deployment | | plus.http.ssl.handshakes | HTTP successful SSL handshakes | build version server_zone | count | The total number of successful SSL handshakes during the aggregation interval. | server zone | | plus.http.ssl.handshakes.failed | HTTP failed SSL handshakes | build version server_zone | count | The total number of failed SSL handshakes during the aggregation interval. | server zone | | plus.http.ssl.session.reuses | HTTP SSL session reuses | build version server_zone | count | The total number of session reuses during SSL handshakes in the aggregation interval. | server zone | | plus.http.ssl.no_common_protocol | Handshakes failed - no common protocol | build version server_zone | count | The number of SSL handshakes failed because of no common protocol during the aggregation interval. | server zone | | plus.http.ssl.no_common_cipher | Handshakes failed - no shared cipher | build version server_zone | count | The number of SSL handshakes failed because of no shared cipher during the aggregation interval. | server zone | | plus.http.ssl.handshake_timeout | Handshakes failed - timeout | build version server_zone | count | The number of SSL handshakes failed because of a timeout during the aggregation interval. | server zone | | plus.http.ssl.peer_rejected_cert | Handshakes failed - certificate rejected | build version server_zone | count | The number of failed SSL handshakes when nginx presented the certificate to the client but it was rejected with a corresponding alert message during the aggregation interval. | server zone | | plus.http.ssl.verify_failures.no_cert | Verify failures - no certificate | build version server_zone | count | SSL certificate verification errors - a client did not provide the required certificate during the aggregation interval. | server zone | | plus.http.ssl.verify_failures.expired_cert | Verify failures - expired cert | build version server_zone | count | SSL certificate verification errors - an expired or not yet valid certificate was presented by a client during the aggregation interval. | server zone | | plus.http.ssl.verify_failures.revoked_cert | Verify failures - revoked cert | build version server_zone | count | SSL certificate verification errors - a revoked certificate was presented by a client during the aggregation interval. | server zone | | plus.http.ssl.verify_failures.other | Verify failures - other | build version server_zone | count | SSL certificate verification errors - other SSL certificate verification errors during the aggregation interval. | server zone | ### NGINX cache statistics | **Metric** | **Display Name** | **Dimensions** | **Type** | **Description** | **Roll-up per** | |----------------------------------------|------------------|-----------------------------|-------|-----------------------------------------------------------------------------------------------------------------------------|---------------| | plus.cache.hit.ratio | Cache hit ratio | build version cache_zone | avg | Cache Hit Ratio The average ratio of cache hits to misses during the aggregation interval. | cache zone | | plus.cache.size | Cache size | build version cache_zone | avg | Cache Size The average size of the cache during the aggregation interval. | cache zone | | plus.cache.max_size | Cache max size | build version cache_zone | max | Cache Max Size The max size of the cache during the aggregation interval. | cache zone | | plus.cache.hit.responses | Cache hit responses | build version cache_zone | count | The total number of responses that were served from the cache during the aggregation interval. | cache zone | | plus.cache.hit.bytes | Cache hit bytes | build version cache_zone | count | The total number of bytes served from the cache during the aggregation interval. | cache zone | | plus.cache.stale.responses | Cache stale responses | build version cache_zone | count | The total number of responses served from stale cache content during the aggregation interval. | cache zone | | plus.cache.stale.bytes | Cache stale bytes | build version cache_zone | count | The total number of bytes served from stale cache content during the aggregation interval. | cache zone | | plus.cache.updating.responses | Cache updating responses | build version cache_zone | count | The total number of responses served from the cache while the cache is being updated during the aggregation interval. | cache zone | | plus.cache.updating.bytes | Cache updating bytes | build version cache_zone | count | The total number of bytes served from the cache while the cache is being updated during the aggregation interval. | cache zone | | plus.cache.revalidated.responses | Cache revalidated responses | build version cache_zone | count | The total number of cache responses that were successfully revalidated with the origin server during the aggregation interval. | cache zone | | plus.cache.revalidated.bytes | Cache revalidated bytes | build version cache_zone | count | The total number of bytes served from the cache after successful revalidation with the origin server during the aggregation interval. | cache zone | | plus.cache.miss.responses | Cache miss responses | build version cache_zone | count | The total number of responses that were not served from the cache (cache misses) during the aggregation interval. | cache zone | | plus.cache.miss.bytes | Cache miss bytes | build version cache_zone | count | The total number of bytes served from the origin server due to cache misses during the aggregation interval. | cache zone | | plus.cache.expired.responses | Cache expired responses | build version cache_zone | count | The total number of cache responses that expired and had to be refreshed from the origin server during the aggregation interval. | cache zone | | plus.cache.expired.bytes | Cache expired bytes | build version cache_zone | count | The total number of bytes served from the cache after expiration and refresh from the origin server during the aggregation interval. | cache zone | | plus.cache.expired.responses_written | Cache expired responses written | build version cache_zone | count | The total number of expired cache responses that were refreshed and written back to the cache during the aggregation interval. | cache zone | | plus.cache.expired.bytes_written | Cache expired bytes written | build version cache_zone | count | The total number of bytes written back to the cache after expiration and refresh from the origin server during the aggregation interval. | cache zone | | plus.cache.bypass.responses | Cache bypass responses | build version cache_zone | count | The total number of responses that bypassed the cache during the aggregation interval. | cache zone | | plus.cache.bypass.bytes | Cache bypass bytes | build version cache_zone | count | The total number of bytes served by bypassing the cache during the aggregation interval. | cache zone | | plus.cache.bypass.responses_written | Cache bypass responses written | build version cache_zone | count | The total number of responses that bypassed the cache and were written back to the cache during the aggregation interval. | cache zone | | plus.cache.bypass.bytes_written | Cache bypass bytes written | build version cache_zone | count | The total number of bytes that bypassed the cache and were written back to the cache during the aggregation interval. | cache zone | ### NGINX worker statistics | **Metric** | **Display Name** | **Dimensions** | **Type** | **Description** | **Roll-up per** | |----------------------------------------|-------------------------------|-----------------------------|-------|-----------------------------------------------------------------------------------------------------------------------------|---------------| | plus.worker.conn.accepted | Worker connections accepted | build version worker_id | count | The total number of client connections accepted by the worker process during the aggregation interval. | worker | | plus.worker.conn.dropped | Worker connections dropped | build version worker_id | count | The total number of client connections dropped by the worker process during the aggregation interval. | worker | | plus.worker.conn.active | Active worker connections | build version worker_id | count | The current number of active client connections that are currently being handled by the worker process during the aggregation interval. | worker | | plus.worker.conn.idle | Idle worker connections | build version worker_id | count | The number of idle client connections that are currently being handled by the worker process during the aggregation interval. | worker | | plus.worker.http.request.total | Total worker HTTP requests | build version worker_id | count | The total number of client requests received by the worker process during the aggregation interval. | worker | | plus.worker.http.request.current | Current worker HTTP requests | build version worker_id | count | The current number of client requests that are currently being processed by the worker process during the aggregation interval. | worker | ### NGINX upstream statistics | **Metric** | **Display Name** | **Dimensions** | **Type** | **Description** | **Roll-up per** | |-----------------------------------|-------------------------------|-----------------------------|-------|-----------------------------------------------------------------------------------------------------------------------------|---------------| | plus.http.upstream.peers.conn.active | Upstream active connections | build version upstream peer.address peer.name | count | Upstream Server Active Connections The number of active client connections during the aggregation interval. | upstream server | | plus.http.upstream.peers.request.count | Upstream HTTP requests | build version upstream peer.address peer.name | count | Upstream Server HTTP Requests The total number of HTTP requests during the aggregation interval. | upstream server | | plus.http.upstream.peers.response.count | Upstream server HTTP responses | build version upstream peer.address peer.name | count | Upstream Server HTTP Responses The total number of HTTP responses during the aggregation interval. | upstream server | | plus.http.upstream.peers.status.1xx | Upstream server HTTP 1xx responses | build version upstream peer.address peer.name | count | Upstream Server HTTP 1xx Responses The total number of HTTP responses with a 1xx status code during the aggregation interval. | upstream server | | plus.http.upstream.peers.status.2xx | Upstream server HTTP 2xx responses | build version upstream peer.address peer.name | count | Upstream Server HTTP 2xx Responses The total number of HTTP responses with a 2xx status code during the aggregation interval. | upstream server | | plus.http.upstream.peers.status.3xx | Upstream server HTTP 3xx responses | build version upstream peer.address peer.name | count | Upstream Server HTTP 3xx Responses The total number of HTTP responses with a 3xx status code during the aggregation interval. | upstream server | | plus.http.upstream.peers.status.4xx | Upstream server HTTP 4xx responses | build version upstream peer.address peer.name | count | Upstream Server HTTP 4xx Responses The total number of HTTP responses with a 4xx status code during the aggregation interval. | upstream server | | plus.http.upstream.peers.status.5xx | Upstream server HTTP 5xx responses | build version upstream peer.address peer.name | count | Upstream Server HTTP 5xx Responses The total number of HTTP responses with a 5xx status code during the aggregation interval. | upstream server | | plus.http.upstream.peers.request.bytes_sent | Upstream server request bytes sent | build version upstream peer.address peer.name | count | The total number of bytes sent in HTTP requests during the aggregation interval. | upstream server | | plus.http.upstream.peers.request.bytes_rcvd | Upstream server request bytes received | build version upstream peer.address peer.name | count | The total number of bytes received in HTTP requests during the aggregation interval. | upstream server | | plus.http.upstream.peers.state.up | Upstream server state up | build version upstream peer.address peer.name | boolean | Upstream Server State Up Current state of upstream servers in deployment. If all upstream servers in the deployment are up, then the value will be 1. If any upstream server is not up, then the value will be 0. | upstream peer | | plus.http.upstream.peers.state.draining | Upstream server state draining | build version upstream peer.address peer.name | boolean | Upstream Server State Draining Current state of upstream servers in deployment. If any of the upstream servers in the deployment are draining, then the value will be 1. If no upstream server is draining, then the value will be 0. | upstream peer | | plus.http.upstream.peers.state.down | Upstream server state down | build version upstream peer.address peer.name | boolean | Upstream Server State Down Current state of upstream servers in deployment. If any of the upstream servers in the deployment are down, then the value will be 1. If no upstream server is down, then the value will be 0. | upstream peer | | plus.http.upstream.peers.state.unavail | Upstream server state unavailable | build version upstream peer.address peer.name | boolean | Upstream Server State Unavailable Current state of upstream servers in deployment. If any of the upstream servers in the deployment are unavailable, then the value will be 1. If no upstream server is unavailable, then the value will be 0. | upstream peer | | plus.http.upstream.peers.state.checking | Upstream server state checking | build version upstream peer.address peer.name | boolean | Upstream Server State Check Current state of upstream servers in deployment. If any of the upstream servers in the deployment is being checked then the value will be 1. If no upstream server is being checked then the value will be 0. | upstream peer | | plus.http.upstream.peers.state.unhealthy | Upstream server state unhealthy | build version upstream peer.address peer.name | boolean | Upstream Server State Unhealthy Current state of upstream servers in deployment. If any of the upstream servers in the deployment are unhealthy then the value will be 1. If no upstream server is unhealthy then the value will be 0. | upstream peer | | plus.http.upstream.peers.fails | Upstream server fails | build version upstream peer.address peer.name | count | Upstream Server Fails The total number of unsuccessful attempts to communicate with the server during the aggregation interval. | upstream peer | | plus.http.upstream.peers.unavail | Upstream server unavailable | build version upstream peer.address peer.name | count | Upstream Server Unavailable The number of times the server became unavailable for client requests (state “unavail”) due to the number of unsuccessful attempts reaching the [max_fails](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#max_fails) threshold during the aggregation interval. | upstream peer | | plus.http.upstream.peers.health_checks.checks | Upstream server health checks | build version upstream peer.address peer.name | count | Upstream Server Health Checks The total number of [health check](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#health_check) requests made during the aggregation interval. | upstream peer | | plus.http.upstream.peers.health_checks.fails | Upstream server health checks fails | build version upstream peer.address peer.name | count | Upstream Server Health Checks Fails The number of failed health checks during the aggregation interval. | upstream peer | | plus.http.upstream.peers.health_checks.unhealthy | Upstream server health checks unhealthy | build version upstream peer.address peer.name | count | Upstream Server Health Checks Unhealthy How many times the server became unhealthy (state “unhealthy”) during the aggregation interval. | upstream peer | | plus.http.upstream.peers.health_checks.last_passed | Upstream server health checks last pass | build version upstream peer.address peer.name | boolean | Upstream Server Health Checks Last Pass last_passed (boolean) indicating if the last health check request was successful and passed [tests](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#match). | upstream peer | | plus.http.upstream.peers.downstart | Upstream server downstart | build version upstream peer.address peer.name | timestamp | Upstream Server Downstart The time when the server became “unavail”, “checking”, or “unhealthy”, as a UTC timestamp. | upstream peer | | plus.http.upstream.peers.response.time | Upstream server response time | build version upstream peer.address peer.name | avg | Upstream Server Response Time The average time to get the [full response](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#var_upstream_response_time) from the server during the aggregation interval. | upstream server | | plus.http.upstream.peers.header.time | Upstream server header time | build version upstream peer.address peer.name | avg | Upstream Server Header Time The average time to get the [response header](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#var_upstream_header_time) from the server | upstream server | | plus.http.upstream.peers.response.time.hist.count | Upstream response time histogram count | build version upstream peer.address peer.name | count | The total number of upstream response time measurements recorded during the aggregation interval. | upstream server | | plus.http.upstream.peers.response.time.hist.sum | Upstream response time histogram sum | build version upstream peer.address peer.name | count | The cumulative sum of upstream response times (in milliseconds) recorded during the aggregation interval. | upstream server | | plus.http.upstream.peers.response.time.hist.bucket.5 | Upstream response time histogram bucket ≤5ms | build version upstream peer.address peer.name | count | The number of upstream responses with a response time at or below 5ms during the aggregation interval. | upstream server | | plus.http.upstream.peers.response.time.hist.bucket.10 | Upstream response time histogram bucket ≤10ms | build version upstream peer.address peer.name | count | The number of upstream responses with a response time at or below 10ms during the aggregation interval. | upstream server | | plus.http.upstream.peers.response.time.hist.bucket.25 | Upstream response time histogram bucket ≤25ms | build version upstream peer.address peer.name | count | The number of upstream responses with a response time at or below 25ms during the aggregation interval. | upstream server | | plus.http.upstream.peers.response.time.hist.bucket.50 | Upstream response time histogram bucket ≤50ms | build version upstream peer.address peer.name | count | The number of upstream responses with a response time at or below 50ms during the aggregation interval. | upstream server | | plus.http.upstream.peers.response.time.hist.bucket.75 | Upstream response time histogram bucket ≤75ms | build version upstream peer.address peer.name | count | The number of upstream responses with a response time at or below 75ms during the aggregation interval. | upstream server | | plus.http.upstream.peers.response.time.hist.bucket.100 | Upstream response time histogram bucket ≤100ms | build version upstream peer.address peer.name | count | The number of upstream responses with a response time at or below 100ms during the aggregation interval. | upstream server | | plus.http.upstream.peers.response.time.hist.bucket.250 | Upstream response time histogram bucket ≤250ms | build version upstream peer.address peer.name | count | The number of upstream responses with a response time at or below 250ms during the aggregation interval. | upstream server | | plus.http.upstream.peers.response.time.hist.bucket.500 | Upstream response time histogram bucket ≤500ms | build version upstream peer.address peer.name | count | The number of upstream responses with a response time at or below 500ms during the aggregation interval. | upstream server | | plus.http.upstream.peers.response.time.hist.bucket.750 | Upstream response time histogram bucket ≤750ms | build version upstream peer.address peer.name | count | The number of upstream responses with a response time at or below 750ms during the aggregation interval. | upstream server | | plus.http.upstream.peers.response.time.hist.bucket.1000 | Upstream response time histogram bucket ≤1000ms | build version upstream peer.address peer.name | count | The number of upstream responses with a response time at or below 1000ms during the aggregation interval. | upstream server | | plus.http.upstream.peers.response.time.hist.bucket.2500 | Upstream response time histogram bucket ≤2500ms | build version upstream peer.address peer.name | count | The number of upstream responses with a response time at or below 2500ms during the aggregation interval. | upstream server | | plus.http.upstream.peers.response.time.hist.bucket.5000 | Upstream response time histogram bucket ≤5000ms | build version upstream peer.address peer.name | count | The number of upstream responses with a response time at or below 5000ms during the aggregation interval. | upstream server | | plus.http.upstream.peers.response.time.hist.bucket.7500 | Upstream response time histogram bucket ≤7500ms | build version upstream peer.address peer.name | count | The number of upstream responses with a response time at or below 7500ms during the aggregation interval. | upstream server | | plus.http.upstream.peers.response.time.hist.bucket.10000 | Upstream response time histogram bucket ≤10000ms | build version upstream peer.address peer.name | count | The number of upstream responses with a response time at or below 10000ms during the aggregation interval. | upstream server | | plus.http.upstream.peers.response.time.hist.bucket.inf | Upstream response time histogram bucket +Inf | build version upstream peer.address peer.name | count | The total number of upstream responses counted across all response time buckets during the aggregation interval. | upstream server | | plus.http.upstream.peers.response.time.hist.median | Upstream response time histogram median | build version upstream peer.address peer.name | avg | The median upstream response time (in milliseconds) during the aggregation interval. | upstream server | | plus.http.upstream.peers.response.time.hist.95th_percentile | Upstream response time histogram 95th percentile | build version upstream peer.address peer.name | avg | The 95th percentile upstream response time (in milliseconds) during the aggregation interval. | upstream server | | plus.http.upstream.zombies | Upstream zombies | build version | avg | Upstream Zombies The current number of servers removed from the group but still processing active client requests | deployment | | plus.http.upstream.keepalives | Upstream keepalive connections | build version | count | Upstream Keepalive Connections The current number of idle [keepalive](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive) connections | deployment | | plus.http.upstream.queue.maxsize | Upstream queue max size | build version | avg | Upstream Queue Max Size The maximum number of requests that can be in the queue at the same time | deployment | | plus.http.upstream.queue.overflows | Upstream queue overflows | build version | sum | Upstream Queue Overflows The total number of requests rejected due to the queue overflow | deployment | | plus.http.upstream.queue.size | Upstream queue size | build version | avg | Upstream Queue Size The current number of requests in the queue | deployment | | plus.http.upstream.peers.ssl.handshakes | Upstream SSL handshakes | build version upstream peer.address peer.name | count | The total number of successful SSL handshakes during the aggregation interval. | upstream peer | | plus.http.upstream.peers.ssl.handshakes.failed | Upstream SSL handshakes failed | build version upstream peer.address peer.name | count | The total number of failed SSL handshakes during the aggregation interval. | upstream peer | | plus.http.upstream.peers.ssl.session.reuses | Upstream SSL session reuses | build version upstream peer.address peer.name | count | The total number of session reuses during SSL handshake in the aggregation interval. | upstream peer | | plus.http.upstream.peers.ssl.no_common_protocol | Upstream SSL no common protocol | build version upstream peer.address peer.name | count | The number of SSL handshakes failed because of no common protocol during the aggregation interval. | upstream peer | | plus.http.upstream.peers.ssl.handshake_timeout | Upstream SSL handshake timeout | build version upstream peer.address peer.name | count | The number of SSL handshakes failed because of a timeout during the aggregation interval. | upstream peer | | plus.http.upstream.peers.ssl.peer_rejected_cert | SSL handshake failed - rejected cert | build version upstream peer.address peer.name | count | The number of failed SSL handshakes when nginx presented the certificate to the client but it was rejected with a corresponding alert message during the aggregation interval. | upstream peer | | plus.http.upstream.peers.ssl.verify_failures.expired_cert | SSL verify failures - expired cert | build version upstream peer.address peer.name | count | SSL certificate verification errors - an expired or not yet valid certificate was presented by a client during the aggregation interval. | upstream peer | | plus.http.upstream.peers.ssl.verify_failures.revoked_cert | SSL verify failures - revoked cert | build version upstream peer.address peer.name | count | SSL certificate verification errors - a revoked certificate was presented by a client during the aggregation interval. | upstream peer | | plus.http.upstream.peers.ssl.verify_failures.hostname_mismatch | SSL verify failures - hostname mismatch | build version upstream peer.address peer.name | count | SSL certificate verification errors - server's certificate doesn't match the hostname during the aggregation interval. | upstream peer | | plus.http.upstream.peers.ssl.verify_failures.other | SSL verify failures - other | build version upstream peer.address peer.name | count | SSL certificate verification errors - other SSL certificate verification errors during the aggregation interval. | upstream peer | | plus.stream.upstream.peers.ssl.handshakes | Stream SSL handshakes total | build version upstream peer.address peer.name | count | The total number of successful SSL handshakes during the aggregation interval. | upstream peer | | plus.stream.upstream.peers.ssl.handshakes.failed | Stream SSL handshakes failed | build version upstream peer.address peer.name | count | The total number of failed SSL handshakes during the aggregation interval. | upstream peer | | plus.stream.upstream.peers.ssl.session.reuses | Stream SSL session reuses | build version upstream peer.address peer.name | count | The total number of session reuses during SSL handshake in the aggregation interval. | upstream peer | | plus.stream.upstream.peers.ssl.no_common_protocol | Stream HS failed - no common protocol | build version upstream peer.address peer.name | count | The number of SSL handshakes failed because of no common protocol during the aggregation interval. | upstream peer | | plus.stream.upstream.peers.ssl.handshake_timeout | Stream SSL handshake timeout | build version upstream peer.address peer.name | count | The number of SSL handshakes failed because of a timeout during the aggregation interval. | upstream peer | | plus.stream.upstream.peers.ssl.peer_rejected_cert | Stream verify failure - rejected cert | build version upstream peer.address peer.name | count | The number of failed SSL handshakes when nginx presented the certificate to the client but it was rejected with a corresponding alert message during the aggregation interval. | upstream peer | | plus.stream.upstream.peers.ssl.verify_failures.expired_cert | Stream verify failure - expired cert | build version upstream peer.address peer.name | count | SSL certificate verification errors - an expired or not yet valid certificate was presented by a client during the aggregation interval. | upstream peer | | plus.stream.upstream.peers.ssl.verify_failures.revoked_cert | Stream verify failure - revoked cert | build version upstream peer.address peer.name | count | SSL certificate verification errors - a revoked certificate was presented by a client during the aggregation interval. | upstream peer | | plus.stream.upstream.peers.ssl.verify_failures.hostname_mismatch | Stream verify failure - hostname mismatch | build version upstream peer.address peer.name | count | SSL certificate verification errors - server's certificate doesn't match the hostname during the aggregation interval. | upstream peer | | plus.stream.upstream.peers.ssl.verify_failures.other | Stream SSL verify failure - other | build version upstream peer.address peer.name | count | SSL certificate verification errors - other SSL certificate verification errors during the aggregation interval. | upstream peer | ### NGINX system statistics | **Metric** | **Display Name** | **Dimensions** | **Type** | **Description** | **Roll-up per** | |----------------------------------------|------------------|-----------------------------|-------|-----------------------------------------------------------------------------------------------------------------------------|---------------| | system.cpu| CPU utilization | | average | System CPU Utilization. | deployment | | system.interface.bytes_rcvd| Interface bytes received | interface | count | System Interface Bytes Received. | deployment | | system.interface.bytes_sent| Interface bytes sent | interface | count | System Interface Bytes Sent. | deployment | | system.interface.packets_rcvd| Interface packets received | interface | count | System Interface Packets Received. | deployment | | system.interface.packets_sent| Interface packets sent | interface | count | System Interface Packets Sent. | deployment | | system.interface.total_bytes| Interface total bytes | interface | count | System Interface Total Bytes, sum of bytes_sent and bytes_rcvd. | deployment | | system.interface.egress_throughput| Interface egress throughput | interface | count | System Interface Egress Throughput, i.e. bytes sent per second| deployment | ### NGINX stream statistics | **Metric** | **Display Name** | **Dimensions** | **Type** | **Description** | **Roll-up per** | |----------------------------------------|-------------------------------|-----------------------------|-------|-----------------------------------------------------------------------------------------------------------------------------|---------------| | plus.stream.limit_conns.passed | Connections passed | build, version, limit_conn_zone | count | The total number of connections that were neither limited nor accounted as limited. | limit conn zone | | plus.stream.limit_conns.rejected | Connections rejected | build, version, limit_conn_zone | count | The total number of connections that were rejected. | limit conn zone | | plus.stream.limit_conns.rejected_dry_run | Connections rejected dry run | build, version, limit_conn_zone | count | The total number of connections accounted as rejected in the dry run mode. | limit conn zone | | plus.stream.request.bytes_rcvd | Request bytes received | build, version, server_zone | count | The total number of bytes received from clients. | server zone | | plus.stream.request.bytes_sent | Request bytes sent | build, version, server_zone | count | The total number of bytes sent to clients. | server zone | | plus.stream.status.2xx | Status 2xx | build, version, server_zone | count | The total number of sessions completed with status codes '2xx'. | server zone | | plus.stream.status.4xx | Status 4xx | build, version, server_zone | count | The total number of sessions completed with status codes '4xx'. | server zone | | plus.stream.status.5xx | Status 5xx | build, version, server_zone | count | The total number of sessions completed with status codes '5xx'. | server zone | | plus.stream.status.connections | Accepted connections | build, version, server_zone | avg | The average number of connections accepted from clients. | server zone | | plus.stream.status.discarded | Connections discarded | build, version, server_zone | avg | The average number of connections completed without creating a session. | server zone | | plus.stream.status.processing | Connections processing | build, version, server_zone | avg | The average number of client connections that are currently being processed. | server zone | | plus.stream.upstream.peers.conn.active | Upstream active connections | build, version, upstream, peer.address, peer.name | count | The current number of connections. | upstream peer | | plus.stream.upstream.peers.downstart | Upstream downstart | build, version, upstream, peer.address, peer.name | timestamp | The time when the server became 'unavail', 'checking', or 'unhealthy', in the ISO 8601 format with millisecond resolution. | upstream peer | | plus.stream.upstream.peers.downtime | Upstream downtime | build, version, upstream, peer.address, peer.name | count | Total time the server was in the 'unavail', 'checking', and 'unhealthy' states. | upstream peer | | plus.stream.upstream.peers.fails | Upstream fails | build, version, upstream, peer.address, peer.name | count | The total number of unsuccessful attempts to communicate with the server. | upstream peer | | plus.stream.upstream.peers.health_checks.checks | Upstream health checks | build, version, upstream, peer.address, peer.name | count | The total number of health check requests made. | upstream peer | | plus.stream.upstream.peers.health_checks.fails | Upstream health checks fails | build, version, upstream, peer.address, peer.name | count | The number of failed health checks. | upstream peer | | plus.stream.upstream.peers.health_checks.last_passed | Upstream last health check pass | build, version, upstream, peer.address, peer.name | boolean | Boolean indicating whether the last health check request was successful and passed tests. | upstream peer | | plus.stream.upstream.peers.health_checks.unhealthy | Upstream health checks unhealthy | build, version, upstream, peer.address, peer.name | count | How many times the server became unhealthy (state 'unhealthy'). | upstream peer | | plus.stream.upstream.peers.request.bytes_rcvd | Upstream request bytes received | build, version, upstream, peer.address, peer.name | count | The total number of bytes received from this server. | upstream peer | | plus.stream.upstream.peers.request.bytes_sent | Upstream request bytes sent | build, version, upstream, peer.address, peer.name | count | The total number of bytes sent to this server. | upstream peer | | plus.stream.upstream.peers.response.time | Upstream response time | build, version, upstream, peer.address, peer.name | avg | The average time to receive the last byte of data. | upstream peer | | plus.stream.upstream.peers.state.checking | Upstream state checking | build, version, upstream, peer.address, peer.name | boolean | Boolean indicating if any of the upstream servers are being checked. | upstream peer | | plus.stream.upstream.peers.state.down | Upstream state down | build, version, upstream, peer.address, peer.name | boolean | Boolean indicating if any of the upstream servers are down. | upstream peer | | plus.stream.upstream.peers.state.draining | Upstream state draining | build, version, upstream, peer.address, peer.name | boolean | Boolean indicating if any of the upstream servers are draining. | upstream peer | | plus.stream.upstream.peers.state.unavail | Upstream state unavailable | build, version, upstream, peer.address, peer.name | boolean | Boolean indicating if any of the upstream servers are unavailable. | upstream peer | | plus.stream.upstream.peers.state.unhealthy | Upstream state unhealthy | build, version, upstream, peer.address, peer.name | boolean | Boolean indicating if any of the upstream servers are unhealthy. | upstream peer | | plus.stream.upstream.peers.state.up | Upstream state up | build, version, upstream, peer.address, peer.name | boolean | Boolean indicating if all upstream servers are up. | upstream peer | | plus.stream.upstream.peers.unavail | Upstream unavailable | build, version, upstream, peer.address, peer.name | count | How many times the server became unavailable for client connections (state 'unavail') due to the number of unsuccessful attempts reaching the max_fails threshold. | upstream peer | | plus.stream.upstream.zombies | Upstream zombies | build, version | avg | The current number of servers removed from the group but still processing active client connections. | deployment | | plus.stream.ssl.handshakes | Stream SSL handshakes total | build version server_zone | count | The total number of successful SSL handshakes during the aggregation interval. | server zone | | plus.stream.ssl.handshakes.failed | Stream SSL handshakes failed | build version server_zone | count | The total number of failed SSL handshakes during the aggregation interval. | server zone | | plus.stream.ssl.session.reuses | Stream SSL session reuses | build version server_zone | count | The total number of session reuses during SSL handshakes in the aggregation interval. | server zone | | plus.stream.ssl.no_common_protocol | Stream HS failed - no common protocol | build version server_zone | count | The number of SSL handshakes failed because of no common protocol during the aggregation interval. | server zone | | plus.stream.ssl.no_common_cipher | Stream HS failed - no shared cipher | build version server_zone | count | The number of SSL handshakes failed because of no shared cipher during the aggregation interval. | server zone | | plus.stream.ssl.handshake_timeout | Stream SSL handshake timeout | build version server_zone | count | The number of SSL handshakes failed because of a timeout during the aggregation interval. | server zone | | plus.stream.ssl.peer_rejected_cert | Stream verify failure - rejected cert | build version server_zone | count | The number of failed SSL handshakes when nginx presented the certificate to the client but it was rejected with a corresponding alert message during the aggregation interval. | server zone | | plus.stream.ssl.verify_failures.no_cert | Stream verify failure - no cert | build version server_zone | count | SSL certificate verification errors - a client did not provide the required certificate during the aggregation interval. | server zone | | plus.stream.ssl.verify_failures.expired_cert | Stream verify failure - expired cert | build version server_zone | count | SSL certificate verification errors - an expired or not yet valid certificate was presented by a client during the aggregation interval. | server zone | | plus.stream.ssl.verify_failures.revoked_cert | Stream verify failure - revoked cert | build version server_zone | count | SSL certificate verification errors - a revoked certificate was presented by a client during the aggregation interval. | server zone | | plus.stream.ssl.verify_failures.other | Stream SSL verify failure - other | build version server_zone | count | SSL certificate verification errors - other SSL certificate verification errors during the aggregation interval. | server zone | | plus.stream.zone_sync.status.bytes_in | Zone sync bytes in | build, version | count | The number of bytes received by all nodes during the aggregation interval. | deployment | | plus.stream.zone_sync.status.bytes_out | Zone sync bytes out | build, version | count | The number of bytes sent by all nodes during the aggregation interval. | deployment | | plus.stream.zone_sync.status.msgs_in | Zone sync messages in | build, version | count | The number of messages received by all nodes during the aggregation interval. | deployment | | plus.stream.zone_sync.status.msgs_out | Zone sync messages out | build, version | count | The number of messages sent by all nodes during the aggregation interval. | deployment | | plus.stream.zone_sync.zones.records_pending | Zone sync records pending | build, version, shared_memory_zone | avg | The average number of records that need to be sent to the cluster during the aggregation interval. | shared memory zone | | plus.stream.zone_sync.zones.records_total | Zone sync records total | build, version, shared_memory_zone | avg | The average number of records stored in the shared memory zone by all nodes during the aggregation interval. | shared memory zone | ### NGINX resolver statistics | **Metric** | **Display Name** | **Dimensions** | **Type** | **Description** | **Roll-up per** | |---------------------------------------|------------------------------|--------------------------------|----------|--------------------------------------------------------------------------------------------|-----------------| | plus.resolvers.requests.name | Resolve name requests | build, version, resolver_zone | count | The number of requests to resolve names to addresses during the aggregation interval. | resolver zone | | plus.resolvers.requests.srv | Resolve SRV requests | build, version, resolver_zone | count | The number of requests to resolve SRV records during the aggregation interval. | resolver zone | | plus.resolvers.requests.addr | Resolve address requests | build, version, resolver_zone | count | The number of requests to resolve addresses to names during the aggregation interval. | resolver zone | | plus.resolvers.responses.noerror | Successful responses | build, version, resolver_zone | count | The number of successful responses during the aggregation interval. | resolver zone | | plus.resolvers.responses.formerr | FORMERR responses | build, version, resolver_zone | count | The number of FORMERR (Format error) responses during the aggregation interval. | resolver zone | | plus.resolvers.responses.servfail | SERVFAIL responses | build, version, resolver_zone | count | The number of SERVFAIL (Server failure) responses during the aggregation interval. | resolver zone | | plus.resolvers.responses.nxdomain | NXDOMAIN responses | build, version, resolver_zone | count | The number of NXDOMAIN (Host not found) responses during the aggregation interval. | resolver zone | | plus.resolvers.responses.notimp | NOTIMP responses | build, version, resolver_zone | count | The number of NOTIMP (Unimplemented) responses during the aggregation interval. | resolver zone | | plus.resolvers.responses.refused | REFUSED responses | build, version, resolver_zone | count | The number of REFUSED (Operation refused) responses during the aggregation interval. | resolver zone | | plus.resolvers.responses.timedout | Timed out requests | build, version, resolver_zone | count | The number of timed out requests during the aggregation interval. | resolver zone | | plus.resolvers.responses.unknown | Unknown error responses | build, version, resolver_zone | count | The number of requests completed with an unknown error during the aggregation interval. | resolver zone | -------------------------------------------------------------------------------- title: "Terraform" url: https://docs.nginx.com/nginxaas-azure/client-tools/terraform/index.md description: Learn how to use the Terraform to manage NGINXaaS for Azure. -------------------------------------------------------------------------------- F5 NGINXaaS for Azure (NGINXaaS) deployments can be managed using Terraform. This document outlines common Terraform workflows for NGINXaaS. ## Prerequisites - [NGINXaaS Prerequisites](/nginxaas-azure/getting-started/prerequisites.md) - [Authenticate Terraform to Azure](https://learn.microsoft.com/en-us/azure/developer/terraform/authenticate-to-azure) - [Install Terraform](https://developer.hashicorp.com/terraform/downloads) ## Workflows ### Create or update a deployment See [NGINXaaS Snippets](https://github.com/nginxinc/nginxaas-for-azure-snippets/tree/main/terraform/deployments/create-or-update) for an example to create or update deployment resources. - [NGINXaaS Managed Identity Documentation](/nginxaas-azure/getting-started/managed-identity-portal.md) - [NGINXaaS Azure Monitor Documentation](/nginxaas-azure/monitoring/enable-metrics.md) ### Create or update a certificate Upload a self-signed certificate created in Azure Key Vault to a deployment. See [NGINXaaS Snippets](https://github.com/nginxinc/nginxaas-for-azure-snippets/tree/main/terraform/certificates) for an example to create or update certificate resources. - [NGINXaaS Certificates Documentation](/nginxaas-azure/getting-started/ssl-tls-certificates/ssl-tls-certificates-portal.md) ### Create or update a configuration Upload an example multi-file NGINX configuration to a deployment. See [NGINXaaS Snippets](https://github.com/nginxinc/nginxaas-for-azure-snippets/tree/main/terraform/configurations) for an example to create or update configurations resources. - [NGINXaaS Configuration Documentation](/nginxaas-azure/getting-started/nginx-configuration/nginx-configuration-portal.md) ## Additional Docs - [Managing an NGINXaaS for Azure deployment](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/nginx_deployment) - [Managing an NGINXaaS for Azure deployment configuration](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/nginx_configuration) - [Managing an NGINXaaS for Azure deployment certificate](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/nginx_certificate) - If you are new to Terraform, see [Terraform Overview](https://www.terraform.io/) -------------------------------------------------------------------------------- title: "Use the njs Scripting language" url: https://docs.nginx.com/nginxaas-azure/quickstart/njs-support/index.md -------------------------------------------------------------------------------- F5 NGINXaaS for Azure (NGINXaaS) supports the open-source [njs module](https://nginx.org/en/docs/http/ngx_http_js_module.html), allowing the extension of NGINX functionality with a subset of the Javascript language. ## Upload NGINX configuration with njs Create an njs script file by uploading a gzipped tar file or create the script file in the editor. See [NGINX Configuration](/nginxaas-azure/getting-started/nginx-configuration/nginx-configuration-portal.md) for a step-by-step guide. **Note:** If specifying an absolute file path as your njs script's `File path`, see the [NGINX Filesystem Restrictions table](/nginxaas-azure/getting-started/nginx-configuration/overview/#nginx-filesystem-restrictions) for the allowed directories the file can be written to. Switch between the language options to see syntax highlighting for NGINX configs or JavaScript. To use njs, enable the `ngx_http_js_module` module and specify the `js_import` directive with your njs file. ```nginx load_module modules/ngx_http_js_module.so; http { js_import http.js; server { location / { js_content http.hello; } } } ``` ## njs validation NGINXaaS will not parse, evaluate, or run any provided njs scripts when validating the NGINX configuration. [Enable logging](/nginxaas-azure/monitoring/enable-logging/) to monitor errors caused by njs scripts. ## "fs" module The njs [File System module](http://nginx.org/en/docs/njs/reference.html#njs_api_fs) provides operations with files. NGINXaaS only allows reading and writing from [specified directories](/nginxaas-azure/getting-started/nginx-configuration.md#nginx-process-restrictions). -------------------------------------------------------------------------------- title: "Recreating a deployment" url: https://docs.nginx.com/nginxaas-azure/quickstart/recreate/index.md -------------------------------------------------------------------------------- Learn how to recreate an existing F5 NGINXaaS for Azure (NGINXaaS) deployment using an Azure Resource Manager (ARM) template. There are two ways to replicate a current NGINXaaS for Azure deployment using ARM templates. You can either delete and recreate the deployment, or you can update the DNS to smoothly transition to the new deployment. ## Prerequisites - [Azure CLI Installation](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli) - You need to be logged in to your Azure account through the CLI if you are using that for template deployment, see [Azure CLI Authentication](https://learn.microsoft.com/en-us/cli/azure/authenticate-azure-cli) ## Export ARM template from existing deployment To export an ARM template for an existing deployment: 1. Navigate to your existing NGINXaaS deployment. 1. Select **Export template** under **Automation** in the left menu. 1. Wait for the template to generate. 1. Select **Download**. ## Delete and recreate strategy The simplest method to recreate a deployment is to delete the original deployment and then recreate it using the ARM template. The ARM template generated through the portal will include the VNET and public IP address used by the original deployment as dependencies. If you plan to change these with the new deployment these entries need to be modified in the template. To recreate the deployment: 1. Export the template as instructed above. 1. Modify, if neccessary, and verify the data in the template for accuracy. 1. Delete the original deployment. 1. Use the exported ARM template to recreate the deployment using the Azure CLI: ```shell az deployment group create \ --subscription= \ --resource-group= \ --template-file= ``` ## DNS migration strategy If you control the DNS associated with the deployment's frontend and have flexibility of the IP address NGINXaaS uses you can recreate a deployment with no downtime. 1. Export the template, as instructed above. 1. If you're using a public IP, create a new public IP resource. If you're using a private IP address, select a new IP address from your VNET. 1. Modify the ARM template to change the NGINXaaS deployment name and reference the new IP address. 1. Use the Azure CLI as above to create a new deployment. 1. Update DNS to refer to the new deployment's IP address. 1. Monitor metrics of the old deployment to watch for requests dropping off as clients use the new IP address from DNS. Note that depending on your DNS settings this could take from a few minutes to a few days. 1. Delete the old deployment. Remember to change your configuration on any firewall or Network Security Group associated with the deployment to allow access to your deployment's new IP address. ## Further reading See the [Azure CLI Deployment Create](https://learn.microsoft.com/en-us/cli/azure/nginx/deployment#az-nginx-deployment-create) documentation for example commands to create deployment resources. You can find example code to manage NGINXaaS deployments and related objects in the NGINXaaS GitHub repository, [NGINXaaS Snippets](https://github.com/nginxinc/nginxaas-for-azure-snippets). -------------------------------------------------------------------------------- title: "Disaster recovery" url: https://docs.nginx.com/nginxaas-azure/disaster-recovery/index.md -------------------------------------------------------------------------------- This guide describes how to configure disaster recovery (DR) for F5 NGINXaaS for Azure deployments in separate (ideally [paired](https://learn.microsoft.com/en-us/azure/reliability/regions-paired)) Azure regions, ensuring upstream access remains available even if the primary NGINXaaS deployment in a region fails. The deployment architecture ensures users can access backend application servers (upstreams) continuously from an alternative region if the primary NGINXaaS deployment becomes unavailable. The solution leverages Terraform, Azure Traffic Manager, Azure Virtual Network (VNet) peering, and unique subnets to support failover. **Architecture Overview** ![The diagram illustrates the configuration of F5 NGINXaaS for Azure deployments for disaster recovery. It shows how the end users can still access the applications provided by AppServer - Primary1 and AppServer - Primary2 if the primary NGINXaaS deployment goes down.](nginxaas-azure/n4a-dr-topology.png) - Each region has its own VNet, subnet, and NGINXaaS for Azure deployment. - The NGINXaaS for Azure deployment needs to have a public frontend to leverage Azure Traffic Manager. - Cross region connectivity ensures that upstreams are reachable from either deployment. We use VNet peering in this guide to establish that connectivity. - Upstreams (for example, VMs) are accessible from either NGINX deployment. ## Prerequisites - Two Azure regions selected for DR. - Unique, non-overlapping VNet and subnet address spaces for each region. - Terraform 1.3+ and AzureRM provider 4.23+. **Note:** Each NGINX deployment **must run on separate subnets and non-overlapping address spaces**. This is critical for [Virtual Network (VNet) peering](https://learn.microsoft.com/en-us/azure/virtual-network/how-to-configure-subnet-peering) between the two regions. For example: - Primary Region Virtual Network Address Space: `10.0.0.0/16` - Secondary Region Virtual Network Address Space: `172.16.0.0/16` ## Configure disaster recovery ### Step 1: Terrraform setup To get started, please review the [Terraform prerequisites](/nginxaas-azure/getting-started/create-deployment/deploy-terraform.md#prerequisites) for NGINXaaS for Azure. The following steps outline Terraform resources required to set up the disaster recovery topology; these resources can be placed in a `main.tf` file, variables used by these resources can go into `variables.tf`, and outputs you need to collect can be defined in `outputs.tf`. The directory structure looks as follows: ```shell $ tree . |-- main.tf |-- outputs.tf `-- variables.tf ``` To execute the Terraform code, `cd` into the directory with these files and run: ```shell terraform init terraform plan terraform apply --auto-approve ``` ### Step 2: Deploy prerequisite infrastructure Each region requires its own VNet, subnet(s), public IP and network security group. #### Terraform code for necessary resources ```hcl # Primary Region resource "azurerm_resource_group" "primary_resource_group" { # ... location = "eastus" } resource "azurerm_public_ip" "primary_pip" { # ... location = "eastus" } resource "azurerm_virtual_network" "primary_virtual_network" { # ... address_space = [var.primary_vnet_addr_space] # - 10.0.0.0/16 location = "eastus" } resource "azurerm_subnet" "primary_subnet_1" { # ... virtual_network_name = azurerm_virtual_network.primary_virtual_network.name address_prefixes = [cidrsubnet(var.vnet_addr_space, 8, 0)] # - 10.0.0.0/24 delegation { name = "nginx" service_delegation { name = "NGINX.NGINXPLUS/nginxDeployments" actions = [ "Microsoft.Network/virtualNetworks/subnets/join/action" ] } } } resource "azurerm_network_security_group" "primary_virtual_network_nsg" { # ... location = "eastus" security_rule { # ... priority = 100 direction = "Inbound" access = "Allow" protocol = "Tcp" } } resource "azurerm_subnet_network_security_group_association" "primary_virtual_network_nsg_association" { subnet_id = azurerm_subnet.primary_subnet_1.id network_security_group_id = azurerm_network_security_group.primary_virtual_network_nsg.id } # Secondary Region resource "azurerm_resource_group" "secondary_resource_group" { # ... location = "centralus" } resource "azurerm_public_ip" "secondary_pip" { # ... location = "centralus" } resource "azurerm_virtual_network" "secondary_virtual_network" { # ... address_space = [var.secondary_vnet_addr_space] # - 172.16.0.0/16 location = "centralus" } resource "azurerm_subnet" "secondary_subnet_1" { # ... virtual_network_name = azurerm_virtual_network.secondary_virtual_network.name address_prefixes = [cidrsubnet(var.vnet_addr_space, 8, 0)] # - 172.16.0.0/24 delegation { name = "nginx" service_delegation { name = "NGINX.NGINXPLUS/nginxDeployments" actions = [ "Microsoft.Network/virtualNetworks/subnets/join/action" ] } } } resource "azurerm_network_security_group" "secondary_virtual_network_nsg" { # ... location = "centralus" security_rule { # ... priority = 100 direction = "Inbound" access = "Allow" protocol = "Tcp" } } resource "azurerm_subnet_network_security_group_association" "secondary_virtual_network_nsg_association" { subnet_id = azurerm_subnet.secondary_subnet_1.id network_security_group_id = azurerm_network_security_group.secondary_virtual_network_nsg.id } ``` ### Step 3: Configure app servers (upstreams) You may already have upstreams in the primary region that you wish to reverse proxy using NGINXaaS. For the sake of completion, the following example shows creation of Primary Subnet 2, NICs for the upstreams and the upstreams themselves. The upstream VMs need to be in a subnet separate from the NGINXaaS deployment subnet in the **primary region**. #### Terraform code for creating and configuring upstreams ```hcl resource "azurerm_subnet" "primary_subnet_2" { # ... virtual_network_name = azurerm_virtual_network.primary_virtual_network.name address_prefixes = [cidrsubnet(var.vnet_addr_space, 8, 1)] # - 10.0.1.0/24 } resource "azurerm_network_interface" "app_server_nic" { # ... count = 2 name = "nginx-vm${count.index + 1}-nic" location = "eastus" ip_configuration { name = "internal" subnet_id = azurerm_subnet.upstreams.id private_ip_address_allocation = "Dynamic" } } resource "azurerm_linux_virtual_machine" "nginx_upstream_vm" { # ... count = 2 name = "nginx-upstream${count.index + 1}" network_interface_ids = [ azurerm_network_interface.app_server_nic[count.index].id, ] source_image_reference { publisher = "Canonical" offer = "0001-com-ubuntu-server-jammy" sku = "22_04-lts-gen2" version = "latest" } custom_data = base64encode(<<-EOF #!/bin/bash sudo apt update sudo apt install nginx -y VM_ID=$(hostname) IP=$(ip addr show $(ip route show default | awk '/default/ {print $5}') | awk '/inet / {print $2}' | cut -d/ -f1) echo "Hello from $VM_ID: $IP" | sudo tee /var/www/html/index.html sudo systemctl enable nginx sudo systemctl restart nginx EOF ) } ``` **Note:** As a best practice, maintain identical upstream resources in your secondary region as in your primary region to ensure full protection and availability in the event of a region-wide outage or disaster. ### Step 4: Peer the VNets Peer the virtual networks so that the upstream app servers are accessible from either primary or secondary NGINXaaS deployment. #### Terraform code for configuring virtual network peering ```hcl resource "azurerm_virtual_network_peering" "primary_vnet_to_secondary_vnet" { name = "peering-primary-vnet-to-secondary-vnet" resource_group_name = var.primary_resource_group virtual_network_name = azurerm_virtual_network.primary_virtual_network.name remote_virtual_network_id = azurerm_virtual_network.secondary_virtual_network.id } resource "azurerm_virtual_network_peering" "secondary_vnet_to_primary_vnet" { name = "peering-secondary-vnet-to-primary-vnet" resource_group_name = var.resource_group_secondary virtual_network_name = azurerm_virtual_network.secondary_virtual_network.name remote_virtual_network_id = azurerm_virtual_network.primary_virtual_network.id } ``` - **Subnet Peering for Overlapping VNets:** If overlapping address spaces are unavoidable, use subnet-level peering to selectively peer only the required subnets. **Note:** As of May 2025, subnet peering is not available by default for all subscriptions. To use this feature, you must have the subscription on which you want to configure subnet peering be registered with Azure. Please review the configuration details and limitations in this [document](https://learn.microsoft.com/en-us/azure/virtual-network/how-to-configure-subnet-peering). ### Step 5: Deploy NGINXaaS for Azure in each region Reverse proxy your upstreams using NGINXaaS. Since the virtual networks are peered, both deployments would be able to access the upstreams. #### Terraform code for deploying and configuring primary and secondary NGINXaaS deployments ```hcl resource "azurerm_nginx_deployment" "primary_nginxaas_deployment" { name = var.primary_deployment_name resource_group_name = var.primary_resource_group location = "eastus" identity { type = "SystemAssigned" } # ... network_interface { subnet_id = azurerm_subnet.primary_subnet_1.id } } resource "azurerm_nginx_configuration" "primary_nginxaas_config" { nginx_deployment_id = azurerm_nginx_deployment.primary_nginxaas_deployment.id root_file = "/etc/nginx/nginx.conf" config_file { content = base64encode(<<-EOT user nginx; worker_processes auto; worker_rlimit_nofile 8192; pid /run/nginx/nginx.pid; events { worker_connections 4000; } error_log /var/log/nginx/error.log error; http { upstream backend_servers { server :80; server :80; keepalive 16; } server { listen 80 default_server; # /health will be used for Azure Traffic Manager Profile location /health { return 200 'nginx proxy alive'; } location / { proxy_pass http://backend_servers; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_next_upstream error timeout http_500; proxy_http_version 1.1; proxy_set_header "Connection" ""; } } } EOT ) virtual_path = "/etc/nginx/nginx.conf" } } # The secondary NGINXaaS deployment is identical to primary resource "azurerm_nginx_deployment" "secondary_nginxaas_deployment" { name = var.secondary_deployment_name resource_group_name = var.secondary_resource_group location = "centralus" identity { type = "SystemAssigned" } # ... network_interface { subnet_id = azurerm_subnet.secondary_subnet_1.id } } resource "azurerm_nginx_configuration" "secondary_nginxaas_config" { nginx_deployment_id = azurerm_nginx_deployment.secondary_nginxaas_deployment.id root_file = "/etc/nginx/nginx.conf" config_file { content = base64encode(<<-EOT user nginx; worker_processes auto; worker_rlimit_nofile 8192; pid /run/nginx/nginx.pid; events { worker_connections 4000; } error_log /var/log/nginx/error.log error; http { upstream backend_servers { server :80; server :80; keepalive 16; } server { listen 80 default_server; # /health will be used for Azure Traffic Manager Profile location /health { return 200 'nginx proxy alive'; } location / { proxy_pass http://backend_servers; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_next_upstream error timeout http_500; proxy_http_version 1.1; proxy_set_header "Connection" ""; } } } EOT ) virtual_path = "/etc/nginx/nginx.conf" } } ``` ### Step 6: DNS and failover - Use Azure Traffic Manager to direct traffic to the primary NGINXaaS deployment. - When the primary deployment is detected as being unhealthy, Azure Traffic Manager updates the public DNS record of your service to point to the public IP of the NGINXaaS deployment in the secondary region. #### Terraform code for configuring Azure Traffic Manager to point to both NGINXaaS deployments ```hcl resource "azurerm_traffic_manager_profile" "nginxaas_failover_monitor" { ... traffic_routing_method = "Priority" # Chooses one deployment or the other dns_config { # relative_name needs to be globally unique # .trafficmanager.net resolves to the public IP of either NGINXaaS deployment relative_name = "nginxaas-global-record" ttl = 60 } monitor_config { protocol = "HTTP" port = 80 path = "/health" #endpoint implemented in NGINXaaS configuration interval_in_seconds = 30 timeout_in_seconds = 9 tolerated_number_of_failures = 3 } } resource "azurerm_traffic_manager_external_endpoint" "primary" { name = "nginx-primary" profile_id = azurerm_traffic_manager_profile.nginxaas_failover_monitor.id priority = 10 # Lower number results in higher priority target = azurerm_nginx_deployment.primary_nginxaas_deployment.ip_address } resource "azurerm_traffic_manager_external_endpoint" "secondary" { name = "nginx-secondary" profile_id = azurerm_traffic_manager_profile.nginxaas_failover_monitor.id priority = 20 target = azurerm_nginx_deployment.secondary_nginxaas_deployment.ip_address } ``` ## Failover process 1. **Monitor**: `/health` endpoint continuously monitors NGINXaaS deployment reachability in both regions. 1. **Failover**: If the primary region deployment is deemed unhealthy, Azure Traffic Manager updates the DNS record for the service to route traffic to the secondary region's NGINXaaS deployment. 1. **Recovery**: Once the primary region deployment recovers, Azure Traffic Manager automatically restores DNS records to the primary endpoint when its health probes detect recovery and confirm the primary endpoint is healthy again. ## Summary By deploying NGINXaaS in separate regions with unique subnets and peered VNets, and configuring upstreams and DNS for failover, this topology ensures high availability and DR for your applications. Lastly, always monitor and test your failover paths. -------------------------------------------------------------------------------- title: "GeoIP2" url: https://docs.nginx.com/nginxaas-azure/quickstart/geoip2/index.md -------------------------------------------------------------------------------- ## Overview F5 NGINXaaS for Azure (NGINXaaS) supports GeoIP2 using the [`ngx_http_geoip2_module` or `ngx_stream_geoip2_module`](https://github.com/leev/ngx_http_geoip2_module) dynamic modules, enabling NGINXaaS to implement various user differentiation strategies. For more information on GeoIP2 with NGINX, see [NGINX GeoIP2](https://docs.nginx.com/nginx/admin-guide/dynamic-modules/geoip2/). NGINXaaS uses your MaxMind license to download GeoIP2 databases, puts them in the right place before NGINX starts, and updates the databases daily to reduce your operational overhead. All GeoIP2 data is deleted once you stop using GeoIP2 or delete your deployment. MaxMind provides a variety of [databases](https://www.maxmind.com/en/geoip-databases), including a lower accuracy [free option](https://www.maxmind.com/en/geolite2/signup). NGINXaaS uses a modified form of [MaxMind's `geoipupdate`](https://github.com/maxmind/geoipupdate). ## Configure To enable GeoIP2 you [update your NGINX configuration](/nginxaas-azure/getting-started/nginx-configuration/overview.md) to include your MaxMind license and the relevant NGINX directives. 1. Log into MaxMind and [generate a `GeoIP.conf`](https://dev.maxmind.com/geoip/updating-databases/#2-obtain-geoipconf-with-account-information) file. 2. Add the `GeoIP.conf` file to your NGINX configuration, using the exact path `/etc/nginx/GeoIP.conf`. The `GeoIP.conf` will be validated, and must include `AccountID`, `LicenseKey`, and `EditionIDs`. Other configuration options in `GeoIP.conf` are ignored. We recommend you enable the **Protected** [icon: toggle-right] toggle button to mark `GeoIP.conf` as a protected file, which will prevent the contents from being read via any Azure client tools. 3. Add the `load_module` directive - the modules are available at `modules/ngx_http_geoip2_module.so` or `modules/ngx_stream_geoip2_module.so`. 4. Add `geoip2` directives to your NGINX configuration as desired. The `EditionIDs` from your `GeoIP.conf` are available at `/usr/local/share/GeoIP` **Note:** NGINXaaS for Azure currently only supports the database directory at the path `/usr/local/share/GeoIP`. There are many different ways to use the `geoip2` directives; For example: ```nginx load_module modules/ngx_http_geoip2_module.so; http { # "GeoLite2-City" is one of the EditionIDs in /etc/nginx/GeoIP.conf geoip2 /usr/local/share/GeoIP/GeoLite2-City.mmdb { $geoip2_city_name city names en; } server { listen 80; server_name localhost; location / { return 200 "Hello $geoip2_city_name"; } } } ``` ## Monitoring All licenses are [validated with MaxMind](https://dev.maxmind.com/license-key-validation-api/) when initially added to your deployment, but MaxMind licenses can expire or be manually revoked. To view the status of your MaxMind license, [enable metrics](/nginxaas-azure/monitoring/enable-metrics.md) for your NGINXaaS deployment and navigate to the Metrics tab. View the `nginxaas.maxmind` metric under the `nginxaas statistics` metric namespace. The `nginxaas.maxmind` metric reports the health of your license through the `status` dimension: | Status | Description | | -------------- | ------------------------------------------------------------------------------------------ | | `active` | The license is valid and in use to update GeoIP2 databases. | | `unauthorized` | MaxMind returned an license error, which usually indicates an issue with the `GeoIP.conf`. | -------------------------------------------------------------------------------- title: "NGINXaaS Managed Identity on dataplane" url: https://docs.nginx.com/nginxaas-azure/quickstart/dataplane-mi/index.md -------------------------------------------------------------------------------- ## Overview F5 NGINX as a Service for Azure (NGINXaaS) supports using a system-assigned or user-assigned Managed Identity (MI) assigned to the NGINXaaS deployment to access other Azure resources in the same virtual network or those that are publicly accessible. NGINX Plus instances that are part of the NGINXaaS deployment now have access to query the identity endpoint in [Instance Metadata Service](https://learn.microsoft.com/en-us/azure/virtual-machines/instance-metadata-service?tabs=windows) to fetch access tokens. **Note:** Exposing the IMDS endpoint externally can allow unintended third parties to retrieve the access tokens associated with the managed identities assigned to the deployment. To mitigate this risk, ensure the endpoint is restricted to internal access or apply appropriate access controls. ## Configuration The example below contains a sample NGINX config that uses [njs](https://nginx.org/en/docs/njs/) and the NGINXaaS MI to query a blob storage container. You can also use the standard MSAL libraries to query IMDS. 1. Assign a system-assigned or user-assigned [identity](/nginxaas-azure/getting-started/managed-identity-portal.md) to the NGINXaaS deployment and grant the identity access to blob storage. 2. Create an NGINX config named `nginx.conf` with the following content: ```nginx user nginx; worker_processes auto; worker_rlimit_nofile 8192; pid /run/nginx/nginx.pid; error_log /var/log/nginx/error.log info; load_module modules/ngx_http_js_module.so; http { js_import /etc/nginx/query_blob.js; error_log /var/log/nginx/error.log info; resolver 168.63.129.16 valid=30s; server { listen 80; location /queryBlob { js_content query_blob.queryBlob; } } } ``` 3. Create a javascript file named `query_blob.js` with the following content: ```javascript async function fetchAccessToken(r) { const resource = "https://storage.azure.com/"; const apiVersion = "2019-08-01"; const imdsEndpoint = `http://169.254.169.254/metadata/identity/oauth2/token?resource=${resource}&api-version=${apiVersion}`; try { const imdsResponse = await ngx.fetch(imdsEndpoint, { headers: { "Metadata": "true" } }); if (imdsResponse.status !== 200) { let resp = JSON.stringify(imdsResponse); r.error(`ERROR: Failed to fetch access token. IMDS returned status: ${resp}`); return null; } let body = await imdsResponse.text(); const imdsResponseBody = JSON.parse(body); let token = imdsResponseBody.access_token; return token; } catch (err) { r.error(`ERROR: Exception occurred while querying IMDS. Details: ${err}`); return null; } } async function queryBlob(r) { const storageAccountName = "test-storage"; const containerName = "test-container"; const apiEndpoint = `https://${storageAccountName}.blob.core.windows.net/${containerName}?restype=container&comp=list`; r.log("fetching access token..."); const accessToken = await fetchAccessToken(r); if (!accessToken) { r.return(401, "Failed to fetch access token."); return; } r.log(`INFO: querying blob storage... ${apiEndpoint}`); try { const reply = await ngx.fetch(apiEndpoint, { method: "GET", headers: { "Authorization": `Bearer ${accessToken}`, "x-ms-version": "2019-12-12" }, verify: false }); const body = await reply.text(); r.return(reply.status, body); } catch (err) { r.error(`ERROR: Exception occurred while querying Blob Storage. Details: ${err}`); r.return(500, "Failed to query Blob Storage."); } } export default { queryBlob }; ``` Sending an HTTP request to the `queryBlob` endpoint triggers njs, which fetches an access token from IMDS and uses it to query blob storage. **Note:** IMDS enforces a rate limit of 5 requests per second. To optimize performance, cache the access token in NGINX instead of retrieving it for every request. -------------------------------------------------------------------------------- title: "Service status" url: https://docs.nginx.com/nginxaas-azure/service-status/index.md -------------------------------------------------------------------------------- You can monitor the health status of the F5 NGINXaaS service at the [F5 Cloud Status page](https://www.f5cloudstatus.com/history?filter=88pmy2nlbd01). The status page provides information related to service degradation and unscheduled downtime. When appropriate, we provide mitigations to minimize the impact to your deployments. You may also subscribe to receive emails, webhooks, and RSS feeds when an incident related to NGINXaaS is created, updated, or resolved. -------------------------------------------------------------------------------- title: "Azure CLI" url: https://docs.nginx.com/nginxaas-azure/client-tools/cli/index.md description: Learn how to setup the Azure CLI to manage NGINXaaS for Azure. -------------------------------------------------------------------------------- F5 NGINXaaS for Azure (NGINXaaS) deployments can be managed using the [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/). This document outlines how to install the CLI tool including the NGINX extension. ## Prerequisites - Install Azure CLI version 2.67.0 or greater: [Azure CLI Installation](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli)) - Log into your Azure account through the CLI: [Azure CLI Authentication](https://learn.microsoft.com/en-us/cli/azure/authenticate-azure-cli). ## Install NGINXaaS extension In order to install and manage your NGINXaaaS deployments using the Azure CLI, you will need to install the `nginx` extension: ```shell az extension add --name nginx --allow-preview true ``` ## Update NGINXaaS extension Ensure you are running the latest version of the `nginx` CLI extension to take advantage of the latest capabilities available on your NGINXaaS deployments: ```shell az extension update --name nginx --allow-preview true ``` -------------------------------------------------------------------------------- title: "Known issues" url: https://docs.nginx.com/nginxaas-azure/known-issues/index.md -------------------------------------------------------------------------------- List of known issues in the latest releases of F5 NGINXaaS for Azure. ### NGINX Plus 37.0 behavior impact The release of NGINX Plus [37.0](/nginx/releases/#pls.37.0.4) introduces the following behavioral changes that may affect upstream applications: - HTTP/1.1 is now the default protocol for connecting to proxy or upstream servers. - Keepalive connections between NGINX and upstream servers are enabled by default. - Upstream shared memory zone requires an additional 1KB of memory per upstream server. To preserve existing behavior, make the following configuration changes to your NGINX Plus [R36-P8](/nginx/releases/#r36) configuration before NGINXaaS is upgraded to NGINX Plus 37.0.4 on the stable upgrade channel (August 17, 2026): - Explicitly set HTTP/1.0 as the default protocol for communicating with upstream or proxy servers: ```shell http { proxy_http_version 1.0; ... } ``` - For all upstream or proxy servers that explicitly use the HTTP/1.1 protocol, use the [proxy_set_header](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_header) directive to ensure the connection is closed after completing the request-response cycle: ```shell location /example { proxy_set_header Connection "close"; proxy_http_version 1.1; ... } ``` - Set the [`keepalive`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive) directive to zero for all HTTP/1.1 upstream blocks where it has not been explicitly defined: ```shell upstream backend { ... keepalive 0; } ``` - Increase the upstream block shared memory [zone](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#zone) size by approximately 1KB per upstream server, rounding up to leave some headroom: ```shell upstream backend { ... zone backend 64k; # Example only. New size depends on previous settings. } ``` For further information on the new behavior of NGINX Plus 37.0, see **NGINX Plus PLS.37.0.4.1 LTS** [Upgrade Notes](/nginx/releases/#pls.37.0.4). ### [icon: bug] Terraform fails to apply due to validation errors, but creates "Failed" resources in Azure (ID-4424) Some validation errors are caught later in the creation process, and can leave behind "Failed" resources in Azure. An example initial failure might look like: ```shell $ terraform apply │ Error: creating Nginx Deployment (Subscription: "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX" │ Resource Group Name: "XXXXXXXX" │ Nginx Deployment Name: "XXXXXXXX"): polling after DeploymentsCreateOrUpdate: polling failed: the Azure API returned the following │ error: │ │ Status: "Failed" │ Code: "NginxSaaSError" │ Message: "{\"Content\":\"{\\\"error\\\":{\\\"code\\\":\\\"CapacityOutOfRange\\\",\\\"message\\\":\\\"The deployment's capacity must │ be between 10 and 500 inclusive for marketplace plan standard. For more information about setting capacity see │ https://docs.nginx.com/nginxaas-azure/quickstart/scaling/.\\\"}}\\n\",\"StatusCode\":400}" ``` The error message describes how to fix the vailidation problem. In the Azure portal, you'll be able to see your NGINXaaS, but it will have a "Failed" status. Future **terraform apply** will fail with **Error: A resource with the ID "..." already exists**. **Workaround**: manually delete the "Failed" resource in Azure portal before re-running **terraform apply**. **terraform import** will not work. ### [icon: bug] Changing IP addresses in `listen` directives fails with "cannot reload nginx: timed out waiting for config to reload" (ID-4366) NGINXaaS uses NGINX's ["change configuration" feature](https://nginx.org/en/docs/control.html#reconfiguration) to update the configuration gracefully without dropping traffic. This starts new workers on the new configuration before shutting down the old workers on the old config. Some kinds of `listen` changes can block new workers from starting up. If you're changing from listening on all IPs to one (for example `listen 1234` -> `listen 127.0.0.1:1234` or vice versa), the config will fail to apply because the old workers and the new workers have an IP conflict. **Workaround**: Change the port as well as the IP address to avoid the conflict, and then make a second config change back to the desired port. ### [icon: bug] Deploying NGINXaaS and Diagnostic Settings for NGINXaaS using a ARM Bicep or JSON template shows an error (ID-4326) While using a single template deployment to deploy both, a NGINXaaS instance and a diagnostic setting for the NGINXaaS instance, you will see a validation error similar to: ``` {"code": "InvalidTemplateDeployment", "message": "The template deployment 'example' is not valid according to the validation procedure. The tracking id is '650afc1e-50d6-476c-bf94-9fc35ffeedd6'. See inner errors for details."} Inner Errors: {"code": "OpenAPISpecValidationFailedForTemplateDeploymentResources", "message": "One or more resources in template deployment preflight validation request failed during OpenApi spec (swagger) validation. Please check error details for the resource identifiers."} Inner Errors: {"code": "HttpPayloadAPISpecValidationFailed", "target": "/subscriptions/ee920d60-90f3-4a92-b5e7-bb284c3a6ce2/resourceGroups/testenv-1b791f58-workload/providers/NGINX.NGINXPLUS/nginxDeployments/myDeployment/providers/Microsoft.Insights/diagnosticSettings/myLoggingSetting", "message": "Failed during request payload validation against the API specification"} ``` **Workaround**: Deploy your NGINXaaS instance and your diagnostic setting in separate templates. ### [icon: bug] Not all NGINX Plus directives and use-cases are supported in NGINXaaS (ID-4331) NGINXaaS currently does not support all NGINX Plus directives and use-cases. We are continually adding new NGINX Plus capabilities into NGINXaaS to close the gap in functionality. You can follow the updates to the supported use-cases by visiting the [Changelog](/nginxaas-azure/changelog/changelog.md). For a comprehensive list of currently allowed directives, please see the [Configuration Directives List](/nginxaas-azure/getting-started/nginx-configuration/nginx-configuration-portal.md#configuration-directives-list). ### [icon: bug] Terraform errors using `package_data` (ID-2752) Specifying a configuration using a `tar.gz` file and the `package_data` directive fails. ```text │ Error: Insufficient config_file blocks │ │ on main.tf line 105, in resource "azurerm_nginx_configuration" "example": │ 105: resource "azurerm_nginx_configuration" "example" { │ │ At least 1 "config_file" blocks are required. ``` **Workaround:** Extract the files from your `tar.gz` and use the `config_file` directive instead of `package_data` ### [icon: bug] Deployment responsiveness takes approximately 5-10 seconds. (ID-872) When creating a new deployment or exposing a new port for traffic, there might be a lag time of 5-10 seconds, during which the Azure Load Balancer does not recognize the new ports, thus preventing making new connections to the NGINX deployment. **Workaround:** Wait 10 seconds to make requests or make multiple requests to the instance with low connect timeout times after creating a new deployment or exposing a new port to the deployment for the first 10 seconds after the deployment reaches the Completed state. ### [icon: bug] NGINXaaS for Azure charges do not render correctly in the Azure Portal cost center. (ID-1660) NGINXaaS for Azure resources appear with a random suffix, and clicking the link does not lead to the NGINXaaS for Azure resource overview page. The charge details show "Unassigned" for all fields, but the charge amount is accurate. ### [icon: bug] Configuration update will not succeed with a failed certificate. (ID-1545) If a configuration update request uses a certificate that is in failed `provisioningState`, the configuration update is rejected. **Workaround:** Update the referenced certificate before updating the configuration. Make sure the certificate provisioning is successful and retry the configuration update. ### [icon: bug] Known networking limitations (ID-625) - NGINXaaS deployments cannot access [Private Endpoints](https://learn.microsoft.com/en-us/azure/private-link/private-endpoint-overview) behind network security groups for private links. Attempts to do so will fail silently. - NGINXaaS deployments cannot access [Private Endpoints](https://learn.microsoft.com/en-us/azure/private-link/private-endpoint-overview) in a globally peered VNET. Attempts to do so will fail silently. - The resource group which contains the public IP resource attached to NGINXaaS deployment cannot be moved across subscriptions. Attempts to do so will result in a validation error. - [Network security group](https://learn.microsoft.com/en-us/azure/virtual-network/network-security-groups-overview) (NSG) flow logs will not be available for IP traffic flowing through an NGINXaaS deployment attached to a customer delegated subnet. Flow logs for other resources on the same subnet will be available as normal. ### [icon: bug] Deployment locked when updating mutliple certificates at once. (ID-767) Attaching multiple certificates to a deployment quickly will result in a deployment conflict and error with a "409" status code. Certificates are a sub-resource of the deployment, and a user cannot attach multiple certificates to a deployment simultaneously. This issue is more likely to occur when attempting to configure multiple certificates using client tools such as Terraform and ARM templates. **Workaround:** If you want to add multiple certificates to a deployment, configure resource dependencies between the certificate resources, which will cause them to be added to the deployment one at a time. **Terraform:** Use [depends_on](https://developer.hashicorp.com/terraform/language/meta-arguments/depends_on) to add a dependency between certificate resources: ```hcl {linenos=false,hl_lines=[16]} resource "azurerm_nginx_certificate" "cert1" { name = "examplecert" nginx_deployment_id = azurerm_nginx_deployment.test.id key_virtual_path = "/src/cert/soservermekey.key" certificate_virtual_path = "/src/cert/server.cert" key_vault_secret_id = azurerm_key_vault_certificate.test.secret_id } resource "azurerm_nginx_certificate" "cert2" { name = "examplecert" nginx_deployment_id = azurerm_nginx_deployment.test.id key_virtual_path = "/src/cert/soservermekey.key" certificate_virtual_path = "/src/cert/server.cert" key_vault_secret_id = azurerm_key_vault_certificate.test.secret_id depends_on = [azurerm_nginx_certificate.cert1] } ``` **ARM Template** Use [dependsOn](https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/resource-dependency) to add a dependency between certificate resources: ```json {linenos=false,hl_lines=[21]} { "type": "NGINX.NGINXPLUS/nginxDeployments/certificates", "apiVersion": "2021-05-01-preview", "name": "[concat(parameters('nginxDeploymentName'), '/', 'cert1')]", "properties": { "certificateVirtualPath": "[parameters('certificateVirtualPath')]", "keyVirtualPath": "[parameters('keyVirtualPath')]", "keyVaultSecretId": "[parameters('keyVaultSecretId')]" } } { "type": "NGINX.NGINXPLUS/nginxDeployments/certificates", "apiVersion": "2021-05-01-preview", "name": "[concat(parameters('nginxDeploymentName'), '/', 'cert2')]", "properties": { "certificateVirtualPath": "[parameters('certificateVirtualPath')]", "keyVirtualPath": "[parameters('keyVirtualPath')]", "keyVaultSecretId": "[parameters('keyVaultSecretId')]" } "dependsOn": ["cert1"] } ``` **Bicep Template** Use [dependsOn](https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/resource-dependencies) to add a dependency between certificate resources: ```bicep {linenos=false,hl_lines=[17]} resource cert1 'NGINX.NGINXPLUS/nginxDeployments/certificates@2021-05-01-preview' = { name: '${nginxDeploymentName}/cert1' properties: { certificateVirtualPath: certificateVirtualPath keyVirtualPath: keyVirtualPath keyVaultSecretId: keyVaultSecretId } } resource cert2 'NGINX.NGINXPLUS/nginxDeployments/certificates@2021-05-01-preview' = { name: '${nginxDeploymentName}/cert2' properties: { certificateVirtualPath: certificateVirtualPath keyVirtualPath: keyVirtualPath keyVaultSecretId: keyVaultSecretId } dependsOn: [cert1] } ``` ### [icon: bug] Terraform errors around capacity for Basic plan deployments (ID-4880) Basic plans have no capacity, but older versions of [`azurerm`](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs) do not handle that well. You may see errors like: - `azurerm_nginx_deployment` falsely detecting capacity changes from 0 to 20 - `UnsupportedOnBasicPlan: The Basic plan does not support scaling.` errors when running `terraform apply` **Solution** Upgrade `azurerm` to version v3.116.0 or higher. -------------------------------------------------------------------------------- title: "Migrate from Custom metrics to Platform metrics" url: https://docs.nginx.com/nginxaas-azure/monitoring/migrate-to-platform-metrics/index.md -------------------------------------------------------------------------------- ## Overview F5 NGINXaaS for Azure previously supported monitoring through [Custom Metrics](https://learn.microsoft.com/en-us/azure/azure-monitor/metrics/metrics-custom-overview), which is a preview feature in Azure. Support for Custom Metrics will be removed in the future. We've added support for Platform Metrics, which is the recommended way to monitor resources in Azure. We strongly recommend switching your deployment's monitoring to Platform Metrics to take advantage of lower latency and better reliability. ## Migration steps Follow the steps in this section to migrate your deployment monitoring from Custom metrics to Platform metrics. 1. Verify that your NGINXaaS deployment meets the [pre-requisites](/nginxaas-azure/monitoring/enable-metrics.md#prerequisites) for Platform metrics to work. 2. If the pre-requisites are met, Platform metrics are enabled by default on all NGINXaaS deployment. Verify that you are able to see the new metrics in Azure Monitor under the `Standard Metrics` namespace. 3. **Migrate existing alert rules** to use Platform metrics instead of Custom metrics: If you have existing alert rules configured for Custom metrics, you need to update them to use the equivalent Platform metrics signals. 1. In the [Azure portal](https://portal.azure.com), go to **Monitor** > **Alerts**. 2. Select **Alert rules**. 3. Select the alert rule you want to migrate, then select **Edit**. 4. In the **Conditions** tab, select the current **Signal name** to modify it. 5. Replace the Custom metrics signal with the corresponding Platform metrics signal: - Platform metrics signals use the `NGINXaaS Standard Metrics` namespace 6. The metric name remains the same between Custom and Platform metrics (for example, `nginx.http.request.count`). No update to the signal name is needed. 7. Review and adjust the alert logic, thresholds, and conditions as needed. 8. Select **Review + create**, then **Create** to save the updated alert rule. **Note:** For a complete list of available Platform metrics, see the [metrics catalog](/nginxaas-azure/monitoring/metrics-catalog.md). To learn more about editing alert rules, refer to the [Edit an existing alert rule](https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/alerts-create-metric-alert-rule#edit-an-existing-alert-rule) section in the Azure documentation. -------------------------------------------------------------------------------- title: "F5 NGINXaaS Product Documentation" url: https://docs.nginx.com/nginxaas/index.md description: Learn how to deliver, manage, and protect your applications using F5 NGINXaaS. -------------------------------------------------------------------------------- Learn how to deliver, manage, and protect your applications using F5 NGINXaaS. ## F5 NGINXaaS F5 NGINXaaS is a SaaS offering that is tightly integrated into your public cloud provider and its ecosystem of services, making applications fast, efficient, and reliable, and bringing advanced traffic services enabled with the commercial version of NGINX, without any of the operational toil. ### Featured content ### [Manage NGINX configuration](/nginxaas/overview/nginx-configuration/nginx-configuration-console/) Follow these steps to create an NGINX configuration for your NGINXaaS deployment ### [Create a deployment in Google Cloud](/nginxaas/google/deploy/create-deployment/deploy-console/) Step-by-step instructions to deploy NGINXaaS in Google Cloud using the NGINXaaS Console ### [Create a deployment in AWS](/nginxaas/aws/deploy/create-deployment/deploy-console/) Step-by-step instructions to deploy NGINXaaS in AWS using the NGINXaaS Console ### Get started with NGINXaaS Access the [NGINXaaS Console](https://console.nginxaas.net/) directly to manage organizations, NGINX configurations, and certificates. To create and manage deployments, subscribe to F5 NGINXaaS using your preferred cloud provider marketplace. ### [NGINXaaS for AWS](/nginxaas/aws/deploy/prerequisites/) Use NGINXaaS alongside your services running in AWS ### [NGINXaaS for Google Cloud](/nginxaas/google/deploy/prerequisites/) Deploy NGINXaaS for use with your Google Cloud services ### Get help ### [Get help](/nginxaas/support/) Get in touch with the NGINX support team for assistance -------------------------------------------------------------------------------- title: "NGINX configuration rules and limits" url: https://docs.nginx.com/nginxaas/aws/deploy/nginx-configuration/configuration-rules/index.md description: Reference for NGINX configuration file restrictions and workflows for F5 NGINXaaS for AWS. -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- title: "NGINX configuration rules and limits" url: https://docs.nginx.com/nginxaas/google/deploy/nginx-configuration/configuration-rules/index.md -------------------------------------------------------------------------------- This document provides details about using NGINX configuration files with your F5 NGINXaaS for Google Cloud deployment, restrictions, and available directives. ## NGINX configuration common user workflows You can upload NGINX configurations to your NGINXaaS for Google Cloud deployment using the Google Cloud console: - [Upload using the console](/nginxaas/google/deploy/nginx-configuration/nginx-configuration-console.md) The topics below explain NGINX configuration restrictions and which directives are supported, unsupported, or cannot be overridden. ## NGINX filesystem restrictions There are limits to where files, including NGINX configuration files, certificate files, and any other files uploaded to the deployment, can be placed on the filesystem. There are also limits on what directories NGINX can access during runtime. These limits help support the separation of roles, enforce the principle of least privilege, and ensure the smooth operation of the system. | Allowed Directory | User can upload files to | NGINX master process can read | NGINX master process can write | NGINX worker process can read | NGINX worker process can write | | -------------------- | -------------------- | -------------------- | -------------------- | -------------------- | -------------------- | | /etc/nginx | [icon: check] | [icon: check] | | | | | /opt | [icon: check] | [icon: check] | [icon: check] | [icon: check] | [icon: check] | | /srv | [icon: check] | [icon: check] | | [icon: check] | | | /tmp | | [icon: check] | [icon: check] | [icon: check] | [icon: check] | | /spool/nginx | | [icon: check] | [icon: check] | [icon: check] | [icon: check] | | /var/cache/nginx | | [icon: check] | [icon: check] | [icon: check] | [icon: check] | | /var/spool/nginx | | [icon: check] | [icon: check] | [icon: check] | [icon: check] | | /var/www | [icon: check] | [icon: check] | | [icon: check] | | For example, `/etc/nginx` is only readable by the NGINX master process, making it a secure location for certificate files that won't be accidentally served due to configuration errors. `/var/www` is a secure location for static content because the NGINX worker process can serve files from it but cannot modify them, ensuring content integrity. `/tmp` is a good choice for storing temporary files with `proxy_temp_path` or `client_body_temp_path` since it is writable by the NGINX worker process. If you need access to additional directories, please [contact us](/nginxaas/google/support.md). ## Disallowed configuration directives The following directives are not supported because of specific limitations. If you include any of these directives in your NGINX configuration, you'll get an error. | Disallowed Directive | Reason | |------------------ | ----------------- | | ssl_engine | No hardware SSL accelerator is available. | | debug_points | NGINXaaS does not provide access to NGINX processes for debugging. | | fastcgi_bind
grpc_bind
memcached_bind
proxy_bind
scgi_bind
uwsgi_bind | Source IP specification for active-active deployments is not allowed. | | quic_bpf | QUIC connection migration is not currently supported for active-active deployments. | | ssl_ech_file | NGINXaaS does not use the [OpenSSL ECH feature branch](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_ech_file). | You may find a few directives are not listed here as either allowed or disallowed. Our team is working on getting these directives supported soon. ## Directives that cannot be overridden The following directives cannot be overridden by the user provided configuration. | Persistent Directive | Value | Reason | |------------------ | ----------------------- | -----------------| | `user` | `nginx` | The `nginx` user has the correct permissions for accessing certificates, policy files and other auxfiles. | | `worker_processes` | `auto` | Set to `auto` to automatically set `worker_processes` to the number of CPU cores. | | `worker_rlimit_nofile` | `524288` | Set for optimal performance of deployments. | | `worker_connections` | `150000` | Set for optimal performance of deployments. | | `pid` | `/run/nginx/nginx.pid` | Set to this value to allow NGINXaaS to automatically manage the NGINX master process. | | `daemon` | `on` | Automatically set to `on` to allow NGINXaaS to manage the NGINX master process. | | `master_process` | `on` | This directive is intended for NGINX developers. | | `worker_cpu_affinity` | `auto` | The value `auto` allows binding worker processes automatically to available CPUs based on the current capacity of the deployment. | For connection and request rate limiting, consider using these NGINX modules: - [limit_conn](https://nginx.org/en/docs/http/ngx_http_limit_conn_module.html#limit_conn) - [limit_req](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req) - [upstream queue](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#queue) ## Configuration directives list NGINXaaS supports a limited set of NGINX directives. The directives with the "app_protect" prefix require [F5 WAF for NGINX](/waf) to be enabled for the deployment. NGINXaaS does not yet support F5 WAF for NGINX custom security policies or logging profiles. Support is limited to the [prebuilt policies](/waf/policies/configuration.md#default-policy) and the [default logging profiles](/waf/logging/logs-overview.md#default-logging-profile-bundles). #### Alphabetical index of directives [absolute_redirect](https://nginx.org/en/docs/http/ngx_http_core_module.html#absolute_redirect)\ [accept_mutex](https://nginx.org/en/docs/ngx_core_module.html#accept_mutex)\ [accept_mutex_delay](https://nginx.org/en/docs/ngx_core_module.html#accept_mutex_delay)\ [access_log (ngx_http_log_module)](https://nginx.org/en/docs/http/ngx_http_log_module.html#access_log)\ [access_log (ngx_stream_log_module)](https://nginx.org/en/docs/stream/ngx_stream_log_module.html#access_log)\ [add_after_body](https://nginx.org/en/docs/http/ngx_http_addition_module.html#add_after_body)\ [add_before_body](https://nginx.org/en/docs/http/ngx_http_addition_module.html#add_before_body)\ [add_header](https://nginx.org/en/docs/http/ngx_http_headers_module.html#add_header)\ [add_trailer](https://nginx.org/en/docs/http/ngx_http_headers_module.html#add_trailer)\ [addition_types](https://nginx.org/en/docs/http/ngx_http_addition_module.html#addition_types)\ [aio](https://nginx.org/en/docs/http/ngx_http_core_module.html#aio)\ [aio_write](https://nginx.org/en/docs/http/ngx_http_core_module.html#aio_write)\ [alias](https://nginx.org/en/docs/http/ngx_http_core_module.html#alias)\ [allow (ngx_http_access_module)](https://nginx.org/en/docs/http/ngx_http_access_module.html#allow)\ [allow (ngx_stream_access_module)](https://nginx.org/en/docs/stream/ngx_stream_access_module.html#allow)\ [ancient_browser](https://nginx.org/en/docs/http/ngx_http_browser_module.html#ancient_browser)\ [ancient_browser_value](https://nginx.org/en/docs/http/ngx_http_browser_module.html#ancient_browser_value)\ [app_protect_compressed_requests_action](/waf/policies/directives/#global-directives:~:text=app_protect_compressed_requests_action)\ [app_protect_cookie_seed](/waf/policies/directives/#global-directives:~:text=app_protect_cookie_seed)\ [app_protect_cpu_thresholds](/waf/policies/directives/#global-directives:~:text=app_protect_cpu_thresholds)\ [app_protect_custom_log_attribute](/waf/policies/directives/#f5-waf-for-nginx-directives:~:text=app_protect_custom_log_attribute)\ [app_protect_enable](/waf/policies/directives/#f5-waf-for-nginx-directives:~:text=modules/ngx_http_app_protect_module.so-,app_protect_enable)\ [app_protect_failure_mode_action](/waf/policies/directives/#global-directives:~:text=app_protect_failure_mode_action)\ [app_protect_physical_memory_util_thresholds](/waf/policies/directives/#global-directives:~:text=app_protect_physical_memory_util_thresholds)\ [app_protect_policy_file]()\ [app_protect_reconnect_period_seconds](/waf/policies/directives/#global-directives:~:text=app_protect_reconnect_period_seconds)\ [app_protect_request_buffer_overflow_action](/waf/policies/directives/#global-directives:~:text=app_protect_request_buffer_overflow_action)\ [app_protect_security_log](/waf/policies/directives/#f5-waf-for-nginx-directives:~:text=app_protect_security_log)\ [app_protect_security_log_enable](/waf/policies/directives/#f5-waf-for-nginx-directives:~:text=app_protect_security_log_enable)\ [app_protect_user_defined_signatures](/waf/policies/directives/#global-directives:~:text=app_protect_user_defined_signatures)\ [auth_basic](https://nginx.org/en/docs/http/ngx_http_auth_basic_module.html#auth_basic)\ [auth_basic_user_file](https://nginx.org/en/docs/http/ngx_http_auth_basic_module.html#auth_basic_user_file)\ [auth_delay](https://nginx.org/en/docs/http/ngx_http_core_module.html#auth_delay)\ [auth_http](https://nginx.org/en/docs/mail/ngx_mail_auth_http_module.html#auth_http)\ [auth_http_header](https://nginx.org/en/docs/mail/ngx_mail_auth_http_module.html#auth_http_header)\ [auth_http_pass_client_cert](https://nginx.org/en/docs/mail/ngx_mail_auth_http_module.html#auth_http_pass_client_cert)\ [auth_http_timeout](https://nginx.org/en/docs/mail/ngx_mail_auth_http_module.html#auth_http_timeout)\ [auth_jwt](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt)\ [auth_jwt_claim_set](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_claim_set)\ [auth_jwt_header_set](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_header_set)\ [auth_jwt_key_cache](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_cache)\ [auth_jwt_key_file](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_file)\ [auth_jwt_key_request](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_request)\ [auth_jwt_leeway](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_leeway)\ [auth_jwt_require](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_require)\ [auth_jwt_type](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_type)\ [auth_oidc](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#auth_oidc)\ [auth_request](https://nginx.org/en/docs/http/ngx_http_auth_request_module.html#auth_request)\ [auth_request_set](https://nginx.org/en/docs/http/ngx_http_auth_request_module.html#auth_request_set)\ [autoindex](https://nginx.org/en/docs/http/ngx_http_autoindex_module.html#autoindex)\ [autoindex_exact_size](https://nginx.org/en/docs/http/ngx_http_autoindex_module.html#autoindex_exact_size)\ [autoindex_format](https://nginx.org/en/docs/http/ngx_http_autoindex_module.html#autoindex_format)\ [autoindex_localtime](https://nginx.org/en/docs/http/ngx_http_autoindex_module.html#autoindex_localtime)\ [break](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#break)\ [connect_timeout](https://nginx.org/en/docs/ngx_mgmt_module.html#connect_timeout)\ [charset](https://nginx.org/en/docs/http/ngx_http_charset_module.html#charset)\ [charset_map](https://nginx.org/en/docs/http/ngx_http_charset_module.html#charset_map)\ [charset_types](https://nginx.org/en/docs/http/ngx_http_charset_module.html#charset_types)\ [chunked_transfer_encoding](https://nginx.org/en/docs/http/ngx_http_core_module.html#chunked_transfer_encoding)\ [client_body_buffer_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_buffer_size)\ [client_body_in_file_only](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_in_file_only)\ [client_body_in_single_buffer](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_in_single_buffer)\ [client_body_temp_path](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_temp_path)\ [client_body_timeout](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_timeout)\ [client_header_buffer_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_header_buffer_size)\ [client_header_timeout](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_header_timeout)\ [client_id](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_id)\ [client_max_body_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size)\ [client_secret](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_secret)\ [config_url](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#config_url)\ [connection_pool_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#connection_pool_size)\ [cookie_name](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#cookie_name)\ [create_full_put_path](https://nginx.org/en/docs/http/ngx_http_dav_module.html#create_full_put_path)\ [daemon](https://nginx.org/en/docs/ngx_core_module.html#daemon)\ [dav_access](https://nginx.org/en/docs/http/ngx_http_dav_module.html#dav_access)\ [dav_methods](https://nginx.org/en/docs/http/ngx_http_dav_module.html#dav_methods)\ [debug_connection](https://nginx.org/en/docs/ngx_core_module.html#debug_connection)\ [default_type](https://nginx.org/en/docs/http/ngx_http_core_module.html#default_type)\ [deny (ngx_http_access_module)](https://nginx.org/en/docs/http/ngx_http_access_module.html#deny)\ [deny (ngx_stream_access_module)](https://nginx.org/en/docs/stream/ngx_stream_access_module.html#deny)\ [directio](https://nginx.org/en/docs/http/ngx_http_core_module.html#directio)\ [directio_alignment](https://nginx.org/en/docs/http/ngx_http_core_module.html#directio_alignment)\ [disable_symlinks](https://nginx.org/en/docs/http/ngx_http_core_module.html#disable_symlinks)\ [empty_gif](https://nginx.org/en/docs/http/ngx_http_empty_gif_module.html#empty_gif)\ [enforce_initial_report](https://nginx.org/en/docs/ngx_mgmt_module.html#enforce_initial_report)\ [env](https://nginx.org/en/docs/ngx_core_module.html#env)\ [error_log](https://nginx.org/en/docs/ngx_core_module.html#error_log)\ [error_log_tag](https://nginx.org/en/docs/http/ngx_http_core_module.html#error_log_tag)\ [error_page](https://nginx.org/en/docs/http/ngx_http_core_module.html#error_page)\ [etag](https://nginx.org/en/docs/http/ngx_http_core_module.html#etag)\ [events](https://nginx.org/en/docs/ngx_core_module.html#events)\ [expires](https://nginx.org/en/docs/http/ngx_http_headers_module.html#expires)\ [extra_auth_args](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#extra_auth_args)\ [f4f](https://nginx.org/en/docs/http/ngx_http_f4f_module.html#f4f)\ [f4f_buffer_size](https://nginx.org/en/docs/http/ngx_http_f4f_module.html#f4f_buffer_size)\ [fastcgi_buffer_size](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_buffer_size)\ [fastcgi_buffering](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_buffering)\ [fastcgi_buffers](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_buffers)\ [fastcgi_busy_buffers_size](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_busy_buffers_size)\ [fastcgi_cache](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache)\ [fastcgi_cache_background_update](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_background_update)\ [fastcgi_cache_bypass](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_bypass)\ [fastcgi_cache_key](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_key)\ [fastcgi_cache_lock](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_lock)\ [fastcgi_cache_lock_age](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_lock_age)\ [fastcgi_cache_lock_timeout](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_lock_timeout)\ [fastcgi_cache_max_range_offset](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_max_range_offset)\ [fastcgi_cache_methods](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_methods)\ [fastcgi_cache_min_uses](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_min_uses)\ [fastcgi_cache_path](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_path)\ [fastcgi_cache_revalidate](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_revalidate)\ [fastcgi_cache_use_stale](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_use_stale)\ [fastcgi_cache_valid](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_valid)\ [fastcgi_catch_stderr](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_catch_stderr)\ [fastcgi_connect_timeout](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_connect_timeout)\ [fastcgi_force_ranges](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_force_ranges)\ [fastcgi_hide_header](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_hide_header)\ [fastcgi_ignore_client_abort](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_ignore_client_abort)\ [fastcgi_ignore_headers](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_ignore_headers)\ [fastcgi_index](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_index)\ [fastcgi_intercept_errors](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_intercept_errors)\ [fastcgi_keep_conn](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_keep_conn)\ [fastcgi_limit_rate](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_limit_rate)\ [fastcgi_max_temp_file_size](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_max_temp_file_size)\ [fastcgi_next_upstream](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_next_upstream)\ [fastcgi_next_upstream_timeout](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_next_upstream_timeout)\ [fastcgi_next_upstream_tries](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_next_upstream_tries)\ [fastcgi_no_cache](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_no_cache)\ [fastcgi_param](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_param)\ [fastcgi_pass](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_pass)\ [fastcgi_pass_header](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_pass_header)\ [fastcgi_pass_request_body](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_pass_request_body)\ [fastcgi_pass_request_headers](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_pass_request_headers)\ [fastcgi_read_timeout](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_read_timeout)\ [fastcgi_request_buffering](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_request_buffering)\ [fastcgi_send_lowat](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_send_lowat)\ [fastcgi_send_timeout](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_send_timeout)\ [fastcgi_socket_keepalive](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_socket_keepalive)\ [fastcgi_split_path_info](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_split_path_info)\ [fastcgi_store](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_store)\ [fastcgi_store_access](http://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_store_access)\ [fastcgi_temp_file_write_size](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_temp_file_write_size)\ [fastcgi_temp_path](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_temp_path)\ [flv](https://nginx.org/en/docs/http/ngx_http_flv_module.html#flv)\ [geo (ngx_http_geo_module)](https://nginx.org/en/docs/http/ngx_http_geo_module.html#geo)\ [geo (ngx_stream_geo_module)](https://nginx.org/en/docs/stream/ngx_stream_geo_module.html#geo)\ [grpc_buffer_size](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_buffer_size)\ [grpc_connect_timeout](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_connect_timeout)\ [grpc_hide_header](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_hide_header)\ [grpc_ignore_headers](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ignore_headers)\ [grpc_intercept_errors](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_intercept_errors)\ [grpc_next_upstream](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_next_upstream)\ [grpc_next_upstream_timeout](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_next_upstream_timeout)\ [grpc_next_upstream_tries](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_next_upstream_tries)\ [grpc_pass](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_pass)\ [grpc_pass_header](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_pass_header)\ [grpc_read_timeout](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_read_timeout)\ [grpc_send_timeout](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_send_timeout)\ [grpc_set_header](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_set_header)\ [grpc_socket_keepalive](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_socket_keepalive)\ [grpc_ssl_certificate](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_certificate)\ [grpc_ssl_certificate_cache](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_certificate_cache)\ [grpc_ssl_certificate_key](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_certificate_key)\ [grpc_ssl_ciphers](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_ciphers)\ [grpc_ssl_conf_command](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_conf_command)\ [grpc_ssl_crl](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_crl)\ [grpc_ssl_name](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_name)\ [grpc_ssl_password_file](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_password_file)\ [grpc_ssl_protocols](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_protocols)\ [grpc_ssl_server_name](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_server_name)\ [grpc_ssl_session_reuse](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_session_reuse)\ [grpc_ssl_trusted_certificate](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_trusted_certificate)\ [grpc_ssl_verify](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_verify)\ [grpc_ssl_verify_depth](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_verify_depth)\ [gunzip](https://nginx.org/en/docs/http/ngx_http_gunzip_module.html#gunzip)\ [gunzip_buffers](https://nginx.org/en/docs/http/ngx_http_gunzip_module.html#gunzip_buffers)\ [gzip](https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip)\ [gzip_buffers](https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_buffers)\ [gzip_comp_level](https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_comp_level)\ [gzip_disable](https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_disable)\ [gzip_http_version](https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_http_version)\ [gzip_min_length](https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_min_length)\ [gzip_proxied](https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_proxied)\ [gzip_static](https://nginx.org/en/docs/http/ngx_http_gzip_static_module.html#gzip_static)\ [gzip_types](https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_types)\ [gzip_vary](https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_vary)\ [hash (ngx_http_upstream_module)](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#hash)\ [hash (ngx_stream_upstream_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#hash)\ [health_check (ngx_http_upstream_hc_module)](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#health_check)\ [health_check (ngx_stream_upstream_hc_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#health_check)\ [health_check_timeout](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#health_check_timeout)\ [hls](https://nginx.org/en/docs/http/ngx_http_hls_module.html#hls)\ [hls_buffers](https://nginx.org/en/docs/http/ngx_http_hls_module.html#hls_buffers)\ [hls_forward_args](https://nginx.org/en/docs/http/ngx_http_hls_module.html#hls_forward_args)\ [hls_fragment](https://nginx.org/en/docs/http/ngx_http_hls_module.html#hls_fragment)\ [hls_mp4_buffer_size](https://nginx.org/en/docs/http/ngx_http_hls_module.html#hls_mp4_buffer_size)\ [hls_mp4_max_buffer_size](https://nginx.org/en/docs/http/ngx_http_hls_module.html#hls_mp4_max_buffer_size)\ [http](https://nginx.org/en/docs/http/ngx_http_core_module.html#http)\ [http2_body_preread_size](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_body_preread_size)\ [http2_chunk_size](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_chunk_size)\ [http2_idle_timeout](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_idle_timeout)\ [http2_max_concurrent_pushes](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_max_concurrent_pushes)\ [http2_max_concurrent_streams](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_max_concurrent_streams)\ [http2_max_field_size](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_max_field_size)\ [http2_max_header_size](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_max_header_size)\ [http2_max_requests](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_max_requests)\ [http2_push](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_push)\ [http2_push_preload](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_push_preload)\ [http2_recv_buffer_size](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_recv_buffer_size)\ [http2_recv_timeout](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_recv_timeout)\ [http3](http://nginx.org/en/docs/http/ngx_http_v3_module.html#http3)\ [http3_hq](http://nginx.org/en/docs/http/ngx_http_v3_module.html#http3_hq)\ [http3_max_concurrent_streams](http://nginx.org/en/docs/http/ngx_http_v3_module.html#http3_max_concurrent_streams)\ [http3_stream_buffer_size](http://nginx.org/en/docs/http/ngx_http_v3_module.html#http3_stream_buffer_size)\ [if](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#if)\ [if_modified_since](https://nginx.org/en/docs/http/ngx_http_core_module.html#if_modified_since)\ [ignore_invalid_headers](https://nginx.org/en/docs/http/ngx_http_core_module.html#ignore_invalid_headers)\ [image_filter](http://nginx.org/en/docs/http/ngx_http_image_filter_module.html#image_filter)\ [image_filter_buffer](http://nginx.org/en/docs/http/ngx_http_image_filter_module.html#image_filter_buffer)\ [image_filter_interlace](http://nginx.org/en/docs/http/ngx_http_image_filter_module.html#image_filter_interlace)\ [image_filter_jpeg_quality](http://nginx.org/en/docs/http/ngx_http_image_filter_module.html#image_filter_jpeg_quality)\ [image_filter_sharpen](http://nginx.org/en/docs/http/ngx_http_image_filter_module.html#image_filter_sharpen)\ [image_filter_transparency](http://nginx.org/en/docs/http/ngx_http_image_filter_module.html#image_filter_transparency)\ [image_filter_webp_quality](http://nginx.org/en/docs/http/ngx_http_image_filter_module.html#image_filter_webp_quality)\ [imap_auth](https://nginx.org/en/docs/mail/ngx_mail_imap_module.html#imap_auth)\ [imap_capabilities](https://nginx.org/en/docs/mail/ngx_mail_imap_module.html#imap_capabilities)\ [imap_client_buffer](https://nginx.org/en/docs/mail/ngx_mail_imap_module.html#imap_client_buffer)\ [include](https://nginx.org/en/docs/ngx_core_module.html#include)\ [index](https://nginx.org/en/docs/http/ngx_http_index_module.html#index)\ [internal](https://nginx.org/en/docs/http/ngx_http_core_module.html#internal)\ [internal_redirect](http://nginx.org/en/docs/http/ngx_http_internal_redirect_module.html#internal_redirect)\ [ip_hash](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#ip_hash)\ [issuer](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#issuer)\ [js_access (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_access)\ [js_body_filter](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_body_filter)\ [js_content](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_content)\ [js_fetch_buffer_size (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_fetch_buffer_size)\ [js_fetch_buffer_size (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_fetch_buffer_size)\ [js_fetch_ciphers (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_fetch_ciphers)\ [js_fetch_ciphers (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_fetch_ciphers)\ [js_fetch_max_response_buffer_size (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_fetch_max_response_buffer_size)\ [js_fetch_max_response_buffer_size (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_fetch_max_response_buffer_size)\ [js_fetch_protocols (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_fetch_protocols)\ [js_fetch_protocols (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_fetch_protocols)\ [js_fetch_timeout (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_fetch_timeout)\ [js_fetch_timeout (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_fetch_timeout)\ [js_fetch_trusted_certificate (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_fetch_trusted_certificate)\ [js_fetch_trusted_certificate (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_fetch_trusted_certificate)\ [js_fetch_verify (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_fetch_verify)\ [js_fetch_verify (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_fetch_verify)\ [js_fetch_verify_depth (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_fetch_verify_depth)\ [js_fetch_verify_depth (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_fetch_verify_depth)\ [js_filter (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_filter)\ [js_header_filter](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_header_filter)\ [js_import (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_import)\ [js_import (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_import)\ [js_include (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_include)\ [js_include (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_include)\ [js_path (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_path)\ [js_path (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_path)\ [js_periodic (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_periodic)\ [js_periodic (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_periodic)\ [js_preload_object (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_preload_object)\ [js_preload_object (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_preload_object)\ [js_preread (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_preread)\ [js_set (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_set)\ [js_set (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_set)\ [js_var (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_var)\ [js_var (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_var)\ [js_shared_dict_zone (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_shared_dict_zone)\ [js_var (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_var)\ [js_var (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_var)\ [keepalive](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive)\ [keepalive_disable](https://nginx.org/en/docs/http/ngx_http_core_module.html#keepalive_disable)\ [keepalive_min_timeout](https://nginx.org/en/docs/http/ngx_http_core_module.html#keepalive_min_timeout)\ [keepalive_requests (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#keepalive_requests)\ [keepalive_time (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#keepalive_time)\ [keepalive_timeout (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#keepalive_timeout)\ [keyval (ngx_http_keyval_module)](https://nginx.org/en/docs/http/ngx_http_keyval_module.html#keyval)\ [keyval (ngx_stream_keyval_module)](https://nginx.org/en/docs/stream/ngx_stream_keyval_module.html#keyval)\ [keyval_zone (ngx_http_keyval_module)](https://nginx.org/en/docs/http/ngx_http_keyval_module.html#keyval_zone)\ [keyval_zone (ngx_stream_keyval_module)](https://nginx.org/en/docs/stream/ngx_stream_keyval_module.html#keyval_zone)\ [large_client_header_buffers](https://nginx.org/en/docs/http/ngx_http_core_module.html#large_client_header_buffers)\ [least_conn (ngx_http_upstream_module)](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#least_conn)\ [least_conn (ngx_stream_upstream_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#least_conn)\ [least_time (ngx_http_upstream_module)](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#least_time)\ [least_time (ngx_stream_upstream_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#least_time)\ [license_token](https://nginx.org/en/docs/ngx_mgmt_module.html#license_token)\ [limit_conn (ngx_http_limit_conn_module)](https://nginx.org/en/docs/http/ngx_http_limit_conn_module.html#limit_conn)\ [limit_conn (ngx_stream_limit_conn_module)](https://nginx.org/en/docs/stream/ngx_stream_limit_conn_module.html#limit_conn)\ [limit_conn_dry_run (ngx_http_limit_conn_module)](https://nginx.org/en/docs/http/ngx_http_limit_conn_module.html#limit_conn_dry_run)\ [limit_conn_dry_run (ngx_stream_limit_conn_module)](https://nginx.org/en/docs/stream/ngx_stream_limit_conn_module.html#limit_conn_dry_run)\ [limit_conn_log_level (ngx_http_limit_conn_module)](https://nginx.org/en/docs/http/ngx_http_limit_conn_module.html#limit_conn_log_level)\ [limit_conn_log_level (ngx_stream_limit_conn_module)](https://nginx.org/en/docs/stream/ngx_stream_limit_conn_module.html#limit_conn_log_level)\ [limit_conn_status](https://nginx.org/en/docs/http/ngx_http_limit_conn_module.html#limit_conn_status)\ [limit_conn_zone (ngx_http_limit_conn_module)](https://nginx.org/en/docs/http/ngx_http_limit_conn_module.html#limit_conn_zone)\ [limit_conn_zone (ngx_stream_limit_conn_module)](https://nginx.org/en/docs/stream/ngx_stream_limit_conn_module.html#limit_conn_zone)\ [limit_except](https://nginx.org/en/docs/http/ngx_http_core_module.html#limit_except)\ [limit_rate](https://nginx.org/en/docs/http/ngx_http_core_module.html#limit_rate)\ [limit_rate_after](https://nginx.org/en/docs/http/ngx_http_core_module.html#limit_rate_after)\ [limit_req](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req)\ [limit_req_dry_run](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_dry_run)\ [limit_req_log_level](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_log_level)\ [limit_req_status](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_status)\ [limit_req_zone](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_zone)\ [limit_zone](https://nginx.org/en/docs/http/ngx_http_limit_conn_module.html#limit_zone)\ [lingering_close](https://nginx.org/en/docs/http/ngx_http_core_module.html#lingering_close)\ [lingering_time](https://nginx.org/en/docs/http/ngx_http_core_module.html#lingering_time)\ [lingering_timeout](https://nginx.org/en/docs/http/ngx_http_core_module.html#lingering_timeout)\ [listen (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#listen)\ [listen (ngx_mail_core_module)](https://nginx.org/en/docs/mail/ngx_mail_core_module.html#listen)\ [load_module](https://nginx.org/en/docs/ngx_core_module.html#load_module)\ [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location)\ [lock_file](http://nginx.org/en/docs/ngx_core_module.html#lock_file)\ [log_format (ngx_http_log_module)](https://nginx.org/en/docs/http/ngx_http_log_module.html#log_format)\ [log_format (ngx_stream_log_module)](https://nginx.org/en/docs/stream/ngx_stream_log_module.html#log_format)\ [log_not_found](https://nginx.org/en/docs/http/ngx_http_core_module.html#log_not_found)\ [log_subrequest](https://nginx.org/en/docs/http/ngx_http_core_module.html#log_subrequest)\ [mail](https://nginx.org/en/docs/mail/ngx_mail_core_module.html#mail)\ [map (ngx_http_map_module)](https://nginx.org/en/docs/http/ngx_http_map_module.html#map)\ [map (ngx_stream_map_module)](https://nginx.org/en/docs/stream/ngx_stream_map_module.html#map)\ [map_hash_bucket_size (ngx_http_map_module)](https://nginx.org/en/docs/http/ngx_http_map_module.html#map_hash_bucket_size)\ [map_hash_bucket_size (ngx_stream_map_module)](https://nginx.org/en/docs/stream/ngx_stream_map_module.html#map_hash_bucket_size)\ [map_hash_max_size (ngx_http_map_module)](https://nginx.org/en/docs/http/ngx_http_map_module.html#map_hash_max_size)\ [map_hash_max_size (ngx_stream_map_module)](https://nginx.org/en/docs/stream/ngx_stream_map_module.html#map_hash_max_size)\ [master_process](https://nginx.org/en/docs/ngx_core_module.html#master_process)\ [match (ngx_http_upstream_hc_module)](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#match)\ [match (ngx_stream_upstream_hc_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#match)\ [max_errors](https://nginx.org/en/docs/mail/ngx_mail_core_module.html#max_errors)\ [max_headers](https://nginx.org/en/docs/http/ngx_http_core_module.html#max_headers)\ [max_ranges](https://nginx.org/en/docs/http/ngx_http_core_module.html#max_ranges)\ [memcached_buffer_size](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_buffer_size)\ [memcached_connect_timeout](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_connect_timeout)\ [memcached_gzip_flag](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_gzip_flag)\ [memcached_next_upstream](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_next_upstream)\ [memcached_next_upstream_timeout](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_next_upstream_timeout)\ [memcached_next_upstream_tries](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_next_upstream_tries)\ [memcached_pass](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_pass)\ [memcached_read_timeout](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_read_timeout)\ [memcached_send_timeout](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_send_timeout)\ [memcached_socket_keepalive](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_socket_keepalive)\ [merge_slashes](https://nginx.org/en/docs/http/ngx_http_core_module.html#merge_slashes)\ [mgmt](https://nginx.org/en/docs/ngx_mgmt_module.html#mgmt)\ [min_delete_depth](https://nginx.org/en/docs/http/ngx_http_dav_module.html#min_delete_depth)\ [mirror](https://nginx.org/en/docs/http/ngx_http_mirror_module.html#mirror)\ [mirror_request_body](https://nginx.org/en/docs/http/ngx_http_mirror_module.html#mirror_request_body)\ [modern_browser](https://nginx.org/en/docs/http/ngx_http_browser_module.html#modern_browser)\ [modern_browser_value](https://nginx.org/en/docs/http/ngx_http_browser_module.html#modern_browser_value)\ [more_clear_headers](https://github.com/openresty/headers-more-nginx-module?tab=readme-ov-file#more_clear_headers)\ [more_clear_input_headers](https://github.com/openresty/headers-more-nginx-module?tab=readme-ov-file#more_clear_input_headers)\ [more_set_headers](https://github.com/openresty/headers-more-nginx-module?tab=readme-ov-file#more_set_headers)\ [more_set_input_headers](https://github.com/openresty/headers-more-nginx-module?tab=readme-ov-file#more_set_input_headers)\ [mp4](https://nginx.org/en/docs/http/ngx_http_mp4_module.html#mp4)\ [mp4_buffer_size](https://nginx.org/en/docs/http/ngx_http_mp4_module.html#mp4_buffer_size)\ [mp4_limit_rate](https://nginx.org/en/docs/http/ngx_http_mp4_module.html#mp4_limit_rate)\ [mp4_limit_rate_after](https://nginx.org/en/docs/http/ngx_http_mp4_module.html#mp4_limit_rate_after)\ [mp4_max_buffer_size](https://nginx.org/en/docs/http/ngx_http_mp4_module.html#mp4_max_buffer_size)\ [mp4_start_key_frame](https://nginx.org/en/docs/http/ngx_http_mp4_module.html#mp4_start_key_frame)\ [mqtt](https://nginx.org/en/docs/stream/ngx_stream_mqtt_filter_module.html#mqtt)\ [mqtt_rewrite_buffer_size](https://nginx.org/en/docs/stream/ngx_stream_mqtt_filter_module.html#mqtt_rewrite_buffer_size)\ [mqtt_set_connect](https://nginx.org/en/docs/stream/ngx_stream_mqtt_filter_module.html#mqtt_set_connect)\ [msie_padding](https://nginx.org/en/docs/http/ngx_http_core_module.html#msie_padding)\ [msie_refresh](https://nginx.org/en/docs/http/ngx_http_core_module.html#msie_refresh)\ [multi_accept](https://nginx.org/en/docs/ngx_core_module.html#multi_accept)\ [ntlm](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#ntlm)\ [oidc_provider](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#oidc_provider)\ [open_file_cache](https://nginx.org/en/docs/http/ngx_http_core_module.html#open_file_cache)\ [open_file_cache_errors](https://nginx.org/en/docs/http/ngx_http_core_module.html#open_file_cache_errors)\ [open_file_cache_min_uses](https://nginx.org/en/docs/http/ngx_http_core_module.html#open_file_cache_min_uses)\ [open_file_cache_valid](https://nginx.org/en/docs/http/ngx_http_core_module.html#open_file_cache_valid)\ [open_log_file_cache (ngx_http_log_module)](https://nginx.org/en/docs/http/ngx_http_log_module.html#open_log_file_cache)\ [open_log_file_cache (ngx_stream_log_module)](https://nginx.org/en/docs/stream/ngx_stream_log_module.html#open_log_file_cache)\ [otel_exporter](https://nginx.org/en/docs/ngx_otel_module.html#otel_exporter)\ [otel_service_name](https://nginx.org/en/docs/ngx_otel_module.html#otel_service_name)\ [otel_trace](https://nginx.org/en/docs/ngx_otel_module.html#otel_trace)\ [otel_trace_context](https://nginx.org/en/docs/ngx_otel_module.html#otel_trace_context)\ [otel_span_name](https://nginx.org/en/docs/ngx_otel_module.html#otel_span_name)\ [otel_span_attr](https://nginx.org/en/docs/ngx_otel_module.html#otel_span_attr)\ [output_buffers](https://nginx.org/en/docs/http/ngx_http_core_module.html#output_buffers)\ [override_charset](https://nginx.org/en/docs/http/ngx_http_charset_module.html#override_charset)\ [pass](https://nginx.org/en/docs/stream/ngx_stream_pass_module.html#pass)\ [pid](https://nginx.org/en/docs/ngx_core_module.html#pid)\ [pop3_auth](https://nginx.org/en/docs/mail/ngx_mail_pop3_module.html#pop3_auth)\ [pop3_capabilities](https://nginx.org/en/docs/mail/ngx_mail_pop3_module.html#pop3_capabilities)\ [port_in_redirect](https://nginx.org/en/docs/http/ngx_http_core_module.html#port_in_redirect)\ [postpone_output](https://nginx.org/en/docs/http/ngx_http_core_module.html#postpone_output)\ [preread_buffer_size (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#preread_buffer_size)\ [preread_timeout (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#preread_timeout)\ [protocol](https://nginx.org/en/docs/mail/ngx_mail_core_module.html#protocol)\ [proxy_buffer](https://nginx.org/en/docs/mail/ngx_mail_proxy_module.html#proxy_buffer)\ [proxy_buffer_size (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffer_size)\ [proxy_buffer_size (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_buffer_size)\ [proxy_buffering](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffering)\ [proxy_buffers](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffers)\ [proxy_busy_buffers_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_busy_buffers_size)\ [proxy_cache](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache)\ [proxy_cache_background_update](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_background_update)\ [proxy_cache_bypass](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_bypass)\ [proxy_cache_convert_head](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_convert_head)\ [proxy_cache_key](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_key)\ [proxy_cache_lock](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_lock)\ [proxy_cache_lock_age](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_lock_age)\ [proxy_cache_lock_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_lock_timeout)\ [proxy_cache_max_range_offset](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_max_range_offset)\ [proxy_cache_methods](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_methods)\ [proxy_cache_min_uses](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_min_uses)\ [proxy_cache_path](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_path)\ [proxy_cache_purge](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_purge)\ [proxy_cache_revalidate](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_revalidate)\ [proxy_cache_use_stale](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_use_stale)\ [proxy_cache_valid](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_valid)\ [proxy_connect_timeout (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_connect_timeout)\ [proxy_connect_timeout (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_connect_timeout)\ [proxy_cookie_domain](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cookie_domain)\ [proxy_cookie_flags](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cookie_flags)\ [proxy_cookie_path](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cookie_path)\ [proxy_download_rate (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_download_rate)\ [proxy_force_ranges](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_force_ranges)\ [proxy_half_close (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_half_close)\ [proxy_headers_hash_bucket_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_headers_hash_bucket_size)\ [proxy_headers_hash_max_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_headers_hash_max_size)\ [proxy_hide_header](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_hide_header)\ [proxy_http_version](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_http_version)\ [proxy_ignore_client_abort](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ignore_client_abort)\ [proxy_ignore_headers](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ignore_headers)\ [proxy_intercept_errors](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_intercept_errors)\ [proxy_limit_rate](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_limit_rate)\ [proxy_max_temp_file_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_max_temp_file_size)\ [proxy_method](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_method)\ [proxy_next_upstream (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream)\ [proxy_next_upstream (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_next_upstream)\ [proxy_next_upstream_timeout (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream_timeout)\ [proxy_next_upstream_timeout (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_next_upstream_timeout)\ [proxy_next_upstream_tries (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream_tries)\ [proxy_next_upstream_tries (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_next_upstream_tries)\ [proxy_no_cache](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_no_cache)\ [proxy_pass (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass)\ [proxy_pass (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_pass)\ [proxy_pass_error_message](https://nginx.org/en/docs/mail/ngx_mail_proxy_module.html#proxy_pass_error_message)\ [proxy_pass_header](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass_header)\ [proxy_pass_request_body](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass_request_body)\ [proxy_pass_request_headers](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass_request_headers)\ [proxy_protocol (ngx_mail_proxy_module)](https://nginx.org/en/docs/mail/ngx_mail_proxy_module.html#proxy_protocol)\ [proxy_protocol (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_protocol)\ [proxy_protocol_timeout (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#proxy_protocol_timeout)\ [proxy_read_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_read_timeout)\ [proxy_redirect](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_redirect)\ [proxy_requests (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_requests)\ [proxy_request_buffering](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_request_buffering)\ [proxy_responses (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_responses)\ [proxy_send_lowat](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_send_lowat)\ [proxy_send_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_send_timeout)\ [proxy_session_drop (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_session_drop)\ [proxy_set_body](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_body)\ [proxy_set_header](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_header)\ [proxy_smtp_auth](https://nginx.org/en/docs/mail/ngx_mail_proxy_module.html#proxy_smtp_auth)\ [proxy_socket_keepalive (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_socket_keepalive)\ [proxy_socket_keepalive (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_socket_keepalive)\ [proxy_ssl (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl)\ [proxy_ssl_certificate (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_certificate)\ [proxy_ssl_certificate (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_certificate)\ [proxy_ssl_certificate_cache (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_certificate_cache)\ [proxy_ssl_certificate_cache (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_certificate_cache)\ [proxy_ssl_certificate_key (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_certificate_key)\ [proxy_ssl_certificate_key (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_certificate_key)\ [proxy_ssl_ciphers (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_ciphers)\ [proxy_ssl_ciphers (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_ciphers)\ [proxy_ssl_conf_command (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_conf_command)\ [proxy_ssl_conf_command (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_conf_command)\ [proxy_ssl_crl (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_crl)\ [proxy_ssl_crl (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_crl)\ [proxy_ssl_name (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_name)\ [proxy_ssl_name (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_name)\ [proxy_ssl_password_file (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_password_file)\ [proxy_ssl_password_file (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_password_file)\ [proxy_ssl_protocols (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_protocols)\ [proxy_ssl_protocols (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_protocols)\ [proxy_ssl_server_name (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_server_name)\ [proxy_ssl_server_name (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_server_name)\ [proxy_ssl_session_reuse (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_session_reuse)\ [proxy_ssl_session_reuse (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_session_reuse)\ [proxy_ssl_trusted_certificate (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_trusted_certificate)\ [proxy_ssl_trusted_certificate (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_trusted_certificate)\ [proxy_ssl_verify (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_verify)\ [proxy_ssl_verify (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_verify)\ [proxy_ssl_verify_depth (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_verify_depth)\ [proxy_ssl_verify_depth (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_verify_depth)\ [proxy_store](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_store)\ [proxy_store_access](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_store_access)\ [proxy_temp_file_write_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_temp_file_write_size)\ [proxy_temp_path](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_temp_path)\ [proxy_timeout (ngx_mail_proxy_module)](https://nginx.org/en/docs/mail/ngx_mail_proxy_module.html#proxy_timeout)\ [proxy_timeout (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_timeout)\ [proxy_upload_rate (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_upload_rate)\ [queue](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#queue)\ [quic_active_connection_id_limit](http://nginx.org/en/docs/http/ngx_http_v3_module.html#quic_active_connection_id_limit)\ [quic_gso](http://nginx.org/en/docs/http/ngx_http_v3_module.html#quic_gso)\ [quic_host_key](http://nginx.org/en/docs/http/ngx_http_v3_module.html#quic_host_key)\ [quic_retry](http://nginx.org/en/docs/http/ngx_http_v3_module.html#quic_retry)\ [random (ngx_http_upstream_module)](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#random)\ [random (ngx_stream_upstream_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#random)\ [random_index](https://nginx.org/en/docs/http/ngx_http_random_index_module.html#random_index)\ [read_ahead](https://nginx.org/en/docs/http/ngx_http_core_module.html#read_ahead)\ [read_timeout](https://nginx.org/en/docs/ngx_mgmt_module.html#read_timeout)\ [real_ip_header](https://nginx.org/en/docs/http/ngx_http_realip_module.html#real_ip_header)\ [real_ip_recursive](https://nginx.org/en/docs/http/ngx_http_realip_module.html#real_ip_recursive)\ [recursive_error_pages](https://nginx.org/en/docs/http/ngx_http_core_module.html#recursive_error_pages)\ [redirect_uri](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#redirect_uri)\ [referer_hash_bucket_size](https://nginx.org/en/docs/http/ngx_http_referer_module.html#referer_hash_bucket_size)\ [referer_hash_max_size](https://nginx.org/en/docs/http/ngx_http_referer_module.html#referer_hash_max_size)\ [request_pool_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#request_pool_size)\ [reset_timedout_connection](https://nginx.org/en/docs/http/ngx_http_core_module.html#reset_timedout_connection)\ [resolver (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver)\ [resolver (ngx_mail_core_module)](https://nginx.org/en/docs/mail/ngx_mail_core_module.html#resolver)\ [resolver (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#resolver)\ [resolver (ngx_stream_upstream_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#resolver)\ [resolver (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#resolver)\ [resolver_timeout (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver_timeout)\ [resolver_timeout (ngx_mail_core_module)](https://nginx.org/en/docs/mail/ngx_mail_core_module.html#resolver_timeout)\ [resolver_timeout (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#resolver_timeout)\ [resolver_timeout (ngx_stream_upstream_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#resolver_timeout)\ [resolver_timeout (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#resolver_timeout)\ [return (ngx_http_rewrite_module)](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#return)\ [return (ngx_stream_return_module)](https://nginx.org/en/docs/stream/ngx_stream_return_module.html#return)\ [rewrite](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#rewrite)\ [rewrite_log](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#rewrite_log)\ [root](https://nginx.org/en/docs/http/ngx_http_core_module.html#root)\ [satisfy](https://nginx.org/en/docs/http/ngx_http_core_module.html#satisfy)\ [scgi_buffer_size](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_buffer_size)\ [scgi_buffering](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_buffering)\ [scgi_buffers](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_buffers)\ [scgi_busy_buffers_size](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_busy_buffers_size)\ [scgi_cache](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache)\ [scgi_cache_background_update](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_background_update)\ [scgi_cache_bypass](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_bypass)\ [scgi_cache_key](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_key)\ [scgi_cache_lock](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_lock)\ [scgi_cache_lock_age](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_lock_age)\ [scgi_cache_lock_timeout](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_lock_timeout)\ [scgi_cache_max_range_offset](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_max_range_offset)\ [scgi_cache_methods](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_methods)\ [scgi_cache_min_uses](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_min_uses)\ [scgi_cache_path](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_path)\ [scgi_cache_purge](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_purge)\ [scgi_cache_revalidate](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_revalidate)\ [scgi_cache_use_stale](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_use_stale)\ [scgi_cache_valid](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_valid)\ [scgi_connect_timeout](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_connect_timeout)\ [scgi_force_ranges](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_force_ranges)\ [scgi_hide_header](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_hide_header)\ [scgi_ignore_client_abort](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_ignore_client_abort)\ [scgi_ignore_headers](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_ignore_headers)\ [scgi_intercept_errors](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_intercept_errors)\ [scgi_limit_rate](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_limit_rate)\ [scgi_max_temp_file_size](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_max_temp_file_size)\ [scgi_next_upstream](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_next_upstream)\ [scgi_next_upstream_timeout](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_next_upstream_timeout)\ [scgi_next_upstream_tries](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_next_upstream_tries)\ [scgi_no_cache](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_no_cache)\ [scgi_param](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_param)\ [scgi_pass](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_pass)\ [scgi_pass_header](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_pass_header)\ [scgi_pass_request_body](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_pass_request_body)\ [scgi_pass_request_headers](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_pass_request_headers)\ [scgi_read_timeout](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_read_timeout)\ [scgi_request_buffering](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_request_buffering)\ [scgi_send_timeout](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_send_timeout)\ [scgi_socket_keepalive](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_socket_keepalive)\ [scgi_store](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_store)\ [scgi_store_access](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_store_access)\ [scgi_temp_file_write_size](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_temp_file_write_size)\ [scgi_temp_path](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_temp_path)\ [scope](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#scope)\ [secure_link](https://nginx.org/en/docs/http/ngx_http_secure_link_module.html#secure_link)\ [secure_link_md5](https://nginx.org/en/docs/http/ngx_http_secure_link_module.html#secure_link_md5)\ [secure_link_secret](https://nginx.org/en/docs/http/ngx_http_secure_link_module.html#secure_link_secret)\ [send_lowat](https://nginx.org/en/docs/http/ngx_http_core_module.html#send_lowat)\ [send_timeout (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#send_timeout)\ [send_timeout (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#send_timeout)\ [sendfile](https://nginx.org/en/docs/http/ngx_http_core_module.html#sendfile)\ [sendfile_max_chunk](https://nginx.org/en/docs/http/ngx_http_core_module.html#sendfile_max_chunk)\ [server (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#server)\ [server (ngx_mail_core_module)](https://nginx.org/en/docs/mail/ngx_mail_core_module.html#server)\ [server (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#server)\ [server (ngx_stream_upstream_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#server)\ [server_name (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#server_name)\ [server_name (ngx_mail_core_module)](https://nginx.org/en/docs/mail/ngx_mail_core_module.html#server_name)\ [server_name (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#server_name)\ [server_name_in_redirect](https://nginx.org/en/docs/http/ngx_http_core_module.html#server_name_in_redirect)\ [server_tokens](https://nginx.org/en/docs/http/ngx_http_core_module.html#server_tokens)\ [session_log](https://nginx.org/en/docs/http/ngx_http_session_log_module.html#session_log)\ [session_log_format](https://nginx.org/en/docs/http/ngx_http_session_log_module.html#session_log_format)\ [session_log_zone](https://nginx.org/en/docs/http/ngx_http_session_log_module.html#session_log_zone)\ [session_store](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#session_store)\ [session_timeout](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#session_timeout)\ [set (ngx_http_rewrite_module)](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#set)\ [set (ngx_stream_set_module)](https://nginx.org/en/docs/stream/ngx_stream_set_module.html#set)\ [set_real_ip_from (ngx_http_realip_module)](https://nginx.org/en/docs/http/ngx_http_realip_module.html#set_real_ip_from)\ [set_real_ip_from (ngx_mail_realip_module)](https://nginx.org/en/docs/mail/ngx_mail_realip_module.html#set_real_ip_from)\ [set_real_ip_from (ngx_stream_realip_module)](https://nginx.org/en/docs/stream/ngx_stream_realip_module.html#set_real_ip_from)\ [slice](https://nginx.org/en/docs/http/ngx_http_slice_module.html#slice)\ [smtp_auth](https://nginx.org/en/docs/mail/ngx_mail_smtp_module.html#smtp_auth)\ [smtp_capabilities](https://nginx.org/en/docs/mail/ngx_mail_smtp_module.html#smtp_capabilities)\ [smtp_client_buffer](https://nginx.org/en/docs/mail/ngx_mail_smtp_module.html#smtp_client_buffer)\ [smtp_greeting_delay](https://nginx.org/en/docs/mail/ngx_mail_smtp_module.html#smtp_greeting_delay)\ [source_charset](https://nginx.org/en/docs/http/ngx_http_charset_module.html#source_charset) [split_clients (ngx_http_split_clients_module)](https://nginx.org/en/docs/http/ngx_http_split_clients_module.html#split_clients)\ [split_clients (ngx_stream_split_clients_module)](https://nginx.org/en/docs/stream/ngx_stream_split_clients_module.html#split_clients)\ [ssi](https://nginx.org/en/docs/http/ngx_http_ssi_module.html#ssi)\ [ssi_last_modified](https://nginx.org/en/docs/http/ngx_http_ssi_module.html#ssi_last_modified)\ [ssi_min_file_chunk](https://nginx.org/en/docs/http/ngx_http_ssi_module.html#ssi_min_file_chunk)\ [ssi_silent_errors](https://nginx.org/en/docs/http/ngx_http_ssi_module.html#ssi_silent_errors)\ [ssi_types](https://nginx.org/en/docs/http/ngx_http_ssi_module.html#ssi_types)\ [ssi_value_length](https://nginx.org/en/docs/http/ngx_http_ssi_module.html#ssi_value_length)\ [ssl (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl)\ [ssl (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl)\ [ssl (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl)\ [ssl_buffer_size](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_buffer_size)\ [ssl_certificate (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_certificate)\ [ssl_certificate (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_certificate)\ [ssl_certificate (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_certificate)\ [ssl_certificate (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_certificate)\ [ssl_certificate_cache (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_certificate_cache)\ [ssl_certificate_cache (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_certificate_cache)\ [ssl_certificate_key (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_certificate_key)\ [ssl_certificate_key (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_certificate_key)\ [ssl_certificate_key (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_certificate_key)\ [ssl_certificate_key (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_certificate_key)\ [ssl_ciphers (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_ciphers)\ [ssl_ciphers (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_ciphers)\ [ssl_ciphers (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_ciphers)\ [ssl_client_certificate (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_client_certificate)\ [ssl_client_certificate (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_client_certificate)\ [ssl_client_certificate (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_client_certificate)\ [ssl_conf_command (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_conf_command)\ [ssl_conf_command (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_conf_command)\ [ssl_conf_command (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_conf_command)\ [ssl_crl (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_crl)\ [ssl_crl (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_crl)\ [ssl_crl (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_crl)\ [ssl_dhparam (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_dhparam)\ [ssl_dhparam (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_dhparam)\ [ssl_early_data](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_early_data)\ [ssl_ecdh_curve (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_ecdh_curve)\ [ssl_ecdh_curve (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_ecdh_curve)\ [ssl_ecdh_curve (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_ecdh_curve)\ [ssl_handshake_timeout](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_handshake_timeout)\ [ssl_name](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_name)\ [ssl_object_cache_inheritable](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_object_cache_inheritable)\ [ssl_ocsp](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_ocsp)\ [ssl_ocsp_cache](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_ocsp_cache)\ [ssl_ocsp_responder](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_ocsp_responder)\ [ssl_password_file (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_password_file)\ [ssl_password_file (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_password_file)\ [ssl_password_file (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_password_file)\ [ssl_prefer_server_ciphers (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_prefer_server_ciphers)\ [ssl_prefer_server_ciphers (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_prefer_server_ciphers)\ [ssl_prefer_server_ciphers (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_prefer_server_ciphers)\ [ssl_preread (ngx_stream_ssl_preread_module)](http://nginx.org/en/docs/stream/ngx_stream_ssl_preread_module.html#var_ssl_preread_protocol)\ [ssl_protocols (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_protocols)\ [ssl_protocols (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_protocols)\ [ssl_protocols (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_protocols)\ [ssl_protocols (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_protocols)\ [ssl_reject_handshake](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_reject_handshake)\ [ssl_server_name](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_server_name)\ [ssl_session_cache (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_session_cache)\ [ssl_session_cache (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_session_cache)\ [ssl_session_cache (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_session_cache)\ [ssl_session_ticket_key (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_session_ticket_key)\ [ssl_session_ticket_key (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_session_ticket_key)\ [ssl_session_ticket_key (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_session_ticket_key)\ [ssl_session_tickets (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_session_tickets)\ [ssl_session_tickets (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_session_tickets)\ [ssl_session_tickets (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_session_tickets)\ [ssl_session_timeout (ngx_http_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_session_timeout)\ [ssl_session_timeout (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_session_timeout)\ [ssl_session_timeout (ngx_stream_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_session_timeout)\ [ssl_stapling](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_stapling)\ [ssl_stapling_file](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_stapling_file)\ [ssl_stapling_responder](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_stapling_responder)\ [ssl_stapling_verify](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_stapling_verify)\ [ssl_trusted_certificate (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_trusted_certificate)\ [ssl_trusted_certificate (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_trusted_certificate)\ [ssl_trusted_certificate (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_trusted_certificate)\ [ssl_trusted_certificate (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_trusted_certificate)\ [ssl_verify](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_verify)\ [ssl_verify_client (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_verify_client)\ [ssl_verify_client (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_verify_client)\ [ssl_verify_client (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_verify_client)\ [ssl_verify_depth (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_verify_depth)\ [ssl_verify_depth (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_verify_depth)\ [ssl_verify_depth (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_verify_depth)\ [ssl_verify_depth (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_verify_depth)\ [starttls](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#starttls)\ [state (ngx_http_upstream_module)](http://nginx.org/en/docs/http/ngx_http_upstream_module.html#state)\ [status_zone (ngx_http_api_module)](https://nginx.org/en/docs/http/ngx_http_api_module.html#status_zone)\ [sticky](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#sticky)\ [sticky_cookie_insert](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#sticky_cookie_insert)\ [stream (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#stream)\ [stub_status](https://nginx.org/en/docs/http/ngx_http_stub_status_module.html#stub_status)\ [sub_filter](https://nginx.org/en/docs/http/ngx_http_sub_module.html#sub_filter)\ [sub_filter_last_modified](https://nginx.org/en/docs/http/ngx_http_sub_module.html#sub_filter_last_modified)\ [sub_filter_once](https://nginx.org/en/docs/http/ngx_http_sub_module.html#sub_filter_once)\ [sub_filter_types](https://nginx.org/en/docs/http/ngx_http_sub_module.html#sub_filter_types)\ [subrequest_output_buffer_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#subrequest_output_buffer_size)\ [tcp_nodelay (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#tcp_nodelay)\ [tcp_nodelay (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#tcp_nodelay)\ [tcp_nopush](https://nginx.org/en/docs/http/ngx_http_core_module.html#tcp_nopush)\ [thread_pool](https://nginx.org/en/docs/ngx_core_module.html#thread_pool)\ [timeout](https://nginx.org/en/docs/mail/ngx_mail_core_module.html#timeout)\ [timer_resolution](https://nginx.org/en/docs/ngx_core_module.html#timer_resolution)\ [try_files](https://nginx.org/en/docs/http/ngx_http_core_module.html#try_files)\ [types](https://nginx.org/en/docs/http/ngx_http_core_module.html#types)\ [types_hash_bucket_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#types_hash_bucket_size)\ [types_hash_max_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#types_hash_max_size)\ [underscores_in_headers](https://nginx.org/en/docs/http/ngx_http_core_module.html#underscores_in_headers)\ [uninitialized_variable_warn](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#uninitialized_variable_warn)\ [upstream (ngx_http_upstream_module)](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#upstream)\ [upstream (ngx_stream_upstream_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#upstream)\ [upstream_conf](https://nginx.org/en/docs/http/ngx_http_upstream_conf_module.html#upstream_conf)\ [usage_report](https://nginx.org/en/docs/ngx_mgmt_module.html#usage_report)\ [use](https://nginx.org/en/docs/ngx_core_module.html#use)\ [user](https://nginx.org/en/docs/ngx_core_module.html#user)\ [userid](https://nginx.org/en/docs/http/ngx_http_userid_module.html#userid)\ [userid_domain](https://nginx.org/en/docs/http/ngx_http_userid_module.html#userid_domain)\ [userid_expires](https://nginx.org/en/docs/http/ngx_http_userid_module.html#userid_expires)\ [userid_flags](https://nginx.org/en/docs/http/ngx_http_userid_module.html#userid_flags)\ [userid_mark](https://nginx.org/en/docs/http/ngx_http_userid_module.html#userid_mark)\ [userid_name](https://nginx.org/en/docs/http/ngx_http_userid_module.html#userid_name)\ [userid_p3p](https://nginx.org/en/docs/http/ngx_http_userid_module.html#userid_p3p)\ [userid_path](https://nginx.org/en/docs/http/ngx_http_userid_module.html#userid_path)\ [userid_service](https://nginx.org/en/docs/http/ngx_http_userid_module.html#userid_service)\ [uuid_file](https://nginx.org/en/docs/ngx_mgmt_module.html#uuid_file)\ [uwsgi_buffer_size](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_buffer_size)\ [uwsgi_buffering](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_buffering)\ [uwsgi_buffers](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_buffers)\ [uwsgi_busy_buffers_size](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_busy_buffers_size)\ [uwsgi_cache](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache)\ [uwsgi_cache_background_update](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_background_update)\ [uwsgi_cache_bypass](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_bypass)\ [uwsgi_cache_key](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_key)\ [uwsgi_cache_lock](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_lock)\ [uwsgi_cache_lock_age](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_lock_age)\ [uwsgi_cache_lock_timeout](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_lock_timeout)\ [uwsgi_cache_max_range_offset](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_max_range_offset)\ [uwsgi_cache_methods](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_methods)\ [uwsgi_cache_min_uses](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_min_uses)\ [uwsgi_cache_path](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_path)\ [uwsgi_cache_purge](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_purge)\ [uwsgi_cache_revalidate](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_revalidate)\ [uwsgi_cache_use_stale](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_use_stale)\ [uwsgi_cache_valid](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_valid)\ [uwsgi_connect_timeout](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_connect_timeout)\ [uwsgi_force_ranges](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_force_ranges)\ [uwsgi_hide_header](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_hide_header)\ [uwsgi_ignore_client_abort](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ignore_client_abort)\ [uwsgi_ignore_headers](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ignore_headers)\ [uwsgi_intercept_errors](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_intercept_errors)\ [uwsgi_limit_rate](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_limit_rate)\ [uwsgi_max_temp_file_size](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_max_temp_file_size)\ [uwsgi_modifier1](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_modifier1)\ [uwsgi_modifier2](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_modifier2)\ [uwsgi_next_upstream](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_next_upstream)\ [uwsgi_next_upstream_timeout](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_next_upstream_timeout)\ [uwsgi_next_upstream_tries](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_next_upstream_tries)\ [uwsgi_no_cache](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_no_cache)\ [uwsgi_param](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_param)\ [uwsgi_pass](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_pass)\ [uwsgi_pass_header](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_pass_header)\ [uwsgi_pass_request_body](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_pass_request_body)\ [uwsgi_pass_request_headers](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_pass_request_headers)\ [uwsgi_read_timeout](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_read_timeout)\ [uwsgi_request_buffering](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_request_buffering)\ [uwsgi_send_timeout](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_send_timeout)\ [uwsgi_socket_keepalive](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_socket_keepalive)\ [uwsgi_ssl_certificate](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_certificate)\ [uwsgi_ssl_certificate_cache](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_certificate_cache)\ [uwsgi_ssl_certificate_key](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_certificate_key)\ [uwsgi_ssl_conf_command](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_conf_command)\ [uwsgi_ssl_crl](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_crl)\ [uwsgi_ssl_name](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_name)\ [uwsgi_ssl_password_file](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_password_file)\ [uwsgi_ssl_protocols](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_protocols)\ [uwsgi_ssl_server_name](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_server_name)\ [uwsgi_ssl_session_reuse](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_session_reuse)\ [uwsgi_ssl_trusted_certificate](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_trusted_certificate)\ [uwsgi_ssl_verify](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_verify)\ [uwsgi_ssl_verify_depth](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_verify_depth)\ [uwsgi_store](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_store)\ [uwsgi_store_access](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_store_access)\ [uwsgi_temp_file_write_size](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_temp_file_write_size)\ [uwsgi_temp_path](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_temp_path)\ [valid_referers](https://nginx.org/en/docs/http/ngx_http_referer_module.html#valid_referers)\ [variables_hash_bucket_size (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#variables_hash_bucket_size)\ [variables_hash_bucket_size (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#variables_hash_bucket_size)\ [variables_hash_max_size (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#variables_hash_max_size)\ [variables_hash_max_size (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#variables_hash_max_size)\ [worker_aio_requests](https://nginx.org/en/docs/ngx_core_module.html#worker_aio_requests)\ [worker_connections](https://nginx.org/en/docs/ngx_core_module.html#worker_connections)\ [worker_cpu_affinity](https://nginx.org/en/docs/ngx_core_module.html#worker_cpu_affinity)\ [worker_priority](https://nginx.org/en/docs/ngx_core_module.html#worker_priority)\ [worker_processes](https://nginx.org/en/docs/ngx_core_module.html#worker_processes)\ [worker_rlimit_core](https://nginx.org/en/docs/ngx_core_module.html#worker_rlimit_core)\ [worker_rlimit_nofile](https://nginx.org/en/docs/ngx_core_module.html#worker_rlimit_nofile)\ [worker_shutdown_timeout](https://nginx.org/en/docs/ngx_core_module.html#worker_shutdown_timeout)\ [working_directory](https://nginx.org/en/docs/ngx_core_module.html#working_directory)\ [xclient](https://nginx.org/en/docs/mail/ngx_mail_proxy_module.html#xclient)\ [xml_entities](https://nginx.org/en/docs/http/ngx_http_xslt_module.html#xml_entities)\ [xslt_last_modified](https://nginx.org/en/docs/http/ngx_http_xslt_module.html#xslt_last_modified)\ [xslt_param](https://nginx.org/en/docs/http/ngx_http_xslt_module.html#xslt_param)\ [xslt_string_param](http://nginx.org/en/docs/http/ngx_http_xslt_module.html#xslt_string_param)\ [xslt_stylesheet](https://nginx.org/en/docs/http/ngx_http_xslt_module.html#xslt_stylesheet)\ [xslt_types](https://nginx.org/en/docs/http/ngx_http_xslt_module.html#xslt_types)\ [zone (ngx_http_upstream_module)](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#zone)\ [zone (ngx_stream_upstream_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#zone)\ -------------------------------------------------------------------------------- title: "NGINX configuration rules and limits" url: https://docs.nginx.com/nginxaas/overview/nginx-configuration/configuration-rules/index.md -------------------------------------------------------------------------------- This document provides details about using NGINX configuration files with your F5 ${product} deployment, restrictions, and available directives. ## NGINX configuration common user workflows You can upload NGINX configurations to your ${product} deployment using the NGINXaaS console: - [Upload using the console](/nginxaas/overview/nginx-configuration/nginx-configuration-console.md) The topics below explain NGINX configuration restrictions and which directives are supported, unsupported, or cannot be overridden. ## NGINX filesystem restrictions There are limits to where files, including NGINX configuration files, certificate files, and any other files uploaded to the deployment, can be placed on the filesystem. There are also limits on what directories NGINX can access during runtime. These limits help support the separation of roles, enforce the principle of least privilege, and ensure the smooth operation of the system. | Allowed Directory | User can upload files to | NGINX master process can read | NGINX master process can write | NGINX worker process can read | NGINX worker process can write | | -------------------- | -------------------- | -------------------- | -------------------- | -------------------- | -------------------- | | /etc/nginx | [icon: check] | [icon: check] | | | | | /opt | [icon: check] | [icon: check] | [icon: check] | [icon: check] | [icon: check] | | /srv | [icon: check] | [icon: check] | | [icon: check] | | | /tmp | | [icon: check] | [icon: check] | [icon: check] | [icon: check] | | /spool/nginx | | [icon: check] | [icon: check] | [icon: check] | [icon: check] | | /var/cache/nginx | | [icon: check] | [icon: check] | [icon: check] | [icon: check] | | /var/spool/nginx | | [icon: check] | [icon: check] | [icon: check] | [icon: check] | | /var/www | [icon: check] | [icon: check] | | [icon: check] | | For example, `/etc/nginx` is only readable by the NGINX master process, making it a secure location for certificate files that won't be accidentally served due to configuration errors. `/var/www` is a secure location for static content because the NGINX worker process can serve files from it but cannot modify them, ensuring content integrity. `/tmp` is a good choice for storing temporary files with `proxy_temp_path` or `client_body_temp_path` since it is writable by the NGINX worker process. If you need access to additional directories, please [contact us](/nginxaas/support.md). ## Disallowed configuration directives The following directives are not supported because of specific limitations. If you include any of these directives in your NGINX configuration, you'll get an error. | Disallowed Directive | Reason | |------------------ | ----------------- | | ssl_engine | No hardware SSL accelerator is available. | | debug_points | NGINXaaS does not provide access to NGINX processes for debugging. | | fastcgi_bind
grpc_bind
memcached_bind
proxy_bind
scgi_bind
uwsgi_bind | Source IP specification for active-active deployments is not allowed. | | quic_bpf | QUIC connection migration is not currently supported for active-active deployments. | You may find a few directives are not listed here as either allowed or disallowed. Our team is working on getting these directives supported soon. ## Directives that cannot be overridden The following directives cannot be overridden by the user provided configuration. | Persistent Directive | Value | Reason | |------------------ | ----------------------- | -----------------| | `user` | `nginx` | The `nginx` user has the correct permissions for accessing certificates, policy files and other auxfiles. | | `worker_processes` | `auto` | Set to `auto` to automatically set `worker_processes` to the number of CPU cores. | | `worker_rlimit_nofile` | `524288` | Set for optimal performance of deployments. | | `worker_connections` | `150000` | Set for optimal performance of deployments. | | `pid` | `/run/nginx/nginx.pid` | Set to this value to allow NGINXaaS to automatically manage the NGINX master process. | | `daemon` | `on` | Automatically set to `on` to allow NGINXaaS to manage the NGINX master process. | | `master_process` | `on` | This directive is intended for NGINX developers. | | `worker_cpu_affinity` | `auto` | The value `auto` allows binding worker processes automatically to available CPUs based on the current capacity of the deployment. | For connection and request rate limiting, consider using these NGINX modules: - [limit_conn](https://nginx.org/en/docs/http/ngx_http_limit_conn_module.html#limit_conn) - [limit_req](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req) - [upstream queue](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#queue) ## Listener restrictions NGINXaaS reserves the following listen ports for internal use. - `49151` - `49152` NGINXaaS does not support specific hostnames or IP addresses (other than localhost / loopback addresses) to be used as a `listen` address. You must use an unspecified address to listen on all IPs (such as `0.0.0.0`, `[::0]` or `*`) or a local address for use-cases where you do not want the listener exposed via the service endpoint (such as `127.0.0.1` or `localhost`). ## Configuration directives list NGINXaaS supports a limited set of NGINX directives. The directives with the "app_protect" prefix require [F5 WAF for NGINX](/waf) to be enabled for the deployment. NGINXaaS does not yet support F5 WAF for NGINX custom security policies or logging profiles. Support is limited to the [prebuilt policies](/waf/policies/configuration.md#default-policy) and the [default logging profiles](/waf/logging/logs-overview.md#default-logging-profile-bundles). #### Alphabetical index of directives [absolute_redirect](https://nginx.org/en/docs/http/ngx_http_core_module.html#absolute_redirect)\ [accept_mutex](https://nginx.org/en/docs/ngx_core_module.html#accept_mutex)\ [accept_mutex_delay](https://nginx.org/en/docs/ngx_core_module.html#accept_mutex_delay)\ [access_log (ngx_http_log_module)](https://nginx.org/en/docs/http/ngx_http_log_module.html#access_log)\ [access_log (ngx_stream_log_module)](https://nginx.org/en/docs/stream/ngx_stream_log_module.html#access_log)\ [add_after_body](https://nginx.org/en/docs/http/ngx_http_addition_module.html#add_after_body)\ [add_before_body](https://nginx.org/en/docs/http/ngx_http_addition_module.html#add_before_body)\ [add_header](https://nginx.org/en/docs/http/ngx_http_headers_module.html#add_header)\ [add_trailer](https://nginx.org/en/docs/http/ngx_http_headers_module.html#add_trailer)\ [addition_types](https://nginx.org/en/docs/http/ngx_http_addition_module.html#addition_types)\ [aio](https://nginx.org/en/docs/http/ngx_http_core_module.html#aio)\ [aio_write](https://nginx.org/en/docs/http/ngx_http_core_module.html#aio_write)\ [alias](https://nginx.org/en/docs/http/ngx_http_core_module.html#alias)\ [allow (ngx_http_access_module)](https://nginx.org/en/docs/http/ngx_http_access_module.html#allow)\ [allow (ngx_stream_access_module)](https://nginx.org/en/docs/stream/ngx_stream_access_module.html#allow)\ [ancient_browser](https://nginx.org/en/docs/http/ngx_http_browser_module.html#ancient_browser)\ [ancient_browser_value](https://nginx.org/en/docs/http/ngx_http_browser_module.html#ancient_browser_value)\ [app_protect_compressed_requests_action](/waf/policies/directives/#global-directives:~:text=app_protect_compressed_requests_action)\ [app_protect_cookie_seed](/waf/policies/directives/#global-directives:~:text=app_protect_cookie_seed)\ [app_protect_cpu_thresholds](/waf/policies/directives/#global-directives:~:text=app_protect_cpu_thresholds)\ [app_protect_custom_log_attribute](/waf/policies/directives/#f5-waf-for-nginx-directives:~:text=app_protect_custom_log_attribute)\ [app_protect_enable](/waf/policies/directives/#f5-waf-for-nginx-directives:~:text=modules/ngx_http_app_protect_module.so-,app_protect_enable)\ [app_protect_failure_mode_action](/waf/policies/directives/#global-directives:~:text=app_protect_failure_mode_action)\ [app_protect_physical_memory_util_thresholds](/waf/policies/directives/#global-directives:~:text=app_protect_physical_memory_util_thresholds)\ [app_protect_policy_file]()\ [app_protect_reconnect_period_seconds](/waf/policies/directives/#global-directives:~:text=app_protect_reconnect_period_seconds)\ [app_protect_request_buffer_overflow_action](/waf/policies/directives/#global-directives:~:text=app_protect_request_buffer_overflow_action)\ [app_protect_security_log](/waf/policies/directives/#f5-waf-for-nginx-directives:~:text=app_protect_security_log)\ [app_protect_security_log_enable](/waf/policies/directives/#f5-waf-for-nginx-directives:~:text=app_protect_security_log_enable)\ [app_protect_user_defined_signatures](/waf/policies/directives/#global-directives:~:text=app_protect_user_defined_signatures)\ [auth_basic](https://nginx.org/en/docs/http/ngx_http_auth_basic_module.html#auth_basic)\ [auth_basic_user_file](https://nginx.org/en/docs/http/ngx_http_auth_basic_module.html#auth_basic_user_file)\ [auth_delay](https://nginx.org/en/docs/http/ngx_http_core_module.html#auth_delay)\ [auth_http](https://nginx.org/en/docs/mail/ngx_mail_auth_http_module.html#auth_http)\ [auth_http_header](https://nginx.org/en/docs/mail/ngx_mail_auth_http_module.html#auth_http_header)\ [auth_http_pass_client_cert](https://nginx.org/en/docs/mail/ngx_mail_auth_http_module.html#auth_http_pass_client_cert)\ [auth_http_timeout](https://nginx.org/en/docs/mail/ngx_mail_auth_http_module.html#auth_http_timeout)\ [auth_jwt](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt)\ [auth_jwt_claim_set](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_claim_set)\ [auth_jwt_header_set](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_header_set)\ [auth_jwt_key_cache](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_cache)\ [auth_jwt_key_file](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_file)\ [auth_jwt_key_request](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_key_request)\ [auth_jwt_leeway](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_leeway)\ [auth_jwt_require](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_require)\ [auth_jwt_type](https://nginx.org/en/docs/http/ngx_http_auth_jwt_module.html#auth_jwt_type)\ [auth_oidc](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#auth_oidc)\ [auth_request](https://nginx.org/en/docs/http/ngx_http_auth_request_module.html#auth_request)\ [auth_request_set](https://nginx.org/en/docs/http/ngx_http_auth_request_module.html#auth_request_set)\ [autoindex](https://nginx.org/en/docs/http/ngx_http_autoindex_module.html#autoindex)\ [autoindex_exact_size](https://nginx.org/en/docs/http/ngx_http_autoindex_module.html#autoindex_exact_size)\ [autoindex_format](https://nginx.org/en/docs/http/ngx_http_autoindex_module.html#autoindex_format)\ [autoindex_localtime](https://nginx.org/en/docs/http/ngx_http_autoindex_module.html#autoindex_localtime)\ [break](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#break)\ [connect_timeout](https://nginx.org/en/docs/ngx_mgmt_module.html#connect_timeout)\ [charset](https://nginx.org/en/docs/http/ngx_http_charset_module.html#charset)\ [charset_map](https://nginx.org/en/docs/http/ngx_http_charset_module.html#charset_map)\ [charset_types](https://nginx.org/en/docs/http/ngx_http_charset_module.html#charset_types)\ [chunked_transfer_encoding](https://nginx.org/en/docs/http/ngx_http_core_module.html#chunked_transfer_encoding)\ [client_body_buffer_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_buffer_size)\ [client_body_in_file_only](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_in_file_only)\ [client_body_in_single_buffer](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_in_single_buffer)\ [client_body_temp_path](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_temp_path)\ [client_body_timeout](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_timeout)\ [client_header_buffer_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_header_buffer_size)\ [client_header_timeout](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_header_timeout)\ [client_id](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_id)\ [client_max_body_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size)\ [client_secret](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#client_secret)\ [config_url](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#config_url)\ [connection_pool_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#connection_pool_size)\ [cookie_name](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#cookie_name)\ [create_full_put_path](https://nginx.org/en/docs/http/ngx_http_dav_module.html#create_full_put_path)\ [daemon](https://nginx.org/en/docs/ngx_core_module.html#daemon)\ [dav_access](https://nginx.org/en/docs/http/ngx_http_dav_module.html#dav_access)\ [dav_methods](https://nginx.org/en/docs/http/ngx_http_dav_module.html#dav_methods)\ [debug_connection](https://nginx.org/en/docs/ngx_core_module.html#debug_connection)\ [default_type](https://nginx.org/en/docs/http/ngx_http_core_module.html#default_type)\ [deny (ngx_http_access_module)](https://nginx.org/en/docs/http/ngx_http_access_module.html#deny)\ [deny (ngx_stream_access_module)](https://nginx.org/en/docs/stream/ngx_stream_access_module.html#deny)\ [directio](https://nginx.org/en/docs/http/ngx_http_core_module.html#directio)\ [directio_alignment](https://nginx.org/en/docs/http/ngx_http_core_module.html#directio_alignment)\ [disable_symlinks](https://nginx.org/en/docs/http/ngx_http_core_module.html#disable_symlinks)\ [empty_gif](https://nginx.org/en/docs/http/ngx_http_empty_gif_module.html#empty_gif)\ [enforce_initial_report](https://nginx.org/en/docs/ngx_mgmt_module.html#enforce_initial_report)\ [env](https://nginx.org/en/docs/ngx_core_module.html#env)\ [error_log](https://nginx.org/en/docs/ngx_core_module.html#error_log)\ [error_page](https://nginx.org/en/docs/http/ngx_http_core_module.html#error_page)\ [etag](https://nginx.org/en/docs/http/ngx_http_core_module.html#etag)\ [events](https://nginx.org/en/docs/ngx_core_module.html#events)\ [expires](https://nginx.org/en/docs/http/ngx_http_headers_module.html#expires)\ [extra_auth_args](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#extra_auth_args)\ [f4f](https://nginx.org/en/docs/http/ngx_http_f4f_module.html#f4f)\ [f4f_buffer_size](https://nginx.org/en/docs/http/ngx_http_f4f_module.html#f4f_buffer_size)\ [fastcgi_buffer_size](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_buffer_size)\ [fastcgi_buffering](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_buffering)\ [fastcgi_buffers](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_buffers)\ [fastcgi_busy_buffers_size](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_busy_buffers_size)\ [fastcgi_cache](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache)\ [fastcgi_cache_background_update](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_background_update)\ [fastcgi_cache_bypass](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_bypass)\ [fastcgi_cache_key](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_key)\ [fastcgi_cache_lock](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_lock)\ [fastcgi_cache_lock_age](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_lock_age)\ [fastcgi_cache_lock_timeout](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_lock_timeout)\ [fastcgi_cache_max_range_offset](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_max_range_offset)\ [fastcgi_cache_methods](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_methods)\ [fastcgi_cache_min_uses](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_min_uses)\ [fastcgi_cache_path](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_path)\ [fastcgi_cache_revalidate](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_revalidate)\ [fastcgi_cache_use_stale](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_use_stale)\ [fastcgi_cache_valid](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_cache_valid)\ [fastcgi_catch_stderr](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_catch_stderr)\ [fastcgi_connect_timeout](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_connect_timeout)\ [fastcgi_force_ranges](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_force_ranges)\ [fastcgi_hide_header](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_hide_header)\ [fastcgi_ignore_client_abort](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_ignore_client_abort)\ [fastcgi_ignore_headers](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_ignore_headers)\ [fastcgi_index](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_index)\ [fastcgi_intercept_errors](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_intercept_errors)\ [fastcgi_keep_conn](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_keep_conn)\ [fastcgi_limit_rate](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_limit_rate)\ [fastcgi_max_temp_file_size](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_max_temp_file_size)\ [fastcgi_next_upstream](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_next_upstream)\ [fastcgi_next_upstream_timeout](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_next_upstream_timeout)\ [fastcgi_next_upstream_tries](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_next_upstream_tries)\ [fastcgi_no_cache](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_no_cache)\ [fastcgi_param](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_param)\ [fastcgi_pass](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_pass)\ [fastcgi_pass_header](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_pass_header)\ [fastcgi_pass_request_body](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_pass_request_body)\ [fastcgi_pass_request_headers](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_pass_request_headers)\ [fastcgi_read_timeout](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_read_timeout)\ [fastcgi_request_buffering](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_request_buffering)\ [fastcgi_send_lowat](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_send_lowat)\ [fastcgi_send_timeout](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_send_timeout)\ [fastcgi_socket_keepalive](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_socket_keepalive)\ [fastcgi_split_path_info](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_split_path_info)\ [fastcgi_store](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_store)\ [fastcgi_store_access](http://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_store_access)\ [fastcgi_temp_file_write_size](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_temp_file_write_size)\ [fastcgi_temp_path](https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_temp_path)\ [flv](https://nginx.org/en/docs/http/ngx_http_flv_module.html#flv)\ [geo (ngx_http_geo_module)](https://nginx.org/en/docs/http/ngx_http_geo_module.html#geo)\ [geo (ngx_stream_geo_module)](https://nginx.org/en/docs/stream/ngx_stream_geo_module.html#geo)\ [grpc_buffer_size](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_buffer_size)\ [grpc_connect_timeout](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_connect_timeout)\ [grpc_hide_header](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_hide_header)\ [grpc_ignore_headers](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ignore_headers)\ [grpc_intercept_errors](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_intercept_errors)\ [grpc_next_upstream](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_next_upstream)\ [grpc_next_upstream_timeout](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_next_upstream_timeout)\ [grpc_next_upstream_tries](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_next_upstream_tries)\ [grpc_pass](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_pass)\ [grpc_pass_header](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_pass_header)\ [grpc_read_timeout](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_read_timeout)\ [grpc_send_timeout](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_send_timeout)\ [grpc_set_header](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_set_header)\ [grpc_socket_keepalive](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_socket_keepalive)\ [grpc_ssl_certificate](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_certificate)\ [grpc_ssl_certificate_cache](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_certificate_cache)\ [grpc_ssl_certificate_key](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_certificate_key)\ [grpc_ssl_ciphers](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_ciphers)\ [grpc_ssl_conf_command](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_conf_command)\ [grpc_ssl_crl](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_crl)\ [grpc_ssl_name](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_name)\ [grpc_ssl_password_file](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_password_file)\ [grpc_ssl_protocols](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_protocols)\ [grpc_ssl_server_name](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_server_name)\ [grpc_ssl_session_reuse](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_session_reuse)\ [grpc_ssl_trusted_certificate](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_trusted_certificate)\ [grpc_ssl_verify](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_verify)\ [grpc_ssl_verify_depth](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_ssl_verify_depth)\ [gunzip](https://nginx.org/en/docs/http/ngx_http_gunzip_module.html#gunzip)\ [gunzip_buffers](https://nginx.org/en/docs/http/ngx_http_gunzip_module.html#gunzip_buffers)\ [gzip](https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip)\ [gzip_buffers](https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_buffers)\ [gzip_comp_level](https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_comp_level)\ [gzip_disable](https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_disable)\ [gzip_http_version](https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_http_version)\ [gzip_min_length](https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_min_length)\ [gzip_proxied](https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_proxied)\ [gzip_static](https://nginx.org/en/docs/http/ngx_http_gzip_static_module.html#gzip_static)\ [gzip_types](https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_types)\ [gzip_vary](https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_vary)\ [hash (ngx_http_upstream_module)](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#hash)\ [hash (ngx_stream_upstream_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#hash)\ [health_check (ngx_http_upstream_hc_module)](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#health_check)\ [health_check (ngx_stream_upstream_hc_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#health_check)\ [health_check_timeout](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#health_check_timeout)\ [hls](https://nginx.org/en/docs/http/ngx_http_hls_module.html#hls)\ [hls_buffers](https://nginx.org/en/docs/http/ngx_http_hls_module.html#hls_buffers)\ [hls_forward_args](https://nginx.org/en/docs/http/ngx_http_hls_module.html#hls_forward_args)\ [hls_fragment](https://nginx.org/en/docs/http/ngx_http_hls_module.html#hls_fragment)\ [hls_mp4_buffer_size](https://nginx.org/en/docs/http/ngx_http_hls_module.html#hls_mp4_buffer_size)\ [hls_mp4_max_buffer_size](https://nginx.org/en/docs/http/ngx_http_hls_module.html#hls_mp4_max_buffer_size)\ [http](https://nginx.org/en/docs/http/ngx_http_core_module.html#http)\ [http2_body_preread_size](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_body_preread_size)\ [http2_chunk_size](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_chunk_size)\ [http2_idle_timeout](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_idle_timeout)\ [http2_max_concurrent_pushes](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_max_concurrent_pushes)\ [http2_max_concurrent_streams](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_max_concurrent_streams)\ [http2_max_field_size](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_max_field_size)\ [http2_max_header_size](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_max_header_size)\ [http2_max_requests](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_max_requests)\ [http2_push](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_push)\ [http2_push_preload](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_push_preload)\ [http2_recv_buffer_size](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_recv_buffer_size)\ [http2_recv_timeout](https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_recv_timeout)\ [http3](http://nginx.org/en/docs/http/ngx_http_v3_module.html#http3)\ [http3_hq](http://nginx.org/en/docs/http/ngx_http_v3_module.html#http3_hq)\ [http3_max_concurrent_streams](http://nginx.org/en/docs/http/ngx_http_v3_module.html#http3_max_concurrent_streams)\ [http3_stream_buffer_size](http://nginx.org/en/docs/http/ngx_http_v3_module.html#http3_stream_buffer_size)\ [if](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#if)\ [if_modified_since](https://nginx.org/en/docs/http/ngx_http_core_module.html#if_modified_since)\ [ignore_invalid_headers](https://nginx.org/en/docs/http/ngx_http_core_module.html#ignore_invalid_headers)\ [image_filter](http://nginx.org/en/docs/http/ngx_http_image_filter_module.html#image_filter)\ [image_filter_buffer](http://nginx.org/en/docs/http/ngx_http_image_filter_module.html#image_filter_buffer)\ [image_filter_interlace](http://nginx.org/en/docs/http/ngx_http_image_filter_module.html#image_filter_interlace)\ [image_filter_jpeg_quality](http://nginx.org/en/docs/http/ngx_http_image_filter_module.html#image_filter_jpeg_quality)\ [image_filter_sharpen](http://nginx.org/en/docs/http/ngx_http_image_filter_module.html#image_filter_sharpen)\ [image_filter_transparency](http://nginx.org/en/docs/http/ngx_http_image_filter_module.html#image_filter_transparency)\ [image_filter_webp_quality](http://nginx.org/en/docs/http/ngx_http_image_filter_module.html#image_filter_webp_quality)\ [imap_auth](https://nginx.org/en/docs/mail/ngx_mail_imap_module.html#imap_auth)\ [imap_capabilities](https://nginx.org/en/docs/mail/ngx_mail_imap_module.html#imap_capabilities)\ [imap_client_buffer](https://nginx.org/en/docs/mail/ngx_mail_imap_module.html#imap_client_buffer)\ [include](https://nginx.org/en/docs/ngx_core_module.html#include)\ [index](https://nginx.org/en/docs/http/ngx_http_index_module.html#index)\ [internal](https://nginx.org/en/docs/http/ngx_http_core_module.html#internal)\ [internal_redirect](http://nginx.org/en/docs/http/ngx_http_internal_redirect_module.html#internal_redirect)\ [ip_hash](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#ip_hash)\ [issuer](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#issuer)\ [js_access (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_access)\ [js_body_filter](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_body_filter)\ [js_content](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_content)\ [js_fetch_buffer_size (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_fetch_buffer_size)\ [js_fetch_buffer_size (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_fetch_buffer_size)\ [js_fetch_ciphers (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_fetch_ciphers)\ [js_fetch_ciphers (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_fetch_ciphers)\ [js_fetch_max_response_buffer_size (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_fetch_max_response_buffer_size)\ [js_fetch_max_response_buffer_size (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_fetch_max_response_buffer_size)\ [js_fetch_protocols (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_fetch_protocols)\ [js_fetch_protocols (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_fetch_protocols)\ [js_fetch_timeout (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_fetch_timeout)\ [js_fetch_timeout (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_fetch_timeout)\ [js_fetch_trusted_certificate (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_fetch_trusted_certificate)\ [js_fetch_trusted_certificate (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_fetch_trusted_certificate)\ [js_fetch_verify (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_fetch_verify)\ [js_fetch_verify (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_fetch_verify)\ [js_fetch_verify_depth (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_fetch_verify_depth)\ [js_fetch_verify_depth (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_fetch_verify_depth)\ [js_filter (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_filter)\ [js_header_filter](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_header_filter)\ [js_import (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_import)\ [js_import (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_import)\ [js_include (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_include)\ [js_include (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_include)\ [js_path (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_path)\ [js_path (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_path)\ [js_periodic (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_periodic)\ [js_periodic (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_periodic)\ [js_preload_object (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_preload_object)\ [js_preload_object (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_preload_object)\ [js_preread (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_preread)\ [js_set (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_set)\ [js_set (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_set)\ [js_var (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_var)\ [js_var (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_var)\ [js_shared_dict_zone (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_shared_dict_zone)\ [js_var (ngx_http_js_module)](https://nginx.org/en/docs/http/ngx_http_js_module.html#js_var)\ [js_var (ngx_stream_js_module)](https://nginx.org/en/docs/stream/ngx_stream_js_module.html#js_var)\ [keepalive](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive)\ [keepalive_disable](https://nginx.org/en/docs/http/ngx_http_core_module.html#keepalive_disable)\ [keepalive_min_timeout](https://nginx.org/en/docs/http/ngx_http_core_module.html#keepalive_min_timeout)\ [keepalive_requests (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#keepalive_requests)\ [keepalive_time (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#keepalive_time)\ [keepalive_timeout (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#keepalive_timeout)\ [keyval (ngx_http_keyval_module)](https://nginx.org/en/docs/http/ngx_http_keyval_module.html#keyval)\ [keyval (ngx_stream_keyval_module)](https://nginx.org/en/docs/stream/ngx_stream_keyval_module.html#keyval)\ [keyval_zone (ngx_http_keyval_module)](https://nginx.org/en/docs/http/ngx_http_keyval_module.html#keyval_zone)\ [keyval_zone (ngx_stream_keyval_module)](https://nginx.org/en/docs/stream/ngx_stream_keyval_module.html#keyval_zone)\ [large_client_header_buffers](https://nginx.org/en/docs/http/ngx_http_core_module.html#large_client_header_buffers)\ [least_conn (ngx_http_upstream_module)](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#least_conn)\ [least_conn (ngx_stream_upstream_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#least_conn)\ [least_time (ngx_http_upstream_module)](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#least_time)\ [least_time (ngx_stream_upstream_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#least_time)\ [license_token](https://nginx.org/en/docs/ngx_mgmt_module.html#license_token)\ [limit_conn (ngx_http_limit_conn_module)](https://nginx.org/en/docs/http/ngx_http_limit_conn_module.html#limit_conn)\ [limit_conn (ngx_stream_limit_conn_module)](https://nginx.org/en/docs/stream/ngx_stream_limit_conn_module.html#limit_conn)\ [limit_conn_dry_run (ngx_http_limit_conn_module)](https://nginx.org/en/docs/http/ngx_http_limit_conn_module.html#limit_conn_dry_run)\ [limit_conn_dry_run (ngx_stream_limit_conn_module)](https://nginx.org/en/docs/stream/ngx_stream_limit_conn_module.html#limit_conn_dry_run)\ [limit_conn_log_level (ngx_http_limit_conn_module)](https://nginx.org/en/docs/http/ngx_http_limit_conn_module.html#limit_conn_log_level)\ [limit_conn_log_level (ngx_stream_limit_conn_module)](https://nginx.org/en/docs/stream/ngx_stream_limit_conn_module.html#limit_conn_log_level)\ [limit_conn_status](https://nginx.org/en/docs/http/ngx_http_limit_conn_module.html#limit_conn_status)\ [limit_conn_zone (ngx_http_limit_conn_module)](https://nginx.org/en/docs/http/ngx_http_limit_conn_module.html#limit_conn_zone)\ [limit_conn_zone (ngx_stream_limit_conn_module)](https://nginx.org/en/docs/stream/ngx_stream_limit_conn_module.html#limit_conn_zone)\ [limit_except](https://nginx.org/en/docs/http/ngx_http_core_module.html#limit_except)\ [limit_rate](https://nginx.org/en/docs/http/ngx_http_core_module.html#limit_rate)\ [limit_rate_after](https://nginx.org/en/docs/http/ngx_http_core_module.html#limit_rate_after)\ [limit_req](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req)\ [limit_req_dry_run](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_dry_run)\ [limit_req_log_level](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_log_level)\ [limit_req_status](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_status)\ [limit_req_zone](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_zone)\ [limit_zone](https://nginx.org/en/docs/http/ngx_http_limit_conn_module.html#limit_zone)\ [lingering_close](https://nginx.org/en/docs/http/ngx_http_core_module.html#lingering_close)\ [lingering_time](https://nginx.org/en/docs/http/ngx_http_core_module.html#lingering_time)\ [lingering_timeout](https://nginx.org/en/docs/http/ngx_http_core_module.html#lingering_timeout)\ [listen (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#listen)\ [listen (ngx_mail_core_module)](https://nginx.org/en/docs/mail/ngx_mail_core_module.html#listen)\ [load_module](https://nginx.org/en/docs/ngx_core_module.html#load_module)\ [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location)\ [lock_file](http://nginx.org/en/docs/ngx_core_module.html#lock_file)\ [log_format (ngx_http_log_module)](https://nginx.org/en/docs/http/ngx_http_log_module.html#log_format)\ [log_format (ngx_stream_log_module)](https://nginx.org/en/docs/stream/ngx_stream_log_module.html#log_format)\ [log_not_found](https://nginx.org/en/docs/http/ngx_http_core_module.html#log_not_found)\ [log_subrequest](https://nginx.org/en/docs/http/ngx_http_core_module.html#log_subrequest)\ [mail](https://nginx.org/en/docs/mail/ngx_mail_core_module.html#mail)\ [map (ngx_http_map_module)](https://nginx.org/en/docs/http/ngx_http_map_module.html#map)\ [map (ngx_stream_map_module)](https://nginx.org/en/docs/stream/ngx_stream_map_module.html#map)\ [map_hash_bucket_size (ngx_http_map_module)](https://nginx.org/en/docs/http/ngx_http_map_module.html#map_hash_bucket_size)\ [map_hash_bucket_size (ngx_stream_map_module)](https://nginx.org/en/docs/stream/ngx_stream_map_module.html#map_hash_bucket_size)\ [map_hash_max_size (ngx_http_map_module)](https://nginx.org/en/docs/http/ngx_http_map_module.html#map_hash_max_size)\ [map_hash_max_size (ngx_stream_map_module)](https://nginx.org/en/docs/stream/ngx_stream_map_module.html#map_hash_max_size)\ [master_process](https://nginx.org/en/docs/ngx_core_module.html#master_process)\ [match (ngx_http_upstream_hc_module)](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#match)\ [match (ngx_stream_upstream_hc_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#match)\ [max_errors](https://nginx.org/en/docs/mail/ngx_mail_core_module.html#max_errors)\ [max_ranges](https://nginx.org/en/docs/http/ngx_http_core_module.html#max_ranges)\ [memcached_buffer_size](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_buffer_size)\ [memcached_connect_timeout](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_connect_timeout)\ [memcached_gzip_flag](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_gzip_flag)\ [memcached_next_upstream](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_next_upstream)\ [memcached_next_upstream_timeout](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_next_upstream_timeout)\ [memcached_next_upstream_tries](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_next_upstream_tries)\ [memcached_pass](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_pass)\ [memcached_read_timeout](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_read_timeout)\ [memcached_send_timeout](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_send_timeout)\ [memcached_socket_keepalive](https://nginx.org/en/docs/http/ngx_http_memcached_module.html#memcached_socket_keepalive)\ [merge_slashes](https://nginx.org/en/docs/http/ngx_http_core_module.html#merge_slashes)\ [mgmt](https://nginx.org/en/docs/ngx_mgmt_module.html#mgmt)\ [min_delete_depth](https://nginx.org/en/docs/http/ngx_http_dav_module.html#min_delete_depth)\ [mirror](https://nginx.org/en/docs/http/ngx_http_mirror_module.html#mirror)\ [mirror_request_body](https://nginx.org/en/docs/http/ngx_http_mirror_module.html#mirror_request_body)\ [modern_browser](https://nginx.org/en/docs/http/ngx_http_browser_module.html#modern_browser)\ [modern_browser_value](https://nginx.org/en/docs/http/ngx_http_browser_module.html#modern_browser_value)\ [more_clear_headers](https://github.com/openresty/headers-more-nginx-module?tab=readme-ov-file#more_clear_headers)\ [more_clear_input_headers](https://github.com/openresty/headers-more-nginx-module?tab=readme-ov-file#more_clear_input_headers)\ [more_set_headers](https://github.com/openresty/headers-more-nginx-module?tab=readme-ov-file#more_set_headers)\ [more_set_input_headers](https://github.com/openresty/headers-more-nginx-module?tab=readme-ov-file#more_set_input_headers)\ [mp4](https://nginx.org/en/docs/http/ngx_http_mp4_module.html#mp4)\ [mp4_buffer_size](https://nginx.org/en/docs/http/ngx_http_mp4_module.html#mp4_buffer_size)\ [mp4_limit_rate](https://nginx.org/en/docs/http/ngx_http_mp4_module.html#mp4_limit_rate)\ [mp4_limit_rate_after](https://nginx.org/en/docs/http/ngx_http_mp4_module.html#mp4_limit_rate_after)\ [mp4_max_buffer_size](https://nginx.org/en/docs/http/ngx_http_mp4_module.html#mp4_max_buffer_size)\ [mp4_start_key_frame](https://nginx.org/en/docs/http/ngx_http_mp4_module.html#mp4_start_key_frame)\ [mqtt](https://nginx.org/en/docs/stream/ngx_stream_mqtt_filter_module.html#mqtt)\ [mqtt_rewrite_buffer_size](https://nginx.org/en/docs/stream/ngx_stream_mqtt_filter_module.html#mqtt_rewrite_buffer_size)\ [mqtt_set_connect](https://nginx.org/en/docs/stream/ngx_stream_mqtt_filter_module.html#mqtt_set_connect)\ [msie_padding](https://nginx.org/en/docs/http/ngx_http_core_module.html#msie_padding)\ [msie_refresh](https://nginx.org/en/docs/http/ngx_http_core_module.html#msie_refresh)\ [multi_accept](https://nginx.org/en/docs/ngx_core_module.html#multi_accept)\ [ntlm](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#ntlm)\ [oidc_provider](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#oidc_provider)\ [open_file_cache](https://nginx.org/en/docs/http/ngx_http_core_module.html#open_file_cache)\ [open_file_cache_errors](https://nginx.org/en/docs/http/ngx_http_core_module.html#open_file_cache_errors)\ [open_file_cache_min_uses](https://nginx.org/en/docs/http/ngx_http_core_module.html#open_file_cache_min_uses)\ [open_file_cache_valid](https://nginx.org/en/docs/http/ngx_http_core_module.html#open_file_cache_valid)\ [open_log_file_cache (ngx_http_log_module)](https://nginx.org/en/docs/http/ngx_http_log_module.html#open_log_file_cache)\ [open_log_file_cache (ngx_stream_log_module)](https://nginx.org/en/docs/stream/ngx_stream_log_module.html#open_log_file_cache)\ [otel_exporter](https://nginx.org/en/docs/ngx_otel_module.html#otel_exporter)\ [otel_service_name](https://nginx.org/en/docs/ngx_otel_module.html#otel_service_name)\ [otel_trace](https://nginx.org/en/docs/ngx_otel_module.html#otel_trace)\ [otel_trace_context](https://nginx.org/en/docs/ngx_otel_module.html#otel_trace_context)\ [otel_span_name](https://nginx.org/en/docs/ngx_otel_module.html#otel_span_name)\ [otel_span_attr](https://nginx.org/en/docs/ngx_otel_module.html#otel_span_attr)\ [output_buffers](https://nginx.org/en/docs/http/ngx_http_core_module.html#output_buffers)\ [override_charset](https://nginx.org/en/docs/http/ngx_http_charset_module.html#override_charset)\ [pass](https://nginx.org/en/docs/stream/ngx_stream_pass_module.html#pass)\ [pid](https://nginx.org/en/docs/ngx_core_module.html#pid)\ [pop3_auth](https://nginx.org/en/docs/mail/ngx_mail_pop3_module.html#pop3_auth)\ [pop3_capabilities](https://nginx.org/en/docs/mail/ngx_mail_pop3_module.html#pop3_capabilities)\ [port_in_redirect](https://nginx.org/en/docs/http/ngx_http_core_module.html#port_in_redirect)\ [postpone_output](https://nginx.org/en/docs/http/ngx_http_core_module.html#postpone_output)\ [preread_buffer_size (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#preread_buffer_size)\ [preread_timeout (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#preread_timeout)\ [protocol](https://nginx.org/en/docs/mail/ngx_mail_core_module.html#protocol)\ [proxy_buffer](https://nginx.org/en/docs/mail/ngx_mail_proxy_module.html#proxy_buffer)\ [proxy_buffer_size (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffer_size)\ [proxy_buffer_size (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_buffer_size)\ [proxy_buffering](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffering)\ [proxy_buffers](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffers)\ [proxy_busy_buffers_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_busy_buffers_size)\ [proxy_cache](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache)\ [proxy_cache_background_update](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_background_update)\ [proxy_cache_bypass](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_bypass)\ [proxy_cache_convert_head](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_convert_head)\ [proxy_cache_key](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_key)\ [proxy_cache_lock](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_lock)\ [proxy_cache_lock_age](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_lock_age)\ [proxy_cache_lock_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_lock_timeout)\ [proxy_cache_max_range_offset](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_max_range_offset)\ [proxy_cache_methods](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_methods)\ [proxy_cache_min_uses](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_min_uses)\ [proxy_cache_path](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_path)\ [proxy_cache_purge](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_purge)\ [proxy_cache_revalidate](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_revalidate)\ [proxy_cache_use_stale](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_use_stale)\ [proxy_cache_valid](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_valid)\ [proxy_connect_timeout (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_connect_timeout)\ [proxy_connect_timeout (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_connect_timeout)\ [proxy_cookie_domain](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cookie_domain)\ [proxy_cookie_flags](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cookie_flags)\ [proxy_cookie_path](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cookie_path)\ [proxy_download_rate (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_download_rate)\ [proxy_force_ranges](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_force_ranges)\ [proxy_half_close (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_half_close)\ [proxy_headers_hash_bucket_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_headers_hash_bucket_size)\ [proxy_headers_hash_max_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_headers_hash_max_size)\ [proxy_hide_header](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_hide_header)\ [proxy_http_version](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_http_version)\ [proxy_ignore_client_abort](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ignore_client_abort)\ [proxy_ignore_headers](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ignore_headers)\ [proxy_intercept_errors](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_intercept_errors)\ [proxy_limit_rate](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_limit_rate)\ [proxy_max_temp_file_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_max_temp_file_size)\ [proxy_method](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_method)\ [proxy_next_upstream (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream)\ [proxy_next_upstream (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_next_upstream)\ [proxy_next_upstream_timeout (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream_timeout)\ [proxy_next_upstream_timeout (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_next_upstream_timeout)\ [proxy_next_upstream_tries (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream_tries)\ [proxy_next_upstream_tries (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_next_upstream_tries)\ [proxy_no_cache](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_no_cache)\ [proxy_pass (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass)\ [proxy_pass (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_pass)\ [proxy_pass_error_message](https://nginx.org/en/docs/mail/ngx_mail_proxy_module.html#proxy_pass_error_message)\ [proxy_pass_header](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass_header)\ [proxy_pass_request_body](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass_request_body)\ [proxy_pass_request_headers](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass_request_headers)\ [proxy_protocol (ngx_mail_proxy_module)](https://nginx.org/en/docs/mail/ngx_mail_proxy_module.html#proxy_protocol)\ [proxy_protocol (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_protocol)\ [proxy_protocol_timeout (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#proxy_protocol_timeout)\ [proxy_read_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_read_timeout)\ [proxy_redirect](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_redirect)\ [proxy_requests (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_requests)\ [proxy_request_buffering](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_request_buffering)\ [proxy_responses (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_responses)\ [proxy_send_lowat](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_send_lowat)\ [proxy_send_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_send_timeout)\ [proxy_session_drop (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_session_drop)\ [proxy_set_body](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_body)\ [proxy_set_header](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_header)\ [proxy_smtp_auth](https://nginx.org/en/docs/mail/ngx_mail_proxy_module.html#proxy_smtp_auth)\ [proxy_socket_keepalive (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_socket_keepalive)\ [proxy_socket_keepalive (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_socket_keepalive)\ [proxy_ssl (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl)\ [proxy_ssl_certificate (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_certificate)\ [proxy_ssl_certificate (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_certificate)\ [proxy_ssl_certificate_cache (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_certificate_cache)\ [proxy_ssl_certificate_cache (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_certificate_cache)\ [proxy_ssl_certificate_key (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_certificate_key)\ [proxy_ssl_certificate_key (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_certificate_key)\ [proxy_ssl_ciphers (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_ciphers)\ [proxy_ssl_ciphers (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_ciphers)\ [proxy_ssl_conf_command (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_conf_command)\ [proxy_ssl_conf_command (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_conf_command)\ [proxy_ssl_crl (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_crl)\ [proxy_ssl_crl (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_crl)\ [proxy_ssl_name (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_name)\ [proxy_ssl_name (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_name)\ [proxy_ssl_password_file (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_password_file)\ [proxy_ssl_password_file (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_password_file)\ [proxy_ssl_protocols (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_protocols)\ [proxy_ssl_protocols (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_protocols)\ [proxy_ssl_server_name (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_server_name)\ [proxy_ssl_server_name (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_server_name)\ [proxy_ssl_session_reuse (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_session_reuse)\ [proxy_ssl_session_reuse (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_session_reuse)\ [proxy_ssl_trusted_certificate (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_trusted_certificate)\ [proxy_ssl_trusted_certificate (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_trusted_certificate)\ [proxy_ssl_verify (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_verify)\ [proxy_ssl_verify (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_verify)\ [proxy_ssl_verify_depth (ngx_http_proxy_module)](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ssl_verify_depth)\ [proxy_ssl_verify_depth (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_ssl_verify_depth)\ [proxy_store](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_store)\ [proxy_store_access](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_store_access)\ [proxy_temp_file_write_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_temp_file_write_size)\ [proxy_temp_path](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_temp_path)\ [proxy_timeout (ngx_mail_proxy_module)](https://nginx.org/en/docs/mail/ngx_mail_proxy_module.html#proxy_timeout)\ [proxy_timeout (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_timeout)\ [proxy_upload_rate (ngx_stream_proxy_module)](https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_upload_rate)\ [queue](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#queue)\ [quic_active_connection_id_limit](http://nginx.org/en/docs/http/ngx_http_v3_module.html#quic_active_connection_id_limit)\ [quic_gso](http://nginx.org/en/docs/http/ngx_http_v3_module.html#quic_gso)\ [quic_host_key](http://nginx.org/en/docs/http/ngx_http_v3_module.html#quic_host_key)\ [quic_retry](http://nginx.org/en/docs/http/ngx_http_v3_module.html#quic_retry)\ [random (ngx_http_upstream_module)](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#random)\ [random (ngx_stream_upstream_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#random)\ [random_index](https://nginx.org/en/docs/http/ngx_http_random_index_module.html#random_index)\ [read_ahead](https://nginx.org/en/docs/http/ngx_http_core_module.html#read_ahead)\ [read_timeout](https://nginx.org/en/docs/ngx_mgmt_module.html#read_timeout)\ [real_ip_header](https://nginx.org/en/docs/http/ngx_http_realip_module.html#real_ip_header)\ [real_ip_recursive](https://nginx.org/en/docs/http/ngx_http_realip_module.html#real_ip_recursive)\ [recursive_error_pages](https://nginx.org/en/docs/http/ngx_http_core_module.html#recursive_error_pages)\ [redirect_uri](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#redirect_uri)\ [referer_hash_bucket_size](https://nginx.org/en/docs/http/ngx_http_referer_module.html#referer_hash_bucket_size)\ [referer_hash_max_size](https://nginx.org/en/docs/http/ngx_http_referer_module.html#referer_hash_max_size)\ [request_pool_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#request_pool_size)\ [reset_timedout_connection](https://nginx.org/en/docs/http/ngx_http_core_module.html#reset_timedout_connection)\ [resolver (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver)\ [resolver (ngx_mail_core_module)](https://nginx.org/en/docs/mail/ngx_mail_core_module.html#resolver)\ [resolver (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#resolver)\ [resolver (ngx_stream_upstream_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#resolver)\ [resolver (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#resolver)\ [resolver_timeout (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver_timeout)\ [resolver_timeout (ngx_mail_core_module)](https://nginx.org/en/docs/mail/ngx_mail_core_module.html#resolver_timeout)\ [resolver_timeout (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#resolver_timeout)\ [resolver_timeout (ngx_stream_upstream_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#resolver_timeout)\ [resolver_timeout (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#resolver_timeout)\ [return (ngx_http_rewrite_module)](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#return)\ [return (ngx_stream_return_module)](https://nginx.org/en/docs/stream/ngx_stream_return_module.html#return)\ [rewrite](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#rewrite)\ [rewrite_log](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#rewrite_log)\ [root](https://nginx.org/en/docs/http/ngx_http_core_module.html#root)\ [satisfy](https://nginx.org/en/docs/http/ngx_http_core_module.html#satisfy)\ [scgi_buffer_size](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_buffer_size)\ [scgi_buffering](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_buffering)\ [scgi_buffers](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_buffers)\ [scgi_busy_buffers_size](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_busy_buffers_size)\ [scgi_cache](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache)\ [scgi_cache_background_update](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_background_update)\ [scgi_cache_bypass](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_bypass)\ [scgi_cache_key](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_key)\ [scgi_cache_lock](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_lock)\ [scgi_cache_lock_age](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_lock_age)\ [scgi_cache_lock_timeout](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_lock_timeout)\ [scgi_cache_max_range_offset](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_max_range_offset)\ [scgi_cache_methods](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_methods)\ [scgi_cache_min_uses](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_min_uses)\ [scgi_cache_path](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_path)\ [scgi_cache_purge](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_purge)\ [scgi_cache_revalidate](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_revalidate)\ [scgi_cache_use_stale](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_use_stale)\ [scgi_cache_valid](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_cache_valid)\ [scgi_connect_timeout](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_connect_timeout)\ [scgi_force_ranges](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_force_ranges)\ [scgi_hide_header](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_hide_header)\ [scgi_ignore_client_abort](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_ignore_client_abort)\ [scgi_ignore_headers](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_ignore_headers)\ [scgi_intercept_errors](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_intercept_errors)\ [scgi_limit_rate](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_limit_rate)\ [scgi_max_temp_file_size](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_max_temp_file_size)\ [scgi_next_upstream](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_next_upstream)\ [scgi_next_upstream_timeout](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_next_upstream_timeout)\ [scgi_next_upstream_tries](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_next_upstream_tries)\ [scgi_no_cache](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_no_cache)\ [scgi_param](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_param)\ [scgi_pass](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_pass)\ [scgi_pass_header](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_pass_header)\ [scgi_pass_request_body](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_pass_request_body)\ [scgi_pass_request_headers](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_pass_request_headers)\ [scgi_read_timeout](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_read_timeout)\ [scgi_request_buffering](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_request_buffering)\ [scgi_send_timeout](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_send_timeout)\ [scgi_socket_keepalive](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_socket_keepalive)\ [scgi_store](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_store)\ [scgi_store_access](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_store_access)\ [scgi_temp_file_write_size](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_temp_file_write_size)\ [scgi_temp_path](https://nginx.org/en/docs/http/ngx_http_scgi_module.html#scgi_temp_path)\ [scope](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#scope)\ [secure_link](https://nginx.org/en/docs/http/ngx_http_secure_link_module.html#secure_link)\ [secure_link_md5](https://nginx.org/en/docs/http/ngx_http_secure_link_module.html#secure_link_md5)\ [secure_link_secret](https://nginx.org/en/docs/http/ngx_http_secure_link_module.html#secure_link_secret)\ [send_lowat](https://nginx.org/en/docs/http/ngx_http_core_module.html#send_lowat)\ [send_timeout (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#send_timeout)\ [send_timeout (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#send_timeout)\ [sendfile](https://nginx.org/en/docs/http/ngx_http_core_module.html#sendfile)\ [sendfile_max_chunk](https://nginx.org/en/docs/http/ngx_http_core_module.html#sendfile_max_chunk)\ [server (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#server)\ [server (ngx_mail_core_module)](https://nginx.org/en/docs/mail/ngx_mail_core_module.html#server)\ [server (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#server)\ [server (ngx_stream_upstream_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#server)\ [server_name (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#server_name)\ [server_name (ngx_mail_core_module)](https://nginx.org/en/docs/mail/ngx_mail_core_module.html#server_name)\ [server_name (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#server_name)\ [server_name_in_redirect](https://nginx.org/en/docs/http/ngx_http_core_module.html#server_name_in_redirect)\ [server_tokens](https://nginx.org/en/docs/http/ngx_http_core_module.html#server_tokens)\ [session_log](https://nginx.org/en/docs/http/ngx_http_session_log_module.html#session_log)\ [session_log_format](https://nginx.org/en/docs/http/ngx_http_session_log_module.html#session_log_format)\ [session_log_zone](https://nginx.org/en/docs/http/ngx_http_session_log_module.html#session_log_zone)\ [session_store](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#session_store)\ [session_timeout](https://nginx.org/en/docs/http/ngx_http_oidc_module.html#session_timeout)\ [set (ngx_http_rewrite_module)](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#set)\ [set (ngx_stream_set_module)](https://nginx.org/en/docs/stream/ngx_stream_set_module.html#set)\ [set_real_ip_from (ngx_http_realip_module)](https://nginx.org/en/docs/http/ngx_http_realip_module.html#set_real_ip_from)\ [set_real_ip_from (ngx_mail_realip_module)](https://nginx.org/en/docs/mail/ngx_mail_realip_module.html#set_real_ip_from)\ [set_real_ip_from (ngx_stream_realip_module)](https://nginx.org/en/docs/stream/ngx_stream_realip_module.html#set_real_ip_from)\ [slice](https://nginx.org/en/docs/http/ngx_http_slice_module.html#slice)\ [smtp_auth](https://nginx.org/en/docs/mail/ngx_mail_smtp_module.html#smtp_auth)\ [smtp_capabilities](https://nginx.org/en/docs/mail/ngx_mail_smtp_module.html#smtp_capabilities)\ [smtp_client_buffer](https://nginx.org/en/docs/mail/ngx_mail_smtp_module.html#smtp_client_buffer)\ [smtp_greeting_delay](https://nginx.org/en/docs/mail/ngx_mail_smtp_module.html#smtp_greeting_delay)\ [source_charset](https://nginx.org/en/docs/http/ngx_http_charset_module.html#source_charset) [split_clients (ngx_http_split_clients_module)](https://nginx.org/en/docs/http/ngx_http_split_clients_module.html#split_clients)\ [split_clients (ngx_stream_split_clients_module)](https://nginx.org/en/docs/stream/ngx_stream_split_clients_module.html#split_clients)\ [ssi](https://nginx.org/en/docs/http/ngx_http_ssi_module.html#ssi)\ [ssi_last_modified](https://nginx.org/en/docs/http/ngx_http_ssi_module.html#ssi_last_modified)\ [ssi_min_file_chunk](https://nginx.org/en/docs/http/ngx_http_ssi_module.html#ssi_min_file_chunk)\ [ssi_silent_errors](https://nginx.org/en/docs/http/ngx_http_ssi_module.html#ssi_silent_errors)\ [ssi_types](https://nginx.org/en/docs/http/ngx_http_ssi_module.html#ssi_types)\ [ssi_value_length](https://nginx.org/en/docs/http/ngx_http_ssi_module.html#ssi_value_length)\ [ssl (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl)\ [ssl (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl)\ [ssl (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl)\ [ssl_buffer_size](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_buffer_size)\ [ssl_certificate (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_certificate)\ [ssl_certificate (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_certificate)\ [ssl_certificate (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_certificate)\ [ssl_certificate (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_certificate)\ [ssl_certificate_cache (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_certificate_cache)\ [ssl_certificate_cache (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_certificate_cache)\ [ssl_certificate_key (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_certificate_key)\ [ssl_certificate_key (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_certificate_key)\ [ssl_certificate_key (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_certificate_key)\ [ssl_certificate_key (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_certificate_key)\ [ssl_ciphers (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_ciphers)\ [ssl_ciphers (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_ciphers)\ [ssl_ciphers (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_ciphers)\ [ssl_client_certificate (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_client_certificate)\ [ssl_client_certificate (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_client_certificate)\ [ssl_client_certificate (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_client_certificate)\ [ssl_conf_command (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_conf_command)\ [ssl_conf_command (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_conf_command)\ [ssl_conf_command (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_conf_command)\ [ssl_crl (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_crl)\ [ssl_crl (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_crl)\ [ssl_crl (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_crl)\ [ssl_dhparam (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_dhparam)\ [ssl_dhparam (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_dhparam)\ [ssl_early_data](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_early_data)\ [ssl_ecdh_curve (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_ecdh_curve)\ [ssl_ecdh_curve (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_ecdh_curve)\ [ssl_ecdh_curve (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_ecdh_curve)\ [ssl_handshake_timeout](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_handshake_timeout)\ [ssl_name](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_name)\ [ssl_object_cache_inheritable](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_object_cache_inheritable)\ [ssl_ocsp](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_ocsp)\ [ssl_ocsp_cache](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_ocsp_cache)\ [ssl_ocsp_responder](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_ocsp_responder)\ [ssl_password_file (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_password_file)\ [ssl_password_file (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_password_file)\ [ssl_password_file (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_password_file)\ [ssl_prefer_server_ciphers (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_prefer_server_ciphers)\ [ssl_prefer_server_ciphers (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_prefer_server_ciphers)\ [ssl_prefer_server_ciphers (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_prefer_server_ciphers)\ [ssl_preread (ngx_stream_ssl_preread_module)](http://nginx.org/en/docs/stream/ngx_stream_ssl_preread_module.html#var_ssl_preread_protocol)\ [ssl_protocols (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_protocols)\ [ssl_protocols (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_protocols)\ [ssl_protocols (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_protocols)\ [ssl_protocols (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_protocols)\ [ssl_reject_handshake](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_reject_handshake)\ [ssl_server_name](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_server_name)\ [ssl_session_cache (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_session_cache)\ [ssl_session_cache (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_session_cache)\ [ssl_session_cache (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_session_cache)\ [ssl_session_ticket_key (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_session_ticket_key)\ [ssl_session_ticket_key (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_session_ticket_key)\ [ssl_session_ticket_key (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_session_ticket_key)\ [ssl_session_tickets (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_session_tickets)\ [ssl_session_tickets (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_session_tickets)\ [ssl_session_tickets (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_session_tickets)\ [ssl_session_timeout (ngx_http_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_session_timeout)\ [ssl_session_timeout (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_session_timeout)\ [ssl_session_timeout (ngx_stream_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_session_timeout)\ [ssl_stapling](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_stapling)\ [ssl_stapling_file](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_stapling_file)\ [ssl_stapling_responder](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_stapling_responder)\ [ssl_stapling_verify](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_stapling_verify)\ [ssl_trusted_certificate (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_trusted_certificate)\ [ssl_trusted_certificate (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_trusted_certificate)\ [ssl_trusted_certificate (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_trusted_certificate)\ [ssl_trusted_certificate (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_trusted_certificate)\ [ssl_verify](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_verify)\ [ssl_verify_client (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_verify_client)\ [ssl_verify_client (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_verify_client)\ [ssl_verify_client (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_verify_client)\ [ssl_verify_depth (ngx_http_ssl_module)](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_verify_depth)\ [ssl_verify_depth (ngx_mail_ssl_module)](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_verify_depth)\ [ssl_verify_depth (ngx_stream_ssl_module)](https://nginx.org/en/docs/stream/ngx_stream_ssl_module.html#ssl_verify_depth)\ [ssl_verify_depth (ngx_mgmt_module)](https://nginx.org/en/docs/ngx_mgmt_module.html#ssl_verify_depth)\ [starttls](https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#starttls)\ [state (ngx_http_upstream_module)](http://nginx.org/en/docs/http/ngx_http_upstream_module.html#state)\ [status_zone (ngx_http_api_module)](https://nginx.org/en/docs/http/ngx_http_api_module.html#status_zone)\ [sticky](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#sticky)\ [sticky_cookie_insert](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#sticky_cookie_insert)\ [stream (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#stream)\ [stub_status](https://nginx.org/en/docs/http/ngx_http_stub_status_module.html#stub_status)\ [sub_filter](https://nginx.org/en/docs/http/ngx_http_sub_module.html#sub_filter)\ [sub_filter_last_modified](https://nginx.org/en/docs/http/ngx_http_sub_module.html#sub_filter_last_modified)\ [sub_filter_once](https://nginx.org/en/docs/http/ngx_http_sub_module.html#sub_filter_once)\ [sub_filter_types](https://nginx.org/en/docs/http/ngx_http_sub_module.html#sub_filter_types)\ [subrequest_output_buffer_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#subrequest_output_buffer_size)\ [tcp_nodelay (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#tcp_nodelay)\ [tcp_nodelay (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#tcp_nodelay)\ [tcp_nopush](https://nginx.org/en/docs/http/ngx_http_core_module.html#tcp_nopush)\ [thread_pool](https://nginx.org/en/docs/ngx_core_module.html#thread_pool)\ [timeout](https://nginx.org/en/docs/mail/ngx_mail_core_module.html#timeout)\ [timer_resolution](https://nginx.org/en/docs/ngx_core_module.html#timer_resolution)\ [try_files](https://nginx.org/en/docs/http/ngx_http_core_module.html#try_files)\ [types](https://nginx.org/en/docs/http/ngx_http_core_module.html#types)\ [types_hash_bucket_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#types_hash_bucket_size)\ [types_hash_max_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#types_hash_max_size)\ [underscores_in_headers](https://nginx.org/en/docs/http/ngx_http_core_module.html#underscores_in_headers)\ [uninitialized_variable_warn](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#uninitialized_variable_warn)\ [upstream (ngx_http_upstream_module)](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#upstream)\ [upstream (ngx_stream_upstream_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#upstream)\ [upstream_conf](https://nginx.org/en/docs/http/ngx_http_upstream_conf_module.html#upstream_conf)\ [usage_report](https://nginx.org/en/docs/ngx_mgmt_module.html#usage_report)\ [use](https://nginx.org/en/docs/ngx_core_module.html#use)\ [user](https://nginx.org/en/docs/ngx_core_module.html#user)\ [userid](https://nginx.org/en/docs/http/ngx_http_userid_module.html#userid)\ [userid_domain](https://nginx.org/en/docs/http/ngx_http_userid_module.html#userid_domain)\ [userid_expires](https://nginx.org/en/docs/http/ngx_http_userid_module.html#userid_expires)\ [userid_flags](https://nginx.org/en/docs/http/ngx_http_userid_module.html#userid_flags)\ [userid_mark](https://nginx.org/en/docs/http/ngx_http_userid_module.html#userid_mark)\ [userid_name](https://nginx.org/en/docs/http/ngx_http_userid_module.html#userid_name)\ [userid_p3p](https://nginx.org/en/docs/http/ngx_http_userid_module.html#userid_p3p)\ [userid_path](https://nginx.org/en/docs/http/ngx_http_userid_module.html#userid_path)\ [userid_service](https://nginx.org/en/docs/http/ngx_http_userid_module.html#userid_service)\ [uuid_file](https://nginx.org/en/docs/ngx_mgmt_module.html#uuid_file)\ [uwsgi_buffer_size](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_buffer_size)\ [uwsgi_buffering](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_buffering)\ [uwsgi_buffers](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_buffers)\ [uwsgi_busy_buffers_size](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_busy_buffers_size)\ [uwsgi_cache](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache)\ [uwsgi_cache_background_update](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_background_update)\ [uwsgi_cache_bypass](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_bypass)\ [uwsgi_cache_key](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_key)\ [uwsgi_cache_lock](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_lock)\ [uwsgi_cache_lock_age](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_lock_age)\ [uwsgi_cache_lock_timeout](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_lock_timeout)\ [uwsgi_cache_max_range_offset](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_max_range_offset)\ [uwsgi_cache_methods](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_methods)\ [uwsgi_cache_min_uses](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_min_uses)\ [uwsgi_cache_path](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_path)\ [uwsgi_cache_purge](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_purge)\ [uwsgi_cache_revalidate](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_revalidate)\ [uwsgi_cache_use_stale](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_use_stale)\ [uwsgi_cache_valid](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_cache_valid)\ [uwsgi_connect_timeout](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_connect_timeout)\ [uwsgi_force_ranges](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_force_ranges)\ [uwsgi_hide_header](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_hide_header)\ [uwsgi_ignore_client_abort](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ignore_client_abort)\ [uwsgi_ignore_headers](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ignore_headers)\ [uwsgi_intercept_errors](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_intercept_errors)\ [uwsgi_limit_rate](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_limit_rate)\ [uwsgi_max_temp_file_size](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_max_temp_file_size)\ [uwsgi_modifier1](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_modifier1)\ [uwsgi_modifier2](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_modifier2)\ [uwsgi_next_upstream](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_next_upstream)\ [uwsgi_next_upstream_timeout](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_next_upstream_timeout)\ [uwsgi_next_upstream_tries](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_next_upstream_tries)\ [uwsgi_no_cache](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_no_cache)\ [uwsgi_param](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_param)\ [uwsgi_pass](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_pass)\ [uwsgi_pass_header](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_pass_header)\ [uwsgi_pass_request_body](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_pass_request_body)\ [uwsgi_pass_request_headers](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_pass_request_headers)\ [uwsgi_read_timeout](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_read_timeout)\ [uwsgi_request_buffering](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_request_buffering)\ [uwsgi_send_timeout](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_send_timeout)\ [uwsgi_socket_keepalive](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_socket_keepalive)\ [uwsgi_ssl_certificate](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_certificate)\ [uwsgi_ssl_certificate_cache](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_certificate_cache)\ [uwsgi_ssl_certificate_key](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_certificate_key)\ [uwsgi_ssl_conf_command](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_conf_command)\ [uwsgi_ssl_crl](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_crl)\ [uwsgi_ssl_name](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_name)\ [uwsgi_ssl_password_file](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_password_file)\ [uwsgi_ssl_protocols](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_protocols)\ [uwsgi_ssl_server_name](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_server_name)\ [uwsgi_ssl_session_reuse](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_session_reuse)\ [uwsgi_ssl_trusted_certificate](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_trusted_certificate)\ [uwsgi_ssl_verify](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_verify)\ [uwsgi_ssl_verify_depth](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_ssl_verify_depth)\ [uwsgi_store](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_store)\ [uwsgi_store_access](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_store_access)\ [uwsgi_temp_file_write_size](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_temp_file_write_size)\ [uwsgi_temp_path](https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_temp_path)\ [valid_referers](https://nginx.org/en/docs/http/ngx_http_referer_module.html#valid_referers)\ [variables_hash_bucket_size (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#variables_hash_bucket_size)\ [variables_hash_bucket_size (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#variables_hash_bucket_size)\ [variables_hash_max_size (ngx_http_core_module)](https://nginx.org/en/docs/http/ngx_http_core_module.html#variables_hash_max_size)\ [variables_hash_max_size (ngx_stream_core_module)](https://nginx.org/en/docs/stream/ngx_stream_core_module.html#variables_hash_max_size)\ [worker_aio_requests](https://nginx.org/en/docs/ngx_core_module.html#worker_aio_requests)\ [worker_connections](https://nginx.org/en/docs/ngx_core_module.html#worker_connections)\ [worker_cpu_affinity](https://nginx.org/en/docs/ngx_core_module.html#worker_cpu_affinity)\ [worker_priority](https://nginx.org/en/docs/ngx_core_module.html#worker_priority)\ [worker_processes](https://nginx.org/en/docs/ngx_core_module.html#worker_processes)\ [worker_rlimit_core](https://nginx.org/en/docs/ngx_core_module.html#worker_rlimit_core)\ [worker_rlimit_nofile](https://nginx.org/en/docs/ngx_core_module.html#worker_rlimit_nofile)\ [worker_shutdown_timeout](https://nginx.org/en/docs/ngx_core_module.html#worker_shutdown_timeout)\ [working_directory](https://nginx.org/en/docs/ngx_core_module.html#working_directory)\ [xclient](https://nginx.org/en/docs/mail/ngx_mail_proxy_module.html#xclient)\ [xml_entities](https://nginx.org/en/docs/http/ngx_http_xslt_module.html#xml_entities)\ [xslt_last_modified](https://nginx.org/en/docs/http/ngx_http_xslt_module.html#xslt_last_modified)\ [xslt_param](https://nginx.org/en/docs/http/ngx_http_xslt_module.html#xslt_param)\ [xslt_string_param](http://nginx.org/en/docs/http/ngx_http_xslt_module.html#xslt_string_param)\ [xslt_stylesheet](https://nginx.org/en/docs/http/ngx_http_xslt_module.html#xslt_stylesheet)\ [xslt_types](https://nginx.org/en/docs/http/ngx_http_xslt_module.html#xslt_types)\ [zone (ngx_http_upstream_module)](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#zone)\ [zone (ngx_stream_upstream_module)](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#zone)\ -------------------------------------------------------------------------------- title: "Overview" url: https://docs.nginx.com/nginxaas/aws/deploy/ssl-tls-certificates/overview/index.md description: Reference for supported SSL/TLS certificate types and how to add them to an F5 NGINXaaS for AWS deployment. -------------------------------------------------------------------------------- Use F5 NGINXaaS for AWS to secure traffic by adding SSL/TLS certificates to a deployment. ## Supported certificate types and formats NGINX supports the following certificate formats: - PEM format certificates. You can upload these certificates as text, as files, or as secrets from [AWS Secrets Manager](https://docs.aws.amazon.com/secretsmanager/latest/userguide/intro.html). Encrypt your certificates, keys, and PEM files using one of these algorithms: - RSA - ECC/ECDSA ## Add SSL/TLS certificates NGINXaaS for AWS supports two ways to manage your certificates and keys securely: **NGINXaaS console**: Manage certificates alongside the NGINX configurations that reference them. See [Add certificates using the NGINXaaS Console](/nginxaas/aws/deploy/ssl-tls-certificates/ssl-tls-certificates-console.md). **AWS Secrets Manager**: Fetch secrets directly from [AWS Secrets Manager](https://docs.aws.amazon.com/secretsmanager/latest/userguide/intro.html), keeping credentials within AWS. See [Add certificates from AWS Secrets Manager](/nginxaas/aws/deploy/ssl-tls-certificates/ssl-tls-certificates-secrets-manager.md). ## Certificate rotation NGINXaaS for AWS supports automatic and manual rotation for AWS Secrets Manager certificates: **Automatic rotation**: Let NGINXaaS for AWS pick up new certificate versions automatically with no configuration changes needed. See [Rotate an AWS Secrets Manager certificate (automatic)](/nginxaas/aws/deploy/ssl-tls-certificates/ssl-tls-certificates-secrets-manager.md#rotate-an-aws-secrets-manager-certificate-automatic). **Manual rotation**: When you need to update certificates immediately, use **Reapply Configuration** in the console to refetch secrets right away. See [Rotate an AWS Secrets Manager certificate (manual)](/nginxaas/aws/deploy/ssl-tls-certificates/ssl-tls-certificates-secrets-manager.md#rotate-an-aws-secrets-manager-certificate-manual). -------------------------------------------------------------------------------- title: "Overview" url: https://docs.nginx.com/nginxaas/google/deploy/ssl-tls-certificates/overview/index.md -------------------------------------------------------------------------------- Use F5 NGINXaaS for Google Cloud (NGINXaaS) to secure traffic by adding SSL/TLS certificates to a deployment. ## Supported certificate types and formats NGINX supports the following certificate formats: - PEM format certificates. You can upload these certificates as text, as files, or as secrets from [Secret Manager](https://docs.cloud.google.com/secret-manager/docs/overview). Encrypt your certificates, keys, and PEM files using one of these standards: - RSA - ECC/ECDSA ## Add SSL/TLS certificates NGINXaaS supports two ways to manage your certificates and keys securely: **NGINXaaS console**: Manage certificates alongside the NGINX configurations that reference them. See [Add certificates using the NGINXaaS Console](/nginxaas/google/deploy/ssl-tls-certificates/ssl-tls-certificates-console.md). **Google Secret Manager**: Fetch secrets directly from [Secret Manager](https://docs.cloud.google.com/secret-manager/docs/overview), keeping credentials within Google Cloud. See [Add certificates from Secret Manager](/nginxaas/google/deploy/ssl-tls-certificates/ssl-tls-certificates-secret-manager.md). ## Certificate rotation NGINXaaS supports automatic and manual rotation for Secret Manager certificates: **Automatic rotation**: Let NGINXaaS pick up new certificate versions automatically with no configuration changes needed. See [Rotate a Secret Manager certificate (automatic)](/nginxaas/google/deploy/ssl-tls-certificates/ssl-tls-certificates-secret-manager.md#rotate-a-secret-manager-certificate-automatic). **Manual rotation**: When you need to update certificates immediately, use **Reapply Configuration** in the console to refetch secrets right away. See [Rotate a Secret Manager certificate (manual)](/nginxaas/google/deploy/ssl-tls-certificates/ssl-tls-certificates-secret-manager.md#rotate-a-secret-manager-certificate-manual). -------------------------------------------------------------------------------- title: "Add certificates from AWS Secrets Manager" url: https://docs.nginx.com/nginxaas/aws/deploy/ssl-tls-certificates/ssl-tls-certificates-secrets-manager/index.md description: Fetch SSL/TLS certificates for F5 NGINXaaS for AWS directly from AWS Secrets Manager. -------------------------------------------------------------------------------- F5 NGINXaaS for AWS can fetch secrets directly from [AWS Secrets Manager](https://docs.aws.amazon.com/secretsmanager/latest/userguide/intro.html) to use as certificates and keys in your NGINX configuration, ensuring your credentials remain securely within AWS. ## Prerequisites If you haven't already done so, [create an NGINXaaS deployment](/nginxaas/aws/deploy/create-deployment/deploy-console.md) with an IAM role. See [Identity and access management](/nginxaas/aws/deploy/access-management.md) for more information. ### IAM role permissions policy To allow NGINXaaS for AWS access to your AWS Secrets Manager secrets, you must attach a permissions policy to your IAM role. The policy must allow the `secretsmanager:GetSecretValue` action. For example, the following policy allows access to the specified secret. ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "secretsmanager:GetSecretValue", "Resource": "arn:aws:secretsmanager:us-east-1:123456789012:secret:secretName-AbCdEf" } ] } ``` See [AWS Secrets Manager identity-based policies](https://docs.aws.amazon.com/secretsmanager/latest/userguide/auth-and-access_iam-policies.html#auth-and-access_examples_identity_read) for more examples. NGINXaaS for AWS also supports attribute-based access control (ABAC) by restricting access based on tag attributes. For example, the following policy allows only the specified NGINXaaS deployment to access the secret. ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "secretsmanager:GetSecretValue", "Resource": "arn:aws:secretsmanager:us-east-1:123456789012:secret:secretName-AbCdEf", "Condition": { "StringEquals": { "aws:PrincipalTag/NGINXaaS:DeploymentName": "test-deployment" } } } ] } ``` The session tags passed in the request to fetch the secret will appear in `AssumeRole` [events in CloudTrail](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_session-tags.html#id_session-tags_ctlogs). The following tags are supported: - `NGINXaaS:OrganizationID` - `NGINXaaS:DeploymentID` - `NGINXaaS:DeploymentName` ## Add an SSL/TLS certificate to AWS Secrets Manager To add an SSL/TLS certificate and key as a secret to AWS Secrets Manager, 1. Make sure your certificate and key file(s) are in one of the [accepted formats](/nginxaas/aws/deploy/ssl-tls-certificates/overview.md#supported-certificate-types-and-formats). 1. Follow AWS's [instructions to create a secret in AWS Secrets Manager](https://docs.aws.amazon.com/secretsmanager/latest/userguide/create_secret.html) and set your certificate and private key file contents as the `plaintext` secret value. **Note:** There are many ways to manage your SSL/TLS certificates and keys. For example, you can include the PEM certificate data in the same secret as your private key. The `ssl_certificate` directive supports a single file containing multiple certificates and a key. See NGINX's [Configuring HTTPS servers](https://nginx.org/en/docs/http/configuring_https_servers.html) guide for more details. ## Use an AWS Secrets Manager certificate in an NGINX configuration To add your AWS Secrets Manager certificate and key to an NGINX configuration in the NGINXaaS console, 1. Select **Configurations** in the left menu. 2. Select the ellipsis (three dots) next to the configuration you want to edit, and select **Edit**. 3. Select [icon: plus] **Add File**. 4. Select **Cloud Provider Secret** as the type of file you want to add. 5. Select **AWS Secrets Manager** as the **Cloud Secret Manager**. 6. Provide the required information: | Field | Description | Note | |---------------------------- | ---------------------------- | ---- | | Secret ARN | The Amazon Resource Name (ARN) of the secret in AWS Secrets Manager | The ARN must match the format `arn::secretsmanager:::secret:-<6_RANDOM_CHARACTERS>`. | | Version Stage | The staging label of the secret version. | Version stage is optional and cannot be specified at the same time as Version ID. If you don't specify a version stage or a version ID, NGINXaaS for AWS fetches the version labeled `AWSCURRENT`. See AWS's [documentation on secret versions](https://docs.aws.amazon.com/secretsmanager/latest/userguide/whats-in-a-secret.html#term_version) for more information. | | Version ID | The unique identifier of the secret version. | Version ID is optional and cannot be specified at the same time as version stage. If you don't specify a version stage or a version ID, NGINXaaS for AWS fetches the version labeled `AWSCURRENT`. | | File Path | NGINXaaS writes the secret to this file path, so it can be used with NGINX directives such as `ssl_certificate` or `ssl_certificate_key` in your NGINX configuration. | The path must be unique within the configuration. See the [NGINX Filesystem Restrictions table](/nginxaas/aws/deploy/nginx-configuration/configuration-rules.md#nginx-filesystem-restrictions) for the allowed directories the file can be written to. | **tip:** If you set the **Version Stage** to `AWSCURRENT` or leave **Version Stage** and **Version ID** unspecified, NGINXaaS for AWS automatically picks up any new secret version AWS Secrets Manager promotes to `AWSCURRENT` without a configuration change. NGINXaaS for AWS applies new versions within four hours. See [Rotate an AWS Secrets Manager certificate (automatic)](#rotate-an-aws-secrets-manager-certificate-automatic) for details. 7. Update the NGINX configuration to reference the certificate you just added by the path value. 8. Select **Add**, **Next**, and then **Save** to save your changes. ## Update your NGINXaaS deployment's NGINX configuration Before updating your NGINXaaS deployment to use your new NGINX configuration, make sure your deployment already has an [IAM role set up](/nginxaas/aws/deploy/access-management.md) with the `secretsmanager:GetSecretValue` permission granted, so it can fetch certificates. Then, in the NGINXaaS console: 1. Select **Deployments**. 1. Select the deployment you want to edit. 1. In the **Configuration Info** panel, select **Edit**. 1. Select the configuration and configuration version created in the last section. 1. Select **Update Configuration**. **Note:** Configurations with AWS Secrets Manager secrets can only be added to AWS deployments. ## Rotate an AWS Secrets Manager certificate (automatic) If you set the **Version Stage** to `AWSCURRENT` or leave **Version Stage** and **Version ID** unspecified, NGINXaaS for AWS fetches the latest secret version. When you [update the value of a secret](https://docs.aws.amazon.com/secretsmanager/latest/userguide/manage_update-secret-value.html) or [configure an AWS Lambda function to rotate the secret](https://docs.aws.amazon.com/secretsmanager/latest/userguide/rotate-secrets_lambda.html), AWS Secrets Manager moves the `AWSCURRENT` label to the new secret version. NGINXaaS for AWS automatically picks up that new version within four hours. If you set the **Version Stage** to a staging label other than `AWSCURRENT`, NGINXaaS for AWS fetches the secret version the staging label points to. When you [move a staging label to point to a different secret in AWS Secrets Manager](https://docs.aws.amazon.com/secretsmanager/latest/apireference/API_UpdateSecretVersionStage.html), NGINXaaS for AWS automatically picks up that secret within four hours. No configuration changes are required in either case. To confirm your deployment is using an updated certificate, check the **Certificates** list for the new serial number or inspect the certificate at your deployment's endpoint. ## Rotate an AWS Secrets Manager certificate (manual) To immediately refetch secrets without editing your NGINX configuration, use **Reapply Configuration**. This is useful in the following scenarios: - **New secret version**: You've uploaded a new certificate and want NGINXaaS for AWS to use it right away. - **Updated IAM role or permissions**: You've updated your IAM role trust policy or permissions policy and want NGINXaaS for AWS to retry immediately. To reapply your configuration: 1. In the NGINXaaS console, go to your deployment. 2. Select **Reapply Configuration** in the **Configuration Info** panel. NGINXaaS for AWS reapplies your current configuration version and immediately refetches all referenced secrets. ## Monitor secret fetch events NGINXaaS for AWS generates an event each time it fetches or fails to fetch a secret from AWS Secrets Manager. Use these events to track successful rotations and diagnose access failures. ### Event types | Event type | Description | |---|---| | Successful Secret Fetch from AWS | The secret was fetched from AWS Secrets Manager and applied to NGINX. | | Failed Secret Fetch from AWS | NGINXaaS for AWS couldn't fetch the secret. The event message includes the error details. | ### View events in the console - Select **Overview** in the left menu, then select **Events**. To narrow results to a specific deployment, filter by its object ID using the controls at the top of the page. - For a summary of recent events for a specific deployment, select **Deployments**, select the deployment, and look for the **Recent Events** card. Select **See Events Details** to go to the full Events page pre-filtered for that deployment. ### Common failure messages and remediation | Message | Likely cause | Remediation | |---|---|---| | `operation error Secrets Manager: GetSecretValue, get identity: get credentials: failed to refresh cached credentials, operation error STS: AssumeRole...` | The IAM role's trust policy is not configured correctly. | Verify the IAM role trust policy allows `sts:AssumeRole` and `sts:TagSession` on the NGINXaaS principal. | | `AccessDeniedException... no identity-based policy allows the secretsmanager:GetSecretValue action` | The IAM role's permissions policy is not configured correctly. | Verify the IAM role has a permissions policy allowing `secretsmanager:GetSecretValue` on the secret ARN and any tag attribute conditions are met. | | `ResourceNotFoundException: Secrets Manager can't find the specified secret...` | The secret ARN doesn't exist, or the referenced version stage or version ID doesn't point to an existing version. | Confirm the secret ARN is correct and that the specified version stage or version ID is assigned to an existing version. | ## What's next [Upload an NGINX Configuration](/nginxaas/aws/deploy/nginx-configuration/nginx-configuration-console.md) -------------------------------------------------------------------------------- title: "Add certificates from Secret Manager" url: https://docs.nginx.com/nginxaas/google/deploy/ssl-tls-certificates/ssl-tls-certificates-secret-manager/index.md -------------------------------------------------------------------------------- F5 NGINXaaS for Google Cloud (NGINXaaS) can fetch secrets directly from [Secret Manager](https://docs.cloud.google.com/secret-manager/docs/overview) to use as certificates and keys in your NGINX configuration, ensuring your credentials remain securely within Google Cloud. ## Prerequisites If you haven't already done so, complete the following prerequisites: - Enable the [Secret Manager API](https://docs.cloud.google.com/secret-manager/docs/configuring-secret-manager#enable-the-secret-manager-api). - [Create an NGINXaaS deployment](/nginxaas/google/deploy/create-deployment/deploy-console.md). - Configure Workload Identity Federation (WIF). See [the documentation on setting up WIF](/nginxaas/google/deploy/access-management.md#configure-wif) for exact steps. - [Grant access to the WIF principal](/nginxaas/google/deploy/access-management.md#grant-access-to-the-wif-principal-with-your-desired-roles) with the **Secret Manager Secret Accessor** role. ## Add an SSL/TLS certificate to Secret Manager To add an SSL/TLS certificate and key as a secret to Secret Manager, 1. Make sure your certificate and key file(s) are in one of the [accepted formats](/nginxaas/google/deploy/ssl-tls-certificates/overview.md#supported-certificate-types-and-formats). 1. Follow Google's [instructions to upload your certificate and key file(s) to Secret Manager](https://docs.cloud.google.com/secret-manager/docs/creating-and-accessing-secrets#console_1). **Note:** There are many ways to manage your SSL/TLS certificates and keys. For example, you can include the PEM certificate data in the same secret as your private key. The ssl_certificate directive supports a single file containing multiple certificates and a key. See NGINX's [Configuring HTTPS servers](https://nginx.org/en/docs/http/configuring_https_servers.html) guide for more details. ## Use a Secret Manager certificate in an NGINX configuration To add your Secret Manager certificate and key to an NGINX configuration in the NGINXaaS console, 1. Select **Configurations** in the left menu. 2. Select the ellipsis (three dots) next to the configuration you want to edit, and select **Edit**. 3. Select [icon: plus] **Add File**. 4. Select **Cloud Provider Secret** as the type of file you want to add. 5. Select **Google Secret Manager** as the **Cloud Secret Manager**. 6. Provide the required information: | Field | Description | Note | |---------------------------- | ---------------------------- | ---- | | Google Secret ID | The resource name of the secret in Secret Manager | The resource name must match the format `projects/$PROJECT_ID/secrets/$SECRET_ID/versions/$VERSION`, where `$VERSION` can be a specific version ID (for example, `3`), a custom alias, or the special version ID `latest`. | | File Path | The secret will be written to this file path, so it can be used with NGINX directives such as `ssl_certificate` or `ssl_certificate_key` in your NGINX configuration. | The path must be unique within the configuration. See the [NGINX Filesystem Restrictions table](/nginxaas/google/deploy/nginx-configuration/configuration-rules.md#nginx-filesystem-restrictions) for the allowed directories the file can be written to. | **tip:** If you set `$VERSION` to `latest`, NGINXaaS automatically picks up any new secret version you add to Secret Manager without a configuration change. NGINXaaS applies new versions within four hours. See [Rotate a Secret Manager certificate (automatic)](#rotate-a-secret-manager-certificate-automatic) for details. 7. Update the NGINX configuration to reference the certificate you just added by the path value. 8. Select **Add**, **Next**, and then **Save** to save your changes. ## Update your NGINXaaS deployment's NGINX configuration Before updating your NGINXaaS deployment to use your new NGINX configuration, make sure your deployment already has a [workload identity pool provider set up](/nginxaas/google/deploy/access-management.md#configure-wif) with the **Secret Manager Secret Accessor** role granted, so it can fetch certificates. Then, in the NGINXaaS console: 1. Select **Deployments**. 1. Select the deployment you want to edit. 1. In the **Configuration Info** panel, select **Edit**. 1. Select the configuration and configuration version created in the last section. 1. Select **Update Configuration**. **Note:** Configurations with Google Secret Manager secrets can only be added to Google deployments. ## Rotate a Secret Manager certificate (automatic) If you set the version ID of your secret to `latest`, NGINXaaS fetches the latest secret version. When you [add a new secret version in Secret Manager](https://docs.cloud.google.com/secret-manager/docs/add-secret-version#add-a-secret-version), NGINXaaS automatically picks up that version within four hours. If you set the version ID of your secret to a custom alias, NGINXaaS fetches the secret version the alias points to. When you [update the alias to point to a different version in Secret Manager](https://docs.cloud.google.com/secret-manager/docs/assign-alias-to-secret-version), NGINXaaS automatically picks up that version within four hours. No configuration changes are required in either case. To confirm your deployment is using an updated certificate, check the **Certificates** list for the new serial number or inspect the certificate at your deployment's endpoint. ## Rotate a Secret Manager certificate (manual) To immediately refetch secrets without editing your NGINX configuration, use **Reapply Configuration**. This is useful in the following scenarios: - **New secret version**: You've uploaded a new certificate and want NGINXaaS to use it right away. - **WIF or permissions fix**: You've updated a WIF provider or granted Secret Manager permissions and want NGINXaaS to retry immediately. To reapply your configuration: 1. In the NGINXaaS console, go to your deployment. 2. Select **Reapply Configuration** in the **Configuration Info** panel. NGINXaaS reapplies your current configuration version and immediately refetches all referenced secrets. ## Monitor secret fetch events NGINXaaS generates an event each time it fetches or fails to fetch a secret from Secret Manager. Use these events to track successful rotations and diagnose access failures. ### Event types | Event type | Description | |---|---| | Successful Secret Fetch from Google | The secret was fetched from Secret Manager and applied to NGINX. | | Failed Secret Fetch from Google | NGINXaaS couldn't fetch the secret. The event message includes the error details. | ### View events in the console - Select **Overview** in the left menu, then select **Events**. To narrow results to a specific deployment, filter by its object ID using the controls at the top of the page. - For a summary of recent events for a specific deployment, select **Deployments**, select the deployment, and look for the **Recent Events** card. Select **See Events Details** to go to the full Events page pre-filtered for that deployment. ### Common failure messages and remediation | Message | Likely cause | Remediation | |---|---|---| | `Failed to fetch secret ... PermissionDenied: Permission 'secretmanager.versions.access' denied` | The Workload Identity Federation principal doesn't have the required IAM role on the secret. | Verify the WIF principal has the Secret Manager Secret Accessor role on the project or secret. | | `Failed to fetch secret ... NotFound: Secret [...] has no alias [latest]` | No versions exist for the referenced secret, or the specified version alias or number doesn't exist. | Confirm the secret has at least one enabled version and that the resource name in your configuration uses a valid version or alias. | ## What's next [Upload an NGINX Configuration](/nginxaas/google/deploy/nginx-configuration/nginx-configuration-console.md) -------------------------------------------------------------------------------- title: "About" url: https://docs.nginx.com/nginxaas/overview/about/index.md -------------------------------------------------------------------------------- ## What is F5 ${product}? ${product} is a service offering that is tightly integrated into your cloud platform and its ecosystem, making applications fast, efficient, and reliable with full lifecycle management of advanced NGINX traffic services. ${product} is powered by [NGINX Plus](https://www.nginx.com/products/nginx/), which extends NGINX Open Source with advanced functionality and provides customers with a complete application delivery solution. ${product} handles the NGINX Plus license management automatically. ## Capabilities The key capabilities of ${product} are: - Simplifies onboarding by providing a fully managed, ready-to-use NGINX service, eliminating the need for infrastructure setup, manual upgrades, or operational overhead. - Lowers operational overhead in running and optimizing NGINX. - Simplifies NGINX deployments with fewer moving parts (edge routing is built into the service). - Supports migration of existing NGINX configurations to the cloud with minimal effort. - Integrates with your cloud provider's ecosystem. - ${product} keeps your SSL private keys and other secrets secure and easy to manage. - Adopts a consumption-based pricing model to align infrastructure costs to actual usage. - ${product} is used to create, update, and delete NGINX configurations, certificates, and deployments. - ${product} automatically adapts to application traffic demands through autoscaling. - Each ${product} deployment has dedicated network and compute resources. There is no possibility of noisy neighbor problems or data leakage between deployments. - Makes collecting and reviewing access logs and NGINX metrics painless and accessible. - ${product} supports request tracing. See the [Application Performance Management with NGINX Variables](https://www.f5.com/company/blog/nginx/application-tracing-nginx-plus) blog to learn more about tracing. - Supports HTTP to HTTPS, HTTPS to HTTP, and HTTP to HTTP redirects. ${product} also provides the ability to create new rules for redirecting. See [How to Create NGINX Rewrite Rules](https://blog.nginx.org/blog/creating-nginx-rewrite-rules) for more details. ## Availability ${product} is available for use with multiple public cloud providers. See specific documentation for more details: - [NGINXaaS for AWS](/nginxaas/aws/overview.md) - [NGINXaaS for Google Cloud](/nginxaas/google/overview.md) -------------------------------------------------------------------------------- title: "Add certificates using the Console" url: https://docs.nginx.com/nginxaas/aws/deploy/ssl-tls-certificates/ssl-tls-certificates-console/index.md description: Add, update, and delete SSL/TLS certificates for F5 NGINXaaS for AWS using the NGINXaaS Console. -------------------------------------------------------------------------------- ## What's next [Upload an NGINX Configuration](/nginxaas/aws/deploy/nginx-configuration/nginx-configuration-console.md) -------------------------------------------------------------------------------- title: "Add certificates using the Console" url: https://docs.nginx.com/nginxaas/google/deploy/ssl-tls-certificates/ssl-tls-certificates-console/index.md -------------------------------------------------------------------------------- ## What's next [Upload an NGINX Configuration](/nginxaas/google/deploy/nginx-configuration/nginx-configuration-console.md) -------------------------------------------------------------------------------- title: "Add certificates using the NGINXaaS Console" url: https://docs.nginx.com/nginxaas/overview/ssl-tls-certificates/ssl-tls-certificates-console/index.md -------------------------------------------------------------------------------- You can manage SSL/TLS certificates for F5 ${product} using the NGINXaaS console. ## Add an SSL/TLS certificate to NGINXaaS - Select **Certificates** in the left menu. - Select [icon: plus] **Add Certificate**. - In the **Add Certificate** panel, provide the required information: | Field | Description | |---------------------------- | ---------------------------- | | Name | A unique name for the certificate. | | Type | Select the type of certificate you are adding: SSL certificate and key, or CA certificate bundle. | | Certificate Import Options | Choose how you want to import the certificate. Enter the certificate text or upload a file. | - Repeat the same steps to add as many certificates as needed. ### Use a certificate in an NGINX configuration To use a certificate in an NGINX configuration, follow these steps: - Select **Configurations** in the left menu. - Select the ellipsis (three dots) next to the configuration you want to edit, and select **Edit**. - Select **Continue** to open the configuration editor. - In your configuration, select [icon: plus] **Add File** and either choose to use an existing certificate or add a new one. - If you want to add a new certificate, select **New SSL Certificate or CA Bundle** and follow the steps mentioned in [Add an SSL/TLS certificate to NGINXaaS](#add-an-ssltls-certificate-to-nginxaas). - If you want to use an existing certificate, select **Existing SSL Certificate or CA Bundle** and use the menu to choose a certificate from the list of certificates you have already added. - Provide the required path information: | Field | Description | Note | |---------------------------- | ---------------------------- | ---- | | Certificate File Path | This path can match one or more ssl_certificate directive file arguments in your NGINX configuration. | The certificate path must be unique within the same deployment. | | Key File Path | This path can match one or more ssl_certificate_key directive file arguments in your NGINX configuration. | The key path must be unique within the same deployment. See the [NGINX Filesystem Restrictions table](/nginxaas/overview/nginx-configuration/configuration-rules.md#nginx-filesystem-restrictions) for the allowed directories the file can be written to. | - Update the NGINX configuration to reference the certificate you just added by the path value. - Select **Continue** and then **Save** to save your changes. ### Edit an SSL/TLS certificate 1. On the left menu, select **Configurations**. 1. On the list of configurations, select the ellipsis (three dots) icon next to the configuration you want to update. 1. Select **Edit**. 1. Update the "Description" field as needed and select **Next**. 1. Modify the configuration file(s) as needed. - Select a config file in the file tree to update its content. - Select **Add File** to stage new content such as certificates, configuration files, or upload non-configuration files. - Select **File Actions** to rename, move, delete, or copy files as you build out the overall configuration filesystem. - Warnings, errors, and recommendations appear in the editor as you modify your configuration content. - You can hover over an NGINX configuration directive to view helpful descriptions and usage details. 1. Select **Next** when you have completed your changes. 1. Review the changes using the "Inline" or "Side-by-side" views and select **Save**. You will see a notification confirming that the configuration was updated successfully, and a new config version will be available to apply to your NGINXaaS deployments. ### Delete an SSL/TLS certificate - Select **Certificates** in the left menu. - On the list of certificates, select the ellipsis (three dots) icon next to the certificate you want to delete. - Select **Delete**. - Confirm that you want to delete the certificate. **Note:** Deleting a TLS/SSL certificate currently used by a NGINXaaS deployment will cause an error. ## Add certificates via aux files You can upload certificate and key files as part of your NGINX configuration filesystem by selecting **Add File** > **Upload Auxiliary / Other File** in the [NGINXaaS Console configuration editor](/nginxaas/overview/nginx-configuration/nginx-configuration-console.md), then reference them with the `ssl_certificate` and `ssl_certificate_key` directives. **Note:** Storing certificates, private keys, or any sensitive data in aux files is not recommended for security reasons. Use this method only when the other certificate management approaches are not suitable. -------------------------------------------------------------------------------- title: "Billing overview" url: https://docs.nginx.com/nginxaas/aws/billing/overview/index.md description: Overview of how F5 NGINXaaS for AWS is billed through the AWS Marketplace. -------------------------------------------------------------------------------- **Note:** **NGINXaaS for AWS** is currently in **Early Access**. Contact us for information about billing. F5 NGINXaaS for AWS is purchased and billed through the AWS Marketplace. Contact us if you are interested in participating in our Early Access offering by sending an email to nginxaas-early-access@f5.com. -------------------------------------------------------------------------------- title: "Billing overview" url: https://docs.nginx.com/nginxaas/google/billing/overview/index.md -------------------------------------------------------------------------------- F5 NGINXaaS for Google Cloud is deployed into your Google Cloud subscription, where your deployment resource is visible and integrated with Google Cloud’s ecosystem. The underlying infrastructure, software maintenance, availability, and scaling are fully managed by F5, abstracting operational complexities. Billing occurs hourly and is tracked in the Google Cloud Cost Management Dashboard. ## Pricing plans F5 NGINXaaS for Google Cloud is offered on an Enterprise plan, delivering enterprise-grade performance, scalability, and security backed by a 99.95% uptime SLA. The pricing model consists of three billing components, ensuring transparent and predictable costs based on resource usage. ### Pricing components | Tier | Fixed price per hour | NCU price per hour | Data processing per GB | Google Cloud Regions | |--------|---------------------|----------------------------|-----------------------|-----------------------------------------------------------------------------------------------------------------------| | Tier 1 | $0.10 | $0.008 | $0.0096 | us-east1, us-east4, us-west1, us-west2, us-west3, us-west4, us-central1, europe-west1, europe-west4, europe-north1, asia-south2 | | Tier 2 | $0.133 | $0.0106 | $0.0127 | europe-west2, europe-west3, asia-southeast1, asia-south1 | | Tier 3 | $0.166 | $0.0132 | $0.0159 | europe-central2 | ## NGINX Capacity Unit (NCU) An NGINX Capacity Unit (NCU) quantifies the capacity for a deployment. Resources are metered hourly based on the capacity utilized, enabling customers to scale up or down dynamically. The minimum billing interval is 5 min, ensuring accurate alignment of cost and usage. A single NCU consists of: - Bandwidth – 2.2 Mbps - Connections – 3000 ## Billing examples ### Deployment with 20 NCUs processing 100 GB of data for 1 hour - Fixed price: $0.10/hour - NCU usage: 20 NCUs * $0.008/hour = $0.16/hour - Data processing: 100 GB * $0.0096/GB = $0.96 **Total cost for 1 hour: $0.10 + $0.16 + $0.96 = $1.22** ### Deployment using 30 NCUs for 2 hours and scaled to 50 NCUs for another hour, processing 200 GB of data - Fixed price: $0.10/hour * 3 hours = $0.30 - NCU usage: (30 NCUs * $0.008/hour * 2 hours) + (50 NCUs * $0.008/hour * 1 hour) = $0.88 - Data processing: 200 GB * $0.0096/GB = $1.92 **Total cost for 3 hours: $0.30 + $0.88 + $1.92 = $3.10** ## Review billing data Billing data for F5 NGINXaaS for Google Cloud is reported per deployment and can be accessed through the Google Cloud Cost Management Dashboard. Usage metrics and costs are updated hourly, allowing customers to monitor and optimize resource allocation effectively. ## Canceling your NGINXaaS for Google Cloud subscription You can unsubscribe from NGINXaaS for Google Cloud by visiting the [Google Cloud Marketplace Orders](https://console.cloud.google.com/marketplace/orders) page. Please note the following behavior when you cancel your subscription: - Upon cancelation, all active deployments will immediately transition to a suspended state. In the suspended state, deployments will no longer be operational and cannot process traffic. - While in this state, you will still have access to your deployments via the NGINXaaS Console, allowing you to view or delete them. However, it will no longer be possible to update existing deployments or create new ones. - Despite the suspension of deployments, you will retain the ability to view, edit, create, and delete configurations and SSL certificates through the console. If you decide to re-subscribe to NGINXaaS for Google Cloud after canceling your subscription, all previously suspended deployments will remain deactivated. You will need to recreate your deployments from scratch. We recommend carefully reviewing your deployments and configurations before initiating the cancelation process to avoid any unintended data loss. ## Free trial You can sign up for a free trial of NGINXaaS for Google Cloud through the [Google Cloud Marketplace](https://console.cloud.google.com/marketplace/product/f5-7626-networks-public/nginxaas-google-cloud). The free trial provides up to USD 100 in credits for a maximum of 30 days, whichever comes first, to help you explore NGINXaaS for Google Cloud and its features. During the trial, you will be billed for solution provider fees and credited for those same fees at the same time, up to USD 100. Additionally, you will still be billed for any applicable infrastructure usage charges during the trial period. At the end of the trial period, you will be charged under the Enterprise plan unless you stop or end the subscription. You may cancel the trial at any time by canceling your plan before the trial ends. -------------------------------------------------------------------------------- title: "Create or upload using the NGINXaaS Console" url: https://docs.nginx.com/nginxaas/aws/deploy/nginx-configuration/nginx-configuration-console/index.md description: Create, upload, and update NGINX configurations for F5 NGINXaaS for AWS using the NGINXaaS Console. -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- title: "Create or upload using the NGINXaaS Console" url: https://docs.nginx.com/nginxaas/google/deploy/nginx-configuration/nginx-configuration-console/index.md -------------------------------------------------------------------------------- ## What's next [Monitor your deployment](/nginxaas/google/monitoring/enable-monitoring.md) -------------------------------------------------------------------------------- title: "Deploy using the NGINXaaS Console" url: https://docs.nginx.com/nginxaas/aws/deploy/create-deployment/deploy-console/index.md description: Create, configure, and connect an F5 NGINXaaS for AWS deployment using the NGINXaaS Console. -------------------------------------------------------------------------------- ## Overview This guide explains how to deploy F5 NGINXaaS for AWS using the [AWS Management Console](https://console.aws.amazon.com) and the [NGINXaaS Console](https://console.nginxaas.net/). The deployment process involves creating a new deployment, configuring the deployment, and testing the deployment. ## Before you begin Before you can deploy NGINXaaS, follow the steps in the [Prerequisites](/nginxaas/aws/deploy/prerequisites/) topic to subscribe to the NGINXaaS for AWS offering in the AWS Marketplace. ## Access the NGINXaaS Console - Open the [NGINXaaS Console](https://console.nginxaas.net/). - Log in with your preferred identity provider. - Select the Geography you want to work in. ## Create or import an NGINX configuration In the NGINXaaS Console, 1. On the left menu, select **Configurations**. 1. Select **Add Configuration** to add a new NGINX configuration. 1. You can either create a new configuration from scratch or copy an existing configuration: - Select **New configuration** to create a new config. - Provide a name for your configuration and an optional description. - Select to start with the "F5 NGINXaaS Default" (a basic NGINX setup) or an empty configuration file. - Select **Next**. - Select **Copy existing configuration** to use one of the existing configuration files in your account as template. - Provide a name for your configuration and an optional description. - Use the **Choose configuration to copy** list to select the configuration file you want to copy. - Use the **Choose configuration version to copy** list to select the version of the configuration file you want to copy. - Select **Next**. 1. Modify the configuration file as needed. - Select **Add File** to stage new content such as certificates, configuration files, or upload non-configuration files. - Select **File Actions** to rename, move, delete, or copy files as you build out the overall configuration filesystem. - Warnings, errors, and recommendations appear in the editor as you modify your configuration content. - You can hover over an NGINX configuration directive to view helpful descriptions and usage details. 1. Select **Save**. ## Create a new deployment Create a new NGINXaaS deployment using the NGINXaaS Console: 1. On the left menu, select **Deployments**. 1. Select [icon: plus] **Add Deployment**, then select **AWS** to create a new AWS deployment. - Enter a unique **Name**. - Add an optional description for your deployment. - Change the [**NCUs**](/nginxaas/aws/overview.md#nginx-capacity-unit-ncu) if needed. - The default value of `20` works for most common scenarios. - Enable **WAF** if you want [F5 WAF for NGINX](/waf) enabled for your deployment. - Select the AWS **Region** where you want the NGINXaaS deployment to be created. - Enter an **IPv4 CIDR Block** for the deployment's private network IP space. - NGINXaaS only accepts block sizes between `/24` and `/18`. - For more information on choosing a VPC CIDR block, refer to AWS's [VPC CIDR blocks](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-cidr-blocks.html) documentation. **Note:** NGINXaaS uses VPC peering for network connectivity to your upstream services. VPC peering requires all peered VPCs to have non-overlapping CIDR blocks so unique routing rules can be defined for each IP range. When selecting an IPv4 CIDR block for your AWS deployment, make sure the range doesn't match or overlap with any VPCs you plan to peer to the deployment's network. - In the **Apply NGINX Configuration** section, select an NGINX configuration [you created earlier](#create-or-import-an-nginx-configuration) from the **Choose Configuration** list. - Select a **Configuration Version** from the list. - Select **Managed Public Endpoint** or **Private Endpoint** under Service Frontend. - Refer to the [Service Frontend](/nginxaas/aws/overview.md#service-frontend) documentation for more information on these two frontend types. - If you selected **Managed Public Endpoint**, select **+ Add ACL Rule** to add an **Access Control List (ACL)** rule. If you don't add any ACL rules, no traffic is allowed to the deployment. For each rule, set: - **Protocol**: `Any`, `TCP`, or `UDP`. - **Port Range**: Only selectable when you choose a specific protocol. - **Source Prefixes**: A required list of IPv4 or IPv6 CIDR blocks to allow traffic from. Use `0.0.0.0/0` to allow traffic from all IPv4 addresses, and `::/0` to allow traffic from all IPv6 addresses. - **Description**: An optional description for the rule. - If you selected **Private Endpoint**, select **+ Add Entry** to add an entry to the **PrivateLink Connection Allow List**. If you don't add any entries, no PrivateLink connections are allowed. - Populate the allow list with either AWS account IDs or VPC endpoint IDs -- you can't mix both types in the same allow list: - **AWS account IDs**: You can add these now, or at any point later. Connections from an allowed AWS account are automatically accepted. - **VPC endpoint IDs**: After you create the deployment and its [interface VPC endpoint](#private-endpoint-traffic), add these entries to this allow list. This provides a stricter way to accept secure PrivateLink connections than trusting entire AWS account IDs. 1. Select **Add Deployment** to create the deployment. Your new deployment will appear in the list of deployments. The status of the deployment is "Pending" while the deployment is being created. Once the deployment is complete, the status changes to "Ready". ## Edit your deployment In the NGINXaaS Console, 1. Open the details of your deployment by selecting its name from the list of deployments. - You can view the details of your deployment, including the status, region, VPC endpoint service, NGINX configuration, and more. 1. Select **Edit** to modify the deployment. - From this form, you can edit the description or reserved [NCUs](/nginxaas/aws/overview.md#nginx-capacity-unit-ncu). - You can apply a different NGINX configuration or configuration version. - Enable or disable WAF. - Configure upstream connectivity by adding entries to the **VPC Peering Connections** list as detailed further in [Upstream connectivity](#upstream-connectivity). - You can also modify the **Service Frontend** configuration, including frontend type, ACL rules (Managed Public Endpoint) or PrivateLink Connection Allow List entries (Private Endpoint) from here. - This is where you can add VPC endpoint IDs to the allow list after you [create an interface VPC endpoint](#private-endpoint-traffic). - You can configure monitoring here. For detailed instructions, see [Enable Monitoring](/nginxaas/aws/monitoring/enable-monitoring.md) 1. Select **Save Changes**. To modify the contents of the NGINX configuration, see [Update an NGINX Configuration](/nginxaas/aws/deploy/nginx-configuration/nginx-configuration-console.md#update-an-nginx-configuration). ## Set up connectivity ### Private Endpoint traffic If you selected **Private Endpoint** as the service frontend type, complete the following steps to allow client access. If you selected **Managed Public Endpoint**, skip this section. To let clients in your AWS VPC connect directly to your deployment, create an interface VPC endpoint that targets your deployment's PrivateLink Endpoint Service Name: **Note:** NGINXaaS doesn't currently support cross-region PrivateLink connections. You can only create an interface VPC endpoint in the same region as your deployment. 1. After your deployment is created, open its Details tab and find the **PrivateLink Endpoint Service Name** under **Cloud Settings** > **Service Frontend**, for example `com.amazonaws.vpce.us-east-1.vpce-svc-0c0d939ca9a7ce020`. 1. Create an interface VPC endpoint that targets this Service Name. For step-by-step instructions, see AWS's [Connect to an endpoint service as the service consumer](https://docs.aws.amazon.com/vpc/latest/privatelink/create-endpoint-service.html#connect-to-endpoint-service) documentation. When prompted for **Service name**, enter the PrivateLink Endpoint Service Name from the previous step. 1. Note the new interface endpoint's **VPC Endpoint Id**, for example `vpce-0123456789abcdef0`, shown in the AWS VPC console **Endpoints** list. 1. Ensure your deployment's **PrivateLink Connection Allow List** includes the AWS account ID or VPC endpoint ID to accept the PrivateLink connection. - To add an entry to the allow list, go to your deployment's Details tab, select **Edit**, and add the VPC endpoint ID or AWS account ID to the allow list. 1. Select **Save Changes** to allow NGINXaaS to accept the PrivateLink connection request. - The VPC endpoint ID under **Cloud Settings** > **Service Frontend** > **PrivateLink Connection Allow List** changes to an accepted state. This confirms the deployment accepted the connection. **Note:** The allow list can contain either AWS account IDs or VPC endpoint IDs, but not both. If it uses AWS account IDs, NGINXaaS automatically accepts every PrivateLink connection from those accounts. If it uses VPC endpoint IDs, NGINXaaS accepts each endpoint's PrivateLink connection only after you add its ID to the list. Removing an entry disconnects that PrivateLink connection from the deployment. ### Upstream connectivity To let your NGINXaaS deployment reach applications in your upstream network, create a VPC peering connection between your upstream VPC and the deployment's VPC: **Note:** NGINXaaS doesn't currently support cross-region VPC peering connections. A peering connection from any region other than the deployment's region will be rejected. 1. Open your deployment's Details tab and note its **AWS Account ID** and **VPC ID**. 1. From your upstream AWS account, create a VPC peering connection request targeting the deployment's AWS Account ID and VPC ID. For step-by-step instructions, see AWS's [Create a VPC peering connection](https://docs.aws.amazon.com/vpc/latest/peering/create-vpc-peering-connection.html) documentation. 1. Note the resulting **VPC Peering Connection ID**, for example `pcx-0123456789abcdef0`, shown in the AWS VPC console **Peering Connections** list. 1. On your deployment's Details tab, select **Edit**, go to **Cloud Details** > **Upstream Network**, select **+ Add Entry**, and add the VPC Peering Connection ID. 1. Select **Save Changes** to allow NGINXaaS to accept the peering connection request. - The VPC peering connection ID under **Cloud Settings** > **Upstream Network** changes to an accepted state. This confirms the deployment accepted the connection. 1. You must update your upstream VPC's route tables, network ACLs, and security groups to allow traffic to and from the deployment's VPC CIDRs. - Open your deployment's Details tab and note its **IPv4 CIDR** and **IPv6 CIDR** (if you plan to use IPv6). - See AWS's [Update your route tables for a VPC peering connection](https://docs.aws.amazon.com/vpc/latest/peering/vpc-peering-routing.html) and [Configure security group rules for your VPC](https://docs.aws.amazon.com/vpc/latest/userguide/working-with-security-group-rules.html) documentation. **Note:** Your upstream VPC CIDRs must not overlap with the deployment's VPC CIDRs, or the CIDRs of other peered upstream VPCs. If CIDRs overlap, VPC peering will fail. See [Upstream network](/nginxaas/aws/overview.md#upstream-network) for more information. ## Test your deployment How you test your deployment depends on the service frontend type you selected: - **Managed Public Endpoint**: connect to the **Service Endpoint** URL shown on your deployment's Details tab, under **Cloud Settings** > **Service Frontend**, for example `http://speak-lp-9e9707dede.us-east-1.depl.nginxaas.net`. - **Private Endpoint**: connect through the interface VPC endpoint you created in [Private Endpoint traffic](#private-endpoint-traffic). Because the endpoint is only reachable from within your VPC, you need a resource inside that VPC to test the connection: 1. Launch an EC2 instance in the same VPC and subnet (or a subnet that can route to it) as your interface VPC endpoint. 1. Open the interface endpoint's details in the AWS VPC console and note its **DNS names**. 1. From the EC2 instance, connect to your NGINX configuration's listening port using one of the endpoint's DNS names, for example `curl https://vpce-0123456789abcdef0-abc12345.vpce-svc-0c0d939ca9a7ce020.us-east-1.vpce.amazonaws.com`. ## What's next - [Monitor your deployment](/nginxaas/aws/monitoring/enable-monitoring.md) - [Manage certificates in AWS Secrets Manager](/nginxaas/aws/deploy/ssl-tls-certificates/ssl-tls-certificates-secrets-manager.md) -------------------------------------------------------------------------------- title: "Deploy using the NGINXaaS Console" url: https://docs.nginx.com/nginxaas/google/deploy/create-deployment/deploy-console/index.md description: Create, configure, and connect an F5 NGINXaaS for Google Cloud deployment using the NGINXaaS Console. -------------------------------------------------------------------------------- ## Overview This guide explains how to deploy F5 NGINXaaS for Google Cloud (NGINXaaS) using [Google Cloud Console](https://console.cloud.google.com) and the NGINXaaS Console. The deployment process involves creating a new deployment, configuring the deployment, and testing the deployment. ## Before you begin Before you can deploy NGINXaaS, follow the steps in the [Prerequisites](/nginxaas/google/deploy/prerequisites/) topic to subscribe to the NGINXaaS for Google Cloud offering in the Google Cloud Marketplace. ## Access the NGINXaaS Console Once you have completed the subscription process, you can access the NGINXaaS Console. - Open the [NGINXaaS Console](https://console.nginxaas.net/). - Log in with your preferred identity provider. - Select the Geography you want to work in. **Note:** Select the Geography that supports the region where you plan to deploy. See [NGINXaaS for Google Cloud supported regions](/nginxaas/google/overview.md#supported-regions). ## Create or import an NGINX configuration In the NGINXaaS Console, 1. On the left menu, select **Configurations**. 1. Select **Add Configuration** to add a new NGINX configuration. 1. You can either create a new configuration from scratch or copy an existing configuration: - Select **New configuration** to create a new config. - Provide a name for your configuration and an optional description. - Select to start with the "F5 NGINXaaS Default" (a basic NGINX setup) or an empty configuration file. - Select **Next**. - Select **Copy existing configuration** to use one of the existing configuration files in your account as template. - Provide a name for your configuration and an optional description. - Use the **Choose configuration to copy** list to select the configuration file you want to copy. - Use the **Choose configuration version to copy** list to select the version of the configuration file you want to copy. - Select **Next**. 1. Modify the configuration file as needed. - Select **Add File** to stage new content such as certificates, configuration files, or upload non-configuration files. - Select **File Actions** to rename, move, delete, or copy files as you build out the overall configuration filesystem. - Warnings, errors, and recommendations appear in the editor as you modify your configuration content. - You can hover over an NGINX configuration directive to view helpful descriptions and usage details. 1. Select **Save**. ## Create a new deployment Next, create a new NGINXaaS deployment using the NGINXaaS Console: 1. On the left menu, select **Deployments**. 1. Select [icon: plus] **Add Deployment** to create a new deployment. - Enter a **Name**. - Add an optional description for your deployment. - Change the **NCU Capacity** if needed. - The default value of `20 NCU` should be adequate for most scenarios. - Enable **WAF** if you want [F5 WAF for NGINX](/waf) enabled for your deployment. - In the Apply Configuration section, select an NGINX configuration [you created earlier](#create-or-import-an-nginx-configuration) from the **Choose Configuration** list. - Select a **Configuration Version** from the list. - In the Cloud Details section, select a **Region**. - Make a note of this region — you'll need to create a network attachment in the same region. - Optional: In the Cloud Details section, enter a network attachment ID or select one in the **Network attachment** list. A network attachment connects this deployment to your upstream servers, and is required for most use cases. - The network attachment ID is formatted like the following example: `projects/my-google-project/regions/us-east1/networkAttachments/my-network-attachment`. - If you haven't created a network attachment yet, you can leave this field empty and [add one after creating your deployment](#create-a-network-attachment). - Select **Managed Public Endpoint** or **Private Endpoint** under Service Frontend. - Refer to the [Service Frontend](/nginxaas/google/overview.md#service-frontend) documentation for more information on these two frontend types. - Select **Submit** to begin the deployment process. Your new deployment will appear in the list of deployments. The status of the deployment will be "Pending" while the deployment is being created. Once the deployment is complete, the status will change to "Ready". Make a note of the deployment's **Region** and **Project ID**, both visible under the `Cloud Info` section for your deployment. You'll need them in the next section to create and connect a network attachment. ### Create a network attachment To connect your NGINXaaS deployment to your upstream servers, create a [network attachment](https://cloud.google.com/vpc/docs/about-network-attachments) and add it to your deployment. **Note:** NGINXaaS for Google Cloud is supported in the following regions per geography: |NGINXaaS Geography | Google Cloud Regions | |-----------|---------| | US | us-east1, us-east4, us-west1, us-west2, us-west3, us-west4, us-central1 | | EU | europe-west1, europe-west2, europe-west3, europe-west4, europe-north1, europe-central2 | | APAC | asia-southeast1, asia-south1, asia-south2, asia-northeast1, asia-northeast2, asia-northeast3 | | CA | northamerica-northeast1, northamerica-northeast2 | 1. Access the [Google Cloud Console](https://console.cloud.google.com/). 1. Create a consumer VPC network and subnetwork. See [Google's documentation on creating a VPC and subnet](https://cloud.google.com/vpc/docs/create-modify-vpc-networks#console_1) for a step-by-step guide. **Note:** Create the network attachment in the same region as your deployment. 1. Create a network attachment in your new subnet. See [Google's documentation on creating a network attachment](https://cloud.google.com/vpc/docs/create-manage-network-attachments#create-network-attachments) for a step-by-step guide. Configure the **Connection preference** on the Network Attachment resource using one of the following options: - **Accept connections from selected projects** (recommended): This option helps maintain security by ensuring only trusted providers can connect to your service by letting you manually approve trusted connections. Add your NGINXaaS deployment's **Project ID** to the accept list to connect it to your upstream servers. - **Automatically accept connections for all projects**: This option lets you skip adding your deployment's Project ID to an accept list. **Note:** For development and testing purposes, or in scenarios where speed and simplicity are prioritized over security, you have the option to configure the **Connection Preference** to **Automatically accept connections for all projects**. Please note that this approach is inherently less secure and may expose your service to unintended or unauthorized access. We encourage you to exercise caution if using the less restrictive option and to avoid using it in production or sensitive environments. 1. Copy the network attachment value. You can find it in the Google Cloud Console by following the steps below: 1. Go to Network Attachments at the following link: https://console.cloud.google.com/net-services/psc/list/networkAttachments?project=my-google-project (replace `my-google-project` in the URL with your project name). 1. Open the desired network attachment and copy the value from the `Network Attachment` field. **Example format:** `projects/my-google-project/regions/us-east1/networkAttachments/my-network-attachment`. 1. Add the network attachment value to your deployment, either while [creating the deployment](#create-a-new-deployment) or by [editing an existing deployment](#configure-your-deployment). This connects your NGINXaaS deployment to your upstream servers. You can add or remove a network attachment after creating a deployment by [editing the deployment](#configure-your-deployment). **Note:** You cannot switch a deployment directly from one network attachment to another. To change the network attachment, first remove the existing one, wait for the deployment to return to a `Ready` state, then add the new network attachment. This operation involves service downtime as the network attachment is removed from the deployment. **Note:** Deployments created before Sept 8, 2026 do not support updating the network attachment. ## Configure your deployment In the NGINXaaS Console, 1. To open the details of your deployment, select its name from the list of deployments. - You can view the details of your deployment, including the status, region, network attachment, NGINX configuration, and more. 1. Select **Edit** to modify the deployment description, NCU Capacity, and WAF enablement. - You can also configure monitoring from here. Detailed instructions can be found in [Enable Monitoring](/nginxaas/google/monitoring/enable-monitoring.md) - You can add or remove a network attachment from the Cloud Details section. See [Create a network attachment](#create-a-network-attachment) for details on switching from one network attachment to another. 1. Select **Update** to save your changes. 1. Select the Configuration tab to view the current NGINX configuration associated with the deployment. 1. Select **Update Configuration** to change the NGINX configuration associated with the deployment. 1. To modify the contents of the NGINX configuration, see [Update an NGINX Configuration](/nginxaas/google/deploy/nginx-configuration/nginx-configuration-console.md#update-an-nginx-configuration). ## Set up connectivity (Private Endpoint only) If you selected **Private Endpoint** as the service frontend type, complete the following steps to allow client access. If you selected **Managed Public Endpoint**, skip this section. ### Internal traffic To set up private connectivity to your NGINXaaS deployment, create a [Private Service Connect (PSC) endpoint](https://docs.cloud.google.com/vpc/docs/configure-private-service-connect-services) in the same VPC as your internal clients. 1. Go to the [Google Cloud Console](https://console.cloud.google.com/) and select the project where you want to create networking resources for your F5 NGINXaaS deployment. 1. Create or reuse a [VPC network](https://cloud.google.com/vpc/docs/create-modify-vpc-networks). 1. Create a PSC endpoint. See [Google's documentation on creating an endpoint](https://docs.cloud.google.com/vpc/docs/configure-private-service-connect-services#create-endpoint) for a step-by-step guide. - For **Target service**, enter your NGINXaaS deployment's Service Attachment, which is visible on the `Deployment Details` section for your deployment. ### External traffic To set up public connectivity for external clients, configure a [Private Service Connect (PSC) backend](https://cloud.google.com/vpc/docs/private-service-connect-backends) for your NGINXaaS deployment. 1. Go to the [Google Cloud Console](https://console.cloud.google.com/) and select the project where you want to create networking resources for your F5 NGINXaaS deployment. 1. Create or reuse a [VPC network](https://cloud.google.com/vpc/docs/create-modify-vpc-networks). 1. Create a proxy-only subnet in your consumer VPC. See [Google's documentation on creating a proxy-only subnet](https://cloud.google.com/load-balancing/docs/tcp/set-up-ext-reg-tcp-proxy-zonal#console_1) for a step-by-step guide. 1. Create a public IP address. See [Google's documentation on reserving a static address](https://cloud.google.com/load-balancing/docs/tcp/set-up-ext-reg-tcp-proxy-zonal#console_3) for a step-by-step guide. 1. Create a Private Service Connect Network Endpoint Group (PSC NEG). See [Google's documentation on creating a NEG](https://cloud.google.com/vpc/docs/access-apis-managed-services-private-service-connect-backends#console) for a step-by-step guide. - Set **Network endpoint group type** to **Private Service Connect NEG (Regional)**. - Set **Target** to **Published service**. - For **Target service**, enter your NGINXaaS deployment's Service Attachment, which is visible on the `Deployment Details` section for your deployment. - For **Producer port**, enter the port your NGINX server is listening on. If you're using the default NGINX config, enter port `80`. - For **Network** and **Subnetwork** select your consumer VPC network and subnet. 1. Create a regional external proxy Network Load Balancer. See [Google's documentation on configuring the load balancer](https://cloud.google.com/load-balancing/docs/tcp/set-up-ext-reg-tcp-proxy-zonal#console_6) for a step-by-step guide. - For **Network**, select your consumer VPC network. - For **Backend configuration**, follow [Google's step-by-step guide to add a backend](https://cloud.google.com/vpc/docs/access-apis-managed-services-private-service-connect-backends#console_5). - In the **Frontend configuration** section, - For **IP address**, select the public IP address created earlier. - For **Port number**, enter the same port as your NEG's Producer port, for example, port `80`. Each listening port configured on NGINX requires its own PSC network endpoint group with a matching port. You can use the following helper script to automate these steps: #### Show helper script ```bash #!/bin/bash set -euo pipefail # Default values PROJECT="" REGION="" NETWORK="" SUBNET="" SA_URI="" PORTS="80" PROXY_SUBNET="psc-proxy-subnet" VIPNAME="psc-vip" # Prerequisites: # - gcloud CLI installed and configured # - An existing projectID and a VPC network created in that project # - A valid Service Attachment URI from F5 NGINXaaS # Function to display usage usage() { cat << EOF Usage: $0 --project PROJECT --region REGION --network NETWORK --subnet SUBNET --service-attachment SA_URI [--ports PORTS] Options: --project GCP Project ID --region GCP Region --network VPC Network name --subnet GCP Subnet for Backend Connectivity (must be in the same region and network) --service-attachment Service Attachment Self Link --ports Comma-separated list of ports (default: 80) --help Show this help message Note: Proxy subnet and public IP will be automatically created as 'psc-proxy-subnet' and 'psc-vip' respectively. These resources will not be deleted, if deleted this script will create new ones. Example: $0 --project my-project --region us-central1 --network my-vpc --subnet my-subnet \\ --service-attachment "projects/producer-proj/regions/us-central1/serviceAttachments/my-service" \\ --ports "80,443,8080" EOF } # Parse command line arguments while [[ $# -gt 0 ]]; do case $1 in --project) PROJECT="$2" shift 2 ;; --region) REGION="$2" shift 2 ;; --network) NETWORK="$2" shift 2 ;; --service-attachment) SA_URI="$2" shift 2 ;; --ports) PORTS="$2" shift 2 ;; --subnet) SUBNET="$2" shift 2 ;; --help|-h) usage exit 0 ;; *) echo "Unknown option: $1" usage exit 1 ;; esac done # Validate required parameters missing_params=() [[ -z "$PROJECT" ]] && missing_params+=("--project") [[ -z "$REGION" ]] && missing_params+=("--region") [[ -z "$NETWORK" ]] && missing_params+=("--network") [[ -z "$SUBNET" ]] && missing_params+=("--subnet") [[ -z "$SA_URI" ]] && missing_params+=("--service-attachment") if [[ ${#missing_params[@]} -gt 0 ]]; then echo "Error: Missing required parameters: ${missing_params[*]}" usage exit 1 fi # Create proxy-only subnet (skip if exists) echo "Creating proxy-only subnet if it doesn't already exist..." if ! gcloud compute networks subnets describe $PROXY_SUBNET --region=$REGION --project=$PROJECT >/dev/null 2>&1; then gcloud compute networks subnets create $PROXY_SUBNET \ --project=$PROJECT --region=$REGION \ --network=$NETWORK \ --range=192.168.1.0/24 \ --purpose=REGIONAL_MANAGED_PROXY \ --role=ACTIVE fi echo "Using proxy-only subnet: $PROXY_SUBNET" # Create regional VIP address (skip if exists) echo "Creating regional VIP address..." if ! gcloud compute addresses describe $VIPNAME --region=$REGION --project=$PROJECT >/dev/null 2>&1; then gcloud compute addresses create $VIPNAME --region=$REGION --project=$PROJECT fi VIP=$(gcloud compute addresses describe $VIPNAME --region=$REGION --project=$PROJECT --format='get(address)') echo "Using VIP address: $VIP" # Convert comma-separated ports to array IFS=',' read -ra PORTS_ARRAY <<< "$PORTS" for P in "${PORTS_ARRAY[@]}"; do echo "Processing port $P..." # Create Network Endpoint Group (skip if exists) if ! gcloud compute network-endpoint-groups describe psc-neg-$P --region=$REGION --project=$PROJECT >/dev/null 2>&1; then gcloud compute network-endpoint-groups create psc-neg-$P \ --project=$PROJECT --region=$REGION \ --network-endpoint-type=private-service-connect \ --psc-target-service="$SA_URI" \ --network=$NETWORK \ --subnet=$SUBNET \ --producer-port=$P fi # Create Backend Service (skip if exists) - NO HEALTH CHECKS for PSC if ! gcloud compute backend-services describe be-$P --region=$REGION --project=$PROJECT >/dev/null 2>&1; then gcloud compute backend-services create be-$P \ --project=$PROJECT --region=$REGION \ --protocol=TCP --load-balancing-scheme=EXTERNAL_MANAGED # Add backend to service gcloud compute backend-services add-backend be-$P \ --project=$PROJECT --region=$REGION \ --network-endpoint-group=psc-neg-$P \ --network-endpoint-group-region=$REGION fi # Create Target TCP Proxy (skip if exists) if ! gcloud compute target-tcp-proxies describe tp-$P --region=$REGION --project=$PROJECT >/dev/null 2>&1; then gcloud compute target-tcp-proxies create tp-$P \ --project=$PROJECT --region=$REGION --backend-service=be-$P fi # Create Forwarding Rule (skip if exists) if ! gcloud compute forwarding-rules describe fr-$P --region=$REGION --project=$PROJECT >/dev/null 2>&1; then gcloud compute forwarding-rules create fr-$P \ --project=$PROJECT --region=$REGION \ --address=$VIP --network=$NETWORK \ --target-tcp-proxy=tp-$P --target-tcp-proxy-region=$REGION \ --ports=$P --load-balancing-scheme=EXTERNAL_MANAGED \ --network-tier=PREMIUM --ip-protocol=TCP fi echo "Completed setup for port $P" done echo "Setup complete! Public Virtual IP: $VIP" ``` ## Test your deployment 1. To test your deployment, connect to the IP address created in [Set up connectivity](/nginxaas/google/deploy/create-deployment/deploy-console.md#set-up-connectivity-private-endpoint-only) or the service endpoint created with your managed public endpoint deployment. **Note:** The deployment is privately deployed in your subnet. If you want to route traffic to an application over the public internet, consider setting up [Cloud NAT](https://docs.cloud.google.com/nat/docs/overview). ## What's next - [Monitor your deployment](/nginxaas/google/monitoring/enable-monitoring.md) - [Manage certificates in Google Secret Manager](/nginxaas/google/deploy/ssl-tls-certificates/ssl-tls-certificates-secret-manager.md) -------------------------------------------------------------------------------- title: "Deploy using the Terraform provider" url: https://docs.nginx.com/nginxaas/google/deploy/create-deployment/deploy-terraform/index.md description: Create, and configure an F5 NGINXaaS for Google Cloud deployment using the Terraform provider. -------------------------------------------------------------------------------- ## Overview This guide explains how to create an F5 NGINXaaS for Google Cloud (NGINXaaS) deployment using the [`F5Networks/f5ads` Terraform provider](https://registry.terraform.io/providers/F5Networks/f5ads/latest/docs). This guide only covers creating a deployment. ## Prerequisites - Confirm that you meet the [NGINXaaS Prerequisites](/nginxaas/google/deploy/prerequisites.md). - Access to a Google Cloud project to manage your network resources. - Access to the F5 NGINXaaS console to create client credentials. See [Programmatic authentication with client credentials](/nginxaas/overview/client-credentials.md). - [Install Terraform](https://learn.hashicorp.com/tutorials/terraform/install). **Note:** The `F5Networks/f5ads` provider is in alpha (`0.1.0-alpha.1`). Pin this exact version in your `required_providers` block, since breaking changes can occur between alpha releases. ## Configure the provider Add the `f5ads` provider to your Terraform configuration and pin it to the exact alpha version: ```hcl terraform { required_providers { f5ads = { source = "F5Networks/f5ads" version = "0.1.0-alpha.1" } } } provider "f5ads" { geo = "us" } ``` The provider needs your F5 ADS client credentials to authenticate. You can supply them in the provider block, or with environment variables: ```hcl provider "f5ads" { geo = "us" client_id = "your-client-id" client_secret = "your-client-secret" } ``` ```shell export F5ADS_GEO="us" export F5ADS_CLIENT_ID="your-client-id" export F5ADS_CLIENT_SECRET="your-client-secret" ``` ## Create GCP network resources Include the following snippet if you need to create a VPC, subnet, and [network attachment](https://cloud.google.com/vpc/docs/about-network-attachments) for your deployment. Refer to the [`hashicorp/google` provider](https://registry.terraform.io/providers/hashicorp/google/latest/docs) for instructions on authentication and setting up your Google Cloud project. ```hcl provider "google" { region = "us-east1" } resource "google_compute_network" "vpc" { name = "my-vpc" auto_create_subnetworks = false } resource "google_compute_subnetwork" "subnet" { name = "my-subnet" ip_cidr_range = "10.0.0.0/24" region = "us-east1" network = google_compute_network.vpc.id } resource "google_compute_network_attachment" "attachment" { name = "my-network-attachment" region = "us-east1" connection_preference = "ACCEPT_AUTOMATIC" subnetworks = [google_compute_subnetwork.subnet.id] } ``` **Note:** The `f5ads` provider currently only supports network attachments with `connection_preference` set to `ACCEPT_AUTOMATIC`, which accepts connections from all projects. Support for `ACCEPT_MANUAL` is planned for an upcoming release. See [Google's documentation on creating a network attachment](https://cloud.google.com/vpc/docs/create-manage-network-attachments#create-network-attachments) for details. ## Create a deployment **Note:** Create or reuse an existing configuration in the [F5 ADS Console](/nginxaas/google/deploy/nginx-configuration/nginx-configuration-console.md) first, then supply its [Configuration ID and Configuration Version ID](/nginxaas/google/deploy/nginx-configuration/nginx-configuration-console.md#get-nginx-configuration-version-information) below. Choose a frontend type for your deployment. Refer to [Service frontend](/nginxaas/google/overview.md#service-frontend) for more information on these two frontend types. To use a managed public endpoint: ```hcl resource "f5ads_deployment" "example" { name = "my-deployment" capacity = 20 waf_enabled = true nginx_config_id = "cfg_example123" nginx_config_version_id = "cv_example456" google_cloud_properties = { region = "us-east1" network_attachment = google_compute_network_attachment.attachment.id frontend = { managed_public_endpoint = { acl = [ { source_prefixes = ["0.0.0.0/0"] port_range = "80" protocol = "tcp" }, { source_prefixes = ["0.0.0.0/0"] port_range = "443" protocol = "tcp" }, ] } } } } ``` To use a private endpoint instead, replace the `managed_public_endpoint` block with a `private_endpoint` block: ```hcl frontend = { private_endpoint = { service_attachment_accept_list = [ "my-gcp-project", ] } } ``` Create the deployment: ```shell terraform init terraform plan terraform apply ``` -------------------------------------------------------------------------------- title: "Enable F5 WAF for NGINX" url: https://docs.nginx.com/nginxaas/overview/app-protect/enable-waf/index.md description: Enable F5 WAF for NGINX on an NGINXaaS deployment using the NGINXaaS Console. -------------------------------------------------------------------------------- ## Overview This guide explains how to enable F5 WAF for NGINX on a F5 NGINX as a Service (NGINXaaS) deployment. [F5 WAF for NGINX](https://docs.nginx.com/nginx-app-protect-waf/v5) provides web application firewall (WAF) security protection for your web applications, including OWASP Top 10; response inspection; Meta characters check; HTTP protocol compliance; evasion techniques; disallowed file types; JSON & XML well-formedness; sensitive parameters & Data Guard. ## Enable F5 WAF for NGINX F5 WAF for NGINX is disabled by default and needs to be explicitly enabled on an NGINXaaS deployment. Follow these steps: ### Using the NGINXaaS Console - Open the [NGINXaaS Console](https://console.nginxaas.net/). - Log in with your preferred identity provider. - Select the Geography you want to work in. 1. Go to your NGINXaaS deployment. 2. Edit your deployment. 3. Enable **WAF** for your deployment. 4. Select **Save Changes** to begin the deployment process. ## What's next [Configure F5 WAF for NGINX](/nginxaas/overview/app-protect/configure-waf.md) -------------------------------------------------------------------------------- title: "Manage configs using the NGINXaaS Console" url: https://docs.nginx.com/nginxaas/overview/nginx-configuration/nginx-configuration-console/index.md -------------------------------------------------------------------------------- You can apply an NGINX configuration to your F5 ${product} deployment using the NGINXaaS Console. ## Access the NGINXaaS Console - Open the [NGINXaaS Console](https://console.nginxaas.net/). - Log in with your preferred identity provider. - Select the Geography you want to work in. ## Create or import an NGINX configuration In the NGINXaaS Console, 1. On the left menu, select **Configurations**. 1. Select **Add Configuration** to add a new NGINX configuration. 1. You can either create a new configuration from scratch or copy an existing configuration: - Select **New configuration** to create a new config. - Provide a name for your configuration and an optional description. - Select to start with the "F5 NGINXaaS Default" (a basic NGINX setup) or an empty configuration file. - Select **Next**. - Select **Copy existing configuration** to use one of the existing configuration files in your account as template. - Provide a name for your configuration and an optional description. - Use the **Choose configuration to copy** list to select the configuration file you want to copy. - Use the **Choose configuration version to copy** list to select the version of the configuration file you want to copy. - Select **Next**. 1. Modify the configuration file as needed. - Select **Add File** to stage new content such as certificates, configuration files, or upload non-configuration files. - Select **File Actions** to rename, move, delete, or copy files as you build out the overall configuration filesystem. - Warnings, errors, and recommendations appear in the editor as you modify your configuration content. - You can hover over an NGINX configuration directive to view helpful descriptions and usage details. 1. Select **Save**. ## Update an NGINX configuration 1. On the left menu, select **Configurations**. 1. On the list of configurations, select the ellipsis (three dots) icon next to the configuration you want to update. 1. Select **Edit**. 1. Update the "Description" field as needed and select **Next**. 1. Modify the configuration file(s) as needed. - Select a config file in the file tree to update its content. - Select **Add File** to stage new content such as certificates, configuration files, or upload non-configuration files. - Select **File Actions** to rename, move, delete, or copy files as you build out the overall configuration filesystem. - Warnings, errors, and recommendations appear in the editor as you modify your configuration content. - You can hover over an NGINX configuration directive to view helpful descriptions and usage details. 1. Select **Next** when you have completed your changes. 1. Review the changes using the "Inline" or "Side-by-side" views and select **Save**. You will see a notification confirming that the configuration was updated successfully, and a new config version will be available to apply to your NGINXaaS deployments. ## Get NGINX Configuration version information In the NGINXaaS Console, 1. On the left menu, select **Configurations**. 1. Select your desired configuration. 2. Select **Details**. 3. Note the NGINX Configuration **Object ID** and **Latest Version ID** for your reference. ## Delete NGINX configuration files 1. On the left menu, select **Configurations**. 1. On the list of configurations, select the ellipsis (three dots) icon next to the configuration you want to delete. 1. Select **Delete**. 1. Confirm that you want to delete the configuration. -------------------------------------------------------------------------------- title: "Overview and architecture" url: https://docs.nginx.com/nginxaas/aws/overview/index.md description: Overview of F5 NGINXaaS for AWS architecture, capabilities, and how it integrates with your AWS environment. -------------------------------------------------------------------------------- ## What is NGINXaaS for AWS? F5 NGINXaaS for AWS is a SaaS offering that is tightly integrated into AWS and its ecosystem of services, making applications fast, efficient, and reliable. It brings advanced traffic management capabilities from the commercial version of NGINX, without any of the operational toil. [NGINX Plus](https://www.nginx.com/products/nginx/) powers NGINXaaS for AWS, which extends NGINX Open Source with advanced functionality and provides customers with a complete application delivery solution. NGINXaaS handles the NGINX Plus license management automatically. ### [Prerequisites](/nginxaas/aws/deploy/prerequisites/) Follow these steps to prepare for your NGINXaaS deployment ### [Create a deployment](/nginxaas/aws/deploy/create-deployment/deploy-console/) Step-by-step instructions to deploy NGINXaaS using the NGINXaaS Console ### [Add certificates](/nginxaas/aws/deploy/ssl-tls-certificates/ssl-tls-certificates-console/) Instructions to add SSL/TLS certificates to your NGINXaaS deployment using the NGINXaaS Console ### [Get help](/nginxaas/support/) Contact F5 support for assistance with NGINXaaS for AWS ## Capabilities The key capabilities of NGINXaaS for AWS are: - Simplifies onboarding and use of NGINX by providing a fully managed, ready-to-use service, eliminating the need for infrastructure setup or manual upgrades. - Lowers operational overhead in running and optimizing NGINX. - Simplifies NGINX deployments with fewer moving parts (edge routing is built into the service). - Supports migration of existing NGINX configurations to the cloud with minimal effort. - Integrates with the AWS ecosystem. - Adopts a consumption-based pricing to align infrastructure costs to actual usage by billing transactions using AWS. ## NGINXaaS for AWS architecture ![Architecture diagram showing how NGINXaaS integrates with AWS. At the top, inside the AWS IaaS layer, NGINX Plus is managed using UI, API, and Terraform, alongside NGINXaaS. Admins connect to this layer. Below, in the Customer VPC, end users connect through Edge Routing to multiple App Servers (labeled App Server 1). NGINX Plus directs traffic to these app servers. The Customer VPC also connects with AWS services such as AWS Secrets Manager, Amazon CloudWatch, and other AWS services. Green arrows show traffic flow from end users through edge routing and NGINX Plus to app servers, while blue arrows show admin access.](nginxaas/aws/nginxaas-aws-cloud-architecture.svg) - The NGINXaaS Console is used to create, update, and delete NGINX configurations, certificates and NGINXaaS deployments - NGINXaaS automatically adapts to application traffic demands through autoscaling - Each NGINXaaS deployment has dedicated network and compute resources. There is no possibility of noisy neighbor problems or data leakage between deployments - NGINXaaS acts as a load balancer, API gateway, and reverse proxy, enabling you to keep your application workloads secure within your AWS account while serving traffic reliably and efficiently - NGINXaaS for AWS supports the following capabilities: - HTTP, HTTP/2, HTTP/3, and gRPC traffic - Layer 4 and Layer 7 load balancing with [configurable balancing methods](https://docs.nginx.com/nginx/admin-guide/load-balancer/http-load-balancer/#method) - IPv4 and IPv6 traffic - UDP, TCP, and QUIC protocols - Private or public internet client ingress - [Request tracing](https://www.f5.com/company/blog/nginx/application-tracing-nginx-plus) - HTTP to HTTPS, HTTPS to HTTP, and HTTP to HTTP redirects - NGINXaaS also provides the ability to create new rules for redirecting. See [How to Create NGINX Rewrite Rules | NGINX](https://blog.nginx.org/blog/creating-nginx-rewrite-rules) for more details ### Service frontend The service frontend of an NGINXaaS deployment controls how client ingress traffic reaches your deployment. There are two frontend types: managed public endpoint and private endpoint. #### Managed public endpoint A managed public endpoint frontend allows client access over the internet through a public DNS name created by NGINXaaS in its network. **This frontend type is suitable for:** - Serving public web applications to end users over the internet - Proxying traffic from clients outside AWS - Testing NGINXaaS configurations before you set up a [Private endpoint](/nginxaas/aws/overview.md#private-endpoint) frontend **Access control** Access control list (ACL) rules control traffic to a managed public endpoint deployment. If you don’t provide ACL rules, no traffic is allowed. An ACL rule includes the following settings: - **Source prefixes**: A list of CIDR blocks to allow traffic from - Use `0.0.0.0/0`, `::0/0` to allow traffic from all source IP addresses - **Protocol**: The network protocol to allow - Valid values are **TCP** and **UDP** - Required when you specify a port or port range - **Port range**: A single port or port range to allow traffic from - If you don’t specify a port range, traffic is allowed from any port - Required when you specify a protocol #### Private endpoint A private endpoint frontend allows client access through your network by using [AWS PrivateLink](https://aws.amazon.com/privatelink/). To set up connectivity, create an [interface VPC endpoint](https://docs.aws.amazon.com/vpc/latest/privatelink/create-interface-endpoint.html) in your own VPC that connects to the VPC endpoint service provisioned for your NGINXaaS deployment. This approach enables any applications or clients in your VPC to connect directly to the NGINXaaS deployment via private networking. For step-by-step instructions, see [Set up connectivity](/nginxaas/aws/deploy/create-deployment/deploy-console.md#set-up-connectivity-private-endpoint-only). **This frontend type is suitable for:** - Situations where you need greater control over traffic to the NGINXaaS deployment - Environments where all clients exist within your AWS network - Internal services that shouldn't be exposed to the internet **Access control** A PrivateLink connection allow list restricts which AWS account IDs or VPC endpoint IDs can connect to the deployment. If you don't specify any entries in the allow list, no PrivateLink connections will be accepted. The allow list can be modified at any time to add or remove access permission for any AWS accounts or VPC endpoints. ### Upstream network NGINXaaS uses [AWS VPC Peering](https://docs.aws.amazon.com/vpc/latest/peering/what-is-vpc-peering.html) to connect privately to your upstream applications. A VPC peering connection brings the deployment into your application network and supports secure, private connectivity to your upstream services. By managing your own VPC routing and security group rules, you control traffic flow and can apply your preferred security controls. To connect NGINXaaS to your upstream VPC, you must [create a VPC peering connection](https://docs.aws.amazon.com/vpc/latest/peering/create-vpc-peering-connection.html) from your AWS account targeting the NGINXaaS deployment's AWS Account ID and VPC ID, then add the peering connection ID to your deployment. **Note:** Upstream VPC CIDRs must not overlap with the NGINXaaS deployment VPC CIDRs, or the CIDRs of other peered upstream VPCs. If CIDRs overlap, VPC peering will fail. ### NGINX Capacity Unit (NCU) An NGINX Capacity Unit (NCU) quantifies the capacity of an NGINX deployment based on its underlying compute resources. This abstraction lets you specify capacity in NCUs without considering hardware differences between regions. You can reserve a minimum capacity for your deployment. The deployment automatically scales up or down based on traffic demand and makes sure it never drops below the reserved minimum. ### Geographical controllers NGINXaaS for AWS has a global presence, with management requests served by regional controllers. A geographical controller (GC) is a control plane that serves users within a defined geographic boundary while addressing data residency and localization requirements. For example, a US geographical controller serves customers in the United States. NGINXaaS currently operates in three geographies: US, EU, and Asia Pacific (APAC). ### Supported regions NGINXaaS for AWS is supported in the following regions per geography: |NGINXaaS Geography | AWS Regions | |-----------|---------| | US | us-east-1, us-east-2, us-west-1, us-west-2 | | EU | eu-west-1, eu-west-2, eu-west-3, eu-central-1, eu-north-1 | | APAC | ap-northeast-1, ap-northeast-2, ap-south-1, ap-south-2, ap-southeast-1, ap-southeast-4 | | CA | ca-central-1, ca-west-1 | ## Current limitations We are committed to enhancing NGINXaaS for AWS and welcome your feedback to help shape the future of our service. If there are features you'd like to see prioritized, we encourage you to submit a [support ticket](/nginxaas/aws/support.md) to share your suggestions. Here are the current constraints you should be aware of while using NGINXaaS for AWS: - User Role-Based Access Control (RBAC) is not yet supported, but this enhancement is on our roadmap as we improve access control for multi-user environments. - PrivateLink and upstream VPC peering connections must remain within the same AWS region as your deployment. Cross-region connections are not currently supported. - NGINXaaS deployments on AWS can only support up to 50 unique listen ports. - While NGINXaaS deployments on AWS can be configured for UDP and QUIC traffic, it requires that the deployment is listening for that traffic on IPv6. Note: this does not require the incoming client traffic, or upstream traffic to be IPv6. ## What's next To get started, check the [NGINXaaS for AWS prerequisites](/nginxaas/aws/deploy/prerequisites.md) -------------------------------------------------------------------------------- title: "Overview and architecture" url: https://docs.nginx.com/nginxaas/google/overview/index.md -------------------------------------------------------------------------------- ## What is NGINXaaS for Google Cloud? F5 NGINXaaS for Google Cloud is a SaaS offering that is tightly integrated into Google Cloud and its ecosystem of services, making applications fast, efficient, and reliable. It brings advanced traffic management capabilities from the commercial version of NGINX, without any of the operational toil. [NGINX Plus](https://www.nginx.com/products/nginx/) powers NGINXaaS for Google Cloud, which extends NGINX Open Source with advanced functionality and provides customers with a complete application delivery solution. NGINXaaS handles the NGINX Plus license management automatically. ### [Prerequisites](/nginxaas/google/deploy/prerequisites/) Follow these steps to prepare for your NGINXaaS deployment ### [Create a deployment](/nginxaas/google/deploy/create-deployment/deploy-console/) Step-by-step instructions to deploy NGINXaaS using the NGINXaaS Console ### [Add certificates](/nginxaas/google/deploy/ssl-tls-certificates/ssl-tls-certificates-console/) Instructions to add SSL/TLS certificates to your NGINXaaS deployment using the NGINXaaS Console ## Capabilities The key capabilities of NGINXaaS for Google Cloud are: - Simplifies onboarding by providing a fully managed, ready-to-use NGINX service, eliminating the need for infrastructure setup, manual upgrades, or operational overhead. - Lowers operational overhead in running and optimizing NGINX. - Simplifies NGINX deployments with fewer moving parts (edge routing is built into the service). - Supports migration of existing NGINX configurations to the cloud with minimal effort. - Integrates with the Google Cloud ecosystem. - Adopts a consumption-based pricing to align infrastructure costs to actual usage by billing transactions using Google. ## NGINXaaS for Google Cloud architecture ![Architecture diagram showing how NGINXaaS integrates with Google Cloud. At the top, inside the Google Cloud IaaS layer, NGINX Plus is managed using UI, API, and Terraform, alongside NGINXaaS. Admins connect to this layer. Below, in the Customer VPC, end users connect through Edge Routing to multiple App Servers (labeled App Server 1). NGINX Plus directs traffic to these app servers. The Customer VPC also connects with Google Cloud services such as Secret Manager, Monitoring, and other services. Green arrows show traffic flow from end users through edge routing and NGINX Plus to app servers, while blue arrows show admin access.](nginxaas/google/nginxaas-google-cloud-architecture.svg) - The NGINXaaS Console is used to create, update, and delete NGINX configurations, certificates and NGINXaaS deployments - NGINXaaS automatically adapts to application traffic demands through autoscaling - Each NGINXaaS deployment has dedicated network and compute resources. There is no possibility of noisy neighbor problems or data leakage between deployments - NGINXaaS can route traffic to upstreams even if the upstream servers are located in different geographies. See [Known Issues](/nginxaas/google/known-issues.md) for any networking restrictions. - NGINXaaS supports request tracing. See the [Application Performance Management with NGINX Variables](https://www.f5.com/company/blog/nginx/application-tracing-nginx-plus) blog to learn more about tracing. - Supports HTTP to HTTPS, HTTPS to HTTP, and HTTP to HTTP redirects. NGINXaaS also provides the ability to create new rules for redirecting. See [How to Create NGINX Rewrite Rules | NGINX](https://blog.nginx.org/blog/creating-nginx-rewrite-rules) for more details. ### Service frontend The service frontend of an NGINXaaS deployment controls how client ingress traffic reaches your deployment. There are two frontend types: managed public endpoint and private endpoint. #### Managed public endpoint A managed public endpoint frontend allows client access over the internet through a public DNS name created by NGINXaaS in its network. **This frontend type is suitable for:** - Serving public web applications to end users over the internet - Proxying traffic from clients outside Google Cloud - Testing NGINXaaS configurations before you set up a [Private Endpoint](/nginxaas/google/overview.md#private-endpoint) frontend **Access control** Access control list (ACL) rules control traffic to a managed public endpoint deployment. If you don’t provide ACL rules, no traffic is allowed. An ACL rule includes the following settings: - **Source prefixes**: A list of CIDR blocks to allow traffic from - Use `0.0.0.0/0` to allow traffic from all source IP addresses - **Protocol**: The network protocol to allow - Valid values are **TCP** and **UDP** - Required when you specify a port range - **Port range**: A single port or port range to allow traffic from - If you don’t specify a port range, traffic is allowed from any port - Required when you specify a protocol #### Private endpoint A private endpoint frontend allows client access through your network by using Google’s [Private Service Connect (PSC)](https://cloud.google.com/vpc/docs/private-service-connect). To set up connectivity, create either a [PSC endpoint](https://docs.cloud.google.com/vpc/docs/private-service-connect#endpoints) for internal traffic or a [PSC backend](https://cloud.google.com/vpc/docs/private-service-connect#backends) for external traffic. This approach brings the NGINXaaS deployment into your client network through an NGINXaaS-created service attachment, so application clients can connect directly into your network. For step-by-step instructions, see [Set up connectivity](/nginxaas/google/deploy/create-deployment/deploy-console.md#set-up-connectivity-private-endpoint-only). **This frontend type is suitable for:** - Situations where you need greater control over traffic to the NGINXaaS deployment - Environments where all clients exist within your Google Cloud network - Internal services that shouldn't be exposed to the internet **Access control** A service attachment accept list restricts which Google project IDs can connect to the deployment. If you don’t specify any project IDs in the accept list, traffic from all projects is allowed. ### Upstream network NGINXaaS uses Google [Private Service Connect](https://cloud.google.com/vpc/docs/private-service-connect) (PSC) to connect securely to your applications. A [PSC interface](https://cloud.google.com/vpc/docs/private-service-connect#interfaces) brings the deployment into your application network and supports secure connectivity to your applications. By using your own networking resources, you control traffic flow and can apply your preferred security controls. To connect the NGINXaaS PSC interface to your network, you must create a [network attachment](https://cloud.google.com/vpc/docs/about-network-attachments). For steps, see [Create a network attachment](/nginxaas/google/deploy/create-deployment/deploy-console.md#create-a-network-attachment). ### NGINX Capacity Unit (NCU) An NGINX Capacity Unit (NCU) quantifies the capacity of an NGINX deployment based on its underlying compute resources. This abstraction lets you specify capacity in NCUs without considering hardware differences between regions. You can reserve a minimum capacity for your deployment. The deployment automatically scales up or down based on traffic demand and makes sure it never drops below the reserved minimum. ### Geographical controllers NGINXaaS for Google Cloud has a global presence, with management requests served by regional controllers. A geographical controller (GC) is a control plane that serves users within a defined geographic boundary while addressing data residency and localization requirements. For example, a US geographical controller serves customers in the United States. NGINXaaS currently operates in three geographies: US, EU, and Asia Pacific (APAC). ### Supported regions NGINXaaS for Google Cloud is supported in the following regions per geography: |NGINXaaS Geography | Google Cloud Regions | |-----------|---------| | US | us-east1, us-east4, us-west1, us-west2, us-west3, us-west4, us-central1 | | EU | europe-west1, europe-west2, europe-west3, europe-west4, europe-north1, europe-central2 | | APAC | asia-southeast1, asia-south1, asia-south2, asia-northeast1, asia-northeast2, asia-northeast3 | | CA | northamerica-northeast1, northamerica-northeast2 | ## Current limitations We are committed to enhancing NGINXaaS for Google Cloud and welcome your feedback to help shape the future of our service. If there are features you'd like to see prioritized, we encourage you to submit a [support ticket](/nginxaas/google/support.md) to share your suggestions. Here are the current constraints you should be aware of while using NGINXaaS for Google Cloud: - NGINXaaS is [supported in a limited number of regions](/nginxaas/google/overview.md#supported-regions). We are continually working to expand support across additional regions. - User Role-Based Access Control (RBAC) is not yet supported, but this enhancement is on our roadmap as we improve access control for multi-user environments. - NGINXaaS deployments on Google Cloud do not support IPv6 traffic. - NGINXaaS deployments on Google Cloud only support UDP traffic when using **Managed Public Endpoint** frontend service. ## What's next To get started, check the [NGINXaaS for Google Cloud prerequisites](/nginxaas/google/deploy/prerequisites.md) -------------------------------------------------------------------------------- title: "Prerequisites" url: https://docs.nginx.com/nginxaas/aws/deploy/prerequisites/index.md description: Steps to subscribe to F5 NGINXaaS for AWS in the AWS Marketplace before you deploy. -------------------------------------------------------------------------------- Before you can deploy F5 NGINXaaS for AWS, you must subscribe to the AWS Marketplace offering. ## Early Access to the F5 NGINXaaS for AWS Marketplace offering F5 NGINXaaS for AWS is currently in Early Access. Contact us if you are interested in participating in our Early Access offering by sending an email to nginxaas-early-access@f5.com. ## What's next [Create a Deployment](/nginxaas/aws/deploy/create-deployment/deploy-console.md) -------------------------------------------------------------------------------- title: "Prerequisites" url: https://docs.nginx.com/nginxaas/google/deploy/prerequisites/index.md -------------------------------------------------------------------------------- Before you can deploy F5 NGINXaaS for Google Cloud (NGINXaaS), you need to complete some prerequisites. ## Subscribe to the NGINXaaS for Google Cloud offering If it's your first time using NGINXaaS for Google Cloud, you need to find the offering in the Google Cloud Marketplace and subscribe to it: ### Get the offering in the Google Cloud Marketplace 1. Find the [NGINXaaS for Google Cloud product listing in the Google Cloud Marketplace](https://console.cloud.google.com/marketplace/product/f5-7626-networks-public/nginxaas-google-cloud). 1. Login with your Google Cloud account. 1. Select **Subscribe** to subscribe to the NGINXaaS for Google Cloud offering. 1. Currently the **Enterprise** plan is the only plan supported. This option is selected automatically. - You can use the [usage and cost estimator](/nginxaas/google/billing/usage-and-cost-estimator/) to calculate the cost of your deployment based on your expected usage. 1. Select the billing account you want to use for this deployment. 1. Agree to the terms of service and privacy policy. 1. Select **Subscribe** and a message will confirm that your order request has been sent to F5, Inc. - Once F5 NGINXaaS receives your order request, it will be automatically approved. - This approval process may take several seconds to complete in the background. 1. Next, select **Sign up with F5, Inc.** to proceed. - In the **Welcome to F5 NGINXaaS for Google Cloud** window, select **Close Window** to return to the Google Cloud Console. 1. You can now log into the NGINXaaS for Google Cloud console by selecting **Manage on provider** from the Google Cloud Marketplace. ## What's next [Create a Deployment](/nginxaas/google/deploy/create-deployment/deploy-console.md) -------------------------------------------------------------------------------- title: "Configure F5 WAF for NGINX" url: https://docs.nginx.com/nginxaas/overview/app-protect/configure-waf/index.md description: Configure F5 WAF for NGINX security features by editing the NGINX configuration file. -------------------------------------------------------------------------------- ## Overview This guide explains how to configure the F5 WAF for NGINX security features. ## Configure To use F5 WAF for NGINX, apply the following changes to the NGINX config file. 1. Load the F5 WAF for NGINX module on the main context: ```nginx load_module modules/ngx_http_app_protect_module.so; ``` 2. Set the enforcer address: ```nginx app_protect_enforcer_address 127.0.0.1:50000; ``` **Note:** The app_protect_enforcer_address directive is a required directive for F5 WAF for NGINX to work and must match `127.0.0.1:50000` 3. Enable F5 WAF for NGINX with the `app_protect_enable` directives in the appropriate scope. The `app_protect_enable` directive may be set in the `http`, `server`, and `location` contexts. It is recommended to have a basic policy enabled in the `http` or `server` context to process malicious requests in a more complete manner. ```nginx app_protect_enable on; ``` 4. Configure the pre-defined policy to use with the `app_protect_policy_file` directive (either the `app_protect_default_policy` or `app_protect_strict_policy`). ```nginx app_protect_policy_file app_protect_strict_policy; ``` Sample Config with F5 WAF for NGINX configured: ```nginx user nginx; worker_processes auto; worker_rlimit_nofile 8192; pid /run/nginx/nginx.pid; load_module modules/ngx_http_app_protect_module.so; events { worker_connections 4000; } error_log /var/log/nginx/error.log debug; http { access_log off; server_tokens ""; app_protect_enforcer_address 127.0.0.1:50000; server { listen 80 default_server; location / { app_protect_enable on; app_protect_policy_file app_protect_strict_policy; proxy_pass http://127.0.0.1:80/proxy/$request_uri; } location /proxy { default_type text/html; return 200 "Hello World\n"; } } } ``` -------------------------------------------------------------------------------- title: "Disable F5 WAF for NGINX" url: https://docs.nginx.com/nginxaas/overview/app-protect/disable-waf/index.md description: Disable F5 WAF for NGINX on an NGINXaaS deployment using the NGINXaaS Console. -------------------------------------------------------------------------------- ## Overview This guide explains how to disable F5 WAF for NGINX on an NGINX as a Service (NGINXaaS) deployment. ## Before you start You must remove the WAF directives from your NGINX config file before attempting to disable WAF. ## Disable F5 WAF for NGINX ### Using the NGINXaaS Console - Open the [NGINXaaS Console](https://console.nginxaas.net/). - Log in with your preferred identity provider. - Select the Geography you want to work in. 1. Go to your NGINXaaS deployment. 2. Edit your deployment. 3. Disable **WAF** for your deployment. 4. Select **Save Changes** to begin the deployment process. -------------------------------------------------------------------------------- title: "NGINX config restrictions" url: https://docs.nginx.com/nginxaas/aws/deploy/nginx-configuration/nginx-config-restrictions/index.md description: Reference for NGINX configuration restrictions specific to F5 NGINXaaS for AWS deployments. -------------------------------------------------------------------------------- NGINXaaS for AWS enforces additional restrictions on NGINX configurations to support its managed network interfaces. These restrictions are validated when you upload a configuration; configurations that violate any restriction are rejected before they are applied. ## Listen address restrictions For general listen address and port restrictions that apply to all NGINXaaS deployments, see [Listener restrictions](/nginxaas/overview/nginx-configuration/configuration-rules.md#listener-restrictions). ## IPv6 requirement for UDP and QUIC NGINXaaS for AWS requires using an IPv6 address on listener ports configured for UDP or QUIC protocols. For example, a UDP listener can be configured as `listen [::]:53 udp;`. If a port configured for UDP or QUIC does not include an IPv6 listening address, the NGINX configuration cannot be applied to an AWS deployment. **Note:** Configuring the NGINX deployment to listen on IPv6 does not restrict or otherwise impact the IP address type used for incoming client traffic or upstream traffic. NGINXaaS ensures that IPv4 and IPv6 client traffic is always supported to the frontend service, and you can configure upstream traffic to use IPv4, IPv6, or both based on your application's needs. ## proxy_protocol consistency All `listen` directives for a given port (across every server block in the configuration) must have a consistent `proxy_protocol` setting. Enabling `proxy_protocol` on some listeners for a port while leaving it disabled on others is not supported. ## IPv4 and IPv6 across server blocks All `listen` directives for the same port must use the same address family: - **Merging within a single server block**: When a server block contains both an IPv4 listener and an IPv6 listener on the same port, NGINXaaS collapses them into a single port entry that carries both address families. That entry includes IPv6, so the NLB listener for that port is created as an IPv6 target group. - **Cross-server-block mixing is not allowed**: If port _N_ is referenced with an IPv6 address in one server block, every other server block that references port _N_ must also use IPv6. An IPv4-only listener for the same port in a different server block is not supported. ## Unique listen port limit A configuration can define at most 50 unique listen ports across the entire configuration. Configurations with more than 50 unique listen ports are not supported. ## What's next [Monitor your deployment](/nginxaas/aws/monitoring/enable-monitoring.md) -------------------------------------------------------------------------------- title: "Enable monitoring" url: https://docs.nginx.com/nginxaas/aws/monitoring/enable-monitoring/index.md description: Learn how to enable, view, and disable Amazon CloudWatch metric export for F5 NGINXaaS for AWS deployments. -------------------------------------------------------------------------------- ## Overview Monitoring your application's performance is crucial for maintaining its reliability and efficiency. F5 NGINXaaS for AWS integrates with Amazon CloudWatch so you can collect, correlate, and analyze metrics for a thorough understanding of your application's health and behavior. NGINXaaS for AWS exports metrics using the [Amazon CloudWatch Embedded Metric Format (EMF)](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format.html). When you enable metric export, NGINXaaS writes EMF records to a CloudWatch Logs log group (the **Metric Group**) in a log stream named `metrics`. CloudWatch automatically extracts these records into metrics under the `NGINXaaS` namespace, where you can chart and query them like any other CloudWatch metric. ## Before you begin - Configure a **Role ARN** in the **Identity** section of your deployment. NGINXaaS uses this IAM role to export metrics to CloudWatch. See [Identity and access management](/nginxaas/aws/deploy/access-management.md) for how to create the role and attach the [Policy for CloudWatch Logs](/nginxaas/aws/deploy/access-management.md#step-3-add-inline-policies-to-your-role). - Grant the IAM role the permissions required to export metrics: - `logs:CreateLogStream` and `logs:PutLogEvents`: required for NGINXaaS to write EMF records to the metric log group. - `logs:CreateLogGroup`: required only if you want NGINXaaS to create the log group for you. If you pre-create the log group yourself, you can omit this permission. See [AWS's documentation on controlling access to CloudWatch Logs with IAM](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/auth-and-access-control-cwl.html) for more information. - To view metrics in the CloudWatch console, your own AWS identity needs read access such as `cloudwatch:GetMetricData` and `cloudwatch:ListMetrics`. This is separate from the deployment's IAM role. See [AWS's documentation on identity and access management for Amazon CloudWatch](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/auth-and-access-control-cw.html) for more information. - Configure the [`status_zone`](https://nginx.org/en/docs/http/ngx_http_status_module.html#status_zone) directive in your `server` blocks and the [`zone`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#zone) directive in your `upstream` blocks to collect HTTP request and response statistics, stream connection metrics, upstream statistics, and memory statistics. See the [Metrics Catalog](/nginxaas/aws/monitoring/metrics-catalog.md) for configuration requirements. ## Export NGINXaaS metrics to CloudWatch To enable exporting metrics, turn on the **Export Metrics to CloudWatch** toggle when creating or updating a deployment. To create a deployment, see [our documentation on creating an NGINXaaS deployment](/nginxaas/aws/deploy/create-deployment/) for a step-by-step guide. To update an existing deployment, in the NGINXaaS console, 1. On the navigation menu, select **Deployments**. 1. Select the deployment you want to update and select **Edit**. 1. In the **Identity** section, make sure the **Role ARN** field is populated with an IAM role that has the [required permissions](#before-you-begin). Metric export fails without it. 1. In the **Observability** section, turn on the **Export Metrics to CloudWatch** toggle. 1. In the **Metric Group Name** field, enter the name of the CloudWatch Logs log group you want to receive exported metrics. If you leave this field at its default, NGINXaaS uses a log group named after the deployment object ID. 1. Select **Update**. Within the metric log group, NGINXaaS writes EMF records to a log stream named `metrics`. CloudWatch extracts these records into metrics under the `NGINXaaS` namespace. ## View NGINXaaS metrics in Amazon CloudWatch See the [Metrics Catalog](/nginxaas/aws/monitoring/metrics-catalog.md) for a full list of metrics NGINXaaS for AWS provides. ### CloudWatch Metrics Explorer Log in to the [AWS Management Console](https://console.aws.amazon.com/), 1. Go to the **CloudWatch** console. 1. Select **Classic Metrics** and then select the **NGINXaaS** tile in the **Custom Namespaces** section. Refer to the [AWS's CloudWatch Metrics Explorer](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Metrics-Explorer.html) documentation to learn how you can create charts and queries. **Note:** Metrics can take a few minutes to appear after you enable export, because CloudWatch must first ingest the EMF records and extract them into the `NGINXaaS` namespace. To inspect the raw EMF records, open the `metrics` log stream in your metric log group with [CloudWatch Logs Insights](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AnalyzingLogData.html). ## Disable NGINXaaS metric export to CloudWatch To stop exporting metrics, update your NGINXaaS deployment to turn off the metric export toggle. To update the deployment, in the NGINXaaS console, 1. On the navigation menu, select **Deployments**. 1. Select the deployment you want to update and select **Edit**. 1. In the **Observability** section, turn off the **Export Metrics to CloudWatch** toggle. 1. Select **Update**. ## Troubleshooting If Amazon CloudWatch is not showing any metrics, check for **Failed Metric Export to CloudWatch** events from your NGINXaaS deployment. In the NGINXaaS console: 1. On the navigation menu, select **Events**. 1. Select **Add Filter**. 1. Select **Affected Object** and the name of your NGINXaaS deployment. Events are deleted after 14 days. ## What's next - [Enable NGINX logs](/nginxaas/aws/monitoring/enable-nginx-logs.md) - [Metrics catalog](/nginxaas/aws/monitoring/metrics-catalog.md) - [Identity and access management](/nginxaas/aws/deploy/access-management.md) -------------------------------------------------------------------------------- title: "Enable monitoring" url: https://docs.nginx.com/nginxaas/google/monitoring/enable-monitoring/index.md -------------------------------------------------------------------------------- Monitoring your application's performance is crucial for maintaining its reliability and efficiency. F5 NGINXaaS for Google Cloud (NGINXaaS) seamlessly integrates with Google Cloud services, allowing you to collect, correlate, and analyze metrics for a thorough understanding of your application's health and behavior. ## Prerequisites - Enable the [Cloud Monitoring API](https://cloud.google.com/monitoring/api/enable-api). - Configure Workload Identity Federation (WIF). See [our documentation on setting up WIF](/nginxaas/google/deploy/access-management.md#configure-wif) for exact steps. - Grant your principal access to one of the following IAM roles depending on your needs: - `roles/monitoring.viewer` — View metrics in Cloud Monitoring (read-only access) - `roles/monitoring.editor` — View metrics and create or import dashboards See [Google's documentation on controlling access to Cloud Monitoring with IAM](https://cloud.google.com/monitoring/access-control) for more information. - Configure the [`status_zone`](https://nginx.org/en/docs/http/ngx_http_status_module.html#status_zone) directive in your `server` blocks and the [`zone`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#zone) directive in your `upstream` blocks to collect HTTP request and response statistics, stream connection metrics, upstream statistics, and memory statistics. See the [Metrics Catalog](/nginxaas/google/monitoring/metrics-catalog.md) for configuration requirements. ## Export NGINXaaS metrics to a Google Cloud Project To enable sending metrics to your desired Google Cloud project, you must specify the project ID when creating or updating a deployment. To create a deployment, see [our documentation on creating an NGINXaaS deployment](/nginxaas/google/deploy/create-deployment/) for a step-by-step guide. To update the deployment, in the NGINXaaS console, 1. On the navigation menu, select **Deployments**. 1. Select the deployment you want to update and select **Edit**. 1. Enter the project you want metrics to be sent to under **Metric Project ID**. 1. Select **Update**. ## View NGINXaaS metrics in Google Cloud Monitoring See the [Metrics Catalog](/nginxaas/google/monitoring/metrics-catalog.md) for a full list of metrics NGINXaaS provides. ### Google Cloud Console's Metrics Explorer Log in to your [Google Cloud Console](https://console.cloud.google.com/), 1. Go to your metric project. 2. Search for "Metrics Explorer". Refer to the [Google's Metrics Explorer](https://cloud.google.com/monitoring/charts/metrics-explorer) documentation to learn how you can create charts and queries. ### Import a pre-built dashboard To help you quickly visualize important metrics and logs from your NGINXaaS deployments, you can import a pre-configured dashboard into Google Cloud Monitoring. The dashboard displays key metrics such as request and connection rates, response status codes, upstream health, and access and error logs. To import the dashboard: 1. Copy the dashboard JSON configuration #### Show dashboard JSON ```json { "displayName": "NGINXaaS", "dashboardFilters": [ { "filterType": "METRIC_LABEL", "labelKey": "nginxaas_deployment_name", "templateVariable": "nginxaas_deployment_name", "valueType": "STRING_ARRAY" } ], "description": "", "labels": {}, "mosaicLayout": { "columns": 48, "tiles": [ { "height": 16, "width": 29, "widget": { "title": "Request + Connection rates", "id": "", "xyChart": { "chartOptions": { "displayHorizontal": false, "mode": "COLOR", "showLegend": false }, "dataSets": [ { "breakdowns": [], "dimensions": [], "legendTemplate": "Connections/second", "measures": [], "minAlignmentPeriod": "60s", "plotType": "LINE", "sort": [], "targetAxis": "Y2", "timeSeriesQuery": { "outputFullDuration": false, "timeSeriesFilter": { "aggregation": { "alignmentPeriod": "60s", "crossSeriesReducer": "REDUCE_SUM", "groupByFields": [], "perSeriesAligner": "ALIGN_RATE" }, "filter": "metric.type=\"workload.googleapis.com/nginx.http.connections\" resource.type=\"generic_node\" metric.label.\"nginx_connections_outcome\"=\"ACCEPTED\" ${nginxaas_deployment_name}" }, "unitOverride": "" } }, { "breakdowns": [], "dimensions": [], "legendTemplate": "Requests/second", "measures": [], "minAlignmentPeriod": "60s", "plotType": "LINE", "sort": [], "targetAxis": "Y1", "timeSeriesQuery": { "outputFullDuration": false, "timeSeriesFilter": { "aggregation": { "alignmentPeriod": "60s", "crossSeriesReducer": "REDUCE_SUM", "groupByFields": [], "perSeriesAligner": "ALIGN_RATE" }, "filter": "metric.type=\"workload.googleapis.com/nginx.http.requests\" resource.type=\"generic_node\" ${nginxaas_deployment_name}" }, "unitOverride": "" } } ], "thresholds": [], "y2Axis": { "label": "Conn/s", "scale": "LINEAR" }, "yAxis": { "label": "Req/s", "scale": "LINEAR" } } } }, { "xPos": 29, "height": 16, "width": 18, "widget": { "title": "Current Connections", "id": "", "scorecard": { "breakdowns": [], "dimensions": [], "measures": [], "sparkChartView": { "sparkChartType": "SPARK_LINE" }, "thresholds": [], "timeSeriesQuery": { "outputFullDuration": false, "timeSeriesFilter": { "aggregation": { "alignmentPeriod": "60s", "crossSeriesReducer": "REDUCE_SUM", "groupByFields": [ "metric.label.\"nginx_connections_outcome\"" ], "perSeriesAligner": "ALIGN_MEAN" }, "filter": "metric.type=\"workload.googleapis.com/nginx.http.connection.count\" resource.type=\"generic_node\" ${nginxaas_deployment_name}" }, "unitOverride": "" } } } }, { "yPos": 16, "height": 14, "width": 29, "widget": { "title": "Response rate by Zone + Status", "id": "", "xyChart": { "chartOptions": { "displayHorizontal": false, "mode": "COLOR", "showLegend": false }, "dataSets": [ { "breakdowns": [], "dimensions": [], "legendTemplate": "", "measures": [], "minAlignmentPeriod": "60s", "plotType": "LINE", "sort": [], "targetAxis": "Y1", "timeSeriesQuery": { "outputFullDuration": false, "timeSeriesFilter": { "aggregation": { "alignmentPeriod": "60s", "crossSeriesReducer": "REDUCE_SUM", "groupByFields": [ "metric.label.\"nginx_zone_name\"", "metric.label.\"nginx_status_range\"" ], "perSeriesAligner": "ALIGN_RATE" }, "filter": "metric.type=\"workload.googleapis.com/nginx.http.response.status\" resource.type=\"generic_node\" ${nginxaas_deployment_name}" }, "unitOverride": "" } } ], "thresholds": [], "yAxis": { "label": "", "scale": "LINEAR" } } } }, { "yPos": 16, "xPos": 29, "height": 9, "width": 9, "widget": { "title": "Current Requests", "id": "", "scorecard": { "breakdowns": [], "dimensions": [], "measures": [], "sparkChartView": { "sparkChartType": "SPARK_LINE" }, "thresholds": [], "timeSeriesQuery": { "outputFullDuration": false, "timeSeriesFilter": { "aggregation": { "alignmentPeriod": "60s", "crossSeriesReducer": "REDUCE_SUM", "groupByFields": [], "perSeriesAligner": "ALIGN_MEAN" }, "filter": "metric.type=\"workload.googleapis.com/nginx.http.request.processing.count\" resource.type=\"generic_node\" metric.label.\"nginx_zone_name\"=monitoring.regex.full_match(\"..*\") ${nginxaas_deployment_name}", "pickTimeSeriesFilter": { "direction": "TOP", "numTimeSeries": 12, "rankingMethod": "METHOD_MEAN" } }, "unitOverride": "" } } } }, { "yPos": 16, "xPos": 38, "height": 9, "width": 9, "widget": { "title": "Config updates", "id": "", "scorecard": { "breakdowns": [], "dimensions": [], "measures": [], "sparkChartView": { "sparkChartType": "SPARK_LINE" }, "thresholds": [], "timeSeriesQuery": { "outputFullDuration": true, "timeSeriesFilter": { "aggregation": { "alignmentPeriod": "60s", "crossSeriesReducer": "REDUCE_SUM", "groupByFields": [ "metric.label.\"nginxaas_deployment_name\"" ], "perSeriesAligner": "ALIGN_DELTA" }, "filter": "metric.type=\"workload.googleapis.com/nginx.config.reloads\" resource.type=\"generic_node\" ${nginxaas_deployment_name}", "pickTimeSeriesFilter": { "direction": "TOP", "numTimeSeries": 12, "rankingMethod": "METHOD_MEAN" } }, "unitOverride": "" } } } }, { "yPos": 25, "xPos": 29, "height": 12, "width": 18, "widget": { "title": "Upstream Status", "id": "", "xyChart": { "chartOptions": { "displayHorizontal": false, "mode": "COLOR", "showLegend": false }, "dataSets": [ { "breakdowns": [], "dimensions": [], "legendTemplate": "", "measures": [], "minAlignmentPeriod": "60s", "plotType": "STACKED_AREA", "sort": [], "targetAxis": "Y1", "timeSeriesQuery": { "outputFullDuration": false, "timeSeriesFilter": { "aggregation": { "alignmentPeriod": "60s", "crossSeriesReducer": "REDUCE_SUM", "groupByFields": [ "metric.label.\"nginx_peer_state\"" ], "perSeriesAligner": "ALIGN_MEAN" }, "filter": "metric.type=\"workload.googleapis.com/nginx.http.upstream.peer.count\" resource.type=\"generic_node\" ${nginxaas_deployment_name}", "pickTimeSeriesFilter": { "direction": "TOP", "numTimeSeries": 12, "rankingMethod": "METHOD_MAX" } }, "unitOverride": "" } } ], "thresholds": [], "yAxis": { "label": "", "scale": "LINEAR" } } } }, { "yPos": 30, "height": 14, "width": 29, "widget": { "title": "Upstream Response rate by Zone + Status", "id": "", "xyChart": { "chartOptions": { "displayHorizontal": false, "mode": "COLOR", "showLegend": false }, "dataSets": [ { "breakdowns": [], "dimensions": [], "legendTemplate": "", "measures": [], "minAlignmentPeriod": "60s", "plotType": "LINE", "sort": [], "targetAxis": "Y1", "timeSeriesQuery": { "outputFullDuration": false, "timeSeriesFilter": { "aggregation": { "alignmentPeriod": "60s", "crossSeriesReducer": "REDUCE_SUM", "groupByFields": [ "metric.label.\"nginx_zone_name\"", "metric.label.\"nginx_status_range\"" ], "perSeriesAligner": "ALIGN_RATE" }, "filter": "metric.type=\"workload.googleapis.com/nginx.http.upstream.peer.responses\" resource.type=\"generic_node\" ${nginxaas_deployment_name}" }, "unitOverride": "" } } ], "thresholds": [], "yAxis": { "label": "", "scale": "LINEAR" } } } }, { "yPos": 44, "height": 16, "width": 48, "widget": { "title": "Error Logs", "logsPanel": { "filter": "resource.type=\"generic_node\"\nlabels.\"log.file.name\"=\"error.log\"\nlabels.\"log.file.path\"=~\"/var/log/nginx/.*\\.log\"", "resourceNames": [] } } }, { "yPos": 60, "height": 16, "width": 48, "widget": { "title": "Access Logs", "logsPanel": { "filter": "resource.type=\"generic_node\"\nlabels.\"log.file.name\"=\"access.log\"\nlabels.\"log.file.path\"=~\"/var/log/nginx/.*\\.log\"", "resourceNames": [] } } } ] } } ``` 2. Go to the [Google Cloud Console](https://console.cloud.google.com/). 3. Go to your metrics project. 4. Search for **Monitoring** and select **Dashboards**. 5. Select **Create Custom Dashboard**. 6. Select the settings icon (⚙), and select **JSON** to switch to JSON editor mode. 7. Replace the default JSON with the dashboard configuration you copied. 8. Select **Apply Changes**. **Note:** The dashboard includes an **nginxaas_deployment_name** filter. Use this filter to view metrics for a specific NGINXaaS deployment or select multiple deployments to compare their performance. ## Disable exporting NGINXaaS metrics to a Google Cloud project To disable sending metrics to your Google Cloud project, update your NGINXaaS deployment to remove the reference to your project ID. To update the deployment, in the NGINXaaS console, 1. On the navigation menu, select **Deployments**. 1. Select the deployment you want to update and select **Edit**. 1. Remove the project ID under **Metric Project ID**. 1. Select **Update**. ## Troubleshooting If Google Cloud Monitoring is not showing any metrics, check for **Failed Metric Export to Google** events from your NGINXaaS deployment. In the NGINXaaS console: 1. On the navigation menu, select **Events**. 1. Select **Add Filter**. 1. Select **Affected Object** and the name of your NGINXaaS deployment. Events are deleted after 14 days. -------------------------------------------------------------------------------- title: "Usage and cost estimator" url: https://docs.nginx.com/nginxaas/google/billing/usage-and-cost-estimator/index.md --------------------------------------------------------------------------------

Cost Estimation for Enterprise Plan

Estimate Monthly Cost

Total Monthly Payment --
Show calculations

hours * ( fixed/hr + NCUs * per NCU/hr ) + GB * per GB =

-------------------------------------------------------------------------------- title: "Manage users and organizations" url: https://docs.nginx.com/nginxaas/overview/manage-users-organizations/index.md -------------------------------------------------------------------------------- ## Overview This document explains how to manage users and organizations in F5 NGINXaaS using the NGINXaaS console. Before you start, ensure you understand the following concepts: - **NGINXaaS Organization**: An NGINXaaS Organization can be created at any time, to host your NGINXaaS resources for your team or business unit. The organization must be linked to an active cloud marketplace subscription in order to manage and utilize NGINXaaS deployments in that cloud. - **User**: NGINXaaS Users are granted access to all resources in the NGINXaaS Organization. User authentication is performed securely via your allowed login method(s), requiring a matching identity. - Individuals can be added as users to multiple NGINXaaS Organizations, and can switch between them using the steps documented below. - Note that once a user has gained access to an NGINXaaS Organization through a particular login method, they must continue to use that login method to gain access to that NGINXaaS Organization. If the same human user authenticates through two different login methods, the resulting user identities are distinct from the perspective of NGINXaaS. - **Authentication settings**: Authentication settings are specific to the enabled login method. - **Login Methods**: NGINXaaS authenticates users through the following login methods: - Microsoft social login - Google social login - Email and password You can configure the NGINXaaS Organization to allow any or all of these methods. By default, all listed methods are enabled. See [user login](/nginxaas/overview/user-login.md) for further details. - **Google Authorized Domains**: If Google social login is enabled, authorized users can configure the list of domains with which users must be associated. - This can be used to restrict access to Google identities within your Google Cloud Organization or Google Workspace, or other known, trusted Workspaces. For example, your Google Cloud Organization may have users created under the `example.com` domain. By setting the Authorized Domains in your NGINXaaS Organization to only allow `example.com`, users attempting to log in with the same email associated with `alternative.net` Google Workspace would not be authenticated. - By default, an NGINXaaS Organization has an empty authorized domains list, which accepts matching users from any Google Workspace. - **Microsoft Authorized Tenants**: If Microsoft social login is enabled, authorized users can configure the list of Azure tenant IDs to which users must belong. - For example, all team members seeking to gain access to your NGINXaaS Organization have an Entra identity within a particular Azure tenant. You can add the Azure tenant ID to the list of Microsoft Authorized Tenants. This will restrict anyone with an Entra identity outside that Azure tenant from accessing your NGINXaaS Organization. - By default, users of all Azure tenants will be allowed to match with the new user entries you add to your organization. ## Access the NGINXaaS Console - Open the [NGINXaaS Console](https://console.nginxaas.net/). - Log in with your preferred identity provider. - Select the Geography you want to work in. Once logged in, you can create and manage [NGINX configurations](/nginxaas/overview/nginx-configuration/nginx-configuration-console.md) and [SSL/TLS certificates](/nginxaas/overview/ssl-tls-certificates/ssl-tls-certificates-console.md). If you want to create an NGINXaaS deployment, subscribe to your preferred cloud provider(s) (such as [AWS](/nginxaas/aws/deploy/prerequisites.md#subscribe-to-the-nginxaas-for-aws-offering) or [Google Cloud](/nginxaas/google/deploy/prerequisites.md#subscribe-to-the-nginxaas-for-google-cloud-offering)). ## Create an organization - **Logging in for the first time**: If you are not already a member of an existing organization, you will be prompted to create a new organization. Enter an optional **Organization Name** and select **Submit**. - **When already logged in**: In the [NGINXaaS Console](https://console.nginxaas.net/), select your profile icon in the top right corner and choose **Switch Organization**. Select **Add Organization**, enter an **Organization Name**, and choose **Create and Select** to create and switch to the new organization. **Note:** Choose a clear, recognizable name for your organization. Avoid leaving the name empty or using generic titles, as a distinct organization name helps team members easily identify and switch to the correct organization when collaborating. You can create NGINX configurations and upload SSL/TLS certificates within an organization without a cloud subscription. If you want to create an NGINXaaS deployment, please subscribe to your preferred cloud provider(s) in the cloud marketplace. ## Add or edit a user An existing NGINXaaS Organization user can add additional users following these steps: 1. Navigate to the **Users** page on the left menu, then select **Add User**. 1. Enter the **Email** address for the user to be added. 1. Select **Create User** to save the changes. The new user will appear in the list of users on the **Users** page. The next time they log in, they will be able to access this NGINXaaS Organization. ## Modify an organization's Authentication Settings 1. Select **Organization Details** under the **Settings** section on the left menu. 1. Select **Edit** in the **Authentication Settings** section. 1. Tick the login methods that you wish to enable. 1. To add a new Google authorized domain, select **Add Domain** and enter the new domain. 1. To remove an existing Google authorized domain, select the Recycle Bin button next to it. 1. To add a new Authorized Microsoft tenant, select **Add Tenant ID** and enter the Azure tenant ID. 1. To remove an existing Authorized Microsoft tenant, select the Recycle Bin button next to it. 1. Select **Update** to save changes. **Note:** You cannot remove a Google authorized domain or an Authorized Microsoft tenant from an organization's authentication settings if the action will lock out existing users of the organization. To modify the authentication settings you must first delete every NGINXaaS user that is associated with the Google authorized domain or Authorized Microft tenant that you wish to exclude. ### Modify the name of an organization 1. Select **Organization Details** under the **Settings** section on the left menu. 1. Select **Edit** in the **Organization Info** section. 1. Enter new name in the **Organization Name** field, then select **Update** to save changes. ## Switch organizations To switch to a different NGINXaaS Organization (or to create a new organization), select the profile symbol in the top right corner and choose **Switch Organization**. This opens a page showing the list of all NGINXaaS Organizations linked to your user identity. Select the organization you want to switch to, or select **Add Organization** to set up a new one. ## Delete a user An authenticated user can delete other users, but not their own user. Deletion is irreversible; the deleted user will no longer be able to access the NGINXaaS Organization. To delete a user in an NGINXaaS Organization: 1. Select **Organization Details** under the **Settings** section on the left menu. 1. Select the ellipsis (three dots) menu next to the user you want to delete. 1. Select **Delete** in the menu. The deleted user will no longer appear in the **Users** page. ## What's next [Add an NGINX configuration using the NGINXaaS Console](/nginxaas/overview/nginx-configuration/nginx-configuration-console.md) -------------------------------------------------------------------------------- title: "Usage dashboard" url: https://docs.nginx.com/nginxaas/google/billing/usage-dashboard/index.md -------------------------------------------------------------------------------- # Usage dashboard The Usage dashboard provides detailed insights into your NGINX as a Service resource consumption for the entire customer organization, encompassing deployments across all geographies and regions. View hourly usage metrics to track costs, analyze patterns, and optimize your deployments. Use the dashboard to analyze daily traffic patterns, plan capacity requirements, and optimize geographic distribution across regions. Use this dashboard to get daily and monthly usage trends, monitor peak traffic hours to plan scaling strategies and compare performance across different regions to optimize deployment location. ## Detailed per-deployment analysis This dashboard provides granular, per-deployment usage analysis. View individual deployment consumption, region-specific usage, and hourly patterns for each deployment. ## Viewing your usage data Access your usage dashboard to view consumption metrics across all your deployments. The dashboard displays hourly usage data that you can filter and analyze based on your needs. ## Available filters ### Time period selection Choose the time range for your usage from the predefined ranges. The default time range is last 30 days. ### Filtering options Narrow down your usage data using these filters. You can combine multiple filters to get more precise usage insights: | Filter | Description | Use Case | |--------|-------------|----------| | **Region** | Filter by cloud region | Analyze regional usage patterns | | **Deployment** | Filter by specific deployment | Focus on individual deployment consumption | #### Filtering workflow To analyze your usage data, follow this step-by-step filtering process: 1. **Select a time range** - Choose from the available time periods as described in [Time Period Selection](#time-period-selection) 2. **Filter by region** - Narrow down the data to specific cloud regions where your deployments are located 3. **Filter by deployment** - Focus on specific deployments within the selected region for detailed analysis This hierarchical filtering approach allows you to progressively narrow your analysis from organization-wide usage to specific deployment performance. ## Usage metrics explained ### Hourly data points Usage data is collected and displayed on an hourly basis. Each hour shows: - **Timestamp**: The specific hour when usage was recorded - **Location Details**: Region and deployment information - **Consumption Metrics**: Detailed usage measurements ### Usage meters Your usage dashboard tracks several types of consumption: | Meter Type | Description | |------------|-------------| | **Fixed** | Base service cost | | **NCU (NGINX Compute Units)** | Processing capacity used | | **Data Processed** | Volume of data handled | For more information about NCU, see the [overview](overview.md). ## Data retention Historical usage data is available for **up to one year**, allowing for comprehensive analysis and trend identification. -------------------------------------------------------------------------------- title: "Enable NGINX logs" url: https://docs.nginx.com/nginxaas/aws/monitoring/enable-nginx-logs/index.md description: Learn how to configure, export, view, and disable NGINX access and error log export to Amazon CloudWatch Logs. -------------------------------------------------------------------------------- ## Overview F5 NGINXaaS for AWS integrates with Amazon CloudWatch Logs to collect NGINX error and access logs. When you enable log export, NGINXaaS writes NGINX logs to a CloudWatch Logs log group (the **Log Group**) in a log stream named `logs`. ## Before you begin - Configure a **Role ARN** in the **Identity** section of your deployment. NGINXaaS uses this IAM role to export logs to CloudWatch. See [Identity and access management](/nginxaas/aws/deploy/access-management.md) for how to create the role and attach the [Policy for CloudWatch Logs](/nginxaas/aws/deploy/access-management.md#step-3-add-inline-policies-to-your-role). - Grant the IAM role the permissions required to export logs: - `logs:CreateLogStream` and `logs:PutLogEvents`: required for NGINXaaS to write logs to the log group. - `logs:CreateLogGroup`: required only if you want NGINXaaS to create the log group for you. If you pre-create the log group yourself, you can omit this permission. See [AWS's documentation on controlling access to CloudWatch Logs with IAM](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/auth-and-access-control-cwl.html) for more information. - To view logs in the CloudWatch console, your own AWS identity needs read access such as `logs:GetLogEvents` and `logs:FilterLogEvents`. This is separate from the deployment's IAM role. ## Set up error logs NGINX error logs are disabled by default. You can enable error logs by adding **error_log** directives to your NGINX configuration to specify the location of the logs and formats. The log path should always be configured to be inside **/var/log/nginx**. While you should configure log files in the **/var/log/nginx** directory, you can change the filename and severity level. For example, the following line in the NGINX configuration sends errors to the `nginx-error.log` file, and limits messages to a severity level of **emerg**: ```nginx error_log /var/log/nginx/nginx-error.log emerg; ``` Alternatively, you can disable error logs completely with the following line: ```nginx error_log /dev/null; ``` To learn more about how to specify `error_log` in different configuration levels, see the documentation of the [error_log](https://nginx.org/en/docs/ngx_core_module.html?#error_log) directive. ## Set up access logs NGINX access logs are disabled by default. You can enable access logs by adding **access_log** directives to your NGINX configuration to specify the location of the logs and formats. The log path should always be configured to be inside **/var/log/nginx**. ```nginx http { log_format myfmt '$remote_addr - $remote_user [$time_local] ' '"$request" $status $body_bytes_sent ' '"$http_referer" "$http_user_agent" "$gzip_ratio"'; access_log /var/log/nginx/nginx-access.log myfmt; # ... } ``` **Note:** The **$time_local** variable includes the date and time for each log. It helps with ordering logs after export. To explicitly disable access logs, apply the following config: ```nginx http { access_log off; } ``` or ```nginx http { access_log /dev/null; } ``` To learn more about how to specify `access_log` in different configuration levels and their effect, see [access_log](https://nginx.org/en/docs/http/ngx_http_log_module.html#access_log) **Note:** Keep NGINX logs in the **/var/log/nginx** directory. Otherwise, you may lose data from your logs. ## Set up F5 WAF for NGINX security logs You can enable security logs by adding **app_protect_security_log** directives to your NGINX configuration to specify the location of the logs and logging formats. The log path should always be configured under **/var/log/app_protect**. ```nginx app_protect_security_log_enable on; app_protect_security_log log_default /var/log/app_protect/security.log; ``` NGINXaaS does not support custom logging profiles and is limited to the [default logging profiles](/waf/logging/logs-overview.md#default-logging-profile-bundles). **Note:** WAF logs should always be stored under the **/var/log/app_protect** directory. You may lose logging data if you choose any other log paths. ## Export NGINX logs to CloudWatch To enable exporting logs, turn on the **Export Logs to CloudWatch** toggle when creating or updating a deployment. To create a deployment, see [our documentation on creating an NGINXaaS deployment](/nginxaas/aws/deploy/create-deployment/) for a step-by-step guide. To update an existing deployment, in the NGINXaaS console, 1. On the navigation menu, select **Deployments**. 1. Select the deployment you want to update and select **Edit**. 1. In the **Identity** section, make sure the **Role ARN** field is populated with an IAM role that has the [required permissions](#before-you-begin). Log export fails without it. 1. In the **Observability** section, turn on the **Export Logs to CloudWatch** toggle. 1. In the **Log Group Name** field, enter the name of the CloudWatch Logs log group you want to receive exported logs. If you leave this field at its default, NGINXaaS uses a log group named after the deployment object ID. 1. Select **Update**. ## View NGINX logs in Amazon CloudWatch Logs In the [AWS Management Console](https://console.aws.amazon.com/), 1. Go to the **CloudWatch** console and select **Logs** > **Log Management** > **Log groups**. 1. In the **Filter log groups** field, enter the name of your log group to find it quickly. 1. Select your log group and open the `logs` log stream, or use **Logs Insights** to query across streams. Refer to the [AWS's CloudWatch Logs Insights](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AnalyzingLogData.html) documentation to learn how you can create queries. Within the log group, NGINX access and error logs are written to a log stream named `logs`, which you can use to separate NGINX logs from the rest of your log group's streams. You can also filter based on the following fields using a CloudWatch Logs Insights `filter` or `stats` query, for example, * `filename` * `nginxaas_deployment_location` * `nginxaas_deployment_name` * `nginxaas_deployment_object_id` * `nginxaas_namespace` * `nginxaas_organization_object_id` For example, to query for a deployment's /var/log/nginx/access.log entries: 1. In the CloudWatch console, select **Logs** > **Logs Insights**. 1. Select the log group for your deployment (default value is the deployment object ID). 1. In the query editor, enter the following query to filter for access log entries ```text fields @timestamp, @message | filter @logStream = "logs" and attributes.filename = "/var/log/nginx/access.log" | sort @timestamp desc ``` The same procedure can be used to query via the newer **Log Analytics** feature in the CloudWatch console. See [AWS's documentation on Log Analytics](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/LogAnalytics.html) for more information. ## Disable NGINX log export to CloudWatch To stop exporting logs, update your NGINXaaS deployment to turn off the log export toggle. To update the deployment, in the NGINXaaS console, 1. On the navigation menu, select **Deployments**. 1. Select the deployment you want to update and select **Edit**. 1. In the **Observability** section, turn off the **Export Logs to CloudWatch** toggle. 1. Select **Update**. ## Troubleshooting If Amazon CloudWatch Logs is not showing any logs, check for **Failed Log Export to CloudWatch** events from your NGINXaaS deployment. In the NGINXaaS console: 1. On the navigation menu, select **Events**. 1. Select **Add Filter**. 1. Select **Affected Object** and the name of your NGINXaaS deployment. Events are deleted after 14 days. ## What's next - [Enable monitoring](/nginxaas/aws/monitoring/enable-monitoring.md) - [Metrics catalog](/nginxaas/aws/monitoring/metrics-catalog.md) - [Identity and access management](/nginxaas/aws/deploy/access-management.md) -------------------------------------------------------------------------------- title: "Enable NGINX logs" url: https://docs.nginx.com/nginxaas/google/monitoring/enable-nginx-logs/index.md -------------------------------------------------------------------------------- F5 NGINXaaS for Google (NGINXaaS) supports integrating with Google Cloud services to collect NGINX error and access logs, and F5 WAF for NGINX security logs. ## Prerequisites - Enable the [Cloud Logging API](https://docs.cloud.google.com/logging/docs/api/enable-api). - Configure Workload Identity Federation (WIF). See [our documentation on setting up WIF](/nginxaas/google/deploy/access-management.md#configure-wif) for exact steps. - Grant a project-level role or grant your principal access to the `roles/logging.viewer` role. See [Google's documentation on controlling access to Cloud Logging with IAM](https://cloud.google.com/logging/docs/access-control). ## Setting up error logs NGINX error logs are disabled by default. You can enable error logs by adding **error_log** directives to your NGINX configuration to specify the location of the logs and formats. The log path should always be configured to be inside **/var/log/nginx**. While you should configure log files in the **/var/log/nginx** directory, you can change the filename and severity level. For example, the following line in the NGINX configuration sends errors to the `nginx-error.log` file, and limits messages to a severity level of **emerg**: ```nginx error_log /var/log/nginx/nginx-error.log emerg; ``` Alternatively, you can disable error logs completely with the following line: ```nginx error_log /dev/null; ``` To learn more about how to specify `error_log` in different configuration levels, see the documentation of the [error_log](https://nginx.org/en/docs/ngx_core_module.html?#error_log) directive. ## Setting up access logs NGINX access logs are disabled by default. You can enable access logs by adding **access_log** directives to your NGINX configuration to specify the location of the logs and formats. The log path should always be configured to be inside **/var/log/nginx**. ```nginx http { log_format myfmt '$remote_addr - $remote_user [$time_local] ' '"$request" $status $body_bytes_sent ' '"$http_referer" "$http_user_agent" "$gzip_ratio"'; access_log /var/log/nginx/nginx-access.log myfmt; # ... } ``` **Note:** The **$time_local** variable includes the date and time for each log. It helps with ordering logs after export. To explicitly disable access logs, apply the following config: ```nginx http { access_log off; } ``` or ```nginx http { access_log /dev/null; } ``` To learn more about how to specify `access_log` in different configuration levels and their effect, see [access_log](https://nginx.org/en/docs/http/ngx_http_log_module.html#access_log) **Note:** Keep NGINX logs in the **/var/log/nginx** directory. Otherwise, you may lose data from your logs. ## Setting up F5 WAF for NGINX security logs You can enable security logs by adding **app_protect_security_log** directives to your NGINX configuration to specify the location of the logs and logging formats. The log path should always be configured under **/var/log/app_protect**. ```nginx app_protect_security_log_enable on; app_protect_security_log log_default /var/log/app_protect/security.log; ``` NGINXaaS does not support custom logging profiles and is limited to the [default logging profiles](/waf/logging/logs-overview.md#default-logging-profile-bundles). **Note:** WAF logs should always be stored under the **/var/log/app_protect** directory. You may lose logging data if you choose any other log paths. ## Export NGINX logs to a Google Cloud Project To enable sending logs to your desired Google Cloud project, you must specify the project ID when creating or updating a deployment. To create a deployment, see [our documentation on creating an NGINXaaS deployment](/nginxaas/google/deploy/create-deployment/) for a step-by-step guide. To update the deployment, in the NGINXaaS console, 1. On the left menu, select **Deployments**. 1. Select the deployment you want to update and select **Edit**. 1. Enter the project you want metrics to be sent to under **Log Project ID**. 1. Select **Update**. ## View NGINX logs in Google Cloud Logging In the [Google Cloud Console](https://console.cloud.google.com/), 1. Go to your log project. 2. Search for "Logs Explorer". Refer to the [Google's Logs Explorer](https://cloud.google.com/logging/docs/view/logs-explorer-interface) documentation to learn how you can create queries. NGINX access and error logs sent to Cloud Logging will have the log name `nginx-logs` which can be used to filter NGINX logs from the rest of your project logs. You can also filter based on log labels, for example, * `filename` * `nginxaas_organization_object_id` * `nginxaas_deployment_location` * `nginxaas_deployment_name` * `nginxaas_deployment_object_id` * `nginxaas_namespace` ## Disable Exporting NGINX logs to a Google Cloud Project To disable sending logs to your Google Cloud project, update your NGINXaaS deployment to remove the reference to your project ID. To update the deployment, in the NGINXaaS console, 1. On the navigation menu, select **Deployments**. 1. Select the deployment you want to update and select **Edit**. 1. Remove the project ID under **Log Project ID**. 1. Select **Update**. ## Troubleshooting If Google Cloud Logging is not showing any logs, check for **Failed Log Export to Google** events from your NGINXaaS deployment. In the NGINXaaS console: 1. On the navigation menu, select **Events**. 1. Select **Add Filter**. 1. Select **Affected Object** and the name of your NGINXaaS deployment. Events are deleted after 14 days. -------------------------------------------------------------------------------- title: "User login" url: https://docs.nginx.com/nginxaas/overview/user-login/index.md -------------------------------------------------------------------------------- ## Overview NGINXaaS authenticates users through three login methods: - Microsoft social login - Google social login - Email and password If a user signs in through Microsoft social or Google social, they interact with [Google authentication](https://docs.cloud.google.com/architecture/identity/overview-google-authentication) or the [Microsoft identity platform](https://learn.microsoft.com/en-us/entra/identity-platform/v2-overview) to establish their identity. Alternatively, the user can register a email address and password directly with our service. ## Sign up with email and password If a user wishes to register with an email address and password, they must follow these steps: 1. Select **Sign In** on the main console page. 1. On the login page, select **Sign up** in the email address input form. 1. Enter a valid email address and select **Continue**. 1. Check your email account for a six-digit verification code. 1. Enter the verification code in the browser. 1. Choose a password and select **Continue**. 1. Authenticate with our service using the email and password you have chosen. ## Password reset If you want to reset your password, follow these steps: 1. Select **Sign In** on the main console page. 1. On the login page, enter email address and select **Continue**. 1. On the password entry page, select **Reset password**. 1. On the next page, confirm the email address and select **Continue**. 1. Check your email account for a six-digit verification code. 1. Enter the verification code in the browser. 1. Enter a new password and then select **Reset password**. 1. Use the new password to authenticate with our service. ## Password standards A valid password must contain: - At least 14 characters - No more than 2 identical characters in a row - At least one special character (!@#$%^&*) - At least one lower case character (a-z), one upper case character (A-Z) and one number (0-9) ## Entra configuration for Microsoft social login Depending on the configuration of an Entra tenant, administrators may need to follow these steps before users can log in with F5's Entra app and the Microsoft identity platform. 1. Sign in to the service on the main console page 1. Select **Continue with Microsoft** 1. After signing in with Microsoft, on the **Permissions requested** prompt for the **F5 - Inc Social Login** Entra app, select **Accept** 1. At this point, you can abort the login to NGINXaaS if you wish 1. Go to the Azure portal and go to **Entra** -> **Enterprise Applications** 1. Select the **F5 Inc - Social Login** Entra app 1. Under **Permissions**, select **Grant admin consent** 1. Users of your Entra tenant should now be able to use Microsoft social login to authenticate with NGINXaaS See [Entra guidance on granting admin consent](https://learn.microsoft.com/en-us/entra/identity/enterprise-apps/grant-admin-consent?pivots=portal) for further details. - The app ID (client ID) of the **F5 Inc - Social Login** Entra app is `4aef7a9d-810c-4746-8a6a-6c5d43e4e5f2`. - The tenant ID is `dd3dfd2f-6a3b-40d1-9be0-bf8327d81c50`. - The app requires the following Microsoft Graph API permissions: `email`, `openid`, `profile` and `User.Read`. -------------------------------------------------------------------------------- title: "Metrics catalog" url: https://docs.nginx.com/nginxaas/aws/monitoring/metrics-catalog/index.md description: Reference catalog of Amazon CloudWatch metrics exported by F5 NGINXaaS for AWS deployments. -------------------------------------------------------------------------------- ## Overview F5 NGINXaaS for AWS provides a rich set of metrics that you can use to monitor the health and performance of your NGINXaaS deployment. This document provides a catalog of the metrics that are available for monitoring NGINXaaS for AWS. NGINXaaS exports these metrics to Amazon CloudWatch using the [Embedded Metric Format (EMF)](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format.html). CloudWatch extracts them into metrics under the `NGINXaaS` namespace. See [Enable monitoring](/nginxaas/aws/monitoring/enable-monitoring.md) to configure metric export. The following metrics are reported by NGINXaaS for AWS in Amazon CloudWatch under the `NGINXaaS` namespace. The labels allow you to filter or split your queries in Amazon CloudWatch providing you with a granular view over the metrics reported. ## Metrics - [NGINX config statistics](#nginx-config-statistics) - [NGINX connections statistics](#nginx-connections-statistics) - [NGINX requests and response statistics](#nginx-requests-and-response-statistics) - [NGINX SSL statistics](#nginx-ssl-statistics) - [NGINX cache statistics](#nginx-cache-statistics) - [NGINX memory statistics](#nginx-memory-statistics) - [NGINX upstream statistics](#nginx-upstream-statistics) - [NGINX stream statistics](#nginx-stream-statistics) ### NGINX config statistics All NGINXaaS deployments collect these metrics automatically. No additional NGINX configuration is required. | Metric | Labels | Type | Description | Roll-up per | | ------ | ------ | ---- | ----------- | ------------| | nginx.config.reloads | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location | count | The total number of NGINX configuration reloads since NGINX was last started. | deployment | ### NGINX connections statistics All NGINXaaS deployments collect these metrics automatically. No additional NGINX configuration is required. | Metric | Labels | Type | Description | Roll-up per | | ------ | ------ | ---- | ----------- | ------------| | nginx.http.connections | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_connections_outcome | count | The total number of client connections since NGINX was last started, categorized by outcome (accepted, active, dropped, idle). | deployment | | nginx.http.connection.count | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_connections_outcome | gauge | The current number of client connections, categorized by outcome (accepted, active, dropped, idle). | deployment | ### NGINX requests and response statistics To collect these metrics, configure the `status_zone` directive in your NGINX configuration. Add a `status_zone` directive to your `server` or `location` blocks to enable zone-specific request and response tracking. Example: ```nginx server { listen 80; status_zone my_server_zone; location / { proxy_pass http://backend; } } ``` | Metric | Labels | Type | Description | Roll-up per | | ------ | ------ | ---- | ----------- | ------------| | nginx.http.request.count | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_zone_type | gauge | The total number of client requests received since the last collection interval. | zone | | nginx.http.requests | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_zone_type | count | The total number of client requests received since NGINX was last started or reloaded. | zone | | nginx.http.responses | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_zone_type | count | The total number of HTTP responses sent to clients since NGINX was last started or reloaded. | zone | | nginx.http.response.count | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_status_range, nginx_zone_name, nginx_zone_type | gauge | The total number of HTTP responses sent to clients since the last collection interval, grouped by status code range. | zone | | nginx.http.response.status | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_status_range, nginx_zone_name, nginx_zone_type | count | The total number of responses since NGINX was last started or reloaded, grouped by status code range. | zone | | nginx.http.request.processing.count | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_zone_type | gauge | The number of client requests that are currently being processed. | zone | | nginx.http.request.discarded | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_zone_type | count | The total number of requests completed without sending a response. | zone | | nginx.http.request.io | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_io_direction, nginx_zone_name, nginx_zone_type | count | The total number of HTTP bytes transferred (receive/transmit). | zone | | nginx.http.limit_conn.requests | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_limit_conn_outcome, nginx_zone_name | count | The total number of connections to an endpoint with a limit_conn directive. | zone | | nginx.http.limit_req.requests | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_limit_req_outcome, nginx_zone_name | count | The total number of requests to an endpoint with a limit_req directive. | zone | ### NGINX SSL statistics NGINX automatically collects these metrics when you configure SSL/TLS in your NGINX deployment. To collect SSL metrics, configure SSL certificates and enable HTTPS listeners in your NGINX configuration. Example: ```nginx server { listen 443 ssl; ssl_certificate /path/to/cert.pem; ssl_certificate_key /path/to/key.pem; } ``` | Metric | Labels | Type | Description | Roll-up per | | ------ | ------ | ---- | ----------- | ------------| | nginx.ssl.handshakes | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_ssl_status, nginx_ssl_handshake_reason | count | The total number of SSL handshakes (successful and failed). | deployment | | nginx.ssl.certificate.verify_failures | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_ssl_verify_failure_reason | count | The total number of SSL certificate verification failures, categorized by reason. | deployment | | nginx.ssl.certificate.expiry.time | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, file_path, public_key_algorithm, serial_number, subject_common_name | gauge | The Unix timestamp, in seconds, at which an SSL/TLS certificate expires. | deployment | **Note:** The `nginx.ssl.certificate.expiry.time` metric reports the expiry time of the certificate as an absolute Unix timestamp. To chart or alert on the time remaining, subtract the current time. For example, in PromQL: `nginx.ssl.certificate.expiry.time - time()`. ### NGINX cache statistics To collect cache metrics, configure caching in your NGINX configuration using the `proxy_cache_path` and `proxy_cache` directives. Example: ```nginx http { proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=my_cache:10m; server { location / { proxy_cache my_cache; proxy_pass http://backend; } } } ``` | Metric | Labels | Type | Description | Roll-up per | | ------ | ------ | ---- | ----------- | ------------| | nginx.cache.bytes_read | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_cache_outcome, nginx_cache_name | count | The total number of bytes read from the cache or proxied server. | cache | | nginx.cache.responses | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_cache_outcome, nginx_cache_name | count | The total number of responses read from the cache or proxied server. | cache | | nginx.cache.memory.limit | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_cache_name | gauge | The limit on the maximum size of the cache specified in the configuration. | cache | | nginx.cache.memory.usage | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_cache_name | gauge | The current size of the cache. | cache | ### NGINX memory statistics These metrics track shared memory zone usage. NGINX automatically collects memory statistics when you configure zones using the `status_zone` directive or other directives that create shared memory zones. | Metric | Labels | Type | Description | Roll-up per | | ------ | ------ | ---- | ----------- | ------------| | nginx.slab.page.free | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name | gauge | The current number of free memory pages in the shared memory zone. | zone | | nginx.slab.page.limit | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name | gauge | The total number of memory pages (free and used) in the shared memory zone. | zone | | nginx.slab.page.usage | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name | gauge | The current number of used memory pages in the shared memory zone. | zone | | nginx.slab.page.utilization | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name | gauge | The current percentage of used memory pages in the shared memory zone. | zone | | nginx.slab.slot.usage | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_slab_slot_limit, nginx_zone_name | gauge | The current number of used memory slots. | zone | | nginx.slab.slot.free | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_slab_slot_limit, nginx_zone_name | gauge | The current number of free memory slots. | zone | | nginx.slab.slot.allocations | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_slab_slot_limit, nginx_slab_slot_allocation_result, nginx_zone_name | count | The number of attempts to allocate memory of specified size. | zone | ### NGINX upstream statistics To collect upstream metrics, define `upstream` blocks in your NGINX configuration and reference them in your proxy configuration. Add `zone` directives to track upstream statistics. Example: ```nginx upstream backend { zone backend_zone 64k; server 10.0.0.1:8080; server 10.0.0.2:8080; } server { status_zone my_server; location / { proxy_pass http://backend; } } ``` | Metric | Labels | Type | Description | Roll-up per | | ------ | ------ | ---- | ----------- | ------------| | nginx.http.upstream.keepalive.count | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name | gauge | The current number of idle keepalive connections per HTTP upstream. | upstream | | nginx.http.upstream.peer.io | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_io_direction, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | count | The total number of bytes transferred per HTTP upstream peer. | peer | | nginx.http.upstream.peer.connection.count | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | gauge | The average number of active connections per HTTP upstream peer. | peer | | nginx.http.upstream.peer.count | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_peer_state, nginx_zone_name, nginx_upstream_name | gauge | The current count of peers on the HTTP upstream grouped by state. | upstream | | nginx.http.upstream.peer.fails | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | count | The total number of unsuccessful attempts to communicate with the HTTP upstream peer. | peer | | nginx.http.upstream.peer.header.time | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | gauge | The average time to get the response header from the HTTP upstream peer. | peer | | nginx.http.upstream.peer.health_checks | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_health_check, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | count | The total number of health check requests made to an HTTP upstream peer. | peer | | nginx.http.upstream.peer.requests | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | count | The total number of client requests forwarded to the HTTP upstream peer. | peer | | nginx.http.upstream.peer.response.time | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | gauge | The average time to get the full response from the HTTP upstream peer. | peer | | nginx.http.upstream.peer.responses | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_status_range, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | count | The total number of responses obtained from the HTTP upstream peer grouped by status range. | peer | | nginx.http.upstream.peer.unavailables | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | count | The total number of times the server became unavailable for client requests. | peer | | nginx.http.upstream.peer.state | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_peer_state, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | gauge | Current state of an upstream peer in deployment (1 if deployed, 0 if not). | peer | | nginx.http.upstream.queue.limit | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name | gauge | The maximum number of requests that can be in the queue at the same time. | upstream | | nginx.http.upstream.queue.overflows | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name | count | The total number of requests rejected due to the queue overflow. | upstream | | nginx.http.upstream.queue.usage | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name | gauge | The current number of requests in the queue. | upstream | | nginx.http.upstream.zombie.count | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name | gauge | The current number of upstream peers removed from the group but still processing active client requests. | upstream | ### NGINX stream statistics To collect stream metrics, configure the `stream` context in your NGINX configuration and add `status_zone` directives to your stream servers. Example: ```nginx stream { server { listen 12345; status_zone tcp_server; proxy_pass backend_stream; } upstream backend_stream { zone stream_backend 64k; server 10.0.0.1:12345; } } ``` | Metric | Labels | Type | Description | Roll-up per | | ------ | ------ | ---- | ----------- | ------------| | nginx.stream.io | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_io_direction, nginx_zone_name | count | The total number of Stream bytes transferred (receive/transmit). | zone | | nginx.stream.connection.accepted | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name | count | The total number of connections accepted from clients. | zone | | nginx.stream.connection.discarded | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name | count | Total number of connections completed without creating a session. | zone | | nginx.stream.connection.processing.count | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name | gauge | The number of client connections that are currently being processed. | zone | | nginx.stream.session.status | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_status_range, nginx_zone_name | count | The total number of completed sessions grouped by status range. | zone | | nginx.stream.upstream.peer.io | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_io_direction, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | count | The total number of Stream upstream peer bytes transferred. | peer | | nginx.stream.upstream.peer.connection.count | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | gauge | The current number of Stream upstream peer connections. | peer | | nginx.stream.upstream.peer.connection.time | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | gauge | The average time to connect to the stream upstream peer. | peer | | nginx.stream.upstream.peer.connections | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | count | The total number of client connections forwarded to this stream upstream peer. | peer | | nginx.stream.upstream.peer.count | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_peer_state, nginx_zone_name, nginx_upstream_name | gauge | The current number of stream upstream peers grouped by state. | upstream | | nginx.stream.upstream.peer.fails | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name, nginx_peer_address | count | The total number of unsuccessful attempts to communicate with the stream upstream peer. | peer | | nginx.stream.upstream.peer.health_checks | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_health_check, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | count | The total number of health check requests made to the stream upstream peer. | peer | | nginx.stream.upstream.peer.response.time | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | gauge | The average time to receive the last byte of data for the stream upstream peer. | peer | | nginx.stream.upstream.peer.ttfb.time | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | gauge | The average time to receive the first byte of data for the stream upstream peer. | peer | | nginx.stream.upstream.peer.unavailables | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | count | How many times the server became unavailable for client connections due to the number of unsuccessful attempts reaching the max_fails threshold. | peer | | nginx.stream.upstream.peer.state | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_peer_state, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | count | Current state of upstream peers in deployment (1 if any peer matches state, 0 if none). | peer | | nginx.stream.upstream.zombie.count | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name | gauge | The current number of peers removed from the group but still processing active client connections. | upstream | ## References For more information, see: - [Enable monitoring](/nginxaas/aws/monitoring/enable-monitoring.md) - [Enable NGINX logs](/nginxaas/aws/monitoring/enable-nginx-logs.md) - [Identity and access management](/nginxaas/aws/deploy/access-management.md) -------------------------------------------------------------------------------- title: "Metrics catalog" url: https://docs.nginx.com/nginxaas/google/monitoring/metrics-catalog/index.md description: Reference catalog of NGINX metrics exported by F5 NGINXaaS for Google Cloud deployments. -------------------------------------------------------------------------------- ## Overview F5 NGINXaaS for Google Cloud (NGINXaaS) provides a rich set of metrics that you can use to monitor the health and performance of your NGINXaaS deployment. This document provides a catalog of the metrics that are available for monitoring NGINXaaS for Google Cloud. The following metrics are reported by NGINXaaS for Google Cloud in Google Cloud Monitoring. The metrics are categorized by the namespace used in Google Cloud Monitoring. The labels allow you to filter or split your queries in Google Cloud Monitoring providing you with a granular view over the metrics reported. ## Metrics - [NGINX config statistics](#nginx-config-statistics) - [NGINX connections statistics](#nginx-connections-statistics) - [NGINX requests and response statistics](#nginx-requests-and-response-statistics) - [NGINX SSL statistics](#nginx-ssl-statistics) - [NGINX cache statistics](#nginx-cache-statistics) - [NGINX memory statistics](#nginx-memory-statistics) - [NGINX upstream statistics](#nginx-upstream-statistics) - [NGINX stream statistics](#nginx-stream-statistics) ### NGINX config statistics All NGINXaaS deployments collect these metrics automatically. No additional NGINX configuration is required. | Metric | Labels | Type | Description | Roll-up per | | ------ | ------ | ---- | ----------- | ------------| | nginx.config.reloads | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location | count | The total number of NGINX configuration reloads since NGINX was last started. | deployment | ### NGINX connections statistics All NGINXaaS deployments collect these metrics automatically. No additional NGINX configuration is required. | Metric | Labels | Type | Description | Roll-up per | | ------ | ------ | ---- | ----------- | ------------| | nginx.http.connections | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_connections_outcome | count | The total number of client connections since NGINX was last started, categorized by outcome (accepted, active, dropped, idle). | deployment | | nginx.http.connection.count | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_connections_outcome | gauge | The current number of client connections, categorized by outcome (accepted, active, dropped, idle). | deployment | ### NGINX requests and response statistics To collect these metrics, configure the `status_zone` directive in your NGINX configuration. Add a `status_zone` directive to your `server` or `location` blocks to enable zone-specific request and response tracking. Example: ```nginx server { listen 80; status_zone my_server_zone; location / { proxy_pass http://backend; } } ``` | Metric | Labels | Type | Description | Roll-up per | | ------ | ------ | ---- | ----------- | ------------| | nginx.http.request.count | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_zone_type | gauge | The total number of client requests received since the last collection interval. | zone | | nginx.http.requests | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_zone_type | count | The total number of client requests received since NGINX was last started or reloaded. | zone | | nginx.http.responses | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_zone_type | count | The total number of HTTP responses sent to clients since NGINX was last started or reloaded. | zone | | nginx.http.response.count | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_status_range, nginx_zone_name, nginx_zone_type | gauge | The total number of HTTP responses sent to clients since the last collection interval, grouped by status code range. | zone | | nginx.http.response.status | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_status_range, nginx_zone_name, nginx_zone_type | count | The total number of responses since NGINX was last started or reloaded, grouped by status code range. | zone | | nginx.http.request.processing.count | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_zone_type | gauge | The number of client requests that are currently being processed. | zone | | nginx.http.request.discarded | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_zone_type | count | The total number of requests completed without sending a response. | zone | | nginx.http.request.io | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_io_direction, nginx_zone_name, nginx_zone_type | count | The total number of HTTP bytes transferred (receive/transmit). | zone | | nginx.http.limit_conn.requests | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_limit_conn_outcome, nginx_zone_name | count | The total number of connections to an endpoint with a limit_conn directive. | zone | | nginx.http.limit_req.requests | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_limit_req_outcome, nginx_zone_name | count | The total number of requests to an endpoint with a limit_req directive. | zone | ### NGINX SSL statistics NGINX automatically collects these metrics when you configure SSL/TLS in your NGINX deployment. To collect SSL metrics, configure SSL certificates and enable HTTPS listeners in your NGINX configuration. Example: ```nginx server { listen 443 ssl; ssl_certificate /path/to/cert.pem; ssl_certificate_key /path/to/key.pem; } ``` | Metric | Labels | Type | Description | Roll-up per | | ------ | ------ | ---- | ----------- | ------------| | nginx.ssl.handshakes | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_ssl_status, nginx_ssl_handshake_reason | count | The total number of SSL handshakes (successful and failed). | deployment | | nginx.ssl.certificate.verify_failures | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_ssl_verify_failure_reason | count | The total number of SSL certificate verification failures, categorized by reason. | deployment | | nginx.ssl.certificate.expiry.time | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, file_path, public_key_algorithm, serial_number, subject_common_name | gauge | The Unix timestamp, in seconds, at which an SSL/TLS certificate expires. | deployment | **Note:** The `nginx.ssl.certificate.expiry.time` metric reports the expiry time of the certificate as an absolute Unix timestamp. To chart or alert on the time remaining, subtract the current time. For example, in PromQL: `nginx.ssl.certificate.expiry.time - time()`. ### NGINX cache statistics To collect cache metrics, configure caching in your NGINX configuration using the `proxy_cache_path` and `proxy_cache` directives. Example: ```nginx http { proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=my_cache:10m; server { location / { proxy_cache my_cache; proxy_pass http://backend; } } } ``` | Metric | Labels | Type | Description | Roll-up per | | ------ | ------ | ---- | ----------- | ------------| | nginx.cache.bytes_read | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_cache_outcome, nginx_cache_name | count | The total number of bytes read from the cache or proxied server. | cache | | nginx.cache.responses | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_cache_outcome, nginx_cache_name | count | The total number of responses read from the cache or proxied server. | cache | | nginx.cache.memory.limit | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_cache_name | gauge | The limit on the maximum size of the cache specified in the configuration. | cache | | nginx.cache.memory.usage | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_cache_name | gauge | The current size of the cache. | cache | ### NGINX memory statistics These metrics track shared memory zone usage. NGINX automatically collects memory statistics when you configure zones using the `status_zone` directive or other directives that create shared memory zones. | Metric | Labels | Type | Description | Roll-up per | | ------ | ------ | ---- | ----------- | ------------| | nginx.slab.page.free | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name | gauge | The current number of free memory pages in the shared memory zone. | zone | | nginx.slab.page.limit | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name | gauge | The total number of memory pages (free and used) in the shared memory zone. | zone | | nginx.slab.page.usage | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name | gauge | The current number of used memory pages in the shared memory zone. | zone | | nginx.slab.page.utilization | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name | gauge | The current percentage of used memory pages in the shared memory zone. | zone | | nginx.slab.slot.usage | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_slab_slot_limit, nginx_zone_name | gauge | The current number of used memory slots. | zone | | nginx.slab.slot.free | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_slab_slot_limit, nginx_zone_name | gauge | The current number of free memory slots. | zone | | nginx.slab.slot.allocations | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_slab_slot_limit, nginx_slab_slot_allocation_result, nginx_zone_name | count | The number of attempts to allocate memory of specified size. | zone | ### NGINX upstream statistics To collect upstream metrics, define `upstream` blocks in your NGINX configuration and reference them in your proxy configuration. Add `zone` directives to track upstream statistics. Example: ```nginx upstream backend { zone backend_zone 64k; server 10.0.0.1:8080; server 10.0.0.2:8080; } server { status_zone my_server; location / { proxy_pass http://backend; } } ``` | Metric | Labels | Type | Description | Roll-up per | | ------ | ------ | ---- | ----------- | ------------| | nginx.http.upstream.keepalive.count | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name | gauge | The current number of idle keepalive connections per HTTP upstream. | upstream | | nginx.http.upstream.peer.io | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_io_direction, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | count | The total number of bytes transferred per HTTP upstream peer. | peer | | nginx.http.upstream.peer.connection.count | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | gauge | The average number of active connections per HTTP upstream peer. | peer | | nginx.http.upstream.peer.count | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_peer_state, nginx_zone_name, nginx_upstream_name | gauge | The current count of peers on the HTTP upstream grouped by state. | upstream | | nginx.http.upstream.peer.fails | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | count | The total number of unsuccessful attempts to communicate with the HTTP upstream peer. | peer | | nginx.http.upstream.peer.header.time | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | gauge | The average time to get the response header from the HTTP upstream peer. | peer | | nginx.http.upstream.peer.health_checks | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_health_check, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | count | The total number of health check requests made to an HTTP upstream peer. | peer | | nginx.http.upstream.peer.requests | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | count | The total number of client requests forwarded to the HTTP upstream peer. | peer | | nginx.http.upstream.peer.response.time | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | gauge | The average time to get the full response from the HTTP upstream peer. | peer | | nginx.http.upstream.peer.responses | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_status_range, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | count | The total number of responses obtained from the HTTP upstream peer grouped by status range. | peer | | nginx.http.upstream.peer.unavailables | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | count | The total number of times the server became unavailable for client requests. | peer | | nginx.http.upstream.peer.state | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_peer_state, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | gauge | Current state of an upstream peer in deployment (1 if deployed, 0 if not). | peer | | nginx.http.upstream.queue.limit | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name | gauge | The maximum number of requests that can be in the queue at the same time. | upstream | | nginx.http.upstream.queue.overflows | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name | count | The total number of requests rejected due to the queue overflow. | upstream | | nginx.http.upstream.queue.usage | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name | gauge | The current number of requests in the queue. | upstream | | nginx.http.upstream.zombie.count | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name | gauge | The current number of upstream peers removed from the group but still processing active client requests. | upstream | ### NGINX stream statistics To collect stream metrics, configure the `stream` context in your NGINX configuration and add `status_zone` directives to your stream servers. Example: ```nginx stream { server { listen 12345; status_zone tcp_server; proxy_pass backend_stream; } upstream backend_stream { zone stream_backend 64k; server 10.0.0.1:12345; } } ``` | Metric | Labels | Type | Description | Roll-up per | | ------ | ------ | ---- | ----------- | ------------| | nginx.stream.io | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_io_direction, nginx_zone_name | count | The total number of Stream bytes transferred (receive/transmit). | zone | | nginx.stream.connection.accepted | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name | count | The total number of connections accepted from clients. | zone | | nginx.stream.connection.discarded | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name | count | Total number of connections completed without creating a session. | zone | | nginx.stream.connection.processing.count | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name | gauge | The number of client connections that are currently being processed. | zone | | nginx.stream.session.status | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_status_range, nginx_zone_name | count | The total number of completed sessions grouped by status range. | zone | | nginx.stream.upstream.peer.io | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_io_direction, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | count | The total number of Stream upstream peer bytes transferred. | peer | | nginx.stream.upstream.peer.connection.count | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | gauge | The current number of Stream upstream peer connections. | peer | | nginx.stream.upstream.peer.connection.time | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | gauge | The average time to connect to the stream upstream peer. | peer | | nginx.stream.upstream.peer.connections | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | count | The total number of client connections forwarded to this stream upstream peer. | peer | | nginx.stream.upstream.peer.count | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_peer_state, nginx_zone_name, nginx_upstream_name | gauge | The current number of stream upstream peers grouped by state. | upstream | | nginx.stream.upstream.peer.fails | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name, nginx_peer_address | count | The total number of unsuccessful attempts to communicate with the stream upstream peer. | peer | | nginx.stream.upstream.peer.health_checks | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_health_check, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | count | The total number of health check requests made to the stream upstream peer. | peer | | nginx.stream.upstream.peer.response.time | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | gauge | The average time to receive the last byte of data for the stream upstream peer. | peer | | nginx.stream.upstream.peer.ttfb.time | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | gauge | The average time to receive the first byte of data for the stream upstream peer. | peer | | nginx.stream.upstream.peer.unavailables | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | count | How many times the server became unavailable for client connections due to the number of unsuccessful attempts reaching the max_fails threshold. | peer | | nginx.stream.upstream.peer.state | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_peer_state, nginx_zone_name, nginx_upstream_name, nginx_peer_address, nginx_peer_name | count | Current state of upstream peers in deployment (1 if any peer matches state, 0 if none). | peer | | nginx.stream.upstream.zombie.count | nginxaas_organization_object_id, nginxaas_namespace, nginxaas_deployment_object_id, nginxaas_deployment_name, nginxaas_deployment_location, nginx_zone_name, nginx_upstream_name | gauge | The current number of peers removed from the group but still processing active client connections. | upstream | ## References For more information, see: - [Enable monitoring](/nginxaas/google/monitoring/enable-monitoring.md) - [Enable NGINX logs](/nginxaas/google/monitoring/enable-nginx-logs.md) - [Identity and access management](/nginxaas/google/deploy/access-management.md) -------------------------------------------------------------------------------- title: "Programmatic authentication with client credentials" url: https://docs.nginx.com/nginxaas/overview/client-credentials/index.md description: Learn how to set up OAuth2 client credentials for programmatic access to the NGINXaaS API. -------------------------------------------------------------------------------- ## Overview This guide explains how to create and use client credentials for automating access to NGINXaaS APIs. Client credentials enable automation tools such as CI/CD pipelines to manage certain NGINXaaS resources without requiring user login. Client credentials work the same way regardless of which cloud your NGINXaaS organization is associated with. To authenticate, you exchange your client credentials (client ID and secret) for a short-lived access token from the NGINXaaS token endpoint. This access token is then used in the Authorization header of your API requests. Access tokens have limited validity, after which you'll need to request a new one using the same credentials. Client credentials are scoped to an organization and expire after a set period (up to 1 year, 6 months recommended). ## Before you begin - You must be logged in to the [NGINXaaS Console](https://console.nginxaas.net/). ## Create client credentials Follow these steps to create a new client credential through the NGINXaaS console: 1. Log in to [NGINXaaS Console](https://console.nginxaas.net/). 1. Select **Settings** > **Client Credentials** from the left navigation menu. 1. Select **+ Add Client** to create a new credential. 1. Enter a unique name for your client in the text field that appears. 1. Choose an expiration date (maximum 1 year from today; 6 months is recommended). This setting is immutable after creation. 1. Select **Create**. 1. A popup window displays your client secret. **Note:** The client secret appears only once. Save it immediately in a secure location, such as a password manager or secrets vault. Your client credentials can access the Certificates, Configs, and Deployments resources for your NGINXaaS organization. ### Client limits Organizations are limited to a maximum of 10 client credentials. To request an increase to this limit, contact the [NGINX Support](/nginxaas/support.md). ## Retrieve client information Follow these steps to view information about your clients: 1. Log in to [NGINXaaS Console](https://console.nginxaas.net/). 1. Select **Settings** > **Client Credentials** from the left navigation menu. 1. You can see all the available client credentials for your organization in the **Client Credentials** page. The table shows the following metadata for each credential: - Client Name - Client ID - Token Endpoint - Date Created - Expires On 1. Check the expiration status indicator next to **Expires On**: - **Green dot**: Credential is active and valid - **Red dot**: Credential has expired **Note:** You can only view the client secret once, when they're created. If you need to use credentials again, you must have saved the secret when you initially created the client. ## Delete client credentials Follow these steps to delete a client credential: 1. Log in to [NGINXaaS Console](https://console.nginxaas.net/). 1. Select **Settings** > **Client Credentials** from the left navigation menu. 1. Locate the credential to delete in the table. 1. Select the three vertical dots icon (⋮) at the end of the row. 1. Select **Delete** from the menu. After deletion: - Existing access tokens continue to work for 1 full hour after they're issued. **Note:** Expired credentials are not automatically removed. You must manually delete credentials that are past their expiration date. ## Use client credentials for authentication Learn how to obtain and use access tokens from your client credentials to authenticate API requests. **Note:** Client credential access to these resources isn’t officially supported yet. ### Exchange credentials for an access token Use the client credentials to obtain an access token from the token endpoint. **Endpoint**: `POST https://.api.nginxaas.net/api/v1/auth/token` **Example using cURL**: ```bash curl -X POST "https://.api.nginxaas.net/api/v1/auth/token" \ -H "Content-Type: application/json" \ -d '{ "client_id": "", "client_secret": "", "grant_type": "client_credentials" }' ``` **Response**: ```json { "access_token": "", "token_type": "Bearer", "expires_in": 3600 } ``` ### Use the access token Include the access token in the Authorization header when making API requests. **Example**: ```bash curl -X GET "https://.api.nginxaas.net/api/v1/deployments" \ -H "Authorization: Bearer " ``` ### Access token validity - **Duration**: Access tokens are valid for 1 hour from issuance - **Reauthentication**: When a token expires, request a new one using the same client credentials, if the client credentials are still active - **Scope**: Tokens are also scoped to the organization associated with the client credentials ## Security best practices - **Store secrets securely**: Store client credentials in a secure place - **Delete unused credentials**: Remove clients that are no longer needed - **Follow the recommended expiration**: Avoid using clients with a very long expiration, a good default to begin with is 6 months ## Troubleshooting Common issues when authenticating with client credentials and how to resolve them. ### Invalid or expired credentials If you attempt to authenticate with invalid or expired credentials, you will receive a `401 Unauthorized` response. In this case: - Verify your client ID and secret are correct - Check if the client has expired - Create new credentials if necessary **Example error response**: ```json { "error": "unauthorized", "error_description": "token is expired/invalid" } ``` ### Forbidden APIs Client credentials can only access the Certificates, Configs, and Deployments APIs. Attempting to access any other API with client credentials returns a `403 Forbidden` response. In this case: - Verify you are using the correct API endpoint - Ensure the API is one of the supported resources **Example error response**: ```json { "error": "forbidden", "error_description": "Client credentials do not have permission to access this resource" } ``` ## Quick reference | Property | Value | |----------|-------| | Client secret visibility | Only shown once during creation | | Default expiration | 6 months (recommended) | | Maximum expiration | 1 year | | Client limit per organization | 10 clients contact [NGINX Support](/nginxaas/support.md) to increase | | Access token validity | 1 hour | | Supported resources | Deployments, Configs, Certificates | | Token endpoint | `https://.api.nginxaas.net/api/v1/auth/token` | -------------------------------------------------------------------------------- title: "Identity and access management" url: https://docs.nginx.com/nginxaas/aws/deploy/access-management/index.md description: Configure AWS IAM roles so F5 NGINXaaS for AWS can access CloudWatch and Secrets Manager on your behalf. -------------------------------------------------------------------------------- F5 NGINXaaS for AWS uses AWS Identity and Access Management (IAM) roles to integrate with AWS services. An NGINXaaS deployment configured with an AWS IAM role has access to the following capabilities: - Export logs to CloudWatch Logs - Export metrics to CloudWatch Logs using EMF - Fetch secrets from AWS Secrets Manager to use in your NGINX Configuration NGINXaaS acts as a third party accessing your AWS account. To prevent the [confused deputy problem](https://docs.aws.amazon.com/IAM/latest/UserGuide/confused-deputy.html), NGINXaaS uses an external ID when assuming your provided IAM role. To learn more about this pattern, see [AWS documentation on third-party access](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_common-scenarios_third-party.html). The external ID is always your NGINXaaS Organization ID. ## Prerequisites - An AWS account with permissions to create IAM roles and policies - Your NGINXaaS organization ID, available in the **Organization Info** section of the **Organization Details** page (**Settings** > **Organization Details** on the navigation menu) ## IAM role options You have flexibility in how to structure IAM roles for your NGINXaaS deployments. Choose an architecture based on your security requirements and operational needs. ### Option 1: Single role for all deployments with shared policies One IAM role is shared by all deployments, with access policies that apply to all deployments. **Pros:** - Simple to set up and maintain - Fewer roles to manage in your AWS account - Easier onboarding for new deployments **Cons:** - Cannot differentiate permissions between deployments - All deployments have access to the same resources - Less granular security boundaries **Best for:** Development environments, proof-of-concept deployments, or when all deployments require identical permissions. ### Option 2: Single role for all deployments with per-deployment policies One IAM role is shared by all deployments, but access policies are restricted to specific deployments using the deployment ID in conditions. **Pros:** - Fewer roles to manage in your AWS account - Each deployment can have tailored permissions - Policies can restrict access per deployment using [session tags](/nginxaas/aws/deploy/ssl-tls-certificates/ssl-tls-certificates-secrets-manager.md#iam-role-permissions-policy) or principal conditions **Cons:** - More complex policy conditions - All deployments must use the same role ARN - Policy updates affect all deployments **Best for:** Multi-environment setups where you want per-deployment resource access control without managing multiple roles. ### Option 3: One role per deployment Each deployment has its own dedicated IAM role with tailored access policies. **Pros:** - Maximum granularity and security isolation - Each deployment can have tailored permissions - Clear 1:1 mapping between role and deployment simplifies auditing and access revocation **Cons:** - More roles to manage in your AWS account - More setup and maintenance overhead - Policy duplication if deployments have similar needs **Best for:** Production environments, strict security requirements, or when deployments require significantly different permissions. ## Key concepts ### External ID NGINXaaS provides your organization with a unique **External ID** (your organization ID). This ID is used by the NGINXaaS dataplane to assume your IAM role. The External ID prevents unauthorized access by binding the trust relationship to your specific organization. ## How it works NGINXaaS periodically calls the AWS [AssumeRole](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html) API to obtain temporary credentials for your IAM role. These credentials are valid for the duration of the assume role session (minimum 15 minutes) and are renewed before they expire. Be aware of the following propagation delays when making changes: - **Updating a role's trust policy**: Because credentials are cached for the assume role session duration, trust policy changes can take up to 15 minutes to take effect. - **Updating a role's permissions policy**: Permissions policy changes take effect almost immediately. ## Configure IAM roles ### Step 1: Create a trust policy Create a JSON file named `trust-policy.json` with the following content. Replace `$ORG_ID` with your organization ID, `$DEPLOYMENT_AWS_ACCOUNT_ID` with the NGINXaaS AWS Account ID from the **Deployment Data** section of the **Details** tab, and `$DEPLOYMENT_ID` with your deployment ID (Option 3 only): #### Trust policy for Option 1 or 2 (shared role) ```json { "Version": "2012-10-17", "Statement": [ { "Sid": "AllowNGINXaaSDeploymentToAssumeRole", "Effect": "Allow", "Principal": { "AWS": ["254908360701", "207423186715", "785021966578", "775935274660", "275859940910"] }, "Action": "sts:AssumeRole", "Condition": { "StringEquals": { "sts:ExternalId": "$ORG_ID" } } }, { "Sid": "AllowTagging", "Effect": "Allow", "Principal": { "AWS": ["254908360701", "207423186715", "785021966578", "775935274660", "275859940910"] }, "Action": "sts:TagSession" } ] } ``` #### Trust policy for Option 3 (per-deployment role) For per-deployment roles, restrict the **Principal** to the specific IAM role ARN that NGINXaaS uses for your deployment: ```json { "Version": "2012-10-17", "Statement": [ { "Sid": "AllowNGINXaaSDeploymentToAssumeRole", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::$DEPLOYMENT_AWS_ACCOUNT_ID:role/$DEPLOYMENT_ID" }, "Action": "sts:AssumeRole", "Condition": { "StringEquals": { "sts:ExternalId": "$ORG_ID" } } }, { "Sid": "AllowTagging", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::$DEPLOYMENT_AWS_ACCOUNT_ID:role/$DEPLOYMENT_ID" }, "Action": "sts:TagSession" } ] } ``` ### Step 2: Create the IAM role To create the IAM role, follow [Creating a role with a custom trust policy](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-custom.html) in the AWS documentation, using the trust policy JSON from Step 1. Record the role **ARN** for use in your NGINXaaS deployment configuration. Alternatively, to use the AWS CLI, replace `$ROLE_NAME` with a descriptive name (for example, `NGINXaaS-CloudWatch-Role`): ```bash aws iam create-role --role-name $ROLE_NAME \ --assume-role-policy-document file://trust-policy.json ``` **Example output:** ```json { "Role": { "Path": "/", "RoleName": "NGINXaaS-CloudWatch-Role", "RoleId": "AIDA...", "Arn": "arn:aws:iam::123456789012:role/NGINXaaS-CloudWatch-Role", "CreateDate": "2024-01-15T10:30:00+00:00", "AssumeRolePolicyDocument": { ... } } } ``` Record the role ARN for use in your NGINXaaS deployment configuration. ### Step 3: Add inline policies to your role Add inline policies to your role to grant the permissions your deployment requires. The following sections provide example policies for common use cases. #### Policy for CloudWatch Logs For a full guide on monitoring and logging with NGINXaaS, see [Enable monitoring](/nginxaas/aws/monitoring/enable-monitoring.md) and [Enable NGINX logs](/nginxaas/aws/monitoring/enable-nginx-logs.md). To add this policy in the AWS Management Console, follow [Adding inline policies](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_manage-attach-detach.html#add-policies-console) in the AWS documentation. Use the JSON below and name the policy `nginxaas-cloudwatch-logs`. ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents" ], "Resource": "arn:aws:logs:*:*:log-group:nginxaas/*" } ] } ``` Alternatively, to use the AWS CLI: ```bash aws iam put-role-policy --role-name $ROLE_NAME \ --policy-name nginxaas-cloudwatch-logs \ --policy-document '{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents" ], "Resource": "arn:aws:logs:*:*:log-group:nginxaas/*" } ] }' ``` #### Policy for Secrets Manager Replace `$SECRET_ARN` with the ARN of your secret. For a full guide on using AWS Secrets Manager with NGINXaaS, see [Add certificates from AWS Secrets Manager](/nginxaas/aws/deploy/ssl-tls-certificates/ssl-tls-certificates-secrets-manager.md). To add this policy in the AWS Management Console, follow [Adding inline policies](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_manage-attach-detach.html#add-policies-console) in the AWS documentation. Use the JSON below and name the policy `nginxaas-secrets-manager`. ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "secretsmanager:GetSecretValue" ], "Resource": "$SECRET_ARN" } ] } ``` Alternatively, to use the AWS CLI: ```bash aws iam put-role-policy --role-name $ROLE_NAME \ --policy-name nginxaas-secrets-manager \ --policy-document '{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "secretsmanager:GetSecretValue" ], "Resource": "$SECRET_ARN" } ] }' ``` ### Step 4: Add the role ARN to your NGINXaaS deployment In the NGINXaaS Console: 1. On the navigation menu, select **Deployments**. 1. Select the deployment you want to update and select **Edit**. 1. Enter your role ARN, for example, `arn:aws:iam::123456789012:role/NGINXaaS-CloudWatch-Role`, in the **Role ARN** field under the **Identity** section. 1. Select **Save Changes**. ## Monitor role assumption events NGINXaaS for AWS generates an event when it fails to assume your IAM role. Use these events to diagnose trust policy or permission issues. ### Event types | Event type | Description | |---|---| | AWS role assumption failed | NGINXaaS couldn't assume the IAM role configured for your deployment. The event message includes the error details. | ### View events in the console - Select **Overview** in the left menu, then select **Events**. To narrow results to a specific deployment, filter by its object ID using the controls at the top of the page. - For a summary of recent events for a specific deployment, select **Deployments**, select the deployment, and look for the **Recent Events** card. Select **See Events Details** to go to the full Events page pre-filtered for that deployment. ### Common failure messages and remediation | Message | Likely cause | Remediation | |---|---|---| | `The AWS IAM Role hasn't been set up correctly.` | The role ARN does not exist in the customer account, the trust policy's `sts:ExternalId` condition does not match the NGINXaaS organization ID, or (Option 3 only) the Principal in the trust policy does not match the deployment's IAM role ARN. AWS returns the same error for all cases. | Confirm the role ARN in your deployment's **Identity** section refers to an existing IAM role. If the role exists, verify the `sts:ExternalId` condition matches your NGINXaaS organization ID. For Option 3, also verify the trust policy Principal matches the deployment's IAM role ARN (`arn:aws:iam::$DEPLOYMENT_AWS_ACCOUNT_ID:role/$DEPLOYMENT_ID`). | ## What's next - [Monitor your deployment](/nginxaas/aws/monitoring/enable-monitoring.md) - [Enable NGINX Logs](/nginxaas/aws/monitoring/enable-nginx-logs.md) - [Add certificates from AWS Secrets Manager](/nginxaas/aws/deploy/ssl-tls-certificates/ssl-tls-certificates-secrets-manager.md) - [AWS IAM User Guide](https://docs.aws.amazon.com/iam/) - [AWS IAM Best Practices](https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html) -------------------------------------------------------------------------------- title: "Identity and access management" url: https://docs.nginx.com/nginxaas/google/deploy/access-management/index.md -------------------------------------------------------------------------------- F5 NGINXaaS for Google Cloud (NGINXaaS) uses Workload Identity Federation (WIF) to integrate with Google Cloud services. For example, with WIF configured, your NGINXaaS deployment can perform the following integrations: - export logs to Cloud Logging - export metrics to Cloud Monitoring - fetch secrets from Secret Manager To learn more, see [Google's Workload Identity Federation documentation](https://cloud.google.com/iam/docs/workload-identity-federation). ## Prerequisites - In the project you're configuring WIF in, you need the following roles to create a workload identity pool, provider, and policy bindings: - [iam.workloadIdentityPoolAdmin](https://cloud.google.com/iam/docs/roles-permissions/iam#iam.workloadIdentityPoolAdmin) - [resourcemanager.projectIamAdmin](https://cloud.google.com/iam/docs/roles-permissions/resourcemanager#resourcemanager.projectIamAdmin) - An NGINXaaS deployment. See [our documentation on creating an NGINXaaS deployment](/nginxaas/google/deploy/create-deployment/) for a step-by-step guide. ## Configure WIF ### Create a Workload Identity Pool and Provider 1. Create a workload identity pool. See [Google's documentation on configuring Workload Identity Federation](https://cloud.google.com/iam/docs/workload-identity-federation-with-other-providers#create-pool-provider) for a step-by-step guide. 1. Create an OIDC workload identity pool provider. See [Google's documentation on creating a workload identity pool provider](https://cloud.google.com/iam/docs/workload-identity-federation-with-other-providers#create-pool-provider) for a step-by-step guide. Set up the provider settings as follows: - `Issuer URL` must be `https://accounts.google.com`. - `Allowed audiences` must contain the full canonical resource name of the workload identity pool provider, for example, `https://iam.googleapis.com/projects//locations//workloadIdentityPools//providers/`. If `Allowed audiences` is empty, the full canonical resource name of the workload identity pool provider will be included by default. - Add the following **attribute mapping**: `google.subject=assertion.sub`. - Add the following **attribute condition**: `assertion.sub=='$NGINXAAS_SERVICE_ACCOUNT_UNIQUE_ID'`, where `$NGINXAAS_SERVICE_ACCOUNT_UNIQUE_ID` is the unique ID of your NGINXaaS deployment's service account. This ID can be found in the `F5 NGINXaaS Service Account Unique ID` field under the **Cloud Info** section in the **Details** tab of your deployment. ### Grant access to the WIF principal with your desired roles Depending on your use case, you will need to grant certain roles on specific resources. See [Google's documentation on granting access](https://cloud.google.com/iam/docs/workload-identity-federation-with-other-providers#access) for more information. #### Grant access to export logs to a Google project To grant access to export logs to a Google project, `$LOG_PROJECT_ID`, in the [Google Cloud Console](https://console.cloud.google.com/), 1. Go to the `$LOG_PROJECT_ID` project. 1. Go to the **IAM** page. 1. Select **Grant Access**. 1. Enter your principal, for example, `principal://iam.googleapis.com/projects/$WIF_PROJECT_NUMBER/locations/global/workloadIdentityPools/$WIF_POOL_ID/subject/$NGINXAAS_SERVICE_ACCOUNT_UNIQUE_ID`. 1. Assign the **Logs Writer** role. Alternatively, to use the Google Cloud CLI, you can run the following `gcloud` command: ```bash gcloud projects add-iam-policy-binding "$LOG_PROJECT_ID" \ --member="principal://iam.googleapis.com/projects/$WIF_PROJECT_NUMBER/locations/global/workloadIdentityPools/$WIF_POOL_ID/subject/$NGINXAAS_SERVICE_ACCOUNT_UNIQUE_ID" \ --role='roles/logging.logWriter' ``` #### Grant access to export metrics to a Google project To grant access to export metrics to a Google project, `$METRIC_PROJECT_ID` in the [Google Cloud Console](https://console.cloud.google.com/), perform the following steps. 1. Go to the `$METRIC_PROJECT_ID` project. 1. Go to the **IAM** page. 1. Select **Grant Access**. 1. Enter your principal, for example, `principal://iam.googleapis.com/projects/$WIF_PROJECT_NUMBER/locations/global/workloadIdentityPools/$WIF_POOL_ID/subject/$NGINXAAS_SERVICE_ACCOUNT_UNIQUE_ID`. 1. Assign the **Monitoring Metric Writer** role. Alternatively, to use the Google Cloud CLI, you can run the following `gcloud` command: ```bash gcloud projects add-iam-policy-binding "$METRIC_PROJECT_ID" \ --member="principal://iam.googleapis.com/projects/$WIF_PROJECT_NUMBER/locations/global/workloadIdentityPools/$WIF_POOL_ID/subject/$NGINXAAS_SERVICE_ACCOUNT_UNIQUE_ID" \ --role='roles/monitoring.metricWriter' ``` #### Grant access to fetch a secret from Secret Manager To grant access to fetch a secret, `$SECRET_ID`, in the [Google Cloud Console](https://console.cloud.google.com/), 1. Go to the secret, `$SECRET_ID`, in Secret Manager. 1. Select the **Permissions** tab. 1. Select **Grant Access**. 1. Enter your principal, for example, `principal://iam.googleapis.com/projects/$WIF_PROJECT_NUMBER/locations/global/workloadIdentityPools/$WIF_POOL_ID/subject/$NGINXAAS_SERVICE_ACCOUNT_UNIQUE_ID`. 1. Assign the **Secret Manager Secret Accessor** role. Alternatively, to use the Google Cloud CLI, you can run the following `gcloud` command: ```bash gcloud secrets add-iam-policy-binding "$SECRET_ID" \ --member="principal://iam.googleapis.com/projects/$WIF_PROJECT_NUMBER/locations/global/workloadIdentityPools/$WIF_POOL_ID/subject/$NGINXAAS_SERVICE_ACCOUNT_UNIQUE_ID" \ --role='roles/secretmanager.secretAccessor' ``` If you would like to fetch more than one secret, you will need to grant access on each secret or grant access on the project your secrets are in. ### Update your NGINXaaS deployment with the name of your workload identity pool provider In the NGINXaaS Console, 1. On the navigation menu, select **Deployments**. 1. Select the deployment you want to update and select **Edit**. 1. Enter your provider name, for example, `projects//locations//workloadIdentityPools//providers/`, under **Workload Identity Pool Provider Name**. 1. Select **Update**. ## What's next - [Monitor your deployment](/nginxaas/google/monitoring/enable-monitoring.md) - [Enable NGINX Logs](/nginxaas/google/monitoring/enable-nginx-logs.md) -------------------------------------------------------------------------------- title: "Glossary" url: https://docs.nginx.com/nginxaas/aws/glossary/index.md description: Definitions for terms and acronyms used in F5 NGINXaaS for AWS documentation. -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- title: "Glossary" url: https://docs.nginx.com/nginxaas/google/glossary/index.md -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- title: "Glossary" url: https://docs.nginx.com/nginxaas/overview/glossary/index.md -------------------------------------------------------------------------------- This document provides definitions for terms and acronyms commonly used in F5 ${product} documentation. | Term | Description | | ------------------------ | -------------------------------------------------------------------------------------| | Geographical Controller (GC)| Geographical Controller (GC) is a control plane that serves users in a given geographical boundary while taking into account concerns relating to data residency and localization. Example: A US geographical controller serves US customers. | | Network attachment | A Google Cloud resource that connects your NGINXaaS for Google Cloud deployment to upstream applications in your VPC network. [More information](https://cloud.google.com/vpc/docs/about-network-attachments). | | NGINX Configuration | NGINXaaS deployments are configured with standard NGINX configuration file syntax - the same syntax you'd use for NGINX running outside NGINXaaS. See the [NGINX configuration reference documentation](https://nginx.org/en/docs/). | NGINXaaS Deployment | An instance of the highly available NGINX Plus managed service. A deployment can be created in any supported cloud provider where your organization holds an active NGINXaaS marketplace subscription. | NGINXaaS Organization | The account that hosts your NGINXaaS configurations and deployments. An Organization can be linked to marketplace subscriptions for each cloud provider you want to deploy into, and can have unlimited users added for collaboration on your NGINXaaS resources. | | NGINXaaS User | NGINXaaS Users are granted access to all resources in the NGINXaaS Organization. User authentication is performed securely via Google Cloud, requiring a matching identity. Individuals can be added as users to multiple NGINXaaS Organizations, and can switch between them. | | Service Frontend | Each NGINXaaS deployment uses one of two service frontend types: **Managed public endpoint**, which makes the deployment available for client access over the public internet, or **Private endpoint**, which enables client access from trusted private network connections.
For details, see the docs for each cloud provider:
- [AWS](/nginxaas/aws/overview.md#service-frontend)
- [Google Cloud](/nginxaas/google/overview.md#service-frontend) | | Upstream Network | The network that hosts your applications and workloads NGINXaaS proxies traffic to. These backend services or origin servers, are typically defined in an NGINX `upstream` block. For instructions on connecting your NGINXaaS deployment to an upstream network, see the docs for each supported cloud provider. | | VPC Endpoint | An AWS resource that establishes a PrivateLink connection from your AWS VPC to the NGINXaaS deployment. When you configure an NGINXaaS for AWS deployment's service frontend as **Private endpoint**, you can create an interface VPC endpoint that target's the deployment's VPC endpoint service. For details, see the [AWS PrivateLink documentation](https://docs.aws.amazon.com/vpc/latest/privatelink/privatelink-share-your-services.html). | | VPC Peering | A method for connecting an NGINXaaS deployment's network to an upstream network so traffic to your backend services routes directly over a private network. | -------------------------------------------------------------------------------- title: "Known issues" url: https://docs.nginx.com/nginxaas/aws/known-issues/index.md description: List of known issues in the latest release of F5 NGINXaaS for AWS. -------------------------------------------------------------------------------- List of known issues in the latest release of F5 NGINXaaS for AWS. _There are currently no known issues._ **Note:** You may also want to be familiar with the documented [NGINXaaS for AWS limitations](/nginxaas/aws/overview.md#current-limitations). -------------------------------------------------------------------------------- title: "Known issues" url: https://docs.nginx.com/nginxaas/google/known-issues/index.md -------------------------------------------------------------------------------- List of known issues in the latest release of F5 NGINXaaS for Google Cloud (NGINXaaS). _There are currently no known issues._ **Note:** You may also want to be familiar with the documented [NGINXaaS limitations](nginxaas/google/overview.md#current-limitations). -------------------------------------------------------------------------------- title: "Get help" url: https://docs.nginx.com/nginxaas/support/index.md -------------------------------------------------------------------------------- To contact support about F5 ${product}: 1. Go to the [MyF5 portal](https://my.f5.com) and log in with your F5 account. 1. Go to the **Support** section and select **Create a Case**. 1. Fill out the **Subject** and **Description** sections, and include the following details about the specific issue, and then select **Next**: 1. Identifying Information: - Deployment identifiers (Object ID and name) - Organization identifiers (Object ID and name) 1. Issue Details: - A detailed description of the issue - The specific resources involved (for example: deployment, configuration, certificate, and so on) - The operation you were performing when the issue occurred (for example: subscribing from cloud marketplace, creating a deployment, uploading a configuration, adding a certificate, and so on) - Any error messages or request IDs displayed - The time when the issue occurred 1. Select **NGINX Products** as the product family and **NGINXaaS** as the product module, and then select **Next**. 1. Enter the following required fields, at a minimum, and then select **Next**: - **Reason for contact** - **Subscription ID** (Select **No subscription available** or your F5 subscription ID if you have a private offer.) - **Cloud Provider** - **Assign a priority** 1. Complete the fields in the **Contact Information** section for your case and then select **Submit**. -------------------------------------------------------------------------------- title: "Support" url: https://docs.nginx.com/nginxaas/aws/support/index.md description: How to contact F5 support for help with NGINXaaS for AWS. -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- title: "Support" url: https://docs.nginx.com/nginxaas/google/support/index.md -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- title: "F5 WAF for NGINX" url: https://docs.nginx.com/waf/index.md -------------------------------------------------------------------------------- ## About Defend your applications and APIs with a software security solution that seamlessly integrates into DevOps environments as a lightweight web application firewall (WAF), layer 7 denial-of-service (DoS) protection, bot protection, API security, and threat intelligence services. ## Featured content ### [Overview](/waf/fundamentals/overview) Learn about how F5 WAF for NGINX works and how it can be used to protect your applications ### [Install F5 WAF for NGINX](/waf/install) Explore the methods available to deploy F5 WAF for NGINX in your environment ### [Changelog](/waf/changelog) Review the latest changes and improvements to F5 WAF for NGINX -------------------------------------------------------------------------------- title: "Apply security policy updates without reloading NGINX using apreload" url: https://docs.nginx.com/waf/configure/apreload/index.md -------------------------------------------------------------------------------- This document describes how to use `apreload`, a tool for updating F5 WAF for NGINX configuration without reloading NGINX. It interacts independently to NGINX, and can be used when any F5 WAF for NGINX files are modified, such as policies, logging profiles or global settings. _apreload_ can handle changes in policy content, with the exception of policy names. While _apreload_ can update F5 WAF for NGINX configuration alone, an NGINX reload will update both NGINX and F5 WAF for NGINX configuration. `apreload` should be ran as the same user as NGINX to avoid file access errors. ```text USAGE: /opt/app_protect/bin/apreload: Optional arguments with default values: -apply Apply new configuration in enforcer (default true) -i string Path to the config set. Ex. /opt/app_protect/config/config_set.json (default "/opt/app_protect/config/config_set.json") -policy-map-location string Path to policy map location (default "/opt/app_protect/bd_config/policy_path.map") -t Test and prepare configuration without updating enforcement -wait-for-enforcer Wait until updated config is loaded (default true) Optionally, using --help will issue this help message. ``` In a Kubernetes environment, you can invoke it using _kubectl_: ```shell kubectl -n exec -it -c waf-nginx -- bash /opt/app_protect/bin/apreload ``` The result can then be viewed in the `waf-config-mgr` container logs. ```shell kubectl -n logs -c config-mgr sudo docker logs waf-config-mgr ``` ## Concurrent apreload executions Concurrent NGINX reloads are enqueued and so are calls to _apreload_ by the F5 NGINX for WAF. When calling _apreload_ directly, it is possible to run it while the previous execution is still in progress. In this case, _apreload_ will wait until the current execution completes. A new execution applies the new configuration; until it completes, the most recent configuration remains in effect. In a scenario where an execution from an NGINX reload is followed by a direct _ap_reload_ call, the NGINX workers with the new NGINX configuration will be loaded as soon as the Enforcer finishes processing the existing configuration. Once complete, the most recent F5 WAF for NGINX configuration will be loaded using with the same NGINX worker instances. ## Limitations with HTTP and XFF header modifications _apreload_ will not apply these two policy modifications: - New [user-defined HTTP headers](/waf/policies/user-headers.md) - - It **will** apply changes to _existing_ user-defined headers. - [XFF trust modifications](/waf/policies/xff-headers.md) If you want to apply either of the two, reload NGINX instead of using _apreload_. ## apreload events _apreload_ events use the same format as operation log events written in the NGINX error log, reporting `configuration_load_success` or `configuration_load_failure` with JSON formatted details. If any of the configuration files are invalid, _apreload_ will discover that and return the proper error message in the `configuration_load_failure event`. The enforcer will continue to run with the previous working configuration. For more information, see the [Operation logs](/waf/logging/operation-logs.md) topic. -------------------------------------------------------------------------------- title: "Configure policies" url: https://docs.nginx.com/waf/policies/configuration/index.md -------------------------------------------------------------------------------- This page describes the security features available with F5 WAF for NGINX and how to configure policies. To convert policies from an existing F5 WAF solution, read the [Build and use the converter tools](/waf/configure/converters.md). The [Build and use the compiler tool](/waf/configure/compiler.md) topic explains how to transform policy files into a format parseable by F5 WAF for NGINX. ## Supported security policy features | Feature | Description | | ----------------------------------- | ----------- | | [Allowed methods](/waf/policies/allowed-methods.md) | Checks allowed HTTP methods. By default, all the standard HTTP methods are allowed. | | [Attack signatures](/waf/policies/attack-signatures.md) | The default policy covers the OWASP top 10 attack patterns. Specific signature sets can be added or disabled. | | [Bot signatures](/waf/policies/bot-signatures.md) | Bot signatures and headers can be inspected to authenticate the identity of a client making a request. | | [Brute force attack preventions](/waf/policies/brute-force-attacks.md) | Configure parameters to secure areas of a web application from brute force attacks. | | [Cookie enforcement](/waf/policies/cookie-enforcement.md) | By default all cookies are allowed and not enforced for integrity. The user can add specific cookies, wildcards or explicit, that will be enforced for integrity. It is also possible to set the cookie attributes: HttpOnly, Secure and SameSite for cookies found in the response. | | [Data guard](/waf/policies/data-guard.md) | Detects and masks Credit Card Number (CCN) and/or U.S. Social Security Number (SSN) and/or custom patterns in HTTP responses. Disabled by default. | | [Deny and Allow IP lists](/waf/policies/deny-allow-ip.md) | **Deprecated**. See [IP address lists](/waf/policies/ip-address-lists.md) | | [Do-nothing](/waf/policies/do-nothing.md) | Do-nothing allows you to avoid inspecting or parsing a URL. | | [Disallowed file type extensions](/waf/policies/disallowed-extensions.md) | Support any file type, and includes a predefined list of file types by default | | [Evasion techniques](/waf/policies/evasion-techniques.md) | All evasion techniques are enabled by default, and can be disabled individually. These include directory traversal, bad escaped characters and more. | | [Filetypes](/waf/policies/filetypes.md) | The filetype feature allows you to selectively allow filetypes. | | [Geolocation](/waf/policies/geolocation.md) | The geolocation feature allows you to configure enforcement based on the location of an object using the two-letter ISO code representing a country. | | [GraphQL protection](/waf/policies/graphql-protection.md) | GraphQL protection allows you to configure enforcement for GraphQL, an API query language. | | [gRPC protection](/waf/policies/grpc-protection.md) | gRPC protection detects malformed content, parses well-formed content, and extracts the text fields for detecting attack signatures and disallowed meta-characters. In addition, it enforces size restrictions and prohibition of unknown fields. The Interface Definition Language (IDL) files for the gRPC API must be attached to the profile. gRPC protection is available for unary or bidirectional traffic. | | [HTTP compliance](/waf/policies/http-compliance.md) | All HTTP protocol compliance checks are enabled by default except for GET with body and POST without body. It is possible to enable any of these two. Some of the checks enabled by default can be disabled, but others, such as bad HTTP version and null in request are performed by the NGINX parser and F5 WAF for NGINX only reports them. These checks cannot be disabled. | | [IP address lists](/waf/policies/ip-address-lists.md) | Organize lists of allowed and forbidden IP addresses across several lists with common attributes. | | [IP intelligence](/waf/policies/ip-intelligence.md) | Configure the IP Intelligence feature to customize enforcement based on the source IP of the request, limiting access from IP addresses with questionable reputation. | | [JWT protection](/waf/policies/jwt-protection.md) | JWT protection allows you to configure policies based on properties of JSON web tokens, such as their header and signature properties. | | [Override rules](/waf/policies/override-rules.md) | Override rules allow you to override default policy settings under specific conditions. | | [Response signatures](/waf/policies/response-signatures.md) | Response signatures allow you to inspect HTTP responses, selectively allowing specific response codes or lengths. | | [Server technology signatures](/waf/policies/server-technology-signatures.md) | Support adding signatures per added server technology. | | [Time-based signature staging](/waf/policies/time-based-signature-staging.md) | Time-based signature staging allows you to stage signatures for a specific period of time. During the staging period, violations of staged signatures are logged but not enforced. After the staging period ends, violations of staged signatures are enforced according to the policy's enforcement mode. | | [Threat campaigns](/waf/policies/threat-campaigns.md) | These are patterns that detect all the known attack campaigns. They are very accurate and have almost no false positives, but are very specific and do not detect malicious traffic that is not part of those campaigns. The default policy enables threat campaigns but it is possible to disable it through the respective violation. | | [User-defined browser control](/waf/policies/user-browers.md) | Allow or deny specific browsers, and define custom browsers | | [User-defined HTTP headers](/waf/policies/user-headers.md) | Handling headers as a special part of requests | | [User-defined URLs and parameters](/waf/policies/user-urls-parameters.md) | Use user-defined properties when configuring violations. | | [User-defined signatures](/waf/policies/user-signatures.md) | Create and configure user-defined signatures for enforcement | | [XFF trusted headers](/waf/policies/xff-headers.md) | Disabled by default, and can accept an optional list of custom XFF headers. | | [XML and JSON content](/waf/policies/xml-json-content.md) | XML content and JSON content profiles detect malformed content and signatures in the element values. Default policy checks maximum structure depth. It is possible to enable more size restrictions: maximum total length of XML/JSON data, maximum number of elements and more. | ## Additional policy features | Feature | Description | | -------------- | ----------- | | Blocking pages | The user can customize all blocking pages. By default the AJAX response pages are disabled, but the user can enable them. | | Enforcement by violation rating | By default block requests that are declared as threats, which are rated 4 or 5. It is possible to change this behavior: either disable enforcement by Violation Rating or block also request with Violation Rating 3 - needs examination. | | Large request blocking | To increase the protection of resources at both the NGINX Plus and upstream application tiers, all requests that are larger than 10 MB in size are blocked. When these requests are blocked, a `VIOL_REQUEST_MAX_LENGTH` violation will be logged.| | Malformed cookie | Requests with cookies that are not RFC compliant are blocked by default. This can be disabled. | | Parameter parsing | Support only auto-detect parameter value type and acts according to the result: plain alphanumeric string, XML or JSON. | | Request size checks | Upper limit of request size as dictated by the maximum buffer size of 10 MB; Size checks for: URL, header, Query String, whole request (when smaller than the maximum buffer), cookie, POST data. By default all the checks are enabled with the exception of POST data and whole request. The user can enable or disable every check and customize the size limits. | | Status code restriction | Illegal status code in the range of 4xx and 5xx. By default only these are allowed: 400, 401, 404, 407, 417, 503. The user can modify this list or disable the check altogether. | | Sensitive parameters | The default policy masks the “password” parameter in the security log, and can be customized for more | ## General configuration F5 WAF for NGINX ships with two reference policies, both with a default enforcement mode set to Blocking: - The **default** policy which is identical to the base template and provides OWASP Top 10 and Bot security protection out of the box. - The **strict** policy contains more restrictive criteria for blocking traffic than the default policy. It is meant to be used for protecting sensitive applications that require more security but with higher risk of false positives. You can use these policies as-is, but they are often the starting points for customizations according to the needs of the applications F5 WAF NGINX protects. ### Configuration overview The F5 WAF for NGINX security policy configuration uses a declarative format based on a pre-defined base template. The policy is represented in a JSON file which you can edit to add, modify and remove security capabilities in reference to the base template. The way the policy is integrated into the NGINX configuration is through referencing the JSON file (Using the full path) in the `nginx.conf` file. **Note:** F5 WAF for NGINX provides a [JSON Schema](https://json-schema.org/) which can be used to validate a JSON policy file for format compliance. The schema file can be generated using a script once F5 WAF for NGINX is installed: `sudo /opt/app_protect/bin/generate_json_schema.pl`. This script will output the schema to a file named `policy.json` into the current working directory. Once the schema file is generated, you can use validation tools such as [AJV](https://ajv.js.org/standalone.html) to validate a JSON policy file. This schema is used for the [Policy parameter reference](/waf/policies/parameter-reference.md). In the following example, the NGINX configuration file with F5 WAF for NGINX is enabled in the HTTP context and the policy _/etc/app_protect/conf/NginxDefaultPolicy.json_ is used: ```nginx user nginx; worker_processes 4; load_module modules/ngx_http_app_protect_module.so; error_log /var/log/nginx/error.log warn; events { worker_connections 65536; } http { include /etc/nginx/mime.types; default_type application/octet-stream; sendfile on; keepalive_timeout 65; app_protect_enable on; # This is how you enable F5 WAF for NGINX in the relevant context/block app_protect_policy_file "/etc/app_protect/conf/NginxDefaultPolicy.json"; # This refers to which policy file to use, which falls back to the default policy app_protect_security_log_enable on; # This section enables logging app_protect_security_log "/etc/app_protect/conf/log_default.json" syslog:server=127.0.0.1:514; # This is where the remote logger is defined in terms of: logging options (defined in the referenced file), log server IP, log server port server { listen 80; server_name localhost; location / { client_max_body_size 0; default_type text/html; proxy_pass http://172.29.38.211:80$request_uri; } } } ``` ### Base template The base template is the common starting point for any policy you write. The default policy reflects the base template without any further modifications, so the terms _base template_ and _default policy_ are used interchangeably. The default policy appears as follows: ```json { "policy" : { "name": "app_protect_default_policy", "template": { "name": "POLICY_TEMPLATE_NGINX_BASE" } } } ``` The default policy enforces violations by **Violation Rating**, the F5 WAF for NGINX computed assessment of the risk of the request based on the triggered violations. - 0: No violation - 1-2: False positive - 3: Needs examination - 4-5: Threat The default policy enables most of the violations and signature sets with Alarm turned **ON**, but not **Block**. These violations and signatures, when detected in a request, affect the violation rating. By default, if the violation rating is calculated to be malicious (4-5) the request will be blocked by the `VIOL_RATING_THREAT` violation. This is true even if the other violations and signatures detected in that request have the Block flag turned OFF. It is the `VIOL_RATING_THREAT` violation having the Block flag turned ON that caused the blocking, but indirectly the combination of all the other violations and signatures in Alarm caused the request to be blocked. By default, other requests which have a lower violation rating are not blocked, except for some specific violations described below. This is to minimize false positives. However, you can change the default behavior. For example, if you want to add blocking on a violation rating of 3 as well, enable blocking for the `VIOL_RATING_NEED_EXAMINATION` violation. The following violations and signature sets have a low chance of being false positives and are, therefore, configured by default to block the request regardless of its Violation Rating: - High accuracy attack signatures - Threat campaigns - Malformed request: unparsable header, malformed cookie and malformed body (JSON or XML). ### Default policy F5 WAF for NGINX offers prebuilt bundles for security policies: - _app_protect_default_policy_ - _app_protect_strict_policy_ **Note:** You cannot mix these prebuilt bundles with custom policy bundles within the same `nginx.conf` file. Example: ```nginx ... location / { # F5 WAF for NGINX app_protect_enable on; app_protect_policy_file app_protect_strict_policy; app_protect_security_log_enable on; app_protect_security_log log_all stderr; proxy_pass http://127.0.0.1:8080/; } ``` #### Updating default policy bundles **Note:** This section assumes that you have built a [compiler image](/waf/configure/compiler.md) named `waf-compiler-1.0.0:custom`. To generate versions of the default policies that include the latest security updates, use the `-factory-policy` option instead of a source policy file. For instance, to create an updated version of the `app_protect_default_policy`, use the following command: ```shell docker run \ -v $(pwd):$(pwd) \ waf-compiler-1.0.0:custom \ -factory-policy default -o $(pwd)/new_default_policy.tgz ``` To create an updated version of the `app_protect_strict_policy`, use: ```shell docker run \ -v $(pwd):$(pwd) \ waf-compiler-1.0.0:custom \ -factory-policy strict -o $(pwd)/new_strict_policy.tgz ``` After creating the updated version of a policy, reference it in the `nginx.conf` file: ```nginx app_protect_policy_file /policies_mount/new_default_policy.tgz; ``` ### Strict policy The strict policy is recommended as a starting point for applications requiring a higher level of security. Similar to policies, it is customized from the base template, so it detects and blocks everything the default policy does. To obtain the strict policy, execute the following command: ```shell sudo docker run --rm -v $(pwd):$(pwd) \ --entrypoint='' \ private-registry.nginx.com/nap/waf-compiler:1.0.0 \ cat /etc/app_protect/conf/NginxStrictPolicy.json ``` Replace the `1.0.0` with the actual release version. In addition the strict policy also **blocks** the following: - Requests that have a Violation Rating of 3, "Needs examination". This occurs because the `VIOL_RATING_NEED_EXAMINATION` violation's block flag is enabled in the strict policy. - Requests with the `VIOL_EVASION` violation (evasion techniques). - Requests with violations that restrict options in the request and response: HTTP method, response status code and disallowed file types. **Note:** Other violations, specifically attack signatures and metacharacters, which are more prone to false positives, still have only Alarm turned on, without blocking, contributing to the Violation Rating as in the Default policy. In addition, the Strict policy also enables the following features in **alarm only** mode: - **Data Guard**: masking Credit Card Number (CCN), US Social Security Number (SSN) and custom patterns found in HTTP responses. - **HTTP response data leakage signatures**: preventing exfiltration of sensitive information from the servers. - **More restrictive limitations**: mainly sizing and parsing of JSON and XML payloads. - **Cookie attribute insertion**: the Strict policy adds the **Secure** and **SameSite=lax** attributes to every cookie set by the application server. These attributes are enforced by the browsers and protect against session hijacking and CSRF attacks respectively. ## Policy authoring and tuning The policy JSON file specifies the settings that are different from the base template, such as enabling more signatures, disabling some violations, adding server technologies, etc. These will be shown in the next sections. There are two ways to tune those settings: - Within the `policy` structure property, the organic structure of the policy. - Within the `modifications` structure property that contains a list of changes expressed in a generic manner. Both options are equivalent in their semantic expression power, but different syntactically and are designated for different use cases. But before that, let's look at an example - disabling a specific attack signature. Signature 200001834 disabled in the `policy` property: ```json { "policy": { "name": "signature_exclude_1", "signatures": [ { "signatureId": 200001834, "enabled": false } ] } } ``` As you can see, this is expressed using the `signatures` property that contains configuration of individual signatures in a policy. If you want to modify other parts of the policy, you would use different JSON properties. The same configuration in the `modifications` array looks like this: ```json { "policy": { "name": "signature_exclude_2" }, "modifications": [ { "entityChanges": { "enabled": false }, "entity": { "signatureId": 200001834 }, "entityType": "signature", "action": "add-or-update" } ] } ``` Note the generic schema that can express manipulation in any policy element: `entity`, `entityType`, `action` etc. The `modifications` array is a flat list of individual changes applied to the policy after evaluating the `policy` block. So when to use `policy` and when to use `modifications`? There are some recommended practice guidelines for that: - Use `policy` to express the security policy as you intended it to be: the features you want to enable, disable, the signature sets, server technologies and other related configuration attributes. This part of the policy is usually determined when the application is deployed and changes at a relatively slow pace. - Use `modifications` to express **exceptions** to the intended policy. These exceptions are usually the result of fixing false positive incidents and failures in tests applied to those policies. Usually these are granular modifications, typically disabling checks of individual signatures, metacharacters and sub-violations. These changes are more frequent. - Use `modifications` also for **removing** individual collection elements from the base template, for example disallowed file types. It is a good practice to separate the `modifications` to a different file and have the main policy file reference the former, as the two parts have different lifecycles. The sections just below review the common policy feature configurations using examples. For the full reference of the `policy` JSON properties see the Declarative Policy guide. ### Policy enforcement modes A policy's enforcement mode can be: - **Blocking:** Any illegal or suspicious requests are logged and blocked. This is the default enforcement mode for the default policy and any added policy unless changed to Transparent. - **Transparent:** Any illegal or suspicious requests are logged but not blocked. Individual security features can be defined as blocked or transparent in the policy. Here are examples of both: #### Blocking ```json { "policy": { "name": "policy_name", "template": { "name": "POLICY_TEMPLATE_NGINX_BASE" }, "applicationLanguage": "utf-8", "enforcementMode": "blocking" } } ``` #### Transparent ```json { "policy": { "name": "policy_name", "template": { "name": "POLICY_TEMPLATE_NGINX_BASE" }, "applicationLanguage": "utf-8", "enforcementMode": "transparent" } } ``` ### Enabling violations Adding and enabling additional security features to the policy can be done by specifying the violation name and the `alarm` block state to `true`. To set different states for sub-violations within the violation, enable the violation first, then specifying and configure the sub-violations. A violation may have its own section that provides additional configuration granularity for a specific violation/sub-violation. **Note:** The attack signature violation `VIOL_ATTACK_SIGNATURE` cannot be configured. It is determined by the combination of the [signature sets](/waf/policies/attack-signatures.md#signature-sets ) on the policy. In this example, we enable a violation and a sub-violation: `VIOL_JSON_FORMAT` and `VIOL_PARAMETER_VALUE_METACHAR`. The example defines the blocking and alarm setting for each violation. These settings override the default configuration set above in the `enforcementMode` directive. Be aware, however, that in a transparent policy no violations are blocked, even if specific violations are set to `block: true` in the configuration. ```json { "policy": { "name": "policy_name", "template": { "name": "POLICY_TEMPLATE_NGINX_BASE" }, "applicationLanguage": "utf-8", "enforcementMode": "blocking", "blocking-settings": { "violations": [ { "name": "VIOL_JSON_FORMAT", "alarm": true, "block": true }, { "name": "VIOL_PARAMETER_VALUE_METACHAR", "alarm": false, "block": false } ] } } } ``` -------------------------------------------------------------------------------- title: "Log types" url: https://docs.nginx.com/waf/logging/logs-overview/index.md -------------------------------------------------------------------------------- F5 WAF for NGINX generates three types of logs: - [Security logs](/waf/logging/security-logs.md), which record HTTP requests and how F5 WAF processes them, including violations and detected signatures. - [Operation logs](/waf/logging/operation-logs.md), covering F5 WAF events such as startup, shutdown and reconfiguration. - [Debug logs](/waf/logging/debug-logs.md), recording technical messages at different levels of severity used to debug and resolve incidents and error behaviors. F5 WAF for NGINX can also be configured to add additional data to the NGINX [access logs](/waf/logging/access-logs.md). **Note:** F5 WAF for NGINX and NGINX itself do not have logs for auditing system user interactions. These events should be examined in the orchestration system controlling NGINX (such as NGINX Ingress Controller), or by tracking configuration files and their systemd invocations. F5 WAF for NGINX uses a separate logging mechanism to NGINX's default one, which is responsible for access logs. | Type | Log configuration | Configuration contexts | File destination | Syslog destination | | --------- | ----------------- | -----------------------| ---------------- | ------------------ | | Security | `app_protect_security_log` directive referencing `security_log.json` file | `nginx.conf`: http, server, location | Yes, either `stderr`, or an absolute path to a local file are supported | Yes | | Operation | `error_log` directive, part of core NGINX | `nginx.conf` - global | Yes, NGINX error log | Yes, NGINX error log | | Debug | `/etc/app_protect/bd/logger.cfg.` Log file name is the redirection in the invocation of the bd command line in the start script | Global (not part of nginx.conf) | Yes. Log file is in `/var/log/app_protect` default debug directory. No file rotation currently | No | ## Security logs To configure [security logs](/waf/logging/security-logs.md), you need to [compile a security policy](/waf/configure/compiler.md#compile-a-security-policy) JSON file into a bundle before applying it. ### Default logging profile bundles There are several pre-compiled logging profile bundles available: - log_default (equivalent to log_illegal) - log_all - log_illegal - log_blocked - log_grpc_all - log_grpc_blocked - log_grpc_illegal These logging profiles can be referenced by their names, excluding the file path and the `tgz` extension. For instance: ```nginx ... location / { # F5 WAF for NGINX app_protect_enable on; app_protect_security_log_enable on; app_protect_security_log log_blocked syslog:server=log-server:514; proxy_pass http://127.0.0.1:8080/; } ``` ### Security log destination View [app_protect_security_log](/waf/logging/security-logs.md#app_protect_security_log) to configure the log file destination. ### WAF Enforcer container logs When `stderr` is set as the destination for security logs in the `app_protect_security_log` directive, these logs are accessible via the `waf-enforcer` container. To view them, use the following command: ```shell docker logs waf-enforcer ``` Or in Kubernetes: ```shell kubectl logs deployment.apps/nap5-deployment -c waf-enforcer ``` ## Debug logs Logs for internal components of F5 WAF for NGINX can be accessed by executing `docker logs` or `kubectl logs` on one of the deployment containers. For example: ```shell docker logs waf-config-mgr ``` For more information, see the [Debug logs](/waf/logging/debug-logs.md) topic. ## NGINX access log F5 WAF for NGINX can be configured to add additional data to the NGINX [access log](/waf/logging/access-logs.md). ## logrotate support F5 WAF for NGINX supports logrotate. It is typically run periodically using a cron job: more information is available in its [Linux man page](https://linux.die.net/man/8/logrotate). If your system has logrotate available, F5 WAF for NGINX log files will rotate automatically based on the default configuration file. The default logrotate configuration file is `/etc/logrotate.d`: ```none { size 1M copytruncate notifempty create 644 nginx nginx rotate 20 } ``` | Option name | Description | | ----------- | ------------| | _size_ | Log files are rotated only if they grow larger than the value of _size_. | _copytruncate_ | Truncate the original log file in place after creating a copy, instead of moving the old log file and creating a new one. | _create_ | _mode owner group_ - The log file is created immediately after rotation with the permissions specified by _mode_. _owner_ specifies the user name who will own the log file, and _group_ specifies the group the log file will belong to. | _rotate_ | _count_ - Log files are rotated _count_ times before being removed. You can modify the attributes and add directories to rotate in the file `/etc/logrotate.d/app_protect.conf`. All logs in the `/var/log/app_protect/` folder will be rotated, which can include the security log if configured accordingly. **Note:** The default log rotation policy is provided as a default policy can be customized for your use cases. To output security logs to the `/var/log/app_protect/` folder, update `/etc/nginx/nginx.conf`: ```nginx app_protect_security_log_enable on; app_protect_security_log "/opt/app_protect/share/defaults/log_illegal.json" /var/log/app_protect/security.log; ``` -------------------------------------------------------------------------------- title: "Overview" url: https://docs.nginx.com/waf/fundamentals/overview/index.md -------------------------------------------------------------------------------- [F5 WAF for NGINX](https://www.f5.com/products/nginx/nginx-app-protect) (formerly NGINX App Protect WAF) is an advanced, lightweight and high-performance web application firewall (WAF) for applications and APIs. It provides protection for the OWASP Top 10, with additional functionality: - HTTP response inspection and protocol compliance - Data schema validation (JSON & XML) - Meta character checking - Disallowing file types For more details, see the [Supported security policy features](/waf/policies/configuration.md#supported-security-policy-features). It is platform-agnostic and supports a range of deployment options: 1. [Virtual machine or bare metal](/waf/install/virtual-environment.md) - NGINX and WAF components operate on the host system - Ideal for existing NGINX virtual environments 1. [Docker](/waf/install/docker.md) - NGINX and WAF components are deployed as containers - Ideal for environments with multiple deployment stages 1. [Kubernetes](/waf/install/kubernetes.md) - Integrates NGINX and WAF components in a single pod - Ideal for scalable, cloud-native environments For more details, see the [Technical specifications](/waf/fundamentals/technical-specifications.md). F5 WAF for NGINX is part of the [NGINX One](https://www.f5.com/products/nginx/one) premium packages and runs natively on [NGINX Plus](https://www.f5.com/products/nginx/nginx-plus) and [NGINX Ingress Controller](https://www.f5.com/products/nginx/nginx-ingress-controller). -------------------------------------------------------------------------------- title: "Virtual machine or bare metal" url: https://docs.nginx.com/waf/install/virtual-environment/index.md -------------------------------------------------------------------------------- This page describes how to install F5 WAF for NGINX in a virtual machine or bare metal environment. ## Before you begin To complete this guide, you will need the following prerequisites: - A [supported operating system](/waf/fundamentals/technical-specifications.md#supported-operating-systems). - Ensure you have an active F5 WAF for NGINX subscription (purchased or trial) and have downloaded the associated [SSL certificate, private key, and JWT license](#download-your-subscription-credentials) file from the MyF5 Customer Portal. Depending on your deployment type, you may have additional requirements: You should read the [IP intelligence](/waf/policies/ip-intelligence.md) topics for additional set-up configuration if you want to use them immediately. **Note:** Security mechanisms like SELinux or AppArmor may potentially block necessary file access for the NGINX process and any component containers. For more information, view the [Configure SELinux](/waf/configure/selinux.md) topic. ## Default security policy and logging profile F5 WAF for NGINX uses built-in default security policy and logging profile after installation. To use custom policies or logging profiles, update your NGINX configuration file accordingly. ## Download your subscription credentials To use NGINX Plus, you will need to download the JWT license file associated with your F5 WAF for NGINX WAF subscription from the [MyF5](https://my.f5.com/manage/s/) Customer Portal: 1. Log in to [MyF5](https://my.f5.com/manage/s/). 1. Go to **My Products & Plans > Subscriptions** to see your active subscriptions. 1. Find your NGINX subscription, and select the **Subscription ID** for details. 1. Download the **SSL Certificate**, **Private Key** and **JSON Web Token** files from the subscription page. **Note:** Starting from [NGINX Plus Release 33](nginx/releases.md#r33), a JWT file is required for each NGINX Plus instance. For more information, see [About Subscription Licenses](/solutions/about-subscription-licenses.md). ## Platform-specific instructions Navigate to your chosen operating system, which are alphabetically ordered. ### Alpine Linux Add the F5 WAF for NGINX signing key: ```shell sudo wget -O /etc/apk/keys/app-protect-security-updates.rsa.pub https://cs.nginx.com/static/keys/app-protect-security-updates.rsa.pub ``` Add the F5 WAF for NGINX repository: ```shell printf "https://pkgs.nginx.com/app-protect/alpine/v`egrep -o '^[0-9]+\.[0-9]+' /etc/alpine-release`/main\n" | sudo tee -a /etc/apk/repositories printf "https://pkgs.nginx.com/app-protect-security-updates/alpine/v`egrep -o '^[0-9]+\.[0-9]+' /etc/alpine-release`/main\n" | sudo tee -a /etc/apk/repositories ``` Update the repositories, then install the F5 WAF for NGINX package and its dependencies: ```shell sudo apk update sudo apk add openssl ca-certificates app-protect ``` #### Installing a specific version of F5 WAF for NGINX If you need to install a specific version of F5 WAF for NGINX, you can use `apk info` to list available versions, then append it to the package name: ```shell sudo apk info app-protect sudo apk add openssl ca-certificates app-protect= ``` ### Amazon Linux Add the F5 WAF for NGINX repository: ```shell sudo wget -P /etc/yum.repos.d https://cs.nginx.com/static/files/app-protect-amazonlinux2023.repo ``` Add F5 WAF for NGINX dependencies: ```shell sudo wget -P /etc/yum.repos.d https://cs.nginx.com/static/files/dependencies.amazonlinux2023.repo ``` Install the F5 WAF for NGINX package and its dependencies: ```shell sudo dnf install app-protect ``` #### Installing a specific version of F5 WAF for NGINX If you need to install a specific version of F5 WAF for NGINX, you can use `--showduplicates list` to list available versions, then append it to the package name: ```shell sudo dnf --showduplicates list app-protect sudo dnf install app-protect-= ``` ### Debian Add the F5 WAF for NGINX signing key: ```shell wget -qO - https://cs.nginx.com/static/keys/app-protect-security-updates.key | gpg --dearmor | \ sudo tee /usr/share/keyrings/app-protect-security-updates.gpg > /dev/null ``` Add the F5 WAF for NGINX repositories: ```shell printf "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] \ https://pkgs.nginx.com/app-protect/debian `lsb_release -cs` nginx-plus\n" | \ sudo tee /etc/apt/sources.list.d/nginx-app-protect.list printf "deb [signed-by=/usr/share/keyrings/app-protect-security-updates.gpg] \ https://pkgs.nginx.com/app-protect-security-updates/debian `lsb_release -cs` nginx-plus\n" | \ sudo tee /etc/apt/sources.list.d/app-protect-security-updates.list ``` Update the repositories, then install the F5 WAF for NGINX package and its dependencies: ```shell sudo apt-get update sudo apt-get install app-protect ``` #### Installing a specific version of F5 WAF for NGINX If you need to install a specific version of F5 WAF for NGINX, you can use `apt-cache` to list available versions, then append it to the package name: ```shell sudo apt-get update sudo apt-cache policy app-protect sudo apt-get install app-protect= ``` When installing a specific version of F5 WAF for NGINX, you will also need to manually install its package dependencies. You can use the following script to get the dependent packages: ```shell findDeps () { local pkgs=$(apt show $1 2>/dev/null | grep Depends: | grep -oE "(nginx-plus-module|app-protect)-[a-z]+ *\(= *[0-9\+\.-]+~`lsb_release -cs`\)" | tr -d ' ()'); for p in ${pkgs[@]}; do echo $p; findDeps $p; done; } findDeps app-protect= ``` ### Oracle Linux / RHEL / Rocky Linux **Note:** The steps are identical for Oracle Linux, RHEL, and Rocky Linux. In the commands below, replace `` with your operating system major version: `8`, `9`, or `10`. Add the F5 WAF for NGINX repository: ```shell sudo wget -P /etc/yum.repos.d https://cs.nginx.com/static/files/app-protect-.repo ``` Add F5 WAF for NGINX dependencies: ```shell sudo wget -P /etc/yum.repos.d https://cs.nginx.com/static/files/dependencies.repo ``` Enable F5 WAF for NGINX dependencies: ```shell sudo dnf config-manager --set-enabled crb ``` On Oracle Linux 8, also enable the `ol8_codeready_builder` repository: ```shell sudo dnf config-manager --set-enabled ol8_codeready_builder ``` Install the F5 WAF for NGINX package and its dependencies: ```shell sudo dnf install app-protect ``` #### Installing a specific version of F5 WAF for NGINX If you need to install a specific version of F5 WAF for NGINX, you can use `--showduplicates list` to list available versions, then append it to the package name: ```shell sudo dnf --showduplicates list app-protect sudo dnf install app-protect-= ``` ### Ubuntu Add the F5 WAF for NGINX signing key: ```shell wget -qO - https://cs.nginx.com/static/keys/app-protect-security-updates.key | \ gpg --dearmor | sudo tee /usr/share/keyrings/app-protect-security-updates.gpg > /dev/null ``` Add the F5 WAF for NGINX repositories: ```shell printf "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] \ https://pkgs.nginx.com/app-protect/ubuntu `lsb_release -cs` nginx-plus\n" | \ sudo tee /etc/apt/sources.list.d/nginx-app-protect.list printf "deb [signed-by=/usr/share/keyrings/app-protect-security-updates.gpg] \ https://pkgs.nginx.com/app-protect-security-updates/ubuntu `lsb_release -cs` nginx-plus\n" | \ sudo tee /etc/apt/sources.list.d/app-protect-security-updates.list ``` Update the repositories, then install the F5 WAF for NGINX package and its dependencies: ```shell sudo apt-get update sudo apt-get install app-protect ``` #### Installing a specific version of F5 WAF for NGINX If you need to install a specific version of F5 WAF for NGINX, you can use `apt-cache` to list available versions, then append it to the package name: ```shell sudo apt-get update sudo apt-cache policy app-protect sudo apt-get install app-protect= ``` When installing a specific version of F5 WAF for NGINX, you will also need to manually install its package dependencies. You can use the following script to get the dependent packages: ```shell findDeps () { local pkgs=$(apt show $1 2>/dev/null | grep Depends: | grep -oE "(nginx-plus-module|app-protect)-[a-z]+ *\(= *[0-9\+\.-]+~`lsb_release -cs`\)" | tr -d ' ()'); for p in ${pkgs[@]}; do echo $p; findDeps $p; done; } findDeps app-protect= ``` ## Install NGINX Plus license If you have not already copied your NGINX Plus JWT license file to the `/etc/nginx/` directory (for example, if NGINX Plus was installed automatically as a dependency), do so now: ```shell sudo cp .jwt /etc/nginx/license.jwt ``` ## Update configuration files Once you have installed F5 WAF for NGINX, you must load it as a module in the main context of your NGINX configuration. ```nginx load_module modules/ngx_http_app_protect_module.so; ``` And finally, F5 WAF for NGINX can enabled on a _http_, _server_ or _location_ context: ```nginx app_protect_enable on; ``` **Note:** You should only enable F5 WAF for NGINX on _proxy_pass_ and _grpc_pass_ locations. Here are two examples of how these additions could look in configuration files: #### nginx.conf The default path for this file is `/etc/nginx/nginx.conf`. ```nginx {hl_lines=[5]} user nginx; worker_processes auto; # F5 WAF for NGINX load_module modules/ngx_http_app_protect_module.so; error_log /var/log/nginx/error.log notice; pid /var/run/nginx.pid; events { worker_connections 1024; } http { include /etc/nginx/mime.types; default_type application/octet-stream; log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; access_log /var/log/nginx/access.log main; sendfile on; #tcp_nopush on; keepalive_timeout 65; #gzip on; include /etc/nginx/conf.d/*.conf; } ``` #### default.conf The default path for this file is `/etc/nginx/conf.d/default.conf`. ```nginx {hl_lines=[9]} server { listen 80; server_name domain.com; location / { # F5 WAF for NGINX app_protect_enable on; client_max_body_size 0; default_type text/html; proxy_pass http://127.0.0.1:8080/; } } server { listen 8080; server_name localhost; location / { root /usr/share/nginx/html; index index.html index.htm; } # redirect server error pages to the static page /50x.html # error_page 500 502 503 504 /50x.html; location = /50x.html { root /usr/share/nginx/html; } } ``` Once you have updated your configuration files, reload NGINX with the following command: - `nginx -s reload` ## Post-installation checks The following steps check that F5 WAF for NGINX enforcement is operational. They should be ran in the environment with the WAF components. Check that the three processes for F5 WAF for NGINX are running using `ps aux`: - _bd-socket-plugin_ - _nginx: master process_ - _nginx: worker process_ ```shell USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 8 1.3 2.4 3486948 399092 ? Sl 09:11 0:02 /usr/share/ts/bin/bd-socket-plugin tmm_count 4 proc_cpuinfo_cpu_mhz 2000000 total_xml_memory 307200000 total_umu_max_size 3129344 sys_max_account_id 1024 no_static_config root 14 0.0 0.1 71060 26680 ? S 09:11 0:00 nginx: master process /usr/sbin/nginx -c /tmp/policy/test_nginx.conf -g daemon off; root 26 0.0 0.3 99236 52092 ? S 09:12 0:00 nginx: worker process root 28 0.0 0.0 11788 2920 pts/0 Ss 09:12 0:00 bash root 43 0.0 0.0 47460 3412 pts/0 R+ 09:14 0:00 ps aux ``` Verify there are no errors in the file `/var/log/nginx/error.log` and that the policy compiled successfully: ```none 2020/05/10 13:21:04 [notice] 402#402: APP_PROTECT { "event": "configuration_load_start", "configSetFile": "/opt/f5waf/config/config_set.json" } 2020/05/10 13:21:04 [notice] 402#402: APP_PROTECT policy 'app_protect_default_policy' from: /etc/app_protect/conf/NginxDefaultPolicy.json compiled successfully 2020/05/10 13:21:04 [notice] 402#402: APP_PROTECT { "event": "configuration_load_success", "software_version": "1.1.1", "attack_signatures_package":{"revision_datetime":"2019-07-16T12:21:31Z"},"completed_successfully":true} 2020/05/10 13:21:04 [notice] 402#402: using the "epoll" event method 2020/05/10 13:21:04 [notice] 402#402: nginx/1.17.6 (nginx-plus-r20) 2020/05/10 13:21:04 [notice] 402#402: built by gcc 4.8.5 20150623 (Red Hat 4.8.5-36) (GCC) 2020/05/10 13:21:04 [notice] 402#402: OS: Linux 3.10.0-957.27.2.el7.x86_64 2020/05/10 13:21:04 [notice] 402#402: getrlimit(RLIMIT_NOFILE): 1048576:1048576 2020/05/10 13:21:04 [notice] 406#406: start worker processes 2020/05/10 13:21:04 [notice] 406#406: start worker process 407 ``` Check that sending an attack signature in a request returns a response block page containing a support ID: ```shell Request: http://10.240.185.211/?a=