Encrypt Traffic

This document explains how to encrypt traffic between NGINX Instance Manager and the NGINX Agent.


This documentation applies to NGINX Instance Manager 2.0.0 and later.


Overview

Important:
A management server should NEVER be exposed to the public internet. You can mitigate exposure with settings here, but they are not a substitute for preventing unneeded exposure.
See Also:
For instructions on how to configure TLS settings for the NGINX Agent, see NGINX Agent TLS Settings.

NGINX Proxy SSL Termination

Configure the SSL certificate and key inside the NGINX configuration. For instructions, refer to the NGINX SSL Termination guide.

SSL Termination for NGINX OSS

/etc/nginx/conf.d/nms-http.conf
# Main external HTTPS server, needs port 443
server {
    listen 443 ssl http2;
    root /var/www/nms;
    server_name _;

    ssl_protocols       TLSv1.1 TLSv1.2;
    ssl_ciphers         HIGH:!aNULL:!MD5;
    ssl_session_cache   shared:SSL:10m;
    ssl_session_timeout 10m;

    ssl_certificate         /etc/ssl/nginx-manager/nginx-manager.crt;
    ssl_certificate_key     /etc/ssl/nginx-manager/nginx-manager.key;
    ssl_client_certificate  /etc/ssl/nginx-manager/ca.pem;


Mutual Client Certificate Auth Setup (mTLS)

Using client certificates unique to each endpoint allows you to secure and authorize NGINX instances with NGINX Instance Manager. You can run NGINX as a proxy to offload client cert handling.

Use PKI methods to secure your enterprise. Refer to the following instructions for guidance.

Generate a Certificate Authority (CA) for all the methods described below. The CA can be on the Instance Manager server for testing. For production, follow your organization’s standards (typically an offline machine without network connections).

The root CA provides a certificate for an intermediate CA, which should be secured. The root CA (or intermediate CA) issues client and server certificates. The CA (either root or intermediate) signs certificate signing requests (CSRs) and issues certificates. The following examples assume you have the following components setup:

  1. CA server (nginx-manager-ca.example.com) with ca.pem bundle
  2. Instance Manager server (nginx-manager.example.com) with key and crt
  3. nginx-agent instances (nginx-1.example.com) with key and crt

Generate Certificates

Modify the following example according to your needs. There are many ways to generate certificates; these examples are suggestions.

  1. Install OpenSSL if you haven’t already.

  2. Use a script similar to the following example to set up the certificates you need. Save this script as make_certs.sh.

    make_certs.sh
    #!/bin/bash
    
    make_ca() {
        echo "Creating CA certificate and key"
        openssl req \
        -new \
        -newkey rsa:4096 \
        -days 365 \
        -sha256 \
        -nodes \
        -x509 \
        -keyout ca.key \
        -out ca.crt \
        -config ca.cnf \
        -extensions v3_req
        #openssl x509 -in ca.crt -noout -text
    }
    
    make_int() {
        echo "Creating Intermediate CA certificate and key"
        openssl req \
        -new \
        -newkey rsa:4096 \
        -nodes \
        -keyout ca_int.key \
        -out ca_int.csr \
        -config ca-intermediate.cnf \
        -extensions v3_req
        openssl req -in ca_int.csr -noout -verify
        openssl x509 \
        -req \
        -sha256 \
        -days 365 \
        -CA ca.crt \
        -CAkey ca.key \
        -CAcreateserial \
        -in ca_int.csr \
        -out ca_int.crt \
        -extfile ca-intermediate.cnf \
        -extensions v3_req
        openssl verify -CAfile ca.crt ca_int.crt
        echo "Creating CA chain"
        cat ca_int.crt ca.crt > ca.pem
    }
    
    make_server() {
        echo "Creating nginx-manger certificate and key"
        openssl req \
        -new \
        -nodes \
        -keyout server.key \
        -out server.csr \
        -config server.cnf \
        -extensions v3_req
        openssl req -in server.csr -noout -verify
        openssl x509 \
        -req \
        -days 365 \
        -sha256 \
        -CA ca_int.crt \
        -CAkey ca_int.key \
        -CAcreateserial \
        -in server.csr \
        -out server.crt \
        -extfile server.cnf \
        -extensions v3_req
        openssl verify -CAfile ca.pem server.crt
    }
    
    make_agent() {
        echo "Creating Agent certificate and key"
        openssl req \
            -new \
            -newkey rsa:2048 \
            -nodes \
            -keyout agent.key \
            -out agent.csr \
            -config agent.cnf \
            -extensions v3_req
        openssl req -in agent.csr -noout -verify
        openssl x509 \
            -req \
            -days 365 \
            -CA ca.crt \
            -CAkey ca.key \
            -CAcreateserial \
            -in agent.csr \
            -out agent.crt \
            -extfile agent.cnf \
            -extensions v3_req
        openssl verify -CAfile ca.pem agent.crt
        for os in {1..10}
        do
            for nginx in agent
            do
                openssl req \
                -new \
                -newkey rsa:2048 \
                -nodes \
                -keyout $nginx-$os.key \
                -out $nginx-$os.csr \
                -config agent.cnf \
                -extensions v3_req
                openssl req -in $nginx-$os.csr -noout -verify
                openssl x509 \
                -req \
                -days 365 \
                -sha256 \
                -CA ca.crt \
                -CAkey ca.key \
                -CAcreateserial \
                -in $nginx-$os.csr \
                -out $nginx-$os.crt \
                -extfile agent.cnf \
                -extensions v3_req
                openssl verify -CAfile ca.pem $nginx-$os.crt
            done
        done
    }
    
    # MAIN
    make_ca
    make_int
    make_server
    make_agent
    

  3. Put the following OpenSSL .cnf files in the same directory.

    ca.cnf

    ca.cnf

    [req]
    default_bits        = 4096
    distinguished_name  = req_distinguished_name
    prompt              = no
    default_md          = sha256
    req_extensions      = v3_req
    
    [req_distinguished_name]
    countryName                 = US
    stateOrProvinceName         = California
    localityName                = San Francisco
    organizationName            = NGINX, Inc.
    commonName                  = nginx-manager_ca
    
    [v3_req]
    basicConstraints = critical, CA:true
    keyUsage = critical, keyCertSign, cRLSign
    subjectKeyIdentifier = hash
    subjectAltName = @alt_names
    
    [alt_names]
    DNS.1 = nginx-manager_ca.example.com
    DNS.2 = localhost
    IP.1 = 127.0.0.1
    
    ca-intermediate.cnf
    [req]
    default_bits        = 4096
    distinguished_name  = req_distinguished_name
    prompt              = no
    default_md          = sha256
    req_extensions      = v3_req
    
    [req_distinguished_name]
    countryName                 = US
    stateOrProvinceName         = California
    localityName                = San Francisco
    organizationName            = NGINX, Inc.
    commonName                  = nginx-manager_ca_int
    
    [v3_req]
    basicConstraints = critical, CA:true
    keyUsage = critical, keyCertSign, cRLSign
    subjectKeyIdentifier = hash
    subjectAltName = @alt_names
    
    [alt_names]
    DNS.1 = nginx-manager_ca_int.example.com
    DNS.2 = localhost
    IP.1 = 127.0.0.1
    
    server.cnf
    [req]
    prompt             = no
    default_bits       = 4096
    x509_extensions    = v3_req
    req_extensions     = v3_req
    default_md         = sha256
    distinguished_name = req_distinguished_name
    
    [req_distinguished_name]
    countryName                 = US
    stateOrProvinceName         = California
    localityName                = San Francisco
    organizationName            = NGINX, Inc.
    commonName                  = nginx-manager.example.com
    
    [v3_req]
    subjectAltName = @alt_names
    
    [alt_names]
    DNS.1 = nginx-manager.example.com
    DNS.2 = server
    DNS.3 = nginx-manager
    DNS.4 = localhost
    IP.1 = 127.0.0.1
    
    agent.cnf
    [req]
    prompt             = no
    default_bits       = 2048
    x509_extensions    = v3_req
    req_extensions     = v3_req
    default_md         = sha256
    distinguished_name = req_distinguished_name
    
    [req_distinguished_name]
    countryName                 = US
    stateOrProvinceName         = California
    localityName                = San Francisco
    organizationName            = NGINX, Inc.
    commonName                  = agent.example.com
    
    [v3_req]
    subjectAltName = @alt_names
    
    [alt_names]
    DNS.1 = agent.example.com
    DNS.2 = *.example.com
    
    

  4. Make the script executable and then run the script to generate the certificates.

    sudo chmod +x ./make_certs.sh
    sudo ./make_certs.sh
    
  5. Set permissions on the certificates and keys.

    sudo chmod 664 *.crt
    sudo chmod 640 *.key
    
  6. Rename and move the certificates into a directory.

    sudo cp ca.pem /etc/nms/certs/
    sudo cp server.crt /etc/nms/certs/
    sudo cp server.key /etc/nms/certs/
    sudo cp agent.crt /etc/nms/certs/
    sudo cp agent.key /etc/nms/certs/
    
  7. Add the ca.pem to the truststore or directly to the agent.conf. Update the servername

    Linux (CentOs, RedHat)

    sudo cp /etc/nms/certs/ca.pem /etc/pki/ca-trust/source/anchors/nim-ca.crt
    sudo update-ca-trust
    

    Linux (Ubuntu, Debian)

    sudo cp /etc/nms/certs/ca.pem /usr/local/share/ca-certificates/
    sudo update-ca-certificates
    

    Adding the CA directly to agent.conf

    # tls options
    tls:
        # enable tls in the nginx-agent setup for grpcs
        # default to enable to connect with secure connection but without client cert for mtls
        enable: true
        cert: /etc/nms/certs/agent.crt
        key: /etc/nms/certs/agent.key
        ca: /etc/nms/certs/ca.pem
    
  8. Modify the nginx-agent.conf file to resemble the following example. Note the TLS options that are configured. The enable:true option means gRPC is secured. In addition, the specified cert and key tell the nginx-agent to use client cert authentication with the NGINX proxy on the NGINX Instance Manager server. The ca.pem is included because the certs were generated from this CA, which must be the same for both the client and server. If the CA is publicly trusted, you can omit the ca option. Update the server metrics and command name to the name of the server running the NGINX Instance Manager server.

    See Also:
    For additional information about TLS configurations for the NGINX Agent, see the topic [NGINX Agent TLS Settings] getting-started/installation/nginx-agent-tls-settings.md
    /etc/nginx-agent/nginx-agent.conf
    #
    # /etc/nginx-agent/nginx-agent.conf
    #
    # Configuration file for NGINX Agent.
    #
    # This file is to track agent configuration values that are meant to be statically set. There
    # are additional agent configuration values that are set via the API and agent install script
    # which can be found in /etc/nginx-agent/agent-dynamic.conf.
    
    # specify the server grpc port to connect to
    server:
    # host of the control plane
    host: 127.0.0.1
    grpcPort: 443
    metrics: nginx-manager.example.com
    command: nginx-manager.example.com
    # tls options
    tls:
    # enable tls in the nginx-agent setup for grpcs
    # default to enable to connect with secure connection but without client cert for mtls
    enable: true
    cert: /etc/nms/certs/agent.crt
    key: /etc/nms/certs/agent.key
    ca: /etc/nms/certs/ca.pem
    log:
    # set log level (panic, fatal, error, info, debug, trace; default "info")
    level: info
    # set log path. if empty, don't log to file.
    path: /var/log/nginx-agent/
    # data plane status message / 'heartbeat'
    nginx:
    # path of NGINX logs to exclude
    exclude_logs: ""
    
    dataplane:
    sync:
        enable: true
    # poll interval for data plane status
    status:
        poll_interval: 30s
    metrics:
    # specify the size of a buffer to build before sending metrics
    bulk_size: 20
    # specify metrics poll interval
    report_interval: 1m
    collection_interval: 15s
    mode: aggregated
    
    # OSS NGINX default config path
    # path to aux file dirs can also be added
    config_dirs: "/etc/nginx:/usr/local/etc/nginx"
    
  9. Update the NGINX proxy on the NGINX Instance Manager server with the newly generated certs and reload the service.

    # Main external HTTPS server, needs port 443
    server {
        listen 443 ssl http2;
        root /var/www/nms;
        server_name _;
    
        ssl_protocols       TLSv1.1 TLSv1.2;
        ssl_ciphers         HIGH:!aNULL:!MD5;
        ssl_session_cache   shared:SSL:10m;
        ssl_session_timeout 10m;
    
        ssl_certificate         /etc/ssl/nginx-manager/nginx-manager.crt;
        ssl_certificate_key     /etc/ssl/nginx-manager/nginx-manager.key;
        ssl_client_certificate  /etc/ssl/nginx-manager/ca.pem;
    
  10. Restart the agent. Now, gRPC should be using client authentication. This setup also requires using SSL for the UI/API (server-side SSL only).


Troubleshooting

If the NGINX Agent and NGINX Instance Manager have issues communicating, take the following steps to troubleshoot the problem.

  1. Verify access and error logging are enabled to capture detailed information about errors and request processing in log files.

    The access log and error log are enabled by default in the http directive in the main NGINX configuration file:

    # nginx.conf
    http {
        ...
        access_log /var/log/nginx/access.log;
        error_log /var/log/nginx/error.log;
        ...
    }
    
  2. Check the log files for certificate errors. Ensure the server uses the correct certs and Certificate Authority (CA).