Set Up App Protect Instances for Security Monitoring

Learn how to set up NGINX App Protect data plane instances for use with the NGINX Management Suite Security Monitoring and Instance Manager modules.

Overview

NGINX Management Suite Security Monitoring supports the following use cases:

Security Monitoring only: Use only the Security Monitoring module to monitor data from NGINX App Protect WAF instances. You will be able to review the security dashboards to assess potential threats and identify opportunities to fine-tune your policies. Your NGINX App Protect WAF configurations are managed outside of the NGINX Management Suite context.
Security Monitoring and Instance Manager: Use the Security Monitoring module with the NGINX Management Suite Instance Manager module. In addition to monitoring your application security, you will be able to manage your NGINX App Protect WAF configurations and security policies in a single location and push pre-compiled updates to an instance or instance group.

Before You Begin

Complete the following prerequisites before proceeding with the steps in this guide.

If you are new to NGINX App Protect WAF, follow the instructions in the installation and configuration guides to get up and running:
- Install NGINX App Protect WAF on one or more data plane instances. Each data plane instance must have connectivity to the NGINX Management Suite host.
- Configure NGINX App Protect WAF according to your needs on each of the data plane instance.

Review the dependencies with NGINX App Protect WAF and NGINX Plus.

The Security Monitoring module requires the following versions of NGINX App Protect WAF and NGINX Plus for the data plane:

Security Monitoring	NGINX App Protect WAF
1.7.1	Release 4.7.0 - 4.9.0
1.7.0	Release 4.4.0 - 4.7.0
1.6.0	Release 4.3.0 - 4.4.0
1.5.0	Release 4.3.0
1.4.0	Release 4.2.0
1.3.0	Release 3.12.2 - 4.2.0
1.2.0	Release 3.12.2 - 4.1.0
1.1.0	Release 3.12.2 - 4.0.0
1.0.0	Release 3.12.2

Determine your use case: Security Monitoring only or Security Monitoring and Configuration Management.
Install the NGINX Management Suite Security Monitoring module and upload your license.

Install NGINX Agent

NGINX Agent is a companion daemon for NGINX Open Source or NGINX Plus instance that provides:

Remote management of NGINX configurations
Collection and reporting of real-time NGINX performance and operating system metrics
Notifications of NGINX events

Repeat the steps in this section on each NGINX App Protect WAF data plane host to install and configure NGINX Agent for use with Security Monitoring. These settings apply to both of the Security Monitoring use cases.

Use SSH to connect to the data plane host.
Install the NGINX Agent package from the NGINX Management Suite host.

Note: To complete this step, make sure that gpg is installed on your system. You can install NGINX Agent using various command-line tools like curl or wget. If your NGINX Management Suite host is not set up with valid TLS certificates, you can use the insecure flags provided by those tools. See the following examples:
- curl
- wget
- Secure:
  curl https://<NMS_FQDN>/install/nginx-agent | sudo sh
- Insecure:
  curl --insecure https://<NMS_FQDN>/install/nginx-agent | sudo sh
  You can add your NGINX instance to an existing instance group or create one using --instance-group or -g flag when installing NGINX Agent.
  
  The following example shows how to download and run the script with the optional --instance-group flag adding the NGINX instance to the instance group my-instance-group:
  curl https://<NMS_FQDN>/install/nginx-agent > install.sh; chmod u+x install.sh sudo ./install.sh --instance-group my-instance-group
  By default, the install script attempts to use a secure connection when downloading packages. If, however, the script cannot create a secure connection, it uses an insecure connection instead and logs the following warning message:
  Warning: An insecure connection will be used during this nginx-agent installation
  To require a secure connection, you can set the optional flag skip-verify to false.
  
  The following example shows how to download and run the script with an enforced secure connection:
  curl https://<NMS_FQDN>/install/nginx-agent > install.sh chmod u+x install.sh; chmod u+x install.sh sudo sh ./install.sh --skip-verify false
- Secure:
  wget https://<NMS_FQDN>/install/nginx-agent -O - | sudo sh -s --skip-verify false
- Insecure:
  wget --no-check-certificate https://<NMS_FQDN>/install/nginx-agent -O - | sudo sh
  When you install the NGINX Agent, you can use the --instance-group or -g flag to add your NGINX instance to an existing instance group or to a new group that you specify.
  
  The following example downloads and runs the NGINX Agent install script with the optional --instance-group flag, adding the NGINX instance to the instance group my-instance-group:
  wget https://gnms1.npi.f5net.com/install/nginx-agent -O install.sh ; chmod u+x install.sh sudo ./install.sh --instance-group my-instance-group

Edit the /etc/nginx-agent/nginx-agent.conf file to add the nap_monitoring configuration.

Add the lines below to the end of the file. This enables NGINX Agent to send NGINX App Protect messages to the NGINX Management Suite management plane.

dataplane:
   status:
      # poll interval for data plane status - the frequency the NGINX Agent will query the data plane for changes
      poll_interval: 30s
      # report interval for data plane status - the maximum duration to wait before syncing data plane information if no updates have been observed
      report_interval: 24h
events:
   # report data plane events back to the management plane
   enable: true
metrics:
   # specify the size of a buffer to build before sending metrics
   bulk_size: 20
   # specify metrics poll interval
   report_interval: 1m
   collection_interval: 15s
   mode: aggregated

# OSS NGINX default config path
# path to aux file dirs can also be added
config_dirs: "/etc/nginx:/usr/local/etc/nginx:/usr/share/nginx/modules:/etc/nms:/etc/app_protect"

# Enable reporting NGINX App Protect details to the management plane.
extensions:
  - nginx-app-protect
  - nap-monitoring

# Enable reporting NGINX App Protect details to the control plane.
nginx_app_protect:
   # Report interval for NGINX App Protect details - the frequency the NGINX Agent checks NGINX App Protect for changes.
   report_interval: 15s
   # Enable precompiled publication from the NGINX Management Suite (true) or perform compilation on the data plane host (false).
   precompiled_publication: true

# NGINX App Protect Monitoring config
nap_monitoring:
   # Buffer size for collector. Will contain log lines and parsed log lines
   collector_buffer_size: 50000
   # Buffer size for processor. Will contain log lines and parsed log lines
   processor_buffer_size: 50000
   # Syslog server IP address the collector will be listening to
   syslog_ip: "127.0.0.1"
   # Syslog server port the collector will be listening to
   syslog_port: 514

If the location /api directive has not been set up in the nginx.conf file, follow the example below to add it:

server{
           location /api {
               api write=on;
               allow 127.0.0.1;
               deny all;
           }
}

After adding the directive, restart NGINX to apply the changes:

sudo systemctl restart nginx

Important:
You can change the values of syslog_ip and syslog_port to meet your needs. You must use the same values when configuring logging for the Security Monitoring module. If the syslog:<server><port> configuration does not match these settings, the monitoring dashboards will not display any data.

Note:

You can use the NGINX Agent installation script to add the fields for nginx_app_protect and nap_monitoring:

# Download install script via API
curl https://<NMS_FQDN>/install/nginx-agent > install.sh

# Use the flag --nap-monitoring to set the child fields for the field 'nap_monitoring', the
# child field values will be set to the values in the example configuration from above. Specify
# the -m | --nginx-app-protect-mode flag to set up management of NGINX App Protect on the instance.
# In the example below we specify 'precompiled-publication' for the flag value which will make the
# config field 'precompiled_publication' set to 'true', if you would like to set the config field
# 'precompiled_publication' to 'false' you can specify 'none' as the flag value.
sudo sh ./install.sh --nap-monitoring true --nginx-app-protect-mode precompiled-publication

Restart NGINX Agent:
```
sudo systemctl restart nginx-agent
```

Set Up Instances for Security Monitoring Only

Complete the steps in this section if you are only using the Security Monitoring module to monitor your application security. In this use case, you are not using Instance Manager to manage your WAF security policies.

Repeat the steps below on each NGINX App Protect WAF data plane instance.

Use SSH to connect to the data plane host.
Create a new log format definition file with the name /etc/app_protect/conf/log_sm.json and the contents shown below. This defines the log format for the Security Monitoring module.

This configuration sets the maximum accepted request payload to 2048 bytes and the maximum message size to 5k. The latter setting truncates messages larger than 5k.

Add character escaping for the used separator , to be escaped with its standard URL encoding %2C.

{
    "filter": {
        "request_type": "illegal"
    },
    "content": {
        "format": "user-defined",
        "format_string": "%blocking_exception_reason%,%dest_port%,%ip_client%,%is_truncated_bool%,%method%,%policy_name%,%protocol%,%request_status%,%response_code%,%severity%,%sig_cves%,%sig_set_names%,%src_port%,%sub_violations%,%support_id%,%threat_campaign_names%,%violation_rating%,%vs_name%,%x_forwarded_for_header_value%,%outcome%,%outcome_reason%,%violations%,%violation_details%,%bot_signature_name%,%bot_category%,%bot_anomalies%,%enforced_bot_anomalies%,%client_class%,%client_application%,%client_application_version%,%transport_protocol%,%uri%,%request%",
        "escaping_characters": [
            {
                "from": ",",
                "to": "%2C"
            }
        ],
        "max_request_size": "2048",
        "max_message_size": "5k",
        "list_delimiter": "::"
    }
}

Find the context in your NGINX configuration where NGINX App Protect WAF logging is enabled. In the same context, add the app_protect_security_log directive shown in the example below to configure attack data logging for use with the Security Monitoring dashboards.
```
   app_protect_security_log_enable on;
   app_protect_security_log "/etc/app_protect/conf/log_sm.json" syslog:server=127.0.0.1:514;
```
Important:
The syslog:server=<syslog_ip>:<syslog_port> must match the syslog_ip and syslog_port values specified in the NGINX Agent configuration file. The dashboards won’t display any data if these settings don’t match.

Restart NGINX Agent and the NGINX web server.

sudo systemctl restart nginx-agent
sudo systemctl restart nginx

You should now be able to view data from your NGINX App Protect instances in the NGINX Management Suite Security Monitoring dashboards.

Set up Instances for Security Monitoring with Instance Manager

Complete the steps in this section if you want to use the Security Monitoring module and Instance Manager. In this use case, you will use NGINX Management Suite to monitor threats and to manage your NGINX App Protect WAF configurations and security policies.

Take the steps below to update your NGINX App Protect WAF configurations by using Instance Manager.

Log in to the NGINX Management Suite user interface and go to Modules > Instance Manager.
Select Instances or Instance Groups, as appropriate.
Select Edit Config from the Actions menu for the desired instance or instance group.
Next, edit the desired configuration file. You will add directives that reference the security policies bundle and enable the NGINX App Protect WAF logs required by the Security Monitoring dashboards. An example configuration is provided below.
```
   app_protect_policy_file "/etc/nms/NginxDefaultPolicy.tgz";
   app_protect_security_log_enable on;
   app_protect_security_log "/etc/nms/secops_dashboard.tgz" syslog:server=127.0.0.1:514;
```
- Add the app_protect_policy_file directive with a reference to a security policy.
  
  The policy reference must use the .tgz file extension when using Instance Manager to perform precompiled publication of NGINX App Protect WAF policies and log profiles. The file path referenced must exist on the NGINX Management Suite host, but it’s ok if the policy file doesn’t exist yet. If your Instance is not configured for precompiled publication, then use the .json file extension for polcies and log profiles. In this case, the file path referenced in the NGINX configuration must reside on the Instance.
  
  If you are using custom security policies, at this stage, it’s fine to use the default security policy shown in the example above. After completing the steps in this guide, refer to the instructions in Set Up App Protect WAF Configuration Management to add your custom security policy files to NGINX Management Suite and update your NGINX configuration.
- Add the app_protect_security_log_enable on and the app_protect_security_log directive to any NGINX context where NGINX App Protect WAF is enabled and you want to be able to review attack data.
  
  The logging configuration must reference "/etc/nms/secops_dashboard.tgz", as shown in the example.
  
  If the app_protect_security_log_enable setting is already present, just add the app_protect_security_log beneath it in the same context.
  
  Important:
  The syslog:server=<syslog_ip>:<syslog_port> must match the syslog_ip and syslog_port values specified in the NGINX Agent configuration file. The Security Monitoring dashboards won’t display any data if these settings don’t match.
Select Publish to immediately push the configuration file updates out to your NGINX instance or instance group.

You should now be able to view data from your NGINX App Protect WAF instances in the Security Monitoring dashboard.

What’s Next

Grant Users Access to the Security Monitoring Dashboards: Follow the steps in this guide to allow other users in your organization to access the Security Monitoring Dashboards.
If you are using Security Monitoring with Instance Manager, proceed to the Set Up App Protect WAF Configuration Management guide.