NGINX App Protect WAF Security Log

Security Logs

Security Logs Overview

Security logs (also known as Request logs or Traffic logs) contain information on HTTP requests and responses, how F5 NGINX App Protect WAF processes them, and the final decision made based on the configured policy parameters. The policy configuration defines the information contained in the Security log, such as whether requests are passed, blocked or alerted, due to violations, attack signatures, and other criteria.

NGINX App Protect WAF uses its own logging mechanism for request logging rather than NGINX’s access logging mechanism (which is NGINX’s default logging mechanism).

The Security log has the following properties:

• Log Configuration: app_protect_security_log directive referencing security_log.json file

• Configuration contexts: nginx.conf: http, server, location

• File Destination? Yes. You can set the destination to either stderr, or an absolute path to a local file, or you can use syslog, and redirect the log with Netcat and pipe:

  nc -vv -l “[ip]” “[port]” > “[name_of_file]” 2>&1
  

• Syslog Destination? Yes

Directives in nginx.conf

app_protect_security_log_enable

This directive determines whether security logging will be enabled in the respective context.

The security log attributes are determined by the app_protect_security_log directive. The directive can be at the following contexts: http, server and location. When not present at a certain context, the directive is inherited from the context above it: location from server, then from http. If there is no directive at any of these context levels, then the logging is disabled for the respective context.

• Syntax: app_protect_security_log_enable on | off

• Levels: http, server, location

• Example: app_protect_security_log_enable on

Arguments

app_protect_security_log

The security log attributes are determined by the app_protect_security_log directive, if it was enabled in the respective context. The directive can be at the following context levels: http, server and location. Multiple occurrences of this directive are allowed in a single context, and all the configured logs in this context will be used. When not present in a certain context, all the directives are inherited from the context above it: location from server, then from http. If there is no directive at any of these context levels, but logging is enabled then the default is used for the respective context.

• Syntax: app_protect_security_log [LOG-CONFIG-FILE] [DESTINATION]

• Levels: http, server, location

• Examples:

app_protect_security_log "/etc/app_protect/conf/log_default.json" stderr;
app_protect_security_log "/etc/app_protect/conf/log_default.json" /var/log/app_protect/security.log;
app_protect_security_log "/etc/app_protect/conf/log_default.json" syslog:server=localhost:514;
app_protect_security_log "/etc/app_protect/conf/log_default.json" syslog:server=my-specific-machine-name:514;
app_protect_security_log "/etc/app_protect/conf/log_default.json" syslog:server=192.168.12.34:514;
app_protect_security_log "/etc/app_protect/conf/log_default.json" syslog:server=my.domain.com:514;

• Note: When using stderr, make sure that the process bd-socket-plugin is not redirecting the stderr output to file.
- When using the Docker entrypoint.sh startup script from the admin guide, make sure that it doesn’t redirect stderr.
- When using services startup, make sure that the service startup file for nginx-app-protect.service doesn’t redirect stderr.

Arguments

Security Log Configuration File

The file is in JSON format and consists of two parts:

1. filter: which requests are to be logged.
2. content: how the message is formatted.

Filter

The filter is mandatory, although it may be left blank.

Content

This part of the configuration file specifies what will be logged, the format of the message, and size restrictions.

Content is mandatory. If the entire content field or any of its attributes are not defined, system-defined default values are used.

Examples

Default Logging Content

This is the content of /etc/app_protect/conf/log_default.json. It is used by default when app_protect_security_log_enabled on is set, but app_protect_security_log is not:

{
    "filter": {
        "request_type": "illegal"
    },
    "content": {
        "format": "default",
        "max_request_size": "2k",
        "max_message_size": "5k"
    }
}

Log Illegal Requests in Key-Value Format

{
    "filter": {
        "request_type": "illegal"
    },
    "content": {
        "format": "user-defined",
        "format_string": "client_ip=%ip_client%,client_port=%src_port%,request=%request%,violations=%violations%,signature_ids=%sig_ids%",
        "max_request_size": 2000,
        "max_message_size": "5k"
    }
}

Log State Changing Requests

{
    "filter": {
        "request_type": "all"
    },
    "content": {
        "format": "default",
        "max_request_size": "2k",
        "max_message_size": "5k"
    }
}

A Verbose Custom Formatted Message

{
    "filter": {
        "request_type": "illegal"
    },
    "content": {
        "format": "user-defined",
        "format_string": "Request ID %support_id%: %method% %uri% received on %date_time% from IP %ip_client% had the following violations: %violations%",
        "max_request_size": "2k",
        "max_message_size": "5k"
    }
}

Log with Escaped Character and Custom List Prefix / Delimiter / Suffix

{
    "filter": {
        "request_type": "all"
    },
    "content": {
        "format": "default",
        "max_request_size": "2k",
        "max_message_size": "5k",
        "escaping_characters": [
      	    {
                "from": "/",
                "to": "|"
            }
        ],
        "list_prefix": "[",
        "list_delimiter": "::",
        "list_suffix": "]"
    }
}

Note that in the last example:

• any / character in list element string in security log will be replaced with |. For example: string/another_string will become string|another_string.

• all lists will start with [ and end with ] and each element of the list will be separated with ::. For example: first,second,third will become [first::second::third].

NGINX Format Strings

When format = default or grpc, messages are shown in comma separated key-value pairs consisting of the attributes appearing in Available Security Log Attributes with some differences: grpc format contains the gRPC-specific fields and the request body encoded in Base64 and separated from the headers, while the default format contains the whole request in one field in its original encoding. See details there.

Both default and grpc strings start like this:

“attack_type=\"%attack_type%\",blocking_exception_reason=\"%blocking_exception_reason%\”,…"

Syslog Transport

The syslog transport is over TCP. It is currently unsecured, which means that SSL/TLS is not supported. We highly recommend that you do not send the logs directly to their remote destinations, but rather proxy them through a local syslog server residing on the same pod or same VM as NGINX App Protect WAF. The local syslog server will forward them over a secure channel to the remote destination. We recommend you use mutual authentication TLS (mTLS) to avoid any man-in-the-middle attacks attempting to hijack or alter the logs on their way.

It is not guaranteed that all requests that match the filters will indeed reach their destination especially if the system is overwhelmed by the incoming traffic. In this case some log records may be dropped.

Factory Configuration Files

NGINX will provide example configuration files under /opt/app_protect/share/defaults/ with the following settings:

Available Security Log Attributes

The table below lists attributes that are generated in the security logs. When using customized logs (i.e., format=user-defined), you can add or remove entries from the list below. Per each attribute we show whether it is included in each of the predefined formats: default and grpc.

Blocking Observability

When the NGINX App Protect policy is enforced in Transparent Mode, it is easier to know which transactions would be blocked if the Blocking Mode is set to True and also would be able to know which violations and signatures intended for the transaction to be blocked.

To facilitate this, NGINX App Protect introduces a new security log field named json_log, which contains JSON formatted data. This field contains the violations and applicable signature names and IDs associated with a transaction. This field will indicate if a particular violation or signature will create a blocking condition (indicated in the isBlocked property) according to the signature/violation configuration.

Each detected signature is reported within a JSON block with the VIOL_ATTACK_SIGNATURE violation and the isBlocked boolean property.

Here is an example of json_log field for a request with several violations and signatures:

{
  "violations": [
    {
      "enforcementState": {
        "isBlocked": false
      },
      "violation": {
        "name": "VIOL_URL_METACHAR"
      }
    },
    {
      "enforcementState": {
        "isBlocked": true
      },
      "violation": {
        "name": "VIOL_RATING_THREAT"
      }
    },
    {
      "enforcementState": {
        "isBlocked": true
      },
      "signature": {
        "name": "XSS script tag (URI)",
        "signatureId": 200000099
      },
      "violation": {
        "name": "VIOL_ATTACK_SIGNATURE"
      }
    },
    {
      "enforcementState": {
        "isBlocked": true
      },
      "signature": {
        "name": "XSS script tag end (URI)",
        "signatureId": 200000093
      },
      "violation": {
        "name": "VIOL_ATTACK_SIGNATURE"
      }
    }
  ]
}