NGINX App Protect DoS Security Log

Learn about the NGINX App Protect Security Log.

Overview

Security logs contain information about the status of the protected objects. It gives a general picture about each protected object in terms of traffic intensity, health of the backend server, learning and mitigations.

There are several types of logs, each contains different information and published either periodically or upon an important event.

Dictionary

The following table lists all the possible fields in the logs and their meaning.

Field Meaning
date_time the date and time of the event
product always set to app-protect-dos
product_version NGINX App Protect DoS version
unit_hostname host name of the app-protect-dos instance
instance_id instance ID: container id from /proc/self/cgroupor hostname if container is is not available
vs_name A unique identifier (representing the protected object’s name) of the location in the nginx.conf file that this request is associated with. It contains the line number of the containing server block in nginx.conf, the server name, a numeric discriminator that distinguishes between multiple entries within the same server, and the location name.
For example: 34-mydomain.com:0-~/.*php(2).
dos_attack_id unique attack IP per unit_hostname
attack_event Event name as it appears in remote logger.
stress_level a number from 0 to … that reflects stress level.
baseline_tps learned TPS
incoming_rps current RPS
successful_tps current TPS
unsuccessful_rps unsuccessful requests per second (passed to server and not responded: reset / timeout / 5xx
incoming_requests incremental number of incoming requests
successful_transactions incremental number of successful transactions
unsuccessful_requests incremental number of unsuccessful requests (passed to server and not responded: reset / timeout / 5xx
active_connections current number of active server connections
threshold_rps global rate rps threshold
threshold_connections active connections threshold
mitigated_bad_actors
redirect_bad_actor
challenge_bad_actor
block_bad_actor
incremental number of mitigated bad actors. Increments upon any type of bad actors mitigations.
incremental number of http redirections sent to detected bad actors
incremental number of JS challenges sent to detected bad actors
incremental number of blocked bad actors
mitigated_by_signatures
redirect_signature
challenge_signature
block_signature
incremental number of requests mitigated by signatures. Increments upon any type of signatures mitigations.
incremental number of http redirections sent to clients when requests match a signature.
incremental number of JS challenges sent to clients when requests match a signature.
incremental number of blocked requests when requests match a signature.
mitigated_by_global_rate
redirect_global
challenge_global
block_global
incremental number of requests mitigated by global_rate. Increments upon any type of global rate mitigations.
incremental number of http redirections sent to clients upon global rate mitigation.
incremental number of JS challenges sent to clients upon global rate mitigation.
incremental number of blocked requests upon global rate mitigation.
mitigated_slow
redirect_slow
challenge_slow
block_slow
incremental number of mitigated slow requests. Increments upon any type of slow requests mitigations.
incremental number of http redirections sent to clients upon slow request mitigation.
incremental number of JS challenges sent to clients upon slow request mitigation.
incremental number of blocked slow requests.
mitigated_connections incremental number of mitigated by connections mitigation
mitigated_bad_actors_rps
redirect_bad_actor_rps
challenge_bad_actor_rps
block_bad_actor_rps
mitigated_bad_actors rps. Includes any type of bad actors mitigations.
http redirections per second sent to detected bad actors.
JS challenges per second sent to detected bad actors.
blocked bad actors per second.
mitigated_by_signatures_rps
redirect_signature_rps
challenge_signature_rps
block_signature_rps
mitigated_signatures rps. Includes any type of signatures mitigations.
http redirections sent per second to clients when requests match a signature.
JS challenges per second sent to clients when requests match a signature.
blocked requests per second when requests match a signature.
mitigated_slow_rps
redirect_slow_rps
challenge_slow_rps
block_slow_rps
mitigated slow requests per second. Includes any type of slow requests mitigations.
http redirections per second sent to clients upon slow request mitigation.
JS challenges per second sent to clients upon slow request mitigation.
blocked slow requests per second.
mitigated_by_global_rate_rps
redirect_global_rps
challenge_global_rps
block_global_rps
mitigated_global_rate rps. Includes any type of global rate mitigations.
http redirections per second sent to clients upon global rate mitigation.
JS challenges per second sent to clients upon global rate mitigation.
blocked requests per second upon global rate mitigation.
mitigated_connections_rps mitigated_connections rps
source_ip <br> impact_rps ip address of the detected bad actor 1.1.1.1
RPS created by bad actor in the time of the detection (to be calculated as a max hitcount in AMT / 10)
new_bad_actors_detected
bad_actors
the number of newly detected bad actors
the number of bad actors
signature
signature_id
signature_efficiency
signature_accuracy
signature string http.request.method eq GET and http.uri_parameters eq 6
unique signature ID per unit_host
estimated efficiency upon signature detection: percentage of bad traffic covered by the signature
estimated accuracy upon signature detection: percentage of learned good traffic NOT covered by the signature
learning_confidence the possible values are not ready/bad actors only/ready

Events

1a. Attack notification

Reported about attack start/end and ongoing attack major parameters.

a. Example: Attack Started

date_time="Apr 29 2021 13:59:53", product="app-protect-dos", product_version="23+1.54.1-1.el7.ngx", unit_hostname="localhost.localdomain", instance_id="d9a6d8", vs_name="example.com/", dos_attack_id="1", attack_event="Attack started", stress_level="1.00", learning_confidence="Ready", baseline_tps="16", incoming_rps="211", successful_tps="0", unsuccessful_rps="0", incoming_requests="9761", successful_transactions="5301", unsuccessful_requests_count="0", active_connections="80", threshold_rps="41.60", threshold_conns="41.60", mitigated_bad_actors="0", mitigated_by_signatures="0", mitigated_by_global_rate="0", mitigated_slow="0", redirect_global="0", redirect_bad_actor="0", redirect_signature="0", redirect_slow="0", challenge_global="0", challenge_bad_actor="0", challenge_signature="0", challenge_slow="0", block_global="0", block_bad_actor="0", block_signature="0", block_slow="0", mitigated_connections="0", mitigated_bad_actors_rps="0", mitigated_by_signatures_rps="0", mitigated_by_global_rate_rps="0", mitigated_slow_rps="0", redirect_global_rps="0", redirect_bad_actor_rps="0", redirect_signature_rps="0", redirect_slow_rps="0", challenge_global_rps="0", challenge_bad_actor_rps="0", challenge_signature_rps="0", challenge_slow_rps="0", block_global_rps="0", block_bad_actor_rps="0", block_signature_rps="0", block_slow_rps="0", mitigated_connections_rps="0",

b. Example: Attack Ended

date_time="Apr 29 2021 14:05:14", product="app-protect-dos", product_version="23+1.54.1-1.el7.ngx", unit_hostname="localhost.localdomain", instance_id="d9a6d8", vs_name="example.com/", dos_attack_id="1", attack_event="Attack ended", stress_level="0.50", learning_confidence="Ready", baseline_tps="12", incoming_rps="0", successful_tps="0", unsuccessful_rps="0", incoming_requests="363004", successful_transactions="7754", unsuccessful_requests_count="0", active_connections="0", threshold_rps="2121.60", threshold_conns="2121.60", mitigated_bad_actors="161491", mitigated_by_signatures="187495", mitigated_by_global_rate="1057", mitigated_slow="0", redirect_global="1057", redirect_bad_actor="161491", redirect_signature="187495", redirect_slow="0", challenge_global="0", challenge_bad_actor="0", challenge_signature="0", challenge_slow="0", block_global="0", block_bad_actor="0", block_signature="0", block_slow="0", mitigated_connections="0", mitigated_bad_actors_rps="0", mitigated_by_signatures_rps="0", mitigated_by_global_rate_rps="0", mitigated_slow_rps="0", redirect_global_rps="0", redirect_bad_actor_rps="0", redirect_signature_rps="0", redirect_slow_rps="0", challenge_global_rps="0", challenge_bad_actor_rps="0", challenge_signature_rps="0", challenge_slow_rps="0", block_global_rps="0", block_bad_actor_rps="0", block_signature_rps="0", block_slow_rps="0", mitigated_connections_rps="0",

1b. Traffic/Mitigation summary stats

Reported periodically, supplies aggregated stats per protected object periodically.
Corresponds to the metrics reported by main Grafana screen.

a. Example: No Attack

date_time="Apr 29 2021 13:59:31", product="app-protect-dos", product_version="23+1.54.1-1.el7.ngx", unit_hostname="localhost.localdomain", instance_id="d9a6d8", vs_name="example.com/", dos_attack_id="0", attack_event="No Attack", stress_level="0.50", learning_confidence="Ready", baseline_tps="16", incoming_rps="12", successful_tps="12", unsuccessful_rps="0", incoming_requests="5260", successful_transactions="5260", unsuccessful_requests_count="0", active_connections="0", threshold_rps="2121.60", threshold_conns="2121.60", mitigated_bad_actors="0", mitigated_by_signatures="0", mitigated_by_global_rate="0", mitigated_slow="0", redirect_global="0", redirect_bad_actor="0", redirect_signature="0", redirect_slow="0", challenge_global="0", challenge_bad_actor="0", challenge_signature="0", challenge_slow="0", block_global="0", block_bad_actor="0", block_signature="0", block_slow="0", mitigated_connections="0", mitigated_bad_actors_rps="0", mitigated_by_signatures_rps="0", mitigated_by_global_rate_rps="0", mitigated_slow_rps="0", redirect_global_rps="0", redirect_bad_actor_rps="0", redirect_signature_rps="0", redirect_slow_rps="0", challenge_global_rps="0", challenge_bad_actor_rps="0", challenge_signature_rps="0", challenge_slow_rps="0", block_global_rps="0", block_bad_actor_rps="0", block_signature_rps="0", block_slow_rps="0", mitigated_connections_rps="0",

b. Example: Under Attack

date_time="Apr 29 2021 14:04:05", product="app-protect-dos", product_version="23+1.54.1-1.el7.ngx", unit_hostname="localhost.localdomain", instance_id="d9a6d8", vs_name="example.com/", dos_attack_id="1", attack_event="Under Attack", stress_level="0.50", learning_confidence="Ready", baseline_tps="12", incoming_rps="1466", successful_tps="10", unsuccessful_rps="0", incoming_requests="351983", successful_transactions="7663", unsuccessful_requests_count="0", active_connections="0", threshold_rps="506.41", threshold_conns="506.41", mitigated_bad_actors="150563", mitigated_by_signatures="187495", mitigated_by_global_rate="1057", mitigated_slow="0", redirect_global="1057", redirect_bad_actor="150563", redirect_signature="187495", redirect_slow="0", challenge_global="0", challenge_bad_actor="0", challenge_signature="0", challenge_slow="0", block_global="0", block_bad_actor="0", block_signature="0", block_slow="0", mitigated_connections="0", mitigated_bad_actors_rps="1455", mitigated_by_signatures_rps="0", mitigated_by_global_rate_rps="0", mitigated_slow_rps="0", redirect_global_rps="0", redirect_bad_actor_rps="1455", redirect_signature_rps="0", redirect_slow_rps="0", challenge_global_rps="0", challenge_bad_actor_rps="0", challenge_signature_rps="0", challenge_slow_rps="0", block_global_rps="0", block_bad_actor_rps="0", block_signature_rps="0", block_slow_rps="0", mitigated_connections_rps="0",

2. Bad actor detection/expiration

Reports NGINX App Protect DoS decisions about bad actors.

a. Example: Bad Actor Detection

date_time="Apr 29 2021 14:03:01", product="app-protect-dos", product_version="23+1.54.1-1.el7.ngx", unit_hostname="localhost.localdomain", instance_id="d9a6d8", vs_name="example.com/", dos_attack_id="1", attack_event="Bad actor detection", source_ip="5.5.5.9", impact_rps="30",

b. Example: Bad Actor Expired

date_time="Apr 29 2021 14:05:29", product="app-protect-dos", product_version="23+1.54.1-1.el7.ngx", unit_hostname="localhost.localdomain", instance_id="d9a6d8", vs_name="example.com/", dos_attack_id="0", attack_event="Bad actor expired", source_ip="5.5.5.10", impact_rps="271438576",

3. Attack signatures

Reports NGINX App Protect DoS decisions about signatures.

Example: Attack Signature Detected

date_time="Apr 29 2021 14:02:56", product="app-protect-dos", product_version="23+1.54.1-1.el7.ngx", unit_hostname="localhost.localdomain", instance_id="d9a6d8", vs_name="example.com/", dos_attack_id="1", attack_event="Attack signature detected", signature="(http.user_agent_header_exists eq true) and (http.accept contains other-than(application|audio|message|text|image|multipart)) and (http.unknown_header_exists eq true) and (http.headers_count neq 10) and (http.x_forwarded_for_header_exists eq false) and (http.uri_parameters eq 1) and (http.uri_len between 48-63) and (http.accept_header_exists eq true) and (http.hdrorder not-hashes-to 55) and (http.connection_header_exists eq true) and (http.accept_encoding_header_exists eq true) and (http.request.method eq reserved) and (http.cookie_header_exists eq true) and (http.uri_file hashes-to 7) and (http.host_header_exists eq true)", signature_id="809655398", signature_efficiency="72.00", signature_accuracy="100.00",

4. Bad actors detection information

Reports NGINX App Protect DoS information about bad actors

Example: Bad Actors Detected

date_time="Apr 29 2021 14:02:00", product="app-protect-dos", product_version="23+1.54.1-1.el7.ngx", unit_hostname="localhost.localdomain", instance_id="d9a6d8", vs_name="example.com/", dos_attack_id="1", attack_event="Bad actors detected", new_bad_actors_detected="2", bad_actors="2",

Security Log Configuration File

The file is in JSON format and consists of two parts:

  1. filter: which messages are to be logged.
  2. content: how the message is formatted.

Filter

The filter is mandatory, although it may be left blank.

Element Description Type/Values Default
traffic-mitigation-stats This filter element refers to Traffic/Mitigation summary stats. Enumerated values:
- all
- none
all
bad-actors This filter element refers to Bad actor detection/expiration, every 10 seconds. Enumerated values:
- all
- none
- top N
top 10
attack-signatures This filter element refers to Attack Signatures, every 10 seconds. Enumerated values:
- all
- none
- top N
top 10

Content

This part of the configuration file specifies what will be logged, the format of the message, and size restrictions.
Content is mandatory. If the entire content field or any of its attributes are not defined, system-defined default values are used.

Element Meaning Type/Values Mandatory Default
format Selects one of the predefined formats of log messages or a custom format that will be defined by the format_string field. Enumerated values:
- splunk: formatted for Splunk SIEM with F5 plugin.
- arcsight: formatted according to ArcSight Common Event Format (CEF) with custom fields adapted for F5.
- user-defined: custom format defined by the user in the format_string field.
No splunk
max_message_size Limit in KB for the total size of the message. Range of values is between 1k-64k, must not be smaller than the max_request_size. No 5k
format_string_attack_notification Layout template of the logged fields in the log message for Attack notification. Example: “date=%date_time%, vs=%vs_name%, rps=%incoming_rps%” If, and only if, format=user-defined
format_string_traffic_mitigation_stats Layout template of the logged fields in the log message for Traffic/Mitigation summary stats. Example: “date=%date_time%, vs=%vs_name%, attack=%attack_event%, rps=%incoming_rps%” If, and only if, format=user-defined
format_string_bad_actor Layout template of the logged fields in the log message for Bad actor detection/expiration. Example: “date=%date_time%, vs=%vs_name%, attack=%attack_event%, ip=%source_ip% If, and only if, format=user-defined
format_string_signatures Layout template of the logged fields in the log message for Attack Signatures. Example: “date=%date_time%, vs=%vs_name%, attack=%attack_event%, signature=%signature%” If, and only if, format=user-defined
format_string_bad_actors_info Layout template of the logged fields in the log message for Bad actors detection information. Example: “date=%date_time%, vs=%vs_name%, attack=%attack_event%, bad_actors_detected=%new_bad_actors_detected%” If, and only if, format=user-defined

Examples

Example 1:

{
    "filter": {
        "bad-actors": "top 100",
        "attack-signatures": "top 100"
    },
    "content": {
        "format": "splunk",
        "max_message_size": "5k"
    }
}

Example 2:

{
    "filter": {
        "traffic-mitigation-stats": "none",
        "bad-actors": "top 10",
        "attack-signatures": "top 10"
    },
    "content": {
        "format": "user-defined",
        "format_string_attack notification": "date=%date_time%, vs=%vs_name%, rps=%incoming_rps%",
        "format_string_traffic-mitigation-stats": "date=%date_time%, vs=%vs_name%, attack=%attack_event%, rps=%incoming_rps%",
        "format_string_bad_actor": "date=%date_time%, vs=%vs_name%, attack=%attack_event%, ip=%source_ip%",
        "format_string_signatures": "date=%date_time%, vs=%vs_name%, attack=%attack_event%, signature=%signature%",
        "format_string_bad_actors_info": "date=%date_time%, vs=%vs_name%, attack=%attack_event%, bad_actors_detected=%new_bad_actors_detected%",
        "max_message_size": "5k"
    }
}

This documentation applies to the following versions of NGINX App Protect DoS: 1.0.