NGINX App Protect DoS Troubleshooting Guide
Learn about the NGINX App Protect DoS Troubleshooting Guide.
This Troubleshooting Guide is intended to provide guidance to customers in the detection and correction of programming issues in NGINX App Protect DoS. It may also be useful to IT.
Resolving Known Problems
|NGINX is not running (ps -aux)
Reloading NGINX fails
|Check the error log at
Fix the problem and re-run NGINX.
|No original source IP in logs||1. XFF is not configured (or not configured correctly)
2. External Load Balancer doesn’t forward XFF
|NGINX App Protect DoS functionality is not as expected||NGINX App Protect DoS has several logs which can be used for troubleshooting.
Usually, it is best to look for any warning or error messages within the logs.
Refer to Logs Overview
||Increase number of file descriptors.
Refer to worker_rlimit_nofile directive
||Increase the limit using the following command as the root user:
Refer to Issue 4: Too many files are open Error
|More protected objects than expected||The
Consider moving this directive from outer to inner block.
Refer to: NGINX App Protect DoS - Directives and Policy
||There are more nginx processes than allowed.
Either decrease the number of nginx processes (ngx_processes directive in
||App Protect DOS module is not loaded. Add this line to the main (global) context of nginx.conf:
|NGINX struggles handling a high rate of incoming connections||Linux machine should be tuned for optimal performance.
Refer to Tuning NGINX for Performance
||Insufficient memory to allocate all the required resources.
ELK issues are addressed directly in GitHub by posting the issue to Kibana dashboards for F5 NGINX App Protect DoS GitHub repo.
Configure SELinux to allow NGINX App Protect DoS.
The configuration steps are found in the SELinux configuration section of the deployment guide.
If SELinux still denies access to something, it means that one of more security exceptions should be enabled.
The following steps describe how to find the problematic exception and enable it.
- Temporarily add the
httpd_tdomain to the permissive list (this will completely enable all the fields).
In this configuration SELinux will not deny anything related to NGINX as NGINX is labeled with the
semanage permissive -a httpd_t
- Repeat the scenario which made SELinux deny and see that it now works.
- In permissive mode, security exceptions are logged to the default Linux audit log.
Due to the previous step, the permitted exception will be logged.
The log can be found in
- The following command will parse the audit log and build a SELinux command that will permit all the exceptions found in the log:
grep nginx /var/log/audit/audit.log | audit2allow -m nginx
- Compare the generated output to the nginx.te file mentioned in the deployment guide. Add all the missing commands to the nginx.te file and repeat the SELinux configuration mentioned in the deployment guide.
- Delete the
httpd_tdomain from the permissive list:
semanage permissive -d httpd_t
For more information about how to use NGINX Plus with SELinux - check our blog
Send Logs to Support
If there are any problems, collect the troubleshooting information in a tarball and send it to your customer support engineer.
Get package version:
a. Get NGINX App Protect DoS version:
/usr/bin/admd -v > package_versions.txt
b. Get packages version:
rpm -qa nginx-plus* app-protect* >> package_versions.txt
apt list --installed | grep -E 'nginx-plus|app-protect' >> package_versions.txt
c. Get OS version:
cat /etc/os-release > system_version.txt && uname -r >> system_version.txt && cat /proc/version >> system_version.txt
d. Get NGINX App Protect DoS shared memory dump:
admd -c > napd_shmem.txt
e. Get Linux shared memory dump:
ipcs -m > linux_shmem.txt
Create a list of files for tarball:
a. Create a file using your favorite editor (i.e VI editor)
b. Insert the following content into the file created above:
package_versions.txt system_version.txt napd_shmem.txt linux_shmem.txt /var/log/adm/* /var/run/adm/* /var/log/nginx/*
c. Add the path of your NGINX configuration files including all references, for example:
d. Add all policies and log file configuration, for example:
Create the tarball:
tar cvfz logs.tgz `cat logs.txt`
logs.tgzto your customer support.
This documentation applies to the following versions of NGINX App Protect DoS: 4.1.