NGINX Controller Releases
NGINX Controller Version 2.9.0
These release notes provide general information and describe known issues for NGINX Controller version 2.9.0, in the following categories:
Updates
API Management
- Preview: Developer Portal:
This release includes a preview of the API Management Module’s Developer Portal feature. Admins can choose to add API documentation – including descriptions, sample requests, and sample responses – for APIs published through Controller. Users can view the API documentation in the preview of the dev portal.
Known Issues
Kubernetes components that remained configured/running after failed Controller installation prevented reinstall:
When a Controller installation failed after the kubernetes components were installed, the Kubernetes processes continued to run and configuration was not reset. Subsequent attempts to install the Controller fail because required ports are already in use.
Workaround:Check the output of the failed install process and the install log in
/var/log/nginx-controller/to determine why the install process failed. Often, this can occur due to database connectivity or authentication issues.You will need to resolve this root cause prior to attempting reinstallation of controller.
When you are ready to reinstall, run the commands shown below and then follow the prompts to remove the Kubernetes configuration.
sudo kubeadm reset mv $HOME/.kube $HOME/.kube-old
Proceed with re-installing the Controller.
Controller 2.9 release is incompatible with Controller Agent v2.5.0:
There was a non-backwards-compatible change between Agent 2.5.0 and later Agent versions. As a result, API Management configuration publish events from a Controller 2.9 to a v2.5.0 Agent fail with a timeout error.
Workaround:
Install the Agent by running the commands provided in the “Add new instance” dialog from your agent host.API Management module allows upload of multiple JWT authentication policies for the same environment:
When two JWT client group authentication policies are applied to a single environment, publishing to the gateway will fail. Workaround:
Remove the additional JWT authentication policy, then try to publish the config.Upgrade from 2.6.0 Incorrectly Migrated TLS/SSL Policy from API Management Config:
The upgrade from Controller v2.6.0 incorrectly migrated the API Management TLS/SSL Policy resources. Subsequent attempts to edit and publish the APIM config failed.Workaround:
- Create a new TLS/SSL Policy and add it to the affected endpoint.
- Remove the broken TLS/SSL Policy from the endpoint.
- Save and publish the APIM config.
Controller-agent update overwrites agent.conf customizations:
The controller-agent update process does not respect any manual edits you might have made to the agent.conf file.Workaround:
If you have made manual edits to the agent.conf file, take the following steps when you update the agent:- Backup your current agent.conf file prior to executing the agent upgrade procedure.
- Perform the upgrade by reapplying the steps of the New Instance dialog.
- Restore your agent.conf file over the file in
/etc/controller-agent/agent.conf - Restart the controller-agent process.
NGINX Controller Version 2.8.1
Resolved Issues
Upgrade from Controller v2.7.3 Removed JWT Policy from API Management Config
The upgrade from Controller v2.7.3 to Controller v2.8.0 removed a JWT policy from the API Management config. Attempts to publish the APIM config failed due to the missing auth policy. This issue is resolved in Controller v2.8.1.
Unable to Publish API Management Config with TLS/SSL Policy
In the Controller-2.8.0 release, attempts to publish an API Management configuration with a TLS/SSL policy failed with an error message indicating that the certificate could not be loaded. This issue is resolved Controller-2.8.1.
Known Issues
Upgrade from 2.6.0 Incorrectly Migrated TLS/SSL Policy from API Management Config:
The upgrade from Controller v2.6.0 incorrectly migrated the API Management TLS/SSL policy resources. Subsequent attempts to edit and publish the APIM config failed.
Workaround:
- Create a new TLS/SSL policy and add it to the affected endpoint.
- Remove the broken TLS/SSL policy from the endpoint.
- Save and publish the APIM config.
API Management Config Publication Failed After Renaming a JWT Client Group:
Renaming a client group breaks the link between the client group and its previously published JWT policy. Subsequent attempts to publish the API Management config failed.
Workaround:
Do not rename any client groups that are referenced in an active JWT policy.
If you have already done so, take the following steps:
- Delete the policy that references the renamed client group.
- Delete the client group.
- Create a new client group.
- Associate the relevant
.jwkfile with the new client group. - Configure a JWT policy that references the new client group.
- Publish the config.
Controller 2.8 release is incompatible with Controller Agent v2.5.0:
There was a non-backwards-compatible change between Agent 2.5.0 and later Agent versions. As a result, API Management configuration publish events from a Controller 2.8 to a v2.5.0 Agent fail with a timeout error.
Workaround:
Install the Agent by running the commands provided in the “Add new instance” dialog from your agent host.
Controller-agent update overwrites agent.conf customizations:
The controller-agent update process does not respect any manual edits you might have made to the agent.conf file. If you have made manual edits to the agent.conf file, please take the following steps when you update the agent:
- Backup your current agent.conf file prior to executing the agent upgrade procedure.
- Perform the upgrade by reapplying the steps in the “Add new instance” dialog (as noted below).
- Restore your agent.conf file over the file in /etc/controller-agent/agent.conf.
- Restart the controller-agent process.
NGINX Controller Version 2.8.0
These release notes provide general information and describe known issues for NGINX Controller version 2.8.0, in the following categories:
Updates
NGINX Controller runs on Kubernetes:
This release introduces Kubernetes as an orchestration tool for the NGINX Controller containers.
Support for Debian 8 is deprecated in this release.
API Management
JWT Client Groups now available:
API Management users can choose between using API keys and JWTs for Client Groups. JWT Client Groups allow users to specify a
.jwkby providing a URL.Client Group API keys can be stored as hashed values:
API keys can now be stored as SHA-256 hashed values on each API gateway. The JavaScript module
njsmust be installed on each NGINX Plus instance to enable this feature.Updated Default Log Format for API Management:
The default log format for API Management enables the full range of metrics for graphs, dashboards, and alerts. This resolves a previous issue where publishing an APIM configuration overwrote custom log formats.
Known Issues
NGINX Controller 2.8 requires specific Kubernetes packages to be held at version 1.15.3:
After installing the Controller, exclude these packages from package upgrades:
- kubeadm
- kubectl
- kubelet
Install
yum-plugin-versionlock:yum install yum-plugin-versionlockFor each package to exclude, run
yum versionlock <package name>.
On Debian or Ubuntu:
- For each package to exclude, run
apt-mark hold <package name>.
Swap must be disabled on Controller host prior to install/upgrade:
Per the Kubernetes requirements, you must disable swap on the host where Kubernetes will run. This must be done prior to installing the Nginx Controller or upgrading from a previous version (2.7.0 or earlier) of the Controller.
You can find more information about this requirement in the Kubernetes kubeadm documentation.
Please consult your OS Administrator and/or OS-specific documentation for more information about how to permanently disable swap on your machine.
Direct Upgrade from Controller 2.0.0 to Controller 2.8.0 fails:
If you are using v2.0.0 and follow the upgrade process to install v2.8.0, the process will fail with an error.
Workaround:
Upgrade to Controller v2.7.3 first. Then, upgrade to Controller v2.8.0.
The API Management module does not support IP addresses when setting the location of a remote .jwk file for JWT Client Groups:
The API Management module supports the use of JWT for Client Groups beginning in this release.
You can either upload a local .jwk file or specify the URL via which a remote file can be accessed.
If you provide an IP address when specifying the remote URL, a validation error that states the input is invalid will occur.
The remote key file can only be saved if the URL is specified as a fully-qualified domain name (FQDN) .
Workaround:
Provide the FQDN and path to the .jwk file instead of using the server IP address.
API Management module allows upload of multiple JWT authentication policies for the same environment:
The API Management module supports the use of JWT for Client Groups beginning in this release. When creating a Client Group, the system will allow you to create more than one JWT authentication policy for a single environment. If two JWT client group authentication policies are applied to a single environment, publishing to the gateway will fail. When you try to publish the config, you will see the following error: “Only one JWT policy can be added to an environment”.
Workaround:
Remove the additional JWT authentication policy and then try to publish the config.
Upgrade from Controller v2.6.0 Removed TLS/SSL Policy from API Management Config:
The upgrade from Controller v2.6.0 removed a TLS/SSL policy from the API Management config. Attempts to publish the APIM config failed due to the missing auth policy. Attempts to add a new TLS/SSL policy and then publish the config also failed.
Workaround:
Remove, then re-create, the environment that contains the affected policy.
Controller 2.8.0 release is incompatible with Controller Agent v2.5.0:
There was a non-backwards-compatible change between Agent 2.5.0 and later Agent versions. As a result, API Management configuration publish events from a Controller 2.8.0 to a v2.5.0 Agent fail with a timeout error.
Workaround:
Install the Agent by running the commands provided in the “Add new instance” dialog from your agent host.
Controller-agent update overwrites agent.conf customizations:
The controller-agent update process does not respect any manual edits you might have made to the agent.conf file. If you have made manual edits to the agent.conf file, please take the following steps when you update the agent:
- Backup your current agent.conf file prior to executing the agent upgrade procedure.
- Perform the upgrade by reapplying the steps in the “Add new instance” dialog (as noted below).
- Restore your agent.conf file over the file in /etc/controller-agent/agent.conf.
- Restart the controller-agent process.
Supported Versions
- NGINX Plus R15, R16, R17, R18, R19*
* NGINX Controller 2.8 doesn’t provide GUI support for the status_zone directive in the location block for R19.
NGINX Controller Version 2.7.0
These release notes provide general information and describe known issues for NGINX Controller version 2.7.0, in the following categories:
Updates
Analytics
- This release introduces the new NGINX Controller Analytics module. With an Analytics module license, users can monitor, troubleshoot, and get insights into their NGINX Plus environments. For more information, please reach out to [email protected].
API Management
Read-Only Mode for API Management:
User accounts with the “Read-Only” role are able to view API Management configurations but cannot make any edits.
Default Proxy Mode:
HTTP keepalive policies can now be applied to upstream groups. Proxied requests are now made with HTTP/1.1 (previously HTTP/1.0).
Audit Log:
The Audit Log lets you view all actions performed by Controller users over a set date range. Audit log results can be sorted and filtered by each column.
Reusable TLS Policies:
TLS policies are now managed independently and can be associated with one or more Entry Points. A TLS policy includes the protocols, cipher settings, and session cache settings. The TLS certificate and private key are still Entry Point attributes.
Known Issues
Upgrade from Controller v2.6.0 Removed TLS/SSL Policy from API Management Config:
The upgrade from Controller v2.6.0 removed a TLS/SSL policy from the API Management config. Attempts to publish the APIM config failed due to the missing auth policy. Attempts to add a new TLS/SSL policy and then publish the config also failed.
Workaround:
Remove, then re-create, the environment that contains the affected policy.
Controller version displayed in UI is v2.7.3:
For the 2.7.0 release, the Controller UI shows the version number as 2.7.3.
Controller 2.7 release is incompatible with Controller Agent v2.5.0:
There was a non-backwards-compatible change between Agent 2.5.0 and later Agent versions. As a result, API Management configuration publish events from a Controller 2.7 to a v2.5.0 Agent fail with a timeout error.
Workaround:
Upgrade the Agent to the latest version by following the instructions provided below to bring the system back into working order.
Audit Log’s “Apply” button is disabled when redefining a custom range:
Steps to reproduce:
- Create a custom date range on the new Audit Log page and click “Apply”.
- Make changes to the custom date range.
- The “Apply” button is not active.
Workaround:
- Select and apply a predefined date range, then select a custom range.
Upgrade to 2.7.0 doesn’t remove unused Docker container:
During an update from any earlier version of the controller to v2.7, a previously-used Docker container is left running and emits a warning during the upgrade as follows:
“Found orphan containers (controller_amplify-extapi_1) for this project. If you removed or renamed this service in your compose file, you can run this command with the –remove-orphans flag to clean it up.”
In the next release, this container will be cleaned up automatically. To remove it manually in v2.7, you can run the commands shown below:
sudo docker stop controller_amplify-extapi_1 sudo docker rm controller_amplify-extapi_1
Controller-agent update overwrites agent.conf customizations:
The controller-agent update process does not respect any manual edits you might have made to the agent.conf file. If you have made manual edits to the agent.conf file, please take the following steps when you update the agent:
- Backup your current agent.conf file prior to executing the agent upgrade procedure.
- Perform the upgrade by reapplying the steps in the “Add new instance” dialog (as noted below).
- Restore your agent.conf file over the file in /etc/controller-agent/agent.conf.
- Restart the controller-agent process.
Documentation for upgrading the controller agent is out-of-date:
The instructions provided in the Admin guide and in the in-product documentation contain outdated instructions for upgrading the controller agent.
To upgrade the agent, you should run the commands provided in the “Add new instance” dialog from your agent host. This will download and run a newer version of the agent.
Delay in API initialization post upgrade:
For a period of about five minutes following updating the controller installation, a generic “Unknown” error banner appears in the UI when accessing the API Management pages. Approximately five minutes following update completion, this self-resolves and the application behaves normally.
Workaround: You can manually dismiss the “Unknown” error banner.
Deletion of API Definition not reflected in GUI until page refresh:
When you delete an API Definition in the API Module (which is present only if licensed), the deleted definition still appears in the GUI. The definition is, in fact, being deleted correctly. The UI is not updating to reflect this fact.
Workaround: You can refresh the page, or navigate away and then return, to view the correct state of the application.
NGINX Controller Version 2.6.0
These release notes provide general information and describe known issues for NGINX Controller version 2.6.0, in the following categories:
Known Issues
API Management policies applied may inherit a broader scope than intended.
To resolve this issue, upgrade to Controller version 2.7.
Controller-agent update overwrites agent.conf customizations:
The controller-agent update process does not respect any manual edits you might have made to the agent.conf file. If you have made manual edits to the agent.conf file, please take the following steps when you update the agent:
- Backup your current agent.conf file prior to executing the agent upgrade procedure.
- Perform the upgrade by reapplying the steps in the “Add new instance” dialog (as noted below).
- Restore your agent.conf file over the file in /etc/controller-agent/agent.conf.
- Restart the controller-agent process.
Documentation for upgrading the controller agent is out-of-date:
The instructions provided in the Admin guide and in the in-product documentation contain outdated instructions for upgrading the controller agent.
To upgrade the agent, you should run the commands provided in the “Add new instance” dialog from your agent host. This will download and run a newer version of the agent.
API Management returns 504 Gateway Timeout for a period of minutes following upgrade from 2.5.0 to 2.6.0:
For a period of about five minutes following updating the controller installation, a generic “Unknown” error appear in the GUI when accessing the API Management pages. Approximately five minutes following update completion, this self-resolves and the application behaves normally.
Note that the “Unknown” error, which presents as a banner in the UI, must be manually dismissed.
Deleted API definition remains on the page:
When you delete an API Definition in the API Module (present only if licensed), the deleted definition still appears in the GUI. The definition is, in fact, being deleted correctly. The UI is not updating to reflect this fact. You can refresh the page or navigate away and then return to show the correct state of the application.
NGINX Controller Version 2.5.0
These release notes provide general information and describe known issues for NGINX Controller version 2.5.0, in the following categories:
Known Issues
- Clients deploying controller-agent on CentOS 6 need to manually modify the logging level of controller-agent to ERROR level from INFO and restart controller-agent.
- Administrators who delete a JWT policy will still see that policy in the environment select list. They should not select/configure a deleted policy.
- NGINX Plus instances managed by the API Management module must be associated with a single Entry Point
- Amazon Linux 2. Docker installation issues: Controller install script cannot handle the setup of docker. Please install docker before running the controller install script.
NGINX Controller Version 2.4.0
These release notes provide general information and describe known issues for NGINX Controller version 2.4.0, in the following categories:
Updates
Load Balancer
- The Instances page is now read-only, displaying the configuration that is currently running on each instance. This will help ensure nobody accidentally overwrites expected policy. To edit instance configurations, you must use Configuration Templates (Load Balancing > Configurations).
- If an instance is associated to a Load Balancing Configuration Template or an API Management Entry Point, that information is reflected at the bottom of the page.
- Instances that are not associated with a module can be promoted to Configuration Templates at the bottom of the Instances page for load balancing purposes, or associated with an Entry Point for API Management purposes.
- Configuration Templates > Instances
- You can now associate instances to a Configuration using tags.
- Last Modified timestamp now shows the last time the configuration was changed, whether manually or through Controller.
Known Issues
- Amazon Linux 2. Docker installation issues: Controller install script cannot handle the setup of docker. Please install docker before running the controller install script.
- NGINX Plus instances managed by the API Management module must be associated with a single Entry Point.
NGINX Controller Version 2.3.0
These release notes provide general information and describe known issues for NGINX Controller version 2.3.0, in the following categories:
Updates
Platform
- Configure dashboard graph sets using one or more tags
- Support spaces in upstream group name, route alias, system alias
Load Balancer
- Configuration Templates
- Ability to preview config file
- Additional feedback around in-progress config push
API Management
- Ability to edit environment policies
- Usability improvements to API Definitions page
- Entry point hostname color-coding:
- Grey - config not pushed to the entry point
- Green - config pushed, all associated instances are online
- Yellow - config pushed, one or more instances offline & one or more instances online
- Red - config pushed, all instances offline
- Number of unrouted resources is displayed for each environment
- Entry point hostname color-coding:
Known Issues
- Amazon Linux 2. Docker installation issues: Controller install script cannot handle the setup of docker. Please install docker before running the controller install script.
NGINX Controller Version 2.2.0
These release notes provide general information and describe known issues for NGINX Controller version 2.2.0, in the following categories:
Updates
NGINX Controller Version 2.1.0
These release notes provide general information and describe known issues for NGINX Controller version 2.1.0, in the following categories:
NGINX Controller Version 2.0.0
These release notes provide general information and describe known issues for NGINX Controller version 2.0.0, in the following categories:
Updates
Load Balancer
- Configuration Templates
- Config
- Associated Instances
- Version History
- Version Differencing
- Arbitrary Reverting
- Multi-push from ‘Load Balancing > Instances’
- Notifications for multi-push status
- ServiceNow alerting integration
- Ability to enable error_log and access_log in virtual server
- Renamed Amplify metrics
- Rename tab from “Configure” to “Load Balancing”
- Ability to tag instances from the ‘Load Balancing > Instances’ tab
- Config-push deferral - ability to change a config and save it without pushing
- Promote session persistence to its own section (Upstream Group config)
- Offline installer / updater
- Various bug-fixes
Known Issues
- If upgrading from a previous version of Controller, password needs to be updated/reset.
- Load Balancing - Instances page hangs when there are no hosts registered in Controller.
- Load Balancing - Orange indicators will appear when you have unsaved changes, and may take several minutes to accurately reflect the correct status.
- API Management - When configuring multiple rate limit policies for an Environment, each must use the same status code.
NGINX Controller Version 1.0.1
These release notes provide general information and describe known issues for NGINX Controller version 1.0.1, in the following categories:
Updates
- Alerts - fix issue where editing an alert with threshold could result in an error
- Analyzer - fix issue where old SSL cert could still be evident in the report alongside the new cert that replaced it
- Analyzer - show appropriate message when port is missing for the listen directive
- Configure - provide better error descriptions in the UI
- Configure - fix issue where error notifications disappeared too quickly before they could be fully read
- Configure - fix issue where user would see “undefined” error when loading a config that included a stream block
- Configure - fix issue where virtual server SSL block is inactive after a page reload
- Configure - fix issue where empty directives were not copied to the generated config file
- Configure - fix issue where proxy is not allowed in regex location
- Configure - fix issue where the validation for healthcheck URI does not allow for query params
- Configure - fix issue of no validation for duplicate upstreams
- Configure - fix the wrong visualisation of healthcheck config after a push config action fails
- Configure - fix error about server name being too long in the generated config