End of Sale Notice:

F5 NGINX is announcing the End of Sale (EoS) for NGINX Controller API Management Module, effective January 1, 2024.

F5 maintains generous lifecycle policies that allow customers to continue support and receive product updates. Existing NGINX Controller API- Management customers can continue to use the product past the EoS date. License renewals are not available after September 30, 2024.

See our End of Sale announcement for more details.

End of Sale Notice:

F5 NGINX is announcing the End of Sale (EoS) for NGINX Controller Application Delivery Module, effective January 1, 2024.

F5 maintains generous lifecycle policies that allow customers to continue support and receive product updates. Existing NGINX Controller Application Delivery customers can continue to use the product past the EoS date. License renewals are not available after September 30, 2024.

See our End of Sale announcement for more details.

Deploy NGINX Controller as a Resilient Cluster on AWS

This guide explains how to deploy F5 NGINX Controller as a multi-node resilient. cluster on AWS.

Overview

Complete the steps in this guide to deploy F5 NGINX Controller as a resilient, three-node cluster on AWS. A multi-node cluster ensures that NGINX Controller stays up even if one of the control-plane hosts becomes unavailable.

Failure Tolerance

To be resilient, a cluster requires three working nodes. That’s two nodes for a quorum and one node for failure tolerance.

If a node fails in a resilient cluster, NGINX Controller automatically redirects traffic to the other working nodes. A multi-node cluster is operational with only two nodes; however, a two-node cluster isn’t resilient to further failures. If one of the nodes in a multi-node cluster becomes degraded or fails, you must take action as soon as possible to recover or replace the failed node or risk losing resiliency.

Important:
The failover time can take up to 5 minutes when a node fails. During this time, NGINX Controller may be unavailable while services are migrated and restarted. Resiliency will be restored once there are three working nodes in the cluster.

The following table shows how many nodes are needed for a cluster to have a quorum and what the failure tolerance is:

Cluster Size	Quorum	Failure Tolerance
1	1	0
2	2	0
3	2	1

Larger clusters aren’t supported.

Before You Begin

Implementation Considerations

Before installing or configuring NGINX Controller as a multi-node cluster, review the following list of considerations to assist with planning:

Configuring NGINX Controller as a multi-node cluster on AWS requires NGINX Controller 3.14 or later. To upgrade from an earlier version, refer to the Update NGINX Controller steps for instructions.
Data migration is not supported, so it’s not possible to implement a multi-node cluster with local volumes without reinstalling NGINX Controller.
If you plan to run NGINX Controller on AWS EC2 instances, we recommend using NFS shares for the external volumes. Using EBS shares for multi-node clusters is not recommended because of the EBS Availability Zone limitations; for example, the requirement to have EC2 instances and EBS volumes in the same Availability Zone.
Cluster config changes are orchestrated by a primary control plane node that writes to the external config database. Each NGINX Controller control plane node hosts a set of services (pods) that read and write data. Only the node that hosts the pod that manages the config data writes to the external config database.

Prerequisites

Important:
If you plan to run NGINX Controller on AWS EC2 instances, we recommend you use NFS shares for the external volumes. Using EBS shares for multi-node clusters is not recommended because of the EBS Availability Zone limitations.

Things you’ll need before installing NGINX Controller as a resilient cluster:

Three hosts on which you can install NGINX Controller to create a cluster
The controller-installer-<version>.tar.gz package, which you can get from the MyF5 Customer Portal. You need to upload and extract this tarball on each host.
A license file for NGINX Controller
A tool to send API requests, such as Postman or curl
An external volume for the config database

When installing NGINX Controller, you can choose to have NGINX Controller install and manage a self-hosted – also known as “embedded” – PostgreSQL database for you; this is the recommended implementation. Alternatively, you can install your own PostgreSQL database for the config database, which you manage; this is sometimes referred to as an “external config database” because it is externally managed by you. Regardless of whether you use an embedded or an externally managed config database, the config database must be on an external volume for resilient clusters.
An external volume for the analytics database

Configure IAM Roles

Important:
If you plan to run NGINX Controller on AWS EC2 instances, we recommend using NFS shares for the external volumes. Using EBS shares for multi-node clusters is not recommended because of the EBS Availability Zone limitations; for example, the requirement to have EC2 instances and EBS volumes in the same Availability Zone.

If you are installing NGINX Controller on AWS EC2 instances and plan to use EBS volumes for the analytics and/or config database, you will need to add an IAM role like the one shown below. This will also allow the automatic creation of Elastic Load Balancers (ELBs). Additionally, for successful automatic creation of ELBs, all the EC2 instances that are or will be part of the cluster must be tagged with the following key-value pair: kubernetes.io/cluster/NGINX-CONTROLLER : owned

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "autoscaling:DescribeAutoScalingGroups",
        "autoscaling:DescribeLaunchConfigurations",
        "autoscaling:DescribeTags",
        "ec2:DescribeInstances",
        "ec2:DescribeRegions",
        "ec2:DescribeRouteTables",
        "ec2:DescribeSecurityGroups",
        "ec2:DescribeSubnets",
        "ec2:DescribeVolumes",
        "ec2:CreateSecurityGroup",
        "ec2:CreateTags",
        "ec2:CreateVolume",
        "ec2:ModifyInstanceAttribute",
        "ec2:ModifyVolume",
        "ec2:AttachVolume",
        "ec2:AuthorizeSecurityGroupIngress",
        "ec2:CreateRoute",
        "ec2:DeleteRoute",
        "ec2:DeleteSecurityGroup",
        "ec2:DeleteVolume",
        "ec2:DetachVolume",
        "ec2:RevokeSecurityGroupIngress",
        "ec2:DescribeVpcs",
        "elasticloadbalancing:AddTags",
        "elasticloadbalancing:AttachLoadBalancerToSubnets",
        "elasticloadbalancing:ApplySecurityGroupsToLoadBalancer",
        "elasticloadbalancing:CreateLoadBalancer",
        "elasticloadbalancing:CreateLoadBalancerPolicy",
        "elasticloadbalancing:CreateLoadBalancerListeners",
        "elasticloadbalancing:ConfigureHealthCheck",
        "elasticloadbalancing:DeleteLoadBalancer",
        "elasticloadbalancing:DeleteLoadBalancerListeners",
        "elasticloadbalancing:DescribeLoadBalancers",
        "elasticloadbalancing:DescribeLoadBalancerAttributes",
        "elasticloadbalancing:DetachLoadBalancerFromSubnets",
        "elasticloadbalancing:DeregisterInstancesFromLoadBalancer",
        "elasticloadbalancing:ModifyLoadBalancerAttributes",
        "elasticloadbalancing:RegisterInstancesWithLoadBalancer",
        "elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer",
        "elasticloadbalancing:AddTags",
        "elasticloadbalancing:CreateListener",
        "elasticloadbalancing:CreateTargetGroup",
        "elasticloadbalancing:DeleteListener",
        "elasticloadbalancing:DeleteTargetGroup",
        "elasticloadbalancing:DescribeListeners",
        "elasticloadbalancing:DescribeLoadBalancerPolicies",
        "elasticloadbalancing:DescribeTargetGroups",
        "elasticloadbalancing:DescribeTargetHealth",
        "elasticloadbalancing:ModifyListener",
        "elasticloadbalancing:ModifyTargetGroup",
        "elasticloadbalancing:RegisterTargets",
        "elasticloadbalancing:DeregisterTargets",
        "elasticloadbalancing:SetLoadBalancerPoliciesOfListener",
        "iam:CreateServiceLinkedRole",
        "kms:DescribeKey"
      ],
      "Resource": [
        "*"
      ]
    }
  ]
}

Install NGINX Controller

Complete the steps in the NGINX Controller Installation Guide to install NGINX Controller on the first node.

License NGINX Controller

Follow the steps to license NGINX Controller.

Add Nodes to the Cluster

Nodes are additional control-plane hosts that you can add to your cluster to improve uptime resilience. For a resilient cluster, you should have at least three nodes, of which two nodes must always be operational.

Important:
When adding a third node to the cluster for the first time, NGINX Controller may become momentarily unavailable while the cluster is being created. For this reason, we recommend updating NGINX Controller during a planned maintenance window to minimize disruptions.

Take the following steps to add a node to the cluster:

Open the NGINX Controller web interface and log in.
Select the NGINX Controller menu icon, then select Platform.
On the Platform menu, select Cluster.
On the Cluster overview page, select Create Node.
Add a name for the node.
(Optional) Add a description.
Add the hostname or IP address – or both – for the node.
Select Save. The new node appears in the list of nodes on the Cluster overview page with a Configuring status.
Choose the new node’s name in the list, then select View (eye icon). A page displays with command-line instructions for adding the node.
Copy the install.sh command and join-key that are shown.
Open an SSH connection to the node that you’re adding to the cluster.
(Optional) If you’re adding a node that was previously deleted, uninstall NGINX Controller from the node if you haven’t already, and then continue with the remaining steps in this procedure:
```
/opt/nginx-controller/uninstall.sh
```
Upload and extract the controller-installer-<version>.tar.gz tarball.
Run the install.sh command with the join-key that you copied in the previous step. If you get an error that the join-key has expired, you can get a new one by following the steps in this topic to add a node using the web interface or the NGINX Controller REST API.
```
cd controller-installer
./install.sh --join-key <long-value>
```
After the installation finishes, the node status in the web interface changes to Configured.
Repeat these steps for each node that you want to add to the cluster.

See Also:
To add nodes to your cluster using the NGINX Controller REST API, send a POST request to the /platform/nodes endpoint.

Add Load Balancer Alias to FQDN

You must add the hostname or IP address for the load balancer as a CNAME or A record for the domain that’s used as the Fully Qualified Domain Name (FQDN) for NGINX Controller.

To get the hostname or IP address for the load balancer using the NGINX Controller REST API, send a GET request to the /platform/global endpoint.

Delete a Node

There might be situations when you need to delete a node, either temporarily for maintenance or permanently to decommission a node.

If you need to remove a node temporarily, follow the steps in the Add Nodes to the Cluster topic when you are ready to re-add it. Make sure to uninstall NGINX Controller from the node before re-installing NGINX Controller with the new join-key.

Important:
Deleting nodes can cause NGINX Controller to become momentarily unavailable while the cluster is being updated. For this reason, we recommend updating NGINX Controller during a planned maintenance window to minimize disruptions. When deleting nodes, make sure that at least two nodes are always operational. If the cluster has fewer than two working nodes, NGINX Controller may become unresponsive, and you may not be able to add new nodes.

See Also:
To delete nodes from your cluster using the NGINX Controller API Reference, send a DELETE request to the Nodes endpoint.

To delete a node from the cluster using the web interface:

Open the NGINX Controller web interface and log in.
Select the NGINX Controller menu icon, then select Platform.
On the Platform menu, select Cluster.
On the Cluster overview page, choose the node you want to delete, then select Delete (trash icon).
Select Delete to confirm.
To finish deleting a node from the cluster, uninstall NGINX Controller from the node:
1. SSH into the node that you’re deleting from the cluster.
2. Run the NGINX Controller uninstall script:
```
/opt/nginx-controller/uninstall.sh
```

See Also:
To delete nodes from your cluster using the NGINX Controller REST API, send a DELETE request to the /platform/nodes endpoint.

Replace a Failed Node

To be resilient, a cluster requires three working nodes. That’s two nodes for a quorum and one node for failure tolerance.

If one of the nodes in a multi-node cluster becomes degraded or fails, you must take action as soon as possible to recover or replace the failed node or risk losing resiliency.

To replace a failed node:

Delete the failed node.
Add a new node.

Updating a Cluster

When updating NGINX Controller on a multi-node cluster, run the update.sh script on each node individually – the order in which you update the nodes doesn’t matter.

Warning:
Do not update the nodes in a multi-node cluster in parallel. Doing so may result in race conditions for certain jobs, such as database migrations, and may cause the cluster to become unavailable.

Important:
Active users will be logged out from NGINX Controller during an update. We recommend updating NGINX Controller during a planned maintenance window to minimize disruptions.

To update your cluster to a newer version of NGINX Controller, take the following steps:

Before updating the cluster, check each node’s status to confirm the nodes are healthy. Resolve any degradations before updating.
Download the new installer package from the MyF5 Customer Portal.
Extract the installer package and save the contents to each node:
```
tar xzf controller-installer-<version>.tar.gz
```
Run the update script on each node – the order in which you update the nodes doesn’t matter:
```
cd controller-installer
./update.sh
```