Deploy Instance Manager 2.17 and later

Introduction

This guide provides a step-by-step tutorial on how to set up F5 Instance Manager on a Kubernetes cluster using Helm. Learn how to download and use Docker images, and customize your deployment.

Note:

This guide applies to Instance Manager 2.17 and later.

For older versions, view the Deploy Instance Manager 2.16 and earlier topic.

About Instance Manager

NGINX Instance Manager allows you to configure, scale, and manage NGINX Open Source and NGINX Plus instances at scale. Instance Manager provides a REST API and web-based graphical user interface (GUI) for managing NGINX instances across multiple servers, making it easier to configure, monitor, and troubleshoot NGINX deployments.

Instance Manager can be used to manage instances running on-premises, in the cloud, or in hybrid environments, and it supports the deployment of NGINX instances on a variety of operating systems and container platforms.

Instance Manager also includes advanced features like health checks, rolling updates, and configuration backups, which help to ensure the reliability and security of NGINX deployments.

About Helm

Helm charts are pre-configured packages of Kubernetes resources deployed with a single command, which allow you to to define, install, and upgrade Kubernetes applications.

They are composed of a set of files that describe a related group of Kubernetes resources, including deployments, services, and ingress. Helm charts can define and manage dependencies between various applications, allowing for the development of complex, multi-tier applications.

Warning:

NGINX Instance Manager does not support OpenShift.

For best compatibility, use NGINX Ingress Controller.

Before You Begin

To deploy Instance Manager using a Helm chart, you need the following:

Requirements Notes
Docker 20.10 or later (linux/amd64) https://docs.docker.com/get-docker/
Kubernetes 1.21.3 or later (linux/amd64) Ensure your client can access the Kubernetes API server. Note: by default the Helm chart will enable persistent storage using the default storage class configured in your Kubernetes cluster. Documentation around this topic can be found here: Dynamic Volume Provisioning.
kubectl 1.21.3 or later https://kubernetes.io/docs/tasks/tools/#kubectl
Helm 3.10.0 or later https://helm.sh/docs/intro/install/
OpenSSL 1.1.1 or later https://www.openssl.org/source/
tar 1.20 or later

The tar archiving tool is usually installed by default. To see which version of tar you have installed, run tar --version.

If tar is not installed or the version is too old, follow the instructions for your Linux distribution to install a supported version from a package manager such as YUM (CentOS, RHEL) or APT (Debian, Ubuntu).


Obtain the F5 NGINX Instance Manager Images

Using Docker

Configure Docker to Access the NGINX Instance Manager Public Registry

To configure Docker to communicate with the NGINX container registry located at private-registry.nginx.com, follow these steps:

  1. Download your NGINX Instance Manager subscription’s JSON Web Token and license from MyF5.
    • Log in to the MyF5 customer portal.
    • Go to My Products and Plans > Subscriptions.
    • Select the product subscription.
    • Download the JSON Web Token (JWT) and license files.
  2. Open the JWT file you downloaded from MyF5 and copy its contents.
  3. Log in to the Docker registry using the contents of the JWT file:
docker login private-registry.nginx.com --username=<JWT_CONTENTS> --password=none
Note:
Ensure there are no extra characters or whitespaces when copying the contents of the JWT token: they can invalidate the token and cause 401 errors during authentication.

Pull the NGINX Instance Manager images

You can now pull the images needed for NGINX Instance Manager from private-registry.nginx.com.

Replace <version-tag> with the specific version you want to use.

Note:
latest is not a supported tag.
docker pull private-registry.nginx.com/nms/apigw:<version-tag>
docker pull private-registry.nginx.com/nms/core:<version-tag>
docker pull private-registry.nginx.com/nms/dpm:<version-tag>
docker pull private-registry.nginx.com/nms/ingestion:<version-tag>
docker pull private-registry.nginx.com/nms/integrations:<version-tag>
docker pull private-registry.nginx.com/nms/utility:<version-tag>

If necessary, you can push these images to your own private registry.

Push Images to Your Private Registry

After pulling the images, tag them and upload them to your private registry.

Log in to your private registry:

docker login <my-docker-registry>

Tag and push the image. Replace with your registry’s path and with the version you’re using, for example 2.17.0:

docker tag private-registry.nginx.com/nms/apigw:<version-tag> <my-docker-registry>/nms/apigw:<version-tag>
docker push <my-docker-registry>/nms/apigw:<version-tag>
docker tag private-registry.nginx.com/nms/core:<version-tag> <my-docker-registry>/nms/apigw:<version-tag>
docker push <my-docker-registry>/nms/core:<version-tag>
docker tag private-registry.nginx.com/nms/dpm:<version-tag> <my-docker-registry>/nms/apigw:<version-tag>
docker push <my-docker-registry>/nms/dpm:<version-tag>
docker tag private-registry.nginx.com/nms/ingestion:<version-tag> <my-docker-registry>/nms/apigw:<version-tag>
docker push <my-docker-registry>/nms/ingestion:<version-tag>
docker tag private-registry.nginx.com/nms/integrations:<version-tag> <my-docker-registry>/nms/apigw:<version-tag>
docker push <my-docker-registry>/nms/integrations:<version-tag>
docker tag private-registry.nginx.com/nms/utility:<version-tag> <my-docker-registry>/nms/apigw:<version-tag>
docker push <my-docker-registry>/nms/utility:<version-tag>

Direct Helm Usage with a JWT Token

If a private registry is unnecessary, a JWT token can be used as a Docker configuration secret with Helm charts.

Create a docker-registry secret type on the cluster, using the JWT token for the username and none for the password. The name of the docker server is private-registry.nginx.com.

The parameter --docker-username=<JWT Token> must be the contents of the token, not a reference to it.

Note:
Ensure there are no extra characters or whitespaces when copying the contents of the JWT token: they can invalidate the token and cause 401 errors during authentication.
kubectl create secret docker-registry regcred --docker-server=private-registry.nginx.com --docker-username=<JWT Token> --docker-password=none
Warning:

You will encounter a warning that using --password is insecure.

It can be safely ignored in this instance (No password is used), but if others have access to this system, you should delete the JWT token and clear your shell history once finished with the deployment.

Confirm the details of the created secret by running:

kubectl get secret regcred --output=yaml

You can now use the newly created Kubernetes secret in Helm deployments and point the charts directly to the public registry.


Add Helm Repository

Note:
To complete the steps in the section, you need to have Helm 3.10.0 or later installed.

Run the following commands to install the NGINX Management Suite chart from the Helm repository:

helm repo add nginx-stable https://helm.nginx.com/stable
helm repo update

The first command, helm repo add nginx-stable https://helm.nginx.com/stable, adds the nginx-stable repository to your local Helm repository list. This repository contains the Helm charts for deploying NGINX Management Suite.

The second command, helm repo update, updates the local Helm repository list with the newest versions of the charts from the nginx-stable repository. This command ensures you have the most up-to-date version of the charts available for installation.


Create a Helm Deployment values.yaml File

values.yaml is a Helm configuration file you can use to customize the installation of a Helm chart without editing the chart itself. Values can be used to specify different image repositories and tags, set environment variables, configure resource requests and limits, and more.

  1. Create a values.yaml file similar to the following example:

    • Replace <my-docker-registry:port> with your private Docker registry and port (if needed).

    • In the imagePullSecrets section, add the credentials for your private Docker registry.

      See Also:
      For instructions on creating a secret, see the Kubernetes topic Pull an Image from a Private Registry.
    nms-hybrid:
        imagePullSecrets:
            - name: regcred
        apigw:
            image:
                repository: <my-docker-registry:port>/nms/apigw
                tag: <version>
        core:
            image:
                repository: <my-docker-registry:port>/nms/core
                tag: <version>
        dpm:
            image:
                repository: <my-docker-registry:port>/nms/dpm
                tag: <version>
        ingestion:
            image:
                repository: <my-docker-registry:port>/nms/ingestion
                tag: <version>
        integrations:
            image:
                repository: <my-docker-registry:port>/nms/integrations
                tag: <version>
        utility:
            image:
                repository: <my-docker-registry:port>/nms/utility
                tag: <version>
    

    This values.yaml file specifies the Docker images to be used for the apigw, core, dpm, ingestion, integrations and utility components, including the repository (<my-docker-registry:port>) and tag (version) of each image. It also specifies that a secret called regcred should be used for image pulls.

  2. Save and close the values.yaml file.


Manage Network Policies

To enforce existing network policies for NGINX Management Suite, Kubernetes must have a network plugin installed before Helm chart installation.

When enabled, the following network policies will be created in the release namespace.

kubectl get netpol -n nms
NAME           POD-SELECTOR                          AGE
apigw          app.kubernetes.io/name=apigw          4m47s
clickhouse     app.kubernetes.io/name=clickhouse     4m47s
core           app.kubernetes.io/name=core           4m47s
dpm            app.kubernetes.io/name=dpm            4m47s
ingestion      app.kubernetes.io/name=ingestion      4m47s
integrations   app.kubernetes.io/name=integrations   4m47s
utility        app.kubernetes.io/name=integrations   4m47s

To disable the existing network policies, update the values.yaml file as shown below:

 networkPolicies:
	 # Setting this to true enables network policies for NGINX Management Suite.
	 enabled: false

Install Chart

Run the helm install command to install Instance Manager using the Helm chart:

  1. Replace <path-to-your-values.yaml> with the path to the values.yaml file you created.

  2. Replace YourPassword123# with a secure password that contains a combination of uppercase and lowercase letters, numbers, and special characters.

    Important:
    Make sure to copy and save the password for future reference. Only the encrypted password is stored in Kubernetes. There’s no way to recover or reset a lost password.
  • (Optional) Replace <nms-chart-version> with the desired version: the table below lists module compatibility. Omitting this flag will install the latest version.
helm install -n nms --set nms-hybrid.adminPasswordHash=$(openssl passwd -6 'YourPassword123#') nms nginx-stable/nms --create-namespace -f <path-to-your-values.yaml> [--version <chart-version>] --wait

To help you choose the right NGINX Management Suite chart version, refer to the following table, which provides information about the compatible modules for each version.

NGINX Management Suite chart Instance Manager
1.14.0 2.17.0
1.13.0 2.16.0
1.12.1 2.15.1
1.12.0 2.15.0
1.11.0 2.14.0
1.10.1 2.13.1
1.10.0 2.13.1
1.9.0 2.13.0
1.8.2 2.12.0
1.8.1 2.12.0
1.8.0 2.12.0
1.7.2 2.11.0
1.7.1 2.11.0
1.7.0 2.11.0
1.6.0 2.10.1
1.5.2 2.10.0
1.5.1 2.10.0
1.5.0 2.10.0
1.4.0 2.9.1
1.3.1 2.9.0
1.3.0 2.9.0
1.2.1 2.8.0
1.2.0 2.8.0
1.1.2 2.7.0
1.1.1 2.7.0
1.1.0 2.7.0
1.0.0 2.6.0

Validate Deployment

Run the following command to check the status of the deployment:

helm -n nms status nms

This helm command shows the status of the app called nms in the namespace nms. The command displays the release name, chart version, last deployment time, and current status.

The status should be STATUS: deployed if the deployment was successful.


Access Web Interface

You can access the NGINX Management Suite web interface using the external IP address for the API Gateway.

  1. To look up the external IP address for the API Gateway, run the following command:

    kubectl -n nms get svc apigw
    

    This kubectl command retrieves the service named apigw from the namespace nms. It outputs the details of the service, such as its type, port, cluster and external IP addresses.

    The default service type is ClusterIP and the output looks similar to the following example:

    NAME    TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
    apigw   ClusterIP   10.108.57.167   <none>        443/TCP   32s
    

    Using the CLUSTER-IP value, go to https://<CLUSTER-IP>:443/ui.

    For example, https://10.108.57.167/ui.

    This IP address might not be reachable, depending on how the Kubernetes cluster networking was configured. If so, the apigw service type can be changed to a more suitable option, such as LoadBalancer, by changing the Configurable Helm Setting value for nms-hybrid.apigw.service.type .


Add License

A valid license is required to make full use of all the features in Instance Manager.

Refer to the Add a License topic for instructions on how to download and apply a trial license, subscription license, or Flexible Consumption Program license.


Upgrade Instance Manager

To upgrade Instance Manager, take the following steps:

  1. Update the local Helm repository list
  2. Make any necessary adjustments to your values.yaml file
  3. Run the following command to upgrade the NGINX Management Suite deployment:

    • Replace <path-to-your-values.yaml> with the path to the values.yaml file you created.

    • Replace YourPassword123# with a secure password that contains a combination of uppercase and lowercase letters, numbers, and special characters.

      Important:
      Make sure to copy and save the password for future reference. Only the encrypted password is stored in Kubernetes. There’s no way to recover or reset a lost password.
    • (Optional) Replace <nms-chart-version> with the desired version; see the table below for the available versions. Alternatively, you can omit this flag to install the latest version.

    helm upgrade -n nms --set nms-hybrid.adminPasswordHash=$(openssl passwd -6 'YourPassword123#') nms nginx-stable/nms -f <path-to-your-values.yaml> [--version <nms-chart-version>] --wait
    

    This command upgrades an existing Helm chart deployment named nms with a new version of the chart located in the nginx-stable/nms repository. It also sets the value of the nms-hybrid.adminPasswordHash to the hashed version of the provided password and uses a values.yaml file located at the provided path.


Uninstall Instance Manager

Warning:
Uninstalling Instance Manager uninstalls the entire NGINX Management Suite.

To uninstall Instance Manager, run the following command:

helm uninstall --namespace nms nms

This helm command uninstalls the app named nms from the namespace nms.

It deletes all of the Kubernetes resources associated with the app, including any deployments, pods, services, and configmaps.


Configurable Helm Settings

The following table lists the configurable parameters and default values for the NGINX Management Suite platform when installing from a Helm chart.

To modify a configuration for an existing release, run the helm upgrade command and use -f <my-values-file>, where my-values-file is a path to a values file with your desired configuration.

Parameter Description Default
nms-hybrid.adminPasswordHash The hashed value of the password for the admin user.
To generate the hash using openssl, run a command similar to the following example: openssl passwd -1 "YouPassword123#"
N/A
nms-hybrid.nmsClickhouse.enabled Enable this if external ClickHouse is not used. true
nms-hybrid.nmsClickhouse.fullnameOverride Modify the name of ClickHouse resources. clickhouse
nms-hybrid.nmsClickhouse.image.repository Repository name and path public ClickHouse image. clickhouse/clickhouse-server
nms-hybrid.nmsClickhouse.image.tag Tag used for pulling images from registry. 21.3.20.1-alpine
nms-hybrid.nmsClickhouse.image.pullPolicy Image pull policy. IfNotPresent
nms-hybrid.nmsClickhouse.user Username to connect to the ClickHouse server as. N/A
nms-hybrid.nmsClickhouse.password Password for the ClickHouse server. N/A
nms-hybrid.nmsClickhouse.service.name ClickHouse service name. clickhouse
nms-hybrid.nmsClickhouse.service.rpcPort ClickHouse service port. 9000
nms-hybrid.nmsClickhouse.resources.requests.cpu Minimum required CPU on a node to run ClickHouse server. 500m
nms-hybrid.nmsClickhouse.resources.requests.memory Minimum required memory on a node to run ClickHouse server. 1Gi
nms-hybrid.nmsClickhouse.persistence.enabled Use PVCs to persist ClickHouse data. true
nms-hybrid.nmsClickhouse.persistence.existingClaim Name of an existing Persistent Volume Claim (PVC) to use for ClickHouse persistence N/A
nms-hybrid.nmsClickhouse.persistence.storageClass Storage Class to use for creating a ClickHouse PVC
nms-hybrid.nmsClickhouse.persistence.volumeName Name to use a ClickHouse PVC volume
nms-hybrid.nmsClickhouse.persistence.accessMode PVC access mode for ClickHouse ReadWriteOnce
nms-hybrid.nmsClickhouse.persistence.size PVC size ClickHouse 1G
nms-hybrid.nmsClickhouse.tolerations List your Kubernetes tolerations, if any See Kubernetes Taints And Tolerations
nms-hybrid.externalClickhouse.address Address of external ClickHouse service.
nms-hybrid.externalClickhouse.user User of external ClickHouse service.
nms-hybrid.externalClickhouse.password Password of external ClickHouse service.
nms-hybrid.serviceAccount.annotations Set custom annotations for the service account used by NGINX Management Suite. {}
nms-hybrid.apigw.name Name used for API Gateway resources. apigw
nms-hybrid.apigw.tlsSecret By default, this helm chart creates its own Certificate Authority (CA) to self-sign the HTTPS server cert key pairs; these are not managed by NGINX Management Suite. You can bring your own NGINX API Gateway certificates for hosting the HTTPS NGINX Management Suite server by setting “tlsSecret” to an existing Kubernetes secret name in the namespace targeted by the chart. The secret should include tls.crt, tls.key, and ca.pem in the data object. We recommend using a self-provisioned “tlsSecret” for production scenarios.

For an example, refer to the “Use your own certificates” section in Frequently Used Helm Configurations.
nms-hybrid.apigw.image.repository Repository name and path for the apigw image. apigw
nms-hybrid.apigw.image.tag Tag used for pulling images from registry. latest
nms-hybrid.apigw.image.pullPolicy Image pull policy. IfNotPresent
nms-hybrid.apigw.container.port.https Container HTTPS port. 443
nms-hybrid.apigw.service.name Service name. apigw
nms-hybrid.apigw.service.type Service type. Options: ClusterIp, LoadBalancer, NodePort ClusterIp
nms-hybrid.apigw.service.httpsPort Service HTTPS port. 443
nms-hybrid.apigw.resources.requests.cpu Minimum required CPU on a node to run core. 250m
nms-hybrid.apigw.resources.requests.memory Minimum required memory on a node to run core. 256Mi
nms-hybrid.apigw.tolerations List your Kubernetes tolerations, if any See Kubernetes Taints And Tolerations
nms-hybrid.core.name Name used for API Gateway resources. core
nms-hybrid.core.image.repository Repository name and path for the core image. core
nms-hybrid.core.image.tag Tag used for pulling images from registry. latest
nms-hybrid.core.image.pullPolicy Image pull policy. IfNotPresent
nms-hybrid.core.container.port.http Container HTTP port. 8033
nms-hybrid.core.container.port.db Container database port. 7891
nms-hybrid.core.container.port.grpc Container gRPC port. 8038
nms-hybrid.core.service.httpPort Service HTTPS port. 8033
nms-hybrid.core.service.grpcPort Service HTTPS port. 8038
nms-hybrid.core.resources.requests.cpu Minimum required CPU on a node to run core. 500m
nms-hybrid.core.resources.requests.memory Minimum required memory on a node to run core. 512Mi
nms-hybrid.core.persistence.enabled Enable persistence for core service. true
nms-hybrid.core.persistence.claims An array of persistent volume claims for Dqlite and secrets, can be modified to use an existing PVC. See Dqlite and Secrets
nms-hybrid.core.persistence.storageClass Storage Class to use for creating a core PVC
nms-hybrid.core.persistence.volumeName Name to use a core PVC volume
nms-hybrid.core.tolerations List your Kubernetes tolerations, if any See Kubernetes Taints And Tolerations
nms-hybrid.dpm.name Name used for dpm. dpm
nms-hybrid.dpm.image.repository Repository name and path for the dpm image. dpm
nms-hybrid.dpm.image.tag Tag used for pulling images from registry. latest
nms-hybrid.dpm.image.pullPolicy Image pull policy. IfNotPresent
nms-hybrid.dpm.container.port.http Container HTTP port. 8034
nms-hybrid.dpm.container.port.nats Container database port. 9100
nms-hybrid.dpm.container.port.db Container database port. 7890
nms-hybrid.dpm.container.port.grpc Container gRPC port. 8036
nms-hybrid.dpm.service.name Service name. nms
nms-hybrid.dpm.service.httpPort Service HTTPS port. 8034
nms-hybrid.dpm.service.grpcPort Service HTTPS port. 8036
nms-hybrid.dpm.service.natsPort Service HTTPS port. 9100
nms-hybrid.dpm.resources.requests.cpu Minimum required CPU on a node to run dpm. 500m
nms-hybrid.dpm.resources.requests.memory Minimum required memory on a node to run dpm. 512Mi
nms-hybrid.dpm.persistence.enabled Enable persistence for dpm service. true
nms-hybrid.dpm.persistence.claims An array of persistent volume claims for Dqlite and NATS, can be modified to use an existing PVC. See Dqlite and NATS streaming
nms-hybrid.dpm.persistence.storageClass Storage Class to use for creating a dpm PVC
nms-hybrid.dpm.persistence.volumeName Name to use a dpm PVC volume
nms-hybrid.dpm.tolerations List your Kubernetes tolerations, if any See Kubernetes Taints And Tolerations
nms-hybrid.ingestion.name Name used for ingestion. ingestion
nms-hybrid.ingestion.image.repository Repository name and path for the dpm image. ingestion
nms-hybrid.ingestion.image.tag Tag used for pulling images from registry. latest
nms-hybrid.ingestion.image.pullPolicy Image pull policy. IfNotPresent
nms-hybrid.ingestion.replicaCount Number of replicas of ingestion to run. 1
nms-hybrid.ingestion.container.port.grpc Container HTTP port. 8035
nms-hybrid.ingestion.service.name Service name. nms
nms-hybrid.ingestion.service.grpcPort Service HTTPS port. 8035
nms-hybrid.ingestion.resources.requests.cpu Minimum required CPU on a node to run ingestion. 500m
nms-hybrid.ingestion.resources.requests.memory Minimum required memory on a node to run ingestion. 512Mi
nms-hybrid.ingestion.tolerations List your Kubernetes tolerations, if any See Kubernetes Taints And Tolerations
nms-hybrid.integrations.name Name used for integrations. integrations
nms-hybrid.integrations.image.repository Repository name and path for the integrations image. integrations
nms-hybrid.integrations.image.tag Tag used for pulling images from registry. latest
nms-hybrid.integrations.image.pullPolicy Image pull policy. IfNotPresent
nms-hybrid.integrations.container.port.http Container HTTP port. 8037
nms-hybrid.integrations.container.port.db Container database port. 7892
nms-hybrid.integrations.service.name Service name. nms
nms-hybrid.integrations.service.httpPort Service HTTPS port. 8037
nms-hybrid.integrations.resources.requests.cpu Minimum required CPU on a node to run integrations. 500m
nms-hybrid.integrations.resources.requests.memory Minimum required memory on a node to run integrations. 512Mi
nms-hybrid.integrations.persistence.enabled Enable persistence for integrations service. true
nms-hybrid.integrations.persistence.claims An array of persistent volume claims for Dqlite, can be modified to use an existing PVC. See Dqlite - size is 1Gi
nms-hybrid.integrations.persistence.storageClass Storage Class to use for creating a integrations PVC
nms-hybrid.integrations.persistence.volumeName Name to use a integrations PVC volume
nms-hybrid.integrations.tolerations List your Kubernetes tolerations, if any See Kubernetes Taints And Tolerations
nms-hybrid.utility.name Name used for utility. utility
nms-hybrid.utility.image.repository Repository name and path for the utility image. utility
nms-hybrid.utility.image.tag Tag used for pulling images from registry. latest
nms-hybrid.utility.image.pullPolicy Image pull policy. IfNotPresent
Instance Manager Dqlite Storage Configuration
  - name: dqlite
    existingClaim:
    size: 500Mi
    accessMode: ReadWriteOnce
Instance Manager Secrets Storage Configuration
  - name: secrets
    existingClaim:
    size: 128Mi
    accessMode: ReadWriteOnce
Instance Manager NATS Storage Configuration
  - name: nats-streaming
    existingClaim:
    size: 1Gi
    accessMode: ReadWriteOnce
Kubernetes Taints and Tolerations

The following example snippet shows a toleration on an Instance Manager APIGW deployment. In this example, Kubernetes will tolerate the “NoExecute” effect for 60 seconds before evicting the pod from the tainted node.

tolerations:
  - key: "node.kubernetes.io/unreachable"
    operator: "Exists"
    effect: "NoExecute"
    tolerationSeconds: 60
  - key: "node.kubernetes.io/network-unavailable"
    operator: "Exists"
    effect: "NoExecute"
    tolerationSeconds: 60

For more information, refer to the official Kubernetes Taints and Tolerations documentation.


Troubleshooting

For help with common issues and suggested solutions and workarounds, refer to the NGINX Management Suite Troubleshooting Guide.

For guidance on how to create a support package containing system and service details to share with NGINX Customer Support, refer to the guide Create a Support Package from a Helm Installation.