Install NGINX Service Mesh using nginx-meshctl

This topic explains how to download, install, and deploy NGINX Service Mesh.

Overview

This topic contains instructions for downloading and installing NGINX Service Mesh using the nginx-meshctl command line tool.

For Helm users, see how to Install NGINX Service Mesh using Helm.

Prerequisites

Important:
Before installing NGINX Service Mesh, make sure you’ve completed the following steps.

• You have a working Kubernetes cluster, version 1.18 or newer.
• You followed the Kubernetes or OpenShift Platform Setup guide to prepare your cluster to work with NGINX Service Mesh.
• You have the Kubernetes kubectl command-line utility configured on the machine where you want to install NGINX Service Mesh.
• You reviewed the Configuration Options for NGINX Service Mesh.

Download NGINX Service Mesh

Note:
NGINX Microservice Bundle customers can download the nginx-meshctl tool from MyF5.

In order to download NGINX Service Mesh, you’ll need to register for an account at the F5 Downloads site. Once you have registered, click on the Find a Download button to see a list of the available products and select the NGINX_Service_Mesh product line. From the NGINX_Service_Mesh product page, you can select the version you would like to install from the dropdown menu and click on the product name to view the files available for download.

To install and deploy NGINX Service Mesh you need to download the nginx-meshctl binary for your architecture.

In addition to the binary, you also need access to all of the NGINX Service Mesh docker images. There are multiple ways to access these images:

• Pull the Images from the Docker Registry
• Pull the Images from MyF5
• Download and Push Images to Container Registry.

Install the CLI

The NGINX Service Mesh command-line tool – nginx-meshctl – allows you to deploy, remove, and interact with the NGINX Service Mesh control plane. The following sections describe how to install the CLI on Linux, macOS, and Windows.

Install on Linux

1. Download the appropriate binary for your architecture, nginx-meshctl_linux.gz.

2. Unzip the binary.

   gunzip nginx-meshctl_linux.gz
   

3. Move the nginx-meshctl executable in to your PATH.

   sudo mv nginx-meshctl_linux /usr/local/bin/nginx-meshctl
   

4. Ensure the nginx-meshctl is executable.

   sudo chmod +x /usr/local/bin/nginx-meshctl
   

5. Test the installation.

   nginx-meshctl version
   

Install on macOS

1. Download the appropriate binary for your architecture, nginx-meshctl_darwin.gz.

2. Unzip the binary.

   gunzip nginx-meshctl_darwin.gz
   

3. Move the nginx-meshctl executable in to your PATH.

   sudo mv nginx-meshctl_darwin /usr/local/bin/nginx-meshctl
   

4. Ensure the nginx-meshctl is executable.

   sudo chmod +x /usr/local/bin/nginx-meshctl
   

5. Test the installation.

   nginx-meshctl version
   

Install on Windows

1. Download the appropriate binary for your architecture, nginx-meshctl_windows.exe

2. Add the binary to your PATH and rename.

3. Test the installation.

   nginx-meshctl.exe version
   

Images

NGINX Service Mesh distributes the following images:

• nginx-mesh-api: NGINX Service Mesh API Server.
• nginx-mesh-metrics: Gets Pod (and other Kubernetes resources) metrics. Refer to SMI Metrics on GitHub and nginx-meshctl help top for more information.
• nginx-mesh-sidecar: NGINX Service Mesh sidecar.
• nginx-mesh-init: NGINX Service Mesh sidecar init container. Sets up iptables for the sidecar.
• nginx-mesh-cert-reloader: NGINX Service Mesh certificate reloader. Loads and rotates certificates for the NATs server.
• nginx-mesh-csi-driver (OpenShift only): NGINX Service Mesh CSI Driver. Used for securely distributing Spire certificates in an OpenShift environment.

These images can be pulled directly from the NGINX Service Mesh docker registry docker-registry.nginx.com/nsm or downloaded.

In addition, NGINX Service Mesh pulls the following publicly-accessible third-party container images into your Kubernetes cluster in order to function:

Important:
Prometheus, Grafana, Zipkin, and Jaeger will no longer be deployed in NGINX Service Mesh 1.5. You will need to deploy your own servers instead. See the Monitoring and Tracing guide for more information on connecting your own servers.

OpenShift only

Note:
If you are running in an air-gapped environment, and do not have public network access, see the Air Gap Environment section.

Note:
Prometheus, Grafana, and Zipkin/Jaeger are unnecessary if you are using your own deployments of these components. If you are, see the Monitoring and Tracing guide.

Pull Images from Docker Registry

By default, the following images are automatically pulled from the NGINX Service Mesh docker registry when deploying NGINX Service Mesh. The image tag X.Y.Z is the version number, for example 1.0.0.

• docker-registry.nginx.com/nsm/nginx-mesh-api:X.Y.Z
• docker-registry.nginx.com/nsm/nginx-mesh-metrics:X.Y.Z
• docker-registry.nginx.com/nsm/nginx-mesh-sidecar:X.Y.Z
• docker-registry.nginx.com/nsm/nginx-mesh-init:X.Y.Z
• docker-registry.nginx.com/nsm/nginx-mesh-cert-reloader:X.Y.Z
• docker-registry.nginx.com/nsm/nginx-mesh-csi-driver:X.Y.Z

Download and Push Images to Container Registry

In addition to the CLI binary, you can download the NGINX Service Mesh images and push them to a container registry that your cluster can access.

Important:
It is highly recommended that you match the version number when downloading the nginx-meshctl binary and nginx-mesh-images package. We make no compatibility guarantees across versions. For information on how to upgrade your existing mesh, see Upgrade NGINX Service Mesh.

Follow these steps to download, load, tag, and push the images:

1. Download the nginx-mesh-images.X.Y.Z.tar.gz file. Where X.Y.Z is the appropriate version and matches the binary downloaded in the previous section; for example, 1.0.0.

Each image file is a Docker-formatted tar archive. You can use the docker load command to make them accessible by your local Docker daemon. For instructions on how to download these files see the Download NGINX Service Mesh section.

1. Extract the contents of the tar archive and cd into the release directory.

   tar zxvf nginx-mesh-images.X.Y.Z.tar.gz
   cd nginx-mesh-images-X.Y.Z
   

2. Run the docker load command for each of the image files listed below.

   • nginx-mesh-api.X.Y.Z.tar.gz
   • nginx-mesh-metrics.X.Y.Z.tar.gz
   • nginx-mesh-init.X.Y.Z.tar.gz
   • nginx-mesh-sidecar.X.Y.Z.tar.gz
   • nginx-mesh-cert-reloader.X.Y.Z.tar.gz
   • nginx-mesh-csi-driver.X.Y.Z.tar.gz

   for image in $(ls)
   do
     docker load < $image
   done
   

3. Tag each image appropriately for your environment and registry location.

   • nginx-mesh-api
   • nginx-mesh-metrics
   • nginx-mesh-init
   • nginx-mesh-sidecar
   • nginx-mesh-cert-reloader
   • nginx-mesh-csi-driver

   docker tag <image-name>:X.Y.Z <your-docker-registry>/<image-name>:X.Y.Z
   

4. Push each image.

   docker push <your-docker-registry>/<image-name>:X.Y.Z
   

5. When deploying NGINX Service Mesh using nginx-meshctl, set the --registry-server flag to your registry. If using Helm, set the registry.server field to your registry.

Air Gap Environment

If your environment does not have access to public image repositories, then you will need to manually pull the images listed in the table above, and push them to your private registry. The image names and tags must remain the same. For example:

gcr.io/spiffe-io/spire-agent:1.2.0 would become your-registry/spire-agent:1.2.0

nats:2.7.2-alpine3.15 would become your-registry/nats:2.7.2-alpine3.15

When running nginx-meshctl deploy, use the --disable-public-images flag to instruct the mesh to use your --registry-server for all images. For example:

nginx-meshctl deploy --registry-server your-registry --disable-public-images ...

Install the NGINX Service Mesh Control Plane

See Also:
Check out Getting Started with NGINX Service Mesh to learn about the deployment options before proceeding.
You can find the full list of options in the nginx-meshctl Reference.

Important:
nginx-meshctl creates the namespace for the NGINX Service Mesh control plane.
This namespace is dedicated to the NGINX Service Mesh control plane and should not be used for anything else.
If desired, you can specify any name for the namespace via the --namespace argument, but do not create this namespace yourself.

Take the steps below to install the NGINX Service Mesh control plane.

1. Run the nginx-meshctl deploy command using the desired options.

   For example, the following command will deploy NGINX Service Mesh using all of the default settings for the latest release:

   nginx-meshctl deploy
   

   If you are using a private registry to store the NGINX Service Mesh images see the Private Registry guide for instructions.

   For example, nginx-meshctl deploy --registry-server "registry:5000/images" --image-tag X.Y.Z will look for containers registry:5000/images/nginx-mesh-api:X.Y.Z, registry:5000/images/nginx-mesh-sidecar:X.Y.Z, etc.

2. Run the kubectl get pods command to verify that the Pods are up and running.

   Be sure to specify the nginx-mesh namespace when running the command.

   $ kubectl -n nginx-mesh get pods
   NAME                                  READY   STATUS    RESTARTS   AGE
   grafana-855bf57c67-hz4dt              1/1     Running   0          14m
   nats-server-84f8b6f669-xszkc          1/1     Running   0          14m
   nginx-mesh-api-954467945-sc7qh        1/1     Running   0          14m
   nginx-mesh-metrics-57464df46d-qskd2   1/1     Running   0          14m
   prometheus-7bb967bb58-42c8l           1/1     Running   0          14m
   spire-agent-92ktv                     1/1     Running   0          15m
   spire-agent-9dbn6                     1/1     Running   0          15m
   spire-agent-z5cq6                     1/1     Running   0          15m
   spire-server-0                        2/2     Running   0          15m
   zipkin-6fb7df55c4-rsq9g               1/1     Running   0          14m
   

   Note:
   If running in OpenShift, you will see two pods per Spire Agent container.

UDP MTU Sizing

Note:
UDP traffic proxying is currently disabled by default. It can be enabled at deploy time using the --enable-udp flag. Linux kernel 4.18 or greater is required.

NGINX Service Mesh automatically detects and adjusts the eth0 interface to support the 32 bytes of space required for PROXY Protocol V2. See the UDP and eBPF architecture section for more information.

NGINX Service mesh does not detect changes made to the MTU in the pod at runtime. If adding a CNI changes the MTU of the eth0 interface of running pods, please re-roll the affected pods to ensure those changes take place.