End of Sale Notice:

Commercial support for NGINX Service Mesh is available to customers who currently have active NGINX Microservices Bundle subscriptions. F5 NGINX announced the End of Sale (EoS) for the NGINX Microservices Bundles as of July 1, 2023.

See our End of Sale announcement for more details.

Rancher Kubernetes Engine

Learn how to set up Rancher Kubernetes Engine (RKE) for use with NGINX Service Mesh.

Rancher Kubernetes Engine (RKE) is a CNCF-certified Kubernetes distribution that runs entirely within Docker containers. It works on bare-metal and virtualized servers.

Important:
Before deploying NGINX Service Mesh, ensure that no other service meshes exist in your Kubernetes cluster.

Warning:
Rancher has the option to deploy the community NGINX Ingress Controller when configuring an RKE cluster. While this ingress controller may work, NGINX Service Mesh does not guarantee support. It is recommended to use the NGINX Plus Ingress Controller in conjunction with NGINX Service Mesh.

Persistent storage

RKE doesn’t set up any persistent storage for you, but it’s required to run NGINX Service Mesh in a production environment. See Persistent Storage for more information.

Pod Security Policies

When creating a new cluster with RKE, you can configure it to apply a PodSecurityPolicy. If you choose to do this, NGINX Service Mesh requires a few permissions in order to function properly. The following policy is based on the default restricted-psp policy used by RKE, with a few additions and changes to allow the NGINX Service Mesh control plane to work.

Important:
When running a cluster with a PodSecurityPolicy, all of the following resources need to be created/updated before deploying NGINX Service Mesh.

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  annotations:
    serviceaccount.cluster.cattle.io/pod-security: restricted
    serviceaccount.cluster.cattle.io/pod-security-version: "1696"
  labels:
    cattle.io/creator: norman
  name: restricted-psp-nginx-mesh
spec:
  allowPrivilegeEscalation: false
  fsGroup:
    ranges:
    - max: 65535
      min: 1
    rule: MustRunAs
  requiredDropCapabilities:
  - ALL
  runAsUser:
    rule: RunAsAny
  seLinux:
    rule: RunAsAny
  supplementalGroups:
    ranges:
    - max: 65535
      min: 1
    rule: MustRunAs
  hostNetwork: true
  hostPID: true
  volumes:
  - configMap
  - emptyDir
  - projected
  - secret
  - downwardAPI
  - persistentVolumeClaim
  - hostPath

In order for the NGINX Service Mesh sidecar init container to be able to configure iptables rules and BPF programs needed for UDP communication, it needs NET_ADMIN, NET_RAW, SYS_RESOURCE, and SYS_ADMIN capabilities.

If a PodSecurityPolicy is applied to your workloads, then the following additions need to be made in order for the sidecar to work properly:

spec:
  allowedCapabilities:
  - NET_ADMIN
  - NET_RAW
  - SYS_RESOURCE
  - SYS_ADMIN

Important:
If you have separate PodSecurityPolicies for the control plane and your workloads, ensure that they are bound to the proper Service Accounts.

Bind the Policy

The restricted-psp-nginx-mesh policy needs to be bound to the NGINX Service Mesh control plane namespace, using the following resources:

Note:
The ClusterRoleBinding assumes the default namespace of nginx-mesh, but should be changed if you are using a different namespace for the control plane.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: nginx-mesh-psp-role
rules:
- apiGroups: ['policy']
  resources: ['podsecuritypolicies']
  verbs:     ['use']
  resourceNames:
  - restricted-psp-nginx-mesh

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: nginx-mesh-psp-binding
roleRef:
  kind: ClusterRole
  name: nginx-mesh-psp-role
  apiGroup: rbac.authorization.k8s.io
subjects:
- kind: Group
  apiGroup: rbac.authorization.k8s.io
  name: system:serviceaccounts:nginx-mesh