Rancher Kubernetes Engine
Learn how to set up Rancher Kubernetes Engine (RKE) for use with NGINX Service Mesh.
Rancher Kubernetes Engine (RKE) is a CNCF-certified Kubernetes distribution that runs entirely within Docker containers. It works on bare-metal and virtualized servers.
Before deploying NGINX Service Mesh, ensure that no other service meshes exist in your Kubernetes cluster.
RKE doesn’t set up any persistent storage for you, but it’s required to run NGINX Service Mesh in a production environment. See Persistent Storage for more information.
When creating a new cluster with RKE, you can configure it to apply a PodSecurityPolicy. If you choose to do this, NGINX Service Mesh requires a few permissions in order to function properly. The following policy is based on the default
restricted-psp policy used by RKE, with a few additions and changes to allow the NGINX Service Mesh control plane to work.
When running a cluster with a PodSecurityPolicy, all of the following resources need to be created/updated before deploying NGINX Service Mesh.
apiVersion: policy/v1beta1 kind: PodSecurityPolicy metadata: annotations: serviceaccount.cluster.cattle.io/pod-security: restricted serviceaccount.cluster.cattle.io/pod-security-version: "1696" labels: cattle.io/creator: norman name: restricted-psp-nginx-mesh spec: allowPrivilegeEscalation: false fsGroup: ranges: - max: 65535 min: 1 rule: MustRunAs requiredDropCapabilities: - ALL runAsUser: rule: RunAsAny seLinux: rule: RunAsAny supplementalGroups: ranges: - max: 65535 min: 1 rule: MustRunAs hostNetwork: true hostPID: true volumes: - configMap - emptyDir - projected - secret - downwardAPI - persistentVolumeClaim - hostPath
In order for the NGINX Service Mesh sidecar init container to be able to configure
iptables rules and BPF programs needed for UDP communication, it needs
If a PodSecurityPolicy is applied to your workloads, then the following additions need to be made in order for the sidecar to work properly:
spec: allowedCapabilities: - NET_ADMIN - NET_RAW - SYS_RESOURCE - SYS_ADMIN
If you have separate PodSecurityPolicies for the control plane and your workloads, ensure that they are bound to the proper Service Accounts.
restricted-psp-nginx-mesh policy needs to be bound to the NGINX Service Mesh control plane namespace, using the following resources:
The ClusterRoleBinding assumes the default namespace of
nginx-mesh, but should be changed if you are using a different namespace for the control plane.
apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: nginx-mesh-psp-role rules: - apiGroups: ['policy'] resources: ['podsecuritypolicies'] verbs: ['use'] resourceNames: - restricted-psp-nginx-mesh
apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: nginx-mesh-psp-binding roleRef: kind: ClusterRole name: nginx-mesh-psp-role apiGroup: rbac.authorization.k8s.io subjects: - kind: Group apiGroup: rbac.authorization.k8s.io name: system:serviceaccounts:nginx-mesh