Sidecar Proxy Injection
Learn about the configuration options for NGINX Service Mesh sidecar proxy injection.
NGINX Service Mesh works by injecting a sidecar proxy into Kubernetes resources. You can choose to inject the sidecar proxy into the YAML or JSON definitions for your Kubernetes resources in the following ways:
When you inject the sidecar proxy into a Kubernetes resource, the injected config uses the global mTLS setting. You can define the global setting when you deploy NGINX Service Mesh, or use the default setting.
Refer to Secure Mesh Traffic using mTLS for more information.
The sidecar proxy will not be injected into Pods that define multiple container ports with the same port number or for container ports with the SCTP protocol.
UDP and TCP is an exception to this, and may be specified on the same port.
The mesh supports the following Kubernetes resources and API versions for injection:
|Resource Type||API Version|
NGINX Service Mesh uses automatic injection by default. This means that any time a user creates a Kubernetes Pod resource, the NGINX Service Mesh automatically injects the sidecar proxy into the Pod. Automatic injection applies to all namespaces in your Kubernetes cluster.
By default, NGINX Service Mesh can access resources in all Kubernetes namespaces.
To disable this setting, deploy the mesh using the
nginx-meshctl deploy ... --disable-auto-inject
To enable injection for a specific namespace, add the
injector.nsm.nginx.com/auto-inject=enabled label. This will only work if the mesh is deployed with global automatic sidecar injection disabled.
If you add this label to a namespace where Pods already exist you will need to restart those Pods for the sidecar to be injected. By the same token if you remove this label from a namespace where Pods exist and have the sidecar injected, you will need to restart them to remove the sidecar.
You can also enable injection by adding the
--enabled-namespaces flag to your deploy command.
For example, to disable automatic injection in all namespaces and enable it only in the namespaces “prod” and “staging”, you would run the following command:
nginx-meshctl deploy ... --disable-auto-inject --enabled-namespaces="prod,staging"
In an upcoming version, NGINX Service Mesh will be removing the
Similarly, you can deploy NGINX Service Mesh with automatic injection enabled and specify a list of the namespaces that you want to exclude.
To do so, use the
--disabled-namespaces flag when you deploy.
The following deploy command enables automatic injection in all namespaces except “test”:
nginx-meshctl deploy ... --disabled-namespaces="test"
If you need to modify the auto injection settings after you’ve deployed NGINX Service Mesh, you can do so by using the REST API.
For more granular control, you can override the global automatic injection setting on a per-resource basis. To do so, add the following label to the resource’s PodTemplateSpec:
To inject the sidecar into existing resources you must re-roll those resources after installing NGINX Service Mesh.
kubectl rollout restart <resource type>/<resource name>
kubectl rollout restart deployment/frontend
Refer to NGINX Service Mesh Labels and Annotations for more information.
Refer to the Kubernetes
kubectlCheat Sheet documentation for more information about rolling resources.
Before running the
inject command, make sure your Kubernetes user has the permission to
create the resource
inject in APIGroup
nsm.nginx.com. Below is an example of a
ClusterRole with this permission:
apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: nsm-inject rules: - apiGroups: - nsm.nginx.com resources: - inject verbs: - create
If your Kubernetes user account has the
ClusterAdminrole, then no additional permissions are necessary to run the inject command.
To inject the sidecar proxy into a resource manually, use the
nginx-meshctl inject command. Provide the path to the resource definition file and your desired output filename.
nginx-meshctl inject < <resource-file> > <new-resource-file>
For example, the following command will write the updated config for “resource.yaml” to a new file, “resource-injected.yaml”:
nginx-meshctl inject < resource.yaml > resource-injected.yaml
Depending on the network connection and the size of the file you’re injecting, timeouts may occur while running the inject command. If this happens, you can use the
--timeout flag to increase the timeout. The default timeout is 5 seconds.
nginx-meshctl --timeout 10s inject < resource.yaml > resource-injected.yaml
You can set the proxy to ignore ports for either incoming or outgoing traffic. The NGINX Service Mesh applies the configurations at injection time.
For automatic injection, add the following annotations to the PodTemplateSpec in your resource definition:
config.nsm.nginx.com/ignore-incoming-ports: "port1, port2, ..., portN" config.nsm.nginx.com/ignore-outgoing-ports: "port1, port2, ..., portN"
For manual injection, you can use the annotations above or specify the ports when running the
nginx-meshctl inject --ignore-incoming-ports "port1,port2,...,portN", --ignore-outgoing-ports "port1,port2,...,portN" < resource.yaml > resource-injected.yaml
Refer to NGINX Service Mesh Annotations for more information around annotations.
By default, the following ports are ignored by the proxy:
- 53 (DNS)
If mTLS mode is set to
strict, then readiness, liveness, and startup probes using HTTP GET do not work. This is
kubelet not having the correct client certificates. To remedy this, application HTTP/S health probes are
rewritten at injection time. The new probes point to an endpoint on the sidecar proxy, which
redirects the health check to the original destination on the application. This allows the health check to bypass
the SSL verification, while still sending the health check to the intended destination.
NGINX Service Mesh automatically detects and adjusts the
eth0 interface to support the 32 bytes of space required for PROXY Protocol V2. See the UDP and eBPF architecture section for more information.