Release Notes 1.4.0

Updates

NGINX Service Mesh 1.4.0 includes the following updates:

• NGINX Service Mesh has changed its API to follow Kubernetes convention enabling granular controls of the NGINX Service Mesh API using Kubernetes native RBAC.
  • Use the NGINX Service Mesh API
• Support for service-to-service UDP traffic proxying
• The addition of OpenTelemetry tracing along side the existing OpenTracing support to provide rich telemetry options
• Coupling with the cert-manager project to instantly drop into your existing Certificate Authority issuer workflow

Deprecation

• Starting in v1.5.0 release, NGINX Service Mesh will no longer bundle Prometheus, Grafana, or tracing pods.
  • The –disable-tracing and –deploy-grafana flags are deprecated and will be removed in v1.5.0
  • The tracing.disable and deployGrafana helm values are deprecated and will be removed in v1.5.0
  • The config.nsm.nginx.com/tracing-enabled pod annotation is deprecated and will be removed in v1.5.0

Features

• Support for UDP traffic proxying
• Support for OpenTelemetry tracing using the OTLP gRPC Exporter
• Support for cert-manager as an upstream authority
• How to access the NGINX Service Mesh API using Kubernetes native RBAC

Improvements

• Updated to Spire 1.2.0
• Tightened API access

Fixes

• Openshift CSI Driver is automatically cleaned up after mesh removal once all workloads have been re-rolled or deleted

Vulnerabilities

Fixes

This release includes vulnerability fixes for the following issues.

• None

Third Party Updates

This release includes third party updates for the following issues.

• github.com/apache/thrift CVE-2020-13949 HIGH (22)

• busybox CVE-2021-42374, CVE-2021-42375, CVE-2021-42378, CVE-2021-42379, CVE-2021-42380, CVE-2021-42381, CVE-2021-42382, CVE-2021-42383, CVE-2021-42384 HIGH CVE-2021-42385, CVE-2021-42386 MEDIUM (677)

• github.com/containerd/containerd CVE-2021-41103 HIGH (678)

• golang.org/x/crypto/ssh CVE-2020-29652 HIGH (734)

• golang.org/x/text CVE-2021-38561 HIGH (735)

• github.com/containerd/containerd CVE-2021-43816 CRITICAL (776)

Resolved Issues

This release includes fixes for the following issues.

• NGINX Service Mesh cannot be deployed or removed (428)

• Invalid rate limit configurations are allowed (449)

• Optional, default visualization dependencies may cause excessive disk usage (498)

Known Issues

The following issues are known to be present in this release. Look for updates to these issues in future NGINX Service Mesh release notes.

Spire Server crashes after reaching ~1500 certificate rotations (375):

After reaching approximately 1500 certificate rorations, the Spire Server crashes. This condition can be reached by either setting your certificate authority TTL to a low number, for example, {{–mtls-ca-ttl 1m –mtls-svid-ttl 1m}} – or leaving the mesh running for extended periods of time.

Workaround:

Use the default TTL, which would produce the conditions that cause the crash after about 100 years of continuous operation. If you must use a lower TTL that will result in a significant number of certificate rotations, redeploy NGINX Service Mesh to refresh its state before the crash conditions can be reached.

Pods can’t be created if nginx-mesh-api is unreachable (384):

If the nginx-mesh-api Pod cannot be reached by the “sidecar-injector-webhook-cfg.internal.builtin.nsm.nginx” MutatingWebhookConfiguration, then all Pod creations will fail.

Workaround:

If attempting to create Pods that are not going to be injected by NGINX Service Mesh, then the simplest solution is to remove NGINX Service Mesh.

Otherwise, if the nginx-mesh-api Pod is crashing, then the user should verify that their configuration when deploying NGINX Service Mesh is valid. Reinstalling the mesh may also fix connectivity issues.

Deploying a TrafficSplit with an invalid weight value fails but does not return any errors (426):

When deploying a TrafficSplit, it is possible to set the weight value as a negative integer. NGINX Service Mesh does not support negative-integer weights, so the TrafficSplit will appear to deploy successfully but will not take effect. No error or log message is returned to indicate that the TrafficSplit deployment has failed.

Workaround:

Be sure to use positive integers when assigning weight values to TrafficSplits.

Tracer address reported by nginx-meshctl config when no tracer is deployed (440):

If NGINX Service Mesh is deployed without a tracing backend, nginx-meshctl config reports the default tracing backend (jaeger) and the default tracing backend address (“jaeger..svc.cluster.local:6831”). This has no impact on the functionality of the mesh as tracing is disabled.

Workaround:

No workaround necessary.

NGINX Service Mesh DNS Suffix support (519):

NGINX Service Mesh only supports the cluster.local DNS suffix. Services such as Grafana and Prometheus will not work in clusters with a custom DNS suffix.

Workaround:

Ensure your cluster is setup with the default cluster.local DNS suffix.

Duplicate targetPorts in a Service are disregarded (532):

NGINX Service Mesh supports a variety of Service .spec.ports[] configurations and honors each port list item with one exception.

If the Service lists multiple port configurations that duplicate .spec.ports[].targetPort, the duplicates are disregarded. Only one port configuration is honored for traffic forwarding, authentication, and encryption.

Example invalid configuration:

apiVersion: v1
kind: Service
spec:
  ports:
  - port: 8080
    protocol: TCP
    targetPort: 55555
  - port: 9090
    protocol: TCP
    targetPort: 55555

No workaround exists outside of reconfiguring the Service and application. The Service must use unique .spec.ports[].targetPort values (open up multiple ports on the application workload) or route all traffic to the application workload through the same Service port.

Pods fail to deploy if invalid Jaeger tracing address is set (540):

If --tracing-address is set to an invalid Jaeger address when deploying NGINX Service Mesh, all pods will fail to start.

Workaround:

If you use your own Zipkin or Jaeger instance with NGINX Service Mesh, make sure to correctly set --tracing-address when deploying the mesh.

Inject command errors are ambiguous (789):

Inject command may return ambiguous error messages:

{noformat}./build/nginx-meshctl inject < example.yaml Cannot inject NGINX Service Mesh sidecar. Error: error formatting response string: invalid syntax{noformat}

This error can be returned due to invalid encoding, if the control plane is down, if NGINX Service Mesh is not installed, or if it is installed in an unexpected namespace.

Workaround:

When encountering an error from Inject do the following checks:

1. Verify you have valid YAML.
2. Check the status of your install: {{nginx-meshctl status}}
3. Check the status of you Pods: {{kubectl -n nginx-mesh get pods}}

If steps 2 or 3 show failures, or Pods that are not in the Running state, NGINX Service Mesh will need further troubleshooting. Pods may be restarted through {{kubectl}} by using {{rollout restart}} on the Deployments, or by deleting the Pod resources. If the issue persists contact your support agent.

Supported Versions

NGINX Service Mesh supports the following versions:

Kubernetes:

• 1.18-1.22

OpenShift

• 4.8

Helm:

• 3.2+

Rancher:

• 2.5, 2.6

NGINX Plus Ingress Controller:

• 1.12, 2.0, 2.1

Note: The minimum supported version of Kubernetes is 1.19 if you are running NGINX Plus Ingress Controller 2.0. For older Kubernetes versions, use the NGINX Plus Ingress Controller 1.12 version.

SMI Specification:

• Traffic Access: v1alpha2
• Traffic Metrics: v1alpha1 (in progress, supported resources: StatefulSets, Namespaces, Deployments, Pods, DaemonSets)
• Traffic Specs: v1alpha3
• Traffic Split: v1alpha3

NSM SMI Extensions:

• Traffic Specs:

  • RateLimit: v1alpha1,v1alpha2
  • CircuitBreaker: v1alpha1