TransportServer Resource
The TransportServer resource allows you to configure TCP, UDP, and TLS Passthrough load balancing. The resource is implemented as a Custom Resource.
This document is the reference documentation for the TransportServer resource. To see additional examples of using the resource for specific use cases, go to the examples-of-custom-resources folder in our GitHub repo.
Feature Status: The TransportServer resource is available as a preview feature: it is suitable for experimenting and testing; however, it must be used with caution in production environments. Additionally, while the feature is in preview, we might introduce some backward-incompatible changes to the resource specification in the next releases.
Contents
Prerequisites
- For TCP and UDP, the TransportServer resource must be used in conjunction with the GlobalConfiguration resource, which must be created separately.
- For TLS Passthrough, make sure to enable the
-enable-tls-passthrough
command-line argument of the Ingress Controller.
TransportServer Specification
The TransportServer resource defines load balancing configuration for TCP, UDP, or TLS Passthrough traffic. Below are a few examples:
TCP load balancing:
apiVersion: k8s.nginx.org/v1alpha1 kind: TransportServer metadata: name: dns-tcp spec: listener: name: dns-tcp protocol: TCP upstreams: - name: dns-app service: dns-service port: 5353 action: pass: dns-app
UDP load balancing:
apiVersion: k8s.nginx.org/v1alpha1 kind: TransportServer metadata: name: dns-udp spec: listener: name: dns-udp protocol: UDP upstreams: - name: dns-app service: dns-service port: 5353 upstreamParameters: udpRequests: 1 udpResponses: 1 action: pass: dns-app
TLS passthrough load balancing:
apiVersion: k8s.nginx.org/v1alpha1 kind: TransportServer metadata: name: secure-app spec: listener: name: tls-passthrough protocol: TLS_PASSTHROUGH host: app.example.com upstreams: - name: secure-app service: secure-app port: 8443 action: pass: secure-app
Field | Description | Type | Required |
---|---|---|---|
listener |
The listener on NGINX that will accept incoming connections/datagrams. | listener | Yes |
host |
The host (domain name) of the server. Must be a valid subdomain as defined in RFC 1123, such as my-app or hello.example.com . Wildcard domains like *.example.com are not allowed. Required for TLS Passthrough load balancing. |
string |
No* |
upstreams |
A list of upstreams. | []upstream | Yes |
upstreamParameters |
The upstream parameters. | upstreamParameters | No |
action |
The action to perform for a client connection/datagram. | action | Yes |
ingressClassName |
Specifies which Ingress Controller must handle the TransportServer resource. | string |
No |
serverSnippets |
Sets a custom snippet in the server context. |
string |
No |
* – Required for TLS Passthrough load balancing.
Listener
The listener field references a listener that NGINX will use to accept incoming traffic for the TransportServer. For TCP and UDP, the listener must be defined in the GlobalConfiguration resource. When referencing a listener, both the name and the protocol must match. For TLS Passthrough, use the built-in listener with the name tls-passthrough
and the protocol TLS_PASSTHROUGH
.
An example:
listener:
name: dns-udp
protocol: UDP
Field | Description | Type | Required |
---|---|---|---|
name |
The name of the listener. | string |
Yes |
protocol |
The protocol of the listener. | string |
Yes |
Upstream
The upstream defines a destination for the TransportServer. For example:
name: secure-app
service: secure-app
port: 8443
maxFails: 3
failTimeout: 30s
Field | Description | Type | Required |
---|---|---|---|
name |
The name of the upstream. Must be a valid DNS label as defined in RFC 1035. For example, hello and upstream-123 are valid. The name must be unique among all upstreams of the resource. |
string |
Yes |
service |
The name of a service. The service must belong to the same namespace as the resource. If the service doesn’t exist, NGINX will assume the service has zero endpoints and close client connections/ignore datagrams. | string |
Yes |
port |
The port of the service. If the service doesn’t define that port, NGINX will assume the service has zero endpoints and close client connections/ignore datagrams. The port must fall into the range 1..65535 . |
int |
Yes |
maxFails |
Sets the number of unsuccessful attempts to communicate with the server that should happen in the duration set by the failTimeout parameter to consider the server unavailable. The default 1 . |
int |
No |
failTimeout |
Sets the time during which the specified number of unsuccessful attempts to communicate with the server should happen to consider the server unavailable and the period of time the server will be considered unavailable. The default is 10s . |
string |
No |
healthCheck |
The health check configuration for the Upstream. See the health_check directive. Note: this feature is supported only in NGINX Plus. | healthcheck | No |
Upstream.Healthcheck
The Healthcheck defines an active health check. In the example below we enable a health check for an upstream and configure all the available parameters:
name: secure-app
service: secure-app
port: 8443
healthCheck:
enable: true
interval: 20s
timeout: 30s
jitter: 3s
fails: 5
passes: 5
port: 8080
Note: This feature is supported only in NGINX Plus.
Field | Description | Type | Required |
---|---|---|---|
enable |
Enables a health check for an upstream server. The default is false . |
boolean |
No |
interval |
The interval between two consecutive health checks. The default is 5s . |
string |
No |
timeout |
This overrides the timeout set by proxy_timeout which is set in SessionParameters for health checks. The default value is 5s . |
string |
No |
jitter |
The time within which each health check will be randomly delayed. By default, there is no delay. | string |
No |
fails |
The number of consecutive failed health checks of a particular upstream server after which this server will be considered unhealthy. The default is 1 . |
integer |
No |
passes |
The number of consecutive passed health checks of a particular upstream server after which the server will be considered healthy. The default is 1 . |
integer |
No |
port |
The port used for health check requests. By default, the port of the upstream is used. Note: in contrast with the port of the upstream, this port is not a service port, but a port of a pod. | integer |
No |
UpstreamParameters
The upstream parameters define various parameters for the upstreams:
upstreamParameters:
udpRequests: 1
udpResponses: 1
connectTimeout: 60s
nextUpstream: true
nextUpstreamTimeout: 50s
nextUpstreamTries: 1
Field | Description | Type | Required |
---|---|---|---|
udpRequests |
The number of datagrams, after receiving which, the next datagram from the same client starts a new session. See the proxy_requests directive. The default is 0 . |
int |
No |
udpResponses |
The number of datagrams expected from the proxied server in response to a client datagram. See the proxy_responses directive. By default, the number of datagrams is not limited. | int |
No |
connectTimeout |
The timeout for establishing a connection with a proxied server. See the proxy_connect_timeout directive. The default is 60s . |
string |
No |
nextUpstream |
If a connection to the proxied server cannot be established, determines whether a client connection will be passed to the next server. See the proxy_next_upstream directive. The default is true . |
bool | No |
nextUpstreamTries |
The number of tries for passing a connection to the next server. See the proxy_next_upstream_tries directive. The default is 0 . |
int |
No |
nextUpstreamTimeout |
The time allowed to pass a connection to the next server. See the proxy_next_upstream_timeout directive. The default us 0 . |
string |
No |
SessionParameters
The session parameters define various parameters for TCP connections and UDP sessions.
sessionParameters:
timeout: 50s
Field | Description | Type | Required |
---|---|---|---|
timeout |
The timeout between two succesive read or write operations on client or proxied server connections. See proxy_timeout directive. The default is 10m . |
string |
No |
Action
The action defines an action to perform for a client connection/datagram.
In the example below, client connections/datagrams are passed to an upstream dns-app
:
action:
pass: dns-app
Field | Description | Type | Required |
---|---|---|---|
pass |
Passes connections/datagrams to an upstream. The upstream with that name must be defined in the resource. | string |
Yes |
Using TransportServer
You can use the usual kubectl
commands to work with TransportServer resources, similar to Ingress resources.
For example, the following command creates a TransportServer resource defined in transport-server-passthrough.yaml
with the name secure-app
:
$ kubectl apply -f transport-server-passthrough.yaml
transportserver.k8s.nginx.org/secure-app created
You can get the resource by running:
$ kubectl get transportserver secure-app
NAME AGE
secure-app 46sm
In the kubectl get and similar commands, you can also use the short name ts
instead of transportserver
.
Using Snippets
Snippets allow you to insert raw NGINX config into different contexts of NGINX configuration. In the example below, we use snippets to configure access control in a TransportServer:
apiVersion: k8s.nginx.org/v1alpha1
kind: TransportServer
metadata:
name: cafe
spec:
host: cafe.example.com
serverSnippets: |
deny 192.168.1.1;
allow 192.168.1.0/24;
upstreams:
- name: tea
service: tea-svc
port: 80
Snippets are intended to be used by advanced NGINX users who need more control over the generated NGINX configuration.
However, because of the disadvantages described below, snippets are disabled by default. To use snippets, set the enable-snippets
command-line argument.
Disadvantages of using snippets:
- Complexity. To use snippets, you will need to:
- Understand NGINX configuration primitives and implement a correct NGINX configuration.
- Understand how the IC generates NGINX configuration so that a snippet doesn’t interfere with the other features in the configuration.
- Decreased robustness. An incorrect snippet makes the NGINX config invalid which will lead to a failed reload. This will prevent any new configuration updates, including updates for the other TransportServer resource until the snippet is fixed.
- Security implications. Snippets give access to NGINX configuration primitives and those primitives are not validated by the Ingress Controller.
Note: during a period when the NGINX config includes an invalid snippet, NGINX will continue to operate with the latest valid configuration.
Note: to configure snippets in thestream
context, usestream-snippets
ConfigMap key.
Validation
Two types of validation are available for the TransportServer resource:
- Structural validation by the
kubectl
and Kubernetes API server. - Comprehensive validation by the Ingress Controller.
Structural Validation
The custom resource definition for the TransportServer includes structural OpenAPI schema which describes the type of every field of the resource.
If you try to create (or update) a resource that violates the structural schema (for example, you use a string value for the port field of an upstream), kubectl
and Kubernetes API server will reject such a resource:
Example of
kubectl
validation:$ kubectl apply -f transport-server-passthrough.yaml error: error validating "transport-server-passthrough.yaml": error validating data: ValidationError(TransportServer.spec.upstreams[0].port): invalid type for org.nginx.k8s.v1alpha1.TransportServer.spec.upstreams.port: got "string", expected "integer"; if you choose to ignore these errors, turn validation off with --validate=false
Example of Kubernetes API server validation:
$ kubectl apply -f transport-server-passthrough.yaml --validate=false The TransportServer "secure-app" is invalid: []: Invalid value: map[string]interface {}{ ... }: validation failure list: spec.upstreams.port in body must be of type integer: "string"
If a resource is not rejected (it doesn’t violate the structural schema), the Ingress Controller will validate it further.
Comprehensive Validation
The Ingress Controller validates the fields of a TransportServer resource. If a resource is invalid, the Ingress Controller will reject it: the resource will continue to exist in the cluster, but the Ingress Controller will ignore it.
You can check if the Ingress Controller successfully applied the configuration for a TransportServer. For our example secure-app
TransportServer, we can run:
$ kubectl describe ts secure-app
. . .
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal AddedOrUpdated 3s nginx-ingress-controller Configuration for default/secure-app was added or updated
Note how the events section includes a Normal event with the AddedOrUpdated reason that informs us that the configuration was successfully applied.
If you create an invalid resource, the Ingress Controller will reject it and emit a Rejected event. For example, if you create a TransportServer secure-app
with a pass action that references a non-existing upstream, you will get :
$ kubectl describe ts secure-app
. . .
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning Rejected 2s nginx-ingress-controller TransportServer default/secure-app is invalid and was rejected: spec.action.pass: Not found: "some-app"
Note how the events section includes a Warning event with the Rejected reason.
Note: If you make an existing resource invalid, the Ingress Controller will reject it and remove the corresponding configuration from NGINX.
Customization via ConfigMap
The ConfigMap keys (except for stream-snippets
and stream-log-format
) do not affect TransportServer resources.
Limitations
The TransportServer resource is a preview feature. Currently, it comes with the following limitation:
- When using TLS Passthrough, it is not possible to configure Proxy Protocol for port 443 both for regular HTTPS and TLS Passthrough traffic.