Use the NGINX Service Mesh API
Instructions for using the NGINX Service Mesh API.
This topic documents a beta feature. Beta features are provided for you to try out before they are released. You shouldn't use beta features for production purposes.
Warning:
The NGINX Service Mesh API is in beta release status and is not recommended for use in client-side automation. The objects described here are subject to change as the API evolves. No guarantee of backwards compatibility is made.
Overview
The Service Mesh API is a Kubernetes APIService that offers a way to view and manage resources within the mesh. It can be accessed through the Kubernetes API server or via the command line tool.
Note:
Refer to the API documentation for endpoint descriptions, request payloads, and response bodies.
How it works
The v1alpha1.nsm.nginx.com
APIService
extends the Kubernetes API and claims the URL path /apis/nsm.nginx.com/v1alpha1
in the Kubernetes API.
When a request is sent to the path /apis/nsm.nginx.com/v1alpha1
, the Kubernetes aggregation layer proxies the request to the nginx-mesh-api
Service running in the NGINX Service Mesh namespace.
The nginx-mesh-api
Pod runs an extension API server that implements the APIService
, and handles the requests proxied from the Kubernetes aggregation layer. All authentication and authorization decisions are delegated to the Kubernetes API server.
For more information on how authentication and authorization works, see the Authentication Flow section in the Kubernetes docs.
NGINX Service Mesh API Resources
The following APIResourceList
describes the resources and actions that are available through the NGINX Service Mesh API:
{
"kind": "APIResourceList",
"apiVersion": "v1",
"groupVersion": "nsm.nginx.com/v1alpha1",
"resources": [
{
"name": "services",
"singularName": "",
"namespaced": false,
"kind": "",
"verbs": [
"list"
]
},
{
"name": "config",
"singularName": "",
"namespaced": false,
"kind": "",
"verbs": [
"list",
"patch"
]
},
{
"name": "inject",
"singularName": "",
"namespaced": false,
"kind": "",
"verbs": [
"create"
]
}
]
}
This APIResourceList
can be retrieved from the /apis/nsm.nginx.com/v1alpha
endpoint.
Important:
The Kubernetes API discovery rolesystem:discovery
allows read-only access to this endpoint. By default, thesystem:discovery
role is bound to the Kubernetes groupsystem:authenticated
, which allows all authenticated users access to this endpoint. Some managed Kubernetes environments may also bind thesystem:discovery
role to the Kubernetes groupsystem:unauthenticated
, which allows unauthenticated users access to this endpoint as well. For more information on API discovery roles and how to check your cluster’s configuration, see the Kubernetes API discovery roles doc.
Authentication and Authorization
In order to access the NGINX Service Mesh API, you must be authenticated with the Kubernetes API server and authorized to perform the action (for example, list
, create
, patch
) on the nsm.nginx.com
resource (for example, config
, services
, inject
).
You can find information about authenticating with the Kubernetes API server in the Kubernetes Authenticating documentation.
In addition to authenticating, you must be authorized to access an nsm.nginx.com
resource. The following ClusterRole
contains all the permissions needed to perform the available actions on each resource in the nsm.nginx.com
API group:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: nsm-api-full-access
rules:
- apiGroups:
- nsm.nginx.com
resources:
- services
- config
verbs:
- list
- apiGroups:
- nsm.nginx.com
resources:
- config
verbs:
- patch
- apiGroups:
- nsm.nginx.com
resources:
- inject
verbs:
- create
You can also create a ClusterRole
with a subset of these permissions if you do not want to grant a user or Pod full access to the NGINX Service Mesh API. For example, if you would like to allow a user to only list NGINX Service Mesh services, you can define the following ClusterRole
:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: nsm-api-list-services
rules:
- apiGroups:
- nsm.nginx.com
resources:
- services
verbs:
- list
For more information on how to configure authorization in Kubernetes, see their Authorization Overview doc.
Access the NGINX Service Mesh REST API
Before you can access the NGINX Service Mesh API in any of the manners listed below, you must be authenticated to the Kubernetes API server and authorized to access the resource. See the Authentication and Authorization section for more details.
Kubectl proxy
You can access the NGINX Service Mesh REST API by running kubectl
in proxy mode. This method is recommended by Kubernetes because it protects against man-in-the-middle attacks by locating and authenticating to the API server on behalf of the user.
For more information and directions on how to use kubectl proxy
, see the Using kubectl proxy section in the Kubernetes docs.
Example:
kubectl proxy --port=8080 &
curl http://localhost:8080/apis/nsm.nginx.com/v1alpha1
Direct access to Kubernetes API server
To access the NGINX Service Mesh REST API directly though the Kubernetes API server, you can provide the location and credentials of the Kubernetes API server directly to the HTTP client. For more information and directions, see the Without kubectl proxy section in the Kubernetes docs.
Example:
APISERVER=$(kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}')
## If using kubectl version 1.24 or greater:
TOKEN=$(kubectl create token default)
## Otherwise:
TOKEN=$(kubectl get secret $(kubectl get serviceaccount default -o jsonpath='{.secrets[0].name}') -o jsonpath='{.data.token}' | base64 --decode)
curl ${APISERVER}/apis/nsm.nginx.com/v1alpha1 --cacert {PATH_TO_CLUSTER_CA_CERT} -H "Authorization: Bearer $TOKEN"
Use kubectl to send requests to the API
In some cases, shown below, you can use kubectl
to get NGINX Service Mesh API resources.
To get the APIResourceList
:
kubectl get --raw /apis/nsm.nginx.com/v1alpha1
To get the NGINX Service Mesh deployment configuration:
kubectl get --raw /apis/nsm.nginx.com/v1alpha1/config
To get the list of NGINX Service Mesh services:
kubectl get --raw /apis/nsm.nginx.com/v1alpha1/services
Command Line Access
The nginx-meshctl
command line tool acts as a wrapper around the Service Mesh API. You can use the nginx-meshctl
CLI to access the API endpoints, which simplifies human interactions with the REST API. For automation purposes, you can also access the REST API programmatically.
Refer to the CLI documentation for more information on how to use nginx-meshctl
.
Programmatic Access
For programmatic access, we recommend using a Kubernetes client SDK.
Example Use Cases
View the Mesh State by using the REST API
You can use the REST API, nginx-meshctl
command line tool, or kubectl
to view the current state of the mesh.
-
View the configuration of the mesh:
- REST API endpoint:
/apis/nginx.nsm.com/v1alpha1/config
- kubectl command:
kubectl get --raw /apis/nginx.nsm.com/v1alpha1/config
- CLI command:
nginx-meshctl config
- REST API endpoint:
-
View the services participating in the mesh:
- REST API endpoint:
/apis/nsm.nginx.com/v1alpha1/services
- kubectl command:
kubectl get --raw /apis/nginx.nsm.com/v1alpha1/services
- CLI command:
nginx-meshctl services
- REST API endpoint:
Modify the Mesh State by using the REST API
NGINX Service Mesh API is under active development, but provides an endpoint and PATCH method to update a subset of the deployment configuration. The API schema is described in the patchConfig section.
You can PATCH the configuration of NGINX Service Mesh by sending a request to the REST API endpoint:
- REST API endpoint:
/apis/nginx.nsm.com/v1alpha1/config
The supported patch operations are:
add
remove
replace
There are a subset of fields and objects supported for add, remove, and replace. Refer to the PATCH config schema described above for the full reference, the following examples can be referred to for a quick start.
Example: Disable Automatic injection in All Namespaces
The payload shown below disables automatic injection of the sidecar proxy for all namespaces and enables it for only the “prod” and “staging” namespaces.
[
{
"op": "replace",
"field": {
"injection": {
"isAutoInjectEnabled": false
}
}
},
{
"op": "add",
"field": {
"injection": {
"enabledNamespaces": ["prod", "staging"]
}
}
}
]
or
{
"op": "replace",
"field": {
"injection": {
"isAutoInjectEnabled": false,
"enabledNamespaces": ["default", "my-namespace"]
}
}
}
To remove
all values from a list of strings, define the value as an empty list (using replace
with an empty list will have the same effect). For example:
{
"op": "remove",
"field": {
"injection": {
"enabledNamespaces": []
}
}
}
Inject the Sidecar Proxy into Kubernetes Resources
You can use the CLI or the REST API to manually inject the sidecar proxy into a Kubernetes resource definition.
- CLI command:
nginx-meshctl inject
- API endpoint:
/apis/nsm.nginx.com/v1alpha1/inject
The NGINX Service Mesh supports injection for the following Kubernetes resources:
- Deployment
- DaemonSet
- StatefulSet
- ReplicaSet
- ReplicationController
- Job
- Pod
Requests to the /apis/nsm.nginx.com/v1alpha1/inject
endpoint must include the following:
-
Content-Type: multipart/form-data
header -
a JSON or YAML file sent as a form field with a key name of
file
and theContent-Type: octet-stream
header.Usage:
-F file=@my-app.json
The endpoint also supports the following optional form fields:
-
ignoreIncomingPorts
: a string list of ports for the proxy to ignore for incoming traffic; withContent-Type: text/plain
.Usage:
-F "ignoreIncomingPorts=80;type=text/plain"
-
ignoreOutgoingPorts
: a string list of ports for the proxy to ignore for outgoing traffic; withContent-Type: text/plain
.Usage:
-F "ignoreOutgoingPorts=90;type=text/plain"
Example cURL Requests for Sidecar Proxy Injection
Important:
Read the Direct access to Kubernetes API server before trying any of the following examples.
-
Provide a JSON file for injection:
curl https://{APISERVER}/apis/nsm.nginx.com/v1alpha1/inject -X POST -H "Authorization: Bearer $TOKEN" --insecure -H "Content-Type:multipart/form-data" -F file=@my-app.json
-
Provide a YAML file, ignore incoming requests for port 80, and ignore outgoing traffic on port 90:
curl https://{KUBERNETES_APISERVER}/apis/nsm.nginx.com/v1alpha1/inject -X POST -H "Authorization: Bearer $TOKEN" --insecure -H "Content-Type:multipart/form-data" -F file=@my-app.yaml -F "ignoreIncomingPorts=80;type=text/plain" -F "ignoreOutgoingPorts=90;type=text/plain"
-
Ignore incoming requests on multiple ports (80, 90):
curl https://{APISERVER}/apis/nsm.nginx.com/v1alpha1/inject -X POST -H "Authorization: Bearer $TOKEN" --insecure -H "Content-Type:multipart/form-data" -F file=@my-app.yaml -F "ignoreIncomingPorts=80;type=text/plain" -F "ignoreIncomingPorts=90;type=text/plain"
Internal Configuration API Endpoints
There are a few API endpoints that are used by the NGINX Service Mesh CLI and Helm jobs to communicate with the control plane. The list below describes each endpoint, the CLI command and Helm job that calls them, and the Kubernetes permissions needed.
Important:
These endpoints are for internal use only.
Resource Name | REST API Endpoint | Description | CLI Command | Helm Job | Permissions |
---|---|---|---|---|---|
clear | /apis/nsm.nginx.com/v1alpha1/clear |
a POST request to /clear turns all NGINX Service Mesh sidecars transparent |
remove | turn-proxies-transparent | create clear in APIGroup nsm.nginx.com |
resources | /apis/nsm.nginx.com/v1alpha1/resources |
a GET request to /resources returns the list of resources that are injected with the NGINX Service Mesh sidecar |
remove | N/A | list resources in APIGroup nsm.nginx.com |
version | /apis/nsm.nginx.com/v1alpha1/version |
a GET request to /version returns the versions of the control plane components and the sidecars |
version | N/A | list version in APIGroup nsm.nginx.com |