Azure Kubernetes Service

Azure Kubernetes Service (AKS) is a hosted Kubernetes solution created by Microsoft. To use AKS with NGINX Service Mesh, you need to make a few extra configurations. With Kubernetes role-based access control (RBAC) enabled, AKS has --authentication-token-webhook for kubelet set to false. NGINX Service Mesh requires this flag to be set to true.

Update Kubelet Configuration

The below DaemonSet will start up one pod per node, change the setting of --authentication-token-webhook to true, then go to sleep.

apiVersion: apps/v1
kind: DaemonSet
metadata:
  labels:
    component: custom-kubelet-webhook
  name: custom-kubelet-webhook
  namespace: kube-system
spec:
  selector:
    matchLabels:
      component: custom-kubelet-webhook
      tier: node
  template:
    metadata:
      labels:
        component: custom-kubelet-webhook
        tier: node
    spec:
      containers:
      - name: custom-kubelet-webhook
        image: alpine
        imagePullPolicy: IfNotPresent
        command:
          - nsenter
          - --target
          - "1"
          - --mount
          - --uts
          - --ipc
          - --net
          - --pid
          - --
          - sh
          - -c
          - |
            if ! $(grep -q authentication-token-webhook=true /etc/default/kubelet); then sed -i 's/--authorization-mode=Webhook/--authorization-mode=Webhook --authentication-token-webhook=true/g' /etc/default/kubelet; systemctl restart kubelet; fi
            sleep infinity
        resources:
          requests:
            cpu: 50m
        securityContext:
          privileged: true
      dnsPolicy: ClusterFirst
      hostPID: true
      tolerations:
      - effect: NoSchedule
        operator: Exists
      restartPolicy: Always
  updateStrategy:
    type: OnDelete

To apply the DaemonSet:

  1. Save it to a file. For example, aks-kubelet-webook.yaml.

  2. Apply the configuration using kubectl:

    kubectl apply -f aks-kubelet-webook.yaml
    

Removal

To revert the setting back to --authentication-token-webhook=false, use the DaemonSet provided below.

apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  labels:
    component: custom-kubelet-webhook-default
  name: custom-kubelet-webhook-default
  namespace: kube-system
spec:
  selector:
    matchLabels:
      component: custom-kubelet-webhook-default
      tier: node
  template:
    metadata:
      labels:
        component: custom-kubelet-webhook-default
        tier: node
    spec:
      containers:
      - name: custom-kubelet-webhook-default
        image: alpine
        imagePullPolicy: IfNotPresent
        command:
          - nsenter
          - --target
          - "1"
          - --mount
          - --uts
          - --ipc
          - --net
          - --pid
          - --
          - sh
          - -c
          - |
            if $(grep -q authentication-token-webhook=true /etc/default/kubelet); then sed -i 's/--authentication-token-webhook=true//g' /etc/default/kubelet; systemctl restart kubelet; fi
            sleep infinity
        resources:
          requests:
           cpu: 50m
        securityContext:
          privileged: true
      dnsPolicy: ClusterFirst
      hostPID: true
      tolerations:
      - effect: NoSchedule
        operator: Exists
      restartPolicy: Always
  updateStrategy:
    type: OnDelete

To apply the DaemonSet:

  1. Save it to a file. For example: aks-kubelet-webook-default.yaml.

  2. Using kubectl, remove the previous DaemonSet and apply this one:

    kubetl delete -f aks-kubelet-webook.yaml
    kubectl apply -f aks-kubelet-webook-default.yaml