Secure the k2d API endpoint kubernetes.default.svc

[ncresswell]

In the Alpha, the kubernetes API endpoint is not secured for communications with the emulated cluster. This means that any pod for any deployment has complete access to the emulated API, akin to "anonymous" access to the Kube API. This is not secure.

We already generate a kube access token, which is used for remote connections. We should also enforce the use of this when accessing the k2d endpoint from within the docker environment.

[deviantony]

The current implementation uses a secret to interact with the k2d specific API. Currently, only the API endpoint to retrieve the kubeconfig is protected by this secret based authentication. This implementation was developed to prevent anonymous users from retrieving the kubeconfig when k2d is publicly exposed.

Currently, no authentication method is supported for the Kubernetes API. However, since we are already generating a token in the kubeconfig, we can improve this by ensuring that the token is used when communicating with the Kubernetes API as well as generating a stronger token.

[deviantony]

It's worth noting that Kubernetes will inject the necessary credentials by default in all pods/containers running inside the cluster. These credentials can be used to authenticate and execute requests to the Kubernetes API, as explained in this documentation: https://kubernetes.io/docs/tasks/run-application/access-api-from-pod/

Therefore, the current approach is similar, as it allows direct access to the Kubernetes API.

However, it is essential to ensure that the Kubernetes API cannot be accessed from outside the cluster without some form of authentication.

[deviantony]

The approach I suggest for authentication is to leverage the secret passed to k2d (or automatically generated if not present) as the bearer token used for any request to the k2d API:

• This token will be stored on disk under /var/lib/k2d/token in clear text and mounted in every container that is created via k2d

This will allow full access to the Kubernetes API via:

curl -k \
  -H "Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" \
  https://kubernetes.default.svc:6443/api/v1/namespaces

• The token will be part of the kubeconfig file

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: <ENCODED_CERT>
    server: https://<HOST_IP>:6443
  name: k2d
contexts:
- context:
    cluster: k2d
    user: k2d-root
  name: k2d
current-context: k2d
kind: Config
preferences: {}
users:
- name: k2d-root
  user:
    token: <K2D_SECRET_VALUE>

• kubectl and helm can automatically use this token from the kubeconfig
• Introduce a breaking change to the current getting started experience by replacing the custom x-k2d-secret header that is currently used for the kubeconfig retrieval only with the use of the standard Authorization header

This means that the following command:

curl --insecure -H "x-k2d-secret: YOUR_OWN_SECRET" https://YOUR_HOST_IP:6443/k2d/kubeconfig

Would be changed to:

curl --insecure -H "Authorization: Bearer YOUR_OWN_SECRET" https://YOUR_HOST_IP:6443/k2d/kubeconfig

You could then query the Kubernetes API using the Authorization bearer as well:

curl --insecure -H "Authorization: Bearer YOUR_OWN_SECRET" https://YOUR_HOST_IP:6443/api/v1/pods

Consideration:

• The token could be created as a base64 encoded version of the specified/generated secret for portability reasons. However that would force the user to base64 encode their secret before being able to call the API command to retrieve the kubeconfig, adding a bit of friction to the getting started UX.

This implies that instead of running the following command after starting k2d:

curl --insecure -H "Authorization: Bearer YOUR_OWN_SECRET" https://YOUR_HOST_IP:6443/k2d/kubeconfig

You would need to run the following command, therefore relying on the availability of base64 on the system:

curl --insecure \
  -H "Authorization: Bearer $(echo -n 'YOUR_OWN_SECRET' | base64)" \
  https://YOUR_HOST_IP:6443/k2d/kubeconfig

[ncresswell]

I think a reliance on base64 should be ok, just validate that MacOS/Windows OS have it installed by default... as that is where most users would connect from to retrieve the token.