RKE2 Concierge support?

[vvarga007]

Hi

Based on the documentation concierge does not support RKE2.
I am trying to set this up on RKE2, but the concierge throws the following error:

{"level":"error","timestamp":"2024-07-03T13:11:00.961633Z","caller":"go.pinniped.dev/internal/controllerlib/controller.go:222$controllerlib.(*controller).handleKey","message":"kube-cert-agent-controller: { } failed with: failed to get kube-public/cluster-info configmap: configmap \"cluster-info\" not found"}

[cfryanr]

Hi @vvarga007, thanks for asking your question here.

That error is normal on clusters that were not created using kubeadm or similar tooling. Pinniped wants to get the URL of the Kubetnetes API Server for external (outside the cluster) clients to use, so it can advertise it to pinniped get kubeconfig to use during kubeconfig generation.

It might be possible to manually create that configmap to allow the Concierge to get a little further. It would need to contain the kubeconfig key, and the value would need to be similar enough to a standard cluster-info configmap for it to be parsed correctly by the Kubernetes library functions. Pinniped is trying to look at the first value in the clusters array to get the server and certificateAuthorityData fields. I'd be curious if that makes it work.

When that strategy does not work, then the Pinniped Concierge will use the alternate strategy, which is the impersonation proxy. This will open up a port on the cluster where clients can connect to the Pinniped Concierge, authenticate, and then their Kubernetes API requests will be proxied to the actual Kubernetes API server with their identities attached. It looks like that might be the way to go on this Kubernetes distribution. Is that working for you?

[cfryanr]

By the way, here is an example configmap, in case you want to try creating one to see if that makes it work:

apiVersion: v1
kind: ConfigMap
metadata:
  name: cluster-info
  namespace: kube-public
data:
  kubeconfig: |
    apiVersion: v1
    kind: Config
    clusters:
    - cluster:
        # fill this in with the real CA bundle of the Kube API server, base64 encoded
        certificate-authority-data: LS0t...
        # fill this in with the real externally available address of the Kube API server
        server: https://kube-api.server.example.com:6443
      name: ""
    contexts: null
    current-context: ""
    preferences: {}
    users: null

[vvarga007]

Thanks; I will give it a try. But before that, I tried to implement Pinniped without the concierge, as I could freely change the Kube API parameters. For some reason, this also doesn't work.
I get the following message from the supervisor in a browser window after a Keycloak login.

Login succeeded
You have successfully logged in. You may now close this tab.

But the kubectl using the pinniped config throws the following error:

Error: You must be logged in to the server (Unauthorized)

kubelogin works fine with the same settings. So, it must be a pinniped misconfiguration. Any idea what could be the issue here

[cfryanr]

It sounds like you were following this doc? https://pinniped.dev/docs/tutorials/supervisor-without-concierge-demo/

Maybe check the logs of the Kubernetes API Server to see if there is any message at the same timestamp which explains why it did not accept the JWT as authorization.