OpenShift token retrieval from Keycloak may return a stale token

[metlos]

Describe the bug

When using an access token obtained from Keycloak using the token exchange algorithm using a valid openshift token, che server is then unable to perform authenticated request to OpenShift because the token it is able to obtain from keycloak is not the one used to obtain the token.

Che version

latest

Steps to reproduce

0. Let CLUSTER_DOMAIN be the root domain of an OpenShift 4 cluster.
1. Setup Che on OpenShift4 with OpenShift OAuth enabled (the default)
2. Login to the cluster using oc with some administrative account.
3. Setup the OpenShift tokens to expire in 2 minutes to avoid having to wait for a long time:

oc patch oauth cluster --type=json -p '[{"op": "add", "path": "/spec/tokenConfig", "value": {}},{"op": "add", "path": "/spec/tokenConfig/accessTokenMaxAgeSeconds", "value": 120}]'

3. Login using UI into Eclipse Che (let's say with a user "A")
4. Wait for more than 2 minutes
5. Login to openshift using the CLI and the same user oc login -u A --server=https://api.$CLUSTER_DOMAIN:6443
6. Obtain the OpenShift token: oc whoami -t
7. Get a Keycloak access token using the token exchange algorithm:

KC_TOKEN=$(curl -sk --location --request POST 'https://keycloak-eclipse-che.apps.$CLUSTER_DOMAIN/auth/realms/che/protocol/openid-connect/token' \
--data-urlencode 'client_id=che-public' \
--data-urlencode 'grant_type=urn:ietf:params:oauth:grant-type:token-exchange' \
--data-urlencode 'subject_token_type=urn:ietf:params:oauth:token-type:access_token' \
--data-urlencode 'subject_issuer=openshift-v4' \
--data-urlencode "subject_token=$(oc whoami -t)" |jq -r .access_token)

8. Try to perform an operation that requires Che to figure out the OpenShift token from the $KC_TOKEN Keycloak token, e.g.:

curl -vk -X POST -H "Content-type: application/json" -H "Authorization: Bearer $KC_TOKEN" -d '{"apiVersion":"1.0.0","metadata":{"name":"wksp-test"}}' -H "Accept: application/json, */*" https://che-eclipse-che.apps.$CLUSTER_DOMAIN/api/workspace/devfile

Expected behavior

The access token was obtained using a valid OpenShift token, so it should succeed. But the operation fails with unauthorized error.

I would expect that the OpenShift token I used to generated a Keycloak token would be used when converting the Keycloak token back to OpenShift token during the calls made by the che server.

This is most probably because Che server doesn't use the token exchange algorithm to get the OpenShift token (see below). Not that we need to make sure this also works on OpenShift 3.

Runtime

OpenShift 4.6

Installation method

chectl server:deploy -p openshift -a operator

Eclipse Che Logs

N/A

Additional context

It looks like when using token exchange for getting the Keycloak token, the OpenShift token used for that is not persisted in all places inside Keycloak. Che server does not use token exchange for getting the token. Instead, it uses an equivalent of this call:

curl -sk --location 'https://keycloak-eclipse-che.apps.$CLUSTER_DOMAIN/auth/realms/che/broker/openshift-v4/token' -H "Authorization: Bearer $KC_TOKEN" -H "Accept: application/json"|jq -r .access_token

But this returns the OpenShift token last used in the UI login.

When the token exchange algorithm is used for getting the OpenShift token from the Keycloak token (using the Keycloak token obtained from token exchange itself), the OpenShift token used for obtaining that Keycloak token is returned.

curl -sk --location --request POST 'https://keycloak-eclipse-che.apps.$CLUSTER_DOMAIN/auth/realms/che/protocol/openid-connect/token' \
--data-urlencode 'client_id=che-public' \
--data-urlencode 'grant_type=urn:ietf:params:oauth:grant-type:token-exchange' \
--data-urlencode 'requested_token_type=urn:ietf:params:oauth:token-type:access_token' \
--data-urlencode 'requested_issuer=openshift-v4' \
--data-urlencode "subject_token=$KC_TOKEN"|jq -r .access_token

[mshaposhnik]

Spend 2 days already playing with that tokens exchange, and the conclusion is the following:

• Switching our code to use exchange mechanism is useful, but does not solves the problem fully, since keycloak does not refreshes external IDP token even if internal token is changed/refreshed. So even when using the token exchange,
  Openshift token returned is always the same whicn was remembered during user login.
  There is a lot of complains around that fact [1] [2] etc..

The only visible solution is to force to re-login user, when we see the fact that undelying OIDC token is expired.

[1] https://github.com/keycloak/keycloak-community/issues/277
[2] https://lists.jboss.org/pipermail/keycloak-user/2018-February/013127.html

[skabashnyuk]

Not possible to fix in general and is not relevant for devworkspaces.