Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes to enable Admiralty on OpenShift #134

Open
wants to merge 9 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions charts/multicluster-scheduler/templates/cr.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,13 @@ rules:
- delete
- deletecollection
- patch
- apiGroups:
- multicluster.admiralty.io
resources:
- podchaperons/finalizers
- sources/finalizers
verbs:
- update
- apiGroups:
- ""
resources:
Expand Down Expand Up @@ -236,6 +243,10 @@ rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["list"]
- apiGroups: [""]
resources: ["pods/finalizers"]
verbs:
adrienjt marked this conversation as resolved.
Show resolved Hide resolved
- update
- apiGroups: [""]
resources: ["pods/log"]
verbs: ["get"]
Expand Down
105 changes: 105 additions & 0 deletions docs/tutorials/ocp-ibm.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@

# Red Hat OpenShift on IBM Cloud

The [quick start guide](https://admiralty.io/docs/quick_start) provides clear instructions how to use Admiralty on Kubernetes clusters. The only
thing you need to pay special attention to is how to create a kubeconfig secret that would work in your OpenShift cluster on IBM Cloud. This tutorial will
guide you how to create the kubeconfig secret when you use the Red Hat OpenShift on IBM Cloud service as one of your target clusters. The source cluster can be a Kubernetes or OpenShift cluster.

## Prerequisites
- the [Red Hat OpenShift on IBM Cloud service](https://www.ibm.com/cloud/openshift)
- the required [CLI tools](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift-cli) ( e.g., IBMCLOUD CLI and OpenShift CLI (oc) )

## Kubeconfig for Authentication
You can follow this [link](https://cloud.ibm.com/docs/openshift?topic=openshift-access_cluster) to access your OpenShift cluster.
After you connect to the OpenShift cluster, you can use the following IBMCLOUD CLI command
to retrieve the kubeconfig file.
```bash
export KUBECONFIG=~/.kube/config

ibmcloud oc cluster config --cluster <your cluster name> --admin
```
Your config file should look like the following:
```
apiVersion: v1
clusters:
- cluster:
server: <api server>
name: <cluster name>
contexts:
- context:
cluster: <cluster name>
namespace: default
user: <admin user name>
name: <context name>
current-context: <context name>
kind: Config
preferences: {}
users:
- name: <admin user name>
user:
client-certificate: <path to client certificate>
client-key: <path to client key>
```
Let's modify the config file to the following format below:
```
apiVersion: v1
clusters:
- cluster:
server: <api server>
certificate-authority-data: <ca data>
name: <cluster name>
contexts:
- context:
cluster: <cluster name>
namespace: default
user: <admin user name>
name: <context name>
current-context: <context name>
kind: Config
preferences: {}
users:
- name: <admin user name>
user:
token: <service account token>
```
The fields, client-certificate and client-key, are being removed and certificate-authority-data and token fields are added.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you use jq to edit the downloaded kubeconfig, like in the quick start guide, to make this more foolproof/automatisable?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. I can provide the jq command. Shall I create another PR ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, please update this PR to use jq.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You now get the config twice, and the CA cert twice... this could be simplified.

Actually, I'm thinking this should become part of the quick start page itself. Using Tabs/TabItem, the user could select between kind and Red Hat OpenShift on IBM Cloud. (We'd add GKE/EKS/AKS tabs too later.) What do you think?


For the token part, you can follow the instructions in the [quick start guide](https://admiralty.io/docs/quick_start) to get the service account token.

To get the certificate-authority-data, you can use the command below to get the encoded CA data.
```bash
CA_DATA="$(curl https://cacerts.digicert.com/DigiCertGlobalRootCA.crt.pem | base64 -w0)"
```

You are now ready to create a kubeconfig secret for the target cluster. Use the commands below to automate the entire process and have the new kubeconfig content stored in the CONFIG variable:

On the OpenShift target cluster:

```bash
# the namespace where a service account is created
NS=myproj
# the name of your service account
SA_NAME=sa-$NS

# the secret name for your service account
SECRET_NAME=$(oc get serviceaccount $SA_NAME -n $NS --output json | jq -r '.secrets[] | select(.name | contains("token"))' | jq -r '.name')
# the token in the secret
TOKEN=$(oc get secret $SECRET_NAME -n $NS --output json | jq -r '.data.token' | base64 --decode)

# the CA data
CA_CERT="$(curl https://cacerts.digicert.com/DigiCertGlobalRootCA.crt.pem | base64 -w0)"

CONFIG=$( oc config view --minify --raw --output json | jq '.clusters[0].cluster["certificate-authority-data"] = "'$CA_DATA'" | del(.clusters[0].cluster."certificate-authority")' | jq '.users[0].user={token:"'$TOKEN'"}' )
```

On the source cluster, you can then create a secret for the target cluster, where its kubeconfig is stored in the $CONFIG variable.
```
oc create secret generic <secret_name_for_target_cluster> --from-literal=config="$CONFIG"
```
You may also need to adjust the security context constraints (SCCs) as your pod may be configured with the restricted SCC by default in OpenShift. Run the command below when using OpenShift on IBM Cloud:
```
oc adm policy add-scc-to-user ibm-anyuid-scc -z default -n <namespace of your service account>
```

## Summary
In this tutorial, you've learned how to create a kubeconfig secret for the Red Hat OpenShift cluster on IBM Cloud. You can follow the rest of the steps in the [quick start guide](https://admiralty.io/docs/quick_start) to use Admiralty on OpenShift.