Skip to content

Commit

Permalink
Implement CCCMO render to run CCM on bootstrap node
Browse files Browse the repository at this point in the history
This allows master nodes to be initialized with CCM replica from
bootstrap, speeding up process of cluster installation.

Without CCM running on bootstrap node, most of the cluster operator
and operand pods would have to tolerate new taint and a different
configuration of node.kubernetes.io/not-ready (NoSchedule) taint:

- effect: NoSchedule
  key: node.cloudprovider.kubernetes.io/uninitialized
  value: "true"
- effect: NoSchedule
  key: node.kubernetes.io/not-ready

Without this bootstrap addition or taint toleration the cluster with CCM
would never become ready.

To test this feature on AWS build installer and deploy on AWS.

To create a cluster run:
```bash
./hack/build.sh
./bin/openshift-install create manifests
cat <<EOF > manifests/manifest_feature_gate.yaml
---
apiVersion: config.openshift.io/v1
kind: FeatureGate
metadata:
  annotations:
    include.release.openshift.io/self-managed-high-availability: "true"
    include.release.openshift.io/single-node-developer: "true"
    release.openshift.io/create-only: "true"
  name: cluster
spec:
  customNoUpgrade:
    enabled:
    - ExternalCloudProvider
    - CSIMigrationAWS
    - CSIMigrationOpenStack
  featureSet: CustomNoUpgrade
EOF

OPENSHIFT_INSTALL_RELEASE_IMAGE_OVERRIDE=quay.io/dgrigore/release:cccmo-render ./bin/openshift-install create cluster
```
  • Loading branch information
Danil-Grigorev committed May 20, 2021
1 parent 65650b2 commit dd6b99b
Showing 1 changed file with 31 additions and 0 deletions.
31 changes: 31 additions & 0 deletions data/data/bootstrap/files/usr/local/bin/bootkube.sh.template
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ CLUSTER_ETCD_OPERATOR_IMAGE=$(image_for cluster-etcd-operator)
CONFIG_OPERATOR_IMAGE=$(image_for cluster-config-operator)
KUBE_APISERVER_OPERATOR_IMAGE=$(image_for cluster-kube-apiserver-operator)
KUBE_CONTROLLER_MANAGER_OPERATOR_IMAGE=$(image_for cluster-kube-controller-manager-operator)
CLUSTER_CLOUD_CONTROLLER_MANAGER_OPERATOR_IMAGE=$(image_for cluster-cloud-controller-manager-operator)
KUBE_SCHEDULER_OPERATOR_IMAGE=$(image_for cluster-kube-scheduler-operator)
INGRESS_OPERATOR_IMAGE=$(image_for cluster-ingress-operator)

Expand Down Expand Up @@ -230,6 +231,36 @@ then
record_service_stage_success
fi

if [ ! -f cloud-controller-manager-bootstrap.done ]
then
record_service_stage_start "cluster-cloud-controller-manager-bootstrap"
echo "Rendering Cloud Controller Manager core manifests..."

rm --recursive --force cloud-controller-manager-bootstrap

# Copy the CCCMO images configMap to resolve pod images from internal registry in CCCMO render run
# Source config map is located in https://github.com/openshift/cluster-cloud-controller-manager-operator/blob/master/manifests/0000_26_cloud-controller-manager-operator_01_images.configmap.yaml
bootkube_podman_run \
--name copy-cloud-controller-manager-images \
--rm -it \
--entrypoint /bin/bash "${RELEASE_IMAGE_DIGEST}" \
-c "cat /release-manifests/0000_26_cloud-controller-manager-operator_01_images.configmap.yaml" > manifests/ccm-images.yaml

bootkube_podman_run \
--name cloud-controller-render \
--volume "$PWD:/assets:z" \
"${CLUSTER_CLOUD_CONTROLLER_MANAGER_OPERATOR_IMAGE}" \
/render run \
--images-file=/assets/manifests/ccm-images.yaml \
--dest-dir=/assets/cloud-controller-manager-bootstrap \
--cluster-infrastructure-file=/assets/manifests/cluster-infrastructure-02-config.yml

cp cloud-controller-manager-bootstrap/bootstrap/* bootstrap-manifests//

touch cloud-controller-manager-bootstrap.done
record_service_stage_success
fi

if [ ! -f kube-scheduler-bootstrap.done ]
then
record_service_stage_start "kube-scheduler-bootstrap"
Expand Down

0 comments on commit dd6b99b

Please sign in to comment.