⚠⚠⚠ THIS IS A LIVING DOCUMENT AND LIKELY TO CHANGE QUICKLY ⚠⚠⚠
- Cluster and OS pre-requisites
- Install operator-sdk v1.15.0
- The operator is written using operator-sdk v1.15.0 and has the same pre-requisites as it does.
- Git submodules need to be initialized and up to date before running the build command. Please refer to updating Git submodules.
- Verify push/pull access to a container registry from where the cluster can pull image from.
- Instructions to install the community operator can be found in the README. Customers with a Red Hat subscription should instead look at the OpenShift docs.
- Build the operator
- Generate bundle manifests
- Build the bundle image
- Build an operator index
- Create a CatalogSource using the index
- Follow the normal procedure to deploy an operator
To build and push the operator image, execute:
export OPERATOR_IMAGE=quay.io/<insert username>/wmco:$VERSION_TAG
hack/olm.sh build
Building the full WMCO image is a time consuming process given we are building all the submodules. If you are just changing WMCO and not the submodules, you can separately build the submodules as a base image and then just build the WMCO image from this base image:
make base-img
make wmco-img IMG=$OPERATOR_IMAGE
make base-img
will not push the base image as a push is not needed. make wmco-img
will make and push the
$OPERATOR_IMAGE
With the Operator image built and pushed to a remote repository, the operator can be deployed either directly, or through OLM.
For each deployment method a kubeconfig for an OpenShift or OKD cluster must be exported
export KUBECONFIG=<path to kubeconfig>
make deploy IMG=$OPERATOR_IMAGE
The above command will directly deploy all the resources required for the operator to run, as well as the WMCO pod. To remove the installed resources:
make undeploy
hack/olm.sh run -k "<PRIVATE_KEY.PEM>"
Where "<PRIVATE_KEY.PEM>"
is the private key which will be used to configure Windows nodes.
This command builds the operator image, pushes it to remote repository and uses OLM to launch the operator. Executing
the Build step is not required.
In order to build the operator ignoring the existing build image cache, run the above command with the -i
option.
To clean-up the installation, use:
hack/olm.sh cleanup
The following environment variables must be set:
# SSH key to be used for creation of cloud-private-key secret
export KUBE_SSH_KEY_PATH=<path to RSA type ssh key>
export OPERATOR_IMAGE=quay.io/<insert username>/wmco:$VERSION_TAG
export OPENSHIFT_CI=false
# on AWS only:
export AWS_SHARED_CREDENTIALS_FILE=<path to aws credentials file>
Once the above variables are set, run the following script:
hack/run-ci-e2e-test.sh
Additional flags that can be passed to hack/run-ci-e2e-test.sh
are
-s
to skip the deletion of Windows nodes that are created as part of test suite run-m
to represent the number of Windows instances to be configured by the Windows Machine controller-c
to represent the number of Windows instances to be configured by the ConfigMap controller-b
gives an alternative path to the WMCO binary. This option overridden in OpenShift CI. When building the operator locally, the WMCO binary is created as part of the operator image build process and can be found atbuild/_output/bin/windows-machine-config-operator
, this is the default value of this option.-t
specify the test to run. All tests are run if the option is not used. The allowed options are:all
all the tests are runbasic
creation, network and deletion tests are runupgrade
creation, upgrade, reconfiguration and deletion tests are run
Example command to run the full test suite with 2 instances configured by the Windows Machine controller, and 1 configured by the ConfigMap controller, skipping the deletion of all the nodes.
hack/run-ci-e2e-test.sh -s -m 2 -c 1
Please note that you do not need to run hack/olm.sh run
before hack/run-ci-e2e-test.sh
.
To run the WMCO e2e tests on a bare metal or other platform-agnostic infrastructure (platform=none), where there
is no cloud provider specification, the desired number of Windows instances must be provisioned beforehand, and the
instance(s) information must be provided to the e2e test suite through the WINDOWS_INSTANCES_DATA
environment
variable.
To deploy machines using the infrastructure providers supported by WMCO, refer to the specific provider documentation. See Windows instance pre-requisites.
The information you need to collect from each Windows instance is:
- the internal IP address
- the Windows administrator username
Export WINDOWS_INSTANCES_DATA
as an environment variable with the corresponding windows-instances ConfigMap
data section, and a new windows-instances
ConfigMap will be created during the execution of the e2e test suite.
For example:
export WINDOWS_INSTANCES_DATA='
data:
10.1.42.1: |-
username=Administrator
'
where 10.1.42.1
is the IP address and Administrator
is the Windows' administrator username of the Windows instance.
After the WINDOWS_INSTANCES_DATA
environment variable is set and exported you can run:
hack/run-ci-e2e-test.sh
This directory contains resources related to installing the WMCO onto a cluster using OLM.
This step should be done in the case that changes have been made to any of the RBAC requirements, or if any changes were made to the files in config/.
If the operator version needs to be changed, the WMCO_VERSION
variable at the top of the repo's Makefile should
be set to the new version.
If the bundle channel needs to be changed, the CHANNELS
and/or DEFAULT_CHANNELS
variable within the Makefile
should be changed to the desired values.
New bundle manifests can then be generated with:
make bundle
Within the bundle manifests the :latest
tag must be manually removed in order for the e2e tests to work.
If you plan on building a bundle image using these bundle manifests, the image should be set to reflect where the WMCO was pushed to in the build step
make bundle IMG=$OPERATOR_IMAGE
You can skip this step if you want to run the operator for developer testing purposes only
Once the manifests have been generated properly, you can run the following command in the root of this git repository:
make bundle-build BUNDLE_IMG=<BUNDLE_REPOSITORY:$BUNDLE_TAG>
The variables in the command should be changed to match the container image repository you wish to store the bundle in. You can also change the channels based on the release status of the operator. This command should create a new bundle image. Bundle image and operator image are two different images.
You should then push the newly created bundle image to the remote repository:
podman push $BUNDLE_REPOSITORY:$BUNDLE_TAG
You should verify that the new bundle is valid:
operator-sdk bundle validate $BUNDLE_REPOSITORY:$BUNDLE_TAG --image-builder podman
opm has been installed on the build system. All previous pre-requisites must be satisfied as well.
You can skip this step if you want to run the operator for developer testing purposes only
An operator index is a collection of bundles. Creating one is required if you wish to deploy your operator on your own cluster.
opm index add --bundles $BUNDLE_REPOSITORY:$BUNDLE_TAG --tag $INDEX_REPOSITORY:$INDEX_TAG --container-tool podman
You should then push the newly created index image to the remote repository:
podman push $INDEX_REPOSITORY:$INDEX_TAG
An existing operator index can have bundles added to it:
opm index add --from-index $INDEX_REPOSITORY:$INDEX_TAG
and removed from it:
opm index rm --from-index $INDEX_REPOSITORY:$INDEX_TAG
This project contains git submodules for the following components:
- windows-machine-config-bootstrapper
- kubernetes
- ovn-kubernetes
- containernetworking-plugins
- promu
- windows_exporter
To update git submodules you can use the script hack/update_submodules.sh Use the help command for up to date instructions:
hack/update_submodules.sh -h
Alternatively, it can be done manually via:
# To update all git submodules
git submodule update --recursive
# To update a specific submodule
git submodule update --remote <path_to_submodule>