These APIs allows for adding arbitrary manifests to the set generated by openshift-install create manifests
.
A typical use case would be to create a MachineConfig which would make some persistent customization to a group of nodes.
These will only take effect after the machine config operator is up and running so if a change is needed before or during the installation process, another API will be required.
Formats json, yaml, and multi-document yaml are accepted as manifests. Multi-document yaml manifests are split with unique names to avoid any collisions with existing files when they are added to their respective folders (either "manifests" or "openshift").
Yaml patches are also accepted and should be in the yaml-patch format, patches should end with .yml.patch
, yaml.patch
.
Extensions may have a suffix of an underscore followed by some alphanumeric characters, for example .yml.patch_ocp_manifests
or .yaml.patch_ocp_manifests
Here is an example of the format of a Yaml patch file, for more information on the format, please see https://github.com/krishicks/yaml-patch
---
- op: replace
path: /status/infrastructureTopology
value: HighlyAvailable
Note: In environments where TLS is enabled (e.g. api.openshift.com), you will need to use the https
scheme in URLs.
# base64 encoding of the example from https://docs.openshift.com/container-platform/4.6/installing/install_config/installing-customizing.html
content=YXBpVmVyc2lvbjogbWFjaGluZWNvbmZpZ3VyYXRpb24ub3BlbnNoaWZ0LmlvL3YxCmtpbmQ6IE1hY2hpbmVDb25maWcKbWV0YWRhdGE6CiAgbGFiZWxzOgogICAgbWFjaGluZWNvbmZpZ3VyYXRpb24ub3BlbnNoaWZ0LmlvL3JvbGU6IG1hc3RlcgogIG5hbWU6IDk5LW9wZW5zaGlmdC1tYWNoaW5lY29uZmlnLW1hc3Rlci1rYXJncwpzcGVjOgogIGtlcm5lbEFyZ3VtZW50czoKICAgIC0gJ2xvZ2xldmVsPTcnCg==
file=99-openshift-machineconfig-master-kargs.yaml
folder=openshift
curl \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $TOKEN" \
--request POST \
--data "{\"file_name\":\"$file\", \"folder\":\"$folder\", \"content\":\"$content\"}" \
"http://$ASSISTED_SERVICE_IP:$ASSISTED_SERVICE_PORT/api/assisted-install/v2/clusters/$CLUSTER_ID/manifests"
file=99-openshift-machineconfig-master-kargs.yaml
folder=openshift
curl --header "Authorization: Bearer $TOKEN" "http://$ASSISTED_SERVICE_IP:$ASSISTED_SERVICE_PORT/api/assisted-install/v2/clusters/$CLUSTER_ID/manifests/files?file_name=$file&folder=$folder"
Changes to operating systems on OCP nodes can be done by creating what are referred to as MachineConfig objects that are managed by the Machine Config Operator (MCO).
The MachineConfig specification includes an ignition config for configuring the machines at first boot. This config object can be used to do things like modify files, systemd services, and other operating system features running on on OCP machines.
Specifically, the ignition config can be used to configure storage on the nodes (see ignition Config Spec for full details).
The following MachineSet manifest example demonstrates how to add a partition to sda
device on master nodes. I.e. apply the manifest before installation to have a partition named recovery
of size 16 GiB on the master nodes.
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
labels:
machineconfiguration.openshift.io/role: master
name: 10-masters-storage-config
spec:
config:
ignition:
version: 3.2.0
storage:
disks:
- device: /dev/sda
partitions:
- label: recovery
startMiB: 32768
sizeMiB: 16384
filesystems:
- device: /dev/disk/by-partlabel/recovery
label: recovery
format: xfs
The discovery ignition is used to make changes to the CoreOS live iso image which runs before we actually write anything to the target disk. An example use case would be configuring a separate container registry to pull the assisted-installer-agent image from.
The discovery ignition must use version 3.1.0 regardless of the version of the cluster that will eventually be created.
# ignition patch file
{
"ignition_config_override": "{\"ignition\": {\"version\": \"3.1.0\"}, \"storage\": {\"files\": [{\"path\": \"/etc/containers/registries.conf\", \"mode\": 420, \"overwrite\": true, \"user\": { \"name\": \"root\"},\"contents\": {\"source\": \"data:text/plain;base64,dW5xdWFsaWZpZWQtc2VhcmNoLXJlZ2lzdHJpZXMgPSBbInJlZ2lzdHJ5LmFjY2Vzcy5yZWRoYXQuY29tIiwgImRvY2tlci5pbyJdCltbcmVnaXN0cnldXQogICBwcmVmaXggPSAiIgogICBsb2NhdGlvbiA9ICJxdWF5LmlvL29jcG1ldGFsIgogICBtaXJyb3ItYnktZGlnZXN0LW9ubHkgPSBmYWxzZQogICBbW3JlZ2lzdHJ5Lm1pcnJvcl1dCiAgIGxvY2F0aW9uID0gImxvY2FsLnJlZ2lzdHJ5OjUwMDAvb2NwbWV0YWwiCg==\"}}]}}"
}
curl \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $TOKEN" \
--request PATCH \
--data @discovery-ign.json \
"http://$ASSISTED_SERVICE_IP:$ASSISTED_SERVICE_PORT/api/assisted-install/v2/infra-envs/$INFRA_ENV_ID"
curl --header "Authorization: Bearer $TOKEN" "http://$ASSISTED_SERVICE_IP:$ASSISTED_SERVICE_PORT/api/assisted-install/v2/infra-envs/$INFRA_ENV_ID/downloads/'files?file_name=discovery.ign"
These endpoints alter the default install config yaml used when running openshift-install create
commands.
Install config customization reference is available here
Some of this content will be pinned to particular values, but most can be edited.
Note, some of this content will be pinned to particular values by the assisted-installer (can't be overwritten).
You can compose the install-config overrides by creating a json string with the options you wish to set. An example install config with disabled hyperthreading for the control plane:
apiVersion: extensions.hive.openshift.io/v1beta1
baseDomain: example.com
controlPlane:
name: master
hyperthreading: Disabled
compute:
- name: worker
replicas: 5
metadata:
name: test-cluster
platform: ...
pullSecret: '{"auths": ...}'
sshKey: ssh-ed25519 AAAA...
should look like this:
"{\"controlPlane\":{\"hyperthreading\":\"Disabled\"}}"
curl \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $TOKEN" \
--request PATCH \
--data '"{\"controlPlane\":{\"hyperthreading\":\"Disabled\"}}"' \
"http://$ASSISTED_SERVICE_IP:$ASSISTED_SERVICE_PORT/api/assisted-install/v2/clusters/$CLUSTER_ID/install-config"
curl --header "Authorization: Bearer $TOKEN" "http://$ASSISTED_SERVICE_IP:$ASSISTED_SERVICE_PORT/api/assisted-install/v2/clusters/$CLUSTER_ID/install-config"
Since OpenShift 4.12, it is possible to customize the install config to disable some components.
The features (baremetal-operator
, marketplace
, Console
, Insights
, Storage
, CSISnapshot
and openshift-samples-content
) can be disabled by setting the baselineCapabilitySet
and addittionalEnabledCapabilities
parameters in the install-config.yaml
.
Here is an example of disabling all capabilities except openshift-samples:
capabilities:
baselineCapabilitySet: None
additionalEnabledCapabilities:
- openshift-samples
See https://docs.openshift.com/container-platform/latest/installing/cluster-capabilities.html for more information.
Important
Note that disabling capabilities could cause problems in some situations, e.g. disabling the baremetal-operator on a baremetal platform cluster will cause installation to fail.
As with the parent example, install config can be patched with:
curl \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $TOKEN" \
--request PATCH \
--data '"{\"capabilities\": {\"baselineCapabilitySet\": \"None\", \"additionalEnabledCapabilities\": [\"openshift-samples\"]}"' \
"http://$ASSISTED_SERVICE_IP:$ASSISTED_SERVICE_PORT/api/assisted-install/v2/clusters/$CLUSTER_ID/install-config"
and the configuration verified via:
curl --header "Authorization: Bearer $TOKEN" "http://$ASSISTED_SERVICE_IP:$ASSISTED_SERVICE_PORT/api/assisted-install/v2/clusters/$CLUSTER_ID/install-config"
The pointer ignition is used to customize the particular host when it reboots into the installed system. These changes will be made to the host, but will not be tracked as a machine config. Generally a machine config will be a better fit than editing the pointer ignition, but if some change is specific to an individual host it would be made here.
The pointer ignition override version must match the version of the ignition generated by openshift-installer. This means that the version required for the override will change depending on the cluster version being installed.
OCP Version(s) | Ignition version |
---|---|
4.6 | 3.1.0 |
4.7, 4.8, 4.9, 4.10 | 3.2.0 |
curl \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $TOKEN" \
--request PATCH \
--data '{"config": "{\"ignition\": {\"version\": \"3.1.0\"}, \"storage\": {\"files\": [{\"path\": \"/etc/example\", \"contents\": {\"source\": \"data:text/plain;base64,SGVsbG8gZnJvbSBob3N0IDg0Njk2NzdiLThlZGEtNDQzOS1iNDQwLTc3ZGM5M2FkZmNlZgo=\"}}]}}"}' \
"http://$ASSISTED_SERVICE_IP:$ASSISTED_SERVICE_PORT/api/assisted-install/v2/infra-envs/$INFRA_ENV_ID/hosts/$HOST_ID/ignition"
curl --header "Authorization: Bearer $TOKEN" "http://$ASSISTED_SERVICE_IP:$ASSISTED_SERVICE_PORT/api/assisted-install/v2/infra-envs/$INFRA_ENV_ID/$HOST_ID/ignition
This endpoint sets parameters to be passed to the coreos-installer command line in addition to the ones we provide by default. The only parameters that can be set with this endpoint are: ["--append-karg", "--delete-karg", "-n", "--copy-network", "--network-dir", "--save-partlabel", "--save-partindex", "--image-url"]
curl \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $TOKEN" \
--request PATCH \
--data '{"args": ["--append-karg", "nameserver=8.8.8.8", "-n"]}' \
"http://$ASSISTED_SERVICE_IP:$ASSISTED_SERVICE_PORT/api/assisted-install/v2/infra-envs/$INFRA_ENV_ID/hosts/$HOST_ID/installer-args"
When you need the cluster nodes / installation process to trust custom certificates, you can add them using the Assisted infra-env PATCH API's. The additional trust bundle can contain one or more CA certificates in the PEM format, for example:
-----BEGIN CERTIFICATE-----
...base-64-encoded
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
...base-64-encoded
-----END CERTIFICATE-----
Following is an example of using the API. Begin by storing the certificates in
a file called ca.pem
, set TOKEN
, ASSISTED_SERVICE_IP
,
ASSISTED_SERVICE_PORT
and INFRA_ENV_ID
to their appropriate values and run:
curl \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $TOKEN" \
--request PATCH \
--data "$( jq -n --arg BUNDLE "$(cat ca.pem)" '{ "additional_trust_bundle": $BUNDLE }' )" \
"http://$ASSISTED_SERVICE_IP:$ASSISTED_SERVICE_PORT/api/assisted-install/v2/infra-envs/$INFRA_ENV_ID"
Update the InfraEnv resource to modify the kernel arguments used in the live ISO (during the host discovery phase) . Currently, only the append (additional argument) operation is supported.
Note: additional parameters that are needed for InfraEnv creation are omitted from this example.
curl -X 'POST' \
'http://api.openshift.com/api/assisted-install/v2/infra-envs' \
-H 'accept: application/json' \
-H 'Content-Type: application/json' \
-d '{
"kernel_arguments": [
{
"operation": "append",
"value": "audit=0"
},
{
"operation": "append",
"value": "trace=1"
}
]
}'
curl -X 'PATCH' \
'http://api.openshift.com/api/assisted-install/v2/infra-envs/3fa85f64-5717-4562-b3fc-2c963f66afa6' \
-H 'accept: application/json' \
-H 'Content-Type: application/json' \
-d '{
"kernel_arguments": [
{
"operation": "append",
"value": "audit=0"
},
{
"operation": "append",
"value": "trace=1"
}
]
}'
When updating kernel arguments using this API call, all the kernel arguments are replaced. So the way to clear the kernel arguments is just to provide empty list:
curl -X 'PATCH' \
'http://api.openshift.com/api/assisted-install/v2/infra-envs/3fa85f64-5717-4562-b3fc-2c963f66afa6' \
-H 'accept: application/json' \
-H 'Content-Type: application/json' \
-d '{
"kernel_arguments": []
}'
When using the Kubernetes API, the kernel arguments are set in the InfraEnv spec section:
apiVersion: agent-install.openshift.io/v1beta1
kind: InfraEnv
metadata:
name: myinfraenv
namespace: spoke-cluster
spec:
kernelArguments:
- operation: append
value: audit=0
- operation: append
value: trace=1
clusterRef:
name: single-node
namespace: spoke-cluster
pullSecretRef:
name: pull-secret