[OCPCLOUD-1209] Add feature gate to bootstrap mode manifests

[Danil-Grigorev]

- What I did

Added feature gate into list of supported manifests to render during install time. Additionally makes sure openshift only feature gates are excluded from Kubelet config list too, so it won't be confusing when the feature gate is present in Kubelet config, but Kubelet is not running it.

- How to verify it

1. openshift-install create manifests
2. Put featureGate to manifests folder
3. run openshift-install create cluster.

- Description for the changelog

[JoelSpeed]

Looks good, thanks Danil, couple of suggestions, have you spun up a cluster with this yet?

[elmiko]

i gave this pr a test drive in conjunction with using installer patch openshift/installer#4947 , i created the manifests, added the external feature gate and created the cluster.

the cluster is mostly healthy, but the mco is still having trouble with the rendered configurations. i see this output from the installer:

INFO Cluster operator machine-config Progressing is True with : Working towards 4.9.0-0.ci-2021-06-30-105117 
ERROR Cluster operator machine-config Degraded is True with RequiredPoolsFailed: Unable to apply 4.9.0-0.ci-2021-06-30-105117: timed out waiting for the condition during syncRequiredMachineConfigPools: error pool master is not ready, retrying. Status: (pool degraded: true total: 3, ready 0, updated: 0, unavailable: 3) 
INFO Cluster operator machine-config Available is False with : Cluster not available for 4.9.0-0.ci-2021-06-30-105117 
INFO Cluster operator network ManagementStateDegraded is False with :  
ERROR Cluster initialization failed because one or more operators are not functioning properly. 
ERROR The cluster should be accessible for troubleshooting as detailed in the documentation linked below, 
ERROR https://docs.openshift.com/container-platform/latest/support/troubleshooting/troubleshooting-installations.html 
ERROR The 'wait-for install-complete' subcommand can then be used to continue the installation 
FATAL failed to initialize the cluster: Cluster operator machine-config is not available 

when i examine the master machineconfigpool, i see this in the status.conditions:

  - lastTransitionTime: "2021-07-01T17:55:20Z"
    message: 'Node ip-10-0-128-167.us-east-2.compute.internal is reporting: "machineconfig.machineconfiguration.openshift.io
      \"rendered-master-1e07fda5bdcef19ac1e002aab8784c7e\" not found", Node ip-10-0-189-128.us-east-2.compute.internal
      is reporting: "machineconfig.machineconfiguration.openshift.io \"rendered-master-1e07fda5bdcef19ac1e002aab8784c7e\"
      not found", Node ip-10-0-214-72.us-east-2.compute.internal is reporting: "machineconfig.machineconfiguration.openshift.io
      \"rendered-master-1e07fda5bdcef19ac1e002aab8784c7e\" not found"'
    reason: 3 nodes are reporting degraded status on sync
    status: "True"
    type: NodeDegraded

[JoelSpeed]

I have tested this on its own as well and can see that it resolved the issue of the kubelet flags differing, but we are still seeing differences in the kubelet config (jsons available if you would like to see). I think if we try this in conjunction with #2547 we should hopefully see this issue is resolved completely.

IMO, this PR is doing the job it was intended to do as per my conversations with @yuqi-zhang on slack

/lgtm

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Danil-Grigorev, JoelSpeed
To complete the pull request process, please assign kikisdeliveryservice after the PR has been reviewed.
You can assign the PR to them by writing /assign @kikisdeliveryservice in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

New changes are detected. LGTM label has been removed.

[JoelSpeed]

@Danil-Grigorev Can you update this PR to add an exception for the lint rule that's blocking this from passing the verify stage? Qi is doing it for the same place here, just need to copy this line over https://github.com/openshift/machine-config-operator/pull/2547/files#diff-ee4889f36bd8b3bb13f51dd15721cbc3f0eae1cf4d6bb8635dec6de843e78c4dR50

[openshift-ci]

@Danil-Grigorev: The following tests failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
ci/prow/okd-e2e-aws	afce61e	link	/test okd-e2e-aws
ci/prow/e2e-aws-disruptive	afce61e	link	/test e2e-aws-disruptive
ci/prow/e2e-aws-workers-rhel7	afce61e	link	/test e2e-aws-workers-rhel7
ci/prow/e2e-metal-ipi	afce61e	link	/test e2e-metal-ipi
ci/prow/e2e-ovn-step-registry	afce61e	link	/test e2e-ovn-step-registry

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[openshift-ci]

@Danil-Grigorev: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[JoelSpeed]

/close

This was included in #2668

[openshift-ci]

@JoelSpeed: Closed this PR.

In response to this:

/close

This was included in #2668

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.