✨ Add NodeRegistrationOptions.ImagePullPolicy Support

[akshay196]

What this PR does / why we need it:
Add support for ImagePullPolicy in KCP and CABPK.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #6679

[k8s-ci-robot]

Welcome @akshay196!

It looks like this is your first PR to kubernetes-sigs/cluster-api 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/cluster-api has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

[k8s-ci-robot]

Hi @akshay196. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[akshay196]

Hi team,
I am facing some issue testing this PR locally and would like get some suggestions.

1. I had setup cluster-api up locally using Tilt. New CRD is installed properly, but after I apply a KubeadmControlPlane resource with ImagePullPolicy, somehow it always reverts to IfNotPresent even if I set it something else. Here is an example:

$ kubectl get crd kubeadmcontrolplanes.controlplane.cluster.x-k8s.io -o yaml | grep imagePullPolicy -A10
--
                          imagePullPolicy:
                            default: IfNotPresent
                            description: ImagePullPolicy specifies the policy for
                              image pulling during kubeadm "init" and "join" operations.
                              The value of this field must be one of "Always", "IfNotPresent"
                              or "Never". Defaults to "IfNotPresent".
                            enum:
                            - Always
                            - IfNotPresent
                            - Never
                            type: string
--
                          imagePullPolicy:
                            default: IfNotPresent
                            description: ImagePullPolicy specifies the policy for
                              image pulling during kubeadm "init" and "join" operations.
                              The value of this field must be one of "Always", "IfNotPresent"
                              or "Never". Defaults to "IfNotPresent".
                            enum:
                            - Always
                            - IfNotPresent
                            - Never
                            type: string

$ cat kubeadm-controlplane.yaml
kind: KubeadmControlPlane
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
metadata:
  name: "demo-control-plane"
spec:
  replicas: 1
  machineTemplate:
    infrastructureRef:
      kind: GCPMachineTemplate
      apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
      name: "demo-control-plane"
  kubeadmConfigSpec:
    initConfiguration:
      nodeRegistration:
        imagePullPolicy: Always                                    <-------------------- Set to this
        name: '{{ ds.meta_data.local_hostname.split(".")[0] }}'
        kubeletExtraArgs:
          cloud-provider: gce
    clusterConfiguration:
      apiServer:
        timeoutForControlPlane: 20m
        extraArgs:
          cloud-provider: gce
      controllerManager:
        extraArgs:
          cloud-provider: gce
          allocate-node-cidrs: "false"
    joinConfiguration:
      nodeRegistration:
        imagePullPolicy: Always                               <---------------- Set to this
        name: '{{ ds.meta_data.local_hostname.split(".")[0] }}'
        kubeletExtraArgs:
          cloud-provider: gce
  version: "v1.23.0"

$ kubectl apply -f kubeadm-controlplane.yaml
kubeadmcontrolplane.controlplane.cluster.x-k8s.io/demo-control-plane created

$ kubectl get kubeadmcontrolplane demo-control-plane -o yaml
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
kind: KubeadmControlPlane
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"controlplane.cluster.x-k8s.io/v1beta1","kind":"KubeadmControlPlane","metadata":{"annotations":{},"name":"demo-control-plane","namespace":"default"},"spec":{"kubeadmConfigSpec":{"clusterConfiguration":{"apiServer":{"extraArgs":{"cloud-provider":"gce"},"timeoutForControlPlane":"20m"},"controllerManager":{"extraArgs":{"allocate-node-cidrs":"false","cloud-provider":"gce"}}},"initConfiguration":{"nodeRegistration":{"imagePullPolicy":"Always","kubeletExtraArgs":{"cloud-provider":"gce"},"name":"{{ ds.meta_data.local_hostname.split(\".\")[0] }}"}},"joinConfiguration":{"nodeRegistration":{"imagePullPolicy":"Always","kubeletExtraArgs":{"cloud-provider":"gce"},"name":"{{ ds.meta_data.local_hostname.split(\".\")[0] }}"}}},"machineTemplate":{"infrastructureRef":{"apiVersion":"infrastructure.cluster.x-k8s.io/v1beta1","kind":"GCPMachineTemplate","name":"demo-control-plane"}},"replicas":1,"version":"v1.23.0"}}
  creationTimestamp: "2022-12-17T17:57:16Z"
  generation: 1
  name: demo-control-plane
  namespace: default
  resourceVersion: "88831"
  uid: ffb656da-d89f-4f9a-9401-8e991632806d
spec:
  kubeadmConfigSpec:
    clusterConfiguration:
      apiServer:
        extraArgs:
          cloud-provider: gce
        timeoutForControlPlane: 20m0s
      controllerManager:
        extraArgs:
          allocate-node-cidrs: "false"
          cloud-provider: gce
      dns: {}
      etcd: {}
      networking: {}
      scheduler: {}
    format: cloud-config
    initConfiguration:
      localAPIEndpoint: {}
      nodeRegistration:
        imagePullPolicy: IfNotPresent                 <------------- This has automatically changed
        kubeletExtraArgs:
          cloud-provider: gce
        name: '{{ ds.meta_data.local_hostname.split(".")[0] }}'
    joinConfiguration:
      discovery: {}
      nodeRegistration:
        imagePullPolicy: IfNotPresent                   <------------- This has automatically changed
        kubeletExtraArgs:
          cloud-provider: gce
        name: '{{ ds.meta_data.local_hostname.split(".")[0] }}'
  machineTemplate:
    infrastructureRef:
      apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
      kind: GCPMachineTemplate
      name: demo-control-plane
      namespace: default
    metadata: {}
  replicas: 1
  rolloutStrategy:
    rollingUpdate:
      maxSurge: 1
    type: RollingUpdate
  version: v1.23.0

2. Any suggestions on how to verify ImagePullPolicy applied successfully for the cluster provisioned?

[sbueringer]

/ok-to-test

I'll take a look

[sbueringer]

@akshay196

Remaining issues should be:

• ✨ Add NodeRegistrationOptions.ImagePullPolicy Support #7772 (comment)
• running generate conversions again and then adding some convert funcs in our conversion.go files
  • Something like this:

    cluster-api/bootstrap/kubeadm/api/v1alpha3/conversion.go

    Line 274 in f6b822a

    func Convert_v1beta1_User_To_v1alpha3_User(in *bootstrapv1.User, out *User, s apiconversion.Scope) error {

    (for all the ones that conversion gen will drop from zz_generated.conversion.go)

[sbueringer]

@akshay196 Just fyi. That should be the second part that I described now. (you can generate the conversion with make generate-go-conversions)

[akshay196]

That should be the second part that I described now. (you can generate the conversion with make generate-go-conversions)

I executed make generate-go-conversions that drop generated convert funcs from v1alpha4 and upstreamv1beta2. Then I manually added convert funcs to conversion.go and again executed make generate-go-convertsions
Is this correct?

[sbueringer]

Yup sounds correct. I never remember if the final make generate-go-conversions is necessary or not.
But looks like the verify job is happy now. So the generated files are okay.

Now we have to take a look at the unit tests. Seems there is an issue with the conversion somewhere (I"ll take a look)

(btw more context here: https://hackmd.io/aYLvrb_jTXiwg_MnpD804w)

[akshay196]

Yup sounds correct. I never remember if the final make generate-go-conversions is necessary or not.

The final make generate-go-conversions modified AddGeneratedConversionFunc to AddConversionFunc in zz_generated.conversion.go file.

Now we have to take a look at the unit tests. Seems there is an issue with the conversion somewhere (I"ll take a look)
(btw more context here: https://hackmd.io/aYLvrb_jTXiwg_MnpD804w)

Thanks for sharing that. I don't completely understand how conversion happening here. I took reference from PR #4905 and did corresponding changes. I may have made mistakes.
I will refer your link now.

[sbueringer]

@akshay196 No worries. It's unfortunately quite complex. I added some hints as a comment above. After those changes the tests should be green.

[sbueringer]

/test pull-cluster-api-e2e-full-main

[akshay196]

capi-e2e test is failing with this error:

Failed to get logs for Machine clusterclass-changes-b1puz1-8tc5h-md4df, Cluster clusterclass-changes-n51zp0/clusterclass-changes-b1puz1: error creating container exec: Error response from daemon: Container ffedbb0d2611970c9dddc4255f621d4a0a5c2050b0078fd674fdc756ccaab8d7 is not running
Failed to get logs for Machine clusterclass-changes-b1puz1-md-0-pbtfx-98d6fd966-5ks9r, Cluster clusterclass-changes-n51zp0/clusterclass-changes-b1puz1: error creating container exec: Error response from daemon: No such container: clusterclass-changes-b1puz1-md-0-pbtfx-98d6fd966-5ks9r

Is it some flaky test?

[sbueringer]

/retest

[sbueringer]

Probably. Let's see

[sbueringer]

Another flake :/

/retest

[sbueringer]

/lgtm

Let's wait for green e2e tests, but then it would be good if you can squash the commits

[k8s-ci-robot]

LGTM label has been added.

Git tree hash: 4251399300b8a52ca787800fe80774c6a0d4af93

[fabriziopandini]

one nit, otherwise lgtm for me pending squash

[akshay196]

PR rebased on main and all commits are squashed. 🚀

[sbueringer]

Thank you very much!!

/lgtm
/approve

[k8s-ci-robot]

LGTM label has been added.

Git tree hash: 671167c3b3dea77402808730f637f93891e02e66

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sbueringer

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [sbueringer]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[akshay196]

Thank you @sbueringer @fabriziopandini for reviewing.