feat: require and then remove karpenter startup taint

[rschalo]

Fixes #1049

Description
Based on implementation discussion, it is required that userdata scripts for starting kubelet include karpenter.sh/unregistered=true:NoExecute as part of the --register-with-taints flag.

How was this change tested?
Unit testing and local aws/karpenter-provider-aws deployment. This will be ship with aws/karpenter-provider-aws#6388

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[k8s-ci-robot]

Hi @rschalo. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[jonathan-innis]

/reopen

[coveralls]

Pull Request Test Coverage Report for Build 9764619347

Details

• 19 of 23 (82.61%) changed or added relevant lines in 4 files are covered.
• 6 unchanged lines in 3 files lost coverage.
• Overall coverage increased (+0.05%) to 78.664%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
pkg/controllers/nodeclaim/lifecycle/registration.go	9	13	69.23%

Files with Coverage Reduction	New Missed Lines	%
pkg/controllers/nodeclaim/lifecycle/registration.go	1	82.54%
pkg/test/cachesyncingclient.go	2	82.29%
pkg/controllers/nodeclaim/disruption/controller.go	3	54.17%

Totals
Change from base Build 9754782073:	0.05%
Covered Lines:	8598
Relevant Lines:	10930

---

💛 - Coveralls

[coveralls]

Pull Request Test Coverage Report for Build 9766354735

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

• For more information on this, see Tracking coverage changes with pull request builds.
• To avoid this issue with future PRs, see these Recommended CI Configurations.
• For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

• 16 of 20 (80.0%) changed or added relevant lines in 4 files are covered.
• 38 unchanged lines in 4 files lost coverage.
• Overall coverage increased (+0.07%) to 78.68%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
pkg/controllers/nodeclaim/lifecycle/registration.go	8	12	66.67%

Files with Coverage Reduction	New Missed Lines	%
pkg/controllers/nodeclaim/lifecycle/registration.go	1	82.26%
pkg/test/cachesyncingclient.go	2	82.29%
pkg/controllers/nodeclaim/disruption/controller.go	3	54.17%
pkg/controllers/provisioning/provisioner.go	32	79.25%

Totals
Change from base Build 9754782073:	0.07%
Covered Lines:	8606
Relevant Lines:	10938

---

💛 - Coveralls

[jmdeal]

/lgtm

[coveralls]

Pull Request Test Coverage Report for Build 9785861189

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

• For more information on this, see Tracking coverage changes with pull request builds.
• To avoid this issue with future PRs, see these Recommended CI Configurations.
• For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

• 17 of 19 (89.47%) changed or added relevant lines in 4 files are covered.
• 100 unchanged lines in 9 files lost coverage.
• Overall coverage increased (+0.1%) to 78.754%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
pkg/controllers/nodeclaim/lifecycle/registration.go	11	13	84.62%

Files with Coverage Reduction	New Missed Lines	%
pkg/utils/disruption/disruption.go	1	93.75%
pkg/controllers/nodeclaim/lifecycle/registration.go	1	85.71%
pkg/test/expectations/expectations.go	2	93.7%
pkg/scheduling/requirements.go	2	98.01%
pkg/controllers/metrics/pod/controller.go	15	77.42%
pkg/apis/v1/nodepool.go	15	67.86%
pkg/controllers/nodepool/counter/controller.go	15	62.07%
pkg/controllers/nodeclaim/termination/controller.go	17	65.0%
pkg/controllers/provisioning/provisioner.go	32	79.25%

Totals
Change from base Build 9754782073:	0.1%
Covered Lines:	8611
Relevant Lines:	10934

---

💛 - Coveralls

[njtran]

/lgtm
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: njtran, rschalo

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [njtran]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment