New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bug 1876680: Reduce influence of webhooks and convert some errors to non-fatal #697

Merged

openshift-merge-robot merged 5 commits into openshift:master from JoelSpeed:webhook-warnings

Sep 24, 2020

Contributor

JoelSpeed commented Sep 9, 2020 •

edited

Loading

This should allow our validation/defaulting webhooks to respond with some of their validations as warnings rather than errors.

The webhooks have been provided opinionated defaults and requiring these in validation. This may cause issues for existing users and as such, we need to reel back on these changes and ensure that we only error when a value is missing that would cause a Machine to not be created successfully.

This PR should reduce the opinionation and ensure that we don't break any existing installations

openshift-ci-robot requested review from enxebre and michaelgugino

September 9, 2020 16:46

michaelgugino suggested changes

View reviewed changes

Contributor

michaelgugino left a comment

We should do a separate PR for k8s version bump.

For instance, openshift/client-go looks like it should be release-4.6 but we're using something much older.

pkg/apis/machine/v1beta1/machine_webhook.go

-              		return admission.Denied(err.Error())
+              	ok, warnings, errs := h.webhookOperations(m, h.clusterID)
+              	if !ok {
+              		return responseWithWarnings(admission.Denied(errs.Error()), warnings)

Contributor

michaelgugino Sep 9, 2020

This will result in a denial still?

Contributor Author

JoelSpeed Sep 9, 2020

Indeed, the intention here is that within the webhookOperations we should be able to define fatal and non-fatal errors.

Fatal errors are added to errs, non fatal errors are added to warnings.

If there are any fatal errors, we return and admission.Denied as we currently do for any error.

If there are no fatal errors, we return admission.Allowed as we currently do for no errors.

However, in both cases, we can also return non-fatal errors (warnings), which will allow the request to be allowed or denied as appropriate, but send those warnings back to the user even though it is accepted.

For a worked example, imagine everything is happy, as a user I try to set the disk size to 60gb, the validation logic thinks this should be 120gb or more so sets a warning, but there are no other errors, so it returns allowed with a warning disk size should be greater than 120gb, the request to update from the user was accepted, but kubectl shows them this warning.

pkg/apis/machine/v1beta1/machine_webhook.go

               	}
-              	return admission.Allowed("Machine valid")
+              	return responseWithWarnings(admission.Allowed("Machine valid"), warnings)

Contributor

michaelgugino Sep 9, 2020

This seems confusing to me. We're saying it's valid, but also there may be warnings?

Contributor

alexander-demicev Sep 10, 2020

+1

openshift-ci-robot added the needs-rebase label

JoelSpeed force-pushed the webhook-warnings branch from 3ae6a68 to 0a3ceb2 Compare

September 16, 2020 13:17

openshift-ci-robot removed the needs-rebase label

JoelSpeed changed the title ~~Allow admission controllers to respond with warnings~~ Reduce influence of webhooks and convert some errors to non-fatal

JoelSpeed commented

View reviewed changes

pkg/apis/machine/v1beta1/machine_webhook.go

Comment on lines +413 to +404

+              func responseWithWarnings(response admission.Response, warnings []string) admission.Response {
+              	response.AdmissionResponse.Warnings = warnings
+              	return response
+              }

Contributor Author

JoelSpeed Sep 16, 2020

I intend for this to be replaced by kubernetes-sigs/controller-runtime#1157 once it is merged

Member

enxebre Sep 16, 2020

may be add a TODO comment

Contributor Author

JoelSpeed Sep 21, 2020

Since it's merged, and to keep the scope of this PR reduced, I'm gonna raise a follow up PR that we can keep open until a point at which we are happy to merge it rather than adding the todo

enxebre reviewed

View reviewed changes

pkg/apis/machine/v1beta1/machine_webhook.go Outdated

-              	}
-              	if providerSpec.OSDisk.ManagedDisk.StorageAccountType == "" {
-              		errs = append(errs, field.Required(field.NewPath("providerSpec", "osDisk", "managedDisk", "storageAccountType"), "storageAccountType must be provided"))
+              	if providerSpec.OSDisk.DiskSizeGB <= 0 || providerSpec.OSDisk.DiskSizeGB >= 32768{

Member

enxebre Sep 16, 2020

move this into a constant 32768?
Missing space before {

enxebre reviewed

View reviewed changes

pkg/apis/machine/v1beta1/machine_webhook.go Outdated

+              	workspaceWarnings, workspaceErrors := validateVSphereWorkspace(providerSpec.Workspace, field.NewPath("providerSpec", "workspace"))
+              	warnings = append(warnings, workspaceWarnings...)
+              	errs = append(errs, workspaceErrors...)
               	errs = append(errs, validateVSphereNetwork(providerSpec.Network, field.NewPath("providerSpec", "network"))...)
               	if providerSpec.NumCPUs != 0 && providerSpec.NumCPUs < minVSphereCPU {

Member

enxebre Sep 16, 2020

shouldn't we remove the if providerSpec.NumCPUs != 0 check? so wouldn't providerSpec.NumCPUs < minVSphereCPU be enough?

Contributor Author

JoelSpeed Sep 16, 2020

Yep, good point, will fix

Member

enxebre commented Sep 16, 2020

can we also drop defaultGCPServiceAccounts on GCP? we can always add it back later.

enxebre added a commit to enxebre/cluster-api-actuator-pkg that referenced this pull request


          Temporary disable webhooks validation

930d6a9

We will enable this after openshift/machine-api-operator#697 get merged

enxebre mentioned this pull request

Temporary disable webhooks validation openshift/cluster-api-actuator-pkg#187

Closed

Member

enxebre commented Sep 16, 2020

/retitle Bug 1876680: Reduce influence of webhooks and convert some errors to non-fatal

openshift-ci-robot changed the title ~~Reduce influence of webhooks and convert some errors to non-fatal~~ Bug 1876680: Reduce influence of webhooks and convert some errors to non-fatal

openshift-ci-robot added the bugzilla/severity-high label

Contributor

openshift-ci-robot commented Sep 16, 2020

@JoelSpeed: This pull request references Bugzilla bug 1876680, which is valid. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target release (4.6.0) matches configured target release for branch (4.6.0)
bug is in the state NEW, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

In response to this:

Bug 1876680: Reduce influence of webhooks and convert some errors to non-fatal

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot added the bugzilla/valid-bug label

Contributor Author

JoelSpeed commented Sep 16, 2020

can we also drop defaultGCPServiceAccounts on GCP? we can always add it back later.

In my testing, if there was no GCP Service Account then the node wouldn't join the cluster. Based on what I've done in this PR so far, I'd suggest making this a warning if changing at all, WDYT?

JoelSpeed force-pushed the webhook-warnings branch from 0a3ceb2 to 50183c8 Compare

September 16, 2020 14:52

Member

enxebre commented Sep 16, 2020

/approve

Contributor

openshift-ci-robot commented Sep 16, 2020

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: enxebre

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [enxebre]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci-robot added the approved label

JoelSpeed added 5 commits

September 21, 2020 14:01


          Migrate admission webhooks to allow setting warnings

a5d7179


          Reduce opinionation of AWS providerSpec webhook

e195413

The webhooks have been provided opinionated defaults and requiring these
in validation. This may cause issues for existing users and as such, we
need to reel back on these changes and ensure that we only error when a
value is missing that would cause a Machine to not be created
successfully


          Reduce opinionation of Azure providerSpec webhook

8a9d079

The webhooks have been provided opinionated defaults and requiring these
in validation. This may cause issues for existing users and as such, we
need to reel back on these changes and ensure that we only error when a
value is missing that would cause a Machine to not be created
successfully


          Reduce opinionation of vSphere providerSpec webhook

218cccc

The webhooks have been provided opinionated defaults and requiring these
in validation. This may cause issues for existing users and as such, we
need to reel back on these changes and ensure that we only error when a
value is missing that would cause a Machine to not be created
successfully


          Reduce opinionation of GCP providerSpec webhook

0b89f8d

The webhooks have been provided opinionated defaults and requiring these
in validation. This may cause issues for existing users and as such, we
need to reel back on these changes and ensure that we only error when a
value is missing that would cause a Machine to not be created
successfully

JoelSpeed force-pushed the webhook-warnings branch from 50183c8 to 0b89f8d Compare

September 21, 2020 13:01

michaelgugino self-requested a review

September 21, 2020 13:09

JoelSpeed mentioned this pull request

Bug 1909108: Upgrade dependencies to controller-runtime v0.7.0 and K8s 1.20 #705

Merged

enxebre added a commit to enxebre/cluster-api-actuator-pkg that referenced this pull request


          Temporary disable webhooks validation

9c4519a

We will enable this after openshift/machine-api-operator#697 get merged

enxebre mentioned this pull request

Bug 1876680: Temporarily disable webhook validation openshift/cluster-api-actuator-pkg#189

Merged

Member

enxebre commented Sep 23, 2020

/retest

elmiko approved these changes

View reviewed changes

Contributor

elmiko left a comment

this generally looks good to me, i'm withholding adding the label though because i couldn't till if there were still changes requested. happy to revisit if there are no more questions.

Member

enxebre commented Sep 24, 2020

/retest

Member

enxebre commented Sep 24, 2020

/lgtm

openshift-ci-robot assigned enxebre

openshift-ci-robot added the lgtm label

Contributor

openshift-bot commented Sep 24, 2020

/retest

Please review the full test history for this PR and help us cut down flakes.

1 similar comment

Contributor

openshift-bot commented Sep 24, 2020

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-merge-robot merged commit 5cf5a28 into openshift:master

Contributor

openshift-ci-robot commented Sep 24, 2020

@JoelSpeed: All pull requests linked via external trackers have merged:

Bugzilla bug 1876680 has been moved to the MODIFIED state.

In response to this:

Bug 1876680: Reduce influence of webhooks and convert some errors to non-fatal

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

JoelSpeed deleted the webhook-warnings branch

November 18, 2020 14:16

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved bugzilla/severity-high bugzilla/valid-bug lgtm