-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
QEMU: Unblock bootpd if start fails to due to blocking #16789
Conversation
log.Debugf("IP: %s", d.IPAddress) | ||
if unblockErr := firewall.UnblockBootpd(); unblockErr != nil { | ||
klog.Errorf("failed unblocking bootpd from firewall: %v", unblockErr) | ||
exit.Error(reason.IfBootpdFirewall, "ip not found", err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am curious if we could return an error and let the CMD package to exit ? I generally frown upon exit in non-cmd code, but if there is a good reason or makes way cleaner code I am okay with Exit here..
what u thnk?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The reason I did exit.Error
is because it will retry if the first start fails. In the case where we know bootpd is being blocked and we failed to unblock the process it's almost certainly going to fail the retry so I was exiting. I could remove it, the downside would be that it would have to go through the start loop one more time which is 30+ seconds.
log.Debugf("IP: %s", d.IPAddress) | ||
if unblockErr := firewall.UnblockBootpd(); unblockErr != nil { | ||
klog.Errorf("failed unblocking bootpd from firewall: %v", unblockErr) | ||
exit.Error(reason.IfBootpdFirewall, "ip not found", err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
add comment, and say if return errr, it will retry which makes no sense.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
might be able to use re-triable errors pattern.
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: medyagh, spowelljr The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
/ok-to-test |
kvm2 driver with docker runtime
Times for minikube start: 50.6s 53.0s 52.9s 57.0s 51.7s Times for minikube ingress: 28.3s 29.3s 27.8s 27.7s 28.3s docker driver with docker runtime
Times for minikube ingress: 48.9s 33.5s 48.9s 48.4s 48.9s Times for minikube start: 25.8s 25.5s 22.9s 25.6s 23.0s docker driver with containerd runtime
Times for minikube start: 24.1s 24.7s 23.8s 23.1s 22.9s Times for minikube ingress: 31.4s 31.4s 30.5s 31.4s 20.4s |
These are the flake rates of all failed tests.
Too many tests failed - See test logs for more details. To see the flake rates of all tests by environment, click here. |
Closes #16776
Even if you have bootpd unblocked in the builtin macOS firewall, you can run into the situation where it's still being blocked. To resolve it you have to run the unblock command again.
This PR checks the output of a start failure on QEMU, and if it's due to bootpd being blocked runs the unblock command and retries the start again.
Before:
After: