MTL-1819 Resolve Failed Wait Condition #40
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rustydb
changed the title
rustydb
force-pushed
the
MTL-1819-reboot-race-condition
branch
from
cc5f0cf
to
7663848
Compare
rustydb
force-pushed
the
MTL-1819-reboot-race-condition
branch
3 times, most recently
from
544dc89
to
00e7e55
Compare
heemstra
reviewed
Jun 27, 2022
rustydb
force-pushed
the
MTL-1819-reboot-race-condition
branch
from
00e7e55
to
f22590e
Compare
rustydb
force-pushed
the
MTL-1819-reboot-race-condition
branch
4 times, most recently
from
aaefe54
to
eb17689
Compare
heemstra
approved these changes
Jun 27, 2022
metal-md-disks.sh had an oversight, when checking metal.no-wipe=1 the /tmp/metalpave.done file was never created. This prevented the metal_paved function from correctly reporting whether the pave had actually finished or had been skipped. This failure causes the dependent modules running on k8s-masters and k8s-workers to never exit properly when metal.no-wipe=1 was set. Lastly there are a few minor tweaks in this commit: - This also fixes the pave function's 5-second timer which was actually only giving the user 4 seconds due to using -gt instead of -ge. - Additionally this fixes the log truncation in the pave function, the ending line stating 'pave done' was truncating the log-file. - Adds a metal.wipe-delay kernel argument for changing the delay-time from 5 seconds to anything between 2 and 60 seconds. - Lastly this moves /tmp/metalpave.done into a global variable set in metal-lib.sh, making it less error prone to typo mistakes.
rustydb
force-pushed
the
MTL-1819-reboot-race-condition
branch
from
eb17689
to
cdb9892
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary and Scope
Issue Type
Currently, netboots using
metal.no-wipe=1may or may not succeed. This PR fixes a race condition where our k8s-master and k8s-worker modules never exit while waiting for the wipe function to end.metal-md-disks.shhad an oversight, when checking metal.no-wipe=1 the/tmp/metalpave.donefile was never created. This prevented themetal_pavedfunction from correctly reporting whether the pave had actually finished or had been skipped. This failure causes the dependent modules running on k8s-masters and k8s-workers to never exit properly whenmetal.no-wipe=1was set on netboots.Lastly there are a few minor tweaks in this commit:
-gtinstead of-ge.metal.wipe-delaykernel argument for changing the delay-time from 5 seconds to anything between 2 and 60 seconds./tmp/metalpave.doneinto a global variable set inmetal-lib.sh, making it less error prone to typo mistakes.Prerequisites
Idempotency
Risks and Mitigations
This removes risk of dependent dracut modules from running indefinitely, causing the boot to stall.