Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Persist RSS completed marker on all sleds #7186

Open
andrewjstone opened this issue Nov 27, 2024 · 1 comment
Open

Persist RSS completed marker on all sleds #7186

andrewjstone opened this issue Nov 27, 2024 · 1 comment
Labels
bootstrap services For those occasions where you want the rack to turn on Sled Agent Related to the Per-Sled Configuration and Management

Comments

@andrewjstone
Copy link
Contributor

Right now, when we complete RSS we persist a marker file (ledger) that indicates that RSS should not be re-run as it has already completed. This prevents data loss. However, this marker file is only present on the M.2 devices attached to the scrimlet where RSS was run. If RSS runs on the other scrimlet, or one of the sleds is swapped into scimlet position, the marker will not be present.

We want this marker to be present on all sleds. We could require it to be persisted to all sleds before completing RSS by direct copying. However, this means that if a new sled is added to the rack and put in the scrimlet position it will not have the files. We already have a mechanism to gossip around configuration required for early boot: namely the bootstore. We can put this marker key in the bootstore as well and it will propagate asynchronously over the bootstrap network to any sled in the rack. Max delay for an online sled is ~1 second.

This will require some changes to the bootstore to support multiple keys, but this is not unreasonable, and something we have considered in the past.

@andrewjstone andrewjstone added Sled Agent Related to the Per-Sled Configuration and Management bootstrap services For those occasions where you want the rack to turn on labels Nov 27, 2024
@andrewjstone
Copy link
Contributor Author

andrewjstone commented Nov 27, 2024

We also want to put the marker in the bootstore from nexus if it doesn't already exist. See #7185 (comment) for details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bootstrap services For those occasions where you want the rack to turn on Sled Agent Related to the Per-Sled Configuration and Management
Projects
None yet
Development

No branches or pull requests

1 participant