[RFE] Allow fetching older flatcar packages when starting from scratch

[ffilippopoulos]

Current situation

We have a case, after losing our postgres instance backing nebraska, where we want to start a fresh deployment that serves different packages to different groups. In this scenario, we want only our lower environments to run on latest flatcar, while keeping our production ones on previous versions until we are ready to update. For that purpose we create separate groups per environment and pin them to specific version and arch. As I understand, when deploying nebraska from scratch there is no option to fetch older packages and the only packages available can come from the current content of: https://public.update.flatcar-linux.net/v1/update/ (using default config)

I see that there is a sync-update-url url flag that one can use to trigger nebraska to sync from a different source than: https://public.update.flatcar-linux.net/v1/update/. From what I understand, this will allow us to run our own omaha server and serve older packages to cater for our prod. But still we'd have to maintain a separate nebraska deployment per environment at least as long as babysit our custom omaha servers.

Impact

This behaviour makes the postgres deployment more precious than it should be, holding state that could also be retrieved from public channels.

Ideal future situation

This is to propose adding a new flag that would allow nebraska to fetch older package info at startup. I do not know exactly what is going to be the source of truth, maybe: https://www.flatcar.org/releases-json/releases.json.

Additional information

If that is something completely out of the design scope, would there be a suggested workaround to achieve what we want here?

Thank you very much in advance!

[pothos]

Maybe a helper script can be used to populate the DB, see https://github.com/flatcar-linux/flatcar-build-scripts/blob/master/upload_package where you can throw away the ssh/scp part and only use the package registration based on the files like https://update.release.flatcar-linux.net/amd64-usr/3033.2.4/flatcar_production_update.gz - currently the sha1 checksum needs to be calculated but sha256 is in the checksum file, such as https://update.release.flatcar-linux.net/amd64-usr/3033.2.4/flatcar_production_update.gz.sha256

[george-angel]

Hey @pothos thanks, we can do it that way, we were hoping to gauge the appetite for having this functionality in Nebraska itself, to minimise random scripts on our side.

We are happy to PR if it sounds good / useful in principal. If you feel like its out of scope / doesn't belong / not useful - we'll work on some workaround on our side.

Thanks!

[pothos]

I think it makes more sense as a helper script in the Nebraska repo, which gets a list of Flatcar releases to register. Other ideas @joaquimrocha?

[joaquimrocha]

Yes. I think this is something we should have as an external helper. Still in the repo, but as a separate script that uses the rest API.

[george-angel]

Hello! We have ended up solving this with a generic Postgres backup / restore approach and so didn't need or implement a script to load existing packages via Nebraska API.

Feel free to close this issue if you prefer, we will not be working on this script for now.

Thank you for help and support.