Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge ostree-ext #875

Merged
merged 1,349 commits into from
Nov 7, 2024
Merged

Merge ostree-ext #875

merged 1,349 commits into from
Nov 7, 2024

Conversation

cgwalters
Copy link
Collaborator

@cgwalters cgwalters commented Nov 5, 2024

xref ostreedev/ostree-rs-ext#675

Basically I see bootc as the successor/inheritor of the original ostree-container work.

It will help us close the loop faster on things to merge it into this project.

TODO followups:

  • Remove more unreferenced files (e.g. GHA workflows)
  • Add bootc internals ostree-container or so we can take over the ostree container verb

cgwalters and others added 30 commits September 19, 2023 12:52
Add a man page (which we'll figure out how to install later)
describing the state of things.
We need to support e.g.
`rpm-ostree compose container-encapsulate --format=oci manifest.yaml oci:foo:42`
This patch allows container image deployment stored in a local directory in addition to the supported remote OCI registry, OCI archive tarball, and local container storage. For example, the following command is now supported:

sudo ostree container image deploy --imgref ostree-unverified-image:dir:/var/home/core/fcos --sysroot /var/home/core/sysroot --stateroot fedora-coreos

The reason for making this change relates to the effort to create an FCOS qemu image from a container image input using OSBuild. OSBuild stores container image sources in a local directory in order to solve another bug and, thus, should not be changed. Therefore, to allow OSBuild to deploy from a container image, this patch is needed.
…ner-deploy

Allows container deployment from local directory
- Cover containers-storage and oci-archive too
- Test that we can reserialize back into the same string
container: Add more tests for image reference
This change adds an integration test for deploying from a container image
stored in a local directory. It is directly related a recent change to
support the `dir:` transport when using `ostree container image deploy`.
container/encapsulate: Fix pushing to OCI directories with a tag
…-deploy-integration-test

container: Add dir container deploy test
Just two small changes, plus a new man page.
term_size is archived.
Looking at doing a bootc reinstall, we need the ability
to retain existing deployments for this.
I copy/pasted the mountpoint check into bootc, but it'd
be better to centralize.
This also relates to youki-dev/oci-spec-rs#143

Basically getting the version from the config requires a helper,
this is a lot more convenient.
lib/container: Add APIs to retrieve version conveniently
We're really verbose with `env RUST_LOG=debug`...most of
these spans shouldn't be emitting any arguments at all . And
even then let's lower to debug level for them too.
This ends up recursing into the manifest/config objects
which is a wall of spam with `env RUST_LOG=debug`.  Just print
the merge commit to start here.
Quiet extremely verbose tracing::debug
Mainly want to use new apis in bootc.
If we have a low amount of high-size packages then we will drain with an
index higher than the size of the Vec which triggers a panic.

Fixes: coreos/rpm-ostree#4646
chunking: Handle low amount of high-size packages
I don't recall now the rationale for the previous code; if
we're using this path we always want to be in a container-native
mode.  Currently the "unpack to pure ostree commit" gets triggered
if one uses this on a container image created via raw
`ostree container encapsulate`.

Anyone who wants the prior behavior should instead unencapsulate
and deploy the underlying commit.
cgwalters and others added 12 commits October 24, 2024 12:13
This allows us to accept images to that don't have any metadata
keys starting with `ostree`.

Signed-off-by: Colin Walters <walters@verbum.org>
Because dnf5 broke it and keeping both working is annoying
and we didn't really need it.

Signed-off-by: Colin Walters <walters@verbum.org>
This (with cautious optimism) appears to fix containers#657.

Signed-off-by: John Eckersberg <jeckersb@redhat.com>
Remove decompress_bridge and move decompression inline
This relies on the new tar API.

Closes: ostreedev/ostree-rs-ext#655

Signed-off-by: Colin Walters <walters@verbum.org>
See containers#856

Basically we need to handle absolute paths here too.

Signed-off-by: Colin Walters <walters@verbum.org>
tar: Propagate PAX extensions (including xattrs)
…ot-absolute

tar: Handle absolute hardlinked paths to sysroot
@cgwalters
Copy link
Collaborator Author

Ah, messed this one up, will redo later

@cgwalters cgwalters force-pushed the merge-ostree-rs-ext branch from 52dd6c6 to 2358336 Compare November 6, 2024 23:19
@cgwalters cgwalters marked this pull request as ready for review November 6, 2024 23:20
@cgwalters
Copy link
Collaborator Author

OK I redid this one, now with proper git history

@cgwalters
Copy link
Collaborator Author

cgwalters commented Nov 6, 2024

Ugh, right I think I need to git merge --signoff

EDIT: Well it helps to signoff the merge commit but, I think in practice I guess we can't use subtree merges if we want to require signoffs on each commit, we need to use git filter-branch to edit them.

…s-ext

Signed-off-by: Colin Walters <walters@verbum.org>
@cgwalters cgwalters force-pushed the merge-ostree-rs-ext branch from 2358336 to 23a54d9 Compare November 6, 2024 23:27
Copy link

@p5 p5 Nov 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, GitHub doesn't support GHA workflows in subdirectories (something I've wanted for years!), so these tests will not run.
If you wish to keep these tests, it's probably best moving the contents of ./ostree-ext/.github/workflows/ to ./.github/workflows/.

Any paths inside these workflows will also need updating, since they aren't expecting the old directory structure.

Copy link
Collaborator Author

@cgwalters cgwalters Nov 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, yep there's more followup to do here, though in practice we have decent coverage from the unit tests and our integration tests here cover a lot.

@cgwalters
Copy link
Collaborator Author

I have clicked "Set DCO to pass" to override the signoff on individual commits. Most of these commits are mine anyways, and with this statement I add my Signed-off-by onto all of these commits.

There's a lot shared between bootc and ostree-rs-ext.

Signed-off-by: Colin Walters <walters@verbum.org>
@cgwalters cgwalters requested a review from jeckersb November 7, 2024 22:11
@jeckersb
Copy link
Contributor

jeckersb commented Nov 7, 2024

Yay for one less repo to watch 🥳

@cgwalters cgwalters merged commit affc8b9 into containers:main Nov 7, 2024
23 of 32 checks passed
cgwalters added a commit to cgwalters/ostree-rs-ext that referenced this pull request Nov 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants