Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pubsys: enable the use of crane to publish kits #317

Merged
merged 2 commits into from
Jul 11, 2024

Conversation

sam-berning
Copy link
Contributor

Issue number:

Closes #316

Description of changes:

Adds methods to oci_cli_wrapper::ImageTool for pushing individual-arch kit images and the multi-arch manifest, and implements these methods for both docker and crane.

Additionally, moves the DockerArchitecture enum into oci-cli-wrapper, since it is mainly used while interacting with the OCI CLI tools.

Finally, cleaned up unused dependencies across Twoliter with cargo-machete (barring all of the bindeps dependencies for Twoliter)

Testing done:

Built twoliter, and tried:

$ ../../../target/debug/pubsys --infra-config-path Infra.toml publish-kit --kit-path build/kits/core-kit --vendor bottlerocket --version v1.0.0 --build-id c232d4f-dirty
$ TWOLITER_KIT_IMAGE_TOOL=docker ../../../target/debug/pubsys --infra-config-path Infra.toml publish-kit --kit-path build/kits/core-kit --vendor bottlerocket --version v1.0.0 --build-id c232d4f-dirty
$ TWOLITER_KIT_IMAGE_TOOL=crane ../../../target/debug/pubsys --infra-config-path Infra.toml publish-kit --kit-path build/kits/core-kit --vendor bottlerocket --version v1.0.0 --build-id c232d4f-dirty

All of the above pushed as expected, and left the system in the correct state afterwards (i.e. cleaned up tempdirs and unnecessary docker images)

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

Copy link
Contributor

@jmt-lab jmt-lab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -20,7 +20,6 @@ pipesys = { version = "0.1", path = "../pipesys" }
rand = { version = "0.8", default-features = false, features = ["std", "std_rng"] }
regex = "1"
reqwest = { version = "0.11", default-features = false, features = ["rustls-tls", "blocking"] }
semver = { version = "1", features = ["serde"]}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: could separate all these into a cleanup commit (but it's a good change and fine to include in this PR)

Might be good to add a cargo machete lint to CI.

tools/pubsys/src/kit/publish_kit/mod.rs Outdated Show resolved Hide resolved
Comment on lines +57 to +60
let mut oci_archive = TarArchive::new(&mut oci_file);
oci_archive
.unpack(temp_dir.path())
.context(error::ArchiveExtractSnafu)?;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How much time does this step take? It's sort of nice to have just the tar archive as the output file from rpm2kit, but if it's a meaningful speed-up, we could have the unpacked contents be the "preferred" form and create the tar archive as a temporary file in the (already slow) Docker path.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tested this with just the tar command against a recent build of the core kit. unpack took 16 seconds, and recompressing the contents took 1.5 seconds - so could definitely be an optimization worth taking. docker would take even longer to push an image but 1.5 seconds is pretty insignificant comparatively

tools/pubsys/src/kit/publish_kit/mod.rs Outdated Show resolved Hide resolved
Signed-off-by: Sam Berning <bernings@amazon.com>
Noticed a few crates had unused dependencies so I ran `cargo-machete`
and cleaned up all of the ones it reported, barring the tools that `twoliter`
includes with `bindeps`.

Signed-off-by: Sam Berning <bernings@amazon.com>
Copy link
Contributor

@jmt-lab jmt-lab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@sam-berning sam-berning merged commit bc62668 into bottlerocket-os:develop Jul 11, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add crane support for publishing kits
3 participants