Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

install: add --source-imgref #263

Merged
merged 3 commits into from
Jan 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions docs/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -234,3 +234,18 @@ At the current time, leftover data in `/` is **NOT** automatically cleaned up.
be useful, because it allows the new image to automatically import data from the previous
host system! For example, things like SSH keys or container images can be copied
and then deleted from the original.

### Using `bootc install to-filesystem --source-imgref <imgref>`
ondrejbudai marked this conversation as resolved.
Show resolved Hide resolved

By default, `bootc install` has to be run inside a podman container. With this assumption,
it can escape the container, find the source container image (including its layers) in
the podman's container storage and use it to create the image.

When `--source-imgref <imgref>` is given, `bootc` no longer assumes that it runs inside podman.
Instead, the given container image reference (see [containers-transports(5)](https://github.com/containers/image/blob/main/docs/containers-transports.5.md)
for accepted formats) is used to fetch the image. Note that `bootc install` still has to be
run inside a chroot created from the container image. However, this allows users to use
a different sandboxing tool (e.g. [bubblewrap](https://github.com/containers/bubblewrap)).

This argument is mainly useful for 3rd-party tooling for building disk images from bootable
containers (e.g. based on [osbuild](https://github.com/osbuild/osbuild)).
2 changes: 1 addition & 1 deletion lib/src/blockdev.rs
Original file line number Diff line number Diff line change
Expand Up @@ -197,7 +197,7 @@ pub(crate) fn find_parent_devices(device: &str) -> Result<Vec<String>> {
let kind = dev
.get("TYPE")
.with_context(|| format!("device in hierarchy of {device} missing TYPE"))?;
if kind == "disk" {
if kind == "disk" || kind == "loop" {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes. It would probably make sense to try to use a shared crate for dealing with block devices at some point.

parents.push(name.clone());
} else if kind == "mpath" {
parents.push(name.clone());
Expand Down
137 changes: 101 additions & 36 deletions lib/src/install.rs
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,18 @@ pub(crate) struct InstallTargetOpts {
pub(crate) skip_fetch_check: bool,
}

#[derive(clap::Args, Debug, Clone, Serialize, Deserialize)]
pub(crate) struct InstallSourceOpts {
/// Install the system from an explicitly given source.
///
/// By default, bootc install and install-to-filesystem assumes that it runs in a podman container, and
/// it takes the container image to install from the podman's container registry.
/// If --source-imgref is given, bootc uses it as the installation source, instead of the behaviour explained
/// in the previous paragraph. See skopeo(1) for accepted formats.
#[clap(long)]
pub(crate) source_imgref: Option<String>,
}

#[derive(clap::Args, Debug, Clone, Serialize, Deserialize)]
pub(crate) struct InstallConfigOpts {
/// Disable SELinux in the target (installed) system.
Expand Down Expand Up @@ -137,6 +149,10 @@ pub(crate) struct InstallToDiskOpts {
#[serde(flatten)]
pub(crate) block_opts: InstallBlockDeviceOpts,

#[clap(flatten)]
#[serde(flatten)]
pub(crate) source_opts: InstallSourceOpts,

#[clap(flatten)]
#[serde(flatten)]
pub(crate) target_opts: InstallTargetOpts,
Expand Down Expand Up @@ -209,6 +225,9 @@ pub(crate) struct InstallToFilesystemOpts {
#[clap(flatten)]
pub(crate) filesystem_opts: InstallTargetFilesystemOpts,

#[clap(flatten)]
pub(crate) source_opts: InstallSourceOpts,

#[clap(flatten)]
pub(crate) target_opts: InstallTargetOpts,

Expand All @@ -222,18 +241,25 @@ pub(crate) struct SourceInfo {
/// Image reference we'll pull from (today always containers-storage: type)
pub(crate) imageref: ostree_container::ImageReference,
/// The digest to use for pulls
pub(crate) digest: String,
pub(crate) digest: Option<String>,
/// Whether or not SELinux appears to be enabled in the source commit
pub(crate) selinux: bool,
/// Whether the source is available in the host mount namespace
pub(crate) in_host_mountns: Option<HostMountnsInfo>,
}

/// Information about the host mount namespace
#[derive(Debug, Clone)]
pub(crate) struct HostMountnsInfo {
/// True if the skoepo on host supports containers-storage:
pub(crate) skopeo_supports_containers_storage: bool,
}

// Shared read-only global state
pub(crate) struct State {
pub(crate) source: SourceInfo,
/// Force SELinux off in target system
pub(crate) override_disable_selinux: bool,
/// True if the skoepo on host supports containers-storage:
pub(crate) skopeo_supports_containers_storage: bool,
#[allow(dead_code)]
pub(crate) setenforce_guard: Option<crate::lsm::SetEnforceGuard>,
#[allow(dead_code)]
Expand Down Expand Up @@ -368,6 +394,29 @@ impl SourceInfo {
name: container_info.image.clone(),
};
let digest = crate::podman::imageid_to_digest(&container_info.imageid)?;

let skopeo_supports_containers_storage = skopeo_supports_containers_storage()
.context("Failed to run skopeo (it currently must be installed in the host root)")?;
Self::from(
imageref,
Some(digest),
Some(HostMountnsInfo {
skopeo_supports_containers_storage,
}),
)
}

#[context("Creating source info from a given imageref")]
pub(crate) fn from_imageref(imageref: &str) -> Result<Self> {
let imageref = ostree_container::ImageReference::try_from(imageref)?;
Self::from(imageref, None, None)
}

fn from(
imageref: ostree_container::ImageReference,
digest: Option<String>,
in_host_mountns: Option<HostMountnsInfo>,
) -> Result<Self> {
let cancellable = ostree::gio::Cancellable::NONE;
let commit = Task::new("Reading ostree commit", "ostree")
.args(["--repo=/ostree/repo", "rev-parse", "--single"])
Expand All @@ -386,6 +435,7 @@ impl SourceInfo {
imageref,
digest,
selinux,
in_host_mountns,
})
}
}
Expand Down Expand Up @@ -562,28 +612,39 @@ async fn initialize_ostree_root_from_self(
let sysroot = ostree::Sysroot::new(Some(&gio::File::for_path(rootfs)));
sysroot.load(cancellable)?;

// We need to fetch the container image from the root mount namespace
let skopeo_cmd = run_in_host_mountns("skopeo");
let proxy_cfg = ostree_container::store::ImageProxyConfig {
skopeo_cmd: Some(skopeo_cmd),
..Default::default()
};

let mut temporary_dir = None;
let src_imageref = if state.skopeo_supports_containers_storage {
// We always use exactly the digest of the running image to ensure predictability.
let spec =
crate::utils::digested_pullspec(&state.source.imageref.name, &state.source.digest);
ostree_container::ImageReference {
transport: ostree_container::Transport::ContainerStorage,
name: spec,
let (src_imageref, proxy_cfg) = match &state.source.in_host_mountns {
None => (state.source.imageref.clone(), None),
Some(host_mountns_info) => {
let src_imageref = if host_mountns_info.skopeo_supports_containers_storage {
// We always use exactly the digest of the running image to ensure predictability.
let digest = state
.source
.digest
.as_ref()
.ok_or_else(|| anyhow::anyhow!("Missing container image digest"))?;
let spec = crate::utils::digested_pullspec(&state.source.imageref.name, digest);
ostree_container::ImageReference {
transport: ostree_container::Transport::ContainerStorage,
name: spec,
}
} else {
let td = tempfile::tempdir_in("/var/tmp")?;
let path: &Utf8Path = td.path().try_into().unwrap();
let r = copy_to_oci(&state.source.imageref, path)?;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Unrelated to this change, but I would probably be fine just hard dropping this hacky "copy to oci" code that I don't think we need anymore since c9s has a new enough skopeo and that's fine from my PoV)

Totally optional of course...just reminded by the seeing the code again here. If you want I can also do a prep PR removing it (or feel free to yourself), it seems like it would simplify this change.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ended up doing this in #265 - but I still don't have a really strong opinion - we can merge this before that one if you prefer too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No objections from my side. #265 has a failing test suite currently, though. I think that rebasing #265 on top of this PR might be easier, so I prefer landing this one first. I must admit that it's a tad bit selfish view, though. 👼

temporary_dir = Some(td);
r
};

// We need to fetch the container image from the root mount namespace
let skopeo_cmd = run_in_host_mountns("skopeo");
let proxy_cfg = ostree_container::store::ImageProxyConfig {
skopeo_cmd: Some(skopeo_cmd),
..Default::default()
};

(src_imageref, Some(proxy_cfg))
}
} else {
let td = tempfile::tempdir_in("/var/tmp")?;
let path: &Utf8Path = td.path().try_into().unwrap();
let r = copy_to_oci(&state.source.imageref, path)?;
temporary_dir = Some(td);
r
};
let src_imageref = ostree_container::OstreeImageReference {
// There are no signatures to verify since we're fetching the already
Expand All @@ -601,7 +662,7 @@ async fn initialize_ostree_root_from_self(
let mut options = ostree_container::deploy::DeployOpts::default();
options.kargs = Some(kargs.as_slice());
options.target_imgref = Some(&state.target_imgref);
options.proxy_cfg = Some(proxy_cfg);
options.proxy_cfg = proxy_cfg;
println!("Creating initial deployment");
let target_image = state.target_imgref.to_string();
let state =
Expand Down Expand Up @@ -906,6 +967,7 @@ async fn verify_target_fetch(imgref: &ostree_container::OstreeImageReference) ->
/// Preparation for an install; validates and prepares some (thereafter immutable) global state.
async fn prepare_install(
config_opts: InstallConfigOpts,
source_opts: InstallSourceOpts,
target_opts: InstallTargetOpts,
) -> Result<Arc<State>> {
// We need full root privileges, i.e. --privileged in podman
Expand All @@ -919,16 +981,20 @@ async fn prepare_install(
let rootfs = cap_std::fs::Dir::open_ambient_dir("/", cap_std::ambient_authority())
.context("Opening /")?;

// This command currently *must* be run inside a privileged container.
let container_info = crate::containerenv::get_container_execution_info(&rootfs)?;
if let Some("1") = container_info.rootless.as_deref() {
anyhow::bail!("Cannot install from rootless podman; this command must be run as root");
}

let skopeo_supports_containers_storage = skopeo_supports_containers_storage()
.context("Failed to run skopeo (it currently must be installed in the host root)")?;
let source = match source_opts.source_imgref {
None => {
let container_info = crate::containerenv::get_container_execution_info(&rootfs)?;
// This command currently *must* be run inside a privileged container.
if let Some("1") = container_info.rootless.as_deref() {
anyhow::bail!(
"Cannot install from rootless podman; this command must be run as root"
);
}

let source = SourceInfo::from_container(&container_info)?;
SourceInfo::from_container(&container_info)?
}
Some(source) => SourceInfo::from_imageref(&source)?,
};

// Parse the target CLI image reference options and create the *target* image
// reference, which defaults to pulling from a registry.
Expand Down Expand Up @@ -982,7 +1048,6 @@ async fn prepare_install(
// combines our command line options along with some bind mounts from the host.
let state = Arc::new(State {
override_disable_selinux,
skopeo_supports_containers_storage,
setenforce_guard,
source,
config_opts,
Expand Down Expand Up @@ -1065,7 +1130,7 @@ pub(crate) async fn install_to_disk(opts: InstallToDiskOpts) -> Result<()> {
anyhow::bail!("Not a block device: {}", block_opts.device);
}
}
let state = prepare_install(opts.config_opts, opts.target_opts).await?;
let state = prepare_install(opts.config_opts, opts.source_opts, opts.target_opts).await?;

// This is all blocking stuff
let mut rootfs = {
Expand Down Expand Up @@ -1174,7 +1239,7 @@ pub(crate) async fn install_to_filesystem(opts: InstallToFilesystemOpts) -> Resu
}

// Gather global state, destructuring the provided options
let state = prepare_install(opts.config_opts, opts.target_opts).await?;
let state = prepare_install(opts.config_opts, opts.source_opts, opts.target_opts).await?;

match fsopts.replace {
Some(ReplaceMode::Wipe) => {
Expand Down