rethinking origins #2326
Comments
|
This is an interesting discussion which touches on a lot of things that have come up multiple times in the past. I think there's a trade-off there. "Editing via D-Bus" (i.e. providing high-level commands) allows for better validation and is more UX friendly in many instances (because it's much more "self-documenting"; I'm sure there's a better term for this when applied to UX design). Though "edit via config file" esp. with drop-ins is a pretty well-understood paradigm at this point and fits better with declarative configuration models like Ignition. A lot of friction involved in changing models though. (And FWIW, your argument of circularity does appeal to me :).) Here's an idea:
(Obviously, as you can tell the subcommands there mirror the The advantage of this approach is that it better layers on top of the current model (and e.g. instant validation means you simply can't have an invalid origin on disk), and provides a roadmap on how to extend rpm-ostree features like OSTree layers without necessarily having to add yet another command. And then for the MCO/Ignition, we can build up on that and have a e.g. |
I still think we should do this at some point, but the experiment with using `GKeyfile` for configuration is IMO a failure and the variety of data formats (treefile JSON vs YAML vs origin keyfiles vs container keyfiles) causes a lot of confusion. Prep for coreos#2326
I still think we should do this at some point, but the experiment with using `GKeyfile` for configuration is IMO a failure and the variety of data formats (treefile JSON vs YAML vs origin keyfiles vs container keyfiles) causes a lot of confusion. Prep for #2326
|
Related to this...actually one of the original goals of ostree was to make it easy for people to track the latest from git. I could fill up a whole hour talk about how problematic it is that so few people test e.g. the latest systemd/GNOME/util-linux/etc/etc because of the intermediate "manual packaging" plus the lack of rollbacks. There are people who maintain COPRs of e.g. "kernel-git" and such; we could add something declarative like: (filename: /etc/rpm-ostree/configuration.d/) that would (also given And a much bigger topic related to this - for building from source locally I'd like to make it even more first class of an operation to build e.g. RPMs (or ostree commits) inside a container image and provide those updates to rpm-ostree. There's a lot here; it strongly relates to the kmod issue but generalizes to the whole OS. |
|
I was actually going to work on this but I ran into some immense layers of tech debt in rpm-ostree that I'm working on fixing first. To start, the server side "treefile" format being totally distinct from the client side format. Plus the parser for the client side (origin) being in C. Which led to a big effort around converting to Rust that's going on now. We're close to landing some cleanups in this area that should unblock us from starting the bikeshed of what this "unified declarative format" looks like and making it work. |
|
Another important thread here is I think it would be beneficial if we can at least drive some support for "declarative package management" into dnf(libdnf) directly too. I hit this all the time when building container images; a |
This is a better alternative to coreos/fedora-coreos-config#830 Basically rather than trying to send this out to all FCOS users, it's much saner to allow people to opt-in to it locally. If we'd finished coreos#2326 then this would be something as trivial as: ``` $ echo 'cliwrap: true' > /etc/rpm-ostree.d/cliwrap.yaml $ rpm-ostree rebuild ``` Unfortunately that's not the world we live in, so a whole lot of layers here need crossing to just propagate a boolean. And it interacts in a tricky way with our change detection code. But, it works and will allow people to try this out. Other fixed problems: - Our `rpm --verify` wrapping was broken - Dropping privileges clashed with the default directory being `/root`, so `chdir(/)` too
This is a better alternative to coreos/fedora-coreos-config#830 Basically rather than trying to send this out to all FCOS users, it's much saner to allow people to opt-in to it locally. If we'd finished coreos#2326 then this would be something as trivial as: ``` $ echo 'cliwrap: true' > /etc/rpm-ostree.d/cliwrap.yaml $ rpm-ostree rebuild ``` Unfortunately that's not the world we live in, so a whole lot of layers here need crossing to just propagate a boolean. And it interacts in a tricky way with our change detection code. But, it works and will allow people to try this out. Other fixed problems: - Our `rpm --verify` wrapping was broken - Dropping privileges clashed with the default directory being `/root`, so `chdir(/)` too
This is a better alternative to coreos/fedora-coreos-config#830 Basically rather than trying to send this out to all FCOS users, it's much saner to allow people to opt-in to it locally. If we'd finished coreos#2326 then this would be something as trivial as: ``` $ echo 'cliwrap: true' > /etc/rpm-ostree.d/cliwrap.yaml $ rpm-ostree rebuild ``` Unfortunately that's not the world we live in, so a whole lot of layers here need crossing to just propagate a boolean. And it interacts in a tricky way with our change detection code. But, it works and will allow people to try this out. Other fixed problems: - Our `rpm --verify` wrapping was broken - Dropping privileges clashed with the default directory being `/root`, so `chdir(/)` too
This is a better alternative to coreos/fedora-coreos-config#830 Basically rather than trying to send this out to all FCOS users, it's much saner to allow people to opt-in to it locally. If we'd finished coreos#2326 then this would be something as trivial as: ``` $ echo 'cliwrap: true' > /etc/rpm-ostree.d/cliwrap.yaml $ rpm-ostree rebuild ``` Unfortunately that's not the world we live in, so a whole lot of layers here need crossing to just propagate a boolean. And it interacts in a tricky way with our change detection code. But, it works and will allow people to try this out. Other fixed problems: - Our `rpm --verify` wrapping was broken - Dropping privileges clashed with the default directory being `/root`, so `chdir(/)` too
This is a better alternative to coreos/fedora-coreos-config#830 Basically rather than trying to send this out to all FCOS users, it's much saner to allow people to opt-in to it locally. If we'd finished coreos#2326 then this would be something as trivial as: ``` $ echo 'cliwrap: true' > /etc/rpm-ostree.d/cliwrap.yaml $ rpm-ostree rebuild ``` Unfortunately that's not the world we live in, so a whole lot of layers here need crossing to just propagate a boolean. And it interacts in a tricky way with our change detection code. But, it works and will allow people to try this out. Other fixed problems: - Our `rpm --verify` wrapping was broken - Dropping privileges clashed with the default directory being `/root`, so `chdir(/)` too
This is a better alternative to coreos/fedora-coreos-config#830 Basically rather than trying to send this out to all FCOS users, it's much saner to allow people to opt-in to it locally. If we'd finished coreos#2326 then this would be something as trivial as: ``` $ echo 'cliwrap: true' > /etc/rpm-ostree.d/cliwrap.yaml $ rpm-ostree rebuild ``` Unfortunately that's not the world we live in, so a whole lot of layers here need crossing to just propagate a boolean. And it interacts in a tricky way with our change detection code. But, it works and will allow people to try this out. Other fixed problems: - Our `rpm --verify` wrapping was broken - Dropping privileges clashed with the default directory being `/root`, so `chdir(/)` too
This is a better alternative to coreos/fedora-coreos-config#830 Basically rather than trying to send this out to all FCOS users, it's much saner to allow people to opt-in to it locally. If we'd finished coreos#2326 then this would be something as trivial as: ``` $ echo 'cliwrap: true' > /etc/rpm-ostree.d/cliwrap.yaml $ rpm-ostree rebuild ``` Unfortunately that's not the world we live in, so a whole lot of layers here need crossing to just propagate a boolean. And it interacts in a tricky way with our change detection code. But, it works and will allow people to try this out. Other fixed problems: - Our `rpm --verify` wrapping was broken - Dropping privileges clashed with the default directory being `/root`, so `chdir(/)` too
|
For a long time, It's somewhat analogous to how Anaconda writes /root/anaconda-ks.cfg (which you can see in the container base image too because for not-so-good reasons we currently use Anaconda to build those). But with this issue...you could imagine that we switch to having the compose write And once that happens, it becomes user editable directly, which would close the loop. Or alternatively, perhaps we keep |
This command takes a *derivation* treefile and applies it to the live OSTree container. Currently, only `packages` is supported. But this paves the way for supporting more derivation fields as well as making currently "base" fields only become valid derivation fields too (by promoting it from the `BaseComposeConfigFields` struct to `TreeComposeConfig`). I think this also helps the move towards coreos#2326 by formalizing "client" or "derivation" treefiles more and introducing code which consumes them. What we're doing here is providing a "container" backend for derivation treefiles, but coreos#2326 will provide a "client" backend which uses the core more fully. But the input file itself is the exact same, hence providing symmetry between the two flows. (See also discussions about this in coreos/fedora-coreos-tracker#1054). This will also allow us to drop the `microdnf` dependency, though I haven't yet done that.
Add support for creating a client treefile based on the dropins in `/etc/rpm-ostree/origin.d`. This will currently be used only by the container flow, but could eventually also end up being used client-side as discussed in coreos#2326.
This command reads treefile dropins in `/etc` and applies it to the system. Only the OSTree container flow is supported for now. For the client-side flow, see coreos#2326. In the container flow, only `packages` is currently supported. But this paves the way for supporting more derivation fields as well as making currently "base" fields only become valid derivation fields too (by promoting it from the `BaseComposeConfigFields` struct to `TreeComposeConfig`). What we're doing here is providing a "container" backend for derivation treefiles, but coreos#2326 will provide a "client" backend which uses the core more fully. But the inputs themselves are the exact same, hence providing symmetry between the two flows. (See also discussions about this in coreos/fedora-coreos-tracker#1054). This will also allow us to drop the `microdnf` dependency which will be done in a following patch.
Add support for creating a client treefile based on the dropins in `/etc/rpm-ostree/origin.d`. This will currently be used only by the container flow, but could eventually also end up being used client-side as discussed in coreos#2326.
This command reads treefile dropins in `/etc` and applies it to the system. Only the OSTree container flow is supported for now. For the client-side flow, see coreos#2326. In the container flow, only `packages` is currently supported. But this paves the way for supporting more derivation fields as well as making currently "base" fields only become valid derivation fields too (by promoting it from the `BaseComposeConfigFields` struct to `TreeComposeConfig`). What we're doing here is providing a "container" backend for derivation treefiles, but coreos#2326 will provide a "client" backend which uses the core more fully. But the inputs themselves are the exact same, hence providing symmetry between the two flows. (See also discussions about this in coreos/fedora-coreos-tracker#1054). This will also allow us to drop the `microdnf` dependency which will be done in a following patch.
This command reads treefile dropins in `/etc` and applies it to the system. Only the OSTree container flow is supported for now. For the client-side flow, see coreos#2326. In the container flow, only `packages` is currently supported. But this paves the way for supporting more derivation fields as well as making currently "base" fields only become valid derivation fields too (by promoting it from the `BaseComposeConfigFields` struct to `TreeComposeConfig`). What we're doing here is providing a "container" backend for derivation treefiles, but coreos#2326 will provide a "client" backend which uses the core more fully. But the inputs themselves are the exact same, hence providing symmetry between the two flows. (See also discussions about this in coreos/fedora-coreos-tracker#1054). This will also allow us to drop the `microdnf` dependency which will be done in a following patch.
Add support for creating a client treefile based on the dropins in `/etc/rpm-ostree/origin.d`. This will currently be used only by the container flow, but could eventually also end up being used client-side as discussed in #2326.
This command reads treefile dropins in `/etc` and applies it to the system. Only the OSTree container flow is supported for now. For the client-side flow, see #2326. In the container flow, only `packages` is currently supported. But this paves the way for supporting more derivation fields as well as making currently "base" fields only become valid derivation fields too (by promoting it from the `BaseComposeConfigFields` struct to `TreeComposeConfig`). What we're doing here is providing a "container" backend for derivation treefiles, but #2326 will provide a "client" backend which uses the core more fully. But the inputs themselves are the exact same, hence providing symmetry between the two flows. (See also discussions about this in coreos/fedora-coreos-tracker#1054). This will also allow us to drop the `microdnf` dependency which will be done in a following patch.
I'd like to implement coreos#1265 for both the client-side and the container flow. But to do that, I'd rather not have to wire it through all the layers that need to know about it. In a move towards coreos#2326, I'd like to try to have everything in the `DeployTransaction` -> `RpmOstreeSysrootUpgrader` -> `RpmOstreeContext` flow only deal in treefiles. But to not change the format on disk just yet, we still need to deserialize from and serialize back into a `GKeyFile` at the I/O boundaries (really, libostree API boundaries). We have a function for the former, but not the latter. This patch adds one.
|
I was a Nix user, maybe, 3 weeks ago. The nix workflow, especially surrounding home-manager is sublime. Is renders dotfile management obsolete and I had a working setup that worked across two very different machines. It's really awesome tech, but I think rpm-ostree could do even better. The general rpm-ostree workflow has a lot going for it right now. Low frequency changes (OS updates) are handled by rpm-ostree, where high frequency changes (user) are handled by flatpak. The low-frequency stuff has 100% reproducibility and doesn't seem like low-hanging fruit to me. There's no solution for reproducing medium-frequency changes (tweaks inside of /etc), and reproducing high-frequency changes (copying "mutation-land" config across machines). One bug that I have run into is that Firefox nags me for a password if I open it with my Yubikey plugged in. The relevant config is part of the read-only portion of /etc, so I made an extremely dumb .spec to fix it. That's when it occurred to me, rpm-ostree already has a configuration language for system mutations: ad-hoc .spec/RPM files. A tool, let's call it ToolX, that wanted to make changes to Another approach that could be used here is ToolX might not touch rpm-ostree at all for high-frequency changes. It would install or remove flatpaks, it would configure those by reaching into their sandbox. It could configure wallpapers, or whatever else, with dconf or qdbus. There's a myriad of ways that ToolX could approach dotfiles, it could just write the files out, or it could ask rpm-ostree to deal with it. Ultimately, I think that rpm-ostree would do better with a very minimal opinion on how to deal with this problem. It could provide a small set of tools, which stronger opinions could be built on top of. Maybe you could extend Ansible or Nix to render to an ostree layer. Maybe you could write a brand new tool for reproducing a system in its entirety. I personally believe that such high level opinions shouldn't really go into rpm-ostree. |
I'm curious any more details on what Nix does worse?
In OpenShift's MCO we use Butane and Ignition to manage changes to |
RPM has long solved these problems, and As for other lessons that don't really apply to rpm-ostree/Silverblue yet: Writing .nix files is fraught with papercuts, mostly because editor support is limited. When something goes wrong things become incredibly difficult, especially if home-manager is involved.
The problem here is the n-th boot on a daily driver. There are bold nix users who recreate their entire |
|
Yeah definitely agree there are a lot of warts.
I don't think RPMs have fully solved those problems or a lot of related ones. Scalability: I think releasing RPMs takes a lot more human intervention, whereas with Nixpkgs everything is automated after a commit is merged Flakes: my impression is if you have a really simple project in a GitHub repo, packaging and distributing an RPM takes a good amount of effort. Whereas with a Nix flake you could just drop a trivial flake.nix in your repo.
Yeah I should have mentioned Ignition is getting better at the non-first boot case: coreos/ignition#1285. Not sure if it's quite to the point where it is the tool you're asking for, but if I understand you correctly I think it might be becoming that tool |
|
A lot of work has been done to transform the origin code as a thin layer on top of treefiles, so I think this isn't very far now. The next step I think is still to fully delete |
|
Is there any progress on this? But a simple yaml (Or any other file format) text file would be awesome when it comes to system setup. |
|
Update? |
I'd like to support adding non-RPM content client side too, building on what we have in the treefile (server side) today:
(And maybe
ostree-override-layers)A long time ago in the distant past I totally made up this concept of the "origin" in ostree which uses a keyfile stored outside of the deployment. One thought behind that was that if it was stored inside the deployment it'd be "circular".
I now think it really should have been
/etc/ostree/origin.confor something.In other words, we should have supported directly editing a config file instead of going via DBus for everything.
Also related to this, if we do this the file should be in a nicer format which is probably YAML.
OK now in rpm-ostree we have a lot more sophistication around what we support; I don't think we change ostree at first.
But basically imagine that we support something similar to NixOS /etc/nixos/configuration.nix.
/etc/rpm-ostree/configuration.d strawman
Let's take a subset of the treefile in YAML form as a baseline format. Base keys supported from both would be:
packages ostree-layers. We'd also havepackage-overridesetc. that are only in the client side origin file today.Let's say we support multiple drop-in files; that way e.g. the OpenShift MCO could just write a single file
/etc/rpm-ostree/configuration.d/machine-config-daemon.yaml. (OK well it's messier than that with local package overrides). But more generally you could imagine that e.g. different Ansible playbooks would write separate drop-ins.Then there's a single
rpm-ostree rebuildflag (much likenixos-rebuild switch) that would generate a new deployment based on the changes in/etc. (This would be distinct fromrpm-ostree upgradewhich would actually check any upstream ostree/rpm-md repos for new content; we'd wantrpm-ostree rebuild --upgradeto do both)Now another big thing we do here is ensure that client side changes are captured in the target commit. IOW we should version these configurations so that if e.g. someone messes up content in
/etc/rpm-ostree/configuration.dit's easy to just dorpm-ostree reset --booted-configurationwhich would replace the current copy of/etc/rpm-ostree/configuration.dwith the (immutable) ones from the booted deployment.The text was updated successfully, but these errors were encountered: