Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

filesystem: Explain more about /etc and /var #441

Merged
merged 1 commit into from
Mar 26, 2024
Merged

filesystem: Explain more about /etc and /var #441

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
26 changes: 26 additions & 0 deletions docs/src/building/guidance.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,32 @@ make in the container image to e.g. `/etc/postgresql.conf`
will be applied on update, assuming it is not modified
locally.

### Prefer using drop-in directories

These "locally modified" files can be a source of state drift. The best
pattern to use is "drop-in" directories that are merged dynamically by
the relevant software. systemd supports this comprehensively; see
[drop-ins](https://www.freedesktop.org/software/systemd/man/latest/systemd.unit.html)
for example in units.

And instead of modifying `/etc/sudoers.conf`, it's best practice to add
a file into `/etc/sudoers.d` for example.

Not all software supports this, however; and this is why there
is generic support for `/etc`.

### Configuration in /usr vs /etc

Some software supports generic configuration both `/usr` and `/etc` - systemd,
among others. Because bootc supports *derivation* (the way OCI
containers work) - it is supported and encourged to put configuration
files in `/usr` (instead of `/etc`) where possible, because then
the state is consistently immutable.

One pattern is to replace a configuration file like
`/etc/postgresql.conf` with a symlink to e.g. `/usr/postgres/etc/postgresql.conf`
for example, although this can run afoul of SELinux labeling.

### Secrets

There is a dedicated document for [secrets](secrets.md),
Expand Down
36 changes: 31 additions & 5 deletions docs/src/filesystem.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,22 +22,48 @@ The overall recommendation is to keep all operating system content in `/usr`. S

## `/etc`

The `/etc` directory contains persistent state by default; however,
The `/etc` directory contains mutable persistent state by default; however,
it is suppported to enable the [`etc.transient` config option](https://ostreedev.github.io/ostree/man/ostree-prepare-root.html).

When in persistent mode, it inherits the OSTree semantics of [performing a 3-way merge](https://ostreedev.github.io/ostree/atomic-upgrades/#assembling-a-new-deployment-directory)
across upgrades.
across upgrades. In a nutshell:

- The *new default* `/etc` is used as a base
- The diff between current and previous `/etc` is applied to the new `/etc`
cgwalters marked this conversation as resolved.
Show resolved Hide resolved
- Locally modified files in `/etc` different from the default `/usr/etc` (of the same deployment) will be retained

The implmentation of this defaults to being executed by `ostree-finalize-staged.service`
at shutdown time, before the new bootloader entry is created.

The rationale for this design is that in practice today, many components of a Linux system end up shipping
default configuration files in `/etc`. And even if the default package doesn't, often the software
only looks for config files there by default.

Some other image-based update systems do not have distinct "versions" of `/etc` and
it may be populated only set up at a install time, and untouched thereafter. But
that creates "hysteresis" where the state of the system's `/etc` is strongly
influenced by the initial image version. This can lead to problems
where e.g. a change to `/etc/sudoers.conf` (to give on simple example)
would require external intervention to apply.

For more on configuration file best practices, see [Building](building/guidance.md).

## `/var`

Content in `/var` persists by default; it is however supported to make it or subdirectories
mount points (whether network or `tmpfs`)
mount points (whether network or `tmpfs`). There is exactly one `/var`. If it is
not a distinct partition, then "physically" currently it is a bind mount into
`/ostree/deploy/$stateroot/var` and shared across "deployments" (bootloader entries).

As of OSTree v2024.3, by default [content in /var acts like a Docker VOLUME /var](https://github.com/ostreedev/ostree/pull/3166/commits/f81b9fa1666c62a024d5ca0bbe876321f72529c7).

This means that the content from the container image is copied at initial installation time, and *not updated thereafter*.

The rationale for this is to keep operating system upgrades from touching machine-local data by default.
Note this is very different from the handling of `/etc`. The rationale for this is
that `/etc` should generally only hold small text files, but `/var` should hold arbitrarily
large data (system logs, databases, etc.). Creating multiple copies of it would have

to keep operating system upgrades from touching machine-local data by default.
If the system is rolled back to a previous bootloader entry, the `/var` content remains. This also
makes it possible to "stage" new operating system updates in an alternative root without affecting `/var` content.

Expand Down Expand Up @@ -65,7 +91,7 @@ cases, there are several options (containerizing the app, running it in a system
However, some use cases may find it easier to enable a fully transient writable rootfs by default.
To do this, set the

```
```toml
[root]
transient = true
```
Expand Down