feat: add CPIO packing for companion files #168

RaitoBezarius · 2023-04-30T14:56:18Z

~~Depended on #167 #166.~~

This adds CPIO packing and open credentials and systemd extensions discovery and load.

With this, we should be able to read from drop-in directories, load CPIO archives and combine them as initrds for sysexts (+ measurements).

blitz · 2023-05-04T09:40:09Z

In what situation do we have to create initrds? You only mentioned appending them and cpios can be appended without repacking them.

RaitoBezarius · 2023-05-04T13:18:20Z

In what situation do we have to create initrds? You only mentioned appending them and cpios can be appended without repacking them.

Companion files are not initrd, they are just random files laying around on the ESP.

Therefore, to make them available in your initrd phase, you need to pack them as CPIOs and merge them with the others initrds, hence the need for a CPIO packing phase.

(They are not CPIO on their own at the start, otherwise, it would be cumbersome to manipulate from userspace.)

rust/stub/src/cpio.rs

RaitoBezarius · 2023-05-18T17:26:38Z

@nikstur I think this CPIO stuff is the good opportunity to introduce our "library crate" for UEFI bootloaders stuff.

blitz · 2023-05-04T09:35:12Z

rust/stub/src/cpio.rs

+
+/// Compute the necessary padding based on the provided length
+/// It returns None if no padding is necessary.
+fn compute_pad4(len: usize) -> Option<Vec<u8>> {


This would be easier to handle if it returns a zero-length vector when zero padding is required. This makes the calling code easier because it removes case distinctions.

blitz · 2023-05-04T09:36:33Z

rust/stub/src/cpio.rs

+}
+
+impl Cpio {
+    fn pack_one(&mut self, fname: &CStr16, contents: &[u8], target_dir_prefix: &str, access_mode: u32) -> uefi::Result


From an architectural point, it would be great to separate the CPIO writing from any UEFI stuff. So we could split this file into one module that does only CPIO construction in memory (without using any UEFI types) and another module that adds whatever UEFI magic is required.

RaitoBezarius · 2023-11-15T03:21:37Z

I decided to be serious about the problem space, so here's my strategy.

`pio` library

Instead of forcing a CPIO writer inside the linux-bootloader crate, I decided to create a new library called pio which can be used for writing CPIO in no_std, alloc contexts. (name subject to change but like p for pico I/O :>).

It is relying on embedded-io (I/O in no_std) and snafu (for errors in no_std) and rolls its own very basic cursor type for internal purpose. It could do direct I/O via the existing I/O trait and I could move the cursor thing outside, but baby steps.

pio library is a file format library and as with all file format libraries, let's write tests. Writing UEFI tests is not easy, therefore, I decided to move the testing to userspace via cargo test --target x86_64-unknown-linux-gnu, I don't know if I can automate this well enough without xtask, but that should be the way to go IMHO.

As a development dependency, I bring a… CPIO reader (and writer) which also use the same format (newc) which I will use to perform integration testing in various fashions.

I want simple "unit tests" (alignment, read a simple CPIO, etc.), then I want to torture more and more the library as we go on.

Ideally, we could donate this to rust-osdev or whoever is interested in the maintenance at some point.

`linux-bootloader` library

This library should be responsible to articulate the UEFI APIs to pack CPIO at the high level if I give you a bunch of files via a UEFI filesystem and optionally measure it to the TPM2.

It should also offer a way to merge initrds dynamically.

lanzastub

lanzastub should be discovering system extensions and credentials via linux-bootloader library, measure them if TPM2 is available, send them to the boot linux entrypoint and call the dynamic merger for initrds.

Making this a very simple feature at lanza-level, modulo for new EFI variables to export.

End to end testing

build a way to build systemd-credential easily
use stage 1 assertions to assert about credentials or sysexts in the initrd?
a systemd credential test, put it there, load it, verify you can see the credential in initrd or userspace at least?

I think, if I do this, this PR would be in a good enough shape w.r.t. to testing scenarios.

It is now enclosed in its own subdirectory, but full of `expect`.

It is now possible to discover credentials and systemd extensions and pack them as CPIOs.

With this feature, it is now possible to load dynamic initrds (possibly read from filesystem or generated on the fly) and extend existing initrds. This feature will be useful to implement addons in the future.

RaitoBezarius · 2024-01-02T00:14:53Z

@blitz @nikstur I would appreciate a review on the credentials part, I'm not sure yet what I want to do about local credentials vs. global credentials. I feel like you can always use global credentials and local credentials are almost always useless with lanzaboote except if you use the fat UKIs model? So I suggest removing the code for them from the tool, as it does not make a lot of sense.

For sysexts, I will add more stuff soonish to test them properly.

This is better to perform integration testing and separate the cost of ownership / maintenance.

For dynamic usecases, e.g. credentials or system extension images, we have a need for dynamic merging of initrds.

RaitoBezarius · 2024-01-02T00:17:01Z

Note for reviewers: I will work to remove the fork dependency in this PR.

RaitoBezarius · 2024-01-02T00:55:33Z

Actually, I'm thinking, but I don't think systemd-sysext makes a lot of sense, just merging in /usr or /opt is almost useless for NixOS, so I would rather skip the test for this stuff and leave it as-is.

Now, lzbt can install global credentials directories passed inside the ESP and manage the lifecycle of it in a basic fashion.

As soon as we open in exclusive our own image, we will not be able easily to re-open it because of the locking mechanism. Therefore, we should use the opportunity to get information we may be interested in and avoid us re-opening the loaded image protocol. This is useful to know from where you were loaded for example, e.g. filesystem path.

Now the lanzaboote module can understand passed list of credentials, either global or local and pass them to `lzbt` CLI.

Create a simple test to evaluate credentials mechanism with lanzaboote that does not even make use of TPM2 and just copy garbage around.

…ls and sysexts now systemd just has to pick them up!

And offer a way to debug CPIO representations just with your eyes, Here's a useful snippet to transform that representation into a proper binary CPIO using Python: >>> raw = "<insert the string representation with the proper escaping>" >>> open("/tmp/cpio", "wb").write(bytes(raw.replace("\\x00", "\x00"), "ascii")) Then, you can poke the cpio using `cpio -t < /tmp/cpio` or extract it and `cpio` should complain accordingly.

Until I rebase my changes and push it upstream properly.

Now, systemd is fixed, we can use credentials.

Those are leftovers that didn't fuse correctly with my past commits.

blitz · 2024-02-10T20:02:48Z

I think the separate CPIO module goes in the right direction, but it would make sense to split it off into a separate PR for reviewing.

RaitoBezarius · 2024-02-11T16:13:22Z

Also split.

RaitoBezarius requested review from blitz and nikstur April 30, 2023 14:56

RaitoBezarius mentioned this pull request Apr 30, 2023

feat: support TPM2 PCR unified sections packed as CPIOs #169

Closed

RaitoBezarius force-pushed the cpio-packing branch 2 times, most recently from ac5df5d to f7c720d Compare May 18, 2023 14:21

RaitoBezarius marked this pull request as ready for review May 18, 2023 17:16

RaitoBezarius force-pushed the cpio-packing branch from f7c720d to 1180fd3 Compare May 18, 2023 17:23

RaitoBezarius commented May 18, 2023

View reviewed changes

rust/stub/src/cpio.rs Outdated Show resolved Hide resolved

RaitoBezarius commented May 18, 2023

View reviewed changes

rust/stub/src/cpio.rs Outdated Show resolved Hide resolved

blitz reviewed Aug 25, 2023

View reviewed changes

RaitoBezarius added this to the Release 0.4.0 milestone Sep 30, 2023

RaitoBezarius force-pushed the cpio-packing branch 2 times, most recently from 89ce47b to 6ac63ff Compare October 30, 2023 12:48

RaitoBezarius mentioned this pull request Nov 5, 2023

Tracking issue: Boot security in NixOS NixOS/nixpkgs#265640

Open

RaitoBezarius force-pushed the cpio-packing branch 3 times, most recently from e9ec7a0 to 9ca2548 Compare November 15, 2023 03:13

RaitoBezarius force-pushed the cpio-packing branch 3 times, most recently from 36df0d4 to a60c471 Compare November 15, 2023 07:29

RaitoBezarius added 6 commits January 2, 2024 00:46

feat: add cpio packing for companion files

8890bf7

linux-bootloader: rework the cpio code

2a4f584

It is now enclosed in its own subdirectory, but full of `expect`.

stub/common: add credential/sysext discovery

a366f73

It is now possible to discover credentials and systemd extensions and pack them as CPIOs.

stub(*): support dynamic initrds

4a52120

With this feature, it is now possible to load dynamic initrds (possibly read from filesystem or generated on the fly) and extend existing initrds. This feature will be useful to implement addons in the future.

stub: discover credentials and system extensions and load them

634f012

stub(cargo): use personal fork of uefi-rs for development

9d16eea

RaitoBezarius force-pushed the cpio-packing branch 2 times, most recently from 6bfad96 to 97df618 Compare January 2, 2024 00:12

RaitoBezarius added 3 commits January 2, 2024 01:16

feat(cpio): move cpio archive assembling in its own library called pio

4b38f19

This is better to perform integration testing and separate the cost of ownership / maintenance.

stub: measure companion initrds

6668ee1

stub: merge dynamically initrds

31c3422

For dynamic usecases, e.g. credentials or system extension images, we have a need for dynamic merging of initrds.

RaitoBezarius force-pushed the cpio-packing branch from 97df618 to 3766712 Compare January 2, 2024 00:16

RaitoBezarius added 10 commits January 2, 2024 01:56

linux-bootloader: shuffle things around

3bbcd7d

tool(systemd): support global credentials installation

06f9a35

Now, lzbt can install global credentials directories passed inside the ESP and manage the lifecycle of it in a basic fashion.

modules/lanzaboote: support global credentials and local credentials

afd2002

Now the lanzaboote module can understand passed list of credentials, either global or local and pass them to `lzbt` CLI.

nix/tests/lanzaboote: add credentials-basic test

423ed75

Create a simple test to evaluate credentials mechanism with lanzaboote that does not even make use of TPM2 and just copy garbage around.

linux-bootloader: communicate that we theoretically support credentia…

113bd24

…ls and sysexts now systemd just has to pick them up!

stub: pin to uefi 0.25.0 fork

3289b60

Until I rebase my changes and push it upstream properly.

flake: bump to 24.05+

58722f4

Now, systemd is fixed, we can use credentials.

stub: cargo fmt it

cf5887c

Those are leftovers that didn't fuse correctly with my past commits.

RaitoBezarius force-pushed the cpio-packing branch from 3766712 to cf5887c Compare January 2, 2024 00:56

DDoSolitary mentioned this pull request Jan 13, 2024

Can I automatically run systemd-cryptenroll after bootloader is updated? #61

Open

RaitoBezarius closed this Mar 1, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add CPIO packing for companion files #168

feat: add CPIO packing for companion files #168

RaitoBezarius commented Apr 30, 2023 •

edited

Loading

blitz commented May 4, 2023

RaitoBezarius commented May 4, 2023

RaitoBezarius commented May 18, 2023

blitz May 4, 2023

blitz May 4, 2023

RaitoBezarius commented Nov 15, 2023 •

edited

Loading

RaitoBezarius commented Jan 2, 2024

RaitoBezarius commented Jan 2, 2024

RaitoBezarius commented Jan 2, 2024

blitz commented Feb 10, 2024

RaitoBezarius commented Feb 11, 2024

feat: add CPIO packing for companion files #168

feat: add CPIO packing for companion files #168

Conversation

RaitoBezarius commented Apr 30, 2023 • edited Loading

blitz commented May 4, 2023

RaitoBezarius commented May 4, 2023

RaitoBezarius commented May 18, 2023

blitz May 4, 2023

Choose a reason for hiding this comment

blitz May 4, 2023

Choose a reason for hiding this comment

RaitoBezarius commented Nov 15, 2023 • edited Loading

pio library

linux-bootloader library

lanzastub

End to end testing

RaitoBezarius commented Jan 2, 2024

RaitoBezarius commented Jan 2, 2024

RaitoBezarius commented Jan 2, 2024

blitz commented Feb 10, 2024

RaitoBezarius commented Feb 11, 2024

RaitoBezarius commented Apr 30, 2023 •

edited

Loading

RaitoBezarius commented Nov 15, 2023 •

edited

Loading

`pio` library

`linux-bootloader` library