Investigate Nix on CloudABI / WASI

[Ericson2314]

https://cloudabi.org/ is an interesting idea for bolting cap-safety onto legacy kernels. It would be very interesting to see a port of Nix to this syscall interface. In particular, I'd hope this would allow for very low-overhead sandboxed derivations, as much of the cost in the past has been manipulating the ambient namespaces which this avoids entirely.

CC @shlevy

Edit: Added WASI, which basically CloudABI + Web Assembly

[7c6f434c]

Looking at the CloudABI introduction, it seems to have a different notion of running a program compared to, say POSIX (structured environment instead of command-line arguments, and capability token passing). I am not sure when, and if, a C compiler that would itself work under such rules will appear.

[Ericson2314]

@7c6f434c Actually it's very much designed to be conservative enough that existing software can be shoehorned to work. See https://github.com/NuxiNL/cloudabi-ports; the first step is getting cross compilation to Cloud ABI in Nixpkgs working, cribbing from those bsd-style ports as necessary (should make another issue in Nixpkgs for that). Nix would be a more interesting port, but that's kind of the point :).

https://lwn.net/Articles/674770/ goes into more detail of how things work in the Linux version. Also compare the earlier Capsicum from Academia. (Still not sure to what degree the implementation is shared, but the article sort of hints at it.)

[7c6f434c]

@7c6f434c Actually it's very much designed to be conservative enough that existing software can be shoehorned to work. See https://github.com/NuxiNL/cloudabi-ports; the first step is getting cross compilation to Cloud ABI in Nixpkgs working, cribbing from those bsd-style ports as necessary (should make another issue in Nixpkgs for that). Nix would be a more interesting port, but that's kind of the point :).

Ah indeed, as a cross-compilation target it is interesting. For Nix itself — I guess it is telling that they have cmake but not gnumake?

[Ericson2314]

@7c6f434c Yeah it would probably be easiest to cross compile Nix, and that Nix wouldn't be able to build most packages in Nixpkgs. But it would be really cool for experimental incremental software using the Nix daemon as an incremental cache. (e.g. compiler that forks off fine grained tasks as derivations. It would take a more radically redesigned Nix to make that "fork" maximally performant, but a basic demo of the concept can be made with today's Nix.)

[Ericson2314]

CC @cleverca22, I feel like all this is vaguely up your alley.

[Ericson2314]

Opened NixOS/nixpkgs#59285 for the Nix side of things.

[matthewbauer]

I think we would get more hype/interest if we worked towards using WebAssembly/WASI. It’s got more people behind it, but also incorporates lots of the ideas behind CloudABI:

https://github.com/CraneStation/wasi-sysroot

I need to figure out how to bootstrap libc++ with LLVM before we can do that many interesting things with WebAssembly though.

[Ericson2314]

Thanks for this! https://github.com/CraneStation/wasmtime/blob/master/docs/WASI-overview.md mentions CloudABI a lot, so it makes sense to see this on a spectrum (WASM supports safer models than OS-sandboxed Cish).

[shlevy]

I'm not sure I see the point of this one. At the end of the day most of the work needs to happen with access to global namespaces, because at the end of the day everything still sucks and we're stuck with the build systems as they actually are.

[Ericson2314]

@shlevy Everything indeed still sucks but:

1. I'm mainly interested in doing this for sake of greenfield stuff on top, which can be right from the get go. Let this be the amphibian egg for some layer above chicken for a reinvented Nix chicken egg :).

2. There's no security problem with creating a legacy unix sandbox from a cap-safe context. Were that an extension to WASI/CloudABI, we could shift all global namespace management from Nix to the derivations themselves, analogous to buildFHSUserEnv for building jank software.

[stale]

I marked this as stale due to inactivity. → More info

[stale]

I closed this issue due to inactivity. → More info

[Ericson2314]

Still interested.

[armeenm]

Bump.

[Ericson2314]

Don't worry it won't be closed by stale bot again :)