toolbox: use rkt fly instead of systemd-nspawn

[philips]

Today toolbox uses systemd-nspawn to actual execute the machine. The downside of systemd-nspawn is that there is no propagation. So, you can't say mount something inside of toolbox or unmount something that is seen in the root mount namespace.

Lets do that. Here is a prototype: https://gist.github.com/philips/acc5d812041f7a5baa2d582626cddb2a

xref rkt mount propagation rkt/rkt#1149

[f0]

see also #1497

[dm0-]

There are a few toolbox settings that seem to expect bare systemd-nspawn command-line arguments. If we switch to rkt, users' existing configuration files using these settings won't work:

• TOOLBOX_BIND
• TOOLBOX_ENV

For TOOLBOX_ENV, I'd suggest ignoring it with rkt and adding a TOOLBOX_ENV_FILE option. Since rkt has --set-env-file, it can be used directly, but if the backend program changes again, an env file is trivial to parse and convert to whatever format is necessary.

I'd prefer to ignore TOOLBOX_BIND as well, and maybe add a generic TOOLBOX_RKT_ARGS setting for the same functionality.

There is also the TOOLBOX_DIRECTORY setting, which I believe can just be ignored since a rootfs no longer needs to be extracted for systemd-nspawn.

Any objections to these config changes?

[crawford]

I think the config changes are fine. Toolbox has always been meant to be used interactively, so these flag changes should be manageable by the operator.

[dm0-]

@philips Do you have any object to creating a new tool instead of changing toolbox as @marineam suggested in coreos/toolbox#34?

[crawford]

I don't like the idea of a new tool. Unless it provides some ground-breaking functionality, I don't see a reason to further bloat the OS. Toolbox itself is going to stop working as we upgrade systemd (they removed --share-system). We should rip the bandaid off now.

[dm0-]

@crawford They've actually just removed its documentation, but it's still there. https://github.com/systemd/systemd/blob/master/src/nspawn/nspawn.c#L443 Still, they're clearly moving towards removing it soon, so I agree we should deal with that. (I vaguely recall discussion of the CoreOS toolbox being its only user and the only reason for keeping the option.)

[lucab]

@crawford @dm0- the outcome of the discussion with systemd upstream is that --share-system will disappear at some point, but the same functionality can anyway be toggled via granular env vars, see systemd/systemd#4023. rkt also needs some of those features for pods sharing some host namespaces.

[coreosbot]

Moved to coreos/toolbox#46.