Allow Nix to be installed inside Docker agents as root or without systemd #43

soareschen · 2020-08-17T12:58:15Z

Self hosted runners for GitHub Actions can often be run from Docker containers such as https://github.com/tcardonne/docker-github-runner. In the default setup, the agent runs as root and systemd does not work well with Docker. It is often a hassle to tweak the Docker containers to run as regular user just to support Nix, and we can't always do that when the self hosted runners are maintained by different team of people.

domenkozar · 2020-08-20T18:08:27Z

Refs #40

domenkozar · 2020-09-21T11:17:45Z

@soareschen are you able to specify an docker image? In that case it might be easier to just use NixOS/nix as the base.

dsyer · 2021-05-21T13:14:29Z

You can switch the base image using -P on the command line. But nixos/nix is unlikely to be a faithful analogue to a Github action runtime, so I'm not sure it will help ultimately. Also it doesn't work because it doesn't have some basic things installed on it (and I don't know what those things are because I'm just a user, not a Github expert, so I'm not about to just wade in and try and create my own image). For example:

$ act -P ubuntu-latest=nixos/nix -s GITHUB_TOKEN=...
[CI/FATS] 🚀  Start image=nixos/nix
[CI/FATS]   🐳  docker run image=nixos/nix platform= entrypoint=["/usr/bin/tail" "-f" "/dev/null"] cmd=[]
[CI/FATS] ⭐  Run actions/checkout@v2.3.4
[CI/FATS]   ☁  git clone 'https://github.com/actions/checkout' # ref=v2.3.4
[CI/FATS]   🐳  docker cp src=/home/dsyer/.cache/act/actions-checkout@v2.3.4/ dst=/home/dsyer/dev/kubernetes-intro/_actions/actions-checkout@v2.3.4/
| OCI runtime exec failed: exec failed: container_linux.go:367: starting container process caused: exec: "node": executable file not found in $PATH: unknown
[CI/FATS]   ❌  Failure - actions/checkout@v2.3.4
Error: exit with `FAILURE`: 126

domenkozar · 2021-05-21T13:28:01Z

I'm not familiar with self-hosted setups, would something like NixOS/nixpkgs#116775 be enough or does it need docker for runtime isolation?

catthehacker · 2021-05-21T13:34:35Z

You need nodejs installed for actions to work, certain actions require python as well.

dsyer · 2021-05-24T08:39:12Z

It seems like we need an option to run as root (as per the summary). Could it be as simple as doing a su -u nix before the installation?

domenkozar · 2021-05-25T14:52:29Z

@dsyer wouldn't you expect this action to do nothing if you're running a docker image with Nix preinstalled?

dsyer · 2021-05-25T15:19:35Z

I guess, but it doesn't - instead it barfs. I did get it working with a custom image: nektos/act#696. Not sure where to take that really - I can use it myself, but if others want the same with less hassle, is it better to ask for a change here or in act?

hiroqn · 2021-08-01T08:11:00Z

Our team is using self-hosted runner in container(https://github.com/actions-runner-controller/actions-runner-controller).

If you want to install nix to container, rewrite install-script like this.

        add_config "max-jobs = auto"
        # Allow binary caches for user
        add_config "trusted-users = root $USER"
        add_config "sandbox = false"
        add_config "require-sigs = false"

        # Nix installer flags
        installer_options=(
          --no-daemon
          --no-channel-add
          --nix-extra-conf-file /etc/nix/nix.conf
          --no-modify-profile
        )

SuperSandro2000 · 2021-08-15T14:13:44Z

I just walked into the same problem but while debugging an action with act. I think based on the error output it would be enough to conditionally enable the daemon based on the existence of the /run/systemd/system directory.

| Switching to the Daemon-based Installer
|
| ---- oh no! --------------------------------------------------------------------
| Sorry, the multi-user installation requires systemd on Linux (detected using /run/systemd/system)
|
| We'd love to help if you need it.

domenkozar · 2021-08-19T13:42:56Z

Nix support for Docker needs a lot of love.

I'm not sure I'd like to support it in current form, although I'm not against if someone opens a PR to add something like insideDocker: true setting.

SuperSandro2000 · 2021-08-23T12:12:37Z

I'm not sure I'd like to support it in current form, although I'm not against if someone opens a PR to add something like insideDocker: true setting.

I don't like that option. It would require me to change the action when running locally with act.
I instead created #89 but it still needs testing.

domenkozar · 2021-11-12T14:44:30Z

The installer now does a single-user installation if there's no systemd. Can someone report if it works now?

SuperSandro2000 · 2021-11-12T15:21:53Z

c664ef3 broke it.

 performing a single-user installation of Nix...
| directory /nix does not exist; creating it by running 'mkdir -m 0755 /nix && chown root /nix' using sudo
| copying Nix to /nix/store...........................................
| warning: the group 'nixbld' specified in 'build-users-group' does not exist
| warning: the group 'nixbld' specified in 'build-users-group' does not exist
| installing 'nix-2.4'
| error: the group 'nixbld' specified in 'build-users-group' does not exist
| /tmp/nix-binary-tarball-unpack.ZYarla9mrl/unpack/nix-2.4-x86_64-linux/install: unable to install Nix into your default profile

There are multiple entries for build-user-groups in /tmp/nix.conf which seems to be another bug but I think this is not enough.

$ cat /tmp/nix.conf
max-jobs = auto
trusted-users = root root
experimental-features = nix-command flakes
build-users-group =
max-jobs = auto
trusted-users = root root
experimental-features = nix-command flakes
build-users-group =

SuperSandro2000 · 2021-11-12T15:31:32Z

I got it working like:

sudo -E env PATH=$PATH act -r
docker exec act-Basic-evaluation-checks-tests bash
mkdir -p /etc/nix/
echo "build-users-group =" > /etc/nix/nix.conf
sh <(curl --retry 5 --retry-connrefused -L "${INPUT_INSTALL_URL:-https://nixos.org/nix/install}") --no-channel-add --darwin-use-unencrypted-nix-store-volume --nix-extra-conf-file /tmp/nix.conf

domenkozar · 2021-11-12T17:58:03Z

@SuperSandro2000 How can I reproduce it?

SuperSandro2000 · 2021-11-12T18:06:26Z

@SuperSandro2000 How can I reproduce it?

I cloned the nixpkgs repo, updated the nix-install action, opened a nix-shell -p act and run the following sudo -E env PATH=$PATH act -r and waited until it failed. You can then get into the container with sudo docker exec -it act-Basic-evaluation-checks-tests bash.

lovesegfault · 2021-11-15T05:00:12Z

You can easily reproduce this on my config

$ git clone https://github.com/lovesegfault/nix-config
$ cd nix-config
$ nix develop
$ act push

domenkozar · 2021-11-15T14:19:34Z

It seems that act doesn't set the $USER variable? https://github.com/cachix/install-nix-action/runs/4212609982?check_suite_focus=true

catthehacker · 2021-11-15T17:56:53Z

We stopped doing anything with $USER long time ago, if Docker image has it (either in image spec or in /etc/environment), then act will have it

lovesegfault · 2021-11-15T20:50:18Z

For the record, here's the error I get with the aforementioned reproducer:

| 2021-11-15 20:49:03 (12.5 MB/s) - '/tmp/nix-binary-tarball-unpack.CRijh8lRuM/nix-2.4-x86_64-linux.tar.xz' saved [28432840/28432840]
| 
| Note: a multi-user installation is possible. See https://nixos.org/nix/manual/#sect-multi-user-installation
| Warning: the flag --darwin-use-unencrypted-nix-store-volume
|          is no longer needed and will be removed in the future.
| 
| warning: installing Nix as root is not supported by this script!
| performing a single-user installation of Nix...
| directory /nix does not exist; creating it by running 'mkdir -m 0755 /nix && chown root /nix' using sudo
| copying Nix to /nix/store...........................................
| warning: the group 'nixbld' specified in 'build-users-group' does not exist
| warning: the group 'nixbld' specified in 'build-users-group' does not exist
| installing 'nix-2.4'
| error: the group 'nixbld' specified in 'build-users-group' does not exist
| /tmp/nix-binary-tarball-unpack.CRijh8lRuM/unpack/nix-2.4-x86_64-linux/install: unable to install Nix into your default profile
| child_process.js:830
|     throw err;
|     ^
| 
| Error: Command failed: /run/act/actions/cachix-install-nix-action@v15/lib/install-nix.sh
|     at checkExecSyncError (child_process.js:790:11)
|     at Object.execFileSync (child_process.js:827:15)
|     at Object.<anonymous> (/run/act/actions/cachix-install-nix-action@v15/lib/main.js:4:17)
|     at Module._compile (internal/modules/cjs/loader.js:1085:14)
|     at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
|     at Module.load (internal/modules/cjs/loader.js:950:32)
|     at Function.Module._load (internal/modules/cjs/loader.js:790:12)
|     at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:76:12)
|     at internal/main/run_main_module.js:17:47 {
|   status: 1,
|   signal: null,
|   output: [ null, null, null ],
|   pid: 37,
|   stdout: null,
|   stderr: null
| }
[ci/lints      ]   ❌  Failure - cachix/install-nix-action@v15

domenkozar · 2021-11-19T23:59:46Z

#111 fixes it and adds a test.

domenkozar added the enhancement New feature or request label Aug 20, 2020

catthehacker mentioned this issue Mar 10, 2021

Issue: Failure - Cachix install nektos/act#559

Closed

domenkozar closed this as completed Nov 19, 2021

bbigras mentioned this issue Feb 16, 2022

actions-runner-controller: install-nix.sh: line 20: USER: unbound variable #122

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow Nix to be installed inside Docker agents as root or without systemd #43

Allow Nix to be installed inside Docker agents as root or without systemd #43

soareschen commented Aug 17, 2020

domenkozar commented Aug 20, 2020

domenkozar commented Sep 21, 2020

dsyer commented May 21, 2021

domenkozar commented May 21, 2021

catthehacker commented May 21, 2021 •

edited

Loading

dsyer commented May 24, 2021

domenkozar commented May 25, 2021

dsyer commented May 25, 2021

hiroqn commented Aug 1, 2021

SuperSandro2000 commented Aug 15, 2021

domenkozar commented Aug 19, 2021

SuperSandro2000 commented Aug 23, 2021

domenkozar commented Nov 12, 2021

SuperSandro2000 commented Nov 12, 2021 •

edited

Loading

SuperSandro2000 commented Nov 12, 2021

domenkozar commented Nov 12, 2021

SuperSandro2000 commented Nov 12, 2021 •

edited

Loading

lovesegfault commented Nov 15, 2021

domenkozar commented Nov 15, 2021

catthehacker commented Nov 15, 2021 •

edited

Loading

lovesegfault commented Nov 15, 2021

domenkozar commented Nov 19, 2021

Allow Nix to be installed inside Docker agents as root or without systemd #43

Allow Nix to be installed inside Docker agents as root or without systemd #43

Comments

soareschen commented Aug 17, 2020

domenkozar commented Aug 20, 2020

domenkozar commented Sep 21, 2020

dsyer commented May 21, 2021

domenkozar commented May 21, 2021

catthehacker commented May 21, 2021 • edited Loading

dsyer commented May 24, 2021

domenkozar commented May 25, 2021

dsyer commented May 25, 2021

hiroqn commented Aug 1, 2021

SuperSandro2000 commented Aug 15, 2021

domenkozar commented Aug 19, 2021

SuperSandro2000 commented Aug 23, 2021

domenkozar commented Nov 12, 2021

SuperSandro2000 commented Nov 12, 2021 • edited Loading

SuperSandro2000 commented Nov 12, 2021

domenkozar commented Nov 12, 2021

SuperSandro2000 commented Nov 12, 2021 • edited Loading

lovesegfault commented Nov 15, 2021

domenkozar commented Nov 15, 2021

catthehacker commented Nov 15, 2021 • edited Loading

lovesegfault commented Nov 15, 2021

domenkozar commented Nov 19, 2021

catthehacker commented May 21, 2021 •

edited

Loading

SuperSandro2000 commented Nov 12, 2021 •

edited

Loading

SuperSandro2000 commented Nov 12, 2021 •

edited

Loading

catthehacker commented Nov 15, 2021 •

edited

Loading