cgroup2: does not work with rootless podman

[AkihiroSuda]

$ podman --cgroup-manager=cgroupfs  run -it --rm --runtime=/usr/local/sbin/runc  docker.io/library/alpine  
Error: time="2019-10-30T00:13:24+09:00" level=warning msg="signal: killed"
time="2019-10-30T00:13:24+09:00" level=error msg="container_linux.go:346: starting container process caused \"process_linux.go:297: applying cgroup configuration for process caused \\\"mkdir /sys/fs/cgroup/libpod_parent/libpod-00b78ede55a4d847e53ddfe1e4c1b5498db0175ab22339705883f6b0dc136734: permission denied\\\"\""
container_linux.go:346: starting container process caused "process_linux.go:297: applying cgroup configuration for process caused \"mkdir /sys/fs/cgroup/libpod_parent/libpod-00b78ede55a4d847e53ddfe1e4c1b5498db0175ab22339705883f6b0dc136734: permission denied\"": OCI runtime permission denied error

$ podman --cgroup-manager=systemd  run -it --rm --runtime=/usr/local/sbin/runc  docker.io/library/alpine
Error: time="2019-10-30T00:13:31+09:00" level=error msg="systemd cgroup flag passed, but systemd support for managing cgroups is not available"
systemd cgroup flag passed, but systemd support for managing cgroups is not available: OCI runtime error

Rootful mode seems working without problem (both cgroupfs and systemd).

• runc: d239ca8
• Podman: containers/podman@a56131f
• Host: Ubuntu 19.10, kernel 5.3.0-19-generic #20-Ubuntu with systemd.unified_cgroup_hierarchy=1 cgroup_enable=memory swapaccount=1

[AkihiroSuda]

$ cat /proc/self/cgroup 
0::/user.slice/user-1001.slice/session-1.scope

$ podman --cgroup-manager=cgroupfs run --name=foo -d --runtime=/usr/local/bin/crun docker.io/library/alpine tail -f /dev/null
d81f9a37b4569d30bfcd08f3339398c1eba21f44ca617f95212559527a969145

$ podman exec foo cat /proc/1/cgroup
0::/user.slice/user-1001.slice/user@1001.service/user.slice/podman-11242.scope

$ podman inspect foo | jq -r .[0].OCIConfigPath | xargs jq -r  .linux.cgroupsPath
/libpod_parent/libpod-d81f9a37b4569d30bfcd08f3339398c1eba21f44ca617f95212559527a969145

@giuseppe How is this happening?

[giuseppe]

I think that runc is trying to connect to the systemd system instance. For rootless, it should connect to the user session.

We had the same issue with Podman but it was easy to retrieve the original UID for the rootless user: https://github.com/containers/libpod/blob/e7540d0406c49b22de245246d16ebc6e1778df37/pkg/cgroups/cgroups.go#L374-L392

but I am not sure how to retrieve the uid while in a namespace. We could either parse /proc/self/uid_map or look it up from os.Getenv("USER")

[AkihiroSuda]

Why does --cgroup-manager=cgroupfs use systemd?

[giuseppe]

Why does --cgroup-manager=cgroupfs use systemd?

when running as rootless, if it is not able to create a cgroup using cgroupfs and no limits are set, then it silently ignore errors and use the same cgroups podman was running in. It is the same behaviour Podman has on a cgroups v1 system where cgroups for rootless mode are not supported at all.

[AkihiroSuda]

Hmm,

• podman executes crun in cgroup /user.slice/user-1001.slice/user@1001.service/user.slice/podman-11242.scope via systemd, when systemd is available, regardless to the value of --cgroup-manager arg.
• When --cgroup-manager is cgroupfs, podman specifies /libpod_parent/libpod-d81f9a37b4569d30bfcd08f3339398c1eba21f44ca617f95212559527a969145 as the .linux.cgroupsPath in config.json
• But crun ignores .linux.cgroupsPath=/libpod_parent/libpod-d81f9a37b4569d30bfcd08f3339398c1eba21f44ca617f95212559527a969145 because of EPERM. So the container is executed in /user.slice/user-1001.slice/user@1001.service/user.slice/podman-11242.scope

Is my understanding correct?

[AkihiroSuda]

cgroupfs is being fixed in #2169

[AkihiroSuda]

systemd is being fixed in #2281