Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Carry 2535] rootless: support detach-netns mode #2723

Merged
merged 4 commits into from
Jan 31, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -158,31 +158,40 @@ jobs:
include:
- ubuntu: 20.04
containerd: v1.6.26
rootlesskit: v1.1.1
target: test-integration-rootless
- ubuntu: 20.04
containerd: v1.7.11
rootlesskit: v2.0.0
target: test-integration-rootless
- ubuntu: 22.04
containerd: v1.7.11
rootlesskit: v1.1.1
target: test-integration-rootless
- ubuntu: 22.04
containerd: main
rootlesskit: v2.0.0
target: test-integration-rootless
- ubuntu: 20.04
containerd: v1.6.26
rootlesskit: v1.1.1
target: test-integration-rootless-port-slirp4netns
- ubuntu: 20.04
containerd: v1.7.11
rootlesskit: v2.0.0
target: test-integration-rootless-port-slirp4netns
- ubuntu: 22.04
containerd: v1.7.11
rootlesskit: v1.1.1
target: test-integration-rootless-port-slirp4netns
- ubuntu: 22.04
containerd: main
rootlesskit: v2.0.0
target: test-integration-rootless-port-slirp4netns
env:
UBUNTU_VERSION: "${{ matrix.ubuntu }}"
CONTAINERD_VERSION: "${{ matrix.containerd }}"
ROOTLESSKIT_VERSION: "${{ matrix.rootlesskit }}"
TEST_TARGET: "${{ matrix.target }}"
steps:
- uses: actions/checkout@v4.1.1
Expand All @@ -191,7 +200,7 @@ jobs:
- name: "Register QEMU (tonistiigi/binfmt)"
run: docker run --privileged --rm tonistiigi/binfmt --install all
- name: "Prepare (network driver=slirp4netns, port driver=builtin)"
run: DOCKER_BUILDKIT=1 docker build -t ${TEST_TARGET} --target ${TEST_TARGET} --build-arg UBUNTU_VERSION=${UBUNTU_VERSION} --build-arg CONTAINERD_VERSION=${CONTAINERD_VERSION} .
run: DOCKER_BUILDKIT=1 docker build -t ${TEST_TARGET} --target ${TEST_TARGET} --build-arg UBUNTU_VERSION=${UBUNTU_VERSION} --build-arg CONTAINERD_VERSION=${CONTAINERD_VERSION} --build-arg ROOTLESSKIT_VERSION=${ROOTLESSKIT_VERSION} .
- name: "Test (network driver=slirp4netns, port driver=builtin)"
run: docker run -t --rm --privileged -e WORKAROUND_ISSUE_622=1 ${TEST_TARGET}

Expand Down
7 changes: 4 additions & 3 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -23,13 +23,13 @@ ARG RUNC_VERSION=v1.1.11
ARG CNI_PLUGINS_VERSION=v1.4.0

# Extra deps: Build
ARG BUILDKIT_VERSION=v0.12.4
ARG BUILDKIT_VERSION=v0.13.0-beta2
# Extra deps: Lazy-pulling
ARG STARGZ_SNAPSHOTTER_VERSION=v0.15.1
# Extra deps: Encryption
ARG IMGCRYPT_VERSION=v1.1.9
# Extra deps: Rootless
ARG ROOTLESSKIT_VERSION=v1.1.1
ARG ROOTLESSKIT_VERSION=v2.0.0
ARG SLIRP4NETNS_VERSION=v1.2.2
# Extra deps: bypass4netns
ARG BYPASS4NETNS_VERSION=v0.3.0
Expand Down Expand Up @@ -151,7 +151,8 @@ RUN fname="buildkit-${BUILDKIT_VERSION}.${TARGETOS:-linux}-${TARGETARCH:-amd64}.
curl -o "${fname}" -fSL "https://github.com/moby/buildkit/releases/download/${BUILDKIT_VERSION}/${fname}" && \
grep "${fname}" "/SHA256SUMS.d/buildkit-${BUILDKIT_VERSION}" | sha256sum -c && \
tar xzf "${fname}" -C /out && \
rm -f "${fname}" /out/bin/buildkit-qemu-* /out/bin/buildkit-runc && \
rm -f "${fname}" /out/bin/buildkit-qemu-* /out/bin/buildkit-cni-* /out/bin/buildkit-runc && \
for f in /out/libexec/cni/*; do ln -s ../libexec/cni/$(basename $f) /out/bin/buildkit-cni-$(basename $f); done && \
echo "- BuildKit: ${BUILDKIT_VERSION}" >> /out/share/doc/nerdctl-full/README.md
# NOTE: github.com/moby/buildkit/examples/systemd is not included in BuildKit v0.8.x, will be included in v0.9.x
RUN cd /out/lib/systemd/system && \
Expand Down
2 changes: 0 additions & 2 deletions Dockerfile.d/SHA256SUMS.d/buildkit-v0.12.4

This file was deleted.

2 changes: 2 additions & 0 deletions Dockerfile.d/SHA256SUMS.d/buildkit-v0.13.0-beta2
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
c089e8c4d5765479547c04070676c3803956b98b20b0aa07de671f8ceb4521eb buildkit-v0.13.0-beta2.linux-amd64.tar.gz
a1228cd8d5fbb0d1e150f6ab7690dad9448fc63d99ec84b919d169b4f449a9ee buildkit-v0.13.0-beta2.linux-arm64.tar.gz
6 changes: 6 additions & 0 deletions Dockerfile.d/SHA256SUMS.d/rootlesskit-v2.0.0
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
c24c13b0a7d74ea2ce9c44ca13810844149c93b5d193a08bf21166d08706621c rootlesskit-aarch64.tar.gz
da79e531f38688aad050c93e0c4d2b1f33e888e9dc6301f9866eabfec93bc75f rootlesskit-armv7l.tar.gz
181e9ff0c9ee0286b7b5384893b5dc6498098eee7014de253f053e3b50fdcbdb rootlesskit-ppc64le.tar.gz
709527301c6c4046cbe0ef5043a866ca7cfd105d91dee4a49ef3c85c8e57de5a rootlesskit-riscv64.tar.gz
28a8d3eb8eb6fc49cba4819a3a74be6319e7060707e37ca02425107ed65c034e rootlesskit-s390x.tar.gz
8205e3f96ca9eb576a0b182455fb5ad1067bf019a7fe50a1816f6c04b581723f rootlesskit-x86_64.tar.gz
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ In addition to containerd, the following components should be installed:
- [BuildKit](https://github.com/moby/buildkit) (OPTIONAL): for using `nerdctl build`. BuildKit daemon (`buildkitd`) needs to be running. See also [the document about setting up BuildKit](./docs/build.md).
- v0.11.0 or later is highly recommended. Some features, such as pruning caches with `nerdctl system prune`, do not work with older versions.
- [RootlessKit](https://github.com/rootless-containers/rootlesskit) and [slirp4netns](https://github.com/rootless-containers/slirp4netns) (OPTIONAL): for [Rootless mode](./docs/rootless.md)
- RootlessKit needs to be v0.10.0 or later. v0.14.1 or later is recommended.
- RootlessKit needs to be v0.10.0 or later. v2.0.0 or later is recommended.
- slirp4netns needs to be v0.4.0 or later. v1.1.7 or later is recommended.

These dependencies are included in `nerdctl-full-<VERSION>-<OS>-<ARCH>.tar.gz`, but not included in `nerdctl-<VERSION>-<OS>-<ARCH>.tar.gz`.
Expand Down
5 changes: 5 additions & 0 deletions cmd/nerdctl/container_run_network_linux_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -494,6 +494,11 @@ func TestRunContainerWithMACAddress(t *testing.T) {
}

func TestHostsFileMounts(t *testing.T) {
if rootlessutil.IsRootless() {
if detachedNetNS, _ := rootlessutil.DetachedNetNS(); detachedNetNS != "" {
t.Skip("/etc/hosts is not writable")
}
}
base := testutil.NewBase(t)

base.Cmd("run", "--rm", testutil.CommonImage,
Expand Down
23 changes: 15 additions & 8 deletions cmd/nerdctl/ipfs_linux_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -73,14 +73,21 @@ func TestIPFSAddress(t *testing.T) {

func runIPFSDaemonContainer(t *testing.T, base *testutil.Base) (ipfsAddress string, done func()) {
name := "test-ipfs-address"
base.Cmd("run", "-d", "--name", name, "--entrypoint=/bin/sh", testutil.KuboImage, "-c", "ipfs init && ipfs config Addresses.API /ip4/0.0.0.0/tcp/5001 && ipfs daemon --offline").AssertOK()
iplines := base.Cmd("inspect", name, "-f", "'{{json .NetworkSettings.IPAddress}}'").OutLines()
t.Logf("IPAddress=%v", iplines)
assert.Equal(t, len(iplines), 2)
matches := iplineRegexp.FindStringSubmatch(iplines[0])
t.Logf("ip address matches=%v", matches)
assert.Equal(t, len(matches), 2)
ipfsaddr := fmt.Sprintf("/ip4/%s/tcp/5001", matches[1])
var ipfsaddr string
if detachedNetNS, _ := rootlessutil.DetachedNetNS(); detachedNetNS != "" {
// detached-netns mode can't use .NetworkSettings.IPAddress, because the daemon and CNI has different network namespaces
base.Cmd("run", "-d", "-p", "127.0.0.1:5999:5999", "--name", name, "--entrypoint=/bin/sh", testutil.KuboImage, "-c", "ipfs init && ipfs config Addresses.API /ip4/0.0.0.0/tcp/5999 && ipfs daemon --offline").AssertOK()
ipfsaddr = "/ip4/127.0.0.1/tcp/5999"
} else {
base.Cmd("run", "-d", "--name", name, "--entrypoint=/bin/sh", testutil.KuboImage, "-c", "ipfs init && ipfs config Addresses.API /ip4/0.0.0.0/tcp/5001 && ipfs daemon --offline").AssertOK()
iplines := base.Cmd("inspect", name, "-f", "'{{json .NetworkSettings.IPAddress}}'").OutLines()
t.Logf("IPAddress=%v", iplines)
assert.Equal(t, len(iplines), 2)
matches := iplineRegexp.FindStringSubmatch(iplines[0])
t.Logf("ip address matches=%v", matches)
assert.Equal(t, len(matches), 2)
ipfsaddr = fmt.Sprintf("/ip4/%s/tcp/5001", matches[1])
}
return ipfsaddr, func() {
base.Cmd("kill", "test-ipfs-address").AssertOK()
base.Cmd("rm", "test-ipfs-address").AssertOK()
Expand Down
30 changes: 30 additions & 0 deletions docs/rootless.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,36 @@ $ nerdctl run -it --rm -p 8080:80 --label nerdctl/bypass4netns=true alpine

More detail is available at [https://github.com/rootless-containers/bypass4netns/blob/master/README.md](https://github.com/rootless-containers/bypass4netns/blob/master/README.md)

## Configuring RootlessKit

Rootless containerd recognizes the following environment variables to configure the behavior of [RootlessKit](https://github.com/rootless-containers/rootlesskit):

* `CONTAINERD_ROOTLESS_ROOTLESSKIT_STATE_DIR=DIR`: the rootlesskit state dir. Defaults to `$XDG_RUNTIME_DIR/containerd-rootless`.
* `CONTAINERD_ROOTLESS_ROOTLESSKIT_NET=(slirp4netns|vpnkit|lxc-user-nic)`: the rootlesskit network driver. Defaults to "slirp4netns" if slirp4netns (>= v0.4.0) is installed. Otherwise defaults to "vpnkit".
* `CONTAINERD_ROOTLESS_ROOTLESSKIT_MTU=NUM`: the MTU value for the rootlesskit network driver. Defaults to 65520 for slirp4netns, 1500 for other drivers.
* `CONTAINERD_ROOTLESS_ROOTLESSKIT_PORT_DRIVER=(builtin|slirp4netns)`: the rootlesskit port driver. Defaults to "builtin".
* `CONTAINERD_ROOTLESS_ROOTLESSKIT_SLIRP4NETNS_SANDBOX=(auto|true|false)`: whether to protect slirp4netns with a dedicated mount namespace. Defaults to "auto".
* `CONTAINERD_ROOTLESS_ROOTLESSKIT_SLIRP4NETNS_SECCOMP=(auto|true|false)`: whether to protect slirp4netns with seccomp. Defaults to "auto".
* `CONTAINERD_ROOTLESS_ROOTLESSKIT_DETACH_NETNS=(auto|true|false)`: whether to launch rootlesskit with the "detach-netns" mode.
Defaults to "auto", which is resolved to "true" if RootlessKit >= 2.0 is installed.
The "detached-netns" mode accelerates `nerdctl (pull|push|build)` and enables `nerdctl run --net=host`,
however, there is a relatively minor drawback with BuildKit prior to v0.13:
the host loopback IP address (127.0.0.1) and abstract sockets are exposed to Dockerfile's "RUN" instructions during `nerdctl build` (not `nerdctl run`).
The drawback is fixed in BuildKit v0.13. Upgrading from a prior version of BuildKit needs removing the old systemd unit:
`containerd-rootless-setuptool.sh uninstall-buildkit && rm -f ~/.config/buildkit/buildkitd.toml`

To set these variables, create `~/.config/systemd/user/containerd.service.d/override.conf` as follows:
```ini
[Service]
Environment=CONTAINERD_ROOTLESS_ROOTLESSKIT_DETACH_NETNS="false"
```

And then run the following commands:
```bash
systemctl --user daemon-reload
systemctl --user restart containerd
```

## Troubleshooting

### Hint to Fedora users
Expand Down
36 changes: 21 additions & 15 deletions extras/rootless/containerd-rootless-setuptool.sh
Original file line number Diff line number Diff line change
Expand Up @@ -152,8 +152,13 @@ propagate_env_from() {
cmd_entrypoint_nsenter() {
# No need to call init()
pid=$(cat "$XDG_RUNTIME_DIR/containerd-rootless/child_pid")
n=""
# If RootlessKit is running with `--detach-netns` mode, we do NOT enter the detached netns here
if [ ! -e "$XDG_RUNTIME_DIR/containerd-rootless/netns" ]; then
n="-n"
fi
fahedouch marked this conversation as resolved.
Show resolved Hide resolved
propagate_env_from "$pid" ROOTLESSKIT_STATE_DIR ROOTLESSKIT_PARENT_EUID ROOTLESSKIT_PARENT_EGID
exec nsenter --no-fork --wd="$(pwd)" --preserve-credentials -m -n -U -t "$pid" -- "$@"
exec nsenter --no-fork --wd="$(pwd)" --preserve-credentials -m $n -U -t "$pid" -- "$@"
}

show_systemd_error() {
Expand Down Expand Up @@ -265,14 +270,19 @@ cmd_entrypoint_install_buildkit() {
ERROR "Install containerd first (\`$ARG0 install\`)"
exit 1
fi
BUILDKITD_FLAG="--oci-worker=true --oci-worker-rootless=true --containerd-worker=false"
if buildkitd --help | grep -q bridge; then
# Available since BuildKit v0.13
BUILDKITD_FLAG="${BUILDKITD_FLAG} --oci-worker-net=bridge"
fi
cat <<-EOT | install_systemd_unit "${SYSTEMD_BUILDKIT_UNIT}"
[Unit]
Description=BuildKit (Rootless)
PartOf=${SYSTEMD_CONTAINERD_UNIT}

[Service]
Environment=PATH=$BIN:/sbin:/usr/sbin:$PATH
ExecStart="$REALPATH0" nsenter buildkitd
ExecStart="$REALPATH0" nsenter -- buildkitd ${BUILDKITD_FLAG}
ExecReload=/bin/kill -s HUP \$MAINPID
RestartSec=2
Restart=always
Expand All @@ -291,23 +301,12 @@ cmd_entrypoint_install_buildkit_containerd() {
ERROR "buildkitd (https://github.com/moby/buildkit) needs to be present under \$PATH"
exit 1
fi
if [ ! -f "${XDG_CONFIG_HOME}/buildkit/buildkitd.toml" ]; then
mkdir -p "${XDG_CONFIG_HOME}/buildkit"
cat <<-EOF >"${XDG_CONFIG_HOME}/buildkit/buildkitd.toml"
[worker.oci]
enabled = false

[worker.containerd]
enabled = true
rootless = true
EOF
fi
if ! systemctl --user --no-pager status "${SYSTEMD_CONTAINERD_UNIT}" >/dev/null 2>&1; then
ERROR "Install containerd first (\`$ARG0 install\`)"
exit 1
fi
UNIT_NAME=${SYSTEMD_BUILDKIT_UNIT}
BUILDKITD_FLAG=
BUILDKITD_FLAG="--oci-worker=false --containerd-worker=true --containerd-worker-rootless=true"
if [ -n "${CONTAINERD_NAMESPACE:-}" ]; then
UNIT_NAME="${CONTAINERD_NAMESPACE}-${SYSTEMD_BUILDKIT_UNIT}"
BUILDKITD_FLAG="${BUILDKITD_FLAG} --addr=unix://${XDG_RUNTIME_DIR}/buildkit-${CONTAINERD_NAMESPACE}/buildkitd.sock --root=${XDG_DATA_HOME}/buildkit-${CONTAINERD_NAMESPACE} --containerd-worker-namespace=${CONTAINERD_NAMESPACE}"
Expand All @@ -317,6 +316,10 @@ cmd_entrypoint_install_buildkit_containerd() {
if [ -n "${CONTAINERD_SNAPSHOTTER:-}" ]; then
BUILDKITD_FLAG="${BUILDKITD_FLAG} --containerd-worker-snapshotter=${CONTAINERD_SNAPSHOTTER}"
fi
if buildkitd --help | grep -q bridge; then
# Available since BuildKit v0.13
BUILDKITD_FLAG="${BUILDKITD_FLAG} --containerd-worker-net=bridge"
fi
cat <<-EOT | install_systemd_unit "${UNIT_NAME}"
[Unit]
Description=BuildKit (Rootless)
Expand Down Expand Up @@ -521,7 +524,10 @@ cmd_entrypoint_uninstall_buildkit() {
init
uninstall_systemd_unit "${SYSTEMD_BUILDKIT_UNIT}"
INFO "This uninstallation tool does NOT remove data."
INFO "To remove data, run: \`$BIN/rootlesskit rm -rf ${XDG_DATA_HOME}/buildkit"
INFO "To remove data, run: \`$BIN/rootlesskit rm -rf ${XDG_DATA_HOME}/buildkit\`"
if [ -e "${XDG_CONFIG_HOME}/buildkit/buildkitd.toml" ]; then
INFO "You may also want to remove the daemon config: \`rm -f ${XDG_CONFIG_HOME}/buildkit/buildkitd.toml\`"
fi
}

# CLI subcommand: "uninstall-buildkit-containerd"
Expand Down
31 changes: 30 additions & 1 deletion extras/rootless/containerd-rootless.sh
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
# External dependencies:
# * newuidmap and newgidmap needs to be installed.
# * /etc/subuid and /etc/subgid needs to be configured for the current user.
# * RootlessKit (>= v0.10.0) needs to be installed. RootlessKit >= v0.14.1 is recommended.
# * RootlessKit (>= v0.10.0) needs to be installed. RootlessKit >= v2.0.0 is recommended.
# * Either one of slirp4netns (>= v0.4.0), VPNKit, lxc-user-nic needs to be installed. slirp4netns >= v1.1.7 is recommended.
#
# Recognized environment variables:
Expand All @@ -38,6 +38,15 @@
# * CONTAINERD_ROOTLESS_ROOTLESSKIT_PORT_DRIVER=(builtin|slirp4netns): the rootlesskit port driver. Defaults to "builtin".
# * CONTAINERD_ROOTLESS_ROOTLESSKIT_SLIRP4NETNS_SANDBOX=(auto|true|false): whether to protect slirp4netns with a dedicated mount namespace. Defaults to "auto".
# * CONTAINERD_ROOTLESS_ROOTLESSKIT_SLIRP4NETNS_SECCOMP=(auto|true|false): whether to protect slirp4netns with seccomp. Defaults to "auto".
# * CONTAINERD_ROOTLESS_ROOTLESSKIT_DETACH_NETNS=(auto|true|false): whether to launch rootlesskit with the "detach-netns" mode.
# Defaults to "auto", which is resolved to "true" if RootlessKit >= 2.0 is installed.
# The "detached-netns" mode accelerates `nerdctl (pull|push|build)` and enables `nerdctl run --net=host`,
# however, there is a relatively minor drawback with BuildKit prior to v0.13:
# the host loopback IP address (127.0.0.1) and abstract sockets are exposed to Dockerfile's "RUN" instructions during `nerdctl build` (not `nerdctl run`).
# The drawback is fixed in BuildKit v0.13. Upgrading from a prior version of BuildKit needs removing the old systemd unit:
# `containerd-rootless-setuptool.sh uninstall-buildkit && rm -f ~/.config/buildkit/buildkitd.toml`

# See also: https://github.com/containerd/nerdctl/blob/main/docs/rootless.md#configuring-rootlesskit

set -e
if ! [ -w $XDG_RUNTIME_DIR ]; then
Expand Down Expand Up @@ -69,6 +78,7 @@ if [ -z $_CONTAINERD_ROOTLESS_CHILD ]; then
: "${CONTAINERD_ROOTLESS_ROOTLESSKIT_PORT_DRIVER:=builtin}"
: "${CONTAINERD_ROOTLESS_ROOTLESSKIT_SLIRP4NETNS_SANDBOX:=auto}"
: "${CONTAINERD_ROOTLESS_ROOTLESSKIT_SLIRP4NETNS_SECCOMP:=auto}"
: "${CONTAINERD_ROOTLESS_ROOTLESSKIT_DETACH_NETNS:=auto}"
net=$CONTAINERD_ROOTLESS_ROOTLESSKIT_NET
mtu=$CONTAINERD_ROOTLESS_ROOTLESSKIT_MTU
if [ -z $net ]; then
Expand Down Expand Up @@ -107,6 +117,25 @@ if [ -z $_CONTAINERD_ROOTLESS_CHILD ]; then
export _CONTAINERD_ROOTLESS_SELINUX
fi
fi

case "$CONTAINERD_ROOTLESS_ROOTLESSKIT_DETACH_NETNS" in
auto)
if rootlesskit --help | grep -qw -- "--detach-netns"; then
CONTAINERD_ROOTLESS_ROOTLESSKIT_FLAGS=--detach-netns $CONTAINERD_ROOTLESS_ROOTLESSKIT_FLAGS
AkihiroSuda marked this conversation as resolved.
Show resolved Hide resolved
fi
;;
1 | true)
CONTAINERD_ROOTLESS_ROOTLESSKIT_FLAGS=--detach-netns $CONTAINERD_ROOTLESS_ROOTLESSKIT_FLAGS
;;
0 | false)
# NOP
;;
*)
echo "Unknown CONTAINERD_ROOTLESS_ROOTLESSKIT_DETACH_NETNS value: $CONTAINERD_ROOTLESS_ROOTLESSKIT_DETACH_NETNS"
exit 1
;;
esac

# Re-exec the script via RootlessKit, so as to create unprivileged {user,mount,network} namespaces.
#
# --copy-up allows removing/creating files in the directories by creating tmpfs and symlinks
Expand Down
24 changes: 24 additions & 0 deletions pkg/cmd/container/create.go
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ import (
"github.com/containerd/nerdctl/v2/pkg/namestore"
"github.com/containerd/nerdctl/v2/pkg/platformutil"
"github.com/containerd/nerdctl/v2/pkg/referenceutil"
"github.com/containerd/nerdctl/v2/pkg/rootlessutil"
"github.com/containerd/nerdctl/v2/pkg/strutil"
dockercliopts "github.com/docker/cli/opts"
dockeropts "github.com/docker/docker/opts"
Expand Down Expand Up @@ -418,6 +419,29 @@ func GenerateLogURI(dataStore string) (*url.URL, error) {
}

func withNerdctlOCIHook(cmd string, args []string) (oci.SpecOpts, error) {
if rootlessutil.IsRootless() {
detachedNetNS, err := rootlessutil.DetachedNetNS()
if err != nil {
return nil, fmt.Errorf("failed to check whether RootlessKit is running with --detach-netns: %w", err)
}
if detachedNetNS != "" {
// Rewrite {cmd, args} if RootlessKit is running with --detach-netns, so that the hook can gain
// CAP_NET_ADMIN in the namespaces.
// - Old:
// - cmd: "/usr/local/bin/nerdctl"
// - args: {"--data-root=/foo", "internal", "oci-hook"}
// - New:
// - cmd: "/usr/bin/nsenter"
// - args: {"-n/run/user/1000/containerd-rootless/netns", "-F", "--", "/usr/local/bin/nerdctl", "--data-root=/foo", "internal", "oci-hook"}
oldCmd, oldArgs := cmd, args
cmd, err = exec.LookPath("nsenter")
if err != nil {
return nil, err
}
args = append([]string{"-n" + detachedNetNS, "-F", "--", oldCmd}, oldArgs...)
fahedouch marked this conversation as resolved.
Show resolved Hide resolved
}
}

args = append([]string{cmd}, append(args, "internal", "oci-hook")...)
return func(_ context.Context, _ oci.Client, _ *containers.Container, s *specs.Spec) error {
if s.Hooks == nil {
Expand Down
Loading
Loading