Add bypass4netns integration #808

naoki9911 · 2022-02-14T07:09:18Z

option '--label nerdctl/bypass4netns=true' in run command enables acceleration with bypass4netns

Description

This patch is initial bypass4netns support in nerdctl.
bypass4netns(https://github.com/rootless-containers/bypass4netns) is an accelerator for rootless networking. This improves outgoing or incoming (with --publish option) networking performance.

The performance benchmark with iperf3 on Ubuntu 21.10 on Hyper-V VM is shown below.

iperf3 benchmark	without bypass4netns	with bypass4netns
container -> host	0.398 Gbps	42.2 Gbps
host -> container	20.6 Gbps	47.4 Gbps

Acceleration with bypass4netns is available with --label nerdctl/bypass4netns=true
Example

$ nerdctl run -it --rm -p 8080:80 --label nerdctl/bypass4netns=true alpine

More detail is available at README.md(https://github.com/rootless-containers/bypass4netns/blob/master/README.md)

Signed-off-by: Naoki MATSUMOTO naoki@pibvt.net

AkihiroSuda · 2022-02-14T07:15:58Z

The performance benchmark with iperf3 on GitHub Action is shown below

It should be noted that Vagrant on macOS on GHA is extremely slow environment.
Could you also try it on a "sane" machine?

AkihiroSuda · 2022-02-14T07:18:23Z

pkg/bypass4netnsutil/bypass.go

+		SocketPath:    socketPath,
+		PidFilePath:   pidFilePath,
+		LogFilePath:   logFilePath,
+		IgnoreSubnets: []string{"127.0.0.0/8", "10.0.0.0/8"},


"10.0.0.0/8" shouldn't be hardcoded.
But you may add a documentation about this and call it a day.

Eventually, this should be passed from (equivalent of ) nerdctl network list and nerdctl network inspect.

The documentation can be added in docs/rootless.md

Also please explain that bypass4netns is experimental in https://github.com/containerd/nerdctl/blob/master/docs/experimental.md

naoki9911 · 2022-02-14T07:18:52Z

Thank you for your comment!
I will try the performance benchmark on my laptop.
However, my environment is Ubuntu 21.10 on Hyper-V VM. Is that OK?

AkihiroSuda · 2022-02-14T07:19:06Z

pkg/labels/labels.go

@@ -69,4 +69,7 @@ const (

 	// Platform is the normalized platform string like "linux/ppc64le".
 	Platform = Prefix + "platform"
+
+	// Bypass4netns is the flag to enable acceleration with bypass4netns


Please explain the value format

AkihiroSuda · 2022-02-14T07:22:48Z

cmd/nerdctl/run_linux.go

+	if err != nil {
+		return nil, err
+	}
+	labelMaps := strutil.ConvertKVStringsToMap(strutil.DedupeStrSlice(lab))


We need to support label-file too, but it can be fixed in a separate PR if it is too complicated.
Please add "TODO" comment in that case.

nerdctl/cmd/nerdctl/run.go

Line 843 in b88e714

labelsFilePath, err := cmd.Flags().GetStringSlice("label-file")

AkihiroSuda · 2022-02-14T07:23:17Z

However, my environment is Ubuntu 21.10 on Hyper-V VM. Is that OK?

Yes.

naoki9911 · 2022-02-14T08:10:32Z

I improved the code and updated performance benchmark result

AkihiroSuda · 2022-02-14T08:13:04Z

docs/rootless.md

+| container -> host | 0.398 Gbps           | **42.2 Gbps**         |
+| host -> container | 20.6 Gbps             | **47.4 Gbps**         |
+
+This benchmark can be reproduced with [Vagrantfile(https://github.com/rootless-containers/bypass4netns/blob/master/Vagrantfile)](https://github.com/rootless-containers/bypass4netns/blob/master/Vagrantfile)


blob/master

Please specify the commit hash for reproducibility

AkihiroSuda · 2022-02-14T08:13:37Z

docs/rootless.md

+Container networks which are not contained in the subnets can be broken or can cause problems.
+
+### TODO
+- Remove hard-coded subnets in pkg/bypass4netnsutil/bypass.go L.59 ~ L.60


Remove "L.59 ~ L.60", as line numbers are not permanent

AkihiroSuda · 2022-02-14T08:15:01Z

Could you squash commits?

option '--label nerdctl/bypass4netns=true' in run command enables acceleration with bypass4netns Signed-off-by: Naoki MATSUMOTO <naoki@pibvt.net>

naoki9911 · 2022-02-14T08:23:34Z

I improved and squashed commits into the first one.

AkihiroSuda

Thanks

ktock

This is super cool 🎉

AkihiroSuda · 2022-02-14T09:20:57Z

CI failure (rootful / v1.5.9) is unrelated

=== RUN   TestPushInsecureWithLogin
    testregistry_linux.go:78: hostIP="10.4.0.1", listenIP="0.0.0.0", listenPort=5000, authPort=5100
    testregistry_linux.go:158: Writing "/tmp/TestPushInsecureWithLogin1361150108/004/certs.d3566553694/10.4.0.1:5000/hosts.toml": "\nserver = \"https://10.4.0.1:5000\"\n[host.\"https://10.4.0.1:5000\"]\n  ca = \"/tmp/TestPushInsecureWithLogin1361150108/001/ca1682428042/ca.cert\"\n\t\t"
    push_linux_test.go:93: testImageRef="10.4.0.1:5000/nerdctl-testpushinsecurewithlogin:3.13-org"
    push_linux_test.go:97: assertion failed: expression is false: res.ExitCode == exitCode: index-sha256:83bf22e8705adaaaf[386](https://github.com/containerd/nerdctl/runs/5180902314?check_suite_focus=true#step:6:386)34266c57a41fb2fb9867f61bf942e13b8432b2d7cda4: waiting        |--------------------------------------| 
        elapsed: 0.1 s                                                                 total:   0.0 B (0.0 B/s)                                         
        time="2022-02-14T09:01:38Z" level=info msg="pushing as a reduced-platform image (application/vnd.docker.distribution.manifest.list.v2+json, sha256:83bf22e8705adaaaf38634266c57a41fb2fb9867f61bf942e13b8432b2d7cda4)"
        time="2022-02-14T09:01:38Z" level=warning msg="skipping verifying HTTPS certs for \"10.4.0.1:5000\""
        time="2022-02-14T09:01:38Z" level=fatal msg="content digest sha256:83bf22e8705adaaaf38634266c57a41fb2fb9867f61bf942e13b8432b2d7cda4: not found"
        
--- FAIL: TestPushInsecureWithLogin (6.21s)

merging

naoki9911 · 2022-02-14T09:27:02Z

Thank you for reviewing and merging!

AkihiroSuda added area/rootless Rootless mode enhancement New feature or request impact/major labels Feb 14, 2022

AkihiroSuda reviewed Feb 14, 2022

View reviewed changes

bypass4netns integration

f47c7dd

option '--label nerdctl/bypass4netns=true' in run command enables acceleration with bypass4netns Signed-off-by: Naoki MATSUMOTO <naoki@pibvt.net>

AkihiroSuda approved these changes Feb 14, 2022

View reviewed changes

AkihiroSuda added the status/merge-on-green label Feb 14, 2022

ktock approved these changes Feb 14, 2022

View reviewed changes

AkihiroSuda added this to the v0.17.0 milestone Feb 14, 2022

AkihiroSuda merged commit 8df4284 into containerd:master Feb 14, 2022

AkihiroSuda added the area/performance label Feb 14, 2022

This was referenced Feb 14, 2022

nerdctl-full: add bypass4netns #810

Merged

bypass4netns: "10.0.0.0/8" shouldn't be hardcoded #811

Closed

AkihiroSuda added area/bypass4netns and removed status/merge-on-green labels Feb 14, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add bypass4netns integration #808

Add bypass4netns integration #808

naoki9911 commented Feb 14, 2022 •

edited

Loading

AkihiroSuda commented Feb 14, 2022

AkihiroSuda Feb 14, 2022

AkihiroSuda Feb 14, 2022

AkihiroSuda Feb 14, 2022

naoki9911 commented Feb 14, 2022

AkihiroSuda Feb 14, 2022

AkihiroSuda Feb 14, 2022

AkihiroSuda commented Feb 14, 2022

naoki9911 commented Feb 14, 2022

AkihiroSuda Feb 14, 2022

AkihiroSuda Feb 14, 2022

AkihiroSuda commented Feb 14, 2022

naoki9911 commented Feb 14, 2022

AkihiroSuda left a comment

ktock left a comment

AkihiroSuda commented Feb 14, 2022

naoki9911 commented Feb 14, 2022

Add bypass4netns integration #808

Add bypass4netns integration #808

Conversation

naoki9911 commented Feb 14, 2022 • edited Loading

Description

AkihiroSuda commented Feb 14, 2022

AkihiroSuda Feb 14, 2022

Choose a reason for hiding this comment

AkihiroSuda Feb 14, 2022

Choose a reason for hiding this comment

AkihiroSuda Feb 14, 2022

Choose a reason for hiding this comment

naoki9911 commented Feb 14, 2022

AkihiroSuda Feb 14, 2022

Choose a reason for hiding this comment

AkihiroSuda Feb 14, 2022

Choose a reason for hiding this comment

AkihiroSuda commented Feb 14, 2022

naoki9911 commented Feb 14, 2022

AkihiroSuda Feb 14, 2022

Choose a reason for hiding this comment

AkihiroSuda Feb 14, 2022

Choose a reason for hiding this comment

AkihiroSuda commented Feb 14, 2022

naoki9911 commented Feb 14, 2022

AkihiroSuda left a comment

Choose a reason for hiding this comment

ktock left a comment

Choose a reason for hiding this comment

AkihiroSuda commented Feb 14, 2022

naoki9911 commented Feb 14, 2022

naoki9911 commented Feb 14, 2022 •

edited

Loading