Dev/exclude subnets from traffic shaping2 #921

oOraph · 2023-07-06T09:58:04Z

bandwidth: possibility to exclude some subnets from traffic shapping

what changed:

we had to refactor the bandwidth plugin and switch from a classless qdisc (tbf)
to a classful qdisc (htb).

subnets are to be provided in config or runtimeconfig just like other parameters

unit and integration tests were also adapted in consequence

unrelated changes:

test fixes: the most important tests were just silently skipped due to ginkgo Measure deprecation
(the ones actually checking the effectiveness of the traffic control)

squeed · 2023-07-21T10:26:52Z

@oOraph thanks for this! And very good catch w.r.t. Ginkgo Measure.

Can you check the lint error?

oOraph · 2023-07-21T12:27:00Z

Can you check the lint error?

@squeed Done :)

s1061123

Hi @oOraph I have a question about the design of the PR.

As far as I understood, this PR is going to introduce folloiwng tc filtering config.

tc qdisc add dev <interfaceName> root handle 1: htb default 30
tc class add dev <interfaceName> parent 1: classid 1:30 htb rate <rateInBits> burst <burstInBits>
tc class add dev <interfaceName> parent 1: classid 1:1 htb rate 100000000000
tc filter add dev <interfaceName> parent 1: protocol <protocol> prio 16 u32 match ip dst <subnet> flowid 1:1

It might be good to have, however, on the otherside, user may want to do opposite configurations (adding 'target CIDR' for rate limit), such as

tc qdisc add dev <interfaceName> root handle 1: htb default 30
tc class add dev <interfaceName> parent 1: classid 1:1 htb rate <rateInBits> burst <burstInBits>
tc class add dev <interfaceName> parent 1: classid 1:30 htb rate 100000000000
tc filter add dev <interfaceName> parent 1: protocol <protocol> prio 16 u32 match ip dst <subnet> flowid 1:1

Your proposed config may not fit second one. Is there any way to enhance your desin to support second config and more flexible tc configurations?

s1061123 · 2023-07-24T16:01:03Z

plugins/meta/bandwidth/main.go

+	for _, subnet := range subnets {
+		_, _, err := net.ParseCIDR(subnet)
+		if err != nil {
+			return fmt.Errorf("bad subnet provided %s, details %s", subnet, err)


how about to use %q for subnet (to double quote) and %v for error, to fit to other Errorf in the file?

s1061123 · 2023-07-24T16:03:13Z

plugins/meta/bandwidth/main.go

+	err = validateSubnets(bandwidth.NonShapedSubnets)
+	if err != nil {
+		return err
+	}


Minor, but we could change as other code does

plugins/plugins/meta/bandwidth/main.go

Lines 67 to 69 in 9d9ec6e

if err := json.Unmarshal(stdin, &conf); err != nil {

return nil, fmt.Errorf("failed to parse network configuration: %v", err)

}

if err = validateSubnets(bandwidth.NonShapedSubnets); err != nil { return err }

s1061123 · 2023-07-24T16:44:23Z

plugins/meta/bandwidth/ifb_creator.go

+
+		}
+
+		protocol := syscall.ETH_P_ALL


why protocol is all, instead of ip' or ipv6' (as you commented just above?)

oOraph · 2023-07-27T13:29:57Z

@s1061123 , about #921 (review)
Done as you wished :). now you can either set unshappedSubnets or shappedSubnets to specify subnets to be excluded from/included in traffic shapping. Both parameters are mutually exclusive and will result in error when simultaneously defined

s1061123 · 2023-07-27T13:58:37Z

@oOraph Thank you to incorporate my comments! I will review code tomorrow again!

s1061123

Hi, almost code is reasonable to me. BTW, could you provide document for that, in https://github.com/containernetworking/cni.dev ?

s1061123 · 2023-08-08T06:07:17Z

plugins/meta/bandwidth/bandwidth_linux_test.go

 	var err error

+	name := "myBWnet"


how about to embedded the name in cni config directly (in line 47)?

s1061123 · 2023-08-08T06:20:58Z

plugins/meta/bandwidth/bandwidth_linux_test.go

+		It("Should fail if specified ShapedSubnets are not valid CIDRs", func() {
+			err := validateSubnets([]string{}, []string{"10.0.0.0/8", "hello"})
+			Expect(err).To(HaveOccurred())
+		})


Thank you for improve unit tests. However, on the other side, this file is now the biggest file in the repository. Could you split into several _test.go file for ease of maintenance, if you don't mind it?

done, split the file in three:
gathered all "functional" tests in one file (tests checking the effectiveness of the shaping)
all config related tests in another
left all other tests together in the original test file

s1061123 · 2023-08-08T07:26:27Z

plugins/meta/bandwidth/ifb_creator.go

+
+		// cmd = exec.Command("/usr/sbin/tc", "filter", "add", "dev", interfaceName, "parent", "1:", "protocol", protocol,
+		// "prio", "16", "u32", "match", "ip", "dst", subnet, "flowid", "1:1")
+


May remove this empty line (line:225)

s1061123 · 2023-08-08T07:26:50Z

plugins/meta/bandwidth/ifb_creator.go

+	}
+
+	// Now add filters to redirect subnets to the class 1 if excluded instead of the default one (30)
+


May remove this empty line (line:220)

s1061123 · 2023-08-08T07:27:40Z

plugins/meta/bandwidth/ifb_creator.go

+			keepBytes = 4
+			// prio/pref needs to be changed if we change the protocol, looks like we cannot mix protocols with the same pref
+			prio = 16
+


May remove this empty line (line:249)

oOraph · 2023-08-10T08:02:03Z

Hi, almost code is reasonable to me. BTW, could you provide document for that, in https://github.com/containernetworking/cni.dev ?

done -> containernetworking/cni.dev#130

oOraph · 2023-09-07T06:53:47Z

Hello @s1061123 @MikeZappa87 :). Do you still see anything wrong or missing with this pr ?

s1061123

Hi @oOraph thank you for the great work. Almost Okey to me. I give two minor comment to fix year. That's all. Thanks again!

s1061123 · 2023-09-11T11:24:56Z

plugins/meta/bandwidth/bandwidth_config_test.go

@@ -0,0 +1,563 @@
+// Copyright 2018 CNI authors


Fix copyright year: 2018 -> 2023

s1061123 · 2023-09-11T11:25:41Z

plugins/meta/bandwidth/bandwidth_measure_linux_test.go

@@ -0,0 +1,824 @@
+// Copyright 2018 CNI authors


Fix copyright year: 2018 -> 2023

s1061123

/lgtm

Could you please address doc changes in another PR in doc repo, https://github.com/containernetworking/cni.dev?

oOraph · 2023-11-13T10:36:33Z

/lgtm

Could you please address doc changes in another PR in doc repo, https://github.com/containernetworking/cni.dev?

Sure it's already done here:
containernetworking/cni.dev#130

oOraph · 2023-11-16T17:43:55Z

just rebased on upstream/main to fix lint issue

integration/integration_linux_test.go

MikeZappa87 · 2023-12-28T00:30:39Z

This looks good. I did not run the plugin though.

what changed: we had to refactor the bandwidth plugin and switch from a classless qdisc (tbf) to a classful qdisc (htb). subnets are to be provided in config or runtimeconfig just like other parameters unit and integration tests were also adapted in consequence unrelated changes: test fixes: the most important tests were just silently skipped due to ginkgo Measure deprecation (the ones actually checking the effectiveness of the traffic control) Signed-off-by: Raphael <oOraph@users.noreply.github.com>

…rom shaping Signed-off-by: Raphael <oOraph@users.noreply.github.com>

Signed-off-by: Raphael <oOraph@users.noreply.github.com>

even if json unmarshalling in golang with the standard libs is case unsensitive regarding the keys Signed-off-by: Raphael <oOraph@users.noreply.github.com>

Signed-off-by: Tomofumi Hayashi <tohayash@redhat.com>

nayihz · 2024-04-19T03:27:14Z

Hi all, I am curious why we need to create ifb devices and tc rules in host-side network namespace, rather than in container-side.

oOraph · 2024-04-23T07:47:47Z

Hi all, I am curious why we need to create ifb devices and tc rules in host-side network namespace, rather than in container-side.

Hello @nayihz :). If I understand correctly what you mean we need to apply tc rules on host side because the container side is the user side and we do not want them to arbitrarily edit/cancel them.

oOraph · 2024-04-23T07:49:24Z

@s1061123 @MikeZappa87 thanks for merging. Maybe we should also merge the doc wdyt ?
containernetworking/cni.dev#130

oOraph marked this pull request as draft July 6, 2023 09:58

oOraph force-pushed the dev/exclude_subnets_from_traffic_shapping2 branch from 956ad1c to c1af94f Compare July 6, 2023 09:58

oOraph marked this pull request as ready for review July 6, 2023 09:59

oOraph force-pushed the dev/exclude_subnets_from_traffic_shapping2 branch 6 times, most recently from 2424028 to 644ebf7 Compare July 7, 2023 12:35

oOraph force-pushed the dev/exclude_subnets_from_traffic_shapping2 branch from 644ebf7 to 6021f78 Compare July 21, 2023 12:14

squeed added the needs/review label Jul 24, 2023

squeed requested a review from MikeZappa87 July 24, 2023 15:37

s1061123 reviewed Jul 24, 2023

View reviewed changes

oOraph force-pushed the dev/exclude_subnets_from_traffic_shapping2 branch from 6021f78 to 729c1d1 Compare July 27, 2023 07:44

oOraph requested a review from s1061123 August 4, 2023 08:49

s1061123 reviewed Aug 8, 2023

View reviewed changes

oOraph mentioned this pull request Aug 10, 2023

bandwidth: doc update to take into account new tc parameters containernetworking/cni.dev#130

Open

oOraph force-pushed the dev/exclude_subnets_from_traffic_shapping2 branch from e7ee85a to 2e3f7d6 Compare August 10, 2023 08:01

oOraph requested a review from s1061123 August 10, 2023 08:03

oOraph changed the title ~~Dev/exclude subnets from traffic shapping2~~ Dev/exclude subnets from traffic shaping2 Aug 10, 2023

s1061123 approved these changes Sep 11, 2023

View reviewed changes

oOraph force-pushed the dev/exclude_subnets_from_traffic_shapping2 branch from 2e3f7d6 to 3d93935 Compare September 11, 2023 12:02

oOraph force-pushed the dev/exclude_subnets_from_traffic_shapping2 branch from 3d93935 to 21cabd1 Compare October 17, 2023 14:07

oOraph mentioned this pull request Nov 10, 2023

Bandwidth plugin prevents NP enforcement aws/aws-network-policy-agent#68

Open

s1061123 approved these changes Nov 13, 2023

View reviewed changes

oOraph force-pushed the dev/exclude_subnets_from_traffic_shapping2 branch from 21cabd1 to adcf445 Compare November 16, 2023 17:42

dcbw reviewed Nov 20, 2023

View reviewed changes

integration/integration_linux_test.go Show resolved Hide resolved

oOraph requested a review from dcbw November 21, 2023 16:27

squeed assigned MikeZappa87 Dec 11, 2023

oOraph and others added 5 commits April 8, 2024 15:39

bandwidth: possibility to specify shaped subnets or to exclude some f…

ab0b386

…rom shaping Signed-off-by: Raphael <oOraph@users.noreply.github.com>

bandwidth plugin: split unit tests in several files

c666d14

Signed-off-by: Raphael <oOraph@users.noreply.github.com>

minor case change

78ebd8b

even if json unmarshalling in golang with the standard libs is case unsensitive regarding the keys Signed-off-by: Raphael <oOraph@users.noreply.github.com>

Simplify unit test

ccc1cfa

Signed-off-by: Tomofumi Hayashi <tohayash@redhat.com>

s1061123 force-pushed the dev/exclude_subnets_from_traffic_shapping2 branch from adcf445 to ccc1cfa Compare April 8, 2024 06:42

s1061123 mentioned this pull request Apr 8, 2024

bandwidth: introduce configurable queue length #1025

Closed

MikeZappa87 approved these changes Apr 8, 2024

View reviewed changes

s1061123 merged commit ef076af into containernetworking:main Apr 8, 2024
5 checks passed

h0nIg mentioned this pull request Sep 20, 2024

fix issues with bandwidth plugin and htb introduction #1097

Open

	if err := json.Unmarshal(stdin, &conf); err != nil {
	return nil, fmt.Errorf("failed to parse network configuration: %v", err)
	}


		// cmd = exec.Command("/usr/sbin/tc", "filter", "add", "dev", interfaceName, "parent", "1:", "protocol", protocol,
		// "prio", "16", "u32", "match", "ip", "dst", subnet, "flowid", "1:1")

		}

		// Now add filters to redirect subnets to the class 1 if excluded instead of the default one (30)

Dev/exclude subnets from traffic shaping2 #921

Dev/exclude subnets from traffic shaping2 #921

Conversation

oOraph commented Jul 6, 2023 • edited Loading

squeed commented Jul 21, 2023

oOraph commented Jul 21, 2023 • edited Loading

s1061123 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oOraph commented Jul 27, 2023 • edited Loading

s1061123 commented Jul 27, 2023

s1061123 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oOraph commented Aug 10, 2023

oOraph commented Sep 7, 2023

s1061123 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

s1061123 left a comment

Choose a reason for hiding this comment

oOraph commented Nov 13, 2023

oOraph commented Nov 16, 2023

MikeZappa87 commented Dec 28, 2023

nayihz commented Apr 19, 2024 • edited Loading

oOraph commented Apr 23, 2024 • edited Loading

oOraph commented Apr 23, 2024

oOraph commented Jul 6, 2023 •

edited

Loading

oOraph commented Jul 21, 2023 •

edited

Loading

oOraph commented Jul 27, 2023 •

edited

Loading

nayihz commented Apr 19, 2024 •

edited

Loading

oOraph commented Apr 23, 2024 •

edited

Loading