Multicast statistics implementation #3449

ceclinux · 2022-03-15T03:08:51Z

This PR added multicast statistics support for the following cases:

Supplements current networkpolicy statistics implementation by parsing multicast related flows, which can be displayed as kubectl get antreanetworkpolicystats multicast-networkpolicy-name
Add a node-level antctl command antctl get multicaststats, showing inbound and outbound packet count for each pod interface.
Add an extra kubectl get multicastgroupmembers command. This command shows which pods have joined multicast group for the whole cluster.

rebase updated bin's patch
add tests

codecov-commenter · 2022-03-15T03:15:35Z

Codecov Report

Merging #3449 (8284475) into main (7817152) will decrease coverage by 0.02%.
The diff coverage is 27.78%.

@@            Coverage Diff             @@
##             main    #3449      +/-   ##
==========================================
- Coverage   64.48%   64.46%   -0.03%     
==========================================
  Files         291      293       +2     
  Lines       42795    43224     +429     
==========================================
+ Hits        27598    27864     +266     
- Misses      12974    13130     +156     
- Partials     2223     2230       +7

Flag	Coverage Δ
e2e-tests	`44.74% <12.42%> (?)`
kind-e2e-tests	`50.57% <12.42%> (-0.42%)`	⬇️
unit-tests	`44.56% <26.43%> (-0.20%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
.../agent/apiserver/handlers/networkpolicy/handler.go	`48.97% <0.00%> (ø)`
...g/agent/apiserver/handlers/podinterface/handler.go	`70.83% <ø> (ø)`
pkg/agent/multicast/mcast_controller.go	`51.93% <0.00%> (-6.16%)`	⬇️
pkg/agent/openflow/multicast.go	`0.00% <0.00%> (ø)`
pkg/agent/openflow/pipeline.go	`72.94% <ø> (+0.22%)`	⬆️
pkg/antctl/antctl.go	`50.00% <ø> (ø)`
pkg/controller/stats/aggregator.go	`68.95% <0.00%> (-12.10%)`	⬇️
pkg/querier/querier.go	`55.00% <ø> (ø)`
...kg/apiserver/registry/stats/multicastgroup/rest.go	`11.59% <11.59%> (ø)`
pkg/agent/stats/collector.go	`74.29% <15.38%> (-20.87%)`	⬇️
... and 35 more

ceclinux · 2022-03-15T07:32:45Z

/test-multicast-e2e

ceclinux · 2022-03-15T08:10:07Z

/test-multicast-e2e

ceclinux · 2022-03-15T08:17:32Z

/test-multicast-e2e

ceclinux · 2022-03-15T09:12:41Z

/test-multicast-e2e

ceclinux · 2022-04-06T13:31:39Z

/test-multicast-e2e

ceclinux · 2022-04-07T03:40:41Z

/test-multicast-e2e

ceclinux · 2022-04-07T03:53:09Z

/test-multicast-e2e

wenyingd · 2022-06-13T02:19:43Z

pkg/agent/openflow/multicast.go

@@ -99,6 +100,46 @@ func (f *featureMulticast) multicastOutputFlow(cookieID uint64) binding.Flow {
 		Done()
 }

+func (f *featureMulticast) multicastSkipIGMPMetricFlows() []binding.Flow {
+	return []binding.Flow{


Use this format?

cookieID := f.cookieAllocator.Request(f.category).Raw() var flows []binding.Flow for _, t := range []*Table{MulticastIngressPodMetricTable, MulticastEgressPodMetricTable} { flows = append(flows, t.ofTable.BuildFlow(priorityHigh). Cookie(cookieID). MatchProtocol(binding.ProtocolIGMP). Action().NextTable(). Done()) }

wenyingd · 2022-06-13T02:21:23Z

pkg/agent/openflow/multicast.go

+			Cookie(f.cookieAllocator.Request(f.category).Raw()).
+			MatchProtocol(ipProtocol).
+			MatchSrcIP(podIP).
+			MatchDstIPNet(*types.McastCIDR).


Yes, it looks not necessary.

wenyingd

LGTM

pkg/agent/openflow/client.go

tnqn · 2022-06-13T07:03:37Z

pkg/antctl/antctl.go

+					params: []flagInfo{
+						{
+							name:  "name",
+							usage: "Retrieve Pod interface by name. If present, Namespace must be provided.",


wrong usage

tnqn · 2022-06-13T07:03:45Z

pkg/antctl/antctl.go

+						},
+						{
+							name:      "namespace",
+							usage:     "Get Pod interfaces from specific Namespace",


tnqn · 2022-06-13T07:06:20Z

pkg/apiserver/registry/stats/multicastgroup/rest.go

+		for i, pod := range stats.Pods {
+			podNameSlice[i] = k8s.NamespacedName(pod.Namespace, pod.Name)
+		}
+		return []interface{}{stats.Group, strings.Join(podNameSlice, ", ")}, nil


then we should cut it and append a number and ... like other commands since there will be likely some Pods in a Group?

tnqn · 2022-06-13T07:19:56Z

pkg/controller/stats/aggregator.go

+		for _, mcastGroupInfo := range summary.Multicast {
+			group := mcastGroupInfo.Group
+			reportedGroups.Insert(group)
+			a.groupNodePodsMapMutex.Lock()


acquring lock is not free, seems not worth to lock and unlock repeatedly here to exclude a few code from critical area, which may take more cost

pkg/agent/openflow/pipeline.go

pkg/apiserver/registry/stats/multicastgroup/rest.go

This PR added multicast statistics support for the following cases: - Supplements current networkpolicy statistics implementation by parsing multicast related flows, which can be displayed as kubectl get antreanetworkpolicystats multicast-networkpolicy-name. - Add a node-level antctl command antctl get podmulticaststats, showing inbound and outbound packet count for each pod interface. - Add an extra kubectl get multicastgroups command. This command shows which pods have joined multicast group for the whole cluster. Signed-off-by: ceclinux <src655@gmail.com>

tnqn

LGTM

tnqn · 2022-06-13T15:24:47Z

/test-all
/test-multicast-e2e

tnqn · 2022-06-14T02:03:47Z

/test-multicast-e2e
/test-ipv6-e2e
/test-ipv6-only-e2e

tnqn · 2022-06-14T04:09:41Z

/test-multicast-e2e

tnqn · 2022-06-14T06:22:25Z

/test-multicast-e2e

ceclinux force-pushed the multicast-stats branch from d8ad94d to 3825049 Compare March 15, 2022 09:28

ceclinux mentioned this pull request Mar 23, 2022

Multicast Support in Antrea #2251

Closed

12 tasks

ceclinux force-pushed the multicast-stats branch from 3825049 to 3714028 Compare April 6, 2022 10:35

ceclinux force-pushed the multicast-stats branch 2 times, most recently from 9dcd1a2 to 0b6f728 Compare April 7, 2022 03:15

ceclinux force-pushed the multicast-stats branch 11 times, most recently from 1ecf91c to 7d3bb79 Compare April 12, 2022 12:16

tnqn added this to the Antrea v1.7 release milestone May 13, 2022

tnqn added action/release-note Indicates a PR that should be included in release notes. kind/feature Categorizes issue or PR as related to a new feature. labels May 13, 2022

ceclinux force-pushed the multicast-stats branch from 7d3bb79 to 2f9d2d0 Compare May 13, 2022 06:53

ceclinux changed the title ~~[WIP Multicast] Multicast statistics implementation~~ Multicast statistics implementation May 13, 2022

ceclinux force-pushed the multicast-stats branch from 2f9d2d0 to e78668d Compare May 15, 2022 12:44

ceclinux force-pushed the multicast-stats branch 8 times, most recently from 1fb0f2b to d2153dd Compare June 12, 2022 23:40

wenyingd reviewed Jun 13, 2022

View reviewed changes

ceclinux force-pushed the multicast-stats branch from d2153dd to a7020f2 Compare June 13, 2022 02:51

wenyingd previously approved these changes Jun 13, 2022

View reviewed changes

tnqn reviewed Jun 13, 2022

View reviewed changes

ceclinux dismissed wenyingd’s stale review via 7681214 June 13, 2022 07:59

ceclinux force-pushed the multicast-stats branch 3 times, most recently from 98f7ba1 to 178794b Compare June 13, 2022 10:11

tnqn reviewed Jun 13, 2022

View reviewed changes

pkg/agent/openflow/pipeline.go Show resolved Hide resolved

pkg/apiserver/registry/stats/multicastgroup/rest.go Outdated Show resolved Hide resolved

pkg/apiserver/registry/stats/multicastgroup/rest.go Outdated Show resolved Hide resolved

ceclinux force-pushed the multicast-stats branch 3 times, most recently from 36cd78e to 1169210 Compare June 13, 2022 13:02

tnqn reviewed Jun 13, 2022

View reviewed changes

pkg/apiserver/registry/stats/multicastgroup/rest.go Outdated Show resolved Hide resolved

ceclinux force-pushed the multicast-stats branch from 1169210 to 8284475 Compare June 13, 2022 14:52

tnqn approved these changes Jun 13, 2022

View reviewed changes

tnqn merged commit 6782d71 into antrea-io:main Jun 14, 2022

ceclinux mentioned this pull request Jul 26, 2022

Support Multicast statistics #3294

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Multicast statistics implementation #3449

Multicast statistics implementation #3449

ceclinux commented Mar 15, 2022 •

edited

Loading

codecov-commenter commented Mar 15, 2022 •

edited

Loading

ceclinux commented Mar 15, 2022

ceclinux commented Mar 15, 2022

ceclinux commented Mar 15, 2022

ceclinux commented Mar 15, 2022

ceclinux commented Apr 6, 2022

ceclinux commented Apr 7, 2022

ceclinux commented Apr 7, 2022

wenyingd Jun 13, 2022

ceclinux Jun 13, 2022

wenyingd Jun 13, 2022

wenyingd left a comment

tnqn Jun 13, 2022

ceclinux Jun 13, 2022

tnqn Jun 13, 2022

ceclinux Jun 13, 2022

tnqn Jun 13, 2022

tnqn Jun 13, 2022

ceclinux Jun 13, 2022

tnqn left a comment

tnqn commented Jun 13, 2022

tnqn commented Jun 14, 2022

tnqn commented Jun 14, 2022

tnqn commented Jun 14, 2022

Multicast statistics implementation #3449

Multicast statistics implementation #3449

Conversation

ceclinux commented Mar 15, 2022 • edited Loading

codecov-commenter commented Mar 15, 2022 • edited Loading

Codecov Report

ceclinux commented Mar 15, 2022

ceclinux commented Mar 15, 2022

ceclinux commented Mar 15, 2022

ceclinux commented Mar 15, 2022

ceclinux commented Apr 6, 2022

ceclinux commented Apr 7, 2022

ceclinux commented Apr 7, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wenyingd left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tnqn left a comment

Choose a reason for hiding this comment

tnqn commented Jun 13, 2022

tnqn commented Jun 14, 2022

tnqn commented Jun 14, 2022

tnqn commented Jun 14, 2022

ceclinux commented Mar 15, 2022 •

edited

Loading

codecov-commenter commented Mar 15, 2022 •

edited

Loading