Support configurable interfaces for Pod traffic across Node #2370

wenyingd · 2021-07-09T11:56:31Z

SNAT is performed by Windows host for Pod-to-external traffic, not by OVS
Antrea Agent uses the configurable interface for Pod traffic. The interface
which is configured with Node IP is chosen, if user doesn't configure the
in-band traffic interface.

Signed-off-by: wenyingd wenyingd@vmware.com

codecov-commenter · 2021-07-09T12:07:44Z

Codecov Report

Merging #2370 (7db77e1) into main (b8bc15c) will increase coverage by 5.02%.
The diff coverage is 46.98%.

@@            Coverage Diff             @@
##             main    #2370      +/-   ##
==========================================
+ Coverage   59.86%   64.88%   +5.02%     
==========================================
  Files         284      284              
  Lines       22178    25457    +3279     
==========================================
+ Hits        13277    16518    +3241     
+ Misses       7480     7384      -96     
- Partials     1421     1555     +134

Flag	Coverage Δ
e2e-tests	`55.91% <28.91%> (?)`
kind-e2e-tests	`46.97% <42.85%> (-0.11%)`	⬇️
unit-tests	`42.01% <39.18%> (+0.03%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
pkg/agent/agent_linux.go	`80.00% <0.00%> (-20.00%)`	⬇️
pkg/agent/openflow/pipeline_other.go	`75.00% <ø> (+15.00%)`	⬆️
pkg/agent/util/net.go	`52.94% <0.00%> (+1.65%)`	⬆️
...gent/controller/noderoute/node_route_controller.go	`62.16% <27.77%> (+15.08%)`	⬆️
pkg/agent/agent.go	`58.55% <62.50%> (+8.21%)`	⬆️
pkg/agent/route/route_linux.go	`50.11% <66.66%> (+5.43%)`	⬆️
pkg/agent/config/node_config.go	`100.00% <100.00%> (ø)`
pkg/agent/openflow/client.go	`69.82% <100.00%> (+11.94%)`	⬆️
pkg/agent/openflow/pipeline.go	`82.22% <100.00%> (+9.25%)`	⬆️
pkg/controller/egress/ipallocator/allocator.go	`67.82% <0.00%> (-15.16%)`	⬇️
... and 277 more

wenyingd · 2021-07-12T06:02:10Z

/test-windows-all
/test-ipv6-all
/test-ipv6-only-all

wenyingd · 2021-07-12T09:13:36Z

/test-windows-conformance
/test-windows-networkpolicy
/test-windows-e2e

build/yamls/antrea-windows.yml

jianjuns

In the commit message: "SNAT is performing by" -> "SNAT is performed by".

build/yamls/base/conf/antrea-agent.conf

pkg/agent/controller/noderoute/node_route_controller.go

pkg/agent/agent_windows.go

pkg/agent/agent.go

docs/design/windows-design.md

pkg/agent/agent.go

pkg/agent/openflow/pipeline_windows.go

pkg/agent/util/net.go

tnqn · 2021-07-20T10:55:48Z

pkg/agent/util/net_others.go

+
+// GetVirtualInterfaceByName is not implemented. An error returned when this function is called, which means the required
+// interface is not found.
+func GetVirtualInterfaceByName(ifaceName, ovsBridgeName string) (*net.Interface, error) {


I understand this cause compiliation issue on MAC but this is not the only missing function, can we rename net_linux to net_other instead?

In my mind, some dependent libraries in current net_linux (possibly is netlink) is not supported by MAC, that's why we used to rename it from net_other to net_linux.

pkg/agent/util/net_windows.go

tnqn · 2021-07-20T10:58:53Z

pkg/agent/util/net_windows.go

+func CreateNetNatOnHost(subnetCIDR *net.IPNet) error {
+	netNatName := "antrea-nat"
+	cmd := fmt.Sprintf(`Remove-NetNat -Name %s -Confirm:$false`, netNatName)
+	if _, err := ps.RunCommand(cmd); err != nil {


Can we do "create if not found" instead of "remove and create" to be less disruptive?

Actually, I assumed the PodCIDR is possibly changed on the Windows host, that's why I use "remove and create". But if this assumption doesn't exist, I would change.

To be safe, it can compare whether the value is expected first, then decide whether to remove and create.

Get-NetNat doesn't support using "InternalIPInterfaceAddressPrefix" as filter. I would try to compare with the result fields

jayunit100 · 2021-07-20T20:17:40Z

hi @wenyingd how are you compiling the executable here? when I ran this antrea-agent.exe, I got a "not a valid windows executable" error message.

To test:

I tried to run make windows-bin for me worked, but I couldn't run the antrea-agent.exe.
... It was 73.1 MB in size, I assume that is correct ?
... The url to it is here : https://storage.googleapis.com/jayunit100/antrea-agent-if-ps-fixes-2.exe ... is there anything wrong with it ?

wenyingd · 2021-07-21T01:20:48Z

hi @wenyingd how are you compiling the executable here? when I ran this antrea-agent.exe, I got a "not a valid windows executable" error message.

To test:

I tried to run make windows-bin for me worked, but I couldn't run the antrea-agent.exe.

... It was 73.1 MB in size, I assume that is correct ?

... The url to it is here : https://storage.googleapis.com/jayunit100/antrea-agent-if-ps-fixes-2.exe ... is there anything wrong with it ?

I usually compile the binary on Windows host. I have installed cygwin on Windows host, and run make windows-bin in cygwin shell.

lzhecheng · 2021-07-26T07:38:47Z

/test-windows-networkpolicy

wenyingd · 2021-07-26T09:05:47Z

/test-windows-networkpolicy

wenyingd · 2021-07-26T11:39:10Z

/test-windows-networkpolicy

jianjuns · 2021-07-27T01:15:48Z

@tnqn @jianjuns Any other comments on this PR?

I do not have extra comments.

pkg/agent/agent.go

pkg/agent/agent_test.go

pkg/agent/controller/noderoute/node_route_controller.go

tnqn · 2021-07-27T03:54:38Z

I assume this works for Linux too. Could you remove [Windows] from the title and mentions the configuration works for both platforms in commit message? #2473 is asking similar feature on Linux but I found the PR's title is misleading when I pointed it to that issue.

wenyingd · 2021-07-27T08:21:06Z

/test-all

Jexf · 2021-07-27T10:58:16Z

build/yamls/windows/base/conf/antrea-agent.conf

+# The name of the interface on Node which is used for tunneling or routing the traffic across Nodes.
+# If there are multiple IP addresses configured on the interface, the first one is used.
+# The interface configured with Node IP is used if this parameter is not set.
+#transportInterface:


Sometimes there have different interface name on each node, which will be used to inter-host communication.
Maybe need to use special cidr range for inter-host communication, such as antrea-agent --iface-cidr 172.16.100.0/24, could we add a param transportCIDRRange for this scenarios? @wenyingd @tnqn

Do you mean on each Node, we look up the interface by IP which should fall into the transportCIDRRange?

But it is also possible Nodes' interfaces are in different CIDRs. I feel not easy to cover all cases, but do you already have a use case that Nodes use different interfaces?

wenyingd · 2021-07-28T05:40:28Z

/test-integration
/test-all

wenyingd · 2021-07-28T05:48:55Z

/test-integration
/test-conformance
/test-networkpolicy
/test-e2e
/test-ipv6-all
/test-ipv6-only-all

lzhecheng · 2021-07-28T06:41:32Z

/test-integration

tnqn

LGTM. two minor comments left

pkg/agent/agent.go

pkg/agent/agent_test.go

1. SNAT is performed by Windows host for Pod-to-external traffic, not by OVS 2. Antrea Agent uses the configurable interface for Pod traffic. The interface which is configured with Node IP is chosen, if user doesn't configure the in-band traffic interface. Signed-off-by: wenyingd <wenyingd@vmware.com> Co-authored-by: Quan Tian <qtian@vmware.com>

wenyingd · 2021-07-29T01:49:04Z

/test-all
/test-ipv6-all
/test-ipv6-only-all

tnqn

LGTM, thanks

tnqn · 2021-07-29T04:23:24Z

/test-windows-conformance
/test-windows-networkpolicy
/test-ipv6-only-networkpolicy

All comments have been addressed except #2370 (comment), which we are tracking in #2473. I will merge it after the remaining tests pass.

tnqn · 2021-07-29T12:59:34Z

@wenyingd could you check if the failures of jenkins-windows-conformance and jenkins-windows-networkpolicy are related?

wenyingd · 2021-07-29T13:01:32Z

@wenyingd could you check if the failures of jenkins-windows-conformance and jenkins-windows-networkpolicy are related?

No, it is CI testbed issue. I have passed Windows conformance and networkpolicy tests in my setup many times.

jayunit100 · 2021-08-02T13:11:55Z

hi folks, cool, do we have any releases that has this fix ?

antoninbas · 2021-08-02T16:26:21Z

Hi @jayunit100 - this was just merged last week. It is not a bug fix and will not be backported. It will be included in Antrea v1.3 (end of August).

jayunit100 · 2021-08-29T11:11:44Z

thanks ! anyone have an antrea.exe binary aroujnd i can use?
for some reason, when i compile it locally using the make target for antrea (windows) its flagged as "not a valid windows executable"

lzhecheng · 2021-08-30T01:55:09Z

@jayunit100 hello, currently there's no place to download an exe with this patch but you can compile yourself.

How did you do it? From my side, I usually make windows-bin on a Linux machine and exe files (antrea-agent.exe, antrea-cni.exe) will be under bin/.

wenyingd force-pushed the issue_2337 branch 3 times, most recently from fc73f8f to a652c83 Compare July 12, 2021 05:02

lzhecheng reviewed Jul 13, 2021

View reviewed changes

build/yamls/antrea-windows.yml Outdated Show resolved Hide resolved

wenyingd force-pushed the issue_2337 branch 4 times, most recently from 92298e6 to c2ea073 Compare July 15, 2021 09:46

wenyingd requested review from jianjuns, antoninbas and tnqn July 15, 2021 09:46

wenyingd changed the title ~~[WIP] [Windows] Support configurable interfaces for Pod traffic across Node~~ [Windows] Support configurable interfaces for Pod traffic across Node Jul 15, 2021

jianjuns reviewed Jul 15, 2021

View reviewed changes

pkg/agent/agent_windows.go Show resolved Hide resolved

pkg/agent/agent.go Outdated Show resolved Hide resolved

wenyingd force-pushed the issue_2337 branch 6 times, most recently from 44745b2 to d27871e Compare July 19, 2021 05:43

tnqn reviewed Jul 20, 2021

View reviewed changes

wenyingd force-pushed the issue_2337 branch 4 times, most recently from e995e7f to 40fb21f Compare July 21, 2021 09:32

tnqn reviewed Jul 27, 2021

View reviewed changes

pkg/agent/agent.go Outdated Show resolved Hide resolved

pkg/agent/agent_test.go Show resolved Hide resolved

pkg/agent/controller/noderoute/node_route_controller.go Show resolved Hide resolved

tnqn mentioned this pull request Jul 27, 2021

Antrea supports setting network interface for inter-host communication #2473

Closed

wenyingd force-pushed the issue_2337 branch from 97fc2c6 to 27e477a Compare July 27, 2021 08:20

wenyingd changed the title ~~[Windows] Support configurable interfaces for Pod traffic across Node~~ Support configurable interfaces for Pod traffic across Node Jul 27, 2021

Jexf reviewed Jul 27, 2021

View reviewed changes

tnqn reviewed Jul 28, 2021

View reviewed changes

pkg/agent/agent.go Outdated Show resolved Hide resolved

pkg/agent/agent_test.go Outdated Show resolved Hide resolved

wenyingd force-pushed the issue_2337 branch from 27e477a to 7db77e1 Compare July 28, 2021 13:42

wenyingd requested a review from tnqn July 29, 2021 01:49

tnqn approved these changes Jul 29, 2021

View reviewed changes

tnqn merged commit da17b5d into antrea-io:main Jul 29, 2021

wenyingd deleted the issue_2337 branch October 8, 2021 14:03

lzhecheng mentioned this pull request Oct 18, 2021

[Windows] Use antrea-proxy for Cluster IP Service traffic from Windows host #2235

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support configurable interfaces for Pod traffic across Node #2370

Support configurable interfaces for Pod traffic across Node #2370

wenyingd commented Jul 9, 2021 •

edited

Loading

codecov-commenter commented Jul 9, 2021 •

edited

Loading

wenyingd commented Jul 12, 2021

wenyingd commented Jul 12, 2021

jianjuns left a comment

tnqn Jul 20, 2021

wenyingd Jul 21, 2021

tnqn Jul 20, 2021

wenyingd Jul 21, 2021

tnqn Jul 22, 2021

wenyingd Jul 23, 2021

jayunit100 commented Jul 20, 2021 •

edited

Loading

wenyingd commented Jul 21, 2021 •

edited

Loading

lzhecheng commented Jul 26, 2021

wenyingd commented Jul 26, 2021

wenyingd commented Jul 26, 2021

jianjuns commented Jul 27, 2021

tnqn commented Jul 27, 2021

wenyingd commented Jul 27, 2021

Jexf Jul 27, 2021

jianjuns Jul 27, 2021

wenyingd commented Jul 28, 2021

wenyingd commented Jul 28, 2021

lzhecheng commented Jul 28, 2021

tnqn left a comment

wenyingd commented Jul 29, 2021

tnqn left a comment

tnqn commented Jul 29, 2021

tnqn commented Jul 29, 2021

wenyingd commented Jul 29, 2021 •

edited

Loading

jayunit100 commented Aug 2, 2021

antoninbas commented Aug 2, 2021

jayunit100 commented Aug 29, 2021

lzhecheng commented Aug 30, 2021

Support configurable interfaces for Pod traffic across Node #2370

Support configurable interfaces for Pod traffic across Node #2370

Conversation

wenyingd commented Jul 9, 2021 • edited Loading

codecov-commenter commented Jul 9, 2021 • edited Loading

Codecov Report

wenyingd commented Jul 12, 2021

wenyingd commented Jul 12, 2021

jianjuns left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jayunit100 commented Jul 20, 2021 • edited Loading

wenyingd commented Jul 21, 2021 • edited Loading

lzhecheng commented Jul 26, 2021

wenyingd commented Jul 26, 2021

wenyingd commented Jul 26, 2021

jianjuns commented Jul 27, 2021

tnqn commented Jul 27, 2021

wenyingd commented Jul 27, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wenyingd commented Jul 28, 2021

wenyingd commented Jul 28, 2021

lzhecheng commented Jul 28, 2021

tnqn left a comment

Choose a reason for hiding this comment

wenyingd commented Jul 29, 2021

tnqn left a comment

Choose a reason for hiding this comment

tnqn commented Jul 29, 2021

tnqn commented Jul 29, 2021

wenyingd commented Jul 29, 2021 • edited Loading

jayunit100 commented Aug 2, 2021

antoninbas commented Aug 2, 2021

jayunit100 commented Aug 29, 2021

lzhecheng commented Aug 30, 2021

wenyingd commented Jul 9, 2021 •

edited

Loading

codecov-commenter commented Jul 9, 2021 •

edited

Loading

jayunit100 commented Jul 20, 2021 •

edited

Loading

wenyingd commented Jul 21, 2021 •

edited

Loading

wenyingd commented Jul 29, 2021 •

edited

Loading