Implement propagation of IP routes into apps via DHCP #3690

milan-zededa · 2023-12-22T11:31:01Z

Most of the edge deployments will deploy applications (VMs or containers) that will have
connectivity to both WAN (Internet) and LAN (shop floor, machine floor, intranet).
There may be a single WAN interface and multiple LAN interfaces. With the IP route
configurability of Network instances implemented by this PR (and APIs extensions already
added here), we can automatically (zero-touch) provision routing for applications that have
access to both the Internet and one or more LANs.
This is now possible because EVE allows to:

use DHCP to propagate the connected routes to applications (routes for external networks
that uplink ports are connected to)
configure a set of static IP routes for a network instance and have them propagated
to applications

Another common case is using one application as a network gateway for other applications
running on the same device. The gateway application may provide some network function(s),
such as firewall, IDS, network monitoring, etc. Such application will connect on one side with
the external network(s) using directly attached network adapter(s) or via switch network
instance(s), and the other side will make use of an air-gap local network instance to connect
with other applications running on the device. Propagated static IP routes are necessary
to make the application traffic flow through the gateway app. In theory, multiple network
functions can be chained together in this way using several air-gap network instances
with static IP routes.

PR with examples for eden: lf-edge/eden#938
They allow to easily and quickly perform manual testing of this functionality.
Eventually, I would like to prepare automated tests, but for now examples will do.

codecov · 2023-12-22T11:45:07Z

Codecov Report

Attention: 150 lines in your changes are missing coverage. Please review.

Comparison is base (03b8972) 19.60% compared to head (0237f6b) 19.82%.

❗ Current head 0237f6b differs from pull request most recent head c823cd1. Consider uploading reports for the commit c823cd1 to get more accurate results

Files	Patch %	Lines
pkg/pillar/nireconciler/linux_config.go	63.29%	44 Missing and 14 partials ⚠️
pkg/pillar/cmd/zedagent/parseconfig.go	0.00%	37 Missing ⚠️
pkg/pillar/nireconciler/genericitems/dnsmasq.go	72.07%	29 Missing and 2 partials ⚠️
pkg/pillar/types/zedroutertypes.go	0.00%	6 Missing ⚠️
pkg/pillar/utils/generics/generics.go	0.00%	6 Missing ⚠️
pkg/pillar/dpcreconciler/genericitems/dhcpcd.go	28.57%	5 Missing ⚠️
pkg/pillar/cmd/zedrouter/pubsubhandlers.go	0.00%	3 Missing ⚠️
pkg/pillar/cmd/zedagent/reportinfo.go	0.00%	1 Missing ⚠️
pkg/pillar/cmd/zedrouter/networkinstance.go	0.00%	1 Missing ⚠️
pkg/pillar/dpcreconciler/linux.go	0.00%	0 Missing and 1 partial ⚠️
... and 1 more

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3690      +/-   ##
==========================================
+ Coverage   19.60%   19.82%   +0.21%     
==========================================
  Files         232      231       -1     
  Lines       50789    51047     +258     
==========================================
+ Hits         9959    10118     +159     
- Misses      40104    40190      +86     
- Partials      726      739      +13

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

pkg/pillar/cmd/zedagent/parseconfig.go

eriknordmark · 2024-01-03T11:25:36Z

pkg/pillar/nireconciler/genericitems/dnsmasq.go

+			// From RFC 3442:
+			// If the DHCP server returns both a Classless Static Routes option and
+			// a Router option, the DHCP client MUST ignore the Router option.
+			//
+			// This means that we have to include a default "classless" route
+			// even though the Router option is provided.
+			// We keep the Router option for clients that do not support the Classless


Makes sense to test this with a range of guest VMs including some version(s) of Windows, to make sure no client is upset by seeing both options.

Tested with Windows 10 - looks good, IP routes are installed as expected.

pkg/pillar/nireconciler/linux_config.go

eriknordmark · 2024-01-03T12:01:27Z

pkg/pillar/nireconciler/linux_config.go

+	}
+	var withDefaultRoute bool
+	airGap := ni.bridge.Uplink.IfName == ""
+	if !airGap && (ni.bridge.Uplink.IsMgmt || r.niHasDefRoute(ni)) {


What's the need to check IsMgmt? Is this for a case when the management interface does not have a default route but somehow received all necessary prefix routes to reach the controller etc?

Note that the comment on L820 doesn't talk about IsMgmt.

IsMgmt = ! app-shared

This if-statement is a negation of that comment (we propagate default route if this if-statement is true).

OK, but I still don't understand why you need to check IsMgmt

For management uplink we always want to propagate default route to apps. For app-shared uplink (IsMgmt=false) only if the uplink has default route.

Ah - your terminology might be confusing me.
We have two cases: inject a default route for the app instance (independent of routing on the external network)
propagate a default and/or other routes from the external network to the app instance.

Is it that the same code does the injection (for the management uplinks) and does the propagation?
If so it makes perfect sense, but is hard to understand from reading the code.

This code here (getIntendedDnsmasqCfg function) only decides if app should receive default route for this network instance or not. If yes, then we propagate 0.0.0.0/0 via bridgeIP - i.e. in this function we do not care what the actual gateway is and whether it is received from an external network or injected by user.
In function getIntendedNIL3Cfg we then prepare config for the NI routing table (for the host network stack). There we decide if we should copy default route from uplink routing table or put there a user-injected default route.

pkg/pillar/nireconciler/linux_config.go

naiming-zededa · 2024-01-03T18:04:42Z

pkg/pillar/nireconciler/genericitems/dnsmasq.go

+			if subnet != nil {
+				staticRoutes = append(staticRoutes, types.IPRoute{
+					DstNetwork: subnet,
+					Gateway:    net.IP{0, 0, 0, 0},


why the gateway is 0.0.0.0 here?

This is a link-local route for the whole NI subnet, such as: 10.10.10.0/24 dev eth0 proto dhcp scope link

naiming-zededa · 2024-01-03T18:27:25Z

pkg/pillar/nireconciler/linux_config.go

+		if isAppGW {
+			outputIf = linux.RouteOutIf{BridgeIfName: ni.brIfName}
+		}
+		intendedL3Cfg.PutItem(linux.Route{


do we check here to prevent the app's static route is 0/0 to be installed into host side for IsMgmt?

Currently there is no such check.

How should we handle the case when NI uplink has default route and at the same time user configures static default route (with possibly different GW IP)? Which one should take precedence? I would think that the static route, since it would give the user the option to override the default route received from an external DHCP server.

Yeh, i would think the user explicit one should take precedence.

Done, PR is updated - user-configured default route will override the default route of the uplink.

eriknordmark

Kick off tests

Signed-off-by: Milan Lenco <milan@zededa.com>

eriknordmark

Run tests

Static and connected routes of network instances can be propagated to applications using DHCP option 121. Signed-off-by: Milan Lenco <milan@zededa.com>

Signed-off-by: Milan Lenco <milan@zededa.com>

milan-zededa requested review from uncleDecart and naiming-zededa December 22, 2023 11:31

milan-zededa requested review from rene, rouming and eriknordmark as code owners December 22, 2023 11:31

github-actions bot requested review from christoph-zededa, OhmSpectator, rucoder and shjala December 22, 2023 11:31

milan-zededa changed the title ~~Implement propagation of routes into apps via DHCP~~ Implement propagation of IP routes into apps via DHCP Dec 22, 2023

milan-zededa force-pushed the ni-routes branch 2 times, most recently from 49e7172 to e014ce9 Compare December 22, 2023 12:00

milan-zededa mentioned this pull request Dec 29, 2023

Eden-SDN: add examples for application IP routing configurations lf-edge/eden#938

Merged

eriknordmark reviewed Jan 3, 2024

View reviewed changes

pkg/pillar/cmd/zedagent/parseconfig.go Outdated Show resolved Hide resolved

eriknordmark reviewed Jan 3, 2024

View reviewed changes

pkg/pillar/nireconciler/linux_config.go Outdated Show resolved Hide resolved

eriknordmark reviewed Jan 3, 2024

View reviewed changes

pkg/pillar/nireconciler/linux_config.go Show resolved Hide resolved

eriknordmark reviewed Jan 3, 2024

View reviewed changes

pkg/pillar/nireconciler/linux_config.go Show resolved Hide resolved

naiming-zededa reviewed Jan 3, 2024

View reviewed changes

eriknordmark approved these changes Jan 4, 2024

View reviewed changes

Parse IP routing config of NIs

dfb1e78

Signed-off-by: Milan Lenco <milan@zededa.com>

milan-zededa force-pushed the ni-routes branch from e014ce9 to 0237f6b Compare January 4, 2024 13:27

github-actions bot requested a review from eriknordmark January 4, 2024 13:28

eriknordmark approved these changes Jan 4, 2024

View reviewed changes

milan-zededa added 2 commits January 5, 2024 17:41

Implement propagation of routes into apps via DHCP

f589c0a

Static and connected routes of network instances can be propagated to applications using DHCP option 121. Signed-off-by: Milan Lenco <milan@zededa.com>

Unit test for static and connected routes

c823cd1

Signed-off-by: Milan Lenco <milan@zededa.com>

milan-zededa force-pushed the ni-routes branch from 0237f6b to c823cd1 Compare January 5, 2024 17:28

github-actions bot requested a review from eriknordmark January 5, 2024 17:28

eriknordmark merged commit cf3dbde into lf-edge:master Jan 9, 2024
31 checks passed

milan-zededa mentioned this pull request Mar 21, 2024

[11.0 stable] Implement propagation of IP routes into apps via DHCP #3831

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement propagation of IP routes into apps via DHCP #3690

Implement propagation of IP routes into apps via DHCP #3690

milan-zededa commented Dec 22, 2023 •

edited

Loading

codecov bot commented Dec 22, 2023 •

edited

Loading

eriknordmark Jan 3, 2024

milan-zededa Jan 3, 2024

milan-zededa Jan 10, 2024 •

edited

Loading

eriknordmark Jan 3, 2024

milan-zededa Jan 3, 2024

eriknordmark Jan 4, 2024

milan-zededa Jan 4, 2024

eriknordmark Jan 8, 2024

milan-zededa Jan 8, 2024

naiming-zededa Jan 3, 2024

milan-zededa Jan 4, 2024 •

edited

Loading

naiming-zededa Jan 3, 2024 •

edited

Loading

milan-zededa Jan 4, 2024 •

edited

Loading

naiming-zededa Jan 4, 2024

milan-zededa Jan 5, 2024

eriknordmark left a comment

eriknordmark left a comment

Implement propagation of IP routes into apps via DHCP #3690

Implement propagation of IP routes into apps via DHCP #3690

Conversation

milan-zededa commented Dec 22, 2023 • edited Loading

codecov bot commented Dec 22, 2023 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

milan-zededa Jan 10, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

milan-zededa Jan 4, 2024 • edited Loading

Choose a reason for hiding this comment

naiming-zededa Jan 3, 2024 • edited Loading

Choose a reason for hiding this comment

milan-zededa Jan 4, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

eriknordmark left a comment

Choose a reason for hiding this comment

eriknordmark left a comment

Choose a reason for hiding this comment

milan-zededa commented Dec 22, 2023 •

edited

Loading

codecov bot commented Dec 22, 2023 •

edited

Loading

milan-zededa Jan 10, 2024 •

edited

Loading

milan-zededa Jan 4, 2024 •

edited

Loading

naiming-zededa Jan 3, 2024 •

edited

Loading

milan-zededa Jan 4, 2024 •

edited

Loading