Project network isolation can be enabled if RKE network plugin supports network policies

[janeczku]

What kind of request is this (question/bug/enhancement/feature request):
Enhancement

Steps to reproduce (least amount of steps as possible):
When configuring an RKE network provider that supports Kubernetes network policies, such as the Cisco ACI CNI plugin, Rancher should allow the cluster option to enable Project Network Isolation to be set (enable_network_policy: true).

Currently, when enabling the network isolation with such a compatible network provider the cluster provisioning fails with a validation error.

The validation is implemented here:

rancher/pkg/api/norman/store/cluster/cluster_store.go

Lines 713 to 714 in 41045a8

	if plugin != "canal" {
	return fmt.Errorf("plugin %s should have enableNetworkPolicy %v", plugin, !value)

Other details that may be helpful:

Rancher 2.5.5

[ryansann]

Support Matrix

Rancher RKE Cluster Project Network Isolation (PNI) support per CNI

Network Provider	Supports PNI
Calico	Yes
Canal	Yes
Weave	Yes
ACI	Yes
None/Custom	Yes
Flannel	No

Testing

Currently Rancher only allows for PNI to be enabled when selecting the Canal Network Provider when provisioning an RKE cluster. These changes allow for PNI to be enabled when using other CNIs.

Until this UI issue is complete, you can provision a cluster with PNI enabled by Selecting your CNI, editing the cluster as YAML, and flipping the top-level enable_network_policy field to true in the RKE config.

What follows is an outline of how I tested these changes.

Allowed Network Paths

With PNI enabled, and the below testing setup, these are the network paths that should be allowed

Disallowed Network Paths

And these are the network paths that should not be allowed, since the traffic crosses project boundaries

Steps

1. Launch a cluster with a CNI that supports PNI (see support matrix above) and with PNI enabled
2. Create a project: pni-testing and a namespace within it testing
3. Launch the Rancher Hello World example as a workload in both the default and pni-testing projects, e.g.
   
4. Create an ingress for each hello-world workload, e.g.
   
5. Navigate to the generated xip.io url of the ingress and validate that the hello-world application loads, e.g.
   
   Do this for the hello-world deployed in the default project and the pni-testing project
6. Deploy connection testing workloads

• NOTE: this step uses this simple app
• deploy conn-test workload to default project, which will be configured to talk to the hello-world workload in the same, default project. This is an allowed network path. e.g.
  
• deploy conn-test-cross-proj workload to default project. This workload is configured to talk to the hello-world workload in the pni-testing project. This is a disallowed network path as it crosses project boundaries e.g.
  
• repeat the above with the pni-testing project, i.e. deploy conn-test and conn-test-cross-proj there as well
• validate that all allowed network paths are in fact working and all disallowed network paths are receiving i/o timeouts. To do this, check that the logs of each conn-test workload have entries like got succesful response: 200 OK and each conn-test-cross-proj workload has logs like: error calling http://hello-world.default.svc.cluster.local: Get "http://hello-world.default.svc.cluster.local": dial tcp 10.42.0.8:80: i/o timeout

7. Repeat these steps for clusters provisioned with other CNIs that support PNI

[bmdepesa]

rancher/rancher:v2.5-head 31f00c1

Tested enabling Project Network Isolation with the following CNIs:

• Canal
• Calico
• Weave

Test Case

• Create multiple projects and deploy 2 workloads in each using ranchertest/mytestcontainer image
• From each project, ping a workload in the project and in another project
  • Workloads within the same project have a successful ping
  • Workloads in separate projects are not able to communicate
• Ran automated test test_network_policy

2.6 Forwardport - #32220

UI Validations - #32106