Add default kubernetes network policy support

[smlx]

Checklist

• Affected Issues have been mentioned in the Closing issues section
• Documentation has been written/updated See Document new feature flags #2541.
• PR title is ready for changelog and subsystem label(s) applied

This PR adds a network policy which is applied to Lagoon environment namespaces. It is disabled by default and controlled via a feature flag as documented in #2541.

The default policy currently implements namespace isolation by filtering Ingress traffic. Egress traffic is not filtered.

Because Egress traffic is not filtered, internal cluster services can still be consumed by Lagoon project workloads with this network policy applied. This is required for logging to application-logs.lagoon.svc, and is also common in dedicated Lagoon clusters (e.g. shared Elasticsearch).

Network policies only affects internal cluster traffic. Traffic which routes between Lagoon projects via a public URL (e.g. via an ingress) is not affected. This is the case with e.g. decoupled projects where a frontend consumes a backend API via a public URL.

Similarly to #2481, this depends on uselagoon/lagoon-charts#235. Once that lands, we can remove 83be81f from this PR.

Closing issues

n/a

[Schnitzel]

I think we also need to allow traffic from all monitoring namespaces? (not exactly sure how we could select them, but at least lagoon-monitoring namespaces should probably allow traffic?)

[smlx]

Yes, there are likely to be other exceptions. I'm going to do an audit to check for more.

As the namespace selection relies on labels, we'll have to add a similar name: <name> label to any namespaces that need to be targeted.

edit: we now simply disallow traffic between Lagoon environment namespaces. Other namespaces are not affected.

[tobybellwood]

Confirmed working in local branch.

[ci-active-standby-control-k8s]master-a@nginx:/app$ nc -zvw 5 nginx.ci-active-standby-control-k8s-master-a.svc 8080
nginx.ci-active-standby-control-k8s-master-a.svc (10.96.129.20:8080) open
[ci-active-standby-control-k8s]master-a@nginx:/app$ nc -zvw 5 nginx.ci-active-standby-control-k8s-master-b.svc 8080
nc: nginx.ci-active-standby-control-k8s-master-b.svc (10.96.246.9:8080): Operation timed out
[ci-active-standby-control-k8s]master-a@nginx:/app$ nc -zvw 5 lagoon-core-api.lagoon.svc 80
lagoon-core-api.lagoon.svc (10.96.116.69:80) open
[ci-active-standby-control-k8s]master-a@nginx:/app$ nc -zvw 5 minio.minio.svc 9000
minio.minio.svc (10.96.149.208:9000) open
[ci-active-standby-control-k8s]master-a@nginx:/app$ nc -zvw 5 registry-harbor-portal.registry.svc 80
registry-harbor-portal.registry.svc (10.96.226.103:80) open

Have also pushed a testing/network-policies branch with Calico CNI enabled for local/CI testing. Test to be developed.