Separate control and data planes

[f5yacobucci]

As a route to efficacy and quickly understanding the Gateway API; its implementation and alignment to NGINX as a data plane, we decided on a simplified, but rigid, deployment pattern. To improve our security posture and installation flexibility the control and data planes should be separated as semi-autonomous, distributed components.

Problem
Control plane containers and data plane containers compose into a single Kubernetes Pod.
Control plane containers use OS signals and file system sharing to exchange data.
Control plane and data plane are governed by the same RBAC policies as they reside in one Pod and ServiceAccount.
Control plane and data plane must scale dependently and cannot scale on independent axis.
Compromise of the control plane may impact customer traffic in the data plane.
Compromise of the data plane may expose Kubernetes API server and impact the cluster and allow horizontal movement in the network.
Kubernetes secrets and sensitive data will be shared across containers unnecessarily.
Violation of a basic zero-trust tenet: "The data plane and control plane are logically separated." NIST SP 800-207

Solution
Separate control and data planes across independent Pods and Deployments.

A/C:

Create design document outlining new architecture and deployment pattern. Should cover communication channels between control and data planes, authentication and authorization, interfaces, and data plane agent options. Both planes should eventually scale, only data plane is required for the first iteration.

• Deployment architecture
• Communication channels / protocols
• Authentication and authorization
• Data plane registration and scaling

Stories

Beta Give feedback

1. Add design doc for Control Plane and Data Plane Separation nginx-kubernetes-gateway#344
   documentation +1
2. Write Dockerfile for data plane nginx-kubernetes-gateway#376
   proposal +1
   
3. Add gRPC server and skeleton for Commander Service nginx-kubernetes-gateway#377
   proposal +1
   
4. Implement agent registration #378
   +0
   
5. Implement config management #379
   enhancement +1
   
6. Separate control plane and data plane #375
   backlog enhancement +2
7. Implement token auth for cp/dp connections #380
   backlog enhancement +2
8. Implement certificate generator #381
   backlog enhancement +2
9. Document CP/DP Architecture #374
   backlog documentation enhancement +3
10. Test cp/dp split #382
    backlog enhancement +2
11. Agent re-registration #459
    backlog enhancement +2

Out of scope:

• Control plane scaling is a non-goal for this iteration. Following work will design HA, leader election, sharding or other techniques based on requirements.
• Explicit support for multiple control planes per gateway class or namespace segmentation. Following work will highlight tenant and other isolation use cases.

Aha! Link: https://nginx.aha.io/features/NKG-72

[mpstefan]

I think before we pick this back up, we may want do two things:

1. Define the value to the user of this new architecture
2. Define the architecture we want to achieve with this change, or over time

[mpstefan]

Mistakenly marked as complete. Moving this functionality to a new epic.