Define multiple GatewayClass and Gateways for traffic isolation in K8s

[devcwm]

I have been trying to implement NGF on K8s. Currently basic setup is working as expected.

My question is related to how can I define multiple GatewayClass and Gateway for purpose of having seperates Gateways for HTTPRouts to connect. Fox example, infra tools routes shall use dedicated Infra Gateway, internal tools routes shall use dedicated Internal Gateway and so on.

I'm able to currently define separate NodePort services that allow connections from AWS LB to dedicated ports but they all tie to a single Gateway.

Currently when deploying multiple Gateways or GatewayClasses using NGF, I'm getting The resource is ignored due to a conflicting GatewayClass resource message for GatewayClassConflict. Same thing for Gateways deployments results in conflict.

If NGF currently don't support multiple Gateways declaration, what are my alternatives apart from going back to Ingress rout? Appreciate your help and suggestions.

[pleshakov]

Hi @devcwm

At the moment, NGF only supports one Gateway resource per NGF installation.

For your use case, to support multiple Gateway resources, it is necessary deploy multiple NGFs where each NGF needs to be deployed:

• with a different value of gateway-ctlr-name parameter -- https://docs.nginx.com/nginx-gateway-fabric/reference/cli-help/#flags --

  nginx-gateway-fabric/deploy/manifests/nginx-gateway.yaml

  Line 146 in 4e3d9c4

  - --gateway-ctlr-name=gateway.nginx.org/nginx-gateway-controller

• with a GatewayClass with the value of controllerName that matches the parameter above --

  nginx-gateway-fabric/deploy/manifests/nginx-gateway.yaml

  Line 253 in 4e3d9c4

  controllerName: gateway.nginx.org/nginx-gateway-controller

For example, you can have:

• InfraTools NGF with gateway.nginx.org/nginx-gateway-controller-infra-tools parameter and GatewayClass nginx-infra-tools with controllerName gateway.nginx.org/nginx-gateway-controller-infra-tools. You can create a Gateway resource that references GatewayClass nginx-infra-tools.
• Internal Tools NGF with gateway.nginx.org/nginx-gateway-controller-internal-tools parameter and GatewayClass nginx-internal-tools with controllerName gateway.nginx.org/nginx-gateway-controller-internal-tools. You can create a Gateway resource that references GatewayClass nginx-internal-tools.

If you use helm, you can use nginxGateway.gatewayControllerName and nginxGateway.gatewayClassName parameters to support the use case above. See https://github.com/nginxinc/nginx-gateway-fabric/tree/v1.1.0/deploy/helm-chart

We do have plans to support multiple Gateways per GatewayClass in the future (there is no issue in our GitHub repo yet though)

[devcwm]

@pleshakov thank you for reply. I'll try to implement multiple NGF installation. With this approach one issue I'm facing is resource duplication.

• Do you suggest to change every resource name to -internal-tools or -infra-tools in nginx-gateway.yaml ?
• Also, shall NGF deployment deployed to separate namespace or same one shall be fine as nginx-gateway ?
• For the roles and role bindings, shall they be declared again with separate name or first declaration in NGF deployment should be enough.

Another approach I tried working on to deploy using helm but ArgoCD doesn't deploy helm on OCI from gitops repo. https://github.com/nginxinc/nginx-gateway-fabric/tree/v1.1.0/deploy/helm-chart. Is there a way to use traditional helm charts instead of OCI?

Appreciate your help!

[pleshakov]

@devcwm

With this approach one issue I'm facing is resource duplication.

I suggest having separate manifests including Roles and RoleBinding per NGF installation.
Yep, you can run them in the same namespace.
You can generate such manifests using helm. For example:

helm template  ngf-infra-tools  oci://ghcr.io/nginxinc/charts/nginx-gateway-fabric -n nginx-gateway --set nginxGateway.gatewayControllerName=gateway.nginx.org/nginx-gateway-controller-infra-tools --set nginxGateway.gatewayClassName=nginx-infra-tools

Another approach I tried working on to deploy using helm but ArgoCD doesn't deploy helm on OCI from gitops repo. https://github.com/nginxinc/nginx-gateway-fabric/tree/v1.1.0/deploy/helm-chart. Is there a way to use traditional helm charts instead of OCI?

As a workaround, can you configure ArgoCD to install helm chart from sources. For example, something like below:

git clone https://github.com/nginxinc/nginx-gateway-fabric.git
cd nginx-gateway-fabric
git checkout v1.1.0
helm install ...  deploy/helm-chart

[devcwm]

@pleshakov sounds great, thank you! I got it working by deploying manifests in separate namespaces by changing naming for resources as per your suggestion.

I'll also check out above two options you suggested to generate and deploy using source helm chart. Thank you!

Looking forward to have multiple gateways supported natively.

[mpstefan]

I did add a ticket to track this particular item here: #1443

Can't guarantee it will be in 1.3, but it is definitely something we need to get to.