Do not send traffic to a service until pods are ready.

[uptickmetachu]

I love the simplicity of nginx-gateway-fabric vs kong and other implementations.

One thing I've noticed in my testing is that nginx-gateway-fabric is happy to route traffic to a service while pods are still spinning up.

With the below example HTTPRoute, if the pods for the echo2 service are still waiting to be ready, nginx-gateway-fabric will happily route traffic to non ready pods and the user will receive 500 errors. My preference would be to only route traffic to the other service some pods are ready.

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: echo
spec:
  parentRefs:
  - name: nginx
  hostnames:
    - 'echo-server.com'
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: echo
      kind: Service
      port: 1027
      weight: 50
    - name: echo2
       kind: Service
       port: 1027
       weight: 50

[mpstefan]

Hey @uptickmetachu, thanks for the feedback!

We took a quick look at this yesterday and we're fairly sure what you're running into is an intentional design from the Gateway API, which should be fixable. If you look at the HTTPRouteRule spec and scroll down to the backendRefs section, you'll see:

When a HTTPBackendRef is invalid, 500 status codes MUST be returned for requests that would have otherwise been routed to an invalid backend. If multiple backends are specified, and some are invalid, the proportion of requests that would otherwise have been routed to an invalid backend MUST receive a 500 status code.

For example, if two backends are specified with equal weights, and one is invalid, 50 percent of traffic must receive a 500. Implementations may choose how that 50 percent is determined.

Because of this behavior, we're fairly certain that your endpoint that is not yet ready is actually invalid until it first comes up. If the endpoint is valid, then becomes not ready, NGF should route the traffic as you expect, to only the service that IS ready. Do you happen to have the statuses of the HTTP Route and Gateway when this issue occurs? That would prove the theory.

Also, this guide might be able to help you route traffic when one or more services are unavailable: https://docs.nginx.com/nginx-gateway-fabric/how-to/maintenance/upgrade-apps-without-downtime/

[uptickmetachu]

Thanks for linking to the spec!

You are correct the endpoint is invalid (echo2 service is not yet created) and because of that, 500s are being returned as per the spec.

Kong's behaviour deviates from the spec. We'll just have to work around this with our own operator. Thanks for your reply!