Would Gateway Merging make TLSRoute Unnecessary?

[robscott]

TLSRoute has always been a bit of a strange resource because the only matcher it had (SNI) was already expressible via Gateway. The primary arguments in favor of TLSRoute became:

1. We wanted to allow delegation to different teams, often across namespaces
2. It allowed you to express as many SNIs as your implementation would allow you to match
3. It was relatively consistent with HTTPRoute (just much simpler)

Now let's imagine that @dprotaso's proposal for Gateway merging (#1863) is accepted. In that world, you could use the combination of Gateway merging + TCPRoute to cover 1 and 2. For 1 you simply delegate to a Gateway in a different namespace instead of a TLSRoute, and for 2, the Gateway merging proposal also allows for ~infinite merged Gateways.

Given that the only similarity between HTTPRoute and TLSRoute is hostname/SNI matching, it doesn't really seem worthwhile to keep an entire resource just for the sake of that bit of consistency...

With all of that said, there are still some valid arguments in favor of TLSRoute:

1. It is the only way to do TLS passthrough/SNI-based routing with meshes today. It's unclear how widely used this is, but it would be useful to get some broader feedback here (cc @howardjohn @kflynn @mikemorris).
2. TLSRoute could theoretically be expanded to support something like ALPN matching in the future, it's unclear if/how we'd want to support that on a Gateway resource.

I'm not really sure what the answer is here, but figured I'd at least start a discussion to see if anyone else had an opinion on this. Trying to avoid providing too many ways to do the same thing with this API.

[howardjohn]

Another argument is the persona/RBAC split. Gateway create is a very high privilege, TLSRoute create is less so.

[youngnick]

Thinking more about Rob's point though - some points about doing Gateway merging with another object name (think ListenerSet or ListenerGroup or something):

• separate support option for implementations (you don't have to support the resource)
• separate resource makes RBAC straightforward
• Uses existing parentRef schema so will use Experimental more cleanly
• Can easily add in the extra fields for doing TLSRoute/Other L4 routes without having to make changes (yet) to a v1 resource - Gateway

[mlavacca]

Can easily add in the extra fields for doing TLSRoute/Other L4 routes without having to make changes (yet) to a v1 resource - Gateway

I think this is a good point. In case we have the desire to improve L4 routing with additional capabilities, like ALPN matching, adding those to the Gateway is a bit of a stretch in my opinion.

Also, using a separate resource would make RBACs easier as Nick pointed out, without getting into the ValidatingAdmissionPolicy realm.

[mpstefan]

Adding my 2 cents here... If I understand the Gateway API well enough, I think we'd be introducing more complexity than we'd be replacing for enabling the TLS passthrough use case.

If we rely on the gateway resource to handle the configuring of the "TLS Route," then that essentially means application developers need to cut a ticket to their operations team or own their own Gateway to configure traffic to hit their application - the latter option breaking that role-based design. Not to mention I know some organizations, not knowing the entire breadth of Gateway API, would just give everyone access to the single gateway and get themselves in a bad position, repeating the pain of Ingress.

If we create another resource to separate RBAC, like the ListenerSet or ListenerGroup above, then you have application developers configuring listeners for the cluster which should be under the role of the operator. You'd also be replacing TLSRoute with another, arguably, more complex resource with these listener sets that you'd also have to manage.

I know Rob said above that it's probably not worth it to keep the TLSRoute resource for the sake of consistency, but I honestly think it might be. It is a bit awkward and sparse, but it more closely follows our role-based design and won't prompt the user to learn another configuration path for them to configure traffic to hit their application.

As somehow merging TLSRoute and TCPRoute would be even more awkward, I think keeping the TLSRoute as is might actually be our best option. I could be mistaken on any of the points above though!

[robscott]

Lots of great points here, thanks everyone! Definitely agree that the overall persona + RBAC split is an important consideration, and I think we're coming back to one of the original questions of the API - where a Gateway ends and a Route begins. I've created a doc for the purpose of brainstorming ideas further - any/all ideas are very welcome.

[mikemorris]

As somehow merging TLSRoute and TCPRoute would be even more awkward

I'm honestly still not convinced that this is entirely implausible - adding hostnames to TCPRoute and explicitly defining behavior in the following cases feels like it should be possible:

• attached to a Listener setting mode: Passthrough, functions as TLSRoute does today
  • backendRef without BackendTLSPolicy
    • normal TLSRoute behavior, hostnames on route is valid discriminator
  • backendRef with BackendTLSPolicy applied
    • BackendTLSPolicy is N/A, hostnames still valid discriminator, unclear how to communicate that BackendTLSPolicy is invalid or ignored in this case
• attached to a Listener setting mode: Terminate, functions as TCPRoute does today
  • backendRef without BackendTLSPolicy
    • TODO: hostnames on route invalid/ignored, TLS has already been terminated? unclear
  • backendRef with BackendTLSPolicy applied
    • "re-encrypt", is hostnames on route relevant here?

For further context, @kate-osborn raised questions around these uses cases initially over at https://kubernetes.slack.com/archives/CR0H13KGA/p1718292200173799

Relying on Listener config to determine behavior is a bit awkward for hypothetical mesh use cases for TCPRoute or TLSRoute but still better than removing these route types entirely and maybe safe enough to collapse that distinction with a mesh implementation providing implied mTLS without explicit configuration. istio/istio#49973 is an example of requested functionality for hostnames within mesh routing using HTTPRoute, which both Istio and Linkerd are considering supporting, and I could see being relevant for L4 routing too.

/cc @kflynn

istio/istio#51768 is a weirder issue relating to a mesh application wanting to use an ALPN match for routing, IDK how relevant this use case is though.