Add GEP-3155: Complete Backend mTLS Configuration

[mkosieradzki]

What type of PR is this?
/kind gep

What this PR does / why we need it:
Proposal for backend mTLS configuration

Does this PR introduce a user-facing change?:

NONE

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: mkosieradzki / name: Marcin Kosieradzki (0d84adf, b5f5112, ba3084b, 9f99d93, c19be24, cd2fe15, 9c29265, bc3183e, 42498db, cf4906d, 37590b9, f25dbbc, 3408b5a, edbf961, b3dae17, ef5b22b, a2d458b, c46bb1e, 6d1b13c, 912f965, bb48234, 15fdfd9, 308c7b0, 7426dd8)

[k8s-ci-robot]

Welcome @mkosieradzki!

It looks like this is your first PR to kubernetes-sigs/gateway-api 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/gateway-api has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

[k8s-ci-robot]

Hi @mkosieradzki. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[shaneutt]

/cc @shaneutt

[youngnick]

A few pieces of feedback here, but I think that the number one thing that's needed is use cases that show how some of these proposals work, or justify their existence.

In particular, I'm reluctant to add free-text map[string]string fields to the API without a very good explanation and example use cases.

[LiorLieberman]

/cc

[jaishals]

Hello @mkosieradzki , if we specify client cert on Gateway then can you please share some insights as to how would the scenarios where there are services which do not need MTLS but need only End2End TLS coexist in the same Gateway -> Route configurations.

There could be a route targeting service (music) which needs MTLS and another route targeting service (shopping) which only needs End to End TLS.

These scenarios would not work on same Gateway if we use MTLS to all the services

[robscott]

I think the distinction here is that we're providing a client cert at the Gateway level that should be presented if it's requested by the backend. @mkosieradzki had previously included a per-Service override for this config in BackendTLSPolicy. We ran into issues there where the personas attached to BackendTLSPolicy weren't particularly clear (see #3226 to chime in on that discussion), so this iteration of the GEP will leave per-Service overrides out, but I think that's still very much a longer term goal.

[mkosieradzki]

There could be a route targeting service (music) which needs MTLS and another route targeting service (shopping) which only needs End to End TLS.

To my best understanding whether the Gateway will send the certificate to the Backend or not depends on the backend server configuration, i.e. whether it will send CertificateRequest message during the handshake as per https://datatracker.ietf.org/doc/html/rfc5246#section-7.3 or not.

Therefore if backend is not configured to request a client certificate, the Gateway will not send it, even it is configured. My original rationale for adding per-service overrides was about corner cases:

• where server application expects a certificate issued by a different CA
• where server application can work in two modes: if the cert is available or not, and lack of certificate is perfectly fine for the server (and causes fallback to a different authentication method).

I think we should revisit this scenario as soon as we figure out the #3226.

[mikemorris]

Re all backends - should the client cert be presented even when no BackendTLSPolicy targeting the Service backend is present?

[robscott]

In my longer term vision, I don't think we should limit it to this scope. Ideally Gateway owners could provide a set of CAs they trust for certs from backends at the Gateway level and the validation in BackendTLSPolicy actually becomes more of a per-Service override.

Although I don't think it's strictly necessary, I think it could be acceptable to start with this restriction. Of course I certainly don't want to stick with that limitation long term.

[mikemorris]

Ideally Gateway owners could provide a set of CAs they trust for certs from backends at the Gateway level

I think this makes sense and could potentially address the non-mutual end-to-end TLS functionality @jaishals is describing in #3180 (comment), where the presence of BackendTLSPolicy additional validation is specifically to enforce mutual TLS, restricting the set of acceptable client certificates? Would any additional signal like mode: mutual be needed somewhere to clarify this?

[mikemorris]

I think this is intended to be applicable to the "server"/destination cert, but can you please specify more precisely which cert is affected here?

[robscott]

Let's use "Backend" here to line up with the terminology in https://gateway-api.sigs.k8s.io/geps/gep-2907/#frontend-and-backend.

[mikemorris]

What happens if it doesn't? Suggested status condition on the policy indicated it's invalid?

[robscott]

Apparently we haven't defined how a request should be rejected here:

gateway-api/apis/v1alpha3/backendtlspolicy_types.go

Lines 128 to 130 in c94d401

	// 1. Hostname MUST be used as the SNI to connect to the backend (RFC 6066).
	// 2. Hostname MUST be used for authentication and MUST match the certificate
	// served by the matching backend.

At a minimum I think we can agree that the request should be rejected, ideally we can be more specific here for the sake of writing conformance tests and overall portability.

As far as populating status, I think that most controllers are separated from the dataplane and would not be able to translate runtime TLS validation errors to config time (k8s) status.

[mikemorris]

This is missing an API example - seems to be suggesting the addition of a new field under the Gateway spec stanza https://gateway-api.sigs.k8s.io/reference/spec/#gateway.networking.k8s.io%2fv1.Gateway?

[youngnick]

Yes, I think this should have a quick sketch of what we need to add to Gateway.

[robscott]

Thanks @mkosieradzki! A few more comments, but mostly LGTM.

[robscott]

I think the distinction here is that we're providing a client cert at the Gateway level that should be presented if it's requested by the backend. @mkosieradzki had previously included a per-Service override for this config in BackendTLSPolicy. We ran into issues there where the personas attached to BackendTLSPolicy weren't particularly clear (see #3226 to chime in on that discussion), so this iteration of the GEP will leave per-Service overrides out, but I think that's still very much a longer term goal.

[robscott]

In my longer term vision, I don't think we should limit it to this scope. Ideally Gateway owners could provide a set of CAs they trust for certs from backends at the Gateway level and the validation in BackendTLSPolicy actually becomes more of a per-Service override.

Although I don't think it's strictly necessary, I think it could be acceptable to start with this restriction. Of course I certainly don't want to stick with that limitation long term.

[robscott]

Let's use "Backend" here to line up with the terminology in https://gateway-api.sigs.k8s.io/geps/gep-2907/#frontend-and-backend.

[robscott]

Apparently we haven't defined how a request should be rejected here:

gateway-api/apis/v1alpha3/backendtlspolicy_types.go

Lines 128 to 130 in c94d401

	// 1. Hostname MUST be used as the SNI to connect to the backend (RFC 6066).
	// 2. Hostname MUST be used for authentication and MUST match the certificate
	// served by the matching backend.

At a minimum I think we can agree that the request should be rejected, ideally we can be more specific here for the sake of writing conformance tests and overall portability.

As far as populating status, I think that most controllers are separated from the dataplane and would not be able to translate runtime TLS validation errors to config time (k8s) status.

[robscott]

/ok-to-test

[youngnick]

I've been picky about wording, but once the two wording changes I've asked for are handled, and we have an example of what we are adding to Gateway, then this LGTM.

[robscott]

Thanks @mkosieradzki! Added a few formatting nits but otherwise LGTM.

/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mkosieradzki, robscott, youngnick

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [robscott,youngnick]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[youngnick]

With that last round of changes, this LGTM. I'll leave the final unhold for @robscott though, since he has some outstanding format changes.

/lgtm

[robscott]

Thanks @mkosieradzki!

/lgtm
/hold cancel