External Data Provider mTLS Certificate Management Strategy

[akashsinghal]

Describe the solution you'd like

I'm looking for feedback and recommended guidance on TLS cert management for an external data provider.

I'm contributing to Ratify which is an external data provider for GK. We currently manually generate the ca Bundle and the server certs and key. But there is no cert management for expiry, revocation, etc. We also would like to have a way for graceful cert rotation which does not involve pod restarts of the ED provider as we have HA scenarios in mind. What is the general recommended approach from GK?

Here's what we currently have in mind:

1. Ratify is introducing a local cert file watcher to pick-up changes to certs. This is modelled on how GK handles it's own TLS cert file changes. Ratify generates and stores the certs in a K8s Secret like GK does. Both the Ratify secret and the GK webhook secret are volume mounted at well-known paths that will be watched.
2. The open-policy-agent/cert-controller project seems like a promising option for the ED to use as well. Thanks to recent contributions, the ED Provider CR's caBundle field can now be updated according to changes in the secret contents. We anticipate these changes mean this is the recommended approach?

The above strategy meets most of our requirements. But there doesn't seem to be a graceful way to rotate the certs while guaranteeing uptime. This issue stems from a 60-90 second secret delay in volume mounting. This can lead to cert mismatches. See this issue for more info. Are there any other recommendations?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

[ritazh]

not stale

[maxsmythe]

At some point it'd be good for cert-controller to allow the appending of multiple certs in the caBundle so that we can do a two-phase rotation, eliminating latency as an issue.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.