Supporting per-SpiffeID Intermediate CAs for things like etcd

[solarkennedy]

Opening and issue following this slack thread, even if just for the SEO for others who try to figure out how to use SPIRE with etcd, even if it is close wont-fix.

This use-case is to support applications that assume that a valid client cert is the authz.

Here are some apps that seem to assume this:

• Etcd
• Kiam

One way to support this in spire could be to support an intermediate CA per SpiffeID, and then use that as the trusted CA (instead of the big root). This does have some scalability and HA concerns though.

How do you recommend we use spire in these use-cases? Or alternatively, how should open source applications be "spire-friendly"?

[azdagron]

There might be something we can do here. I'm not sure per-SPIFFE ID intermediates are the way to go. An intermediate per SPIFFE-ID could order in the many thousands (dependent on the number of unique identities in your trust domain). We could scope it down to something that you configure for some subset of SPIFFE IDs (e.g. those matching spiffe://trustdomain.test/etcd/*). However, each SPIRE server (i.e. HA) would need to maintain that intermediate and rotate it along with the SPIRE server CA. They would need to be stored in the datastore layer and provided via an API. It does add a non-trivial amount of complexity.

There is existing software out there that, while not necessarily being SPIFFE aware, still has minimal support for authorization based on SPIFFE IDs by allowing operators to configure authorization policy over the URI SANs of client certificates. A few examples are proxies like Envoy (match_subject_alt_names field) and Ghostunnel (--allow-uri flag).

You may be able to put a proxy sidecar (either one of the above, or your own custom one) in front of etcd to perform the authorization using SPIFFE ID and the translate that into a separate means of authorization (JWT?) to etcd.

This is only half of the story though. The SPIRE Server CA used to mint identities provided by SPIRE over the SPIFFE Workload API is rotated with (configurable) frequency. This would include the proposed per-spiffe ID intermediates. These new CA certificates are appended to the trust bundle (and old ones pruned out). Services relying on the trust bundle to authenticate identities needs to be made aware of these changes or they will stop being able to authenticate identities. Services that supports SPIFFE directly stream these updates from the SPIFFE Workload API or fetch them from the SPIFFE bundle endpoint. For services that can't be augmented, a sidecar process can be used obtain updates, write those updates to disk, and relaunch the service. The spiffe-helper is one example. It's somewhat trivial to write your own using the go-spiffe library.

[evan2645]

One way to support this in spire could be to support an intermediate CA per SpiffeID, and then use that as the trusted CA (instead of the big root).

Most software will fail to validate a leaf certificate if provided with an intermediate as the only "root". See OpenSSL PARTIAL_CHAIN.

It is unfortunate, as it should be allowed per RFC 5280. Go allows it, but most others don't (at least not in their default configuration).

[esweiss]

Would using the dnsname feature of SPIRE work for you, using CN auth on etcd? CN auth is available on etcd 3.2+

[solarkennedy]

Would using the dnsname feature of SPIRE work for you, using CN auth on etcd? CN auth is available on etcd 3.2+

Not super sure without doing a full test setup our current version of etcd (3.4), there are a lot of different options/gotchas:
https://github.com/etcd-io/etcd/blob/master/Documentation/op-guide/security.md

Specifically around DNS and peers. I think I would need to make sure the CN has a real dns record and that it matches the ip that the peer is coming from: etcd-io/etcd#7767
But yea it looks like 3.3+ has --peer-require-cn, and spire 0.8.0 has the option for setting a dnsname and making the first one be the CN. Hmm yea I think that could work!

[evan2645]

Hey @solarkennedy were you able to get this working?

[solarkennedy]

I did, thank you.
However in the end we went with statically provisioned Certs.
It isn't as cool, but it has fewer moving parts and easier to understand for this particular use case (we don't have dynamic clients)