Support MacOS keychain for ca's

[MansM]

As a skopeo user within an enterprise I want to use skopeo
on my local mac talking to a registry from the company running an corporate ca signed cert
in order to do skopeo inspect/copy/etc

the error message I got:
x509: certificate signed by unknown authority

[mtrmac]

Thanks for your report.

The docker/distribution client, at least, uses the system’s CAs as provided by Golang’s x509.SystemCertPool; AFAICS that does use the OS configuration on macOS (using SecTrustSettingsCopyCertificates per src/crypto/x509/root_cgo_darwin.go) — Go does not have its own list of CAs it seems, and skopeo definitely does not have a separate one. So, something else might be going on.

Please provide precise instructions how to reproduce this, and include the output of skopeo --debug $your_command.

[MansM]

we use artifactory as registry
installed skopeo using brew: skopeo version 0.1.31

skopeo --debug inspect docker://internal.registry/openshift/origin-pod:v3.9.0
DEBU[0000] Using registries.d directory /etc/containers/registries.d for sigstore configuration
DEBU[0000] No signature storage configuration found for internal.registry/openshift/origin-pod:v3.9.0
DEBU[0000] Looking for TLS certificates and private keys in /etc/docker/certs.d/internal.registry
DEBU[0000] GET https://internal.registry/v2/
DEBU[0000] Ping https://internal.registry/v2/ err &url.Error{Op:"Get", URL:"https://internal.registry/v2/", Err:x509.UnknownAuthorityError{Cert:(*x509.Certificate)(0xc00042c000), hintErr:error(nil), hintCert:(*x509.Certificate)(nil)}}
DEBU[0000] GET https://internal.registry/v1/_ping
DEBU[0000] Ping https://internal.registry/v1/_ping err &url.Error{Op:"Get", URL:"https://internal.registry/v1/_ping", Err:x509.UnknownAuthorityError{Cert:(*x509.Certificate)(0xc000230000), hintErr:error(nil), hintCert:(*x509.Certificate)(nil)}}
FATA[0000] pinging docker registry returned: Get https://internal.registry/v2/: x509: certificate signed by unknown authority

[mtrmac]

Thanks; that uses the default Golang TLS configuration, with only https://github.com/containers/image/blob/bd10b1b53b2976f215b3f2f848fb8e7cad779aeb/docker/docker_client.go#L137 changed; that doesn’t look like it should change things.

[MansM]

ehm. It could be me (probably), but I have no idea what you are trying to say.
Should something be fixed in skopeo or in docker_client?

[mtrmac]

That’s still unknown; the discrepancy between the certificates you have configured in the macOS keychain and the certificates used by the Golang TLS stack needs to be debugged.

Honestly I’m completely unfamiliar with the macOS keychain API (or, really, debugging any macOS native code), so I’m afraid I’m not likely to be that much help with this, in the very near future at least. #563 (comment) is all the pointers I can provide for that right now.

Maybe if you can find some other Golang-implemented TLS client that does work, without extra configuration, with the CA trusted by the keychain, that could serve as an indication what needs to be done. But, again, AFAICT the Golang standard library intends to have the keychain roots of trust work by default, and skopeo does not do anything explicit to break that, so it seems likely that something non-trivial is going on.

[rhatdan]

@MansM Is this still something you are interested in? Could you help out with a PR? This is not something a bunch of Linux programmers are likely to fix, so we need help from someone with Mac Experience.

[MansM]

I wish I could, but I lack the knowledge on MacOS internals and go code skills

if you want me to run some debug code and provide you with the output I am happily to do so, but for this I am too much a consumer

[lbscorpio]

@MansM How about adding the certificate to your macOS truststore as a workaround? I did that then skopeo working fine.

The command to do that as below, fyi, thanks.

$ sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain <registry_cert.crt>

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[lsm5]

@ashley-cui @rhatdan do we have a mac package for skopeo? Can we use @lbscorpio 's suggestion to the package or should that be left to the end user?

[ashley-cui]

Skopeo's is packaged via brew, but I think its maintained by community. Brew doesn't like sudo commands, so I think this might be left for the end user.

[lsm5]

Skopeo's is packaged via brew, but I think its maintained by community. Brew doesn't like sudo commands, so I think this might be left for the end user.

Thanks @ashley-cui . Closing...

[lsm5]

submitted #1464 to address this in install doc. PTAL

[mtrmac]

@MansM How about adding the certificate to your macOS truststore as a workaround?

Is that a workaround? I understood the original report to mean that CAs added to the trusted certificate keychain are ignored; are you just reporting that this actually works fine?

$ sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain <registry_cert.crt>

Or is there some subtlety to the command above, maybe adding the certificate to an unexpected keychain, or using some unexpected option combination, in a way that is a workaround for some bug that still continues to exist?