tls-gnutls.c should include the digitalSignature key usage bit.

[davidben]

When CUPS is linked against current versions of GnuTLS with TLS 1.3 support, the self-signed certificates it generates are incompatible with Chrome's TLS 1.3 support, and any other clients using BoringSSL.

I believe the problem is this line here:
https://github.com/apple/cups/blob/c4edc06/cups/tls-gnutls.c#L174

It should pass in GNUTLS_KEY_DIGITAL_SIGNATURE | GNUTLS_KEY_KEY_ENCIPHERMENT.

In TLS 1.2, RSA keys may be used in two different ways. The legacy TLS_RSA_WITH_* cipher suites involve the client generating a secret and encrypting it to the private key. This corresponds to the keyEncipherment bit. The modern TLS_ECDHE_RSA_WITH_* cipher suites involve the server signing some ECDH parameters with its RSA key. This corresponds to the digitalSignature bit. In TLS 1.3, only the latter flow exists.

(See https://tools.ietf.org/html/rfc5280#section-4.2.1.3 for more info on key usage bits.)

The certificates generated by CUPS only include the keyEncipherment bit, so they break with TLS 1.3 clients that check the key usage bit, such as BoringSSL. They're additional incompatible with modern ciphers. It looks like GnuTLS detects this and disables TLS_ECDHE_RSA_WITH_* ciphers at TLS 1.2, but it doesn't detect the more global TLS 1.3 incompatibility.

I believe changing that line should both fix the problem at TLS 1.3, as well as enable more modern ciphers for TLS 1.2.

[michaelrsweet]

[master e625fa1] Fix a GNU TLS certificate problem (Issue #5506)

[branch-2.2 324a116] Fix a GNU TLS certificate problem (Issue #5506)