Align X509 PARTIAL_CHAIN behavior with 1.1.1

[samuel40791765]

Issues:

Resolves V1533483320

Description of changes:

Some consumers noticed that a behavior difference between AWS-LC and OpenSSL when the trust store contains certificates that are issued by other certificates that are also in the trust store. A common example of this would be the trust store containing both the intermediate and the root for the cert chain (leaf -> intermediate -> root). The default settings of AWS-LC and OpenSSL require root to be self signed for the chain to be verified.

Many TLS implementations set the X509_V_FLAG_PARTIAL_CHAIN flag however, which allows non self signed certificates to be trusted in the trust store. When X509_V_FLAG_PARTIAL_CHAIN is set, OpenSSL 1.1.1 will only verify leaf and intermediate, since intermediate is a trusted certificate. However, AWS-LC will continue building a certificate chain and include root within the chain of trust. This causes a behavioral difference with X509_STORE_CTX_get0_chain, where AWS-LC will return all 3 certificates (leaf -> intermediate -> root) and OpenSSL 1.1.1 will only return the first two (leaf -> intermediate ).

Our upstream forked a bit before OpenSSL 1.0.2, so we don't have the new behavior. This described behavioral difference was introduced in openssl/openssl@d9b8b89 (along with many others), but the commit introduces too many backwards incompatible changes for us to take as a whole.
This subtle difference was due to OpenSSL 1.1.1 continuously checking for trust while the chain's being established. The search for the next valid cert breaks early as soon as a valid chain has been built. Our current behavior builds the chain with all possible certs first and only breaks the loop if the final cert in the chain is self-signed. We can inherit this part of 1.1.1's new behavior to fix this issue.

Call-outs:

I don't believe this really changes our X509 chain building or verification by much. We're only adding an additional check for trust while the chain is being established and the final chain still needs to go through the same building/verification process that exists in AWS-LC today.

Testing:

Specific test for X509_V_FLAG_PARTIAL_CHAIN

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and the ISC license.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 78.54%. Comparing base (9ff8458) to head (d0d4542).
Report is 4 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1917      +/-   ##
==========================================
+ Coverage   78.51%   78.54%   +0.03%     
==========================================
  Files         585      585              
  Lines       99681    99793     +112     
  Branches    14271    14285      +14     
==========================================
+ Hits        78265    78385     +120     
+ Misses      20782    20773       -9     
- Partials      634      635       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[torben-hansen]

line 383 is now double-work(?).

But you still need it for the SS (true CA) case. Dunno whether intermediate CA trust is the majority case or not for our applications.

[samuel40791765]

Yeah that was one of the reasons I left it there. Even if having a SS root wasn't the majority case, the implicit default of this function was to have one and check_trust on L383 would help check the default scenario.

The other reason was because X509 chain building/verification has too many subtle nuances for me to feel comfortable moving too many pieces around.. We could rearrange the for loop to not do duplicate work for X509_V_FLAG_PARTIAL_CHAIN, but using an early call to break into our existing logic seemed like the safest and most straightforward path.