Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add integration patches/CI for Ruby main and 3.3 #2071

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

samuel40791765
Copy link
Contributor

Issues:

Resolves CryptoAlg-2784

Description of changes:

Ruby 3.3 is relatively the same as 3.2 with just one more test failure due to conflicting error messages.
Ruby's master branch has added much more however.

  1. More tests revolving internal session caching on the SSL client. We don't support these tests and I've adjusted accordingly.
  2. We don't support the ancient MSIE extensions that this PKCS12 commit introduces: ruby/ruby@63e9eaa I don't think there's an actual ask for us to support this, so I've ifdefed out the symbols and skipped the test when building with AWS-LC.
  3. The PKCS8 RSA private key in test/openssl/pkey_rsa.rb generated by certtool isn't parsable by us. I've pinned down the reason to a missing NULL field in the ASN1 contents. I've cut an issue to gnutls, more details can be found there.
    I've replaced the file that Ruby's been using for the time being and replaced the original generation to use OpenSSL instead. OpenSSL's PKCS8 files adhere to the RFC.

Call-outs:

N/A

Testing:

New CI

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and the ISC license.

@samuel40791765 samuel40791765 requested a review from a team as a code owner December 20, 2024 01:38
@codecov-commenter
Copy link

codecov-commenter commented Dec 20, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 79.03%. Comparing base (028cd9f) to head (8c472c9).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2071      +/-   ##
==========================================
- Coverage   79.04%   79.03%   -0.01%     
==========================================
  Files         612      612              
  Lines      106064   106064              
  Branches    14994    14995       +1     
==========================================
- Hits        83833    83824       -9     
- Misses      21579    21587       +8     
- Partials      652      653       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@samuel40791765 samuel40791765 force-pushed the ruby-integration-main branch 2 times, most recently from e146fd0 to 584efab Compare December 21, 2024 00:23
@WillChilds-Klein
Copy link
Contributor

WillChilds-Klein commented Dec 30, 2024

Per a recent test run, it looks like we need to implement PKCS12_set_mac as of last week:

  ./test/runner.rb: OpenSSL::TestPKCS12#test_set_mac_pkcs12kdf: symbol lookup error: /home/runner/work/aws-lc/aws-lc/RUBY_BUILD_ROOT/ruby-src/master/.ext/x86_64-linux/openssl.so: undefined symbol: PKCS12_set_mac
  make: *** [uncommon.mk:963: yes-test-all] Error 127

Even when under review, these integration tests help us respond to upstream changes :)

@WillChilds-Klein
Copy link
Contributor

I've replaced the file that Ruby's been using for the time being and replaced the original generation to use OpenSSL instead.

Are we planning to submit a PR upstream to Ruby?

@samuel40791765
Copy link
Contributor Author

Ahh I'll put up a PR to add support for PKCS12_set_mac. Thanks for the call out!

Are we planning to submit a PR upstream to Ruby?

Yeah I've submitted a PR upstream and cut an issue to GnuTLS as well. ruby/openssl#830

samuel40791765 added a commit that referenced this pull request Jan 22, 2025
Ruby's made a couple larger refactors to require versions later than
OpenSSL 1.1.1.

These changes require us to make a few tweaks to the patch in
#2071 and have exposed a couple minor
symbols that we don't support. Adding support for the ones that aren't
complicated in this commit.

By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license and the ISC license.
@samuel40791765 samuel40791765 force-pushed the ruby-integration-main branch 3 times, most recently from 01e7f25 to 3ee9bb3 Compare January 23, 2025 20:32
manastasova pushed a commit to manastasova/aws-lc that referenced this pull request Jan 30, 2025
Ruby's made a couple larger refactors to require versions later than
OpenSSL 1.1.1.

These changes require us to make a few tweaks to the patch in
aws#2071 and have exposed a couple minor
symbols that we don't support. Adding support for the ones that aren't
complicated in this commit.

By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license and the ISC license.
samuel40791765 added a commit that referenced this pull request Feb 5, 2025
 Ruby has added a binding for an additional function called
`PKCS12_set_mac` in it's master branch: ruby/ruby@c79b435.
This was discovered prior to the PR for Ruby's CI integration with the
master branch being merged: #2071

OpenSSL's implementation of the function directly sets a designated mac
field within the `PKCS12` structure. Our `PKCS12` structure is folded
into a string of bytes along with its length and there aren't any
available fields for us to directly set. This means that AWS-LC's
implementation of `PKCS12_set_mac` has to parse the proper contents from
`PKCS12` and rerun the key and mac generation with the new parameters
provided.

### Call-outs:
1. The parsing logic is similar to `PKCS12_get_key_and_certs`, but the
logic can't be shared. This is because we need to maintain pointers to
certain parts of the CBS parsers in `PKCS12_set_mac`, so that we can
properly rewrite the contents later.
2. I've abstracted the key and mac generation in `PKCS12_create` and
`PKCS12_set_mac` to a single function called `pkcs12_gen_and_write_mac`.
3. OpenSSL 3.0 uses SHA-256 as the default hash, yet OpenSSL 1.1.1 uses
SHA-1. We've decided to go with SHA-1 for now for better 1.1.1 compatibility.
HMAC's security also does not rely on the hash function's collision resistance
property.

By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license and the ISC license.
justsmth
justsmth previously approved these changes Feb 5, 2025
tests/ci/integration/run_ruby_integration.sh Show resolved Hide resolved
justsmth
justsmth previously approved these changes Feb 5, 2025
@samuel40791765 samuel40791765 force-pushed the ruby-integration-main branch 2 times, most recently from a89863d to 356f180 Compare February 10, 2025 19:41
make test-all TESTS="test/rubygems/test*.rb"

if [[ "${branch}" != "master" ]]; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we drop a comment as to why we don't run DRB tests against master?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants