Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update source build documentation and Mozilla CA certificate store to latest (10-14-2020) for libgit2 SSL. #38089

Merged
merged 1 commit into from
Oct 21, 2020

Conversation

mikhail-j
Copy link
Contributor

@mikhail-j mikhail-j commented Oct 18, 2020

@staticfloat

The curl project has updated the Mozilla CA certificate store earlier this week (10/14/2020).

This update coincides with CA certificate list changes to Mozilla NSS 3.57.

3 CA certificates were added:

  • Trustwave Global Certification Authority
  • Trustwave Global ECC P256 Certification Authority
  • Trustwave Global ECC P384 Certification Authority

3 CA certificates were removed:

  • Taiwan GRCA (O=Government Root Certification Authority; C=TW)
  • OISTE WISeKey Global Root GA CA
  • EE Certification Centre Root CA

In addition, @yuyichao added the source build workaround for nghttp2 in #37618. However, the corresponding documentation has not been updated. As a result, I have included a commit to add the Makefile flag in this pull request.

There seems to be a incorrect flag somewhere for the include files of MbedTLS when doing source build with USE_BINARYBUILDER=0. This was the following error I encountered:

In file included from ~/julia_test/julia/deps/srccache/libgit2-0ced29612dacb67eefe0c562a5c1d3aab21cce96/deps/ntlmclient/crypt.h:15,
                 from ~/julia_test/julia/deps/srccache/libgit2-0ced29612dacb67eefe0c562a5c1d3aab21cce96/deps/ntlmclient/ntlm.h:14,
                 from ~/julia_test/julia/deps/srccache/libgit2-0ced29612dacb67eefe0c562a5c1d3aab21cce96/deps/ntlmclient/ntlm.c:20:
~/julia_test/julia/deps/srccache/libgit2-0ced29612dacb67eefe0c562a5c1d3aab21cce96/deps/ntlmclient/crypt_mbedtls.h:12:10: fatal error: mbedtls/md.h: No such file or directory
   12 | #include "mbedtls/md.h"
      |          ^~~~~~~~~~~~~~
compilation terminated.
make[4]: *** [deps/ntlmclient/CMakeFiles/ntlmclient.dir/build.make:63: deps/ntlmclient/CMakeFiles/ntlmclient.dir/ntlm.c.o] Error 1
make[3]: *** [CMakeFiles/Makefile2:339: deps/ntlmclient/CMakeFiles/ntlmclient.dir/all] Error 2
make[2]: *** [Makefile:141: all] Error 2
make[1]: *** [~/julia_test/julia/deps/libgit2.mk:54: scratch/libgit2-0ced29612dacb67eefe0c562a5c1d3aab21cce96/build-compiled] Error 2
make[1]: *** Waiting for unfinished jobs....

I encountered this issue on Ubuntu 20.04 x86_64 with cmake v3.16.3.

As a temporary solution, I used USE_SYSTEM_MBEDTLS=1 to compile Julia successfully.

@vchuravy vchuravy requested a review from staticfloat October 18, 2020 20:50
@staticfloat
Copy link
Member

Thanks Mikhael! I am actually going to fix some of the USE_BINARYBUILDER=0 issues today, so I think we can avoid some of these workarounds if I fix them then you rebase this on top of those fixes. :)

@staticfloat
Copy link
Member

I just merged #38115; can you rebase on top of that? I think you won't need the doc changes anymore.

Also, for your mbedTLS build issue, could you see if #38121 fixes it for you?

@mikhail-j mikhail-j force-pushed the libgit2-cacert-october-2020 branch from 1e01617 to a2a4b4b Compare October 21, 2020 00:02
@mikhail-j
Copy link
Contributor Author

@staticfloat

I have force pushed a git reset to my branch to exclude the commits to the documentation as requested.

In addition, the pull request #38121 seems to have fixed the MbedTLS issue I had encountered 2 days ago. I successfully compiled #38121 using USE_BINARYBUILDER=0 on Ubuntu 20.04 x86_64.

@staticfloat
Copy link
Member

Thanks again! Just as a heads up; eventually, we are going to just be getting this from a JLL, so you'll just be updating this over at Yggdrasil.

@staticfloat staticfloat merged commit f39cb43 into JuliaLang:master Oct 21, 2020
@mikhail-j
Copy link
Contributor Author

@staticfloat @tkelman @ararslan @StefanKarpinski @KristofferC

I initially planned to have the source build process use the latest cacert.pem and cacert.pem.sha256 provided by the curl project.

This idea was rejected and the Julia project now uses the cacert.pem corresponding change date. I have maintained the Mozilla CA certificate store versioning for the past 3 years.

I am not against the usage of Yggdrasil in building Julia from scratch. However, I do not wish to keep maintaining the JLL for the Mozilla CA certificate.

Since BinaryBuilder API provides BinaryBuilderBase.FileSource () that takes a SHA256 checksum, the process can be automated by extracting the provided SHA256 checksum from cacert.pem.sha256 from the curl project. This was not possible with the jlchecksum tool suggested back in October 2017 (#24212).

@mikhail-j mikhail-j deleted the libgit2-cacert-october-2020 branch October 21, 2020 03:41
@staticfloat
Copy link
Member

Yes, thank you so much for your very timely and consistent pull requests. We'll work out a proper system that requires no maintenance from you once we get things set up in Yggdrasil.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants