Skip to content

Commit

Permalink
upgrade of libgit2 0.27.1 (#27241)
Browse files Browse the repository at this point in the history 
* upgrade libgit2 to 0.27.1
* fix for API changes
  • Loading branch information
@simonbyrne
simonbyrne authored Jun 6, 2018
1 parent 89e445b commit 4a303b3
Show file tree
Hide file tree
Showing 15 changed files with 608 additions and 4,646 deletions.
1 change: 0 additions & 1 deletion deps/checksums/libgit2-15e119375018fba121cf58e02a9f17fe22df0df8.tar.gz/md5
View file

This file was deleted.

1 change: 0 additions & 1 deletion deps/checksums/libgit2-15e119375018fba121cf58e02a9f17fe22df0df8.tar.gz/sha512
View file

This file was deleted.

1 change: 1 addition & 0 deletions deps/checksums/libgit2-b0d9952c318a3d1b8917e06ad46b9110c0c28831.tar.gz/md5
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
6059f607530c302aa1b74c77fcbb5fa2
1 change: 1 addition & 0 deletions deps/checksums/libgit2-b0d9952c318a3d1b8917e06ad46b9110c0c28831.tar.gz/sha512
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
6827d19be048be94d87d118c04c37ee9fc8161d1fb5820fd0feadc8b9d99f08835e1c5ccab47ae2406cb4601efce0d30d03bd66d1ec6e25d591c3ba1e0073ec7
31 changes: 1 addition & 30 deletions deps/libgit2.mk
View file
Original file line number Diff line number Diff line change
Expand Up @@ -42,12 +42,7 @@ endif

LIBGIT2_SRC_PATH := $(SRCCACHE)/$(LIBGIT2_SRC_DIR)

$(LIBGIT2_SRC_PATH)/libgit2-ssh.patch-applied: $(LIBGIT2_SRC_PATH)/source-extracted
cd $(LIBGIT2_SRC_PATH) && \
patch -p0 -f < $(SRCDIR)/patches/libgit2-ssh.patch
echo 1 > $@

$(LIBGIT2_SRC_PATH)/libgit2-mbedtls.patch-applied: $(SRCCACHE)/$(LIBGIT2_SRC_DIR)/source-extracted | $(LIBGIT2_SRC_PATH)/libgit2-ssh.patch-applied
$(LIBGIT2_SRC_PATH)/libgit2-mbedtls.patch-applied: $(SRCCACHE)/$(LIBGIT2_SRC_DIR)/source-extracted
cd $(LIBGIT2_SRC_PATH) && \
patch -p1 -f < $(SRCDIR)/patches/libgit2-mbedtls.patch
echo 1 > $@
Expand All @@ -57,25 +52,6 @@ $(LIBGIT2_SRC_PATH)/libgit2-agent-nonfatal.patch-applied: $(LIBGIT2_SRC_PATH)/so
patch -p1 -f < $(SRCDIR)/patches/libgit2-agent-nonfatal.patch
echo 1 > $@

$(LIBGIT2_SRC_PATH)/libgit2-mbedtls-verify.patch-applied: $(LIBGIT2_SRC_PATH)/source-extracted | $(LIBGIT2_SRC_PATH)/libgit2-agent-nonfatal.patch-applied
cd $(LIBGIT2_SRC_PATH) && \
patch -p1 -f < $(SRCDIR)/patches/libgit2-mbedtls-verify.patch
echo 1 > $@

$(LIBGIT2_SRC_PATH)/libgit2-mbedtls-fixup.patch-applied: $(LIBGIT2_SRC_PATH)/source-extracted | $(LIBGIT2_SRC_PATH)/libgit2-mbedtls-verify.patch-applied
cd $(LIBGIT2_SRC_PATH) && \
patch -p1 -f < $(SRCDIR)/patches/libgit2-mbedtls-fixup.patch
echo 1 > $@

$(LIBGIT2_SRC_PATH)/libgit2-ssh-loop.patch-applied: $(LIBGIT2_SRC_PATH)/source-extracted | $(LIBGIT2_SRC_PATH)/libgit2-mbedtls-fixup.patch-applied
cd $(LIBGIT2_SRC_PATH) && \
patch -p1 -f < $(SRCDIR)/patches/libgit2-ssh-loop.patch
echo 1 > $@
$(LIBGIT2_SRC_PATH)/libgit2-bundled_zlib.patch-applied: $(LIBGIT2_SRC_PATH)/source-extracted | $(LIBGIT2_SRC_PATH)/libgit2-ssh-loop.patch-applied
cd $(LIBGIT2_SRC_PATH) && \
patch -p1 -f < $(SRCDIR)/patches/libgit2-bundled_zlib.patch
echo 1 > $@

$(build_datarootdir)/julia/cert.pem:
$(JLDOWNLOAD) $(shell pwd)/cacert-2018-01-17.pem https://curl.haxx.se/ca/cacert-2018-01-17.pem
$(JLCHECKSUM) $(shell pwd)/cacert-2018-01-17.pem
Expand All @@ -84,12 +60,7 @@ $(build_datarootdir)/julia/cert.pem:

$(BUILDDIR)/$(LIBGIT2_SRC_DIR)/build-configured: \
$(LIBGIT2_SRC_PATH)/libgit2-mbedtls.patch-applied \
$(LIBGIT2_SRC_PATH)/libgit2-ssh.patch-applied \
$(LIBGIT2_SRC_PATH)/libgit2-agent-nonfatal.patch-applied \
$(LIBGIT2_SRC_PATH)/libgit2-mbedtls-verify.patch-applied \
$(LIBGIT2_SRC_PATH)/libgit2-mbedtls-fixup.patch-applied \
$(LIBGIT2_SRC_PATH)/libgit2-ssh-loop.patch-applied \
$(LIBGIT2_SRC_PATH)/libgit2-bundled_zlib.patch-applied \

$(BUILDDIR)/$(LIBGIT2_SRC_DIR)/build-configured: $(LIBGIT2_SRC_PATH)/source-extracted $(build_datarootdir)/julia/cert.pem
mkdir -p $(dir $@)
Expand Down
4 changes: 2 additions & 2 deletions deps/libgit2.version
View file
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
LIBGIT2_BRANCH=v0.26.0
LIBGIT2_SHA1=15e119375018fba121cf58e02a9f17fe22df0df8
LIBGIT2_BRANCH=v0.27.1
LIBGIT2_SHA1=b0d9952c318a3d1b8917e06ad46b9110c0c28831
3 changes: 3 additions & 0 deletions deps/patches/libgit2-agent-nonfatal.patch
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ Date: Wed Jul 20 19:59:00 2016 -0400

Make failure to connect to ssh-agent non-fatal

Julia issue: https://github.com/JuliaLang/julia/pull/17459
Upstream: https://github.com/libgit2/libgit2/issues/3866

diff --git a/src/transports/ssh.c b/src/transports/ssh.c
index cfd5736..82d2c63 100644
--- a/src/transports/ssh.c
Expand Down
55 changes: 0 additions & 55 deletions deps/patches/libgit2-bundled_zlib.patch
View file

This file was deleted.

70 changes: 0 additions & 70 deletions deps/patches/libgit2-mbedtls-fixup.patch
View file

This file was deleted.

100 changes: 0 additions & 100 deletions deps/patches/libgit2-mbedtls-verify.patch
View file

This file was deleted.

Loading

11 comments on commit 4a303b3

@staticfloat
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uh-oh, I think this broke something; we seem to be requiring libssl.so now;

$ julia -e 'versioninfo()'
fatal: error thrown and no exception handler available.
InitError(mod=:LibGit2, error=ErrorException("could not load library "libgit2"
libssl.so.10: cannot open shared object file: No such file or directory"))
rec_backtrace at /buildworker/worker/package_linux64/build/src/stackwalk.c:94
record_backtrace at /buildworker/worker/package_linux64/build/src/task.c:246
jl_throw at /buildworker/worker/package_linux64/build/src/task.c:577
jl_errorf at /buildworker/worker/package_linux64/build/src/rtutils.c:77
jl_dlerror at /buildworker/worker/package_linux64/build/src/dlload.c:74 [inlined]
jl_load_dynamic_library_ at /buildworker/worker/package_linux64/build/src/dlload.c:196
jl_get_library at /buildworker/worker/package_linux64/build/src/runtime_ccall.cpp:48
jl_get_library at /buildworker/worker/package_linux64/build/src/runtime_ccall.cpp:56 [inlined]
jl_load_and_lookup at /buildworker/worker/package_linux64/build/src/runtime_ccall.cpp:59
jlplt_git_libgit2_init_17526.clone_1 at /home/travis/julia/lib/julia/sys.so (unknown line)
macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v0.7/LibGit2/src/error.jl:96 [inlined]
__init__ at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v0.7/LibGit2/src/LibGit2.jl:969
jfptr___init___17525.clone_1 at /home/travis/julia/lib/julia/sys.so (unknown line)
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2140
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1540 [inlined]
jl_module_run_initializer at /buildworker/worker/package_linux64/build/src/toplevel.c:90
_julia_init at /buildworker/worker/package_linux64/build/src/init.c:810
julia_init__threading at /buildworker/worker/package_linux64/build/src/task.c:302
unknown function (ip: 0x401517)
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x4015c4)

@andreasnoack
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a fix of this in the pipeline. I think it currently breaks all Travis testing of package on master which is really unfortunate at this stage.

@Nosferican
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

Package name determined from repository url to be 
`src/.jl` not found, repository is not a valid Julia package so the default test script is empty

Is that the reason for 389763894?

@simonbyrne
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this something to do with the buildbot? Why don't I see the same locally?

@staticfloat
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You probably don't see the same locally because you have a libssl.so.10 available. Many distros will have it, but stripped-down testing rigs such as Travis may not, as evidenced by the widespread breakage on Travis right now.

We try not to rely on OpenSSL, instead using MbedTLS for crypto routines and whatnot. Something has changed in this upgrade that makes libgit2 link against libssl once more. When building locally, do an ldd libgit2.so and see what it's linking against; if it links against libssl.so, then something is wrong.

Is there a fix of this in the pipeline

Yes, I recently reverted this commit so that we can get some nightlies back. Compilation was successful, and we're working our way through the test suite now. You can follow along from home right here: https://build.julialang.org/#/builders/89/builds/363

@staticfloat
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The nightlies are working again.

@simonbyrne
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried ldd on a local build with libgit2 0.27.1, but don't see any dependence on libssl.so.10. Any suggestions?

@staticfloat
Copy link
Member

@staticfloat staticfloat commented on 4a303b3 Jun 10, 2018 via email

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@simonbyrne
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes.

@staticfloat
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When you do the cmake step within libgit2, what does your Enabled features look like? E.g. what it should look like is:

-- Enabled features:
 * threadsafe, threadsafe support
 * cURL, cURL for HTTP proxy support
 * HTTPS, using OpenSSL
 * SHA, using mbedTLS
 * http-parser, http-parser support
 * zlib, using bundled zlib
 * SSH, SSH transport support

-- Disabled features:
 * debugpool, debug pool allocator
 * tracing, tracing support
 * SPNEGO, SPNEGO authentication support
 * iconv, iconv encoding conversion support

I'm working on a patch to mbedTLS to unbreak something, and then I'll carry the patch here and submit a PR.

@staticfloat
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR here: #27525

Please sign in to comment.