Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

timer_callback is causing segfaults with curl 7.81.0 #172

Closed
mkitti opened this issue Jan 8, 2022 · 17 comments
Closed

timer_callback is causing segfaults with curl 7.81.0 #172

mkitti opened this issue Jan 8, 2022 · 17 comments

Comments

@mkitti
Copy link

mkitti commented Jan 8, 2022

Several unsupported builds of Julia 1.7.1 are reporting segmentation faults when trying to build with curl 7.81.0:

  1. pacman julia crashes with segmentation fault on ]up julia#43682
  2. cleanup v1.6 LTS conda-forge/julia-feedstock#159 (comment)

The segmentation fault can be replicated as follows:

$ julia -E 'download("https://curl.se")'

signal (11): Segmentation fault
in expression starting at none:1
prune_dead_connections at /home/mkitti/src/curl/lib/url.c:1092
create_conn at /home/mkitti/src/curl/lib/url.c:3869
Curl_connect at /home/mkitti/src/curl/lib/url.c:4129
multi_runsingle at /home/mkitti/src/curl/lib/multi.c:1860
multi_socket at /home/mkitti/src/curl/lib/multi.c:3131
curl_multi_socket_action at /home/mkitti/src/curl/lib/multi.c:3252
curl_multi_socket_action at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/LibCURL/src/lC_curl_h.jl:230 [inlined]
curl_multi_socket_action at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Curl/utils.jl:91 [inlined]
macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Curl/utils.jl:35 [inlined]
#47 at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Curl/Multi.jl:147
lock at ./lock.jl:190
timer_callback at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Curl/Multi.jl:146
jfptr_timer_callback_19000.clone_1 at /home/mkitti/anaconda3/envs/curltest/lib/julia/sys.so (unknown line)
_jl_invoke at /usr/local/src/conda/julia-1.7.1/src/gf.c:2247 [inlined]
jl_apply_generic at /usr/local/src/conda/julia-1.7.1/src/gf.c:2429
jlcapi_timer_callback_18861.clone_1 at /home/mkitti/anaconda3/envs/curltest/lib/julia/sys.so (unknown line)
Curl_update_timer at /home/mkitti/src/curl/lib/multi.c:3365
curl_multi_add_handle at /home/mkitti/src/curl/lib/multi.c:504
curl_multi_add_handle at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/LibCURL/src/lC_curl_h.jl:194 [inlined]
macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Curl/utils.jl:35 [inlined]
#27 at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Curl/Multi.jl:51
lock at ./lock.jl:190
add_handle at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Curl/Multi.jl:44 [inlined]
#9 at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Downloads.jl:345
with_handle at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Curl/Curl.jl:64
#8 at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Downloads.jl:311 [inlined]
arg_write at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/ArgTools/src/ArgTools.jl:112
#7 at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Downloads.jl:310 [inlined]
arg_read at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/ArgTools/src/ArgTools.jl:61 [inlined]
#request#5 at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Downloads.jl:309
request##kw at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Downloads.jl:293 [inlined]
#3 at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Downloads.jl:222 [inlined]
arg_write at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/ArgTools/src/ArgTools.jl:101
#download#2 at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Downloads.jl:221 [inlined]
download at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Downloads.jl:221
_jl_invoke at /usr/local/src/conda/julia-1.7.1/src/gf.c:2247 [inlined]
jl_apply_generic at /usr/local/src/conda/julia-1.7.1/src/gf.c:2429
jl_apply at /usr/local/src/conda/julia-1.7.1/src/julia.h:1788 [inlined]
jl_f__call_latest at /usr/local/src/conda/julia-1.7.1/src/builtins.c:757
#invokelatest#2 at ./essentials.jl:716 [inlined]
invokelatest at ./essentials.jl:714 [inlined]
do_download at ./download.jl:24 [inlined]
download at ./download.jl:20
_jl_invoke at /usr/local/src/conda/julia-1.7.1/src/gf.c:2247 [inlined]
jl_apply_generic at /usr/local/src/conda/julia-1.7.1/src/gf.c:2429
jl_apply at /usr/local/src/conda/julia-1.7.1/src/julia.h:1788 [inlined]
do_call at /usr/local/src/conda/julia-1.7.1/src/interpreter.c:126
eval_value at /usr/local/src/conda/julia-1.7.1/src/interpreter.c:215
eval_stmt_value at /usr/local/src/conda/julia-1.7.1/src/interpreter.c:166 [inlined]
eval_body at /usr/local/src/conda/julia-1.7.1/src/interpreter.c:601
jl_interpret_toplevel_thunk at /usr/local/src/conda/julia-1.7.1/src/interpreter.c:731
jl_toplevel_eval_flex at /usr/local/src/conda/julia-1.7.1/src/toplevel.c:885
jl_toplevel_eval_flex at /usr/local/src/conda/julia-1.7.1/src/toplevel.c:830
jl_toplevel_eval_in at /usr/local/src/conda/julia-1.7.1/src/toplevel.c:944
eval at ./boot.jl:373 [inlined]
exec_options at ./client.jl:270
_start at ./client.jl:495
jfptr__start_40346.clone_1 at /home/mkitti/anaconda3/envs/curltest/lib/julia/sys.so (unknown line)
_jl_invoke at /usr/local/src/conda/julia-1.7.1/src/gf.c:2247 [inlined]
jl_apply_generic at /usr/local/src/conda/julia-1.7.1/src/gf.c:2429
jl_apply at /usr/local/src/conda/julia-1.7.1/src/julia.h:1788 [inlined]
true_main at /usr/local/src/conda/julia-1.7.1/src/jlapi.c:559
jl_repl_entrypoint at /usr/local/src/conda/julia-1.7.1/src/jlapi.c:701
main at julia (unknown line)
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x5626d2001098)
Allocations: 2720 (Pool: 2711; Big: 9); GC: 0
Segmentation fault (core dumped)

In this case curl is configured according to
https://github.com/conda-forge/curl-feedstock/blob/master/recipe/build.sh#L8-L17
and is built with openssl:

$ julia -E "using LibCURL; unsafe_string(LibCURL.curl_version())"
"libcurl/7.81.0-DEV OpenSSL/3.0.0 zlib/1.2.11 libssh2/1.10.0 nghttp2/1.43.0"

I bisected the issue to the following commit and line:
curl/curl@2b3dd01#diff-f25cb725c8f9fcc9a6532bed11a8f9c22b60c17edd2d4a3615bd7ceaa6ad8c18R489
@mkitti
Copy link
Author

mkitti commented Jan 8, 2022

I believe this may have been resolved by #164

@giordano
Copy link
Contributor

giordano commented Jan 8, 2022

I believe this may have been resolved by #164

That's already in Julia master, did you try a nightly build?

@ngam
Copy link

ngam commented Jan 8, 2022

And this hasn't been shipped with the 1.7.1 version? What version of Downloads.jl are we supposed to be using?

@ngam
Copy link

ngam commented Jan 8, 2022 •
edited
Loading

@mkitti Let's make a patch for this and call it a day then? I've been learning towards disabling system libgit2 and mbedtls (to make our linux/osx approach unified)

@mkitti
Copy link
Author

mkitti commented Jan 8, 2022

Julia 1.7.1 uses Downloads.jl 1.5.2:
https://github.com/JuliaLang/julia/blob/v1.7.1/stdlib/Downloads.version

@ngam
Copy link

ngam commented Jan 8, 2022

I believe this may have been resolved by #164

That's already in Julia master, did you try a nightly build?

@mkitti, let's make a dev branch for these types of situations on our end, thoughts? So we can keep testing.

@mkitti
Copy link
Author

mkitti commented Jan 8, 2022

If you want to play Dr. Frankenstein, we could try changing Downloads.version to

DOWNLOADS_BRANCH = release-1.6
DOWNLOADS_SHA1 = 43899e71eed709866f96dd624f1c369a1c8e26a3
DOWNLOADS_GIT_URL := git://github.com/JuliaLang/Downloads.jl.git
DOWNLOADS_TAR_URL = https://api.github.com/repos/JuliaLang/Downloads.jl/tarball/$1

@ngam
Copy link

ngam commented Jan 8, 2022

If you want to play Dr. Frankenstein, we could try changing Downloads.version to

DOWNLOADS_BRANCH = release-1.6
DOWNLOADS_SHA1 = 43899e71eed709866f96dd624f1c369a1c8e26a3
DOWNLOADS_GIT_URL := git://github.com/JuliaLang/Downloads.jl.git
DOWNLOADS_TAR_URL = https://api.github.com/repos/JuliaLang/Downloads.jl/tarball/$1

LOL NO!!! I don't wanna play anything. I just want this to work 😢

@ngam
Copy link

ngam commented Jan 8, 2022 •
edited
Loading

the diff: v1.5.2...v1.6.0

@mkitti
Copy link
Author

mkitti commented Jan 8, 2022

Well that 4 line patch will likely make it "work".

@mkitti
Copy link
Author

mkitti commented Jan 8, 2022

I believe this may have been resolved by #164

That's already in Julia master, did you try a nightly build?

I built master, and it appears to work with conda-forge's build of curl 7.81.0. I also just did a build of 1.7.1 with Downloads.jl 1.6, and that also appears to work with conda-forge's build of curl 7.81.0.

On the JuliaLang side, I believe this is resolved. However, it may be useful for someone to think about other consequences of curl/curl@2b3dd01 so I'll keep this open until sometime next week.

Feel free to close this though.

@mkitti mkitti mentioned this issue Jan 8, 2022
Merged
@ngam
Copy link

ngam commented Jan 8, 2022

I believe this may have been resolved by #164

That's already in Julia master, did you try a nightly build?

I built master, and it appears to work with conda-forge's build of curl 7.81.0. I also just did a build of 1.7.1 with Downloads.jl 1.6, and that also appears to work with conda-forge's build of curl 7.81.0.

On the JuliaLang side, I believe this is resolved. However, it may be useful for someone to think about other consequences of curl/curl@2b3dd01 so I'll keep this open until sometime next week.

Feel free to close this though.

Perf!

Let's just stitch downloads 1.6 in there for now

@giordano
Copy link
Contributor

giordano commented Jan 8, 2022

Note that you don't need to rebuild Julia, you can follow https://github.com/JuliaLang/Pkg.jl/blob/7af3e27e3784ccde287b4bcb87891a22830a7dee/README.md#using-the-development-version-of-pkgjl to use a different version of Downloads.jl than the one shipped by Julia

@ericphanson
Copy link
Contributor

ericphanson commented Jan 9, 2022 •
edited
Loading

My opinion is that these bugfixes should be backported to Julia 1.7.2 (some discussion here: JuliaLang/julia#43430)

@carlocab carlocab mentioned this issue Jan 9, 2022
Closed
6 tasks
carlocab added a commit to carlocab/homebrew-core that referenced this issue Jan 9, 2022
@carlocab
julia: fix segfaults with Curl 7.81
94a8349 
The version bump to curl (Homebrew#92549) is causing segfaults when installing
Julia packages. See Homebrew/discussions#2749. Upstream is already aware
of the issue (JuliaLang/Downloads.jl#172).

I've updated the test to make sure we catch this regression in the
future.
BrewTestBot pushed a commit to Homebrew/homebrew-core that referenced this issue Jan 10, 2022
@carlocab @BrewTestBot
julia: fix segfaults with Curl 7.81
37e6134 
The version bump to curl (#92549) is causing segfaults when installing
Julia packages. See Homebrew/discussions#2749. Upstream is already aware
of the issue (JuliaLang/Downloads.jl#172).

I've updated the test to make sure we catch this regression in the
future.

Closes #92799.

Signed-off-by: BrewTestBot <1589480+BrewTestBot@users.noreply.github.com>
@ericonr ericonr mentioned this issue Jan 22, 2022
Closed
freebsd-git pushed a commit to freebsd/freebsd-ports that referenced this issue Jan 25, 2022
@E-Ryze @fernape
lang/julia: unbreak build when using libcurl 7.81
175c1f5 
See:

 * JuliaLang/Downloads.jl#172
 * conda-forge/julia-feedstock#169 (comment)

PR:	261417
Reported by:	rizor.reardon@yahoo.com
5u623l20 pushed a commit to 5u623l20/freebsd-ports that referenced this issue Jan 27, 2022
@E-Ryze
lang/julia: unbreak build when using libcurl 7.81
90d2595 
See:

 * JuliaLang/Downloads.jl#172
 * conda-forge/julia-feedstock#169 (comment)

PR:	261417
Reported by:	rizor.reardon@yahoo.com
@giordano
Copy link
Contributor

so I'll keep this open until sometime next week.

Can this issue be closed now? A few weeks passed since then

@mkitti
Copy link
Author

mkitti commented Mar 19, 2022

We can close this. Any thoughts on @ericphanson 's backport request?

@giordano
Copy link
Contributor

giordano commented Mar 19, 2022 •
edited
Loading

Any thoughts on @ericphanson 's backport request?

Already done 2 weeks ago: JuliaLang/julia#44433

@cho-m cho-m mentioned this issue May 29, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@giordano @ericphanson @mkitti @ngam