0.7 hangs on module load

[stevengj]

In the course of updating PyCall to Julia 0.7, I encountered a mysterious case where Julia master hangs. For the file Bug.jl from https://gist.github.com/stevengj/a07601824ad607829455651b45bd0a97, I get

julia> include("Bug.jl")
generating < from 0
generating <= from 1
generating == from 2
generating != from 3
generating > from 4

at which point it hangs (0.7.0-DEV.4489 on MacOS).

I've boiled down the gist as much as I can. It seems to be a bit of a Heisenbug, because deleting "random" lines eliminates the hang.

Can others reproduce?

[vchuravy]

I can reproduce on 0.7.0-DEV.4473 and on 0.7.0-DEV.4489 on Linux.

[vchuravy]

Attaching gdb I get a ever growing stack (1710 frames) that includes many of these,

#5  jl_apply_generic (args=0x7fffd1886f40, nargs=<optimized out>) at /home/vchuravy/src/julia/src/gf.c:2095
#6  0x00007f5bc5b4e03a in japi1_abstract_call_method_720 () at compiler/abstractinterpretation.jl:193
#7  0x00007f5bd291c699 in jl_call_fptr_internal (fptr=<optimized out>, fptr=<optimized out>, nargs=<optimized out>, args=<optimized out>, meth=<optimized out>) at /home/vchuravy/src/julia/src/julia_internal.h:383
#8  jl_call_method_internal (nargs=5, args=0x7fffd1887118, meth=<optimized out>) at /home/vchuravy/src/julia/src/julia_internal.h:402
#9  jl_apply_generic (args=0x7fffd1887118, nargs=5) at /home/vchuravy/src/julia/src/gf.c:2098
#10 0x00007f5bc5b4cf32 in japi1_abstract_call_gf_by_type_672 () at compiler/abstractinterpretation.jl:78
#11 0x00007f5bd291c699 in jl_call_fptr_internal (fptr=<optimized out>, fptr=<optimized out>, nargs=<optimized out>, args=<optimized out>, meth=<optimized out>) at /home/vchuravy/src/julia/src/julia_internal.h:383
#12 jl_call_method_internal (nargs=5, args=0x7fffd1887338, meth=<optimized out>) at /home/vchuravy/src/julia/src/julia_internal.h:402
#13 jl_apply_generic (args=0x7fffd1887338, nargs=5) at /home/vchuravy/src/julia/src/gf.c:2098
#14 0x00007f5bc5b4bb36 in japi1_abstract_call_593 () at compiler/abstractinterpretation.jl:663
#15 0x00007f5bd291c699 in jl_call_fptr_internal (fptr=<optimized out>, fptr=<optimized out>, nargs=<optimized out>, args=<optimized out>, meth=<optimized out>) at /home/vchuravy/src/julia/src/julia_internal.h:383
#16 jl_call_method_internal (nargs=6, args=0x7fffd1887560, meth=<optimized out>) at /home/vchuravy/src/julia/src/julia_internal.h:402
#17 jl_apply_generic (args=0x7fffd1887560, nargs=6) at /home/vchuravy/src/julia/src/gf.c:2098
#18 0x00007f5bc5b444e8 in japi1_abstract_eval_call_544 () at compiler/abstractinterpretation.jl:692
#19 0x00007f5bc5b44bd5 in japi1_abstract_eval_528 () at compiler/abstractinterpretation.jl:713
#20 0x00007f5bd291c699 in jl_call_fptr_internal (fptr=<optimized out>, fptr=<optimized out>, nargs=<optimized out>, args=<optimized out>, meth=<optimized out>) at /home/vchuravy/src/julia/src/julia_internal.h:383
#21 jl_call_method_internal (nargs=4, args=0x7fffd1887820, meth=<optimized out>) at /home/vchuravy/src/julia/src/julia_internal.h:402
#22 jl_apply_generic (args=0x7fffd1887820, nargs=4) at /home/vchuravy/src/julia/src/gf.c:2098
#23 0x00007f5bc5b45b92 in japi1_abstract_interpret_523 () at compiler/abstractinterpretation.jl:858
#24 0x00007f5bd291c699 in jl_call_fptr_internal (fptr=<optimized out>, fptr=<optimized out>, nargs=<optimized out>, args=<optimized out>, meth=<optimized out>) at /home/vchuravy/src/julia/src/julia_internal.h:383
#25 jl_call_method_internal (nargs=4, args=0x7fffd1887970, meth=<optimized out>) at /home/vchuravy/src/julia/src/julia_internal.h:402
#26 jl_apply_generic (args=0x7fffd1887970, nargs=4) at /home/vchuravy/src/julia/src/gf.c:2098
#27 0x00007f5bc5baa35a in julia_typeinf_work_54 () at compiler/typeinfer.jl:276
#28 0x00007f5bc5babbdf in japi1_typeinf_51 () at compiler/typeinfer.jl:407
#29 0x00007f5bc5b4fbf7 in japi1_typeinf_edge_741 () at compiler/typeinfer.jl:137
#30 0x00007f5bd291c699 in jl_call_fptr_internal (fptr=<optimized out>, fptr=<optimized out>, nargs=<optimized out>, args=<optimized out>, meth=<optimized out>) at /home/vchuravy/src/julia/src/julia_internal.h:383
#31 jl_call_method_internal (nargs=5, args=0x7fffd1887d20, meth=<optimized out>) at /home/vchuravy/src/julia/src/julia_internal.h:402
#32 jl_apply_generic (args=0x7fffd1887d20, nargs=5) at /home/vchuravy/src/julia/src/gf.c:2098
#33 0x00007f5bc5b4e6db in japi1_abstract_call_method_720 () at compiler/abstractinterpretation.jl:281
#34 0x00007f5bd291c699 in jl_call_fptr_internal (fptr=<optimized out>, fptr=<optimized out>, nargs=<optimized out>, args=<optimized out>, meth=<optimized out>) at /home/vchuravy/src/julia/src/julia_internal.h:383
#35 jl_call_method_internal (nargs=5, args=0x7fffd1887ef8, meth=<optimized out>) at /home/vchuravy/src/julia/src/julia_internal.h:402
#36 jl_apply_generic (args=0x7fffd1887ef8, nargs=5) at /home/vchuravy/src/julia/src/gf.c:2098
#37 0x00007f5bc5b4cf32 in japi1_abstract_call_gf_by_type_672 () at compiler/abstractinterpretation.jl:78
#38 0x00007f5bd291c699 in jl_call_fptr_internal (fptr=<optimized out>, fptr=<optimized out>, nargs=<optimized out>, args=<optimized out>, meth=<optimized out>) at /home/vchuravy/src/julia/src/julia_internal.h:383
#39 jl_call_method_internal (nargs=5, args=0x7fffd1888118, meth=<optimized out>) at /home/vchuravy/src/julia/src/julia_internal.h:402
#40 jl_apply_generic (args=0x7fffd1888118, nargs=5) at /home/vchuravy/src/julia/src/gf.c:2098
#41 0x00007f5bc5b4bb36 in japi1_abstract_call_593 () at compiler/abstractinterpretation.jl:663
#42 0x00007f5bd291c699 in jl_call_fptr_internal (fptr=<optimized out>, fptr=<optimized out>, nargs=<optimized out>, args=<optimized out>, meth=<optimized out>) at /home/vchuravy/src/julia/src/julia_internal.h:383
#43 jl_call_method_internal (nargs=6, args=0x7fffd1888340, meth=<optimized out>) at /home/vchuravy/src/julia/src/julia_internal.h:402
#44 jl_apply_generic (args=0x7fffd1888340, nargs=6) at /home/vchuravy/src/julia/src/gf.c:2098
#45 0x00007f5bc5b444e8 in japi1_abstract_eval_call_544 () at compiler/abstractinterpretation.jl:692
#46 0x00007f5bc5b44bd5 in japi1_abstract_eval_528 () at compiler/abstractinterpretation.jl:713
#47 0x00007f5bc5b45b0e in japi1_abstract_interpret_523 () at compiler/abstractinterpretation.jl:866
#48 0x00007f5bd291c699 in jl_call_fptr_internal (fptr=<optimized out>, fptr=<optimized out>, nargs=<optimized out>, args=<optimized out>, meth=<optimized out>) at /home/vchuravy/src/julia/src/julia_internal.h:383
#49 jl_call_method_internal (nargs=4, args=0x7fffd1888690, meth=<optimized out>) at /home/vchuravy/src/julia/src/julia_internal.h:402
#50 jl_apply_generic (args=0x7fffd1888690, nargs=4) at /home/vchuravy/src/julia/src/gf.c:2098

and if I found the right starting point the whole chain starts in Bug.jl:88 with the macro call off @pycheckv

#1724 0x00007f5bbeca12bd in japi1_pycheckv_33723 () at /home/vchuravy/src/julia/Bug.jl:74
#1725 0x00007f5bd2924d94 in jl_call_fptr_internal (fptr=<optimized out>, fptr=<optimized out>, nargs=<optimized out>, args=<optimized out>, meth=<optimized out>) at /home/vchuravy/src/julia/src/julia_internal.h:383
#1726 jl_call_method_internal (nargs=5, args=0x7fffd18be5d0, meth=0x7f5bc16ff410) at /home/vchuravy/src/julia/src/julia_internal.h:402
#1727 jl_invoke_julia_macro (inmodule=inmodule@entry=0x7f5bc16f8c40, ctx=ctx@entry=0x7fffd18be7e0, args=<optimized out>, args=<optimized out>) at /home/vchuravy/src/julia/src/ast.c:996
#1728 0x00007f5bd2928b15 in jl_expand_macros (expr=<optimized out>, inmodule=0x7f5bc16f8c40, macroctx=0x7fffd18be8b0, onelevel=0) at /home/vchuravy/src/julia/src/ast.c:1058

in particular in line 177:

Expr(:escape, Expr(:call, :ccall, Expr(:macrocall, Symbol("@pysym"), #= Symbol("/home/vchuravy/src/julia/Bug.jl"):177 =#, :(:PyObject_RichCompareBool))::Any, :Cint, Expr(:tuple, :PyPtr, :PyPtr, :Cint)::Any, :o1, :o2, 4)::Any)::Any

[maleadt]

Is this a build with assertions enabled? I've seen similar hangs due to a world-age issue (#26278).

[jmert]

If I wait sufficiently long — on the order of 10-15 minutes — I find that "progress" does eventually get made (in the form of an error being thrown about cglobal calls). I am using LLVM 5.0 instead of 3.9 when building Julia master from source, if that makes a difference.

[vtjnash]

Maybe fixed by #27898?