"LLVM ERROR: Broken module found, compilation aborted!" when compiling a sysimage with a ccallable function and a certain cpu target #34064

KristofferC · 2019-12-09T20:25:43Z

When trying to change the cpu target used when generating "apps" in PackageCompiler (JuliaLang/PackageCompilerX.jl#43) with the "app" at (https://github.com/KristofferC/PackageCompilerX.jl/blob/master/examples/MyApp/src/MyApp.jl) I hit the following CI error

Linux (https://travis-ci.com/KristofferC/PackageCompilerX.jl/jobs/265321335#L264-L267):

<unknown>: error: Cannot represent a difference across sections
Aliasee should be either GlobalValue or ConstantExpr
i32 ()* @julia_main
LLVM ERROR: Broken module found, compilation aborted!

Mac (https://travis-ci.com/KristofferC/PackageCompilerX.jl/jobs/265321336#L197-L212);

signal (11): Segmentation fault: 11
in expression starting at none:0
_ZN12_GLOBAL__N_119X86MachObjectWriter16recordRelocationEPN4llvm16MachObjectWriterERNS1_11MCAssemblerERKNS1_11MCAsmLayoutEPKNS1_10MCFragmentERKNS1_7MCFixupENS1_7MCValueERy at /Users/travis/julia/lib/julia/libLLVM.dylib (unknown line)
_ZN4llvm16MachObjectWriter16recordRelocationERNS_11MCAssemblerERKNS_11MCAsmLayoutEPKNS_10MCFragmentERKNS_7MCFixupENS_7MCValueERy at /Users/travis/julia/lib/julia/libLLVM.dylib (unknown line)
_ZN4llvm11MCAssembler6layoutERNS_11MCAsmLayoutE at /Users/travis/julia/lib/julia/libLLVM.dylib (unknown line)
_ZN4llvm11MCAssembler6FinishEv at /Users/travis/julia/lib/julia/libLLVM.dylib (unknown line)
_ZN12_GLOBAL__N_115MCMachOStreamer10FinishImplEv at /Users/travis/julia/lib/julia/libLLVM.dylib (unknown line)
_ZN4llvm10MCStreamer6FinishEv at /Users/travis/julia/lib/julia/libLLVM.dylib (unknown line)
_ZN4llvm10AsmPrinter14doFinalizationERNS_6ModuleE at /Users/travis/julia/lib/julia/libLLVM.dylib (unknown line)
_ZN4llvm13FPPassManager14doFinalizationERNS_6ModuleE at /Users/travis/julia/lib/julia/libLLVM.dylib (unknown line)
_ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE at /Users/travis/julia/lib/julia/libLLVM.dylib (unknown line)
operator() at /Users/sabae/buildbot/worker/package_macos64/build/src/jitlayers.cpp:1086 [inlined]
jl_dump_native at /Users/sabae/buildbot/worker/package_macos64/build/src/jitlayers.cpp:1095
jl_write_compiler_output at /Users/sabae/buildbot/worker/package_macos64/build/src/precompile.c:93
jl_atexit_hook at /Users/sabae/buildbot/worker/package_macos64/build/src/init.c:230
main at /Users/travis/julia/bin/julia (unknown line)

It can be reprod on the linked branch above with pwd in the root git repo and running:

using PackageCompilerX
 create_sysimage(:MyApp, sysimage_path="MyApp.so", cpu_target="generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1)", project="examples/MyApp", incremental=false)

Note that incremental=false seems to be needed (which means that a new sysimage is bootstrapped from scratch (corecompiler.ji -> sys.ji -> sys.o).

I can reduce it to just a set of raw commands if that would be helpful.

The text was updated successfully, but these errors were encountered:

KristofferC · 2019-12-09T21:08:37Z

On the backport 1.3.1 branch I got

Assertion failed: (isa<X>(Val) && "cast<Ty>() argument of incompatible type!"), function cast, file /Users/kristoffer/julia/usr/include/llvm/Support/Casting.h, line 255.

KristofferC · 2019-12-09T22:15:24Z

Here is an MWE without PackageCompilerX. It seems that the combination for this to occur is:

use sys.ji as sysimage
use the cpu target we use for the binaries
compile a Base.@ccallable function

function get_julia_cmd()
    julia_path = joinpath(Sys.BINDIR, Base.julia_exename())
    cmd = `$julia_path --color=yes --startup-file=no`
end

sysimg_source_path = Base.find_source_file("sysimg.jl")
base_dir = dirname(sysimg_source_path)
tmp_corecompiler_ji = joinpath(@__DIR__, "corecompiler.ji")
tmp_sys_ji = joinpath(@__DIR__, "sys.ji")
compiler_source_path = joinpath(base_dir, "compiler", "compiler.jl")
cd(base_dir) do
    # Create corecompiler.ji
    cmd = `$(get_julia_cmd()) --output-ji $tmp_corecompiler_ji
                              -g0 -O0 $compiler_source_path`
    run(cmd)

    # Use that to create sys.ji
    cmd = `$(get_julia_cmd()) --sysimage=$tmp_corecompiler_ji
                              -g1 -O0 --output-ji=$tmp_sys_ji $sysimg_source_path`
    run(cmd)
end

app_content = """
Base.@ccallable function julia_main()::Cint
    return 0
end
"""

# Create the sysimage
cmd = `$(get_julia_cmd()) --sysimage=$tmp_sys_ji --cpu-target='generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1)'
                          --output-o=sys.o -e $app_content`
run(cmd)

KristofferC · 2019-12-10T09:42:20Z

A workaround is to first create an empty dylib sysimage from sys.ji and then incrementally add the @ccallable function to that sysimage. Works but takes a bit longer time than neeed.

KristofferC · 2020-05-06T16:21:04Z

Still failing, now with:

julia: /home/kc/julia/usr/include/llvm/Support/Casting.h:264: typename llvm::cast_retty<X, Y*>::ret_type llvm::cast(Y*) [with X = llvm::GlobalVariable; Y = llvm::GlobalValue; typename llvm::cast_retty<X, Y*>::ret_type = llvm::GlobalVariable*]: Assertion `isa<X>(Val) && "cast<Ty>() argument of incompatible type!"' failed.

signal (6): Aborted
in expression starting at none:0
gsignal at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
abort at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x7fb078b17728)
__assert_fail at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
cast<llvm::GlobalVariable, llvm::GlobalValue> at /home/kc/julia/usr/include/llvm/Support/Casting.h:264 [inlined]
operator() at /home/kc/julia/src/llvm-multiversioning.cpp:711 [inlined]
fix_gv_uses at /home/kc/julia/src/llvm-multiversioning.cpp:726 [inlined]
runOnModule at /home/kc/julia/src/llvm-multiversioning.cpp:1053
_ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE at /home/kc/julia/usr/bin/../lib/libLLVM-9jl.so (unknown line)
operator() at /home/kc/julia/src/aotcompile.cpp:534 [inlined]
jl_dump_native at /home/kc/julia/src/aotcompile.cpp:545
jl_write_compiler_output at /home/kc/julia/src/precompile.c:90
jl_atexit_hook at /home/kc/julia/src/init.c:218
main at /home/kc/julia/ui/repl.c:228
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
_start at /home/kc/julia/usr/bin/julia (unknown line)
Allocations: 15518740 (Pool: 15510414; Big: 8326); GC: 3

vtjnash · 2020-05-06T16:32:32Z

Possibly we might need https://github.com/JuliaLang/julia/tree/jn/localize-multiversioning for that (though it doesn't currently include the fix)

vchuravy · 2020-09-11T13:16:19Z

Seeing this in #37510 in various forms:

    JULIA /home/vchuravy/builds/julia/usr/lib/julia/sys-o.a
<unknown>:0: error: Cannot represent a difference across sections
<unknown>:0: error: Cannot represent a difference across sections
<unknown>:0: error: Cannot represent a difference across sections
<unknown>:0: error: Cannot represent a difference across sections
<unknown>:0: error: Cannot represent a difference across sections
<unknown>:0: error: Cannot represent a difference across sections
Aliasee should be either GlobalValue or ConstantExpr
i16 (float)* @__truncsfhf2
Aliasee should be either GlobalValue or ConstantExpr
float (i16)* @__gnu_h2f_ieee
Aliasee should be either GlobalValue or ConstantExpr
i16 (float)* @__gnu_f2h_ieee
Aliasee should be either GlobalValue or ConstantExpr
float (i16)* @__extendhfdf2
Aliasee should be either GlobalValue or ConstantExpr
i16 (double)* @__truncdfhf2
Aliasee should be either GlobalValue or ConstantExpr
float (i16)* @__extendhfsf2
LLVM ERROR: Broken module found, compilation aborted!

and

4:44
julia: /buildworker/worker/package_linux32/build/usr/include/llvm/Support/Casting.h:264: typename llvm::cast_retty<X, Y*>::ret_type llvm::cast(Y*) [with X = llvm::GlobalVariable; Y = llvm::GlobalValue; typename llvm::cast_retty<X, Y*>::ret_type = llvm::GlobalVariable*]: Assertion `isa<X>(Val) && "cast<Ty>() argument of incompatible type!"' failed.
signal (6): Aborted
in expression starting at none:0
gsignal at /lib/i386-linux-gnu/libc.so.6 (unknown line)
abort at /lib/i386-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0xf7a830f5)
__assert_fail at /lib/i386-linux-gnu/libc.so.6 (unknown line)
cast<llvm::Function, llvm::Value> at /buildworker/worker/package_linux32/build/usr/include/llvm/Support/Casting.h:264 [inlined]
emit_metadata at /buildworker/worker/package_linux32/build/src/llvm-multiversioning.cpp:999 [inlined]
runOnModule at /buildworker/worker/package_linux32/build/src/llvm-multiversioning.cpp:1068
_ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE at /buildworker/worker/package_linux32/build/usr/bin/../lib/libLLVM-9jl.so (unknown line)
_ZN4llvm6legacy11PassManager3runERNS_6ModuleE at /buildworker/worker/package_linux32/build/usr/bin/../lib/libLLVM-9jl.so (unknown line)
operator() at /buildworker/worker/package_linux32/build/src/aotcompile.cpp:550 [inlined]
jl_dump_native at /buildworker/worker/package_linux32/build/src/aotcompile.cpp:561
jl_write_compiler_output at /buildworker/worker/package_linux32/build/src/precompile.c:94
jl_atexit_hook at /buildworker/worker/package_linux32/build/src/init.c:210
main at /buildworker/worker/package_linux32/build/ui/repl.c:228
__libc_start_main at /lib/i386-linux-gnu/libc.so.6 (unknown line)
_start at /buildworker/worker/package_linux32/build/usr/bin/julia (unknown line)
Allocations: 116324446 (Pool: 116309614; Big: 14832); GC: 222
Aborted (core dumped)

KristofferC · 2020-09-11T13:38:06Z

Looks like the same thing since that PR also uses @ccallable.

Fixes #34064

KristofferC mentioned this issue May 5, 2020

Creating a sysimage with a ccallable function errors #35740

Closed

maleadt added a commit that referenced this issue Sep 11, 2020

Don't rewrite global aliases during multiversioning.

c2db424

Fixes #34064

maleadt mentioned this issue Sep 11, 2020

Multiversioning: support for aliases (from @ccallable) #37530

Merged

simonbyrne mentioned this issue Sep 17, 2020

Support multi-versioning simonbyrne/libcg#5

Closed

maleadt closed this as completed in #37530 Nov 25, 2021

maleadt mentioned this issue Feb 14, 2022

[PackageCompiler] Segmentation fault with CUDA.jl in multiversioning JuliaGPU/CUDA.jl#1365

Closed

AnHeuermann mentioned this issue Apr 7, 2022

Segmentation fault with pre-compiled library importing CUDA JuliaGPU/CUDA.jl#1465

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

"LLVM ERROR: Broken module found, compilation aborted!" when compiling a sysimage with a ccallable function and a certain cpu target #34064

"LLVM ERROR: Broken module found, compilation aborted!" when compiling a sysimage with a ccallable function and a certain cpu target #34064

KristofferC commented Dec 9, 2019 •

edited

Loading

KristofferC commented Dec 9, 2019

KristofferC commented Dec 9, 2019 •

edited

Loading

KristofferC commented Dec 10, 2019

KristofferC commented May 6, 2020

vtjnash commented May 6, 2020

vchuravy commented Sep 11, 2020

KristofferC commented Sep 11, 2020

"LLVM ERROR: Broken module found, compilation aborted!" when compiling a sysimage with a ccallable function and a certain cpu target #34064

"LLVM ERROR: Broken module found, compilation aborted!" when compiling a sysimage with a ccallable function and a certain cpu target #34064

Comments

KristofferC commented Dec 9, 2019 • edited Loading

KristofferC commented Dec 9, 2019

KristofferC commented Dec 9, 2019 • edited Loading

KristofferC commented Dec 10, 2019

KristofferC commented May 6, 2020

vtjnash commented May 6, 2020

vchuravy commented Sep 11, 2020

KristofferC commented Sep 11, 2020

KristofferC commented Dec 9, 2019 •

edited

Loading

KristofferC commented Dec 9, 2019 •

edited

Loading