Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[PackageCompiler] Segmentation fault with CUDA.jl in multiversioning #1365

Closed
vchuravy opened this issue Feb 10, 2022 · 4 comments
Closed

[PackageCompiler] Segmentation fault with CUDA.jl in multiversioning #1365

vchuravy opened this issue Feb 10, 2022 · 4 comments

Comments

@vchuravy
Copy link
Member

Hi, I am getting segfault when I try to include CUDA.jl as dependency in app build PackageCompiler.
Steps to reproduce

]generate MyApp
;cd MyApp
]activate .
]add CUDA
# Add using CUDA to module
using PackageCompiler
create_app(".", "MyApp")

Error is

julia> create_app(".", "MyAppBinary",force=true)
PackageCompiler: bundled artifacts:
  ├── LLVMExtra_jll - 5.370 MiB
  └── OpenSpecFun_jll - 577.252 KiB
  Total artifact file size: 5.934 MiB
✔ [02m:10s] PackageCompiler: compiling base system image (incremental=false)
Precompiling project...
  34 dependencies successfully precompiled in 173 seconds
⠇ [00m:26s] PackageCompiler: compiling incremental system imagejulia: /buildworker/worker/package_linux64/build/usr/include/llvm/Support/Casting.h:269: typename llvm::cast_retty<X, Y*>::ret_type llvm::cast(Y*) [with X = llvm::GlobalVariable; Y = llvm::GlobalValue; typename llvm::cast_retty<X, Y*>::ret_type = llvm::GlobalVariable*]: Assertion `isa<X>(Val) && "cast<Ty>() argument of incompatible type!"' failed.

signal (6): Aborted
in expression starting at none:0
gsignal at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
abort at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x7efe97f9a41e)
__assert_fail at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
cast<llvm::GlobalVariable, llvm::GlobalValue> at /buildworker/worker/package_linux64/build/usr/include/llvm/Support/Casting.h:269 [inlined]
operator() at /buildworker/worker/package_linux64/build/src/llvm-multiversioning.cpp:711
fix_gv_uses at /buildworker/worker/package_linux64/build/src/llvm-multiversioning.cpp:726 [inlined]
runOnModule at /buildworker/worker/package_linux64/build/src/llvm-multiversioning.cpp:1053
_ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE at /opt/julia-1.7.2/bin/../lib/julia/libLLVM-12jl.so (unknown line)
operator() at /buildworker/worker/package_linux64/build/src/aotcompile.cpp:556 [inlined]
jl_dump_native at /buildworker/worker/package_linux64/build/src/aotcompile.cpp:567
jl_write_compiler_output at /buildworker/worker/package_linux64/build/src/precompile.c:94
⠋ [00m:26s] PackageCompiler: compiling incremental system imagejl_atexit_hook at /buildworker/worker/package_linux64/build/src/init.c:211
jl_repl_entrypoint at /buildworker/worker/package_linux64/build/src/jlapi.c:702
main at /buildworker/worker/package_linux64/build/cli/loader_exe.c:42
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
_start at /opt/julia-1.7.2/bin/julia (unknown line)
Allocations: 35210849 (Pool: 35195120; Big: 15729); GC: 27
⠙ [00m:27s] PackageCompiler: compiling incremental system image
ERROR: failed process: Process(`/opt/julia-1.7.2/bin/julia --color=yes --startup-file=no '--cpu-target=generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1)' -O3 --sysimage=/tmp/jl_gATXk1/sys.ji --project=/media/mateusz/Legion2021/MyApp --output-o=/tmp/jl_4r9RmX.o /tmp/jl_zYN2Xp`, ProcessSignaled(6)) [0]

Originally posted by @mashu in #1314 (comment)
@vchuravy
Copy link
Member Author

@mashu this looks like a distinct issue.

Just to confirm this is Julia 1.7.2 (downloaded from the website?) on Linux? Can you post the Manifest you ended up with?

@mashu
Copy link

mashu commented Feb 10, 2022

Yes, this is Julia 1.7.2 downloaded from the official website, but also 1.6.5 and 1.7.1 are affected as far as I recall. I was hoping 1.7.2 would fix this.
More complete steps are below:

(base) mateusz@legion:~$ julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.7.2 (2022-02-06)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

(@v1.7) pkg> generate MyApp
  Generating  project MyApp:
    MyApp/Project.toml
    MyApp/src/MyApp.jl

shell> cd MyApp
/home/mateusz/MyApp

(@v1.7) pkg> activate .
  Activating project at `~/MyApp`

(MyApp) pkg> add CUDA
    Updating registry at `~/.julia/registries/General.toml`
   Resolving package versions...
    Updating `~/MyApp/Project.toml`
  [052768ef] + CUDA v3.8.0
    Updating `~/MyApp/Manifest.toml`
  [621f4979] + AbstractFFTs v1.1.0
  [79e6a3ab] + Adapt v3.3.3
  [ab4f0b2a] + BFloat16s v0.2.0
  [fa961155] + CEnum v0.4.1
  [052768ef] + CUDA v3.8.0
  [d360d2e6] + ChainRulesCore v1.12.0
  [9e997f8a] + ChangesOfVariables v0.1.2
  [34da2185] + Compat v3.41.0
  [ffbed154] + DocStringExtensions v0.8.6
  [e2ba6199] + ExprTools v0.1.8
  [0c68f7d7] + GPUArrays v8.2.1
  [61eb1bfa] + GPUCompiler v0.13.11
  [3587e190] + InverseFunctions v0.1.2
  [92d709cd] + IrrationalConstants v0.1.1
  [692b3bcd] + JLLWrappers v1.4.1
  [929cbde3] + LLVM v4.7.1
  [2ab3a3ac] + LogExpFunctions v0.3.6
  [21216c6a] + Preferences v1.2.3
  [74087812] + Random123 v1.4.2
  [e6cf234a] + RandomNumbers v1.5.3
  [189a3867] + Reexport v1.2.2
  [ae029012] + Requires v1.3.0
  [276daf66] + SpecialFunctions v2.1.2
  [a759f4b9] + TimerOutputs v0.5.15
  [dad2f222] + LLVMExtra_jll v0.0.13+1
  [efe28fd5] + OpenSpecFun_jll v0.5.5+0
  [0dad84c5] + ArgTools
  [56f22d72] + Artifacts
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [8bb1440f] + DelimitedFiles
  [8ba89e20] + Distributed
  [f43a241f] + Downloads
  [b77e0a4c] + InteractiveUtils
  [4af54fe1] + LazyArtifacts
  [b27032c2] + LibCURL
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [a63ad114] + Mmap
  [ca575930] + NetworkOptions
  [44cfe95a] + Pkg
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA
  [9e88b42a] + Serialization
  [1a1011a3] + SharedArrays
  [6462fe0b] + Sockets
  [2f01184e] + SparseArrays
  [10745b16] + Statistics
  [fa267f1f] + TOML
  [a4e569a6] + Tar
  [8dfed614] + Test
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
  [e66e0078] + CompilerSupportLibraries_jll
  [deac9b47] + LibCURL_jll
  [29816b5a] + LibSSH2_jll
  [c8ffd9c3] + MbedTLS_jll
  [14a3606d] + MozillaCACerts_jll
  [4536629a] + OpenBLAS_jll
  [05823500] + OpenLibm_jll
  [83775a58] + Zlib_jll
  [8e850b90] + libblastrampoline_jll
  [8e850ede] + nghttp2_jll
  [3f19e933] + p7zip_jll
Precompiling project...
  1 dependency successfully precompiled in 1 seconds (33 already precompiled)

julia> using PackageCompiler
 │ Package PackageCompiler not found, but a package named PackageCompiler is available from a registry. 
 │ Install package?
 │   (MyApp) pkg> add PackageCompiler 
 └ (y/n) [y]: y
   Resolving package versions...
    Updating `~/MyApp/Project.toml`
  [9b87118b] + PackageCompiler v2.0.4
    Updating `~/MyApp/Manifest.toml`
  [9b87118b] + PackageCompiler v2.0.4
  [05181044] + RelocatableFolders v0.1.3
  [6c6a2e73] + Scratch v1.1.0

julia> using PackageCompiler

(MyApp) pkg> st
     Project MyApp v0.1.0
      Status `~/MyApp/Project.toml`
  [052768ef] CUDA v3.8.0
  [9b87118b] PackageCompiler v2.0.4

julia> create_app(".", "MyApp")
PackageCompiler: bundled artifacts:
  ├── LLVMExtra_jll - 5.370 MiB
  └── OpenSpecFun_jll - 577.252 KiB
  Total artifact file size: 5.934 MiB
✔ [02m:23s] PackageCompiler: compiling base system image (incremental=false)
Precompiling project...
  37 dependencies successfully precompiled in 177 seconds
⠇ [00m:26s] PackageCompiler: compiling incremental system imagejulia: /buildworker/worker/package_linux64/build/usr/include/llvm/Support/Casting.h:269: typename llvm::cast_retty<X, Y*>::ret_type llvm::cast(Y*) [with X = llvm::GlobalVariable; Y = llvm::GlobalValue; typename llvm::cast_retty<X, Y*>::ret_type = llvm::GlobalVariable*]: Assertion `isa<X>(Val) && "cast<Ty>() argument of incompatible type!"' failed.

signal (6): Aborted
in expression starting at none:0
gsignal at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
abort at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x7f167326e41e)
__assert_fail at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
⠋ [00m:26s] PackageCompiler: compiling incremental system imagecast<llvm::GlobalVariable, llvm::GlobalValue> at /buildworker/worker/package_linux64/build/usr/include/llvm/Support/Casting.h:269 [inlined]
operator() at /buildworker/worker/package_linux64/build/src/llvm-multiversioning.cpp:711
fix_gv_uses at /buildworker/worker/package_linux64/build/src/llvm-multiversioning.cpp:726 [inlined]
runOnModule at /buildworker/worker/package_linux64/build/src/llvm-multiversioning.cpp:1053
_ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE at /opt/julia-1.7.2/bin/../lib/julia/libLLVM-12jl.so (unknown line)
operator() at /buildworker/worker/package_linux64/build/src/aotcompile.cpp:556 [inlined]
jl_dump_native at /buildworker/worker/package_linux64/build/src/aotcompile.cpp:567
jl_write_compiler_output at /buildworker/worker/package_linux64/build/src/precompile.c:94
jl_atexit_hook at /buildworker/worker/package_linux64/build/src/init.c:211
jl_repl_entrypoint at /buildworker/worker/package_linux64/build/src/jlapi.c:702
main at /buildworker/worker/package_linux64/build/cli/loader_exe.c:42
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
_start at /opt/julia-1.7.2/bin/julia (unknown line)
Allocations: 42273919 (Pool: 42255007; Big: 18912); GC: 34
⠙ [00m:27s] PackageCompiler: compiling incremental system image
ERROR: failed process: Process(`/opt/julia-1.7.2/bin/julia --color=yes --startup-file=no '--cpu-target=generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1)' -O3 --sysimage=/tmp/jl_5BYb37/sys.ji --project=/home/mateusz/MyApp --output-o=/tmp/jl_sRJzVK.o /tmp/jl_BzVLu9`, ProcessSignaled(6)) [0]

Stacktrace:
  [1] pipeline_error
    @ ./process.jl:531 [inlined]
  [2] run(::Cmd; wait::Bool)
    @ Base ./process.jl:446
  [3] run
    @ ./process.jl:444 [inlined]
  [4] #14
    @ ~/.julia/packages/PackageCompiler/tdgyo/ext/TerminalSpinners.jl:157 [inlined]
  [5] spin(f::PackageCompiler.var"#14#15"{Cmd}, s::PackageCompiler.TerminalSpinners.Spinner{Base.TTY})
    @ PackageCompiler.TerminalSpinners ~/.julia/packages/PackageCompiler/tdgyo/ext/TerminalSpinners.jl:164
  [6] macro expansion
    @ ~/.julia/packages/PackageCompiler/tdgyo/ext/TerminalSpinners.jl:157 [inlined]
  [7] create_sysimg_object_file(object_file::String, packages::Vector{String}, packages_sysimg::Set{Base.PkgId}; project::String, base_sysimage::String, precompile_execution_file::Vector{String}, precompile_statements_file::Vector{String}, cpu_target::String, script::Nothing, sysimage_build_args::Cmd, extra_precompiles::String)
    @ PackageCompiler ~/.julia/packages/PackageCompiler/tdgyo/src/PackageCompiler.jl:356
  [8] create_sysimage(packages::Vector{String}; sysimage_path::String, project::String, precompile_execution_file::Vector{String}, precompile_statements_file::Vector{String}, incremental::Bool, filter_stdlibs::Bool, cpu_target::String, script::Nothing, sysimage_build_args::Cmd, include_transitive_dependencies::Bool, base_sysimage::Nothing, julia_init_c_file::Nothing, version::Nothing, soname::Nothing, compat_level::String, extra_precompiles::String)
    @ PackageCompiler ~/.julia/packages/PackageCompiler/tdgyo/src/PackageCompiler.jl:507
  [9] create_app(package_dir::String, app_dir::String; executables::Nothing, precompile_execution_file::Vector{String}, precompile_statements_file::Vector{String}, incremental::Bool, filter_stdlibs::Bool, force::Bool, c_driver_program::String, cpu_target::String, include_lazy_artifacts::Bool, sysimage_build_args::Cmd, include_transitive_dependencies::Bool)
    @ PackageCompiler ~/.julia/packages/PackageCompiler/tdgyo/src/PackageCompiler.jl:718
 [10] create_app(package_dir::String, app_dir::String)
    @ PackageCompiler ~/.julia/packages/PackageCompiler/tdgyo/src/PackageCompiler.jl:689
 [11] top-level scope
    @ REPL[9]:1

Manifest.txt

@mashu mashu mentioned this issue Feb 10, 2022
Closed
@maleadt
Copy link
Member

maleadt commented Feb 14, 2022

JuliaLang/julia#34064 (comment)
Fixed on 1.8

@maleadt maleadt closed this as completed Feb 14, 2022
@AnHeuermann AnHeuermann mentioned this issue Apr 7, 2022
Closed
@thazhemadam
Copy link
Contributor

Note: The fix will be backported to 1.7.3 as well (see JuliaLang/julia@a6c4674).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@mashu @vchuravy @maleadt @thazhemadam