Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Doesn't work with -O3 #299

Closed
PallHaraldsson opened this issue Apr 27, 2023 · 3 comments
Closed

Doesn't work with -O3 #299

PallHaraldsson opened this issue Apr 27, 2023 · 3 comments
Labels
bug Something isn't working

Comments

@PallHaraldsson
Copy link
Contributor

It's probably a fluke but I thought you would like to know. And I've not seen this on defaults, and only on 1.9.0-rc1

$ julia -O3

julia> @time using PythonCall
[ Info: Precompiling PythonCall [6099a3de-0909-46bc-b1f4-468b9a2dfc0d]
free(): invalid pointer

[34577] signal (6.-6): Aborted
in expression starting at REPL[4]:1
pthread_kill at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
raise at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
abort at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x7ff83ef1d6f5)
unknown function (ip: 0x7ff83ef34d7b)
unknown function (ip: 0x7ff83ef36ac3)
free at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
_PyMem_RawFree at /usr/local/src/conda/python-3.10.6/Objects/obmalloc.c:127 [inlined]
PyMem_RawFree at /usr/local/src/conda/python-3.10.6/Objects/obmalloc.c:595 [inlined]
_PyObject_Free at /usr/local/src/conda/python-3.10.6/Objects/obmalloc.c:2248 [inlined]
_PyObject_Free at /usr/local/src/conda/python-3.10.6/Objects/obmalloc.c:2239 [inlined]
PyObject_Free at /usr/local/src/conda/python-3.10.6/Objects/obmalloc.c:709 [inlined]
object_dealloc at /usr/local/src/conda/python-3.10.6/Objects/typeobject.c:4510
Py_DECREF at /usr/local/src/conda/python-3.11.3/Include/object.h:538 [inlined]
Py_XDECREF at /usr/local/src/conda/python-3.11.3/Include/object.h:602 [inlined]
_PyImport_LoadDynamicModuleWithSpec at /usr/local/src/conda/python-3.11.3/Python/importdl.c:246 [inlined]
_imp_create_dynamic_impl at /usr/local/src/conda/python-3.11.3/Python/import.c:2385 [inlined]
_imp_create_dynamic at /usr/local/src/conda/python-3.11.3/Python/clinic/import.c.h:446
cfunction_vectorcall_FASTCALL at /usr/local/src/conda/python-3.11.3/Objects/methodobject.c:427
do_call_core at /usr/local/src/conda/python-3.11.3/Python/ceval.c:7328 [inlined]
_PyEval_EvalFrameDefault at /usr/local/src/conda/python-3.11.3/Python/ceval.c:5380
_PyEval_EvalFrame at /usr/local/src/conda/python-3.11.3/Include/internal/pycore_ceval.h:73 [inlined]
_PyEval_Vector at /usr/local/src/conda/python-3.11.3/Python/ceval.c:6438 [inlined]
_PyFunction_Vectorcall at /usr/local/src/conda/python-3.11.3/Objects/call.c:393
_PyObject_VectorcallTstate at /usr/local/src/conda/python-3.11.3/Include/internal/pycore_call.h:92 [inlined]
object_vacall at /usr/local/src/conda/python-3.11.3/Objects/call.c:819
PyObject_CallMethodObjArgs at /usr/local/src/conda/python-3.11.3/Objects/call.c:879
import_find_and_load at /usr/local/src/conda/python-3.11.3/Python/import.c:1736 [inlined]
PyImport_ImportModuleLevelObject at /usr/local/src/conda/python-3.11.3/Python/import.c:1835
import_name at /usr/local/src/conda/python-3.11.3/Python/ceval.c:7428 [inlined]
_PyEval_EvalFrameDefault at /usr/local/src/conda/python-3.11.3/Python/ceval.c:3950
_PyEval_EvalFrame at /usr/local/src/conda/python-3.11.3/Include/internal/pycore_ceval.h:73 [inlined]
_PyEval_Vector at /usr/local/src/conda/python-3.11.3/Python/ceval.c:6438
PyEval_EvalCode at /usr/local/src/conda/python-3.11.3/Python/ceval.c:1154
builtin_exec_impl at /usr/local/src/conda/python-3.11.3/Python/bltinmodule.c:1077 [inlined]
builtin_exec at /usr/local/src/conda/python-3.11.3/Python/clinic/bltinmodule.c.h:465
cfunction_vectorcall_FASTCALL_KEYWORDS at /usr/local/src/conda/python-3.11.3/Objects/methodobject.c:443
do_call_core at /usr/local/src/conda/python-3.11.3/Python/ceval.c:7328 [inlined]
_PyEval_EvalFrameDefault at /usr/local/src/conda/python-3.11.3/Python/ceval.c:5380
_PyEval_EvalFrame at /usr/local/src/conda/python-3.11.3/Include/internal/pycore_ceval.h:73 [inlined]
_PyEval_Vector at /usr/local/src/conda/python-3.11.3/Python/ceval.c:6438 [inlined]
_PyFunction_Vectorcall at /usr/local/src/conda/python-3.11.3/Objects/call.c:393
_PyObject_VectorcallTstate at /usr/local/src/conda/python-3.11.3/Include/internal/pycore_call.h:92 [inlined]
object_vacall at /usr/local/src/conda/python-3.11.3/Objects/call.c:819
PyObject_CallMethodObjArgs at /usr/local/src/conda/python-3.11.3/Objects/call.c:879
import_find_and_load at /usr/local/src/conda/python-3.11.3/Python/import.c:1736 [inlined]
PyImport_ImportModuleLevelObject at /usr/local/src/conda/python-3.11.3/Python/import.c:1835
builtin___import___impl at /usr/local/src/conda/python-3.11.3/Python/bltinmodule.c:277 [inlined]
builtin___import__ at /usr/local/src/conda/python-3.11.3/Python/clinic/bltinmodule.c.h:78
cfunction_vectorcall_FASTCALL_KEYWORDS at /usr/local/src/conda/python-3.11.3/Objects/methodobject.c:443
_PyObject_VectorcallTstate at /usr/local/src/conda/python-3.11.3/Include/internal/pycore_call.h:92 [inlined]
_PyObject_CallFunctionVa at /usr/local/src/conda/python-3.11.3/Objects/call.c:536 [inlined]
PyObject_CallFunction at /usr/local/src/conda/python-3.11.3/Objects/call.c:558
PyImport_Import at /usr/local/src/conda/python-3.11.3/Python/import.c:2024
PyImport_Import at /home/pharaldsson/.julia/packages/PythonCall/dsECZ/src/cpython/pointers.jl:299 [inlined]
macro expansion at /home/pharaldsson/.julia/packages/PythonCall/dsECZ/src/Py.jl:131 [inlined]
pyimport at /home/pharaldsson/.julia/packages/PythonCall/dsECZ/src/concrete/import.jl:11
init_consts at /home/pharaldsson/.julia/packages/PythonCall/dsECZ/src/concrete/consts.jl:189
#199 at /home/pharaldsson/.julia/packages/PythonCall/dsECZ/src/PythonCall.jl:87
with_gil at /home/pharaldsson/.julia/packages/PythonCall/dsECZ/src/cpython/gil.jl:10 [inlined]
with_gil at /home/pharaldsson/.julia/packages/PythonCall/dsECZ/src/cpython/gil.jl:9 [inlined]
__init__ at /home/pharaldsson/.julia/packages/PythonCall/dsECZ/src/PythonCall.jl:86
unknown function (ip: 0x7ff7e959e0ff)
_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2731 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2913
jl_apply at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/julia.h:1878 [inlined]
jl_module_run_initializer at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:75
ijl_init_restored_modules at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/module.c:982
register_restored_modules at ./loading.jl:1074
_include_from_serialized at ./loading.jl:1020
_tryrequire_from_serialized at ./loading.jl:1401
unknown function (ip: 0x7ff7e955c0fd)
_require at ./loading.jl:1775
_require_prelocked at ./loading.jl:1619
unknown function (ip: 0x7ff7e9554c36)
macro expansion at ./loading.jl:1607 [inlined]
macro expansion at ./lock.jl:267 [inlined]
require at ./loading.jl:1570
unknown function (ip: 0x7ff7e9551c46)
_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2731 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2913
jl_apply at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/julia.h:1878 [inlined]
call_require at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:466 [inlined]
eval_import_path at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:503
jl_toplevel_eval_flex at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:731
top-level scope at ./timing.jl:273 [inlined]
top-level scope at ./REPL[4]:0
jl_toplevel_eval_flex at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:903
jl_toplevel_eval_flex at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:856
ijl_toplevel_eval_in at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:971
eval at ./boot.jl:370 [inlined]
eval_user_input at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:153
repl_backend_loop at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:249
#start_repl_backend#46 at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:234
start_repl_backend at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:231
_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2731 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2913
#run_repl#59 at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:377
run_repl at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:363
jfptr_run_repl_60577.clone_1 at /home/pharaldsson/.julia/juliaup/julia-1.9.0-rc1+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2731 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2913
#1017 at ./client.jl:421
jfptr_YY.1017_27809.clone_1 at /home/pharaldsson/.julia/juliaup/julia-1.9.0-rc1+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2731 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2913
jl_apply at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/julia.h:1878 [inlined]
jl_f__call_latest at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/builtins.c:774
#invokelatest#2 at ./essentials.jl:816 [inlined]
invokelatest at ./essentials.jl:813 [inlined]
run_main_repl at ./client.jl:405
exec_options at ./client.jl:322
_start at ./client.jl:522
jfptr__start_33350.clone_1 at /home/pharaldsson/.julia/juliaup/julia-1.9.0-rc1+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2731 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2913
jl_apply at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/julia.h:1878 [inlined]
true_main at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/jlapi.c:573
jl_repl_entrypoint at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/jlapi.c:717
main at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/cli/loader_exe.c:59
unknown function (ip: 0x7ff83eebdd8f)
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x401098)
Allocations: 20040012 (Pool: 20031256; Big: 8756); GC: 23
Aborted (core dumped)

[then right after]
$ julia -O3

julia> @time using PythonCall
  2.313616 seconds (1.43 M allocations: 89.947 MiB, 5.32% gc time, 1.59% compilation time)

Potentially this should be reported at Julialang, before 1.9 release (seems imminent) or after, if also happens there.
@PallHaraldsson PallHaraldsson added the bug Something isn't working label Apr 27, 2023
@cjdoris
Copy link
Collaborator

cjdoris commented May 14, 2023

I'm not seeing this on v1.9 - is it still an issue for you?

@PallHaraldsson
Copy link
Contributor Author

PallHaraldsson commented May 14, 2023 •
edited
Loading

It's not an issue for me on on 1.9.0 with -O3:

julia> @time using PythonCall
[ Info: Precompiling PythonCall [6099a3de-0909-46bc-b1f4-468b9a2dfc0d]
    CondaPkg Found dependencies: /home/pharaldsson/.julia/packages/PythonCall/dsECZ/CondaPkg.toml
    CondaPkg Dependencies already up to date
 87.749025 seconds (5.42 M allocations: 391.901 MiB, 0.30% gc time, 0.14% compilation time)

Then on default settings (or higher same):

julia> @time using PythonCall
  1.968430 seconds (835.54 k allocations: 55.378 MiB, 5.06% gc time, 1.57% compilation time)

Note, when this happened before if was ok on default settings then only failed on -O3 and I didn't try to compile in that order. I'm not sure it mattered, likely would have with -O3 first too (or not deterministic). I guess it's your say to close this and I support that, you could always reopen, wait and see if happens again.

The timing, just under 2 sec (and the non-worrying but long precompile time) is a little longer than ideal. But that's a separate issue. Maybe the new PrecompileTools.jl will help.

@cjdoris
Copy link
Collaborator

cjdoris commented May 15, 2023

Ok I'll close - let me know any future issues though thanks.

@cjdoris cjdoris closed this as completed May 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@cjdoris @PallHaraldsson