Skip to content
This repository has been archived by the owner on Oct 31, 2023. It is now read-only.

Segmentation fault (core dumped) #21

Closed
youngkyoonjang opened this issue Oct 25, 2018 · 10 comments
Closed

Segmentation fault (core dumped) #21

youngkyoonjang opened this issue Oct 25, 2018 · 10 comments

Comments

@youngkyoonjang
Copy link

youngkyoonjang commented Oct 25, 2018 •
edited
Loading

It breaks by returning message 'Segmentation fault (core dumped)'.

I think I followed the installation guide correctly. But when I run the code (maskrcnn-benchmark/demo/webcam.py), it breaks by returning 'Segmentation fault (core dumped)'.

The specific location returning the message is
line 27 in the file 'boxlist_ops.py': keep = _box_nms(boxes, score, nms_thresh)
from
line 114 in the file 'inference.py': boxlist = boxlist_nms( .... )
from
line 138 in the file 'inference.py': sampled_boxes.append(self.forward_for_single_feature_map(a, o, b))
from
line 122 in the file 'rpn.py': boxes = self.box_selector_test(anchors, objectness, rpn_box_regression)
from
line 96 in the file 'rpn.py': return self._forward_test(anchors, objectness, rpn_box_regression)
from
line 50 in the file 'generalised_rcnn.py': proposals, proposal_losses = self.rpn(images, features,
targets)
from
line 205 in the file 'predictor.py': predictions = self.model(image_list)
from
file 'webcam.py': composite = coco_demo.run_on_opencv_image(img)

I thought it was because of my small GPU memory problem on my local machine (PC). But when I tried to run the same code on an available deep learning machine which has enough (about 11GB GPU) memory. It still returns the same message and breaks.

What is my problem? It would be grateful if someone has an answer. Thanks in advance.

Best,
Young
@senlinuc
Copy link

senlinuc commented Oct 25, 2018 •
edited
Loading

updating gcc to 4.9 and rebuild can fix it
https://gist.github.com/goldsborough/d466f43e8ffc948ff92de7486c5216d6

@fmassa
Copy link
Contributor

fmassa commented Oct 26, 2018

Hi,

Could you run the python script via gdb and show the traceback?

gdb python
 >> run demo.py
.....
>> bt

Also, Does this segfault when running on the CPU?

Thanks!

@ll490187880
Copy link

@youngkyoonjang Hello,I also encountered the problem, after the output of "Start training" ,the process stopped without any output. And I saw "[1] 8817 segmentation fault (core dumped) python tools/train_net.py --config-file" in the shell.

@fmassa
Copy link
Contributor

fmassa commented Oct 26, 2018

Hi @ll490187880 ,

Could you try running the aforementioned commands and give the output of the stack trace?
Something like

gdb python
>> run "tools/train_net.py --config-file ..."

and once it crashes, run

>> bt

and paste the result?

@ll490187880
Copy link

@fmassa yeah, as follows:
Thread 1 "python" received signal SIGSEGV, Segmentation fault.
0x00007fffac102424 in __gnu_cxx::new_allocator<_object*>::construct<_object*, _object*> (__p=0xb, this=0x5555565456e8) at /usr/include/c++/4.8.2/ext/new_allocator.h:120
120 { ::new((void )__p) _Up(std::forward<_Args>(__args)...); }
(gdb) bt
#0 0x00007fffac102424 in __gnu_cxx::new_allocator<_object>::construct<_object*, _object*> (__p=0xb, this=0x5555565456e8) at /usr/include/c++/4.8.2/ext/new_allocator.h:120
#1 std::allocator_traits<std::allocator<_object*> >::_S_construct<_object*, _object*> (__p=0xb, __a=...) at /usr/include/c++/4.8.2/bits/alloc_traits.h:254
#2 std::allocator_traits<std::allocator<_object*> >::construct<_object*, _object*> (__p=0xb, __a=...) at /usr/include/c++/4.8.2/bits/alloc_traits.h:393
#3 std::vector<_object*, std::allocator<_object*> >::emplace_back<_object*> (this=0x5555565456e8) at /usr/include/c++/4.8.2/bits/vector.tcc:96
#4 std::vector<_object*, std::allocator<_object*> >::push_back(_object*&&) (__x=<unknown type in /local/home/share/lilei/experiment/detection/maskrcnn-benchmark/maskrcnn_benchmark/_C.cpython-36m-x86_64-linux-gnu.so, CU 0x0, DIE 0x9b48c>, this=0x5555565456e8)
at /usr/include/c++/4.8.2/bits/stl_vector.h:920

@youngkyoonjang
Copy link
Author

Hi,

Could you run the python script via gdb and show the traceback?

gdb python
 >> run demo.py
.....
>> bt

Also, Does this segfault when running on the CPU?

Thanks!

@fmassa I did it before following @senlinuc 's comments (updating gcc to 4.9 / rebuilding it):
gdb python
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-110.el7
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
http://www.gnu.org/software/gdb/bugs/...
Reading symbols from /home/yj18885/anaconda3/envs/MaskRCNN-PyTorch1.0/bin/python3.6...done.
(gdb) run webcam.py
Starting program: /home/yj18885/anaconda3/envs/MaskRCNN-PyTorch1.0/bin/python webcam.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Missing separate debuginfo for /home/yj18885/.local/lib/python3.6/site-packages/cv2/.libs/libz-a147dcb0.so.1.2.3
Missing separate debuginfo for /home/yj18885/.local/lib/python3.6/site-packages/numpy/core/../.libs/libgfortran-ed201abd.so.3.0.0
[New Thread 0x7fffe580e700 (LWP 19715)]
[New Thread 0x7fffe500d700 (LWP 19716)]
[New Thread 0x7fffe280c700 (LWP 19717)]
[New Thread 0x7fffe000b700 (LWP 19718)]
[New Thread 0x7fffdd80a700 (LWP 19719)]
[New Thread 0x7fffd9009700 (LWP 19720)]
[New Thread 0x7fffd6808700 (LWP 19721)]
[Thread 0x7fffe280c700 (LWP 19717) exited]
[Thread 0x7fffd6808700 (LWP 19721) exited]
[Thread 0x7fffe500d700 (LWP 19716) exited]
[Thread 0x7fffdd80a700 (LWP 19719) exited]
[Thread 0x7fffe000b700 (LWP 19718) exited]
[Thread 0x7fffe580e700 (LWP 19715) exited]
[Thread 0x7fffd9009700 (LWP 19720) exited]
Detaching after fork from child process 19722.
Detaching after fork from child process 19724.
[New Thread 0x7fffd6808700 (LWP 19726)]
[New Thread 0x7fffd9009700 (LWP 19727)]
[New Thread 0x7fffdd80a700 (LWP 19728)]
[New Thread 0x7fffe000b700 (LWP 19729)]
[New Thread 0x7fff953c1700 (LWP 19730)]
[New Thread 0x7fff94bc0700 (LWP 19731)]
[New Thread 0x7fff8eefe700 (LWP 19732)]
[New Thread 0x7fff8e6fd700 (LWP 19733)]
[New Thread 0x7fff8defc700 (LWP 19734)]
[New Thread 0x7fff8d6fb700 (LWP 19735)]
[New Thread 0x7fff8cefa700 (LWP 19736)]
[New Thread 0x7fff8c6f9700 (LWP 19737)]
[New Thread 0x7fff8bef8700 (LWP 19738)]

Program received signal SIGSEGV, Segmentation fault.
0x00007fff95425424 in construct<_object*, _object*> (__p=0xb, this=0x13259e8) at /usr/include/c++/4.8.2/ext/new_allocator.h:120
120 { ::new((void )__p) _Up(std::forward<_Args>(__args)...); }
Missing separate debuginfos, use: debuginfo-install glibc-2.17-222.el7.x86_64
(gdb) bt
#0 0x00007fff95425424 in construct<_object, _object*> (__p=0xb, this=0x13259e8) at /usr/include/c++/4.8.2/ext/new_allocator.h:120
#1 _S_construct<_object*, _object*> (__p=0xb, __a=...) at /usr/include/c++/4.8.2/bits/alloc_traits.h:254
#2 construct<_object*, _object*> (__p=0xb, __a=...) at /usr/include/c++/4.8.2/bits/alloc_traits.h:393
#3 emplace_back<_object*> (this=0x13259e8) at /usr/include/c++/4.8.2/bits/vector.tcc:96
#4 push_back (__x=<unknown type in /home/yj18885/github/maskrcnn-benchmark/maskrcnn_benchmark/_C.cpython-36m-x86_64-linux-gnu.so, CU 0x0, DIE 0x9b498>, this=0x13259e8)
at /usr/include/c++/4.8.2/bits/stl_vector.h:920
#5 loader_life_support (this=0x7fffffffaa90) at /home/yj18885/anaconda3/envs/MaskRCNN-PyTorch1.0/lib/python3.6/site-packages/torch/lib/include/pybind11/cast.h:44
#6 pybind11::cpp_function::dispatcher (self=, args_in=0x7fff953ca8b8, kwargs_in=0x0)
at /home/yj18885/anaconda3/envs/MaskRCNN-PyTorch1.0/lib/python3.6/site-packages/torch/lib/include/pybind11/pybind11.h:618
#7 0x00007ffff796a08d in _PyCFunction_FastCallDict (func_obj=func_obj@entry=0x7fff9762ae10, args=args@entry=0x7fff95571be0, nargs=, kwargs=kwargs@entry=0x0) at Objects/methodobject.c:231
#8 0x00007ffff796a315 in _PyCFunction_FastCallKeywords (func=func@entry=0x7fff9762ae10, stack=stack@entry=0x7fff95571be0, nargs=, kwnames=kwnames@entry=0x0) at Objects/methodobject.c:294
#9 0x00007ffff7a0349a in call_function (pp_stack=pp_stack@entry=0x7fffffffada0, oparg=oparg@entry=3, kwnames=kwnames@entry=0x0) at Python/ceval.c:4830
#10 0x00007ffff7a0771c in _PyEval_EvalFrameDefault (f=, throwflag=) at Python/ceval.c:3328
#11 0x00007ffff7a0309e in _PyEval_EvalCodeWithName (_co=0x7fff9549adb0, globals=globals@entry=0x7fff9548e798, locals=locals@entry=0x0, args=, argcount=2, kwnames=0x7fff9547a920,
kwargs=0x21281a8, kwcount=2, kwstep=kwstep@entry=1, defs=0x7fff9547ab60, defcount=defcount@entry=2, kwdefs=kwdefs@entry=0x0, closure=0x0, name=name@entry=0x7fff9548f370, qualname=0x7fff9548f370)
at Python/ceval.c:4159
#12 0x00007ffff7a033b2 in fast_function (kwnames=0x2, nargs=, stack=, func=0x7fff9548d6a8) at Python/ceval.c:4971
#13 call_function (pp_stack=pp_stack@entry=0x7fffffffb060, oparg=oparg@entry=4, kwnames=kwnames@entry=0x7fff9547a908) at Python/ceval.c:4851
#14 0x00007ffff7a08184 in _PyEval_EvalFrameDefault (f=, throwflag=) at Python/ceval.c:3344
#15 0x00007ffff7a02730 in _PyFunction_FastCall (co=, args=, nargs=4, globals=) at Python/ceval.c:4912
#16 0x00007ffff7a03654 in fast_function (kwnames=0x0, nargs=, stack=, func=0x7fff95524488) at Python/ceval.c:4947
#17 call_function (pp_stack=pp_stack@entry=0x7fffffffb290, oparg=oparg@entry=3, kwnames=kwnames@entry=0x0) at Python/ceval.c:4851
#18 0x00007ffff7a0771c in _PyEval_EvalFrameDefault (f=, throwflag=) at Python/ceval.c:3328
#19 0x00007ffff7a0309e in _PyEval_EvalCodeWithName (_co=_co@entry=0x7fff955261e0, globals=globals@entry=0x7fff9548eb88, locals=locals@entry=0x0, args=args@entry=0x7fffffffb520, argcount=argcount@entry=4,
kwnames=kwnames@entry=0x7ffff7f93060, kwargs=kwargs@entry=0x7ffff7f93068, kwcount=kwcount@entry=0, kwstep=kwstep@entry=2, defs=0x7fff9549fd48, defcount=1, kwdefs=0x0, closure=0x0, name=0x7fffd0687a78,
qualname=0x7fff955233a0) at Python/ceval.c:4159
#20 0x00007ffff7a0bd55 in _PyFunction_FastCallDict (func=func@entry=0x7fff95524510, args=args@entry=0x7fffffffb520, nargs=4, kwargs=kwargs@entry=0x7fff95507bd0) at Python/ceval.c:5063
#21 0x00007ffff7910abe in _PyObject_FastCallDict (func=func@entry=0x7fff95524510, args=args@entry=0x7fffffffb520, nargs=nargs@entry=4, kwargs=kwargs@entry=0x7fff95507bd0) at Objects/abstract.c:2310
#22 0x00007ffff7910bae in _PyObject_Call_Prepend (func=0x7fff95524510, obj=0x7fff954dd400, args=0x7fff95476f78, kwargs=0x7fff95507bd0) at Objects/abstract.c:2373
#23 0x00007ffff791086a in PyObject_Call (func=0x7fffe48a2808, args=, kwargs=) at Objects/abstract.c:2261
#24 0x00007ffff7a0808b in do_call_core (kwdict=0x7fff95507bd0, callargs=0x7fff95476f78, func=0x7fffe48a2808) at Python/ceval.c:5099
#25 _PyEval_EvalFrameDefault (f=, throwflag=) at Python/ceval.c:3397
#26 0x00007ffff7a0309e in _PyEval_EvalCodeWithName (_co=_co@entry=0x7fffe217f9c0, globals=globals@entry=0x7fffe217c0d8, locals=locals@entry=0x0, args=args@entry=0x7fffffffb920, argcount=argcount@entry=4,
kwnames=kwnames@entry=0x0, kwargs=kwargs@entry=0x0, kwcount=kwcount@entry=0, kwstep=kwstep@entry=2, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0, name=0x7ffff7f96170, qualname=0x7fffe2180470)
at Python/ceval.c:4159
#27 0x00007ffff7a0bd55 in _PyFunction_FastCallDict (func=func@entry=0x7fff977bbf28, args=args@entry=0x7fffffffb920, nargs=4, kwargs=kwargs@entry=0x0) at Python/ceval.c:5063
#28 0x00007ffff7910abe in _PyObject_FastCallDict (func=func@entry=0x7fff977bbf28, args=args@entry=0x7fffffffb920, nargs=nargs@entry=4, kwargs=kwargs@entry=0x0) at Objects/abstract.c:2310
#29 0x00007ffff7910bae in _PyObject_Call_Prepend (func=0x7fff977bbf28, obj=0x7fff954dd400, args=0x7fff953c8678, kwargs=0x0) at Objects/abstract.c:2373
#30 0x00007ffff791086a in PyObject_Call (func=0x7ffff7ed3fc8, args=, kwargs=) at Objects/abstract.c:2261
#31 0x00007ffff7986f11 in slot_tp_call (self=self@entry=0x7fff954dd400, args=args@entry=0x7fff953c8678, kwds=kwds@entry=0x0) at Objects/typeobject.c:6194
#32 0x00007ffff79109cb in _PyObject_FastCallDict (func=0x7fff954dd400, args=, nargs=, kwargs=0x0) at Objects/abstract.c:2331
#33 0x00007ffff7a03208 in call_function (pp_stack=pp_stack@entry=0x7fffffffbbc0, oparg=oparg@entry=3, kwnames=kwnames@entry=0x0) at Python/ceval.c:4854
#34 0x00007ffff7a0771c in _PyEval_EvalFrameDefault (f=, throwflag=) at Python/ceval.c:3328
#35 0x00007ffff7a02730 in _PyFunction_FastCall (co=, args=, nargs=4, globals=) at Python/ceval.c:4912
#36 0x00007ffff7a03654 in fast_function (kwnames=0x0, nargs=, stack=, func=0x7fff955248c8) at Python/ceval.c:4947
#37 call_function (pp_stack=pp_stack@entry=0x7fffffffbe00, oparg=oparg@entry=3, kwnames=kwnames@entry=0x0) at Python/ceval.c:4851
#38 0x00007ffff7a0771c in _PyEval_EvalFrameDefault (f=, throwflag=) at Python/ceval.c:3328
#39 0x00007ffff7a0309e in _PyEval_EvalCodeWithName (_co=_co@entry=0x7fff9547b1e0, globals=globals@entry=0x7fff9546b480, locals=locals@entry=0x0, args=args@entry=0x7fffffffc090, argcount=argcount@entry=4,
kwnames=kwnames@entry=0x7ffff7f93060, kwargs=kwargs@entry=0x7ffff7f93068, kwcount=kwcount@entry=0, kwstep=kwstep@entry=2, defs=0x7fff95477808, defcount=1, kwdefs=0x0, closure=0x0, name=0x7fffd0687a78,
qualname=0x7fff9548ba98) at Python/ceval.c:4159
#40 0x00007ffff7a0bd55 in _PyFunction_FastCallDict (func=func@entry=0x7fff955247b8, args=args@entry=0x7fffffffc090, nargs=4, kwargs=kwargs@entry=0x7fff95507d38) at Python/ceval.c:5063
#41 0x00007ffff7910abe in _PyObject_FastCallDict (func=func@entry=0x7fff955247b8, args=args@entry=0x7fffffffc090, nargs=nargs@entry=4, kwargs=kwargs@entry=0x7fff95507d38) at Objects/abstract.c:2310
#42 0x00007ffff7910bae in _PyObject_Call_Prepend (func=0x7fff955247b8, obj=0x7fff954bcb00, args=0x7fff9555ce10, kwargs=0x7fff95507d38) at Objects/abstract.c:2373
#43 0x00007ffff791086a in PyObject_Call (func=0x7ffff7f859c8, args=, kwargs=) at Objects/abstract.c:2261
#44 0x00007ffff7a0808b in do_call_core (kwdict=0x7fff95507d38, callargs=0x7fff9555ce10, func=0x7ffff7f859c8) at Python/ceval.c:5099
---Type to continue, or q to quit---
#45 _PyEval_EvalFrameDefault (f=, throwflag=) at Python/ceval.c:3397
#46 0x00007ffff7a0309e in _PyEval_EvalCodeWithName (_co=_co@entry=0x7fffe217f9c0, globals=globals@entry=0x7fffe217c0d8, locals=locals@entry=0x0, args=args@entry=0x7fffffffc490, argcount=argcount@entry=4,
kwnames=kwnames@entry=0x0, kwargs=kwargs@entry=0x0, kwcount=kwcount@entry=0, kwstep=kwstep@entry=2, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0, name=0x7ffff7f96170, qualname=0x7fffe2180470)
at Python/ceval.c:4159
#47 0x00007ffff7a0bd55 in _PyFunction_FastCallDict (func=func@entry=0x7fff977bbf28, args=args@entry=0x7fffffffc490, nargs=4, kwargs=kwargs@entry=0x0) at Python/ceval.c:5063
#48 0x00007ffff7910abe in _PyObject_FastCallDict (func=func@entry=0x7fff977bbf28, args=args@entry=0x7fffffffc490, nargs=nargs@entry=4, kwargs=kwargs@entry=0x0) at Objects/abstract.c:2310
#49 0x00007ffff7910bae in _PyObject_Call_Prepend (func=0x7fff977bbf28, obj=0x7fff954bcb00, args=0x7fff95554f30, kwargs=0x0) at Objects/abstract.c:2373
#50 0x00007ffff791086a in PyObject_Call (func=0x7ffff7f85988, args=, kwargs=) at Objects/abstract.c:2261
#51 0x00007ffff7986f11 in slot_tp_call (self=self@entry=0x7fff954bcb00, args=args@entry=0x7fff95554f30, kwds=kwds@entry=0x0) at Objects/typeobject.c:6194
#52 0x00007ffff79109cb in _PyObject_FastCallDict (func=0x7fff954bcb00, args=, nargs=, kwargs=0x0) at Objects/abstract.c:2331
#53 0x00007ffff7a03208 in call_function (pp_stack=pp_stack@entry=0x7fffffffc730, oparg=oparg@entry=3, kwnames=kwnames@entry=0x0) at Python/ceval.c:4854
#54 0x00007ffff7a0771c in _PyEval_EvalFrameDefault (f=, throwflag=) at Python/ceval.c:3328
#55 0x00007ffff7a0309e in _PyEval_EvalCodeWithName (_co=_co@entry=0x7fff95ce0300, globals=globals@entry=0x7fff9546f708, locals=locals@entry=0x0, args=args@entry=0x7fffffffc9c0, argcount=argcount@entry=2,
kwnames=kwnames@entry=0x7ffff7f93060, kwargs=kwargs@entry=0x7ffff7f93068, kwcount=kwcount@entry=0, kwstep=kwstep@entry=2, defs=0x7fff95cfcfb0, defcount=1, kwdefs=0x0, closure=0x0, name=0x7fffd0687a78,
qualname=0x7fff9546b780) at Python/ceval.c:4159
#56 0x00007ffff7a0bd55 in _PyFunction_FastCallDict (func=func@entry=0x7fff9553bea0, args=args@entry=0x7fffffffc9c0, nargs=2, kwargs=kwargs@entry=0x7fff95507c60) at Python/ceval.c:5063
#57 0x00007ffff7910abe in _PyObject_FastCallDict (func=func@entry=0x7fff9553bea0, args=args@entry=0x7fffffffc9c0, nargs=nargs@entry=2, kwargs=kwargs@entry=0x7fff95507c60) at Objects/abstract.c:2310
#58 0x00007ffff7910bae in _PyObject_Call_Prepend (func=0x7fff9553bea0, obj=0x7fff955829e8, args=0x7fff9550b160, kwargs=0x7fff95507c60) at Objects/abstract.c:2373
#59 0x00007ffff791086a in PyObject_Call (func=0x7ffff7ea7908, args=, kwargs=) at Objects/abstract.c:2261
#60 0x00007ffff7a0808b in do_call_core (kwdict=0x7fff95507c60, callargs=0x7fff9550b160, func=0x7ffff7ea7908) at Python/ceval.c:5099
#61 _PyEval_EvalFrameDefault (f=, throwflag=) at Python/ceval.c:3397
#62 0x00007ffff7a0309e in _PyEval_EvalCodeWithName (_co=_co@entry=0x7fffe217f9c0, globals=globals@entry=0x7fffe217c0d8, locals=locals@entry=0x0, args=args@entry=0x7fffffffcdc0, argcount=argcount@entry=2,
kwnames=kwnames@entry=0x0, kwargs=kwargs@entry=0x0, kwcount=kwcount@entry=0, kwstep=kwstep@entry=2, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0, name=0x7ffff7f96170, qualname=0x7fffe2180470)
at Python/ceval.c:4159
#63 0x00007ffff7a0bd55 in _PyFunction_FastCallDict (func=func@entry=0x7fff977bbf28, args=args@entry=0x7fffffffcdc0, nargs=2, kwargs=kwargs@entry=0x0) at Python/ceval.c:5063
#64 0x00007ffff7910abe in _PyObject_FastCallDict (func=func@entry=0x7fff977bbf28, args=args@entry=0x7fffffffcdc0, nargs=nargs@entry=2, kwargs=kwargs@entry=0x0) at Objects/abstract.c:2310
#65 0x00007ffff7910bae in _PyObject_Call_Prepend (func=0x7fff977bbf28, obj=0x7fff955829e8, args=0x7fff955824e0, kwargs=0x0) at Objects/abstract.c:2373
#66 0x00007ffff791086a in PyObject_Call (func=0x7ffff7ee7588, args=, kwargs=) at Objects/abstract.c:2261
#67 0x00007ffff7986f11 in slot_tp_call (self=self@entry=0x7fff955829e8, args=args@entry=0x7fff955824e0, kwds=kwds@entry=0x0) at Objects/typeobject.c:6194
#68 0x00007ffff79109cb in _PyObject_FastCallDict (func=0x7fff955829e8, args=, nargs=, kwargs=0x0) at Objects/abstract.c:2331
#69 0x00007ffff7a03208 in call_function (pp_stack=pp_stack@entry=0x7fffffffd060, oparg=oparg@entry=1, kwnames=kwnames@entry=0x0) at Python/ceval.c:4854
#70 0x00007ffff7a0771c in _PyEval_EvalFrameDefault (f=, throwflag=) at Python/ceval.c:3328
#71 0x00007ffff7a0309e in _PyEval_EvalCodeWithName (_co=0x7fffd05f6270, globals=globals@entry=0x7fffd0658d38, locals=locals@entry=0x0, args=, argcount=2, kwnames=0x0, kwargs=0x7fff954e8b30,
kwcount=0, kwstep=kwstep@entry=1, defs=0x0, defcount=defcount@entry=0, kwdefs=kwdefs@entry=0x0, closure=0x0, name=name@entry=0x7fffd05d86a8, qualname=0x7fffd0657da0) at Python/ceval.c:4159
#72 0x00007ffff7a033b2 in fast_function (kwnames=0x0, nargs=, stack=, func=0x7fff95556400) at Python/ceval.c:4971
#73 call_function (pp_stack=pp_stack@entry=0x7fffffffd310, oparg=oparg@entry=1, kwnames=kwnames@entry=0x0) at Python/ceval.c:4851
#74 0x00007ffff7a0771c in _PyEval_EvalFrameDefault (f=, throwflag=) at Python/ceval.c:3328
#75 0x00007ffff7a02730 in _PyFunction_FastCall (co=, args=, nargs=2, globals=) at Python/ceval.c:4912
#76 0x00007ffff7a03654 in fast_function (kwnames=0x0, nargs=, stack=, func=0x7fff95556378) at Python/ceval.c:4947
#77 call_function (pp_stack=pp_stack@entry=0x7fffffffd550, oparg=oparg@entry=1, kwnames=kwnames@entry=0x0) at Python/ceval.c:4851
#78 0x00007ffff7a0771c in _PyEval_EvalFrameDefault (f=, throwflag=) at Python/ceval.c:3328
#79 0x00007ffff7a02730 in _PyFunction_FastCall (co=, args=, nargs=0, globals=) at Python/ceval.c:4912
#80 0x00007ffff7a03654 in fast_function (kwnames=0x0, nargs=, stack=, func=0x7ffff7f33e18) at Python/ceval.c:4947
#81 call_function (pp_stack=pp_stack@entry=0x7fffffffd790, oparg=oparg@entry=0, kwnames=kwnames@entry=0x0) at Python/ceval.c:4851
#82 0x00007ffff7a0771c in _PyEval_EvalFrameDefault (f=, throwflag=) at Python/ceval.c:3328
#83 0x00007ffff7a0309e in _PyEval_EvalCodeWithName (_co=_co@entry=0x7ffff0429a50, globals=globals@entry=0x7ffff7f4a1b0, locals=locals@entry=0x7ffff7f4a1b0, args=args@entry=0x0, argcount=argcount@entry=0,
kwnames=kwnames@entry=0x0, kwargs=kwargs@entry=0x0, kwcount=kwcount@entry=0, kwstep=kwstep@entry=2, defs=defs@entry=0x0, defcount=defcount@entry=0, kwdefs=kwdefs@entry=0x0, closure=closure@entry=0x0,
name=name@entry=0x0, qualname=qualname@entry=0x0) at Python/ceval.c:4159
#84 0x00007ffff7a036cd in PyEval_EvalCodeEx (_co=_co@entry=0x7ffff0429a50, globals=globals@entry=0x7ffff7f4a1b0, locals=locals@entry=0x7ffff7f4a1b0, args=args@entry=0x0, argcount=argcount@entry=0,
kws=kws@entry=0x0, kwcount=kwcount@entry=0, defs=defs@entry=0x0, defcount=defcount@entry=0, kwdefs=kwdefs@entry=0x0, closure=closure@entry=0x0) at Python/ceval.c:4180
#85 0x00007ffff7a0371b in PyEval_EvalCode (co=co@entry=0x7ffff0429a50, globals=globals@entry=0x7ffff7f4a1b0, locals=locals@entry=0x7ffff7f4a1b0) at Python/ceval.c:731
#86 0x00007ffff7a3f0a2 in run_mod (arena=0x7ffff7f65288, flags=0x7fffffffdaa0, locals=0x7ffff7f4a1b0, globals=0x7ffff7f4a1b0, filename=0x7ffff01c98b0, mod=0x6bd8f8) at Python/pythonrun.c:1025
#87 PyRun_FileExFlags (fp=fp@entry=0x691c50, filename_str=filename_str@entry=0x7ffff0464d10 "webcam.py", start=start@entry=257, globals=globals@entry=0x7ffff7f4a1b0, locals=locals@entry=0x7ffff7f4a1b0,
closeit=closeit@entry=1, flags=flags@entry=0x7fffffffdaa0) at Python/pythonrun.c:978
#88 0x00007ffff7a3f207 in PyRun_SimpleFileExFlags (fp=fp@entry=0x691c50, filename=, closeit=closeit@entry=1, flags=flags@entry=0x7fffffffdaa0) at Python/pythonrun.c:420
---Type to continue, or q to quit---
#89 0x00007ffff7a3f6a3 in PyRun_AnyFileExFlags (fp=fp@entry=0x691c50, filename=, closeit=closeit@entry=1, flags=flags@entry=0x7fffffffdaa0) at Python/pythonrun.c:81
#90 0x00007ffff7a5b6fd in run_file (p_cf=0x7fffffffdaa0, filename=0x603330 L"webcam.py", fp=0x691c50) at Modules/main.c:340
#91 Py_Main (argc=argc@entry=2, argv=argv@entry=0x602010) at Modules/main.c:810
#92 0x0000000000400bbc in main (argc=2, argv=) at ./Programs/python.c:69
(gdb)

And Yes! Segmentation fault (core dumped) happens even when I run on the CPU (python webcam.py --min-image-size 300 MODEL.DEVICE cpu).

@fmassa
Copy link
Contributor

fmassa commented Oct 26, 2018

Thanks for the stack trace!

This is a known issue with gcc < 4.9, see pytorch/pytorch#6987.

While compiling the extensions you probably saw a warning like

Your compiler (g++ 4.8) may be ABI-incompatible with PyTorch!
Please use a compiler that is ABI-compatible with GCC 4.9 and above.
See https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html.

See https://gist.github.com/goldsborough/d466f43e8ffc948ff92de7486c5216d6
for instructions on how to install GCC 4.9 or higher.

The solution is to upgrade to gcc 4.9 or higher following the instructions in https://gist.github.com/goldsborough/d466f43e8ffc948ff92de7486c5216d6

I'm closing the issue, and I'll be adding a note on the TROUBLESHOOTING section mentioning this issue.

Let us know if after updating gcc and recompiling the library (after rm -rf build/ folder) you still have issues.

@fmassa fmassa closed this as completed Oct 26, 2018
@fmassa fmassa mentioned this issue Oct 26, 2018
Merged
@fmassa
Copy link
Contributor

fmassa commented Oct 26, 2018

I've just improved the README with instructions on how to address this issue in #38
Please let me know if it doesn't help in your case.

@youngkyoonjang
Copy link
Author

@

I've just improved the README with instructions on how to address this issue in #38
Please let me know if it doesn't help in your case.

Thanks. @fmassa Now it is running perfect!

@fmassa fmassa mentioned this issue Oct 28, 2018
Closed
@zimenglan-sysu-512 zimenglan-sysu-512 mentioned this issue Nov 6, 2018
Closed
@engineer1109 engineer1109 mentioned this issue Dec 20, 2018
Open
@Jacobew Jacobew mentioned this issue Mar 7, 2019
Closed
@Fanpanda Fanpanda mentioned this issue Jun 28, 2019
Open
@dedoogong dedoogong mentioned this issue Aug 16, 2019
Open
@Jacobew Jacobew mentioned this issue Apr 19, 2020
Merged
@Exception-star
Copy link

Thanks for the stack trace!

This is a known issue with gcc < 4.9, see pytorch/pytorch#6987.

While compiling the extensions you probably saw a warning like

Your compiler (g++ 4.8) may be ABI-incompatible with PyTorch!
Please use a compiler that is ABI-compatible with GCC 4.9 and above.
See https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html.

See https://gist.github.com/goldsborough/d466f43e8ffc948ff92de7486c5216d6
for instructions on how to install GCC 4.9 or higher.

The solution is to upgrade to gcc 4.9 or higher following the instructions in https://gist.github.com/goldsborough/d466f43e8ffc948ff92de7486c5216d6

I'm closing the issue, and I'll be adding a note on the TROUBLESHOOTING section mentioning this issue.

Let us know if after updating gcc and recompiling the library (after rm -rf build/ folder) you still have issues.

#0 0x00007ffff7de6d56 in _dl_relocate_object () from /lib64/ld-linux-x86-64.so.2
#1 0x00007ffff7def7ac in dl_open_worker () from /lib64/ld-linux-x86-64.so.2
#2 0x00007ffff7dea914 in _dl_catch_error () from /lib64/ld-linux-x86-64.so.2
#3 0x00007ffff7deeccb in _dl_open () from /lib64/ld-linux-x86-64.so.2
#4 0x00007ffff792f082 in do_dlopen () from /lib64/libc.so.6
#5 0x00007ffff7dea914 in _dl_catch_error () from /lib64/ld-linux-x86-64.so.2
#6 0x00007ffff792f142 in __libc_dlopen_mode () from /lib64/libc.so.6
#7 0x00007ffff7bcee53 in pthread_cancel_init () from /lib64/libpthread.so.0
#8 0x00007ffff7bcf01c in _Unwind_ForcedUnwind () from /lib64/libpthread.so.0
#9 0x00007ffff7bcd332 in __pthread_unwind () from /lib64/libpthread.so.0
#10 0x00007ffff7bc7e77 in pthread_exit () from /lib64/libpthread.so.0
#11 0x000055555571d6f9 in PyThread_exit_thread ()
at /tmp/build/80754af9/python_1546130271559/work/Python/thread_pthread.h:300
#12 0x0000555555763d93 in t_bootstrap () at /tmp/build/80754af9/python_1546130271559/work/Modules/_threadmodule.c:1030
#13 0x00005555557182a8 in pythread_wrapper (arg=)
at /tmp/build/80754af9/python_1546130271559/work/Python/thread_pthread.h:205
#14 0x00007ffff7bc6e25 in start_thread () from /lib64/libpthread.so.0
#15 0x00007ffff78f0bad in clone () from /lib64/libc.so.6

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@fmassa @youngkyoonjang @ll490187880 @senlinuc @Exception-star