Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

Crash with latest master #4662

Closed
howard0su opened this issue Jan 13, 2017 · 4 comments
Closed

Crash with latest master #4662

howard0su opened this issue Jan 13, 2017 · 4 comments

Comments

@howard0su
Copy link
Contributor

Error:
pure virtual method called
terminate called without an active exception

the crash stack:
#0 0x00007ffff6d26c37 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x00007ffff6d2a028 in __GI_abort () at abort.c:89
#2 0x00007fffd206cb05 in __gnu_cxx::__verbose_terminate_handler() () from /home/howardsu/anaconda2/bin/../lib/libstdc++.so.6
#3 0x00007fffd206aca6 in ?? () from /home/howardsu/anaconda2/bin/../lib/libstdc++.so.6
#4 0x00007fffd206acd3 in std::terminate() () from /home/howardsu/anaconda2/bin/../lib/libstdc++.so.6
#5 0x00007fffd206b78f in __cxa_pure_virtual () from /home/howardsu/anaconda2/bin/../lib/libstdc++.so.6
#6 0x00007fffe8382cc0 in mxnet::Engine::PushSync(std::function<void (mxnet::RunContext)>, mxnet::Context, std::vector<mxnet::engine::Var*, std::allocatormxnet::engine::Var* > const&, std::vector<mxnet::engine::Var*, std::allocatormxnet::engine::Var* > const&, mxnet::FnProperty, int, char const*) (this=0x6c4550, exec_fn=..., exec_ctx=...,
exec_ctx@entry=..., const_vars=..., mutable_vars=..., prop=prop@entry=mxnet::kNormal, priority=priority@entry=0, opr_name=opr_name@entry=0x7fffe927dfbe "SyncCopyCPU2CPU")
at include/mxnet/engine.h:213
#7 0x00007fffe8421a82 in mxnet::NDArray::SyncCopyFromCPU (this=0x7fffa40026c0, data=, size=) at src/ndarray/ndarray.cc:698
#8 0x00007fffe832d6fe in MXNDArraySyncCopyFromCPU (handle=, data=, size=) at src/c_api/c_api.cc:181
#9 0x00007ffff661831c in ffi_call_unix64 () at /home/ilan/minonda/conda-bld/work/Python-2.7.12/Modules/_ctypes/libffi/src/x86/unix64.S:76
#10 0x00007ffff6617a75 in ffi_call (cif=, fn=0x7fffe832d6f0 <MXNDArraySyncCopyFromCPU(NDArrayHandle, void const*, size_t)>, rvalue=,
avalue=0x7fffb3527160) at /home/ilan/minonda/conda-bld/work/Python-2.7.12/Modules/_ctypes/libffi/src/x86/ffi64.c:525
#11 0x00007ffff660f126 in _call_function_pointer (argcount=3, resmem=0x7fffb3527190, restype=, atypes=, avalues=0x7fffb3527160,
pProc=0x7fffe832d6f0 <MXNDArraySyncCopyFromCPU(NDArrayHandle, void const*, size_t)>, flags=4353)
at /home/ilan/minonda/conda-bld/work/Python-2.7.12/Modules/_ctypes/callproc.c:837
#0 0x00007ffff6d26c37 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x00007ffff6d2a028 in __GI_abort () at abort.c:89
#2 0x00007fffd206cb05 in __gnu_cxx::__verbose_terminate_handler() () from /home/howardsu/anaconda2/bin/../lib/libstdc++.so.6
#3 0x00007fffd206aca6 in ?? () from /home/howardsu/anaconda2/bin/../lib/libstdc++.so.6
#4 0x00007fffd206acd3 in std::terminate() () from /home/howardsu/anaconda2/bin/../lib/libstdc++.so.6
#5 0x00007fffd206b78f in __cxa_pure_virtual () from /home/howardsu/anaconda2/bin/../lib/libstdc++.so.6
#6 0x00007fffe8382cc0 in mxnet::Engine::PushSync(std::function<void (mxnet::RunContext)>, mxnet::Context, std::vector<mxnet::engine::Var*, std::allocatormxnet::engine::Var* > const&, std::vector<mxnet::engine::Var*, std::allocatormxnet::engine::Var* > const&, mxnet::FnProperty, int, char const*) (this=0x6c4550, exec_fn=..., exec_ctx=...,
exec_ctx@entry=..., const_vars=..., mutable_vars=..., prop=prop@entry=mxnet::kNormal, priority=priority@entry=0, opr_name=opr_name@entry=0x7fffe927dfbe "SyncCopyCPU2CPU")
at include/mxnet/engine.h:213
#7 0x00007fffe8421a82 in mxnet::NDArray::SyncCopyFromCPU (this=0x7fffa40026c0, data=, size=) at src/ndarray/ndarray.cc:698
#8 0x00007fffe832d6fe in MXNDArraySyncCopyFromCPU (handle=, data=, size=) at src/c_api/c_api.cc:181
#9 0x00007ffff661831c in ffi_call_unix64 () at /home/ilan/minonda/conda-bld/work/Python-2.7.12/Modules/_ctypes/libffi/src/x86/unix64.S:76
#10 0x00007ffff6617a75 in ffi_call (cif=, fn=0x7fffe832d6f0 <MXNDArraySyncCopyFromCPU(NDArrayHandle, void const*, size_t)>, rvalue=,
avalue=0x7fffb3527160) at /home/ilan/minonda/conda-bld/work/Python-2.7.12/Modules/_ctypes/libffi/src/x86/ffi64.c:525
#11 0x00007ffff660f126 in _call_function_pointer (argcount=3, resmem=0x7fffb3527190, restype=, atypes=, avalues=0x7fffb3527160,
pProc=0x7fffe832d6f0 <MXNDArraySyncCopyFromCPU(NDArrayHandle, void const*, size_t)>, flags=4353)
at /home/ilan/minonda/conda-bld/work/Python-2.7.12/Modules/_ctypes/callproc.c:837
#10 0x00007ffff6617a75 in ffi_call (cif=, fn=0x7fffe832d6f0 <MXNDArraySyncCopyFromCPU(NDArrayHandle, void const*, size_t)>, rvalue=,
avalue=0x7fffb3527160) at /home/ilan/minonda/conda-bld/work/Python-2.7.12/Modules/_ctypes/libffi/src/x86/ffi64.c:525
#11 0x00007ffff660f126 in _call_function_pointer (argcount=3, resmem=0x7fffb3527190, restype=, atypes=, avalues=0x7fffb3527160,
pProc=0x7fffe832d6f0 <MXNDArraySyncCopyFromCPU(NDArrayHandle, void const*, size_t)>, flags=4353)
at /home/ilan/minonda/conda-bld/work/Python-2.7.12/Modules/_ctypes/callproc.c:837
#12 _ctypes_callproc (pProc=0x7fffe832d6f0 <MXNDArraySyncCopyFromCPU(NDArrayHandle, void const*, size_t)>, argtuple=0x7fffb3527280, flags=4353, argtypes=,
restype=0x6d6430, checker=0x0) at /home/ilan/minonda/conda-bld/work/Python-2.7.12/Modules/_ctypes/callproc.c:1180
#13 0x00007ffff6606ce3 in PyCFuncPtr_call (self=, inargs=, kwds=0x0) at /home/ilan/minonda/conda-bld/work/Python-2.7.12/Modules/_ctypes/_ctypes.c:3954
#14 0x00007ffff7a29dc3 in PyObject_Call (func=0x7fffb3e86120, arg=, kw=) at Objects/abstract.c:2546
#15 0x00007ffff7adb6c7 in do_call (nk=, na=, pp_stack=0x7fffb3527508, func=0x7fffb3e86120) at Python/ceval.c:4567
#16 call_function (oparg=, pp_stack=0x7fffb3527508) at Python/ceval.c:4372
#17 PyEval_EvalFrameEx (f=, throwflag=) at Python/ceval.c:2987
#18 0x00007ffff7ade1ce in PyEval_EvalCodeEx (co=0x7fffee0de730, globals=, locals=, args=, argcount=2, kws=0x7fffa80a8580, kwcount=0,
defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:3582
#19 0x00007ffff7add1f6 in fast_function (nk=, na=2, n=, pp_stack=0x7fffb3527728, func=0x7fffee0ece60) at Python/ceval.c:4445
#20 call_function (oparg=, pp_stack=0x7fffb3527728) at Python/ceval.c:4370
#21 PyEval_EvalFrameEx (f=, throwflag=) at Python/ceval.c:2987
#22 0x00007ffff7ade1ce in PyEval_EvalCodeEx (co=0x7fffee0de630, globals=, locals=, args=, argcount=3, kws=0x0, kwcount=0, defs=0x0,
defcount=0, closure=0x0) at Python/ceval.c:3582
#23 0x00007ffff7a597e1 in function_call (func=0x7fffee0ecd70, arg=0x7ffff7fc9cd0, kw=0x0) at Objects/funcobject.c:523
#24 0x00007ffff7a29dc3 in PyObject_Call (func=0x7fffee0ecd70, arg=, kw=) at Objects/abstract.c:2546
#25 0x00007ffff7a3c54f in instancemethod_call (func=0x7fffee0ecd70, arg=0x7ffff7fc9cd0, kw=0x0) at Objects/classobject.c:2602
#26 0x00007ffff7a29dc3 in PyObject_Call (func=0x7ffff7ea8dc0, arg=, kw=) at Objects/abstract.c:2546
#27 0x00007ffff7a938c5 in call_method (o=, name=, nameobj=0x7ffff7dbb168 <setitem_str>, format=0x7ffff7b31288 "(OO)") at Objects/typeobject.c:1281
#28 0x00007ffff7a93c2b in slot_mp_ass_subscript (self=, key=, value=) at Objects/typeobject.c:5192
#29 0x00007ffff7ad44e5 in assign_slice (u=0x7ffff7eb4090, v=, w=, x=0x7ffff7ea8e40) at Python/ceval.c:4763

@piiswrong
Copy link
Contributor

piiswrong commented Jan 13, 2017

We should probably revert the "fix cpu/gpu copy" commit.
I can't reproduce this. What's your code.

@piiswrong
Copy link
Contributor

Could you try changing capture by reference to capture by value in NDArray::SyncCopyToCPU and see if error persists

@DuinoDu
Copy link

DuinoDu commented Mar 2, 2017

Hey, I meet the same problem.

pure virtual method called
terminate called without an active exception
Aborted

I crosscompile mxnet with openblas and opencv using arm-hisiv400-linux-g++ and create libmxnet.so successfully.
Then I make a test.cpp.

#include <mxnet/ndarray.h>
int main(int argc, char const *argv[]) {
  auto ctx = mxnet::Context::CPU();
  std::initializer_list<mxnet::index_t> init = {256, 3, 256, 256};
  mxnet::TShape size(init);
  mxnet::NDArray test = mxnet::NDArray(size, ctx);  // if comment this line, no error
  return 0;
}

and compile with

 /opt/hisi-linux/x86-arm/arm-hisiv400-linux/bin/arm-hisiv400-linux-gnueabi-g++ test.cpp -o test -std=c++11 -fopenmp  -I. -L/opt/hisi-linux/x86-arm/arm-hisiv400-linux/lib_mxnet -lmxnet -lopenblas -lz -lopencv_core -lopencv_imgproc -lopencv_imgcodecs -lgomp -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_CUDA=0 -DMSHADOW_USE_SSE=0 -DMKL_EXPERIMENTAL=0

test is created successfully without any error. But when I run it on embedded target, this error occurs.

pure virtual method called
terminate called without an active exception
Aborted

But when I comment "mxnet::NDArray test = mxnet::NDArray(size, ctx)"; , it runs well.

Besides, if I use g++ on my computer(ubuntu14.04-x64), this test runs well also.

So I wonder if there is some setting I ignore when using cross compiling mxnet.

Much thanks.

@ysh329
Copy link
Contributor

ysh329 commented Apr 1, 2017

mark

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants