EXCEPTION_ACCESS_VIOLATION when using LIBSVM.jl on large datasets

[helloinrm]

Hi,
It happens to me that julia will crush when using LIBSVM.jl for classification on large datasets(It is fine on minor datasets such as 100 times 100). In Juno, the error happens when the dataset is 2000 times 2000 artificial random generated. In cmd, the error happens when dataset is 10000 times10000.

I've read all the issues about EXCEPTION_ACCESS_VIOLATION, and haven't find a solution to it.

Here's the bug report:

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x7ffea53ba5a9 -- svm_predict_values at C:\Users\inrm.julia\packages\LIBSVM\5Z99T\deps\libsvm.dll (unknown line)
in expression starting at E:\Coder\github\qisvm\qisvm.jl:705
svm_predict_values at C:\Users\inrm.julia\packages\LIBSVM\5Z99T\deps\libsvm.dll (unknown line)
vcomp_fork at C:\Windows\SYSTEM32\VCOMP140.DLL (unknown line)
vcomp_fork at C:\Windows\SYSTEM32\VCOMP140.DLL (unknown line)
vcomp_fork at C:\Windows\SYSTEM32\VCOMP140.DLL (unknown line)
svm_predict_values at C:\Users\inrm.julia\packages\LIBSVM\5Z99T\deps\libsvm.dll (unknown line)
#svmpredict#4 at C:\Users\inrm.julia\packages\LIBSVM\5Z99T\src\LIBSVM.jl:435
svmpredict at C:\Users\inrm.julia\packages\LIBSVM\5Z99T\src\LIBSVM.jl:402
_jl_invoke at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\gf.c:2141 [inlined]
jl_apply_generic at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\gf.c:2305
svm3 at E:\Coder\github\qisvm\qisvm.jl:362
competition at E:\Coder\github\qisvm\qisvm.jl:692
competition at E:\Coder\github\qisvm\qisvm.jl:670
unknown function (ip: 0000000001E34E8E)
_jl_invoke at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\gf.c:2141 [inlined]
jl_apply_generic at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\gf.c:2305
jl_apply at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\julia.h:1631 [inlined]
do_call at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\interpreter.c:328
eval_value at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\interpreter.c:417
eval_stmt_value at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\interpreter.c:368 [inlined]
eval_body at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\interpreter.c:764
jl_interpret_toplevel_thunk_callback at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\interpreter.c:888
unknown function (ip: FFFFFFFFFFFFFFFE)
unknown function (ip: 0000000038CDAD0F)
unknown function (ip: 0000000000000002)
jl_toplevel_eval_flex at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\toplevel.c:814
jl_parse_eval_all at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\ast.c:873
include_string at .\loading.jl:1075
#200 at C:\Users\inrm.julia\packages\Atom\cYxbS\src\eval.jl:166
withpath at C:\Users\inrm.julia\packages\CodeTools\kosGY\src\utils.jl:30
_jl_invoke at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\gf.c:2141 [inlined]
jl_apply_generic at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\gf.c:2305
withpath at C:\Users\inrm.julia\packages\Atom\cYxbS\src\eval.jl:9
#199 at C:\Users\inrm.julia\packages\Atom\cYxbS\src\eval.jl:163 [inlined]
with_logstate at .\logging.jl:395
with_logger at .\logging.jl:491 [inlined]
#198 at C:\Users\inrm.julia\packages\Atom\cYxbS\src\eval.jl:162 [inlined]
hideprompt at C:\Users\inrm.julia\packages\Atom\cYxbS\src\repl.jl:141
macro expansion at C:\Users\inrm.julia\packages\Atom\cYxbS\src\eval.jl:161 [inlined]
macro expansion at C:\Users\inrm.julia\packages\Media\ItEPc\src\dynamic.jl:24 [inlined]
evalall at C:\Users\inrm.julia\packages\Atom\cYxbS\src\eval.jl:151
_jl_invoke at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\gf.c:2141 [inlined]
jl_apply_generic at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\gf.c:2305
jl_apply at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\julia.h:1631 [inlined]
jl_f__apply at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\builtins.c:627
macro expansion at C:\Users\inrm.julia\packages\Atom\cYxbS\src\eval.jl:36 [inlined]
#172 at .\task.jl:333
unknown function (ip: 00000000182BE893)
_jl_invoke at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\gf.c:2141 [inlined]
jl_apply_generic at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\gf.c:2305
jl_apply at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\julia.h:1631 [inlined]
start_task at /cygdrive/d/buildbot/worker/package_win64/build/src/cygdrive/d/buildbot/worker/package_win64/build/src\task.c:659
Allocations: 92015103 (Pool: 91992416; Big: 22687); GC: 79

you can find in the report that the errors happen in function svm_predict_values().

versioninfo()
Julia Version 1.3.1
Commit 2d57411 (2019-12-30 21:36 UTC)
Platform Info:
OS: Windows (x86_64-w64-mingw32)
CPU: Intel(R) Core(TM) i5-8300H CPU @ 2.30GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-6.0.1 (ORCJIT, skylake)
Environment:
JULIA_EDITOR = "C:\Users\inrm\AppData\Local\atom\app-1.43.0\atom.exe" -a
JULIA_FFTW_PROVIDER = MKL
JULIA_NUM_THREADS = 4

I am using LIBSVM v0.4.0.

Thanks a lot if you can check it!

[bicycle1885]

I think you should avoid crossposting. Also, it would be extremely difficult to help you without any minimal working example of the problem. Please read this guideline: https://github.com/JuliaLang/julia/blob/master/CONTRIBUTING.md#how-to-file-a-bug-report. Thank you!

[helloinrm]

thank you for reminding:) i closed the issue there because i think EXCEPTION_ACCESS_VIOLATION might be a bug from julia(not sure). i am enclosing a minimal working example in a moment.

[helloinrm]

here's a minimal working example:

using LIBSVM
function svm3(X::Array{Float64,2}, y::Array{Float64}, x::Array{Float64})
    model = svmtrain(X, y, kernel = Kernel.Linear)
    (predicted_labels, decision_values) = svmpredict(model, x)
    return predicted_labels
end
n=m=10000
ratio=0.2
train_set=rand(n,m-Int(m*ratio))
test_set=rand(n,Int(m*ratio))
train_label=rand([0.0,1.0],m-Int(m*ratio))
test_label=rand([0.0,1.0],Int(m*ratio))
svm3(train_set, train_label,[test_set train_set])

[StefanKarpinski]

It seems much more likely that there's a bug in the LIBSVM bindings than that this is a Julia bug. There's an awful lot of unsafe_ and pointer calls in that library.

[StefanKarpinski]

In fact, there's so much sketchy looking code in that library that I'm going to close this.

[helloinrm]

ok, thanks!