OverlappingFileLockException while multithreading usage JavaCPP

[Daniel-Alievsky]

I detected a strange problem while testing JavaCPP OpenCV-based functions in multi-threading mode. It occurs not frequently, but stable enough. Unfortunately, I have no a simple test for demonstrating the bug, but, maybe, you will be able to understand, what can be a reason, from the stack-trace.

The bug occurs in a simple method:

    public opencv_core.UMat process(opencv_core.UMat source) {
        int sizeX = kernelSizeX > 0 ? kernelSizeX : kernelSizeY;
        int sizeY = kernelSizeY > 0 ? kernelSizeY : kernelSizeX;
        if (sizeX == 0 && sizeY == 0) {
            return source;
        }
        try (opencv_core.Point anchor = new opencv_core.Point(-1, -1)) {
            try (opencv_core.Size size = new opencv_core.Size(sizeX, sizeY)) {
                opencv_imgproc.blur(source, source, size, anchor, getBorderType().code());
                return source;
            }
        }
    }

It occurs while (!!) creating the point "anchor". The stack trace:

Exception in thread "main" java.lang.ExceptionInInitializerError
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:593)
	at java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:677)
	at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:735)
	at java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:160)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:174)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
	at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:583)
	at net.algart.stare.execution.model.ChainBlock.executeWithAllDependentInputs(ChainBlock.java:445)
	at net.algart.stare.execution.model.Chain.execute(Chain.java:324)
	at net.algart.stare.execution.model.tests.ExecutingChain.main(ExecutingChain.java:89)
	at com.siams.stare.execution.model.tests.ExecutingChain.main(ExecutingChain.java:10)
Caused by: java.lang.ExceptionInInitializerError
	at com.siams.stare.extensions.javacpp.opencv.matrices.filtering.Blur.process(Blur.java:70)
	at com.siams.stare.execution.javacpp.opencv.UMatFilter.processWithCompression(UMatFilter.java:27)
	at com.siams.stare.execution.javacpp.opencv.UMatFilter.process(UMatFilter.java:18)
	at net.algart.stare.execution.Executor.execute(Executor.java:322)
	at net.algart.stare.execution.model.ChainBlock.execute(ChainBlock.java:332)
	at net.algart.stare.execution.model.ChainBlock.executeWithAllDependentInputs(ChainBlock.java:390)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
	at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
	at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:291)
	at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.execLocalTasks(ForkJoinPool.java:1040)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1058)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Caused by: java.nio.channels.OverlappingFileLockException
	at sun.nio.ch.SharedFileLockTable.checkList(FileLockTable.java:255)
	at sun.nio.ch.SharedFileLockTable.add(FileLockTable.java:152)
	at sun.nio.ch.FileChannelImpl.lock(FileChannelImpl.java:1063)
	at java.nio.channels.FileChannel.lock(FileChannel.java:1053)
	at org.bytedeco.javacpp.Loader.cacheResource(Loader.java:421)
	at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:1130)
	at org.bytedeco.javacpp.Loader.load(Loader.java:940)
	at org.bytedeco.javacpp.Loader.load(Loader.java:854)
	at org.bytedeco.javacpp.opencv_core$Point.<clinit>(opencv_core.java:10511)
	... 16 more

The line Blur.java:70 is the following:
try (opencv_core.Point anchor = new opencv_core.Point(-1, -1)) {

The exception is thrown while initializing the class Point, in Loader.load method:

@Name("cv::Point_<int>") @NoOffset public static class Point extends IntPointer {
    static { Loader.load(); }

My Java is jdk1.8.0_181, OS - Windows 8.1.

Do you have any ideas, what could be reasons of such behavior? Is there any way to avoid such problem?

[saudet]

You're not running the latest version. Please try again with the snapshots and let me know if it still happens with 1.4.4-SNAPSHOT: http://bytedeco.org/builds/

[saudet]

Actually, I think I found what the issue is, commit bytedeco/javacpp@7ed4191 should work around this. Thanks for reporting!

[Daniel-Alievsky]

Sounds fine! So, this correction will be included in the next stable release on January? And now it is available in 1.4.4-SNAPSHOT?

[saudet]

Yes and yes, so please try it out! Thanks

[Daniel-Alievsky]

Оbviously, I was just lucky to discover this error: it did not repeat anymore even with old Javacpp.

[saudet]

Workaround now included in JavaCPP 1.4.4! Thanks for reporting

[Daniel-Alievsky]

Thank you very much, Samuel!
Maybe you can comment this issue: opencv/opencv#13472 ? It is interesting, whether the problem is reproduced in you system,

[saudet]

Yeah, I'm getting the same thing you do, but something like below works fine so I'm not sure why anyone would care so much. Just use a workaround like that.

synchronized (opencv_core.class) {
            final boolean haveOpenCL = opencv_core.haveOpenCL();
            final boolean useOpenCL = opencv_core.useOpenCL();
            System.out.println("Executor " + index + ": "
                    + (useOpenCL != haveOpenCL ? "!!! STRANGE SITUATION! " : "")
                    + "haveOpenCL: " + haveOpenCL + "; useOpenCL: " + useOpenCL);
            try {
                Thread.sleep(50);
            } catch (InterruptedException ignored) {
            }
}

[Daniel-Alievsky]

There is more simple workaround, that I'm using: just store this value in a constant while initializing the class :)
public static final boolean GPU_OPTIMIZATION_ENABLED = opencv_core.useOpenCL();
But, first of all, you must know about this. In my previous solutions I just checked useOpenCL() every time, when I choose between Mat-oriented and UMat-oriented procedures, and I was very surprised that my functions sometimes work in 10 times slower. (They just "saw" false result of useOpenCL().)
I still think that OpenCV authors should fix this problem.