OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.

[jolespin]

Is there a way to fix this via conda installation? I can't seem to get this to work.

Essnentially, I'm asking if there is a precompiled version I can install via conda that has USE_OPENMP=1?

I've posted a question on StackOverflow but I haven't gotten any bites:
https://stackoverflow.com/questions/57257712/how-to-fix-openblas-warning-detect-openmp-loop-and-this-application-may-hang

If I do a separate install via this post I fear that my conda environment will break.

These GitHub issues might help:
BinPro/CONCOCT#232
bxlab/metaWRAP#168

[martin-frbg]

Does conda search libopenblas --channel conda-forge bring up anything useful ?
(See https://github.com/conda-forge/openblas-feedstock - this is linked from the "precompiled installation packages" page in the wiki)

[jolespin]

Do you know if any of these use the make command with USE_OPENMP=1?

(metagenomics_env) -bash-4.1$ conda search libopenblas --channel conda-forge
Loading channels: done
# Name                       Version           Build  Channel
libopenblas                   0.2.20      h9ac9557_4  pkgs/main
libopenblas                   0.2.20      h9ac9557_7  pkgs/main
libopenblas                   0.2.20      hae245c1_3  pkgs/main
libopenblas                    0.3.2      h5a2b251_1  pkgs/main
libopenblas                    0.3.2      h7aa55d7_0  pkgs/main
libopenblas                    0.3.2      h9ac9557_0  pkgs/main
libopenblas                    0.3.2      h9ac9557_1  pkgs/main
libopenblas                    0.3.3      h5a2b251_1  pkgs/main
libopenblas                    0.3.3      h5a2b251_2  pkgs/main
libopenblas                    0.3.3      h5a2b251_3  pkgs/main
libopenblas                    0.3.6      h5a2b251_0  pkgs/main
libopenblas                    0.3.6      h5a2b251_1  pkgs/main
libopenblas                    0.3.6      h6e990d7_3  conda-forge
libopenblas                    0.3.6      h6e990d7_4  conda-forge
libopenblas                    0.3.6      h6e990d7_5  conda-forge

I've tried creating my own build and uploading it to conda but it didn't end up uploading for some reason. I downloaded https://anaconda.org/conda-forge/blas/2.10/download/linux-64/blas-2.10-mkl.tar.bz2

then made the following changes:

recipe/meta.yaml
* Removed the sha256 line
* Edited the version:     `version: '2.10.openmp_1'`
recipe/parent/build.sh
* make USE_OPENMP=1 -j${CPU_COUNT}

Then I uploaded to my anaconda cloud:

#### Compressed it
tar -cjvf blas-2.10-mkl-openmp_1.tar.bz2 info

#### Uploaded it
anaconda upload --force blas-2.10-mkl-openmp_1.tar.bz2
Using Anaconda API: https://api.anaconda.org
Using "jolespin" as upload username
Processing 'blas-2.10-mkl-openmp_1.tar.bz2'
Detecting file type...
File type is "conda"
Extracting conda package attributes for upload
Creating package "blas"
Creating release "2.10"
Uploading file "jolespin/blas/2.10/linux-64/blas-2.10-mkl-openmp_1.tar.bz2"

But it's not appearing in my cloud:
https://anaconda.org/jolespin/blas

[martin-frbg]

I expect the repository I linked above will have the build command somewhere in a build.sh or similar. Note that the conda name is openblas not blas - the "mkl" in the name of the one you tried to modify suggests it is probably a completely different BLAS implementation.

[jolespin]

Sorry, my knowledge of blas and openblas are extremely limited. The threads above mentioned that it could be fixed by an install of blas=mkl which is why I'm using the above config. Basically, if I make build and put it in my conda environment I think my conda will break due to incompatability errors so I was wondering if there was already a build that had this. Am I looking in the wrong place since the is openBLAS?

[martin-frbg]

MKL usually refers to the Math Kernel Library which comes with the Intel compilers. Basically a different implementation of the standard BLAS functions, not freely redistributable last I checked.
You really need to discuss this with the anaconda or conda-forge folks - I have no idea about compatibility aspects or build option choices within their ecosystem. If they currently do not build
OpenBLAS with USE_OPENMP=1, perhaps it is because the GNU OpenMP does not support (as in: can hang during) fork calls. (This is discussed in https://gcc.gnu.org/bugzilla/show_bug.cgi?id=60035 - unfortunately there does not appear to be any agreement about possible solutions, and that bug simply goes dormant for years at a time)

[brada4]

The BLAS you upload is MKL. Probably make sure you have only one BLAS source for both numpy and scipy.
OpenBLAS typically is called nomkl and you have to remove mkl packaging before installing same modules from nomkl aka OpenBLAS fame.
Mixing those yields "undefined result" that means everything between working right, producing wrong results, or crashing in not so obvious places.

[martin-frbg]

@isuruf will this be handled by conda-forge/openblas-feedstock ? (BTW if you build with clang/flang,
it is quite likely that the old GOMP fork bug/misfeature is not an issue as clang brings its own libomp.)

[martin-frbg]

Indeed the clang libomp passes the fork utest - both when compiling with clang, and when supplying it disguised as libgomp.so.1 to a gcc-built openblas_utest binary at runtime. (I have not checked if it is a complete run-time replacement for libgomp however - openmp.llvm.org mentions that not all OpenMP 4.0 features are supported, but it is unclear if this page is still up to date, given that it refers to clang 3.8 and gcc 4.9 in some places)
Presumably the Intel OpenMP is also "normally" fork-safe, as I found a reference (https://software.intel.com/en-us/forums/intel-c-compiler/topic/758961) to a deadlock bug that specifically affected compiler versions between 17.0.5 and 18.0.1.

[isuruf]

@martin-frbg, we are building with gcc on linux and can't replace libgomp with libomp because there are some missing symbols in libomp.

[brada4]

@isuruf

• exactly which symbols?
• is clang-flang an option?
• libiomp ?

[martin-frbg]

Actually according to https://software.intel.com/en-us/forums/intel-c-compiler/topic/793552 the LLVM libomp and Intel OpenMP runtime apear to be one and the same

[brada4]

Yep, whichever is in the distribution is valid (sort of for some time)
Exact symbol names would help to understand why it does not work. Maybe gcc header gets included and generates wondwrs, or gfortran just locks on own breed. Who knows.

[isuruf]

@@ -109,10 +23,6 @@
-GOMP_loop_nonmonotonic_dynamic_next
-GOMP_loop_nonmonotonic_dynamic_start
-GOMP_loop_nonmonotonic_guided_next
-GOMP_loop_nonmonotonic_guided_start
@@ -133,10 +43,6 @@
-GOMP_loop_ull_nonmonotonic_dynamic_next
-GOMP_loop_ull_nonmonotonic_dynamic_start
-GOMP_loop_ull_nonmonotonic_guided_next
-GOMP_loop_ull_nonmonotonic_guided_start
@@ -149,10 +55,6 @@
-GOMP_offload_register
-GOMP_offload_register_ver
-GOMP_offload_unregister
-GOMP_offload_unregister_ver
@@ -161,8 +63,6 @@
-GOMP_parallel_loop_nonmonotonic_dynamic
-GOMP_parallel_loop_nonmonotonic_guided
@@ -170,15 +70,6 @@
-GOMP_PLUGIN_acc_thread
-GOMP_PLUGIN_async_unmap_vars
-GOMP_PLUGIN_debug
-GOMP_PLUGIN_error
-GOMP_PLUGIN_fatal
-GOMP_PLUGIN_malloc
-GOMP_PLUGIN_malloc_cleared
-GOMP_PLUGIN_realloc
-GOMP_PLUGIN_target_task_completion
@@ -189,12 +80,8 @@
-GOMP_target_data_ext
-GOMP_target_enter_exit_data
-GOMP_target_ext
-GOMP_target_update_ext
@@ -203,23 +90,745 @@
-omp_get_ancestor_thread_num_8_
-omp_get_initial_device
-omp_get_initial_device_
@@ -244,25 +853,20 @@
-omp_get_partition_place_nums_8_
-omp_get_place_num_procs_8_
-omp_get_place_proc_ids_8_
-omp_get_schedule_8_
-omp_get_team_size_8_
@@ -275,41 +879,36 @@
-omp_set_default_device_8_
-omp_set_dynamic_8_
-omp_set_max_active_levels_8_
-omp_set_nested_8_
-omp_set_num_threads_8_
-omp_set_schedule_8_
-omp_target_alloc
-omp_target_associate_ptr
-omp_target_disassociate_ptr
-omp_target_free
-omp_target_is_present
-omp_target_memcpy
-omp_target_memcpy_rect

@brada4, these symbols are missing. See also http://lists.llvm.org/pipermail/openmp-dev/2017-March/001701.html

[martin-frbg]

https://www.mail-archive.com/gcc@gcc.gnu.org/msg88075.html for the gcc viewpoint. Pity that the two projects can't seem to get along. So we appear to be stuck with either a broken (though some claim POSIXly correct) behaviour on fork() or a library that lacks many modern OpenMP functions.

[isuruf]

I just sent https://reviews.llvm.org/D65714. Other than these symbols, the ones now missing in llvm-openmp 8.0.0 relate to target offloading, which we don't use anyway on conda-forge.

[brada4]

The problem is that GCC (or respective fortran) dares to import those unused unportable functions/stubs, thus theirs OMP API is headers, not the exports provided by compliant omp library.

[starwewe]

update scipy

[Rainyryan]

Conda installs pthreads build by default

conda search libopenblas --channel conda-forge
Name Version Build Channel
libopenblas 0.3.23 pthreads_h80387f5_0 conda-forge
libopenblas 0.3.24 openmp_h2e9423c_0 conda-forge
libopenblas 0.3.24 pthreads_h413a1c8_0 conda-forge

conda install libopenblas=0.3.24=openmp_h2e9423c_0 -c conda-forge

• install the openmp build