problem with concoct binning.sh line 453:61692 segmentation fault #168
Comments
|
I too am having this same issue with OpenBLAS (metawrap V1.2). I tried sym linking the libgslcblas libs as suggested, but no luck. |
When I used concoct of V1.1.8 the result showed this unbinned problem, to resolve this problem I updated metaWRAP to V1.2 and re-run it. Now, I am creating a fresh env to install V1.8 and then continue the task. Hope this trick can help me skip unbinned problem in V1.8 and OpenBLAS problem in V1.2. Best wish for ursky to solve it. |
|
I also met the problem "OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option." when running binning with CONCOCT in a new environment. |
|
In the newest version of metawrap, concoct is installed as is, without specifying any extra dependancies. In other words, it should run since I don't force any fancy libraries on it. But for some reason it doesn't work now. Does @alneberg have any suggestions about what is causing this? I don't know enough about what is under concoct's hood to know what I should change, or what packages could be causing this error. See my later post in BinPro/CONCOCT#232 |
|
For now, you can run concoct with just 1 thread, it seems. |
|
Hi, this bug is really annoying and I can't really understand why it keeps showing up. I'm not really able to maintain the c-part of the code, which is where the problems arise. And it arises because we mix openblas (from numpy) and openmp (from the c-part). However, openblas seems to disable openmp (that's the warning we get). If anyone is proficient with building packages I would greatly appreciate some help. Maybe restructuring the conda package to building openblas with this parameter specified? |
|
Thank you very much for your quick reply and information about the problem! Now I decide to run the two binning methods (metabat2 and maxbin2) except CONCOCT within metaWRAP, and run CONCOCT separately. Running binning with one thread will consume too long time…
LIANG Jinsong
Environmental Biotechnology Lab
Department of Civil Engineering
The University of Hong Kong
Pokfulam Road, Hong Kong
On 5/7/2019 22:06，Johannes Alneberg<notifications@github.com> wrote：
Hi, this bug is really annoying and I can't really understand why it keeps showing up. I'm not really able to maintain the c-part of the code, which is where the problems arise. And it arises because we mix openblas (from numpy) and openmp (from the c-part). However, openblas seems to disable openmp (that's the warning we get). If anyone is proficient with building packages I would greatly appreciate some help. Maybe restructuring the conda package to building openblas with this parameter specified?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.
|
|
Something very strange is going on with CONCOCT. Previously, I was able to get it to work in a stand-alone conda environement, but now I get the Openblas issue every time no matter what I try. See environment below. This is after clearing the downloaded packages in I am beginning suspect this might have something to do with a specific build version of some package (rather than then software version), but I can no longer produce a functional concoct environment to compare. If someone can produce a working stand-alone concoct conda environment, can you post the results of |
|
I was finally able to identify the issue. The problem does not come from any software version, but rather its build. In particular, the To fix this, just install a mkl build of blas. As far as I can tell, any version of blas is supported. In I will be enforcing this in future versions of metawrap, but for now, you guys have a fix. Let me know how it works for you. |
|
When I try conda install blas=2.5=mkl, It seems removed some files automatically like metawrap-modules and metawrap-scripts folders. please see complete bin folder of metawrap_env_V1.2 after conda install. |
|
Looks like metawrap was uninstalled because it conflicted with the new package. I do not see this behavior in v1.2 because it also calls for blas=2.5. Are you sure you had metawrap=1.2? If it still does not work, try re-installing metawrap after you install blas=2.5=mkl. |
|
I am so sorry for this missing report. I retested blas=2.5 and found that this auto removing problem caused by, downgrading salmon to v.0.9.*, command. Because some data are in the metawrap process, do you have any good suggestion on downgrading salmon without removing metawrap? Thank you so much for your patience :) |
|
Like I mentioned in your other post - install metawrap manually so you have direct control over the conda environment without incidentally uninstalling metawrap any time you change something. |
|
Can confirm that fix works! Thanks @ursky ! |
|
I also confirm the command "conda install -y blas=2.5=mkl" can fix the problem.
But I found an issue about high system CPU usage while running binning with CONCOCT.After my server has been rebooted, I execute binning command with CONCOCT method only. When "Starting binning with CONCOCT..." appears in the screen, high system CPU usage shows up and continue for the whole process. You can see the screenshot below. I wonder is it normal?
LIANG Jinsong
Environmental Biotechnology Lab
Department of Civil Engineering
The University of Hong Kong
Pokfulam Road, Hong Kong
On 5/9/2019 15:05，Raphael Eisenhofer<notifications@github.com> wrote：
Can confirm that fix works! Thanks @ursky !
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.
|
|
I don't quite follow... High CPU usage is a good thing - the program is working hard. Is it using more cores than you specified with the |
|
I failed to explain the issue clearly before. Now I try to explain again.
If you type 'top' in the command, you will see the utilization information of system sources, including CPU and memory. You can see one screenshot when I run concoct in metawrap-env after I have fixed the concoct endless warning with 'conda install -y blas=2.5=mkl'. I set the threads number to be 96 for the concoct command, and notice the second line : %CPU (s) : 26.7 us, 62.8 sy. That means only 26.7% of CPU time were used for user (in other words, the percentage of CPU time used for the concoct command was 26.7%), and up to 62.8% of CPU time were used for system itself (not for the command of user). This issue will cause a much higher time consumption in the real world for running concoct.
For your information, I also put the full names of 'us' and 'sy' below:
us: is meaning of "user CPU time"
sy: is meaning of "system CPU time"
LIANG Jinsong
Environmental Biotechnology Lab
Department of Civil Engineering
The University of Hong Kong
Pokfulam Road, Hong Kong
On 5/10/2019 20:56，Gherman V. Uritskiy<notifications@github.com> wrote：
I don't quite follow... High CPU usage is a good thing - the program is working hard. Is it using more cores than you specified with the -t parameter?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.
|
|
Thank you so much for your effort, Concoct works again! |
|
Hello @ursky, I've installed the latest
I think the problem with |
I get the same error, how do you solve it finally? |
|
Hello, I encountered the same error message ‘OpenBLAS Warning: Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option’ while using Metawrap binning (v 1.3.2). I’ve tried blas=2.5=mkl and USE_OPENMP=1 but it didn’t work. Can anyone share how they eventually resolved it? |
Hi @ursky,
After many attempts, I still can't run concoct successfully like Maxbin and Metabat. I installed metaWRAP version 1.2 in a new environment because the result ran by version 1.1.8 without any bins. The first libgslcblas.so error looks like repaired by metaWRAP automatically. And the following problems may abort the task.
Here is my command:
Part of log_file:
The text was updated successfully, but these errors were encountered: