ClearMap gui error #6

tvladyslav · 2022-11-11T23:17:27Z

I follow the alternative start steps

conda activate Clearmap_env
python3 ~/ClearFinder/ClearMap/gui.py # <== fails here after a few minutes

Error compiling Cython file:
------------------------------------------------------------
...
import numpy as np

cimport numpy as cnp
from libc.stdlib cimport malloc, free

from . cimport RankCoreCode
^
------------------------------------------------------------

ClearMap/ImageProcessing/Filter/Rank/RankCoreCode.pyx:23:0: 'ClearMap/ImageProcessing/Filter/Rank.pxd' not found
cc1plus: warning: command-line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++
/home/vlad/.pyxbld/temp.linux-x86_64-3.6/pyrex/ClearMap/ImageProcessing/Filter/Rank/RankCoreCode.cpp:1:2: error: #error Do not use this file, it is the result of a failed Cython compilation.
    1 | #error Do not use this file, it is the result of a failed Cython compilation.
      |  ^~~~~
Traceback (most recent call last):
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/distutils/unixccompiler.py", line 118, in _compile
    extra_postargs)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/distutils/ccompiler.py", line 909, in spawn
    spawn(cmd, dry_run=self.dry_run)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/distutils/spawn.py", line 36, in spawn
    _spawn_posix(cmd, search_path, dry_run=dry_run)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/distutils/spawn.py", line 159, in _spawn_posix
    % (cmd, exit_status))
distutils.errors.DistutilsExecError: command '/home/vlad/anaconda3/envs/Clearmap_env/bin/x86_64-conda-linux-gnu-cc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/site-packages/pyximport/pyximport.py", line 215, in load_module
    inplace=build_inplace, language_level=language_level)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/site-packages/pyximport/pyximport.py", line 191, in build_module
    reload_support=pyxargs.reload_support)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/site-packages/pyximport/pyxbuild.py", line 102, in pyx_to_dll
    dist.run_commands()
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
    _build_ext.build_ext.run(self)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/distutils/command/build_ext.py", line 339, in run
    self.build_extensions()
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
    _build_ext.build_ext.build_extensions(self)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/distutils/command/build_ext.py", line 448, in build_extensions
    self._build_extensions_serial()
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/distutils/command/build_ext.py", line 473, in _build_extensions_serial
    self.build_extension(ext)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/distutils/command/build_ext.py", line 533, in build_extension
    depends=ext.depends)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/distutils/ccompiler.py", line 574, in compile
    self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/distutils/unixccompiler.py", line 120, in _compile
    raise CompileError(msg)
distutils.errors.CompileError: command '/home/vlad/anaconda3/envs/Clearmap_env/bin/x86_64-conda-linux-gnu-cc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/site-packages/pyximport/pyximport.py", line 216, in load_module
    mod = imp.load_dynamic(name, so_path)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/imp.py", line 343, in load_dynamic
    return _load(spec)
  File "ClearMap/ImageProcessing/Filter/Rank/RankCode.pyx", line 1, in init ClearMap.ImageProcessing.Filter.Rank.RankCode
    #cython: language_level=3, boundscheck=False, wraparound=False, nonecheck=False, initializedcheck=False, cdivision=True
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/site-packages/pyximport/pyximport.py", line 462, in load_module
    language_level=self.language_level)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/site-packages/pyximport/pyximport.py", line 231, in load_module
    raise exc.with_traceback(tb)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/site-packages/pyximport/pyximport.py", line 215, in load_module
    inplace=build_inplace, language_level=language_level)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/site-packages/pyximport/pyximport.py", line 191, in build_module
    reload_support=pyxargs.reload_support)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/site-packages/pyximport/pyxbuild.py", line 102, in pyx_to_dll
    dist.run_commands()
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
    _build_ext.build_ext.run(self)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/distutils/command/build_ext.py", line 339, in run
    self.build_extensions()
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
    _build_ext.build_ext.build_extensions(self)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/distutils/command/build_ext.py", line 448, in build_extensions
    self._build_extensions_serial()
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/distutils/command/build_ext.py", line 473, in _build_extensions_serial
    self.build_extension(ext)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/distutils/command/build_ext.py", line 533, in build_extension
    depends=ext.depends)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/distutils/ccompiler.py", line 574, in compile
    self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/distutils/unixccompiler.py", line 120, in _compile
    raise CompileError(msg)
ImportError: Building module ClearMap.ImageProcessing.Filter.Rank.RankCoreCode failed: ["distutils.errors.CompileError: command '/home/vlad/anaconda3/envs/Clearmap_env/bin/x86_64-conda-linux-gnu-cc' failed with exit status 1\n"]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "gui.py", line 1, in <module>
    import utils
  File "/home/vlad/sources/ClearFinder/ClearMap/utils.py", line 24, in <module>
    from ClearMap.Environment import *
  File "/home/vlad/sources/ClearFinder/ClearMap/ClearMap/Environment.py", line 62, in <module>
    import ClearMap.ImageProcessing.Filter.Rank as rnk
  File "/home/vlad/sources/ClearFinder/ClearMap/ClearMap/ImageProcessing/Filter/Rank/__init__.py", line 13, in <module>
    from .Rank import *           #analysis:ignore
  File "/home/vlad/sources/ClearFinder/ClearMap/ClearMap/ImageProcessing/Filter/Rank/Rank.py", line 65, in <module>
    from . import RankCode as code
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/site-packages/pyximport/pyximport.py", line 462, in load_module
    language_level=self.language_level)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/site-packages/pyximport/pyximport.py", line 231, in load_module
    raise exc.with_traceback(tb)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/site-packages/pyximport/pyximport.py", line 216, in load_module
    mod = imp.load_dynamic(name, so_path)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/imp.py", line 343, in load_dynamic
    return _load(spec)
  File "ClearMap/ImageProcessing/Filter/Rank/RankCode.pyx", line 1, in init ClearMap.ImageProcessing.Filter.Rank.RankCode
    #cython: language_level=3, boundscheck=False, wraparound=False, nonecheck=False, initializedcheck=False, cdivision=True
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/site-packages/pyximport/pyximport.py", line 462, in load_module
    language_level=self.language_level)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/site-packages/pyximport/pyximport.py", line 231, in load_module
    raise exc.with_traceback(tb)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/site-packages/pyximport/pyximport.py", line 215, in load_module
    inplace=build_inplace, language_level=language_level)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/site-packages/pyximport/pyximport.py", line 191, in build_module
    reload_support=pyxargs.reload_support)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/site-packages/pyximport/pyxbuild.py", line 102, in pyx_to_dll
    dist.run_commands()
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
    _build_ext.build_ext.run(self)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/distutils/command/build_ext.py", line 339, in run
    self.build_extensions()
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
    _build_ext.build_ext.build_extensions(self)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/distutils/command/build_ext.py", line 448, in build_extensions
    self._build_extensions_serial()
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/distutils/command/build_ext.py", line 473, in _build_extensions_serial
    self.build_extension(ext)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/distutils/command/build_ext.py", line 533, in build_extension
    depends=ext.depends)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/distutils/ccompiler.py", line 574, in compile
    self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
  File "/home/vlad/anaconda3/envs/Clearmap_env/lib/python3.6/distutils/unixccompiler.py", line 120, in _compile
    raise CompileError(msg)
ImportError: Building module ClearMap.ImageProcessing.Filter.Rank.RankCode failed: ['ImportError: Building module ClearMap.ImageProcessing.Filter.Rank.RankCoreCode failed: ["distutils.errors.CompileError: command \'/home/vlad/anaconda3/envs/Clearmap_env/bin/x86_64-conda-linux-gnu-cc\' failed with exit status 1\\n"]\n']

File /home/vlad/.pyxbld/temp.linux-x86_64-3.6/pyrex/ClearMap/ImageProcessing/Filter/Rank/RankCoreCode.cpp indeed contains a single line #error Do not use this file, it is the result of a failed Cython compilation..

For ClearMap2 I did compilation in 2 ways, both succeeded:

As described in readme using compile.py
As described in ClearMap2 compilation guide + fix from ClearAnatomics/ClearMap@f78f5aa

My system: "20.04.5 LTS (Focal Fossa)"inside WSL1.

Any suggestions?

The text was updated successfully, but these errors were encountered:

stegiopast · 2022-11-12T13:19:29Z

Hey,
First of all many thanks for your amazing help. Did you build the conda environment with the ~/ClearFinder/ClearMap/requirements_04_11.yml ? Actually I had this problem and updated the requirements for that reason. Many of the compilation problems arise from ClearMap2 itself. I build it with 20.04 LTS already so it should work for the distro, although I am currently working on 22.04 LTS and updated the conda environment accordingly. Is it possible to change the import command permanently with git submodule ?

stegiopast · 2022-11-12T13:48:04Z

To me it seems like an compiler error. Last time I had this issue it was useful to do the following:

conda install -c conda-forge cxx-compiler
conda install -c conda-forge c-compiler
conda install -c anaconda grpc-cpp

stegiopast · 2022-11-12T13:55:36Z

Did approach number 1 work ? (As described in readme using compile.py)
Compile.py actually only runs the ClearMap import (from ClearMap.Environment import *)
Hence, it is unexpected that ClearMap cant be imported when running gui.py.

stegiopast · 2022-11-12T14:23:09Z

I could reproduce the error. The first compilation has to be executed in the ClearMap2 folder. It is necessary for the first compilation and afterwards ClearMap can be imported via software link.

stegiopast · 2022-11-12T15:10:59Z

I got a different error message though:

/home/stefan/anaconda3/envs/Clearmap_env/compiler_compat/ld: warning: /lib/x86_64-linux-gnu/libm.so.6: unsupported GNU_PROPERTY_TYPE (5) type: 0xc0008002
/home/stefan/anaconda3/envs/Clearmap_env/compiler_compat/ld: warning: /lib/x86_64-linux-gnu/libmvec.so.1: unsupported GNU_PROPERTY_TYPE (5) type: 0xc0008002
/home/stefan/anaconda3/envs/Clearmap_env/compiler_compat/ld: warning: /lib/x86_64-linux-gnu/libc.so.6: unsupported GNU_PROPERTY_TYPE (5) type: 0xc0008002
Traceback (most recent call last):
File "/home/stefan/anaconda3/envs/Clearmap_env/lib/python3.6/site-packages/pyximport/pyximport.py", line 216, in load_module
mod = imp.load_dynamic(name, so_path)
File "/home/stefan/anaconda3/envs/Clearmap_env/lib/python3.6/imp.py", line 343, in load_dynamic
return _load(spec)
ImportError: /home/stefan/.pyxbld/lib.linux-x86_64-3.6/FillingCode.cpython-36m-x86_64-linux-gnu.so.reload1: undefined symbol: _ZSt28__throw_bad_array_new_lengthv

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "compile.py", line 1, in
from ClearMap.Environment import *
File "/home/stefan/ClearFinder/ClearMap/ClearMap2/ClearMap/Environment.py", line 78, in
import ClearMap.ImageProcessing.Experts.Vasculature as vasc
File "/home/stefan/ClearFinder/ClearMap/ClearMap2/ClearMap/ImageProcessing/Experts/Vasculature.py", line 34, in
import ClearMap.ImageProcessing.Binary.Filling as bf
File "/home/stefan/ClearFinder/ClearMap/ClearMap2/ClearMap/ImageProcessing/Binary/Filling.py", line 33, in
from . import FillingCode as code
File "/home/stefan/anaconda3/envs/Clearmap_env/lib/python3.6/site-packages/pyximport/pyximport.py", line 462, in load_module
language_level=self.language_level)
File "/home/stefan/anaconda3/envs/Clearmap_env/lib/python3.6/site-packages/pyximport/pyximport.py", line 231, in load_module
raise exc.with_traceback(tb)
File "/home/stefan/anaconda3/envs/Clearmap_env/lib/python3.6/site-packages/pyximport/pyximport.py", line 216, in load_module
mod = imp.load_dynamic(name, so_path)
File "/home/stefan/anaconda3/envs/Clearmap_env/lib/python3.6/imp.py", line 343, in load_dynamic
return _load(spec)
ImportError: Building module ClearMap.ImageProcessing.Binary.FillingCode failed: ['ImportError: /home/stefan/.pyxbld/lib.linux-x86_64-3.6/FillingCode.cpython-36m-x86_64-linux-gnu.so.reload1: undefined symbol: _ZSt28__throw_bad_array_new_lengthv\n']
/home/stefan/anaconda3/envs/Clearmap_env/lib/python3.6/site-packages/torch/distributed/distributed_c10d.py:86: UserWarning: torch.distributed.reduce_op is deprecated, please use torch.distributed.ReduceOp instead
warnings.warn("torch.distributed.reduce_op is deprecated, please use "

It seems this error occurs in combination with Ubuntu 22.04 as described in this issue:

stan-dev/pystan#294

I used the follwing command to update some C-libraries in anaconda:
cd ~/anaconda3/lib/
rm libstdc++.so libstdc++.so.6
ln -s /usr/lib64/libstdc++.so.6.0.29 libstdc++.so
ln -s /usr/lib64/libstdc++.so.6.0.29 libstdc++.so.6

I removed Clearmap_env and rebuilt it via conda:

conda env remove -n Clearmap_env
conda env create -f ~/ClearFinder/ClearMap/requirements_04_11.yml

Finally, the compilation succeeded

tvladyslav · 2022-11-12T20:07:34Z

Finally I managed to run it. Not sure what helped, most probably a software link, because I didn't need to remember to copy folder - link provides always up to date files.
I'll remove all environments and do everything from scratch.

tvladyslav · 2022-11-12T20:11:23Z

Is it possible to change the import command permanently with git submodule ?

From my point of view best option would be:

Fork ClearMap2 repository into your account
Cherry-pick and merge Fix typo ClearAnatomics/ClearMap#112 as it contains many fixes
Add compile.py (or change import) in your own ClearMap2
Use this updated ClearMap2 as a submodule.
GPLv3 allows all these actions, that is how open source works :)

tvladyslav · 2022-11-12T21:31:46Z

I did everything from scratch. Conclusions:

pip install cellfinder seems doesn't work inside conda environment. Worked without it.
One command required - addressed in Activate Clearmap_env #11
Both GUI are working, ticket can be closed.

tvladyslav mentioned this issue Nov 12, 2022

Optimize imports, small code refactoring #7

Merged

stegiopast closed this as not planned Won't fix, can't repro, duplicate, stale Nov 12, 2022

stegiopast reopened this Nov 12, 2022

stegiopast closed this as completed Nov 17, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ClearMap gui error #6

ClearMap gui error #6

tvladyslav commented Nov 11, 2022

stegiopast commented Nov 12, 2022

stegiopast commented Nov 12, 2022

stegiopast commented Nov 12, 2022

stegiopast commented Nov 12, 2022

stegiopast commented Nov 12, 2022

tvladyslav commented Nov 12, 2022

tvladyslav commented Nov 12, 2022

tvladyslav commented Nov 12, 2022

ClearMap gui error #6

ClearMap gui error #6

Comments

tvladyslav commented Nov 11, 2022

stegiopast commented Nov 12, 2022

stegiopast commented Nov 12, 2022

stegiopast commented Nov 12, 2022

stegiopast commented Nov 12, 2022

stegiopast commented Nov 12, 2022

tvladyslav commented Nov 12, 2022

tvladyslav commented Nov 12, 2022

tvladyslav commented Nov 12, 2022