Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] (macOS) Linking problem? symbol not found in flat namespace '__ZN2at3mps9MPSStream6commitEb' #2951

Open
2 tasks done
debugmenot opened this issue Oct 5, 2023 · 10 comments

Comments

@debugmenot
Copy link

debugmenot commented Oct 5, 2023

Prerequisite

Environment

MacOS 13.6 (Same issue on 14.0 Sonoma)
Apple Silicon M1 Ultra / 64G
PyTorch 2.1 (release)

Impossible to get info from mmcv because of symbol not found error.

Reproduces the problem - code sample

Installed by official instructions.

If building MMCV from source with PyTorch 2.1 problem still exists...

There are no such issues with PyTorch 2.0

Reproduces the problem - command or script

~/o/mmdetection (main|✔) $ mim download mmdet --config rtmdet_tiny_8xb32-300e_coco --dest . (openmmlab)
processing rtmdet_tiny_8xb32-300e_coco...
downloading ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 54.9/54.9 MiB 851.5 kB/s eta 0:00:00
Successfully downloaded rtmdet_tiny_8xb32-300e_coco_20220902_112414-78e30dcc.pth to /Users/madcode/openmmlab/mmdetection
Successfully dumped rtmdet_tiny_8xb32-300e_coco.py to /Users/madcode/openmmlab/mmdetection
~/o/mmdetection (main|✔) $ python demo/image_demo.py demo/demo.jpg rtmdet_tiny_8xb32-300e_coco.py --weights rtmdet_tiny_8xb32-300e_coco_20220902_112414-78e30dcc.pth --device cpu

Reproduces the problem - error message

~/o/mmdetection (main|✔) $ mim download mmdet --config rtmdet_tiny_8xb32-300e_coco --dest . (openmmlab)
processing rtmdet_tiny_8xb32-300e_coco...
downloading ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 54.9/54.9 MiB 851.5 kB/s eta 0:00:00
Successfully downloaded rtmdet_tiny_8xb32-300e_coco_20220902_112414-78e30dcc.pth to /Users/madcode/openmmlab/mmdetection
Successfully dumped rtmdet_tiny_8xb32-300e_coco.py to /Users/madcode/openmmlab/mmdetection
~/o/mmdetection (main|✔) $ python demo/image_demo.py demo/demo.jpg rtmdet_tiny_8xb32-300e_coco.py --weights rtmdet_tiny_8xb32-300e_coco_20220902_112414-78e30dcc.pth --device cpu
Traceback (most recent call last):
File "demo/image_demo.py", line 43, in
from mmdet.apis import DetInferencer
File "/Users/madcode/openmmlab/mmdetection/mmdet/apis/init.py", line 2, in
from .det_inferencer import DetInferencer
File "/Users/madcode/openmmlab/mmdetection/mmdet/apis/det_inferencer.py", line 21, in
from mmdet.evaluation import INSTANCE_OFFSET
File "/Users/madcode/openmmlab/mmdetection/mmdet/evaluation/init.py", line 3, in
from .metrics import * # noqa: F401,F403
File "/Users/madcode/openmmlab/mmdetection/mmdet/evaluation/metrics/init.py", line 5, in
from .coco_metric import CocoMetric
File "/Users/madcode/openmmlab/mmdetection/mmdet/evaluation/metrics/coco_metric.py", line 16, in
from mmdet.datasets.api_wrappers import COCO, COCOeval
File "/Users/madcode/openmmlab/mmdetection/mmdet/datasets/init.py", line 26, in
from .utils import get_loading_pipeline
File "/Users/madcode/openmmlab/mmdetection/mmdet/datasets/utils.py", line 5, in
from mmdet.datasets.transforms import LoadAnnotations, LoadPanopticAnnotations
File "/Users/madcode/openmmlab/mmdetection/mmdet/datasets/transforms/init.py", line 6, in
from .formatting import (ImageToTensor, PackDetInputs, PackReIDInputs,
File "/Users/madcode/openmmlab/mmdetection/mmdet/datasets/transforms/formatting.py", line 11, in
from mmdet.structures.bbox import BaseBoxes
File "/Users/madcode/openmmlab/mmdetection/mmdet/structures/bbox/init.py", line 2, in
from .base_boxes import BaseBoxes
File "/Users/madcode/openmmlab/mmdetection/mmdet/structures/bbox/base_boxes.py", line 9, in
from mmdet.structures.mask.structures import BitmapMasks, PolygonMasks
File "/Users/madcode/openmmlab/mmdetection/mmdet/structures/mask/init.py", line 3, in
from .structures import (BaseInstanceMasks, BitmapMasks, PolygonMasks,
File "/Users/madcode/openmmlab/mmdetection/mmdet/structures/mask/structures.py", line 12, in
from mmcv.ops.roi_align import roi_align
File "/opt/homebrew/Caskroom/miniconda/base/envs/openmmlab/lib/python3.8/site-packages/mmcv/ops/init.py", line 2, in
from .active_rotated_filter import active_rotated_filter
File "/opt/homebrew/Caskroom/miniconda/base/envs/openmmlab/lib/python3.8/site-packages/mmcv/ops/active_rotated_filter.py", line 10, in
ext_module = ext_loader.load_ext(
File "/opt/homebrew/Caskroom/miniconda/base/envs/openmmlab/lib/python3.8/site-packages/mmcv/utils/ext_loader.py", line 13, in load_ext
ext = importlib.import_module('mmcv.' + name)
File "/opt/homebrew/Caskroom/miniconda/base/envs/openmmlab/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ImportError: dlopen(/opt/homebrew/Caskroom/miniconda/base/envs/openmmlab/lib/python3.8/site-packages/mmcv/_ext.cpython-38-darwin.so, 0x0002): symbol not found in flat namespace '__ZN2at3mps9MPSStream6commitEb'

Additional information

No response

@debugmenot
Copy link
Author

debugmenot commented Oct 5, 2023

Building from source:

~/openmmlab $ conda create --name openmmlab_2 python=3.8 -y
Collecting package metadata (current_repodata.json): done
Solving environment: done

(...)

ca-certificates pkgs/main/osx-arm64::ca-certificates-2023.08.22-hca03da5_0
libcxx pkgs/main/osx-arm64::libcxx-14.0.6-h848a8c0_0
libffi pkgs/main/osx-arm64::libffi-3.4.4-hca03da5_0
ncurses pkgs/main/osx-arm64::ncurses-6.4-h313beb8_0
openssl pkgs/main/osx-arm64::openssl-3.0.11-h1a28f6b_2
pip pkgs/main/osx-arm64::pip-23.2.1-py38hca03da5_0
python pkgs/main/osx-arm64::python-3.8.18-hb885b13_0
readline pkgs/main/osx-arm64::readline-8.2-h1a28f6b_0
setuptools pkgs/main/osx-arm64::setuptools-68.0.0-py38hca03da5_0
sqlite pkgs/main/osx-arm64::sqlite-3.41.2-h80987f9_0
tk pkgs/main/osx-arm64::tk-8.6.12-hb8d0fd4_0
wheel pkgs/main/osx-arm64::wheel-0.41.2-py38hca03da5_0
xz pkgs/main/osx-arm64::xz-5.4.2-h80987f9_0
zlib pkgs/main/osx-arm64::zlib-1.2.13-h5a0b063_0

(...)

~/openmmlab $ conda activate openmmlab_2
~/openmmlab $ conda install pytorch torchvision cpuonly -c pytorch (openmmlab_2)
Collecting package metadata (current_repodata.json): done
Solving environment: done

(...)

The following packages will be downloaded:

package                    |            build
---------------------------|-----------------
cpuonly-2.0                |                0           2 KB  pytorch
pytorch-mutex-1.0          |              cpu           3 KB  pytorch
------------------------------------------------------------
                                       Total:           5 KB

The following NEW packages will be INSTALLED:

blas pkgs/main/osx-arm64::blas-1.0-openblas
brotlipy pkgs/main/osx-arm64::brotlipy-0.7.0-py38h1a28f6b_1002
bzip2 pkgs/main/osx-arm64::bzip2-1.0.8-h620ffc9_4
certifi pkgs/main/osx-arm64::certifi-2023.7.22-py38hca03da5_0
cffi pkgs/main/osx-arm64::cffi-1.15.1-py38h80987f9_3
charset-normalizer pkgs/main/noarch::charset-normalizer-2.0.4-pyhd3eb1b0_0
cpuonly pytorch/noarch::cpuonly-2.0-0
cryptography pkgs/main/osx-arm64::cryptography-41.0.3-py38hd4332d6_0
ffmpeg pkgs/main/osx-arm64::ffmpeg-4.2.2-h04105a8_0
filelock pkgs/main/osx-arm64::filelock-3.9.0-py38hca03da5_0
freetype pkgs/main/osx-arm64::freetype-2.12.1-h1192e45_0
gettext pkgs/main/osx-arm64::gettext-0.21.0-h13f89a0_1
giflib pkgs/main/osx-arm64::giflib-5.2.1-h80987f9_3
gmp pkgs/main/osx-arm64::gmp-6.2.1-hc377ac9_3
gmpy2 pkgs/main/osx-arm64::gmpy2-2.1.2-py38h8c48613_0
gnutls pkgs/main/osx-arm64::gnutls-3.6.15-h887c41c_0
icu pkgs/main/osx-arm64::icu-73.1-h313beb8_0
idna pkgs/main/osx-arm64::idna-3.4-py38hca03da5_0
jinja2 pkgs/main/osx-arm64::jinja2-3.1.2-py38hca03da5_0
jpeg pkgs/main/osx-arm64::jpeg-9e-h80987f9_1
lame pkgs/main/osx-arm64::lame-3.100-h1a28f6b_0
lcms2 pkgs/main/osx-arm64::lcms2-2.12-hba8e193_0
lerc pkgs/main/osx-arm64::lerc-3.0-hc377ac9_0
libdeflate pkgs/main/osx-arm64::libdeflate-1.17-h80987f9_1
libgfortran pkgs/main/osx-arm64::libgfortran-5.0.0-11_3_0_hca03da5_28
libgfortran5 pkgs/main/osx-arm64::libgfortran5-11.3.0-h009349e_28
libiconv pkgs/main/osx-arm64::libiconv-1.16-h1a28f6b_2
libidn2 pkgs/main/osx-arm64::libidn2-2.3.4-h80987f9_0
libjpeg-turbo pkgs/main/osx-arm64::libjpeg-turbo-2.0.0-h1a28f6b_0
libopenblas pkgs/main/osx-arm64::libopenblas-0.3.21-h269037a_0
libopus pkgs/main/osx-arm64::libopus-1.3-h1a28f6b_1
libpng pkgs/main/osx-arm64::libpng-1.6.39-h80987f9_0
libtasn1 pkgs/main/osx-arm64::libtasn1-4.19.0-h80987f9_0
libtiff pkgs/main/osx-arm64::libtiff-4.5.1-h313beb8_0
libunistring pkgs/main/osx-arm64::libunistring-0.9.10-h1a28f6b_0
libvpx pkgs/main/osx-arm64::libvpx-1.10.0-hc377ac9_0
libwebp pkgs/main/osx-arm64::libwebp-1.3.2-ha3663a8_0
libwebp-base pkgs/main/osx-arm64::libwebp-base-1.3.2-h80987f9_0
libxml2 pkgs/main/osx-arm64::libxml2-2.10.4-h0dcf63f_1
llvm-openmp pkgs/main/osx-arm64::llvm-openmp-14.0.6-hc6e5704_0
lz4-c pkgs/main/osx-arm64::lz4-c-1.9.4-h313beb8_0
markupsafe pkgs/main/osx-arm64::markupsafe-2.1.1-py38h1a28f6b_0
mpc pkgs/main/osx-arm64::mpc-1.1.0-h8c48613_1
mpfr pkgs/main/osx-arm64::mpfr-4.0.2-h695f6f0_1
mpmath pkgs/main/osx-arm64::mpmath-1.3.0-py38hca03da5_0
nettle pkgs/main/osx-arm64::nettle-3.7.3-h84b5d62_1
networkx pkgs/main/osx-arm64::networkx-3.1-py38hca03da5_0
numpy pkgs/main/osx-arm64::numpy-1.24.3-py38h1398885_0
numpy-base pkgs/main/osx-arm64::numpy-base-1.24.3-py38h90707a3_0
openh264 pkgs/main/osx-arm64::openh264-1.8.0-h98b2900_0
pillow pkgs/main/osx-arm64::pillow-9.4.0-py38h313beb8_1
pycparser pkgs/main/noarch::pycparser-2.21-pyhd3eb1b0_0
pyopenssl pkgs/main/osx-arm64::pyopenssl-23.2.0-py38hca03da5_0
pysocks pkgs/main/osx-arm64::pysocks-1.7.1-py38hca03da5_0
pytorch pytorch/osx-arm64::pytorch-2.1.0-py3.8_0
pytorch-mutex pytorch/noarch::pytorch-mutex-1.0-cpu
pyyaml pkgs/main/osx-arm64::pyyaml-6.0-py38h80987f9_1
requests pkgs/main/osx-arm64::requests-2.31.0-py38hca03da5_0
sympy pkgs/main/osx-arm64::sympy-1.11.1-py38hca03da5_0
torchvision pytorch/osx-arm64::torchvision-0.16.0-py38_cpu
typing_extensions pkgs/main/osx-arm64::typing_extensions-4.7.1-py38hca03da5_0
urllib3 pkgs/main/osx-arm64::urllib3-1.26.16-py38hca03da5_0
x264 pkgs/main/osx-arm64::x264-1!152.20180806-h1a28f6b_0
yaml pkgs/main/osx-arm64::yaml-0.2.5-h1a28f6b_0
zstd pkgs/main/osx-arm64::zstd-1.5.5-hd90d995_0

(...)

~/openmmlab $ python -c 'import torch;print(torch.version)' (openmmlab_2)
2.1.0
~/openmmlab $ git clone https://github.com/open-mmlab/mmcv.git
~/openmmlab $ cd mmcv
~/o/mmcv (main|✔) $ pip install -r requirements/optional.txt (openmmlab_2)
Collecting ninja (from -r requirements/optional.txt (line 1))
Downloading ninja-1.11.1-py2.py3-none-macosx_10_9_universal2.macosx_10_9_x86_64.macosx_11_0_arm64.macosx_11_0_universal2.whl (270 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 270.7/270.7 kB 1.2 MB/s eta 0:00:00
Collecting psutil (from -r requirements/optional.txt (line 2))
Downloading psutil-5.9.5-cp38-abi3-macosx_11_0_arm64.whl (246 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 246.1/246.1 kB 7.0 MB/s eta 0:00:00
Installing collected packages: ninja, psutil
Successfully installed ninja-1.11.1 psutil-5.9.5
~/o/mmcv (main|✔) $ gcc --version (openmmlab_2)
Apple clang version 15.0.0 (clang-1500.0.40.1)
Target: arm64-apple-darwin22.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
~/o/mmcv (main|✔) $ pip install -e . -v

(…)

building output omitted. Only warnings are like:

ld: warning: object file (/Users/madcode/openmmlab/mmcv/build/temp.macosx-11.1-arm64-cpython-38/mmcv/ops/csrc/pytorch/voxelization.o) was built for newer 'macOS' version (13.0) than being linked (11.1)
and so one...

Installed /Users/madcode/openmmlab/mmcv

Successfully installed addict-2.4.0 contourpy-1.1.1 cycler-0.12.0 fonttools-4.43.0 importlib-metadata-6.8.0 importlib-resources-6.1.0 kiwisolver-1.4.5 markdown-it-py-3.0.0 matplotlib-3.7.3 mdurl-0.1.2 mmcv-2.0.1 mmengine-0.8.5 opencv-python-4.8.1.78 packaging-23.2 platformdirs-3.11.0 pygments-2.16.1 pyparsing-3.1.1 python-dateutil-2.8.2 rich-13.6.0 six-1.16.0 termcolor-2.3.0 tomli-2.0.1 yapf-0.40.2 zipp-3.17.0

~/o/mmcv (main|✔) $ python .dev_scripts/check_installation.py (openmmlab_2)
Traceback (most recent call last):
File ".dev_scripts/check_installation.py", line 4, in
from mmcv.ops import box_iou_rotated
File "/Users/madcode/openmmlab/mmcv/mmcv/ops/init.py", line 3, in
from .active_rotated_filter import active_rotated_filter
File "/Users/madcode/openmmlab/mmcv/mmcv/ops/active_rotated_filter.py", line 10, in
ext_module = ext_loader.load_ext(
File "/Users/madcode/openmmlab/mmcv/mmcv/utils/ext_loader.py", line 13, in load_ext
ext = importlib.import_module('mmcv.' + name)
File "/opt/homebrew/Caskroom/miniconda/base/envs/openmmlab_2/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ImportError: dlopen(/Users/madcode/openmmlab/mmcv/mmcv/_ext.cpython-38-darwin.so, 0x0002): symbol not found in flat namespace '__ZN2at3mps9MPSStream6commitEb'

@debugmenot debugmenot changed the title [Bug] Linking problem? symbol not found in flat namespace '__ZN2at3mps9MPSStream6commitEb' [Bug] (macOS) Linking problem? symbol not found in flat namespace '__ZN2at3mps9MPSStream6commitEb' Oct 5, 2023
@fazhang-master
Copy link

image

@zhouzaida
Copy link
Collaborator

Hi @debugmenot , as a workaround, you can pull the main branch to fix this error (#2958) or downgrade your torch to 2.0.0 and we will fix this error for 2.1.0 in the future.

@debugmenot
Copy link
Author

Hi! Any schedule to return back MPS support? 2.1.0 is already released but MPS is disabled :( There are lots of new MPS functions supported in Torch 2.1...

@zhouzaida
Copy link
Collaborator

zhouzaida commented Oct 23, 2023

Hi @debugmenot, the support for MPS backend is in our schedule but with a not high priority. By the way, you can still use the op supported in torch.

@kojipy
Copy link

kojipy commented Jan 18, 2024

@debugmenot
I encountered sama error when I use python 3.10.1. but swithing to python3.9.6 resolved the issue.
I'm not sure but using python3.9.x might potentialy fix this issue.

@LeiChen9
Copy link

LeiChen9 commented Aug 7, 2024

2.0.0, still don't work

@zhiwuz
Copy link

zhiwuz commented Aug 9, 2024

2.0.0, still don't work

+1

@RoderickQiu
Copy link

When I use mmcv 2.0.x it failed. But after changing to both using mmcv 2.1.0 and torch 2.0.x it get fixed.

@AnshVM
Copy link

AnshVM commented Aug 24, 2024

I fixed this by downgrading
pip install mmcv==2.0.0 torch==2.0.0 torchvision==0.15.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants