Building from source OSX: import torchvision crash (with ffmpeg version 4.4)

[AnirudhDagar]

❓ Questions and Help

Hi,
I'm trying to build torchvision from source on OSX and have followed the contributing guidelines. I run the following

git clone https://github.com/pytorch/vision.git
cd vision

# Initialize and activate the environment
conda create -n  vision python=3.8 -y
conda activate vision

pip install flake8 typing mypy pytest scipy numpy
# Install pytorch nightly
conda install pytorch -c pytorch-nightly -c conda-forge

MACOSX_DEPLOYMENT_TARGET=10.9 CC=clang CXX=clang++ python setup.py develop

While building I can see the following clang error.

clang: error: unknown argument: '-print-multiarch'
clang: error: no input files

The setup completes, but on importing the module in python, I run into the issue below.

>>> import torchvision
libc++abi: terminating with uncaught exception of type c10::Error: Type c10::tagged_capsule<vision::video::Video> could not be converted to any of the known types.
Exception raised from operator() at /Users/gollum/anaconda3/envs/vision/lib/python3.8/site-packages/torch/include/ATen/core/jit_type.h:1592 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) + 98 (0x112b1c402 in libc10.dylib)
frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 106 (0x112b1ab7a in libc10.dylib)
frame #2: c10::detail::getTypePtr_<c10::tagged_capsule<vision::video::Video> >::call()::'lambda'()::operator()() const + 298 (0x13021bb3a in video_reader.so)
frame #3: c10::detail::getTypePtr_<c10::tagged_capsule<vision::video::Video> >::call() + 41 (0x13021b989 in video_reader.so)
frame #4: c10::detail::infer_schema::(anonymous namespace)::createArgumentVector(c10::ArrayRef<c10::detail::infer_schema::ArgumentDef>) + 216 (0x1178446b8 in libtorch_cpu.dylib)
frame #5: c10::detail::infer_schema::make_function_schema(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&, c10::ArrayRef<c10::detail::infer_schema::ArgumentDef>, c10::ArrayRef<c10::detail::infer_schema::ArgumentDef>) + 123 (0x11784445b in libtorch_cpu.dylib)
frame #6: torch::jit::Function* torch::class_<vision::video::Video>::defineMethod<torch::class_<vision::video::Video>& torch::class_<vision::video::Video>::def<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(torch::detail::types<void, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::initializer_list<torch::arg>)::'lambda'(c10::tagged_capsule<vision::video::Video>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >)>(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::initializer_list<torch::arg>) + 505 (0x130215a89 in video_reader.so)
frame #7: torch::class_<vision::video::Video>& torch::class_<vision::video::Video>::def<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(torch::detail::types<void, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::initializer_list<torch::arg>) + 113 (0x1301f0fe1 in video_reader.so)
frame #8: _GLOBAL__sub_I_video.cpp + 173 (0x13021be6d in video_reader.so)
frame #9: 0x0 + 4617463815 (0x11338e807 in ???)
frame #10: 0x0 + 4617464850 (0x11338ec12 in ???)
frame #11: 0x0 + 4617443238 (0x1133897a6 in ???)
frame #12: 0x0 + 4617434463 (0x11338755f in ???)
frame #13: 0x0 + 4617434624 (0x113387600 in ???)
frame #14: 0x0 + 4617370136 (0x113377a18 in ???)
frame #15: 0x0 + 4617416162 (0x113382de2 in ???)
frame #16: dlopen_internal(char const*, int, void*) + 185 (0x7fff20441c94 in libdyld.dylib)
frame #17: dlopen + 28 (0x7fff2043007e in libdyld.dylib)
frame #18: py_dl_open + 135 (0x10b4a6d47 in _ctypes.cpython-38-darwin.so)
<omitting python frames>

Complete build log available here: log.txt

Output for clang --version:

Apple clang version 12.0.5 (clang-1205.0.22.9)
Target: x86_64-apple-darwin20.4.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

I don't have a lot of experience with clang and was not able to find a solution for the same. I apologize before since this may be a clang specific issue and not really related to torchvision. Any help troubleshooting this will be extremely helpful. Thanks in advance!

Ps. I was able to build it just fine on another machine following the same steps, it seems like a clang problem on my machine.

[prabhat00155]

I do see this error:

clang: error: unknown argument: '-print-multiarch'
clang: error: no input files

But I don't get any error when importing torchvision.
The error seems to be related to the video reader. Does this crash your python terminal? Which ffmpeg version are you running?

[AnirudhDagar]

Hi @prabhat00155,

Thank you for replying. Yes, it does crash my python terminal.

ffmpeg -version returns the following:

ffmpeg version 4.4 Copyright (c) 2000-2021 the FFmpeg developers
built with Apple clang version 12.0.0 (clang-1200.0.32.29)
configuration: --prefix=/usr/local/Cellar/ffmpeg/4.4_1 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox
libavutil      56. 70.100 / 56. 70.100
libavcodec     58.134.100 / 58.134.100
libavformat    58. 76.100 / 58. 76.100
libavdevice    58. 13.100 / 58. 13.100
libavfilter     7.110.100 /  7.110.100
libavresample   4.  0.  0 /  4.  0.  0
libswscale      5.  9.100 /  5.  9.100
libswresample   3.  9.100 /  3.  9.100
libpostproc    55.  9.100 / 55.  9.100

[prabhat00155]

Hi @AnirudhDagar, I think the issue is related to the ffmpeg version. I originally had ffmpeg==4.2 and it was working fine. After I upgraded it to 4.4, I see this issue. I know there are some known issues with ffmpeg==4.3. If you are not using the video api, you can just uninstall ffmpeg(brew uninstall ffmpeg) and it should work fine.

[AnirudhDagar]

Thanks a tonne @prabhat00155! It was indeed an ffmpeg version issue. For now, I'm not working with the video API, so removing ffmpeg ( brew uninstall --ignore-dependencies ffmpeg) completely worked for me.

If you don't mind I'll leave this open in case someone else stumbles upon the same issue. I'll change the name of this issue for more context. I'm also linking #3188 and #3283 since they seem to mention ffmpeg based video api issues.

[datumbox]

@AnirudhDagar We need to close the issue to keep things tidy. I think users can still find the issue even if it's closed. To help them further you can edit your description and link to the reply that helped you solve it or summarize it in a few words.

If you face any additional issues, don't hesitate to reopen it. Thanks!

[fmassa]

@prabhat00155 what do you think if we skip compiling ffmpeg in torchvision is the version is not compatible, printing a message saying that that version of ffmpeg is not compatible?

[prabhat00155]

@prabhat00155 what do you think if we skip compiling ffmpeg in torchvision is the version is not compatible, printing a message saying that that version of ffmpeg is not compatible?

Yeah, I think we should do that.