torchvision conda packaging fails to resolve when opencv constraint is present #4648
Assignees
Labels
Comments
|
An easier failure case is: Create an env with opencv Try and install |
|
@datumbox just fyi, I confirmed that this is not a dependency issue, but a packaging issue -- i.e. our packaging seems to be at fault. |
|
Sounds good, thanks for investigating. |
|
additional data point, having conda create -n dlibtest python=3.7 dlib -c conda-forge
conda activate dlibtest
conda install "torchvision>=0.10" -c pytorch |
|
I believe the conflict comes from ffmpeg incopatiblity between torchvision and opencv |
|
I thought it was |
Edit: and libjpeg dependency is versioned in 0.10 release vision/packaging/torchvision/meta.yaml Lines 24 to 29 in ca1a620 |
|
Alas, pytorch from nightly also have conflicts: |
|
this is weird, because I don't see a conflict for |
|
Interestingly, changing order of operations works: |
|
the reason changing the order of operations probably works is because it then installs an old For me, the following simply changes pytorch / torchvision to conda-forge: conda create -n pytorch-forge python=3.7 "torchvision>=0.10.1" -c pytorch -c conda-forge
conda activate pytorch-forge$ conda install "dlib" -c conda-forge
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: done
## Package Plan ##
environment location: /home/soumith/miniconda3/envs/pytorch-forge
added / updated specs:
- dlib
The following packages will be downloaded:
package | build
---------------------------|-----------------
pytorch-cpu-1.1.0 | py37he1b5a44_0 50.7 MB conda-forge
------------------------------------------------------------
Total: 50.7 MB
The following NEW packages will be INSTALLED:
cffi conda-forge/linux-64::cffi-1.14.6-py37h036bc23_1
dlib conda-forge/linux-64::dlib-19.22.0-py37hc3f859c_0
libblas conda-forge/linux-64::libblas-3.9.0-12_linux64_mkl
libcblas conda-forge/linux-64::libcblas-3.9.0-12_linux64_mkl
liblapack conda-forge/linux-64::liblapack-3.9.0-12_linux64_mkl
numpy conda-forge/linux-64::numpy-1.21.2-py37h31617e3_0
pycparser conda-forge/noarch::pycparser-2.20-pyh9f0ad1d_2
pytorch-cpu conda-forge/linux-64::pytorch-cpu-1.1.0-py37he1b5a44_0
The following packages will be UPDATED:
jpeg pkgs/main::jpeg-9b-h024ee3a_2 --> conda-forge::jpeg-9d-h36c2ea0_0
The following packages will be SUPERSEDED by a higher-priority channel:
torchvision pytorch::torchvision-0.10.1-py37_cu111 --> conda-forge::torchvision-0.10.1-py37h9e046cd_0_cpu |
|
Following command correctly installs latest PyTorch-1.10 and TorchVision-0.11 RC: If |
|
For now, my workaround is to install torchvision from pip in this case. I also propose to delete torchvision 0.2.2 altogether form conda |
|
I think it should be good to have some CI testing for these widely-used libraries (as in pytorch/pytorch#62590), at least we would know that something stopped working (with hopefully some automatic publication to pytorch.org/installation_commands_testing.txt or something similar) |
|
we just added CI testing for these in pytorch/pytorch-integration-testing#21 yesterday |
|
It would be cool to have these test results auto-published online in some readable way. |
|
Workaround of despite the fact that pytorch was |
|
How do I pip install torchvision for a specific cuda version? pip install torchvision==0.10.1+cu111
# ERROR: Could not find a version that satisfies the requirement torchvision==0.10.1+cu111 (from versions: 0.1.6, 0.1.7, 0.1.8, 0.1.9, 0.2.0, 0.2.1, 0.2.2, 0.2.2.post2, 0.2.2.post3, 0.8.2, 0.9.0, 0.9.1, 0.10.0, 0.10.1, 0.11.0, 0.11.1)
ERROR: No matching distribution found for torchvision==0.10.1+cu111torchvision README has no examples about specifying version constraints or CUDA versions |
|
Also, PyPI page for torchvision doesn't mention torch 1.9.1 and 1.10.0: https://pypi.org/project/torchvision/ @fmassa @soumith, same for main torchvision README |
|
I found a solution: |
|
@vadimkantorov PyPI does not allow publish more than one version of the same version of the binary, nor has a binary larger than 1Gb, this is why CUDA-10.2 is still default there. And installation instructions for CUDA-11.3 are documented in https://pytorch.org/get-started/locally/ Also, please note that CUDA-11.1 is not supported for pytorch-1.10, torchvision-0.11.1 (i.e. binaries were published without any testing) |
|
That said, they're not documented in README of torchvision, and by consequence on PyPI :( |
|
It would also be good if the error of torchvision about incompatibility of torchvision.ops / CUDA directly directly pointed how to fix it and install the good compatible torchvision version corresponding to installed torch and cuda |
|
@malfet It seems when following the official installation instructions and adding $ conda create -n py-cu116 python pytorch torchvision pytorch-cuda=11.6 -c pytorch -c nvidia --dry-runLogLogLogLogGiven that you have self-assigned this issue, do you want to debug further, or can we close this? |
|
It would be super good to have these installation commands added to CI somehow. Having workable / tested installation commands for the common setup (pytorch / torchvision / torchaudio / opencv / ffmpeg / blender) is very needed, especially given different ffmpeg wrappers in torchvision and in torchaudio |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
🐛 Describe the bug
This seems to be somewhat serious to figure out and resolve by the next release or asap.
Here's how the bug manifests.
There's a very common scenario when someone installs
torchvisionandopencvwithin the samecondaenvironment.However, running the following command:
This results in the
torchvisionresolving as thecputorchvision packge fromconda-forge, thepytorchbeing resolved frompytorchchannel, and the worst part, old versions ofpytorchandpytorch-cpubeing installed.Tested with latest anaconda:
conda --versionresolves toconda 4.10.3.Removing
opencvin the above command results intorchvisionandpytorchcorrectly resolving to their CUDA binaries, andpytorch-cpunot being installed.cc: @malfet
Versions
The text was updated successfully, but these errors were encountered: