Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Our CUDA tests don't run on CircleCI Windows #4006

Closed
NicolasHug opened this issue Jun 8, 2021 · 2 comments
Closed

Our CUDA tests don't run on CircleCI Windows #4006

NicolasHug opened this issue Jun 8, 2021 · 2 comments

Comments

@NicolasHug
Copy link
Member

NicolasHug commented Jun 8, 2021

The Windows GPU unittest workflows don't support cuda.

This can be seen in #4005 where the unittest_windows_gpu_py3.8 workflow indicates that torch.cuda.is_available() returns False: https://app.circleci.com/pipelines/github/pytorch/vision/8793/workflows/0d3c21ce-7657-4388-b637-7361f318c441/jobs/638628

SSH-ing into one of those machines shows that the driver is too old:

python -c "import torch; print(torch.cuda.is_available())"
False
C:\Users\circleci\project\env\lib\site-packages\torch\cuda\__init__.py:52: UserWarning: CUDA initialization: The NVIDIA driver on your system is too old (found version 10010). Please update your GPU driver by downloading and installing a new version from the URL: http://www.nvidia.com/Download/index.aspx Alternatively, go to: https://pytorch.org to install a PyTorch version that has been compiled with your version of the CUDA driver. (Triggered internally at  ..\c10\cuda\CUDAFunctions.cpp:115.)
  return torch._C._cuda_getDeviceCount() > 0

@fmassa, who to ping for this? I suspect that all the other domains CI will be affected (EDIT: yes), and possibly those of pytorch?

cc @seemethere

@fmassa
Copy link
Member

fmassa commented Jun 14, 2021

cc @malfet @seemethere could anyone in your team recommend the best way forward with re-enabling those Windows CUDA tests?

@malfet
Copy link
Contributor

malfet commented Jun 14, 2021

@fmassa, you simply need to install latest driver before running the tests, by calling something similar to
https://github.com/pytorch/pytorch/blob/master/.circleci/scripts/windows_cuda_install.sh
Since CUDA drivers are backward compatible it's OK to download say 11.3 and only install Display.Driver component of it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants