Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update pytorch nightly installation to use cpu version of torchtext #3011

Merged
merged 2 commits into from
Mar 9, 2024

Conversation

agunapal
Copy link
Collaborator

@agunapal agunapal commented Mar 9, 2024

Description

The prescribed way to install torchtext nightly is using cpu nightly binaries, since there are no CUDA binaries for torchtext

Related pytorch issue

Current behavior:

INFO: pip is looking at multiple versions of torchtext to determine which version is compatible with other requirements. This could take a while.
Collecting torchtext
  Downloading https://download.pytorch.org/whl/nightly/torchtext-0.17.0.dev20240114%2Bcpu-cp39-cp39-linux_x86_64.whl (2.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 5.1 MB/s eta 0:00:00
  Downloading https://download.pytorch.org/whl/nightly/torchtext-0.17.0.dev20240113%2Bcpu-cp39-cp39-linux_x86_64.whl (2.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 5.1 MB/s eta 0:00:00
  Downloading https://download.pytorch.org/whl/nightly/torchtext-0.17.0.dev20240110%2Bcpu-cp39-cp39-linux_x86_64.whl (2.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 6.3 MB/s eta 0:00:00
  Downloading https://download.pytorch.org/whl/nightly/torchtext-0.17.0.dev20240109%2Bcpu-cp39-cp39-linux_x86_64.whl (2.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 5.1 MB/s eta 0:00:00
  Downloading https://download.pytorch.org/whl/nightly/torchtext-0.17.0.dev20240108%2Bcpu-cp39-cp39-linux_x86_64.whl (2.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 6.5 MB/s eta 0:00:00
  Downloading https://download.pytorch.org/whl/nightly/torchtext-0.17.0.dev20240107%2Bcpu-cp39-cp39-linux_x86_64.whl (2.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 5.0 MB/s eta 0:00:00
  Downloading https://download.pytorch.org/whl/nightly/torchtext-0.17.0.dev20240106%2Bcpu-cp39-cp39-linux_x86_64.whl (2.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 5.1 MB/s eta 0:00:00
INFO: pip is still looking at multiple versions of torchtext to determine which version is compatible with other requirements. This could take a while.
  Downloading https://download.pytorch.org/whl/nightly/torchtext-0.17.0.dev20240105%2Bcpu-cp39-cp39-linux_x86_64.whl (2.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 5.5 MB/s eta 0:00:00
  Downloading https://download.pytorch.org/whl/nightly/torchtext-0.16.0.dev20231010%2Bcpu-cp39-cp39-linux_x86_64.whl (2.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 5.3 MB/s eta 0:00:00
Collecting torchvision
  Downloading https://download.pytorch.org/whl/nightly/cu121/torchvision-0.18.0.dev20240307%2Bcu121-cp39-cp39-linux_x86_64.whl (7.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.1/7.1 MB 91.9 MB/s eta 0:00:00
Collecting torch
  Downloading https://download.pytorch.org/whl/nightly/cu121/torch-2.3.0.dev20240307%2Bcu121-cp39-cp39-linux_x86_64.whl (780.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 780.7/780.7 MB 2.1 MB/s eta 0:00:00
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.
Collecting torchvision
  Downloading https://download.pytorch.org/whl/nightly/cu121/torchvision-0.18.0.dev20240306%2Bcu121-cp39-cp39-linux_x86_64.whl (7.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.1/7.1 MB 118.8 MB/s eta 0:00:00
Collecting torch
  Downloading https://download.pytorch.org/whl/nightly/cu121/torch-2.3.0.dev20240306%2Bcu121-cp39-cp39-linux_x86_64.whl (780.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 780.3/780.3 MB 1.8 MB/s eta 0:00:00

After the fix

Looking in indexes: https://download.pytorch.org/whl/nightly/cpu
Collecting torchtext
  Using cached https://download.pytorch.org/whl/nightly/cpu/torchtext-0.17.0.dev20240308%2Bcpu-cp39-cp39-linux_x86_64.whl (2.0 MB)
Requirement already satisfied: tqdm in /home/ubuntu/anaconda3/envs/ts_test_nightly_1/lib/python3.9/site-packages (from torchtext) (4.65.0)
Requirement already satisfied: requests in /home/ubuntu/anaconda3/envs/ts_test_nightly_1/lib/python3.9/site-packages (from torchtext) (2.31.0)
Requirement already satisfied: torch==2.3.0.dev20240308 in /home/ubuntu/anaconda3/envs/ts_test_nightly_1/lib/python3.9/site-packages (from torchtext) (2.3.0.dev20240308+cu121)
Requirement already satisfied: numpy in /home/ubuntu/anaconda3/envs/ts_test_nightly_1/lib/python3.9/site-packages (from torchtext) (1.26.4)
Collecting torchdata==0.7.1.dev20240307 (from torchtext)
  Using cached https://download.pytorch.org/whl/nightly/cpu/torchdata-0.7.1.dev20240307%2Bcpu-cp39-cp39-linux_x86_64.whl (2.7 MB)
Requirement already satisfied: filelock in /home/ubuntu/anaconda3/envs/ts_test_nightly_1/lib/python3.9/site-packages (from torch==2.3.0.dev20240308->torchtext) (3.13.1)
Requirement already satisfied: typing-extensions>=4.8.0 in /home/ubuntu/anaconda3/envs/ts_test_nightly_1/lib/python3.9/site-packages (from torch==2.3.0.dev20240308->torchtext) (4.8.0)

Fixes #(issue)

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

Feature/Issue validation/testing

Please describe the Unit or Integration tests that you ran to verify your changes and relevant result summary. Provide instructions so it can be reproduced.
Please also list any relevant details for your test configuration.

  • Test A
    Logs for Test A

  • Test B
    Logs for Test B

Checklist:

  • Did you have fun?
  • Have you added tests that prove your fix is effective or that this feature works?
  • Has code been commented, particularly in hard-to-understand areas?
  • Have you made corresponding changes to the documentation?

@agunapal agunapal marked this pull request as ready for review March 9, 2024 02:27
@agunapal agunapal requested review from mreso and msaroufim March 9, 2024 02:27
@agunapal agunapal added this to the v0.10.0 milestone Mar 9, 2024
@agunapal agunapal added this pull request to the merge queue Mar 9, 2024
Merged via the queue into master with commit 616c1ad Mar 9, 2024
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants