Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature request: retry after pip error due to network issue #1846

Open
njzjz opened this issue May 31, 2024 · 0 comments
Open

Feature request: retry after pip error due to network issue #1846

njzjz opened this issue May 31, 2024 · 0 comments

Comments

@njzjz
Copy link
Contributor

njzjz commented May 31, 2024

Description

Sometimes, we get occasional errors when using pip to download dependencies. It's good to have an option to retry pip instead of failing it.

Examples:

+ python -m pip wheel /project --wheel-dir=/tmp/cibuildwheel/built_wheel --no-deps -v
...
    Collecting tensorflow-cpu==2.14.1
      Downloading tensorflow_cpu-2.14.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (198.3 MB)
         ━━╸                                      13.6/198.3 MB 5.0 MB/s eta 0:00:38
    ERROR: Wheel 'tensorflow-cpu' located at /tmp/pip-unpack-i1ci1wt6/tensorflow_cpu-2.14.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl is invalid.

     Downloading tensorflow-2.16.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (218.9 MB)
       ━━━━━━━━━━╸                              59.8/218.9 MB 4.8 MB/s eta 0:00:34
     ERROR: THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.
        unknown package:
            Expected sha256 ae0554471d472b8095f8a5204d878389d0d4bc88f6ef6edcd477b952dff5cfab
                 Got        f816d4661c4e5fbb4ca0e5dffa416ddbd714ea89bdd036026174ccfb70eadf0b

Build log

https://github.com/deepmodeling/deepmd-kit/actions/runs/9308549047/job/25622191044

CI config

No response
@njzjz njzjz mentioned this issue Jun 19, 2024
Merged
github-merge-queue bot pushed a commit to deepmodeling/deepmd-kit that referenced this issue Jun 21, 2024
@njzjz @coderabbitai
chore(ci): workaround to retry error decoding response body from uv (…
0ded21a 
…#3889)

This PR uses a shell wrapper to check if the `error decoding response
body` error message is in the uv stderr and retry if so. It is just a
workaround for astral-sh/uv#2586 and
astral-sh/uv#3514 and hope the upstream can
fix it.

Note that this PR does nothing with cibuildwheel. It's unclear how to
retry with certain errors under its complex logic (feature requested in
pypa/cibuildwheel#1846).

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **Chores**
- Standardized installation process for TensorFlow, Torch, and other
dependencies across workflows by using `uv_with_retry.sh` script to
ensure reliable installations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
github-merge-queue bot pushed a commit to deepmodeling/deepmd-kit that referenced this issue Jun 21, 2024
@njzjz @coderabbitai
chore(ci): workaround to retry error decoding response body from uv (…
3a4bf95 
…#3889)

This PR uses a shell wrapper to check if the `error decoding response
body` error message is in the uv stderr and retry if so. It is just a
workaround for astral-sh/uv#2586 and
astral-sh/uv#3514 and hope the upstream can
fix it.

Note that this PR does nothing with cibuildwheel. It's unclear how to
retry with certain errors under its complex logic (feature requested in
pypa/cibuildwheel#1846).

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **Chores**
- Standardized installation process for TensorFlow, Torch, and other
dependencies across workflows by using `uv_with_retry.sh` script to
ensure reliable installations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
github-merge-queue bot pushed a commit to deepmodeling/deepmd-kit that referenced this issue Jun 21, 2024
@njzjz @coderabbitai
chore(ci): workaround to retry error decoding response body from uv (…
359d19b 
…#3889)

This PR uses a shell wrapper to check if the `error decoding response
body` error message is in the uv stderr and retry if so. It is just a
workaround for astral-sh/uv#2586 and
astral-sh/uv#3514 and hope the upstream can
fix it.

Note that this PR does nothing with cibuildwheel. It's unclear how to
retry with certain errors under its complex logic (feature requested in
pypa/cibuildwheel#1846).

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **Chores**
- Standardized installation process for TensorFlow, Torch, and other
dependencies across workflows by using `uv_with_retry.sh` script to
ensure reliable installations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
njzjz added a commit to njzjz/deepmd-kit that referenced this issue Jul 2, 2024
@njzjz @coderabbitai
chore(ci): workaround to retry error decoding response body from uv (…
bf36e26 
…deepmodeling#3889)

This PR uses a shell wrapper to check if the `error decoding response
body` error message is in the uv stderr and retry if so. It is just a
workaround for astral-sh/uv#2586 and
astral-sh/uv#3514 and hope the upstream can
fix it.

Note that this PR does nothing with cibuildwheel. It's unclear how to
retry with certain errors under its complex logic (feature requested in
pypa/cibuildwheel#1846).

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

- **Chores**
- Standardized installation process for TensorFlow, Torch, and other
dependencies across workflows by using `uv_with_retry.sh` script to
ensure reliable installations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
(cherry picked from commit 359d19b)
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
njzjz added a commit to deepmodeling/deepmd-kit that referenced this issue Jul 3, 2024
@njzjz @coderabbitai
chore(ci): workaround to retry error decoding response body from uv (…
1048015 
…#3889)

This PR uses a shell wrapper to check if the `error decoding response
body` error message is in the uv stderr and retry if so. It is just a
workaround for astral-sh/uv#2586 and
astral-sh/uv#3514 and hope the upstream can
fix it.

Note that this PR does nothing with cibuildwheel. It's unclear how to
retry with certain errors under its complex logic (feature requested in
pypa/cibuildwheel#1846).

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

- **Chores**
- Standardized installation process for TensorFlow, Torch, and other
dependencies across workflows by using `uv_with_retry.sh` script to
ensure reliable installations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
(cherry picked from commit 359d19b)
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
mtaillefumier pushed a commit to mtaillefumier/deepmd-kit that referenced this issue Sep 18, 2024
@njzjz @coderabbitai
chore(ci): workaround to retry error decoding response body from uv (…
70c5b14 
…deepmodeling#3889)

This PR uses a shell wrapper to check if the `error decoding response
body` error message is in the uv stderr and retry if so. It is just a
workaround for astral-sh/uv#2586 and
astral-sh/uv#3514 and hope the upstream can
fix it.

Note that this PR does nothing with cibuildwheel. It's unclear how to
retry with certain errors under its complex logic (feature requested in
pypa/cibuildwheel#1846).

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **Chores**
- Standardized installation process for TensorFlow, Torch, and other
dependencies across workflows by using `uv_with_retry.sh` script to
ensure reliable installations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@njzjz