Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

brew update followed by brew upgrade fails for Python's 2to3 conflict (December 2022) #577

Open
2 of 10 tasks
traversaro opened this issue Dec 20, 2022 · 49 comments · Fixed by casbin-rs/sqlx-adapter#87
Open
2 of 10 tasks

Comments

@traversaro
Copy link

traversaro commented Dec 20, 2022

Description

If one has on a job:

brew update
brew upgrade

the job fails with error:

 ==> Upgrading python@3.10
  3.10.8 -> 3.10.9 

==> Pouring python@3.10--3.10.9.big_sur.bottle.tar.gz
Error: The `brew link` step did not complete successfully
The formula built, but is not symlinked into /usr/local
Could not symlink bin/2to3
Target /usr/local/bin/2to3
already exists. You may want to remove it:
  rm '/usr/local/bin/2to3'

To force the link and overwrite all conflicting files:
  brew link --overwrite python@3.10

To list all files that would be deleted:
  brew link --overwrite --dry-run python@3.10

Possible conflicting files are:
/usr/local/bin/2to3 -> /Library/Frameworks/Python.framework/Versions/3.11/bin/2to3
/usr/local/bin/idle3 -> /Library/Frameworks/Python.framework/Versions/3.11/bin/idle3
/usr/local/bin/pydoc3 -> /Library/Frameworks/Python.framework/Versions/3.11/bin/pydoc3
/usr/local/bin/python3 -> /Library/Frameworks/Python.framework/Versions/3.11/bin/python3
/usr/local/bin/python3-config -> /Library/Frameworks/Python.framework/Versions/3.11/bin/python3-config

This is similar to old issues:

Platforms affected

  • Azure DevOps
  • GitHub Actions - Standard Runners
  • GitHub Actions - Larger Runners

Runner images affected

  • Ubuntu 18.04
  • Ubuntu 20.04
  • Ubuntu 22.04
  • macOS 11
  • macOS 12
  • Windows Server 2019
  • Windows Server 2022

Image version and build link

Image Version: 20221215.1
Link: https://github.com/traversaro/github-actions-brew-update-upgrade-check/actions/runs/3742142787

Is it regression?

20221211.1

Expected behavior

The brew upgrade should exit fine.

Actual behavior

The brew upgrade fails with the error message in the top of the issue.

Repro steps

See https://github.com/traversaro/github-actions-brew-update-upgrade-check/blob/2fa847c627c643716ba10d681ea18c86c0b54dec/.github/workflows/brew-update.yml for a MWE .

@traversaro
Copy link
Author

Possibly related to actions/runner-images#6459 .

@igorboskovic3
Copy link

Hi @traversaro , thank you, we will take a look.

@david-engelmann
Copy link

I'm observing this as well

@ryancurrah
Copy link

ryancurrah commented Dec 21, 2022

Also observed this here https://github.com/ryancurrah/lima-and-qemu/actions/runs/3745432855/jobs/6359863993 and here https://github.com/mook-as/lima-and-qemu/actions/runs/3743467832/jobs/6356414606.

EDIT:

I was able to get it working by removing the existing symlinks.

rm /usr/local/bin/2to3
rm /usr/local/bin/idle3
rm /usr/local/bin/pydoc3
rm /usr/local/bin/python3
rm /usr/local/bin/python3-config

@mikhailkoliada
Copy link

image

It looks like it the universal python's package used in actions/setup-python is installing its symlinks to /usr/local, while I think it should not.

@marko-zivic-93 @dsame @dmitry-shibanov could you please take a look?

@ni4
Copy link

ni4 commented Dec 22, 2022

Got the same issue, and find out that something similar we had previously. Now it looks like:

  # homebrew fails to update python 3.9.1 to 3.9.1.1 due to unlinking failure
  rm /usr/local/bin/2to3 || true
  # homebrew fails to update python from 3.9 to 3.10 due to another unlinking failure
  rm /usr/local/bin/idle3 || true
  rm /usr/local/bin/pydoc3 || true
  rm /usr/local/bin/python3 || true
  rm /usr/local/bin/python3-config || true

Flamefire referenced this issue in boostorg/boost-ci Dec 22, 2022
It may fail creating some symlinks which we don't care about. See e.g. https://github.com/actions/runner-images/issues/6817
@Flamefire
Copy link

It looks like it the universal python's package used in actions/setup-python is installing its symlinks to /usr/local, while I think it should not.

That is not the case for us and the OP:

/usr/local/bin/2to3 -> /Library/Frameworks/Python.framework/Versions/3.11/bin/2to3

There is seemingly a non-brew installed Python from somewhere else

@dsame
Copy link
Contributor

dsame commented Dec 22, 2022

@Flamefire the workflow does not use action/setup-python. It seems the links are created by the macos python package - i see them on the "almost" clean machine with the XCode installed.

@dsame
Copy link
Contributor

dsame commented Dec 22, 2022

@traversaro actully it is a normal behaviour. As you see from the log the installation continues normally with just a warning the links were not overwritten. The problem is brew upgrade exits with non-zero exit code and bash shell runs with set -e option. Thus the step signals the error condition despite brew upgrade is the very last command or not.

The easiest way to make the step to success is to add true after brew upgrade:

      - run: |
          brew update
          brew upgrade || true

@dsame
Copy link
Contributor

dsame commented Dec 23, 2022

Hello @fredroy, i see you forces the links by removing them manually in fact it is better to be done with

brew link --overwrite python@3.10

@maxkratz
Copy link

maxkratz commented Dec 23, 2022

I'm also observing this behavior (just with running brew update and macOS 12): https://github.com/eMoflon/emoflon-ibex-eclipse-build/actions/runs/3764385410/jobs/6398766536

@Flamefire
Copy link

@dsame

@Flamefire the workflow does not use action/setup-python. It seems the links are created by the macos python package - i see them on the "almost" clean machine with the XCode installed.

Yes this is what I also observed and wrote in #577

As you see from the log the installation continues normally with just a warning the links were not overwritten. The problem is brew upgrade exits with non-zero exit code

We also see this when we install a package with brew (e.g. ccache) which requires Python and hence is automatically upgraded. Ignoring the exit code is not feasible in that case as it might hint to anything besides this issue.

i see you forces the links by removing them manually in fact it is better to be done with

brew link --overwrite python@3.10

This is also not feasible for us as the exact Python version required for a package to be installed is not known/may change.

Hence our workaround is to remove the symlinks on failure and retry: https://github.com/boostorg/boost-ci/blob/ac5ae7e901e49351d19de278cbf82ce8d2c44216/ci/setup_ccache.sh#L17-L24

@maxkratz

I'm also observing this behavior (just with running brew update and macOS 12): https://github.com/eMoflon/emoflon-ibex-eclipse-build/actions/runs/3764385410/jobs/6398766536

No, I also made this false assumption. The behavior is not triggered by the update (which just downloads meta information similar to apt-get update) but the following installation of a package: brew install p7zip coreutils grep wget curl (or brew upgrade but you don't use that here)

@maxkratz
Copy link

@maxkratz

I'm also observing this behavior (just with running brew update and macOS 12): https://github.com/eMoflon/emoflon-ibex-eclipse-build/actions/runs/3764385410/jobs/6398766536

No, I also made this false assumption. The behavior is not triggered by the update (which just downloads meta information similar to apt-get update) but the following installation of a package: brew install p7zip coreutils grep wget curl (or brew upgrade but you don't use that here)

Thank you for the clarification, you are right.

lfos pushed a commit to lfos/calcurse that referenced this issue Jan 26, 2024
See actions/setup-python#577 for details.

Signed-off-by: Lukas Fleischer <lfleischer@calcurse.org>
lfos pushed a commit to lfos/calcurse that referenced this issue Jan 26, 2024
See actions/setup-python#577 for details.

Signed-off-by: Lukas Fleischer <lfleischer@calcurse.org>
lfos pushed a commit to lfos/calcurse that referenced this issue Jan 26, 2024
Follow-up to commit 98ff4db (Fix Mac OS GitHub workflow, 2023-04-11).

See actions/setup-python#577 for more details.

Signed-off-by: Lukas Fleischer <lfleischer@calcurse.org>
garrettsummerfi3ld added a commit to garrettsummerfi3ld/OpenSpace that referenced this issue Feb 1, 2024
koolkdev added a commit to koolkdev/wfslib that referenced this issue Feb 9, 2024
Add a better workaround for actions/setup-python#577
koolkdev added a commit to koolkdev/wfs-tools that referenced this issue Feb 9, 2024
jgillich added a commit to getseabird/seabird that referenced this issue Mar 3, 2024
symlink already exists error, see actions/setup-python#577
JCGoran added a commit to neuronsimulator/nrn-build-ci that referenced this issue Mar 10, 2024
When running the CI, we get errors like:

==> Pouring python@3.11--3.11.8.ventura.bottle.tar.gz
Error: The `brew link` step did not complete successfully
The formula built, but is not symlinked into /usr/local
Could not symlink bin/2to3-3.11
Target /usr/local/bin/2to3-3.11
already exists. You may want to remove it:
  rm '/usr/local/bin/2to3-3.11'

To force the link and overwrite all conflicting files:
  brew link --overwrite python@3.11

To list all files that would be deleted:
  brew link --overwrite python@3.11 --dry-run

Possible conflicting files are:
/usr/local/bin/2to3-3.11 ->
/Library/Frameworks/Python.framework/Versions/3.11/bin/2to3-3.11
/usr/local/bin/idle3.11 ->
/Library/Frameworks/Python.framework/Versions/3.11/bin/idle3.11
/usr/local/bin/pydoc3.11 ->
/Library/Frameworks/Python.framework/Versions/3.11/bin/pydoc3.11
/usr/local/bin/python3.11 ->
/Library/Frameworks/Python.framework/Versions/3.11/bin/python3.11
/usr/local/bin/python3.11-config ->
/Library/Frameworks/Python.framework/Versions/3.11/bin/python3.11-config

Hence, we use a fix described here:
actions/setup-python#577 (comment)
hendrikmuhs added a commit to KeyviDev/keyvi that referenced this issue Mar 11, 2024
hendrikmuhs added a commit to KeyviDev/keyvi that referenced this issue Mar 15, 2024
workaround inconsistency problems between github runner image and brew on macos

relates: actions/setup-python#577
sdecugis pushed a commit to freeDiameter/freeDiameter that referenced this issue Mar 30, 2024
KEINOS added a commit to KEINOS/Fork_kagome that referenced this issue Apr 13, 2024
Sometimes `brew upgrade` of Graphviz action fails on macOS runner while upgrading python packages.

This error is based on the below issue:
- actions/setup-python#577

Re-linking python will fix this problem:
- Ref: Homebrew/homebrew-core#165793 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.