Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ignore brew installed packages #21677

Closed

Conversation

johnwparent
Copy link
Contributor

@johnwparent johnwparent commented Jul 2, 2024

Fixes setup bug where Pip attempts to install a downgraded Numpy (<2) on top of Brew's installed numpy@2 on MacOS, which results in:

[1:08:03 AM]    Attempting uninstall: numpy
[1:08:03 AM]      Found existing installation: numpy 2.0.0
[1:08:03 AM]  ERROR: Cannot uninstall numpy 2.0.0, RECORD file not found. Hint: The package was installed by brew.

To reproduce the above error, simply run: ./setup/mac/install_prereqs.sh.

This PR takes one viable approach by simply allowing Pip to disregard what Brew has already installed and install its own numpy version.
Currently I apply this change for all packages installed in the run of pip associated with the numpy install.
This can be updated to limit this call to only the numpy install.

I will note this is essentially the nuclear option here and could potentially cause slight increases in setup times if the overlap between Brew and pip is substantial.

Alternatively, we can use Pip's --no-build-isolation and virtual environments to build numpy in isolation from Brew's python similar to how Spack does things


This change is Reviewable

@jwnimmer-tri
Copy link
Collaborator

@drake-jenkins-bot mac-arm-sonoma-unprovisioned-clang-bazel-experimental-release please
@drake-jenkins-bot mac-arm-ventura-unprovisioned-clang-bazel-experimental-release please
@drake-jenkins-bot mac-arm-ventura-unprovisioned-clang-cmake-experimental-packaging please
@drake-jenkins-bot mac-arm-ventura-unprovisioned-clang-wheel-experimental-release please

@jwnimmer-tri
Copy link
Collaborator

If I'm reading this correctly, the outcome of this PR is that we replace the numpy 2.x that Homebrew installed, with the 1.x version instead? We're not installing a new copy into a separate location which shadows the original -- we're actually nuking the original from orbit. In other words, the only site-packages on the path is /opt/homebrew/lib/python3.12/site-packages and by default that has numpy symlinked into the Cellar, but instead we're replacing that with a 1.x copy? If yes, then that's too severe.

If the only tool we currently have is /opt/homebrew/bin/pip3.12, and it only ever puts packages into the same directory that homebrew uses, then there is really no way to get back to 1.x.

We either need to fix Drake to be compatible with 2.x (#21577, hopefully easy), or change Drake to use a venv (#8392, pretty hard).

@jwnimmer-tri
Copy link
Collaborator

In any case, the in-progress CI results shows that this results in broken packages even just in our CI farm, so this is a no-go.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants