python3Packages.jaxlib-build: share fetch derivation between different build derivations #221390
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@ofborg build python3Packages.jaxlib-build |
12 tasks
samuela
added a commit
to samuela/nixpkgs
that referenced
this pull request
Mar 30, 2023
Based on code from NixOS#221390 by @uri-canva.
|
Found it: it uses go_sdk transitively, which we have to remove like we've already done in envoy and bazel watcher. This one's a bit more tricky because it's not a direct dependency but a transitive one. |
|
Opened an issue to discuss what we can do in general for these kind of issues: #224446. |
|
Got it down to one last issue: On darwin: On linux: Ignore the different group and slightly different display of |
|
Do you have a sense of why it's failing in this PR, but not when I tested it in #219778? AFAIU I used the same code as you, although I did have to update the hash value. |
|
It's not? I can't find ofborg builds of the derivation on the different systems, it looks like it failed the eval? |
AFAICT it only failed eval for jaxlib-bin (#219778 (comment)). The source builds worked for me with |
|
Oh right sorry, the part of sharing it between different configuration works, it's the part of sharing it between linux and darwin that isn't working. Unfortunately since cuda doesn't work on darwin we can't have a single hash, we need either to split it by os, linux(cuda) / macos(non-cuda), or by whether cuda support is enabled. At least that's where I'm at now. I'm rebasing the PR shortly. |
|
Pulled out part of the fix here: #224917 |
Ah gotcha, yeah unfortunately Darwin support is broken for the jaxlib source build atm. aarch64-darwin is not working and I don't think any of us have x86_64-darwin machines to test that build anymore :( |
uri-canva
force-pushed
the
uri/jax
branch
2 times, most recently
from
9cb1430
to
072eacd
Compare
|
@ofborg build python3Packages.jaxlib-build |
ofborg
bot
added
10.rebuild-darwin: 11-100
10.rebuild-linux: 11-100
and removed
10.rebuild-darwin: 101-500
10.rebuild-linux: 501+
10.rebuild-linux: 501-1000
labels
uri-canva
force-pushed
the
uri/jax
branch
2 times, most recently
from
b79cb82
to
2bb182b
Compare
|
@ofborg build python3Packages.jaxlib-build |
uri-canva
commented
Apr 7, 2023
uri-canva
commented
Apr 7, 2023
| }.${stdenv.system} or (throw "unsupported system ${stdenv.system}"); | ||
| "sha256:0pzy18i2pj2hf5rav9x35r44c95ww64cmzgcd0g0ljdhhk5vhaz3" | ||
| else | ||
| "sha256-CyRfPfJc600M7VzR3/SQX/EAyeaXRJwDQWot5h2XnFU="; |
There was a problem hiding this comment.
This hash matches all the systems on ofborg. Success!
samuela
reviewed
Apr 7, 2023
| "--config=avx_posix" | ||
| ] ++ lib.optionals cudaSupport [ | ||
| # ideally we'd add this unconditionally too, but it doesn't work on darwin | ||
| # we make this conditional instead of the system, so that the hash for both the cuda and the non-cuda deps |
There was a problem hiding this comment.
Suggested change
| # we make this conditional instead of the system, so that the hash for both the cuda and the non-cuda deps | |
| # we make this conditional on `cudaSupport` instead of the system, so that the hash for both the cuda and the non-cuda deps |
…t build derivations
|
Result of 18 packages marked as broken and skipped:
25 packages failed to build:
41 packages built:
|
|
Errors appear to be unrelated: |
|
shockingly, python3Packages.jax is now building fine..... i expected it to fail due to #221739 |
|
Thanks for putting this together @uri-canva ! This change will def improve the DX when working with the jaxlib derivation |
12 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of changes
Things done
sandbox = trueset innix.conf? (See Nix manual)nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage./result/bin/)