Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

python3Packages.{jax,jaxlib}: update to 0.4.14 #246712

Merged
merged 6 commits into from
Aug 8, 2023
Merged

python3Packages.{jax,jaxlib}: update to 0.4.14 #246712

merged 6 commits into from
Aug 8, 2023

Conversation

NickCao
Copy link
Member

@NickCao NickCao commented Aug 2, 2023 •
edited
Loading

Description of changes

Continuation of #227145

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 23.11 Release Notes (or backporting 23.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

@ofborg ofborg bot requested review from GaetanLepage and samuela August 2, 2023 05:29
@NickCao NickCao changed the title [WIP] python3Packages.{jax,jaxlib}: update to 0.4.14 python3Packages.{jax,jaxlib}: update to 0.4.14 Aug 2, 2023
@NickCao NickCao marked this pull request as ready for review August 2, 2023 08:59
@ofborg ofborg bot requested a review from ndl August 2, 2023 09:19
@NickCao
Copy link
Member Author

NickCao commented Aug 2, 2023

@ofborg build python3Packages.jax python3Packages.jaxlib python3Packages.tensorflow python3Packages.tensorflow-datasets

@GaetanLepage GaetanLepage mentioned this pull request Aug 2, 2023
Merged
12 tasks
@NickCao
Copy link
Member Author

NickCao commented Aug 2, 2023

bazel deps hash fails to reproduce on aarch64-linux, also need someone to test darwin.

@samuela
Copy link
Member

samuela commented Aug 2, 2023

bazel deps hash fails to reproduce on aarch64-linux, also need someone to test darwin.

IIRC aarch64-linux is not a supported platform but i could be mistaken. The source builds have been broken on darwin for a minute, so we should only need to test the -bin builds, thankfully.

@NickCao
Copy link
Member Author

NickCao commented Aug 3, 2023

IIRC aarch64-linux is not a supported platform but i could be mistaken.

It did build on aarch64-linux: https://hydra.nixos.org/build/229355874

@NickCao
Copy link
Member Author

NickCao commented Aug 3, 2023

@ofborg build python3Packages.jax python3Packages.jaxlib python3Packages.tensorflow python3Packages.tensorflow-datasets python3Packages.numpyro

samuela
samuela reviewed Aug 3, 2023
View reviewed changes
pkgs/development/python-modules/jaxlib/default.nix
Comment on lines +262 to +267
sha256 = (if cudaSupport then {
x86_64-linux = "sha256-8QaXoZq6oITRsYn4RdLUXcKQv3PJ4Q3ItX9PkBwxGBI=";
} else {
x86_64-linux = "sha256-M/h5EZmyiV4QvzgKRjdz7V1LHENUJlc/ig1QAItnWVQ=";
aarch64-linux = "sha256-edkYcdlvOLNGRSanch1fGCZwq8SFn3TzcUNt1LhzG/E=";
}).${stdenv.system} or (throw "jaxlib: unsupported system: ${stdenv.system}");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@uri-canva previously went through the trouble to avoid this kerfuffle in #221390. why does that break now? can we adjust the fetchAttrs such that the hashes match?

Copy link
Member Author

@NickCao NickCao Aug 3, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe but I'm not really interested in going down that rabbit hole... The whole bazel fetching thing is already a nightmare.

@NickCao
Copy link
Member Author

NickCao commented Aug 3, 2023

A few (8 failed, 3559 passed, 578 skipped, 655 warnings) test failures for tensorflow-datasets on aarch64-linux:

ImportError: /nix/store/kh3hv0idxi0vw1n0a1mshrg7ak2flj3b-python3.10-array-record-0.4.0/lib/python3.10/site-packages/array_record/python/array_record_module.so: cannot open shared object file: No such file or directory

Unrelated to this PR: https://hydra.nixos.org/build/229991799

GaetanLepage
GaetanLepage approved these changes Aug 6, 2023
View reviewed changes
Copy link
Contributor

@GaetanLepage GaetanLepage left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @NickCao for taking the burden of this PR.
I am OK with the current changes !

@NickCao NickCao merged commit b98f6d9 into NixOS:master Aug 8, 2023
9 checks passed
@NickCao NickCao deleted the jax-rework branch August 8, 2023 07:58
@NickCao
Copy link
Member Author

NickCao commented