python312Packages.wandb: 0.16.0 -> 0.18.5 #350098
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
GaetanLepage
force-pushed
the
wandb
branch
3 times, most recently
from
0be73c9
to
bf25bc4
Compare
|
nice, wandb update :) |
I am still battling with a few tests but the package seems to be working nicely overall. |
|
I got it to build on linux, but the rust binary fails to compile on darwin: |
GaetanLepage
force-pushed
the
wandb
branch
4 times, most recently
from
89f1b7c
to
b137136
Compare
|
I'm seeing the aarch64-darwin build fail for a different reason: @GaetanLepage is there something different about your setup? FWIW I'm happy to merge without darwin support since building on linux is an improvement over the current status quo. |
|
I am about to push the following which I think is better: index 3de94a2a99f1..676c46295095 100644
--- a/pkgs/development/python-modules/wandb/default.nix
+++ b/pkgs/development/python-modules/wandb/default.nix
@@ -162,22 +162,34 @@ buildPythonPackage rec {
})
];
+ env = {
+ # https://github.com/wandb/wandb/blob/v0.18.5/hatch_build.py#L37-L47
+
+ # Prevent the install script to try building and embedding the `gpu_stats` binary in the wheel.
+ # It will be patched within the `wandb-core` source code.
+ WANDB_BUILD_SKIP_GPU_STATS = true;
+
+ # Prevent the install script to try building and embedding the `wandb-core` binary in the wheel.
+ # It will be patched within the `wandb` source code.
+ WANDB_BUILD_UNIVERSAL = true;
+ };
+
# We manually compile those two dependencies and inject their path to bypass their build scripts
postPatch = ''
- substituteInPlace hatch_build.py \
- --replace-fail \
- 'output = pathlib.Path("wandb", "bin", "wandb-core")' \
- 'return ["${lib.getExe wandb-core}"]' \
- --replace-fail \
- 'output = pathlib.Path("wandb", "bin", "gpu_stats")' \
- 'return ["${lib.getExe gpu-stats}"]'
-
substituteInPlace wandb/util.py \
--replace-fail \
'bin_path = pathlib.Path(__file__).parent / "bin" / "wandb-core"' \
'bin_path = pathlib.Path("${lib.getExe wandb-core}")'
'';Indeed, currently I am still "embedding" the |
|
Also, I removed the |
GaetanLepage
force-pushed
the
wandb
branch
2 times, most recently
from
8325f3e
to
aa17670
Compare
|
|
Ok @samuela, this is good to go according to me. Please, have a look at it to spot eventual mistakes.
|
13 tasks
samuela
reviewed
Oct 22, 2024
There was a problem hiding this comment.
This is great -- thanks so much @GaetanLepage! I'm a little confused how __darwinAllowLocalNetworking is intended to be used, but o/w I think this looks good.
| pytestCheckHook | ||
| ]; | ||
|
|
||
| __darwinAllowLocalNetworking = true; |
There was a problem hiding this comment.
It looks like this is deprecated/renamed:
nixpkgs/pkgs/stdenv/generic/make-derivation.nix
Lines 130 to 137 in 7783a48
but I'm not able to find any docs pointing to why or what to use instead.
Perhaps @NixOS/darwin-maintainers might be able to help us?
There was a problem hiding this comment.
Apparently, it will just be removed from the derivation arguments and we can still use it without any problem.
Comment on lines
+264
to
+265
| # Expects python binary to be named `python3` but nix provides `python3.12` | ||
| # AssertionError: assert ['python3.12', 'main.py'] == ['python3', 'main.py'] |
There was a problem hiding this comment.
❯ ns -p python312
[nix-shell:~]$ which python3
/nix/store/ybnf7k6i9p244bbhsbxizqk65z58cwyr-python3-3.12.6/bin/python3
perhaps I'm missing something here?
There was a problem hiding this comment.
I know, this is weird. But I confirm that these tests fails:
_______________________ test_get_entrypoint[wandb_core] ________________________
[gw0] linux -- Python 3.12.6 /nix/store/wfbjq35kxs6x83c3ncpfxdyl5gbhdx4h-python3-3.12.6/bin/python3.12
@pytest.mark.skipif(
platform.system() == "Windows",
reason="python exec name is different on windows",
)
def test_get_entrypoint():
dir = tempfile.TemporaryDirectory().name
job_source = "artifact"
builder = _configure_job_builder_for_partial(dir, job_source)
metadata = {"python": "3.9.11", "codePathLocal": "main.py", "_partial": "v0"}
program_relpath = builder._get_program_relpath(job_source, metadata)
entrypoint = builder._get_entrypoint(program_relpath, metadata)
> assert entrypoint == ["python3", "main.py"]
E AssertionError: assert ['python3.12', 'main.py'] == ['python3', 'main.py']
E
E At index 0 diff: 'python3.12' != 'python3'
E
E Full diff:
E [
E - 'python3',
E + 'python3.12',
E ? +++
E 'main.py',
E ]
/build/source/tests/unit_tests/test_launch/test_create_job.py:126: AssertionError
drupol
approved these changes
Oct 23, 2024
wegank
added
the
12.approvals: 1
|
@samuela do you have any further concern ? |
|
I took the decision to merge it so we can move on, let's iterate on this later if @samuela has some other concerns. |
This was referenced Nov 1, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Things done
Changelog: https://github.com/wandb/wandb/raw/v0.18.5/CHANGELOG.md
cc @samuela
fixes #349691
nix.conf? (See Nix manual)sandbox = relaxedsandbox = truenix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage./result/bin/)Add a 👍 reaction to pull requests you find important.