Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

python3Packages: get rid of uses of nose #326513

Open
emilazy opened this issue Jul 12, 2024 · 66 comments
Open

python3Packages: get rid of uses of nose #326513

emilazy opened this issue Jul 12, 2024 · 66 comments
Labels
3.skill: sprintable 5. scope: tracking Long-lived issue tracking long-term fixes or multiple sub-problems 6.topic: python

Comments

@emilazy
Copy link
Member

emilazy commented Jul 12, 2024 •
edited
Loading

nose is abandoned upstream and has caused us untold headaches recently (admittedly mostly unrelated to the code itself). We’ve stopped the bleeding by getting #325968 into staging thanks to the incredible work of @jchv, but it’s still broken on master and we ought to do some tidying up so we’re not dragging this around forever and perhaps even to unblock some stuff on master while the staging cycle progresses. Many packages don’t actually need nose any more, despite us including it in our dependency lists. Active upstreams should have already moved to pytest or something else. Some packages will have patches available; if we’re really feeling generous we could send patches to upstream ourselves. Others may be completely unmaintained packages that we could think about dropping.

Please make sure to migrate packages to pyproject = true; when working on them. Additionally, if the upstream is unmaintained, there are no significant reverse dependencies in‐tree or on Wheelodex, and it requires patching or there is no particular reason to imagine we’d want to keep it, consider just dropping the package. For the same reasons we want to get rid of nose, we should take this as an opportunity to reduce future maintenance burden and drop broken, unused, or long‐obsolete packages.

@dotlambda has already done some great work on this – see #325847, #325872, #325885, and #326164 – but I thought it’d be good to have a master list of all affected packages so we can tackle it collectively. If you open a PR for one of these, please edit this issue to link it beside the relevant package(s).

Still to do

Already done

This was generated from staging commit 83881cb with the following awful fish one‐liner: for pkg in (rg -l '\bnose3?\b' | rg -v '/(nose[23]?|cufflinks|enocean|jaconv|annoy|hdmedians|flaky|pycron|snapshottest|ssh-mitm|colour|influxdb)/|(python|cran)-(packages|aliases)\.nix' | sed 's|.*/\([^/]*\)/\([^/]*\)$|\1|'); rg "/$pkg"'[ \n{]' pkgs/top-level/python-packages.nix | awk '{print "python3Packages." $1}'; test $pipestatus[1] = 0 || rg "/$pkg"'[ \n{]' pkgs/top-level/all-packages.nix | awk '{print $1}'; test $pipestatus[1] = 0 || echo "$pkg"; end | sort -u | sed 's/^/* [ ] `/; s/$/`/'; feel free to edit to update the list.
@emilazy emilazy mentioned this issue Jul 12, 2024
Closed
@AndrewKvalheim AndrewKvalheim mentioned this issue Jul 12, 2024
Merged
13 tasks
@lavafroth
Copy link
Contributor

python3Packages.biopandas has dropped pynose in v0.5.1.dev1 (not yet released). We should be able to check that off once the next version releases.

lavafroth added a commit to lavafroth/nixpkgs that referenced this issue Jul 13, 2024
@lavafroth
python3Packages.gocardless-pro: remove nose dependency
324c02a 
Addresses NixOS#326513
Dependency `nose` was removed in favor of pytest in version 1.46.2.
@lavafroth lavafroth mentioned this issue Jul 13, 2024
Merged
13 tasks
AndrewKvalheim added a commit to AndrewKvalheim/nixpkgs that referenced this issue Jul 13, 2024
@AndrewKvalheim
python3Packages.expiringdict: remove nose
fb0f321 
Addresses NixOS#326513 by applying mailgun/expiringdict#49
lavafroth added a commit to lavafroth/nixpkgs that referenced this issue Jul 14, 2024
@lavafroth
python3Packages.gocardless-pro: remove nose dependency
c757e22 
Addresses NixOS#326513
Dependency `nose` was removed in favor of pytest in version 1.46.2.
@cyrinux
Copy link

cyrinux commented Jul 14, 2024

Hi, "paperwork" package also depend on nose and raise error actually.

@pyrox0
Copy link
Member

pyrox0 commented Jul 14, 2024 •
edited
Loading

Seems like paperwork depends on pypillowfight, which has a build error due to nose not being supported. I'm investigating now!

Edit: it also depends on fabulous, which has some more failing tests.

@pyrox0 pyrox0 mentioned this issue Jul 14, 2024
Merged
13 tasks
@pyrox0 pyrox0 added the 5. scope: tracking Long-lived issue tracking long-term fixes or multiple sub-problems label Jul 14, 2024
@pyrox0 pyrox0 mentioned this issue Jul 14, 2024
Merged
13 tasks
@pyrox0
Copy link
Member

pyrox0 commented Jul 15, 2024

ledger-autosync can stay broken for now, per egh/ledger-autosync#143, upstream is working on getting a new release out, which should include all fixes for distutils, nose -> pytest, etc. I could also just update to the latest commit in the meantime if that's what's wanted.

This was referenced Jul 15, 2024
Merged
Merged
lavafroth added a commit to lavafroth/nixpkgs that referenced this issue Jul 15, 2024
@lavafroth
python3Packages.pastescript: remove nose dependency
5a001db 
Addresses NixOS#326513.

Dependency nose was removed in version 3.3.0 (2023-01-03).
Replaced `format` parameter with `pyproject = true` and `build-system`
set to `setuptools`
@lavafroth lavafroth mentioned this issue Jul 15, 2024
Merged
13 tasks
lavafroth added a commit to lavafroth/nixpkgs that referenced this issue Jul 15, 2024
@lavafroth
python3Packages.pastescript: remove nose dependency
e426922 
Addresses NixOS#326513.

Dependency nose was removed in version 3.3.0 (2023-01-03).
Replaced `format` parameter with `pyproject = true` and `build-system`
set to `setuptools`
d-xo pushed a commit to d-xo/nixpkgs that referenced this issue Jul 15, 2024
@AndrewKvalheim @d-xo
python3Packages.expiringdict: remove nose
f94b278 
Addresses NixOS#326513 by applying mailgun/expiringdict#49
d-xo pushed a commit to d-xo/nixpkgs that referenced this issue Jul 15, 2024
@lavafroth @d-xo
python3Packages.gocardless-pro: remove nose dependency
18bf745 
Addresses NixOS#326513
Dependency `nose` was removed in favor of pytest in version 1.46.2.
@mindstorms6 mindstorms6 mentioned this issue Jul 15, 2024
Closed
13 tasks
shahinism pushed a commit to shahinism/nixpkgs that referenced this issue Jul 15, 2024
@AndrewKvalheim @shahinism
python3Packages.expiringdict: remove nose
c0dca70 
Addresses NixOS#326513 by applying mailgun/expiringdict#49
shahinism pushed a commit to shahinism/nixpkgs that referenced this issue Jul 15, 2024
@lavafroth @shahinism
python3Packages.gocardless-pro: remove nose dependency
b65c049 
Addresses NixOS#326513
Dependency `nose` was removed in favor of pytest in version 1.46.2.
@pyrox0
Copy link
Member

pyrox0 commented Jul 15, 2024

sasview, while listed here, does not depend on nose.

@lavafroth
Copy link
Contributor

@pyrox0 sasview has an xhtml2pdf.nix besides the default.nix which depends on nose.

@lavafroth lavafroth mentioned this issue Jul 16, 2024
Merged
13 tasks
This was referenced Aug 13, 2024
Open
Merged
@mindstorms6 mindstorms6 mentioned this issue Aug 15, 2024
Merged
13 tasks
@mmorley0395
Copy link

new to nix with much to learn- but quick question. will these be backported into 24.05 when complete? or better to point a flake to master or unstable for packages that are fixed? thanks!

@emilazy
Copy link
Member Author

emilazy commented Aug 15, 2024

24.05 uses Python 3.11 by default, so the packages here should mostly not be an issue. If you’re using Python 3.12, I would recommend using the unstable release until 24.11 is released.

(Also, most of these packages do already work with 3.12 on 24.11 as we have patched nose ourselves; we’re just trying to get them cleaned up.)

@mmorley0395
Copy link

Interesting-- I was using nix-shell and for some reason my nix-channel was set to unstable so it was pulling from there and therefore using 3.12 i guess. I don't need 3.12 specifically, so updated the channel and all is well now. Thanks for taking the time to explain!

@natsukium natsukium mentioned this issue Aug 17, 2024
Open
13 tasks
@Sigmanificient
Copy link
Member

Sigmanificient commented Aug 20, 2024 •
edited
Loading

Up to date list

131 items: 82.44% done, 10.69% in progress, 11 waiting

python2Packages

  • more-itertools

python3Packages

This was referenced Aug 20, 2024
Open
Merged
Merged
Merged
@Sigmanificient
Copy link
Member

umcompyle6 has doCheck = pythonOlder "3.9"; but when using nix-build -A python312.umcompyle6 tests are still being run, am I missing something @emilazy?

This was referenced Aug 21, 2024
Merged
Merged
Merged
Merged
@Sigmanificient
Copy link
Member

@emilazy should we drop more-itertools?

This was referenced Aug 21, 2024
Merged
Merged
Merged
@natsukium natsukium mentioned this issue Aug 21, 2024
Open
13 tasks
@SuperSandro2000
Copy link
Member

SuperSandro2000 commented Aug 22, 2024 •
edited
Loading

more-itertools

We can't, it is to widely used. Or do you mean the python2Package? Not sure what is even using that at this point.

@Sigmanificient
Copy link
Member

more-itertools

We can't, it is to widely used. Or do you mean the python2Package? Not sure what is even using that at this point.

yup the python2Package

@SuperSandro2000
Copy link
Member

Try removing it and see what breaks. I would be in support to drop eg https://github.com/NixOS/nixpkgs/blob/21576aba43e604a95cc45c25248f39f3989ebe36/pkgs/servers/neard/default.nix

but we need to keep gimp intact

nixpkgs/pkgs/applications/graphics/gimp/default.nix

Line 137 in 21576ab

python2.pkgs.pygtk

I didn't find anything else uses python2 packages on the fast but I didn't search for all variants.

@emilazy
Copy link
Member Author

emilazy commented Aug 22, 2024

Ignore Python 2; it is even more obsolete than nose. GIMP will be solved soon and I have plans for the rest of Python 2.

@emilazy
Copy link
Member Author

emilazy commented Sep 22, 2024

Sorry for my long absence from working on this. I’ve cleaned up the list in the first comment to be easier to follow now that most users have been dealt with, and left a few reviews. I hope we can get this done before 24.11!

@pyrox0 pyrox0 mentioned this issue Sep 22, 2024
Merged
13 tasks
@emilazy
Copy link
Member Author

emilazy commented Sep 22, 2024

PRs of my own in flight:

I’m looking at python3Packages.mhcflurry and python3Packages.uncompyle6, which are now almost the only remaining packages without an open PR.

@emilazy
Copy link
Member Author

emilazy commented Sep 22, 2024

Turns out I was wrong about Python 2.7 packages not mattering… #343822

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
3.skill: sprintable 5. scope: tracking Long-lived issue tracking long-term fixes or multiple sub-problems 6.topic: python
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
10 participants
@trofi @cyrinux @eclairevoyant @SuperSandro2000 @emilazy @pyrox0 @nixos-discourse @Sigmanificient @mmorley0395 @lavafroth