Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tarball: packages.json is missing a couple thousand packages #102508

Closed
Atemu opened this issue Nov 2, 2020 · 3 comments
Closed

tarball: packages.json is missing a couple thousand packages #102508

Atemu opened this issue Nov 2, 2020 · 3 comments
Labels

Comments

@Atemu
Copy link
Member

Atemu commented Nov 2, 2020

Describe the bug
A clear and concise description of what the bug is.

The packages.json inside the tarball contains a list of packages in Nixpkgs.

However, this JSON is missing a bunch of packages only contained in sub-package-sets; most notably ~5000 emacsPackages, ~4000 texlive packages and a whole bunch of smaller package sets.
Also see: repology/repology-rules#396

Since, countrary to most other repositories, we're not a flat list but rather a nested set and also contain functions and other non-packages, generating a flat list of packages is a non-trivial task. We have a script for this task but it only recurses into haskellPackages and rPackages currently.

The new NixOS search have adapted this script and made it recurse into a few more sets which should be nearly all packages users might want to search for.
We should at least port this back to Nixpkgs.

As this JSON isn't used for our package search anymore but rather academic purposes like Repology, I think including all named packages available to the user in Nixpkgs might not be a bad idea.
So, I went at it the other way around and started with all attrsets at the top-level and stripped away all the things that aren't package sets, are a copy of the top-level set or can't be evaluated:

#102509

Notify maintainers

cc Marketing Team as the list in my local JSON went from 63341 to 71370 and 98640 packages long respectively and I'm sure such large numbers would look nice on Repology and the homepage. ;)

This also means that we're now larger than the AUR, possibly even larger than AUR + Arch repos combined. :)

@edolstra @garbas @samueldr

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute:
# a list of nixos modules affected by the problem
module:
@Atemu
Copy link
Member Author

Atemu commented Dec 24, 2020

Unstable has advanced aaand:

image

;)

@Atemu Atemu closed this as completed Dec 24, 2020
@jonringer
Copy link
Contributor

we should probably adjust the repology parser, as almost all of these new packages got added to unique packages.

@Atemu
Copy link
Member Author

Atemu commented Dec 26, 2020

Seems like it's the emacsPackages, the numbers match up (~5000).

repology/repology-rules#447

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants