[RFC 0081] Show unmaintained packages #81

Lassulus · 2020-12-10T22:10:49Z

ryantm · 2020-12-10T23:24:07Z

The "fancy website" basically already exists.

Lassulus · 2020-12-11T08:53:17Z

The "fancy website" basically already exists.

this is indeed quite cool. But I would still prefer a integration in nix/nixpkgs/nixos.

On the repology page I can't see more than 200 packages at once (so creating a complete list seems not possible?). And I'm not entirely sure which packages I really use.

garbas · 2020-12-14T12:04:57Z

The "fancy website" basically already exists.

this is indeed quite cool. But I would still prefer a integration in nix/nixpkgs/nixos.

On the repology page I can't see more than 200 packages at once (so creating a complete list seems not possible?). And I'm not entirely sure which packages I really use.

This information is already in elasticsearch backend of search.nixos.org. We just need to find the right way to show it.

Lassulus · 2020-12-14T17:56:26Z

This information is already in elasticsearch backend of search.nixos.org. We just need to find the right way to show it.

This also sounds lika a good idea, but would be orthogonal to the "feature" proposed in this RFC.

garbas · 2020-12-16T12:06:55Z

When evaluating a system (for example with nixos-rebuild) unmaintained packages should show up, so people are encouraged to maintain them.

Wouldn't this be too verbose? I think that exposing unmaintained package is going to help people maintaining them, but displaying them would mostly annoy users.

Maybe a middle ground would be to show a summary. eg. Your system is using X unmaintained packaged. We're looking for volunteers to maintain them. Follow this link https://....

But that might also rub new users the wrong way: Oh, "half" of NixOS is unmaintained. I wonder what would be a good way to do this.

Ma27 · 2020-12-16T12:28:04Z

But that might also rub new users the wrong way: Oh, "half" of NixOS is unmaintained. I wonder what would be a good way to do this.

Just a random, probably not so well-thought, idea during lunch: what about something like nix why-depends, but for maintainers? This could e.g. be embedded into nixos-rebuild, so that interested people can see where help is needed when they run this dedicated command.

Lassulus · 2020-12-16T12:38:15Z

Wouldn't this be too verbose? I think that exposing unmaintained package is going to help people maintaining them, but displaying them would mostly annoy users.

Maybe a middle ground would be to show a summary. eg. Your system is using X unmaintained packaged. We're looking for volunteers to maintain them. Follow this link https://....

The important part would be to have a list of which packages I use are unmaintained. If I get a number and a list of not maintained packages I don't know how I'm afffected.

I would be happy to know if a package I use is unmaintained. I want to have an option to disable this, so if people get annoyed by it, could disable it with this option. We could of course also make this opt-in instead of opt-out. But I guess most people don't know how many packages in their system are maintained and will never find out if not shown to them.

Yes, showing people, that "half" of NixOS is unmaintained is exactly the purpose of this. Hiding it would not be a good solution IMHO :D

FRidh · 2020-12-16T13:04:19Z

Just a random, probably not so well-thought, idea during lunch: what about something like nix why-depends, but for maintainers? This could e.g. be embedded into nixos-rebuild, so that interested people can see where help is needed when they run this dedicated command.

With flakes it becomes more straightforward to do because you know of a derivation what inputs are used. Right now you could do, in case of your system, nix show-derivation -r $(readlink -f /run/current-system) to get the build-time closure of your system, however, maintainers are not part of the derivation and show-derivation shows a path, not an attribute, so you need to match paths with an evaluated Nixpkgs.

Anyway, users should not be bothered with these kinds of internal things; a user just wants to use the software. Whenever you use free and open source software you should be aware that things may not be maintained or can be unmaintained from one moment to the next, and that you're on your own. If you feel you only want to use software that is maintained, then use an evaluation test.

Lassulus · 2020-12-16T18:19:32Z

Anyway, users should not be bothered with these kinds of internal things; a user just wants to use the software. Whenever you use free and open source software you should be aware that things may not be maintained or can be unmaintained from one moment to the next, and that you're on your own. If you feel you only want to use software that is maintained, then use an evaluation test.

If users are to be expected, that there is unmaintained software, why not show it to them?
I think a warning like unmaintained packages used: bzip2, curl, firefox, .... would not be very annoying. But maybe we can disable the warning on stable systems by default. Fresh users probably use a stable channel anyway.

I also think a warning like this would encourage users to adopt unmaintained packages (at least I would). Maybe we can link to some introductory material of how to become a package maintainer or something along those lines.

Zimmi48 · 2020-12-16T19:13:55Z

It is my impression that the meta.maintainers field does not really reflect any "reality". What does it mean for a package to be maintained? In this RFC, it seems that you define it as "has a maintainer". But if the maintainer hasn't been active (hasn't updated or reviewed updates) recently, then is the package still actually maintained? And if there is no listed maintainer, does it really mean that the package is not maintained?

IMHO this RFC is incomplete as is and should consider the policy to implement on having this field reflect the "reality".

Lassulus · 2020-12-16T19:26:29Z

It is my impression that the meta.maintainers field does not really reflect any "reality". What does it mean for a package to be maintained? In this RFC, it seems that you define it as "has a maintainer". But if the maintainer hasn't been active (hasn't updated or reviewed updates) recently, then is the package still actually maintained? And if there is no listed maintainer, does it really mean that the package is not maintained?

Yes, I suggest here, that a package is maintained when is has an entry in the maintainers field. I don't want to talk about how to update this field here by other tools or manually (that would be another RFC IMHO). I believe the reality and the maintainers field will align when the effects of having people there or not are better visible, like with a warning during evaluation.

Your questions are good questions, but I don't want to answer them in this RFC, because I fear we will go down a rabbit hole from which we will not come back.

Zimmi48 · 2020-12-16T19:31:14Z

I don't want to talk about how to update this field here by other tools or manually (that would be another RFC IMHO). I believe the reality and the maintainers field will align when the effects of having people there or not are better visible, like with a warning during evaluation.

I'm fine with this being a separate RFC, but I don't share your belief of a magical alignment once this field becomes more visible. In particular, your proposal might make more maintained packages get an official maintainer, but it definitely won't help clearing the inactive maintainers that are currently listed.

Your questions are good questions, but I don't want to answer them in this RFC, because I fear we will go down a rabbit hole from which we will not come back.

I share your concern, but I also think that addressing those questions (possibly as part of another RFC) is a prerequisite for this one.

Lassulus · 2020-12-17T18:29:07Z

I share your concern, but I also think that addressing those questions (possibly as part of another RFC) is a prerequisite for this one.

I don't think they are so entangled. This RFC is about transparency and showing our current status to the users/maintainers/interested parties. The current state is the maintainers field. Letting maintainers decay is another topic which is also important but not required for showing this information.

also:

I nominiate @kloenk as a shepherd
Does anyone else want to participate? please speak up :)

garbas · 2020-12-17T22:31:26Z

I'd like to nominate myself for shepherding.

infinisil · 2020-12-17T23:39:08Z

Proof-of-concept NixOS module by searching for unmaintained packages in environment.systemPackages (so no transitive dependencies, which is probably for the better). Uses NixOS/nixpkgs#97023

{ config, ... }: {
_module.checks.unmaintainedPackages =
  let unmaintained = lib.filter (v: v ? pname && v ? meta.maintainers && v.meta.maintainers == []) config.environment.systemPackages;
  in {
    check = unmaintained == [];
    type = "warning";
    message = "These packages you're using don't have a maintainer: ${lib.concatMapStrings (v: "\n- ${v.pname}") unmaintained}";
  };
}

Since some maintainer-less packages are in the default environment.systemPackages, with an empty config this shows as

❯ nix-instantiate '<nixpkgs/nixos>' --arg configuration '{ fileSystems."/".device = "/"; boot.loader.grub.device = "/"; }' -A system
trace: warning: [unmaintainedPackages] These packages you're using don't have a maintainer: 
- bzip2
- gnused
- gnutar

Can be disabled by the user with

{
  _module.checks.unmaintainedPackages.enable = false;
}

spacekookie · 2021-01-07T14:12:50Z

@Zimmi48 would you want to Shepherd this RFC?

Zimmi48 · 2021-01-20T15:06:20Z

Sorry, I don't have enough available time at the moment to be part of a shepherd team.

Mic92 · 2021-01-21T14:16:44Z

@Ma27 @mweinelt @infinisil Does any of you want to be shepherd on this RFC?

Ma27 · 2021-01-21T14:55:55Z

Unfortunately not, but this is only due to a time limitation on my end.

infinisil · 2021-01-21T15:10:47Z

I can be shepherd

Mic92 · 2021-01-21T15:21:26Z

Ok. We have enough shepherds than: @infinisil @garbas @kloenk

Mic92 · 2021-02-18T14:20:14Z

Can we have another meeting for this one?

kloenk · 2021-02-18T15:07:50Z

Best for me would be after the fifth of march. As then School is much less for me.

Lassulus · 2021-03-03T20:15:47Z

lets plan another date: https://when2meet.com/?11230345-jwDak

Lassulus · 2021-03-09T08:41:44Z

alright, so we will meet 2021-03-14 19:00 UTC (which is 20:00 in Europe/Berlin timezone). I guess we will do it on jitsi again.

Lassulus · 2021-03-14T16:08:19Z

meeting link: https://jitsi.lassul.us/rfc81

Mic92 · 2021-03-14T19:59:20Z

RFC 81 Meeting #2 Notes

Opt-in or not

Under the assumption that this feature is expensive, we should make it opt-in by default.

In case this feature can be implemented more efficiently in the future, we can change this to opt-out.

Say max 1% eval time increase if opt-out.

An opt-in prototype could be implemented and included into nixpkgs without this RFC beeing done/accepted

Limitations of current prototype

The current prototype is incapabale of finding packages which are "hidden" inside a string. example "${pkgs.firefox}/bin/firefox"

However it can find all buildInputs, and co. which should be good enough as a start.

Interface proposals

Opt-in

nixos-rebuild --show-unmaintained

Additionally we could inform the user about this flag at the end of a normal nixos-rebuild, like:

Did you know:
You can pass --show-unmaintained to nixos-rebuild 
to show unmaintained packages used in your system

This could be one of many "Did you know"s

Opt-out

At the end of nixos-rebuild, display a count of unmaintaned packages like

Your system depends on 5 unmaintained packages
Run `nixos-rebuild rebuild --show-unmaintained` to see the full list

Implement it by propagated this info in stdenv

stdenv can fill out a $out/nix-support/unmaintained which contains all the buildInputs and co. that don't have a maintainer, plus the $out/nix-support/unmaintained of them (making it recursive). This could be very efficient.

Future work: CI checks for maintainership

Since the goal of this RFC is to have fewer unmaintained packages, in the future we could have a CI check that ensures all packages have a maintainer. Packages would then have to be removed when the last maintained removes themselves.

We can have the next meeting when we have a nixos-rebuild prototype as a PR.

infinisil · 2021-03-19T23:07:31Z

I discussed and worked a bit on this with @garbas today. Some points that were brought up:

If we can have it be opt-out, we should only do so for people not using the stable channel. Reason being that stable could be outdated by many months, giving outdated information on which packages are maintained. In addition, people on unstable are more likely to step up to maintain packages.
We should maybe show the date/commit of the maintainer state that is used in the output, so we can know how up-to-date it is
With the search.nixos.org overhaul, backed by elasticsearch, it's now possible to do a single curl query to get up-to-date information on which packages don't have any maintainers, assuming you have an attribute path to search for

@garbas then created a simple example of how such a search.nixos.org query can be done. With a query.json file with contents

{
  "from": 0,
  "size": 10000,
  "sort": [
    {
      "_score": "desc",
      "package_attr_name": "desc",
      "package_pversion": "desc"
    }
  ],
  "query": {
    "bool": {
      "filter": [
        {
          "term": {
            "type": {
              "value": "package",
              "_name": "filter_packages"
            }
          }
        }
      ],
      "must": [
        {
          "terms": {
            "package_attr_name": [
              "hello",
              "haskellPackages.hello",
              "haskellPackages.gtk2hs-hello"
            ]
          }
        },
        {
          "term": {
            "package_maintainers_set": {
              "value": "No maintainers"
            }
          }
        }
      ]
    }
  }
}

You can do a query for the given attribute paths with the following (the authorization key is public, don't worry)

curl 'https://nixos-search-5886075189.us-east-1.bonsaisearch.net/latest-19-20.09/_search' \
  -H 'Authorization: Basic ejNaRko2eTJtUjpkczhDRXZBTFBmOXB1aTdYRw==' \
  -H 'Content-Type: application/json' \
  -H 'Pragma: no-cache' \
  -H 'Cache-Control: no-cache' \
  --data @query.json

Meanwhile, I worked on a prototype for the idea of having an $out/nix-support/unmaintained be created by stdenv. However I implemented it a bit differently: To have a $out/nix-support/attribute-deps which contains a list of attribute paths that the derivation depends on (recursive build time deps, not runtime deps, for now). I implemented this with the commits 857efbc2 and d33a27b2. It's a bit messy with the whole splicing part, but it basically works (requires stdenv rebuild of course):

$ nix-build -A mbuffer
/nix/store/n3kx0839798la0k8sakvzwravb8grhsi-mbuffer-20200929
$ cat result/nix-support/attribute-deps | sort -u
binutils
pkgsHostTarget.openssl
pkgsHostTarget.perl

Once you have this information, it's very easy to query maintainer information for those packages. Doing it this way is very flexible, as you can use this list of attribute paths for not only maintainer information, but also e.g. for licenses or other things. It also allows integration with the above search.nixos.org query, therefore getting more up-to-date information than the user might have in their nixpkgs version.

However, the code is also easily adjustable to query maintainer information in the same evaluation.

infinisil · 2021-03-19T23:40:48Z

A tiny problem with this approach is that attribute changes can now influence derivations. So e.g. renaming pkgs.foo to pkgs.bar requires rebuilding all reverse-dependencies. Not sure if that's actually a problem, but something to keep in mind.

Here's another example:

$ nix-build -A python3
/nix/store/vipj0kvgnwg22gypla36lhj63ajcck63-python3-3.8.8
$ cat result/nix-support/attribute-deps | sort -u
binutils
gcc
pkgs.gettext
pkgsHostTarget.autoconf
pkgsHostTarget.autoreconfHook
pkgsHostTarget.bzip2
pkgsHostTarget.expat
pkgsHostTarget.gdbm
pkgsHostTarget.gettext
pkgsHostTarget.help2man
pkgsHostTarget.libffi
pkgsHostTarget.lzma
pkgsHostTarget.m4
pkgsHostTarget.ncurses
pkgsHostTarget.nukeReferences
pkgsHostTarget.openssl
pkgsHostTarget.perl
pkgsHostTarget.perlPackages.LocaleGettext
pkgsHostTarget.perlPackages.perl
pkgsHostTarget.pkg-config
pkgsHostTarget.readline
pkgsHostTarget.sqlite
pkgsHostTarget.xz
pkgsHostTarget.zlib
pkgs.perl532

7c6f434c · 2021-03-20T09:13:57Z

A tiny problem with this approach is that attribute changes can now influence derivations. So e.g. renaming `pkgs.foo` to `pkgs.bar` requires rebuilding all reverse-dependencies. Not sure if that's actually a problem, but something to keep in mind.

Does that mean that `linux_latest` and the corresponding versioned attribute would become different attributes? Same for LibreOffice picked by default and explicitly specified as Fresh/Still? That does sound undesirable.

infinisil · 2021-03-21T03:31:16Z

Oh yeah you're right, and that seems like a dealbreaker:

$ nix-instantiate --eval -A linux_latest.__path
[ "linux_latest" ]
$ nix-instantiate --eval -A linux_5_11.__path
[ "linux_5_11" ]

7c6f434c · 2021-03-21T08:55:08Z

Oh yeah you're right, and that seems like a dealbreaker:

Does that code need to be applied globally? Because if it can simply be enabled for a wrapper (so that user can apply it whenever) and maybe default-on for system instances, the rebuilds incurred are dirt cheap, and the user can easily have this information over the used environments (which is presumably the most interesting case, why go package-by-package when you have a summary for your system and your user profile and your project shell).

nixos-discourse · 2021-04-07T15:30:47Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/tweag-nix-dev-update-9/12357/1

Mic92 · 2021-06-24T13:16:51Z

Closing for now until @Lassulus has time to work on a prototype.

nixos-discourse · 2021-10-13T12:40:14Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/get-all-unmaintained-nixpkgs-attributes-that-are-part-of-my-nixos-system/15472/2

nixos-discourse · 2023-04-20T08:48:15Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/how-to-get-package-maintainers-of-all-installed-packages/27307/8

[RFC-0081] Show unmaintained packages

8a057a1

edolstra added the status: new label Dec 14, 2020

Mic92 added status: open for nominations Open for shepherding team nominations and removed status: new labels Dec 17, 2020

Mic92 mentioned this pull request Dec 17, 2020

Meeting 2020-12-17 NixOS/rfc-steering-committee#52

Closed

10 tasks

edolstra changed the title ~~[RFC-0081] Show unmaintained packages~~ [RFC 0081] Show unmaintained packages Dec 18, 2020

spacekookie mentioned this pull request Jan 7, 2021

Meeting 2021-01-07 NixOS/rfc-steering-committee#53

Closed

12 tasks

garbas mentioned this pull request Jan 18, 2021

Adds faceted search NixOS/nixos-search#261

Merged

edolstra mentioned this pull request Jan 21, 2021

Meeting 2021-01-21 NixOS/rfc-steering-committee#54

Closed

12 tasks

kloenk mentioned this pull request Feb 18, 2021

Meeting 2021-02-18 NixOS/rfc-steering-committee#56

Closed

11 tasks

Lassulus added 2 commits March 14, 2021 20:25

Update 0081-show-unmaintained-pkgs.md

2393f93

Update 0081-show-unmaintained-pkgs.md

8bf23b9

edolstra mentioned this pull request Mar 18, 2021

Meeting 2021-03-04 NixOS/rfc-steering-committee#58

Closed

10 tasks

Mic92 mentioned this pull request Mar 18, 2021

Meeting 2021-03-18 NixOS/rfc-steering-committee#57

Closed

12 tasks

lheckemann mentioned this pull request Apr 15, 2021

Meeting 2021-04-01 NixOS/rfc-steering-committee#59

Closed

12 tasks

kloenk mentioned this pull request Apr 15, 2021

Meeting 2021-04-15 NixOS/rfc-steering-committee#60

Closed

13 tasks

edolstra mentioned this pull request May 13, 2021

Meeting 2021-04-29 NixOS/rfc-steering-committee#61

Closed

13 tasks

Mic92 mentioned this pull request May 27, 2021

Meeting 2021-05-27 NixOS/rfc-steering-committee#62

Closed

15 tasks

lheckemann mentioned this pull request Jun 10, 2021

Meeting 2021-06-10 NixOS/rfc-steering-committee#64

Closed

15 tasks

Mic92 closed this Jun 24, 2021

kloenk mentioned this pull request Jun 24, 2021

Meeting 2021-06-24 NixOS/rfc-steering-committee#65

Closed

14 tasks

spacekookie mentioned this pull request Jul 8, 2021

Meeting 2021-07-08 NixOS/rfc-steering-committee#66

Closed

17 tasks

This was referenced Jul 24, 2024

Unmaintained Packages NixOS/nixpkgs#67638

Closed

FEATURE REQUEST: Filter - Maintained / Unmaintained NixOS/nixos-search#812

Open

[RFC 0081] Show unmaintained packages #81

[RFC 0081] Show unmaintained packages #81

Conversation

Lassulus commented Dec 10, 2020

ryantm commented Dec 10, 2020

Lassulus commented Dec 11, 2020

garbas commented Dec 14, 2020

Lassulus commented Dec 14, 2020

garbas commented Dec 16, 2020 • edited Loading

Ma27 commented Dec 16, 2020

Lassulus commented Dec 16, 2020

FRidh commented Dec 16, 2020

Lassulus commented Dec 16, 2020

Zimmi48 commented Dec 16, 2020

Lassulus commented Dec 16, 2020

Zimmi48 commented Dec 16, 2020

Lassulus commented Dec 17, 2020

garbas commented Dec 17, 2020

infinisil commented Dec 17, 2020 • edited Loading

spacekookie commented Jan 7, 2021 • edited Loading

Zimmi48 commented Jan 20, 2021

Mic92 commented Jan 21, 2021

Ma27 commented Jan 21, 2021

infinisil commented Jan 21, 2021

Mic92 commented Jan 21, 2021

Mic92 commented Feb 18, 2021

kloenk commented Feb 18, 2021

Lassulus commented Mar 3, 2021 • edited Loading

Lassulus commented Mar 9, 2021

Lassulus commented Mar 14, 2021

Mic92 commented Mar 14, 2021 • edited Loading

RFC 81 Meeting #2 Notes

Opt-in or not

Limitations of current prototype

Interface proposals

Opt-in

Opt-out

Implement it by propagated this info in stdenv

Future work: CI checks for maintainership

infinisil commented Mar 19, 2021

infinisil commented Mar 19, 2021 • edited Loading

7c6f434c commented Mar 20, 2021 via email

infinisil commented Mar 21, 2021

7c6f434c commented Mar 21, 2021 via email

nixos-discourse commented Apr 7, 2021

Mic92 commented Jun 24, 2021

nixos-discourse commented Oct 13, 2021

nixos-discourse commented Apr 20, 2023

garbas commented Dec 16, 2020 •

edited

Loading

infinisil commented Dec 17, 2020 •

edited

Loading

spacekookie commented Jan 7, 2021 •

edited

Loading

Lassulus commented Mar 3, 2021 •

edited

Loading

Mic92 commented Mar 14, 2021 •

edited

Loading

infinisil commented Mar 19, 2021 •

edited

Loading