[RFC 0091] Pure Object Prototypes

[fare]

In this RFC 0091, we propose to add Pure Object Prototypes, or POP, an object system, to the nixpkgs library. POP improves upon current Nix extension systems by supporting multiple inheritance and default values. These improvements solve modularity issues with existing extension systems. We actually use POP in production in a fork of nixpkgs to support local changes to some packages, by multiple people, with dependencies between changes.

POP can be also made to interoperate both ways with existing extension systems, with simple adapters.

[Rosuavio]

Can you add a rendered link to the rfc?

[fare]

I added a link to the rendered page in the intro and in the issue discussion.

[Profpatsch]

I don’t think this is an actionable change that people feel there is a need for.

Can you structure the RFC in less of a rambling manner and go from a set of concrete problems in the existing code (with lots of examples) to how POP solves these issues in a cleaner manner?

For examples the “Dependencies between extensions“ section is horrid, there is no reason for the reader to try and understand it, since you have not even given a reason they should be interested in this proposal.

I think this needs a serious rework before anybody can be expected to read and understand it.

[FRidh]

I recall your original PR, the other RFC and your feedback on #83 and now I went through this one. It reads a bit like a blog and I agree with @Profpatsch you need to make a stronger case earlier on. Start for example with a list of methods we have now (you did that) and a brief list with significant limitations they have that motivate switching to POP. I do now better understand what problems you resolve, but before adding a new system, I think it is important you also understand the existing ones, which includes the NixOS modules system.

• In the past we had composableDerivation which had a slight overlap with this.
• There is a gist with some ideas with language changes

If you want to propose POP for writing package sets, then I think it is important that it is able to do what we need it to do. That is what you need it for, dependencies between extensions, but also what is listed in #83 (which is far from ready...).

[lheckemann]

This RFC is open for nominations; @fare do you have anybody in mind who could be interested in moving this forward?

[7c6f434c]

I could nominate myself, I guess. I can also pre-disclose my position that this is likely to be valuable, but also won't move forward until there is a comparison with the module system so that trade-offs can be discussed.

[fare]

Where do I learn about modules?

[Rosuavio]

The NxOS manual might be a good place to start.

Also you can also look at some modules from nixpkgs in nixos/modules

Or search threw the options that come from modules in the nixpkgs repo.

It also might be worth taking a look at
nix-community/home-manager and LnL7/nix-darwin

[7c6f434c]

Search through options is probably useless in this context, as most of the options are of the most primitive type not showing anything interesting about modules.

[fare]

I read the module documentation at https://nixos.org/manual/nixos/stable/index.html#sec-writing-modules and while it is very interesting indeed, it's a very different mechanism from the extension system / object system that POP is designed to extend.

• The module system has an imports system the effect of which is not very well defined. That's one case where POP's multiple inheritance could give a good meaning to defining dependencies between module definitions such that each is combined only once in dependency order into the mix.
• The "NixOS module system" actually defines three kinds of entities: modules, configurations, types, each with its own semantics, merging algorithm, etc. (Though, if you squint, modules could a special case of types, and configurations be the type being define by the module system.) The definition for a replacement of each of these entities would require a different use of POP's meta-object protocol.
• The merging algorithms used for modules, configs and types are much more elaborate than the one used by POP by default (mergeAttrset), but POP's meta-object protocol (instantiateMeta) supports specifying an alternate mergeInstance mechanism (though the current "toplevel" functions pop, basePop, kPop, etc., all use the default mechanism and won't let you override it for now), so in this sense modules or something module-like could be done in POP and benefit from POP's features and approach.
• Similarly, the warning mechanism could be achieved in a POP setting by using the topProto feature of instantiateMeta to handle warnings at the end of config merging. The same mechanism could more generally "finalize" the values being defined from the result of the merges to something "user-visible".
• POP has a system of dependencies and defaults that is a partial replacement for the priority system of module options: the defaults in POP replace low-priority settings, and the multiple inheritance of POP ensures that settings by children objects (importing modules) will override those of parent objects (imported modules), without having to assign priority numbers.

So I'm not sure what the question or challenge exactly are with respect to "comparing POP and the NixOS module system". On the one hand, POP is a much simpler mechanism meant to replace the Nix extension system and not the NixOS module system. On the other hand, POP does have extension mechanisms with which the more elaborate aspects of the module system could be plugged in, yielding a variant of the module system that deals with imports as dependencies to be combined each a single time in a proper order.

[fare]

Of course, in comparing POP and the existing module system:

• By hypothesis, if POP could be used to do exactly the same as the existing module system, that would be an extremely expensive way to achieve what you already have.
• If POP can do "better" somehow, it will necessarily be incompatible in various subtle or not subtle ways, that may or may not be "conservative" enough given the existing code base.

I'm actually slightly and happily surprised by how the mechanisms of the NixOS module system could by and large be plugged in as a layer on top of an unmodified POP (or rather, its instantiateMeta protocol) to yield a variant of this module system that might handle imports better.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rfcs-looking-for-shepherds/14203/1

[Mic92]

We are going to close this RFC within 2 weeks time if there are no shepherds.

[Mic92]

Sorry. This RFC did not seem reach the level of interests to find enough shepherds. Therefore we close it.

[sternenseemann]

If this is pursued again, I can offer to be a shepherd. I have been lacking time in recent weeks which is why I've put off volunteering here.

Another thing I think would need to be pondered (maybe during discussion) is not only if POP could replace the module system potentially, but also if it could be faster (maybe with the help of a primop?). Module system evaluation speed is surely one of its main problems, to which e. g. making configuration first class in nix has been proposed even.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-02-30-nixpkgs-architecture-team-meeting-30/25973/1