libexpr: experimental pipe operators

[rhendric]

Motivation

A draft implementation of NixOS/rfcs#148 (excluding builtins.pipe).

Made the following choices about precedence and associativity:

• |> is left-associative and <| is right-associative
• These two operators do not associate with each other; (x: 1) <| 2 |> (x: 3) is a parse error
• Both operators are weaker than ->, the previous weakest operator (part of the motivation here is to reduce parentheses, so the weaker these operators are, the more situations in which they can be used instead of parentheses)

Priorities and Process

Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.

[roberth]

Could you add it to the list of operators, to show the precedence? It's in doc/manual/src/language/operators.md

[roberth]

thought: "a into b", should we give it a name like that? Could be nice.

[rhendric]

🤷 The other direction is a little harder to name, I think? I don't know if this is important.

[MattSturgeon]

"into" follows nicely from the more explicit "a piped into b", though I'm not sure about the other direction (<|).

Maybe that could be read as "from", as in a <| b is "a piped from b"; although I'd personally just read it backwards as "b piped into a".

[roberth]

"taking"?

[rhendric]

Or ‘of’?

This seems like tutorial stuff, not reference stuff; these names would be entirely non-normative and not necessary for understanding the language from first principles. So does it belong in the reference documentation, or in release notes, or both or neither?

[roberth]

Or ‘of’?

This seems like tutorial stuff, not reference stuff; [...] entirely non-normative

Certainly. It was just a

thought

I figured it'd be good to talk about this.

---

I think I like these two, because they're short and literal, with a word per symbol, in order.

• |> pipe into
• <| back pipe

[piegamesde]

Late to the party, but FWIW in the current Lix draft I call them pipe left / pipe right in the code and tend towards calling them pipe (forward) and pipe backward in the documentation. (The forward being implicit and sometimes omitted)

[MattSturgeon]

Just to add some more ideas into the pot, a plumber would usually talk about pipes "feeding" or "supplying" and then "draining" into the wastewater pipes.

E.g:

• "this tap is fed from the cold water supply"
• "the sink drains down a 50mm pipe"
• "the bath is supplied hot water from pipe A"
• "pipe B supplies the bath with cold water"
• "pipe C feeds the shower with cold water"
• "the shower is fed hot water by pipe D"

I don't think this will sound natural in the context of programming, but I thought it was worth reminiscing on my background in construction 😅

I agree "Pipe left/backward" and "pipe right/forward/into" should be fairly natural, simple and obvious

[roberth]

❤️

[roberth]

"taking"?

[roberth]

Channeling @fricklerhandwerk, the reference docs should be at least as complete as this.
EDIT (except for the temporal parts)

[edolstra]

We should be pretty reluctant to add syntactic sugar like this, since it makes the Nix language (even) harder to learn. Such features may be attractive to power users, but increase the barrier to entry for new users, who will have to look up another bit of non-standard syntax. (The use of |> rather than | makes it not super-obvious that this is a pipe operator, unless you're an F# user.)

Since this adds it as an experimental feature, what would be the criterion for stabilizing it?

[MattSturgeon]

We should be pretty reluctant to add syntactic sugar like this, since it makes the Nix language (even) harder to learn. Such features may be attractive to power users, but increase the barrier to entry for new users, who will have to look up another bit of non-standard syntax. (The use of |> rather than | makes it not super-obvious that this is a pipe operator, unless you're an F# user.)

Hasn't this been discussed already on the RFC? (e.g. NixOS/rfcs#148 (comment))

Since this adds it as an experimental feature, what would be the criterion for stabilizing it?

I would assume the criteria for stabilizing would be the RFC getting merged?

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2024-07-17-nix-team-meeting-minutes-162/49255/1

[rhendric]

We should be pretty reluctant to add syntactic sugar like this, since it makes the Nix language (even) harder to learn.

I would say going through the RFC process and gating the implementation behind an experimental feature flag is an appropriate amount of reluctance. What are experimental features for, if we can't do experiments with them?

We can talk all day year about how hard it is to learn different syntaxes, but the proof is in actually trying to teach them to people. It's much easier to do that with an implementation. Otherwise we are stuck in an endless chicken-and-egg loop where the feature doesn't get implemented because people are skeptical about how it will be received, and nobody can assuage the skeptics by trying it out with people at various experience levels because it isn't implemented.

RFC 0148 is currently the highest-thumbed open RFC, with virtually no negative reactions. There is clearly significant interest in the community in seeing some version of this proposal move forward. The original author of the RFC has already expressed their interest in pursuing an implementation in a competing fork of Nix. Is that your vision of how Nix should position itself—‘like that other implementation, but stuck with the syntax from ten years ago!’ Wouldn't it be better if we could run the test and see if the broader, non-power-user community likes this proposal without telling people they have to use a competing implementation to try it out?

[inclyc]

what's the difference between expr_apply and expr_app? It's a bit confusing to me.

[roberth]

Perhaps we could do

Suggested change

	expr_apply
	expr_pipe_apply

and expr_pipe_apply_flipped, expr_pipe_into or expr_pipe_forward for the other production.

[rhendric]

I've renamed the tokens and nonterminals. Please let's not get hung up on making the names perfect; these are internal names that can be improved at any time, whereas releasing this feature is something that other people are waiting for.

[roberth]

Thank you @rhendric!

[Aleksanaa]

The speed at which nix team process PR is changing from one extreme to the other 🤯

[roberth]

To manage expectations, this is a very small and controlled feature, in part thanks to its experimental status and preparation in the form of design discussions on an RFC.
Not to say that those are prerequisites either, by the way. There's other ways to make easy to accept contributions, such as fixes (esp. the kind that doesn't sacrifice something else), or picking up one of the issues in pre-approved directions.
I do agree we're picking up pace though 😃

[roberth]

We should be pretty reluctant to add syntactic sugar like this, since it makes the Nix language (even) harder to learn.

I am absolutely sympathetic to this, but we have to consider the whole system, including lib and its design.
For context, here's my thoughts on a recent PR.
I'd recommend you to read the whole thing, but the crux of it is that the pipe operator(s) remove a strong incentive to add many "convenience" functions that ultimately make the library as a whole harder to comprehend, both when learning the library, and when reading or updating code that uses it.
In this larger context, one or two operators are a small price to pay for a significantly leaner and more effective library.

[anna328p]

There's a small surprise that I noticed: I expected the operators to act like composition and reverse composition, so I could write something like pairsToSet = listToAttrs <| map nameValuePair. but this is not how they behave, and I was confused for a while until I realized it corresponded to listToAttrs (map nameValuePair) and I couldn't write point-free code.