proposal: Go 2: add elixir-like pipe operator

[BourgeoisBear]

I think Elixir's Pipe Operator would be a slam-dunk for Go.

Without a pipe operator, we write things like this:

s := strings.ReplaceAll(strings.ToLower(strings.TrimSpace(x)), "/old/", "/new")

// or this:

http.HandleFunc("/",
    RequireAuthMiddleware(
        SomeOtherMiddleware(
            LogMiddleware(IndexHandler))))

Two complaints: 1) fussy parenthesis accounting, 2) text reads L-to-R, action happens R-to-L.

With a pipe operator, they could be re-written like this:

s := strings.TrimSpace(x) |> strings.ToLower() |> strings.ReplaceAll("/old/", "/new")

http.HandleFunc("/",
   LogMiddleware(IndexHandler) |> SomeOtherMiddleware() |> RequireAuthMiddleware(),
)

It's a tad longer, but easier to read. The operations have more visual separation, and now both text and action flow left-to-right.

Elixir's pipe operator only passes the first parameter. For Go, it would probably make more sense to pipe as many parameters as the previous call returns.

[bcmills]

This operator is closely related to the function composition operator (∘ in mathematical notation): it has more-or-less the same behavior, but applies the operations in the opposite order (sometimes notated as ⨾). See https://en.wikipedia.org/wiki/Function_composition#Alternative_notations.

It is also closely related to Haskell's monad sequencing operator (>>=), which implies that we should also consider the interaction with error-handling. What should happen if one of the functions returns an error type?

[bcmills]

In a language without partial function application or currying, the major question is: which parameter is elided if the function accepts multiple arguments? You've chosen to implicitly pass the result(s) as the first argument(s), but if adopted I would rather we have some explicit indicator (perhaps _, or the C++-style _1, _2, etc. for multiple arguments):

s := strings.TrimSpace(x) |> strings.ToLower(_) |> strings.ReplaceAll(_, "/old/", "/new")

[BourgeoisBear]

I like the "no magic" aspect of Go, so treat error like any other type. In the few cases where it doesn't flow, we can always go old-school.

I also like your suggestion for an explicit indicator. That is even better!

[ianlancetaylor]

Lots of functions return errors, so I wonder how much real world code could really use this.

[BourgeoisBear]

Here is a library with 1,822 stars just to do this for http middleware. Pipe operator would have the advantage of being compile-time syntactic-sugar rather than runtime.

Also, since we don't have operator overloading, pipe operator could really clean up calculations using alternative number types: bignum, complex, matrix, etc...

That the mathematicians bothered to coin ∘ gives me a little more assurance that there's more to the idea than just western luxury decadence.

[bcmills]

@BourgeoisBear, this may also be an interesting use-case to consider for generics. The ∘ operator in SML and Haskell, for example, is just a function with inline call syntax. In SML, that function has the signature:

val o : ('a->'b) * ('c->'a) -> 'c->'b

That has an interesting (lack of) interaction with the the current Contracts proposal, in that that proposal includes neither infix operators nor variadic contracts.

Even including the Rust-like lightweight function syntax from #21498, I suspect the best you could do is something like:

func Apply(type A, B) (x A, f func(A) B) {
	return f(x)
}

func Seq(type A, B, C)(f func(A) B, g func(B) C) func(A) C {
	return func(x A) C {
		return g(f(x))
	}
}

with a call site like:

s := Apply(strings.TrimSpace(x), Seq(strings.ToLower, |x| { strings.ReplaceAll(x, "/old/", "/new") }))

That's not terrible, but it does lack the syntactic smoothness of your proposal.

[BourgeoisBear]

@ianlancetaylor , @bcmills , if a Go pipe operator has to support error handling, here is my first stab (more thinking-aloud than wish-list here, BTW):

Support optional pipeline-continue parameters (check-expression and fail-result-expression) into each pipe operator--which can reference the "explicit indicators" (as suggested by @bcmills) from the preceding command in the pipeline.

If the check-expression evaluates to true, proceed to next stage in the pipeline. Otherwise, skip remainder of pipeline and treat the corresponding fail-result-expression as the final r-value. All fail-result-expressions must have the same result signature as the last expression in the pipeline.

/*
   |>
      always continues to next stage in the pipeline
   |check-expression; fail-result-expression>
      only proceeds to next stage if check-expression evaluates to true
*/

// convert string to int, double it, convert back to string...
s, err := "1234" |>
          strconv.Atoi(_1) |_2 == nil; "NaN", _2> 
          strconv.Itoa(_1 * 2), nil
// s = "2468", err = nil

s, err := "wxy" |>
          strconv.Atoi(_1) |_2 == nil; "NaN", _2>
          strconv.Itoa(_1 * 2), nil
// s = "NaN", err = 'parsing "wxy": invalid syntax'
// strconv.Itoa() is never called

[deanveloper]

Without a pipe operator, we write things like this: [snip]
Two complaints: 1) fussy parenthesis accounting, 2) text reads L-to-R, action happens R-to-L.

The most common solution is to instead separate the calls into separate lines. As a function gets more complex, it should expand vertically, not horizontally.

Simple rewrites of your examples which fix both of your complaints:

s := strings.ToLower(strings.TrimSpace(x))
s = strings.ReplaceAll(s, "/old/", "/new")

http.HandleFunc("/", func (w http.ResponseWriter, r *http.Request) {
    f := LogMiddleware(IndexHandler)
    f = SomeOtherMiddleware(f)
    RequireAuthMiddleware(f)
})

[BourgeoisBear]

@deanveloper, "it should expand vertically, not horizontally" isn't the win here. Elixir-style pipe operator expands vertically just fine. The win is easier-to-read code and less intermediate variable juggling.

Elixir style:

   put_status(conn, :bad_request)
   |> put_view(ErrorView)
   |> render(:raise, exception: e, trace: __STACKTRACE__)

@deanveloper style:

   t := put_status(conn, bad_request)
   t = put_view(t, ErrorView)
   render(t, "raise", e, STACKTRACE)

The value proposition in Go is actually higher, since variables are typed. What if the type that put_status() returns is not the same as what put_view() returns? More variable juggling:

   t := put_status(conn, bad_request)
   q := put_view(t, ErrorView)
   render(q, "raise", e, STACKTRACE)

And those variables are not free of implications! Do they conflict with or shadow any prior variables? Are they used later in the function? The pipe operator sidesteps these implications, and makes it easier to read and write code in a functional style.

[deanveloper]

Actually, because of semicolon placement rules, it would have to expand like the following:

 put_status(conn, :bad_request) |>
    put_view(ErrorView) |>
    render(:raise, exception: e, trace: __STACKTRACE__)

I'll agree that variable juggling is an issue, but I personally don't think it's one that needs solving. It's a large(ish) language change to fix a very minor issue. If a variable gets redeclared/shadowed, it either will cause a compile error (which is very easy to see) or a go vet warning (my bad, go vet doesn't do this by default). If you don't see it because of go vet, it won't affect outer scope so it may not even cause an issue in the first place. Both method chaining and function-call nesting are anti-patterns in Go and are favorable to saving return values and passing them along, making it so that changes to functions, especially internal ones, are easier to make without needing to restructure all of your code.

For instance, what if you want to change your put_status function to return an error as well as t? Well, now it won't work with the pipe operator anymore, so you have to remove the pipe operator, fix your indentation, declare your variables, add error checking, and pass the variable into put_view. And now you're back to not using the pipe operator.

Not to mention that you would have to do this at every single call-site of put_status.

If you didn't have the pipe operator, the only changes to make is to add , err to your variable declaration, and add a quick error check.

Another case: what if you want to print the result of put_status? You don't have access to the variable, so you either need to navigate to your put_view function and put the fmt.Println in there (which is bad, since it should ideally be at the call-site), or you have to remove the pipe operator, add your variable declaration, fix your indentation, pass your variable to put_view, then put the fmt.Println in.

Without the pipe operator? Just add a new line and put in fmt.Println(t).

Not to mention that Go isn't exactly a fan of using special symbols. There are very few symbols in Go that have special meaning that aren't common, which I have brought up in other proposals before. Here's a quick list of all of the ones in Go that aren't exactly commonplace in my eyes:

• := (although this is debatable since it is both a common pseudocode and mathematics operator)
• <- (the channel read/write operator)

[BourgeoisBear]

For instance, what if you want to change your put_status function to return an error as well...

This scenario has already been addressed.

Another case: what if you want to print the result of put_status? You don't have access to the variable, so...

Just add another stage with an anonymous function.

  put_status(conn) |>
  func(c Conn) Conn { fmt.Printf("%+v", c); return c }() |>
  ...

[deanveloper]

Just add another stage with an anonymous function.

I must admit that that's an awful lot of code for a simple debugging statement.

This scenario has already been addressed.

I had missed the comment, sorry. The syntax for indexed piping almost looks like Perl to me, which definitely is not desirable. Not to mention that the proposed syntax is a breaking change since _1 and _2 are both valid identifiers.

[BourgeoisBear]

The syntax for indexed piping almost looks like Perl... _1 and _2...

Use a different syntax then. Just work-shopping the idea here. Was prefixed with "more thinking-aloud than wish-list here, BTW".

@deanveloper, try Elixir. I am not recommending it here, so much as sharing that the pipe operator is one thing they got right. Pipes are also pretty sweet in bash.

[ianlancetaylor]

This proposal doesn't seem to have strong support. It doesn't provide any functionality that we don't already have, it just permits writing function calls in a different order. Therefore, this is a likely decline.

Leave open for a month for final comments.

[griesemer]

Go has function literals/closures and you can trivially write a helper function that does exactly what you want. For instance, with:

func apply(s string, fs ...func(string) string) string {
	for _, f := range fs {
		s = f(s)
	}
	return s
}

you can call apply(s, f1, f2, ...) which calls f1 on s, then f2 on s, etc. Here is a complete, runnable example.

There's no reason to change the language when you can just write a trivial helper function that allows you to use the left-to-right notation you are looking for.

[BourgeoisBear]

@griesemer, not the same thing. Elixir-style pipe operator composes functions where input/outputs may be of different types through the pipeline (i.e. f(string) int -> f(int) float -> f(float) struct...). Also, pipe operator would be compile-time rather than run time.

Golang team has set a high bar for inclusion, and I appreciate that. If this gets shot down for what it is, that's cool. Only answering here to make sure this isn't being shot down for what it isn't.

[griesemer]

@BourgeoisBear Point taken. You can of course also trivially write:

t1 := f1(t0)
t2 := f2(t1)
...
tn := fn(tn_1)

which is perhaps not as elegant but also reads nicely top-down (and avoids overlong lines if these chains are really long). The effect is the same.

Writing the code explicitly also won't raise questions as to which arguments are "flowing in" for things like strings.ReplaceAll (in your example) where some arguments are fixed; all things that add extra complexity to the language for a piece of syntactic sugar. And every time one is in a situation where one might now write f(g(x)) a programmer would have to decide whether to write g(x) |> f().

The overwhelming feedback we hear over and over again from Go users is that the explicitness of Go is what makes the language so easy read and use - even if it comes at the cost of extra typing. A new Go user will know what f(g(x)) means; that's much less true for g(x) |> f() unless they are coming from Elixir.

In short this just doesn't seem to meet the bar for inclusion. It doesn't solve an urgent nor an important problem in Go.

[yousefvand]

JavaScript added pipe operator recently but for Golang it doesn't seem a real improvement toward functional programming without some other things.
Although functions are first citizen in Golang, language doesn't support currying, partial application, ADT to name few of FP concepts out of the box. It doesn't need go so far as Haskell did but it would be really nice to have go-routine power over function pipes.
It seems declarative programming was not in mind from the beginning 😞 and the popularity comes from this minimal imperative design.

[ianlancetaylor]

There were further comments since this was marked as a likely decline (#33361 (comment)) but they do not change the reasoning.

-- for @golang/proposal-review