Add lambda functions in GDScript

[vnen]

Describe the project you are working on

GDScript.

Describe the problem or limitation you are having in your project

There are many times when you need to pass a function around, like when connecting signals, using the Tween node, or even sorting an array. This requires you to declare a function in another place in the file, which has a few issues.

If you're not using the function anywhere else, it's still available publicly to be called, which might not be wanted. It also has to live decoupled from where it's defined, which is not great for reading code, as it requires a context switch. This is particularly problematic if you want to rely on a local variable in scope when you connect the signal (you are forced to add it as a bind).

Describe the feature / enhancement and how it helps to overcome the problem or limitation

With lambdas, you are able to create functions in place. This allows you to make temporary functions without having to define it in the class scope somewhere else, solving the issues above. So you can write the signal callback at the same place you connect it. No need to switch context nor make a publicly available function.

This also allows for more functional constructs, like mapping a function over an array instead of a for loop, or passing a function to sort an array.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

Syntax

The syntax for lambdas can be very similar to functions themselves, reusing the func keyword and using indentation to delimit the body:

func _ready():
    var my_lambda = func(x):
        print(x)

They can also be written in a single line, like regular functions:

func _ready():
    var my_lambda = func(x): print(x)

You can add a name to it, for debugging purposes (useful when looking at call stacks):

func _ready():
    var my_lambda = func my_lambda(x): print(x)

You can pass an inline lambda as a function argument:

func _ready():
    button_down.connect(func():
        print("button was pressed")
    )

Lambdas are treated like Callables so to call those you need to explicitly use its call() method:

func _ready():
    var my_lambda = func(x): print(x)
    my_lambda.call("hello")

They're also closures and capture the local environment. Local variables are passed by copy, so won't be updated in the lambda if changed in the local function.

func _ready():
    var x = 42
    var my_lambda = func(): print(x)
    my_lambda.call() # Prints "42"
    x = "Hello!"
    my_lambda.call() # Prints "42"

This still allows you to create lambdas that act differently given the context, like this:

func make_lambda(x):
    var my_lambda = func(): print(x)
    return my_lambda

func _ready():
    var lambda1 = make_lambda(42)
    var lambda2 = make_lambda("Hello!")

    lambda1.call() # Prints "42"
    lambda2.call() # Prints "Hello!"

Implementation

Under the hood lambdas are like any ol' GDScript function, compiled to the same bytecode. They will be stored in a different place since they won't be publicly accessible, so you can't call a lambda without passing a reference to it.

Lambdas will use a Callable wrapper when they are created at runtime, pointing to the already compiled function. This will be a custom callable that will also have references to the enclosing environment, allowing closures to work as expected. When a function exits it will copy the closed on variables and store on that Callable, so the lambda can still exists after the function where it was created is out of the stack. This can be done efficiently by copying only values that were actually used in the lambda.

In the end it should have about the same performance of any function call.

If this enhancement will not be used often, can it be worked around with a few lines of script?

This can be worked around somewhat by defining the functions elsewhere and using Callables, but it does fall into the same issues described here. In particular, you cannot make this function private to ensure it won't be called outside its scope. You cannot easily work around closures.

Is there a reason why this should be core and not an add-on in the asset library?

It needs to be part of the language implementation.

[menip]

Will regular func be first class? If not, I think we should not reuse func keyword for lambdas.

[PLyczkowski]

Clean, simple and has a lot of functionality, love it.

Would this improve gdscript's refactorability, allowing to easily rename a function everywhere? Currently it has to be a string replacement since functions are stored as strings sometimes, like when connecting signals...

[lentsius-bark]

I was rather perplexed by lambdas in the past, and by considering that I came to understand what they are and why they're useful just by reading your proposal, I'm giving it massive thumbs up!

[nonunknown]

awesome improvement, I would like to propose changes in syntax to be something like javascript/C#:

func _ready():
    var my_lambda = (x) =>
        print(x)

func _ready():
    var my_lambda = (x) => print(x)

func _ready():
    var my_lambda = my_lambda(x) => print(x)

func _ready():
    button_down.connect(()=>
        print("button was pressed")
    )

func _ready():
    var my_lambda = (x)=> print(x)
    my_lambda.call("hello")

func _ready():
    var x = 42
    var my_lambda = ()=> print(x)
    my_lambda.call() # Prints "42"
    x = "Hello!"
    my_lambda.call() # Prints "Hello!"

func make_lambda(x):
    var my_lambda = ()=> print(x)
    return my_lambda

func _ready():
    var lambda1 = make_lambda(42)
    var lambda2 = make_lambda("Hello!")

    lambda1.call() # Prints "42"
    lambda2.call() # Prints "Hello!"

Also I think that lambdas can detect identation like funcs:

---func test():
------print("hello")
------var my_lambda = ()=>
---------print("world")
------print(my_lambda())

this also complements what @menip said.

[vnen]

Will regular func be first class? If not, I think we should not reuse func keyword for lambdas.

Regular functions can already be passed around as Callables. So there is a consistency between both.

[jcostello]

@nonunknown also to use const instead of var for declaring

[vnen]

@nonunknown this just makes the parsing harder and it's inconsistent with the rest of the GDScript style. C# and JS are not indentation based, so they follow a different style.

[nonunknown]

@vnen I was thinking about this now LOL, but at least consider python then?
lets see what godot users thinks, your way is very more GDScript like than mine, I'm just passing my toughts here, and in the end of the day we're going to have lambdas anyway :D

[vnen]

@nonunknown the main difference is not adding parentheses, which could be okay but it's just easier to have those. Lambdas in Phyton are also single expressions, while in GDScript I want them to be full-fledged functions. So using the word lambda is probably more confusing, since it won't work the same as Python.

[nonunknown]

hmm, makes sense!

[hilfazer]

Lambdas as full fledged functions? Sounds good! A static lambda that doesn't have a capture list... can't wait.

Or maybe lambdas could have a capture list like in certain popular demonized language (no, not Java) so we could specify which variables get captured (or just capture all)?

[vnen]

Clean, simple and has a lot of functionality, love it.

Would this improve gdscript's refactorability, allowing to easily rename a function everywhere? Currently it has to be a string replacement since functions are stored as strings sometimes, like when connecting signals...

There's no tool for refactor yet, but you can already (in Godot master) use function names directly instead of strings, which will help when such tool is made.

[menip]

Will regular func be first class? If not, I think we should not reuse func keyword for lambdas.

Regular functions can already be passed around as Callables. So there is a consistency between both.

Awesome! Then I think using func keyword for lambda is fine, with a personal preference for using lambda keyword for lambdas.

[BenjaminNavarro]

How do you pass such a lambda to a function taking other parameters after it? I don't see clearly how comas and indentation would mix here

[menip]

BenjaminNavarro: do you mean like this?

func call_lambda(l, arg):
    l.call(arg)

func _ready():
    var my_lambda = func(x): print(x)

    call_lambda(my_lambda, 2) # prints "2"

Edit: the idea is that for a more complex lambda, you can define separately and pass it in. However, it is still lambda, not function, and is not available globally.

[luizcarlos1405]

It will depend on your style. Functions with many arguments can get weird anyway. If you really prefer doing like this instead of declaring a variable like @menip sugested.

func _ready():
    #1
    call_lambda(func(x):
        print(x)
        print("Not that bad I think")

    , "Other Argument"
    , "Yet another Argument")

    #2
    call_lambda(
        func(x):
            print(x)
            print("Not that bad I think")

        , "Other Argument"
        , "Yet another Argument"
    )

[vnen]

Yeah, the example by @luizcarlos1405 would be the way. You don't need the blank line if you don't want to.

[ee0pdt]

@vnen why require the call method in lambda? Seems it would be even cleaner to allow:

func _ready():
    var my_lambda = func(x): print(x)
    my_lambda("hello")

[BenjaminNavarro]

@menip @luizcarlos1405 @vnen
Ok I see, but I find the coma placement inconsistent between the named and inline lambda cases:

func _ready():
	var my_lambda = func(x):
        print(x)
        print("Not that bad I think")

    # usual multiline function call style
    call_named_lambda(
        my_lambda,
        "Other Argument",
        "Yet another Argument"
    )

	# specific function call style
    call_inline_lambda(
        func(x):
            print(x)
            print("Not that bad I think")
        , "Other Argument"
        , "Yet another Argument"
    )

I guess you could also do this:

    call_inline_lambda(
        func(x):
            print(x)
            print("Not that bad I think")
        ,
		"Other Argument",
        "Yet another Argument"
    )

But it's also a bit odd.

The GDScript style guide makes use of trailing comas so I guess with this lambda syntax we would need to add a dedicated function calls with inline lambdas section somewhere because people have to rely on a different style in this case, which is non-optimal IMO but is probably a limitation of indentation based languages.

[katoneko]

Now that lambdas will solidify higher-order functions in GDScript, how would type annotations work for them? Will you be able to type-annotate a parameter/variable that takes a function? Like var fn: func() -> void. (Sorry if this is the wrong place to ask that, seems OK to me now that we're formally discussing lambdas).

[dalexeev]

@katoneko Something tells me that it would just be var fn: Callable. But I would also like a more complete type system.

You can pass an inline lambda as a function argument:

func _ready():
    button_down.connect(func():
        print("button was pressed")
    )

And if the function takes several arguments, it will be something like this:

f("arg1", (func(): print "Hello!"), "arg3")

?

[vnen]

@BenjaminNavarro so, in my view you should also be able to use trailing commas:

func _ready():
    call_inline_lambda(
        func(x):
            print(x)
            print("Not that bad I think"),
        "Other Argument",
        "Yet another Argument",
    )

But they might get a bit hard to see in this style

[vnen]

Now that lambdas will solidify higher-order functions in GDScript, how would type annotations work for them? Will you be able to type-annotate a parameter/variable that takes a function? Like var fn: func() -> void. (Sorry if this is the wrong place to ask that, seems OK to me now that we're formally discussing lambdas).

@katoneko that was something I thought about, but I don't think there's an easy way to enforce that the signature matches in every situation. So, at least for now, you only have the Callable type.

[vnen]

@vnen why require the call method in lambda? Seems it would be even cleaner to allow:

func _ready():
    var my_lambda = func(x): print(x)
    my_lambda("hello")

@ee0pdt it's the same issue with Callables in general: given GDScript is a dynamically-typed language by default, to make this work I would have to assume that every call is to a Callable, but in order to do that I would have to actually make a Callable on every call, which in turn hurts performance.

It was my preference as well, but I could not figure out a way to make it happen without putting a performance penalty on every call.

[vnen]

What happens if I change the lambda to func(x): print(x). I would expect the lambda argument to shadow the local variable. Is that ok?

@noidexe yes, the argument will shadow the variable, but I think that's fine. We could also throw a warning if that is an issue.

[vnen]

I've change the proposal because passing captures as reference will be too complex and the use cases for that are not common. Captures will be passed as copy instead.

[AnilBK]

Hmm I personally don't really like the .call() syntax. I do see why it is needed though. One thing that came to my mind was if we can allow this for statically typed code as syntactic sugar? It could be a good trade-off.

Wherever the type Callable is statically known it could simply expand a () call to however .call() is represented internally instead at compile time. This way it would have no performance cost at runtime.

This could be confusing if you can only use this when the type is statically known, so I perfectly understand if this is undesired, but it should be at least a possibility I think?

Can't this be solved by introducing a lambda keyword?(I know about your reply to not introducing lambda keyword in twitter.)

Like this:

func _ready():
    var x = 42
    var my_lambda = lambda(): print(x)
    my_lambda() # Prints "42"
    x = "Hello!"
    my_lambda() # Prints "Hello!"

instead of

func _ready():
    var x = 42
    var my_lambda = func(): print(x)
    my_lambda.call() # Prints "42"
    x = "Hello!"
    my_lambda.call() # Prints "Hello!"

The point of introducing the lambda keyword is to make sure that you know that this call is to a Callable and it can directly call .call() under the hood.I don't know how callables work.Does it still affect performance this way?

Additional reason would be readability. Anyone would instantly know that it is a lambda function and not a normal function that got misplaced.

[vnen]

@AnilBK the keyword makes no difference. In the same script you can tell it's a lambda, but once you start passing this around it's not possible anymore given the dynamic nature of GDScript.

For example, imagine two files like this:

# a.gd
class_name A

var x

func _init():
    x = func(a): print(a)

func y(a):
    print(a)

# b.gd
class_name B

func _ready():
    var a = A.new()
    a.x("Hello")
    a.y("Hello")

The call to a.x() have the same as the call to a.y(). However, only one of those is actually a Callable. To make this work I would have to either make a.y also a Callable, which needs to be done at runtime and it's not cheap, or make it check if it's a callable or not, which also affects every call.

Regular functions are also Callables BTW, so this would also work fine:

func _ready():
    var a = A.new()
    a.x.call("Hello")
    a.y.call("Hello")

Except that the a.y Callable is generated at the point of the call, so you lose performance in this style.

So the problem with this .call() syntax is not only with lambdas, but with any function. There isn't really any difference between a regular function and a lambda, wherever you can use one you can also use the other.

[vnen]

Heh, this got me thinking and I might be able to allow the direct callable() syntax for any Callable without impacting regular function calls, although Callable themselves might get a bit slower when they are used in an ambiguous context (that is, when the GDScript compiler can't figure out if it's an actual function or a member with a Callable).

[vnen]

Actually, nevermind. It would be too hacky 😢

[AnilBK]

Oh.
Never mind, Thanks for explaining.

[winston-yallow]

Is there a reason why they will be var x = func(a): print(a) instead of just like func x(a): print(a)? I feel like this syntax is more in line with the rest of GDScript. If you need them somewhere else (eg in a global variable) you can still pass them around.

With this approach one could maybe also allow callable() syntax in the local scope without it being confusing for the user. It will be the same rules for every function, regardless if normal or lambda function:

1. if you can reach the function directly, you can directly call it with callable() syntax
2. if you can only reach the function through a ref, then you need callable.call()

Rule 1. works as lambdas can only be reached by name in the scope where they are defined, hence it is possible to infer that they are callable. To make them available anywhere else, one would explicitly assign them to a variable or pass them around as a method arg. Just like regular functions.

[vnen]

Is there a reason why they will be var x = func(a): print(a) instead of just like func x(a): print(a)? I feel like this syntax is more in line with the rest of GDScript. If you need them somewhere else (eg in a global variable) you can still pass them around.

I prefer to be explicit in this case. Those are not supposed to be nested functions, even if they can act like so. If you define a lambda function you must assign it to something in order to access it later.

With this approach one could maybe also allow callable() syntax in the local scope without it being confusing for the user. It will be the same rules for every function, regardless if normal or lambda function:

Yeah, but this complicates the compiler for only a marginal syntax gain. I really prefer to have it explicit. If I could do it everywhere then it would make sense, but confining to a particular situation doesn't seem very helpful.

[MikeSchulze]

hmm looks for me 'lambda' is more a FuncRef and not a real lambda

var my_lambda = func(x): print(x)
    my_lambda.call("hello")

i can achive the same by

func x(value):
	prints(value)

func test_x():
	var l = funcref( self, "x")
	l.call_func("abc")

The real power of lambda is use in functional style like this

 var values := ["a", "b", "c"]

  values.filter( value -> value == "a")

EDIT: godotengine/godot#38645 gives me the answer ;)

[Calinou]

@MikeSchulze Did you see godotengine/godot#38645?

[MikeSchulze]

@Calinou uh i had now a look ;)

array.filter(func(i): return i < 4)

This saves by day
NICE!!!

[CodeSmith32]

Lambdas sound pretty great!
I just started with Godot, but I'm very used to JS and closure-style lambdas.

I know the documentation remains unwritten; but am I correct in understanding your note, @vnen, that these lambdas capture the scope via copy vs. by reference? i.e., the following would print 1 and not 5?

func action(fn):
    fn.call(5)

func second():
    var x = 1
    var fn = func(new_x): x = new_x

    action(fn)

    print(x)

As a side note, if this copying is true, I assume this can still be accomplished with a reference-based value like a dictionary or an array, correct?

func action(fn):
    fn.call(5)

func second():
    var scope = {x: 1}
    var fn = func(new_x): scope.x = new_x

    action(fn)

    print(scope.x)

Lastly, are nested lambdas supported? Although uncommon, this would be helpful in very complicated algorithms.

func outer():
    var x = 4

    var closure = func(obj):
        var a = 1, b = 3

        obj.func1 = func():
            return a
        obj.func2 = func():
            return b * x