RFC - Annotations

[Kampfkarren]

Rendered

Proposes reserving @name and co for statements and expressions, to pave the way for features like:

@mustuse
local function giveValue(): number

[ghost]

Support because this means inlining could be added and inlining could actually be useful to reduce the overhead of calls on simple statements such as:

@inline
local function GetCatNameFromAge(cat)
  return if cat.age >= 1 then "cat" else "kitten"
end

[Mooshua]

I believe this is really poorly fleshed out.

Support because this means inlining could be added and inlining could actually be useful to reduce the overhead of calls on simple statements @metatablecat

Inlining should not be the motivating scenario given it's already a feature that the compiler already performs.

Additionally, why should we give valuable syntax to something that could be more concisely expressed as a comment? eg:

-- allow unused variable
local x = 1

local x = {
	-- allow undefined variable
	notDefined,
}

I just believe there is a better use for this syntax, such as annotating tables, that users could benefit from.

[m-doescode]

Similar RFC: #824

[Dekkonot]

I just believe there is a better use for this syntax, such as annotating tables

I'd be interested to know what you mean by annotating tables in this case. I'm not able to think of a case where this syntax isn't usable for both annotating functions and tables, but I'm open to it if an example can be provided.

That said, comments aren't a good alternative to this syntax because having to parse comments to control language features is bad. It works for directives at the moment, but they have to be the first tokens in the file and it complicates things dramatically to peek at every comment in a file.

[Mooshua]

@Dekkonot thanks for your response! Sorry for the long delay—Here are my thoughts:

I'd be interested to know what you mean by annotating tables in this case.

My apologies, I was unnecessarily vague there. I believe that this syntax can be used for better semantics than switching on/off type checker features. I’ve included some examples in my (competing) RFC here.

Basically, the gist of it is a rip-off of ECMA Decorators, which are generally praised by the JS community and actively used to enhance previously-imperative code with declarative constructs.

Here’s an example of it being used to enhance defining schemas with a JavaScript ORM.

This C# project uses Attributes (Similar to ECMA decorators, but different semantics) to allow developers to easily automate creating protobuf-like schemas for C# classes.

That said, comments aren't a good alternative to this syntax because having to parse comments to control language features is bad.

Why is it bad? Comments are frequently used to store structured information (eg, jsdoc, xmldoc), so why is it bad for the type checker to use comments too?

The way I see it, it’s already used by the typechecker in a much more limited sense—so it’s positioned well to provide even more annotations to the typechecker in the future.

it complicates things dramatically to peek at every comment in a file.

Luau currently reads and lists all “hot” (--!) comments, it’s just ones that occur after the first lexeme are marked as such and ignored, as you said. While I don’t disagree that this would be complex, it’s not like we’re starting from scratch.

[ghost]

I'm a bit of a rust nut so I would prefer Rust's way of doing annotations but this definitely wont work with Lua syntax, the best approach honestly is to just extend upon the already existing --!macro syntax, since no new grammar needs to be added

--!inline
local function isOdd(n)
  return bit32.band(n, 1) == 1
end

If thats done, typecheck mode could probably be changed to be per-function instead of over the entire script

--!nocheck
local function obviouslyBadType(x: string): number
  return x * 2
end

-- implicitly inferred as nonstrict
...

I guess the issue with that is how you diferentiate setting nocheck/strict onto the entire script, or only on a specific function

[Dekkonot]

@Mooshua

I believe that this syntax can be used for better semantics than switching on/off type checker features.

This RFC specifically does not indicate what it will be used for. It specifically states "[t]his RFC does not propose any annotations, just that the syntax be reserved for future use cases." Whether they end up being annotations or decorators is ultimately irrelevant to this RFC, which is why it's being done; we don't have to agree on the details, just that it should take this syntax. Let's not get caught up bike-shedding quite yet.

@metatablecat

I'm a bit of a rust nut so I would prefer Rust's way of doing annotations but this definitely wont work with Lua syntax, the best approach honestly is to just extend upon the already existing --!macro syntax, since no new grammar needs to be added

As I said before, I would prefer language features not be controlled by comments! It's fine at the moment because the type checker and optimization level don't impact the runtime of any code. Poorly typed code will still compile to the same code and the behavior should be the same between O0 and O2. This follows the use of comments, which is to provide information and not instructions.

If a comment began altering runtime behavior, it would mean any parser or compiler that disregards comments would be wrong. At the moment, disregarding comments for stuff like a linter has no consequences. It may end up being a problem if they became significant, so I can't support it.

[m-doescode]

I think it would be nice to have two different types of annotations. Although this may make it more confusing, it does enable it to be more powerful.

One would simple add information and nothing else onto an object of either type table or function. (e.g. mustuse)

The other would instead pass the object as the first argument to a function and then keep the return value. (like string.pack. See #824)

In essence:

type @somedecorator(text: string, number: number) -- example syntax

@somedecorator("abc",123)
function foo()
    -- blah blah blah...
end

print(getannotation(foo, "somedecorator")) -- "abc", 123
print(getannotations(foo)) -- {"somedecorator"}

-----

function wrapperdecorator(f, initial: any)
    return function(...)
        f(initial, ...)
    end
end

@wrapperdecorator(123)
function myCustomFunction(...)
    print(...)
end

print(myCustomFunction("hello")) -- Output: 123, hello

[Kampfkarren]

--! is also used by more than zero tools in the Lua ecosystem for their own control statements, and thus would have tougher backwards compatibility concerns than a new syntax.

[Kampfkarren]

I also explicitly am avoiding having this RFC make any comment about how user-implemented annotations might work, or even if they are a necessity at all. Putting your idea into the RFC complicates it significantly more, as we now must talk about how to handle ambiguities, stack tracing, etc. I want to keep this very straightforward.

[ccuser44]

--! is also used by more than zero tools in the Lua ecosystem for their own control statements, and thus would have tougher backwards compatibility concerns than a new syntax.

Maybe --@ could be a good syntax for annotations?

[Kampfkarren]

I agree with Dekkonot that comments should not alter runtime behavior, which annotations very well should have the ability to do. --@ is a comment.

[ccuser44]

I agree with Dekkonot that comments should not alter runtime behavior, which annotations very well should have the ability to do. --@ is a comment.

What are examples of ehat they potentislly would modify runtime behavior.

Like how mich is the runtime behavior affected by them?

[Kampfkarren]

This RFC intentionally does not request any specific annotations, but annotations changing runtime behavior is something that very much should be possible. Off the cuff ideas purely for the sake of example are things like force inlining or memory tracking.

[ccuser44]

I think you should add something to the RFC telling that annotations shouldn't be used in a way that they totally change the semantics of a function.

That is, they should only be used for things that do not radically change or alter the semantics of the function.

Force inlining and memory tracking are fine, because they don't really change the semantics of the function. (Although inlining is already done by the compiler so the usefulness of force inlining is not quite clear)
But mustuse should never result in errors, that is, a mustuse shouldn't change the semantics of a script.

[Kampfkarren]

I don't want to put any limitations on what annotations theoretically can do, especially since user defined annotations are a reasonable future development.

[alexmccord]

Worth noting that there could theoretically be use cases where you want to attach an attribute to all returns of a function, or attach an attribute on the module itself, etc, e.g. C# [return: ...] or Rust #![...].

This makes me think that we should reserve syntax that uses an opening and closing delimiter for the sake of future proofing, otherwise we run into the usual ambiguities at the tail end of the syntax. It's also why extending Luau with new syntax without causing breaking changes is a nontrivial problem, the lack of ; to close a statement.

[Kampfkarren]

@alexmccord What do you think about something like @(must_use)?

[alexmccord]

How about @[...], #[...], or [...]?

Edit: oh wait, [...] is potentially ambiguous.

local x = t

[...]
function f() ... end

[Dekkonot]

I'm more in favor of #[...] if we go with that style of delimiters because # is already used in Luau for things so it would feel more consistent. I think @(...) looks bad but I also don't care for @[...].

That said, I can see some people being unhappy with using # instead of @ because of the precedent set by languages like Python and TypeScript... Though I imagine those same people will already be unhappy with @[...] or @(...) instead of @..., so it might not be worth worrying about.

[Mooshua]

I feel like this discussion has just been bikeshedding about the specific syntax to be used rather than the actual semantics and use cases.

We don’t need to make an RFC “reserving” this syntax since it isn’t valid Luau syntax anyways.

I’d prefer if instead we discussed how these annotations would be defined, how these would be used, and what the API surface would look like for user-defined ones, if we choose to support that.

[Dekkonot]

I feel like this discussion has just been bikeshedding about the specific syntax to be used rather than the actual semantics and use cases.

The entire point of this RFC is to settle the question of syntax. It isn't "bikeshedding" for a committee to gather and discuss the color of a bike shed when that is what the meeting was for. The entire point of this discussion is to have a back and forth over syntax; it is not to have a back and forth over how they should work, how they'll be implemented, if/how they should be defined by users, what specific annotations would be implemented, whether these should be "decorators" or "annotations", or anything else you can think of that is not about syntax.

If you wish to argue semantics, you should do so in a different RFC that is intended to settle that question. But please do not attempt to derail this RFC into something it is explicitly not about.

[Mooshua]

It isn't "bikeshedding" for a committee to gather and discuss the color of a bike shed when that is what the meeting was for.

The syntax truly does not matter. Both @ and # are easily accessed on most keyboard layouts around the world. People won’t look at this and go “I won’t use it because it’s an @ and not a #!”

If you wish to argue semantics, you should do so in a different RFC that is intended to settle that question.

I have! Check out #824. But you’re still over here arguing about how you’d type it out.

…And because we have no solid foundation on how these objects should be treated (such as whether or not they can be typed or indexable, and where they can appear) we don’t actually know whether or not the syntax we’ve settled on is legal, intuitive, or ambiguous (eg, not a == b)

So now we’ve spent four months looking at the same proposal that still has gotten no serious traction or progress because we’re stuck up arguing on what color the bike is and not whether or not it will actually have wheels.

[Dekkonot]

I have! Check out #824. But you’re still over here arguing about how you’d type it out.

I would except it's tagged as a draft still. That usually indicates something isn't ready for review. 🙂

[Mooshua]

Alright, I've gone ahead and readied it for review since I'm fairly confident in it for the time being. Didn't realize that would be a blocker, thanks!

[Kampfkarren]

#[] is okay, with the caveat that [] feels like something that might have syntax added onto it someday (Andy showed some interest in that)...what do you think @alexmccord?

@Mooshua Any form of annotations needs to have the right syntax, so the conversation is going to happen at some point anyway. Better to get it done in a dedicated thread for it. Otherwise, an RFC for affixing behavior to annotations is going to be half-syntax and half-behavior discussions, which is a lot harder to follow and more likely to produce suboptimal results for one and/or the other.

[zeux]

This PR is closed as part of RFC migration; please see #1074 (comment) for context.

Additional context: I think it's inevitable that we will end up with this syntax, or a similar syntax. I would recommend using @ in favor of #; # might make sense for Rust but doesn't make sense for Luau as it's already used in the grammar and means something completely unrelated to the idea of annotations. I would recommend not using the word "annotations" here and instead use the word "attributes". We already have (type) annotations and we should not overload the terms. Finally, I'd be interested in seeing a smaller scope proposal that proposes a couple specific attributes that bind to a few specific constructs, maybe focusing on functions - thinking through use cases, a lot of them end up function-centric, and I'm worried that the full attribute syntax is going to require a lot of parser and grammar changes, where function-specific attributes may be easier to specify or implement. That said, there's certainly use cases for adding attributes to local variable declarations as well as statements.

Off the top of my head, a few things I could see using attributes for in the future:

• @inline et al to guide function inlining when our heuristics are insufficient
• @native et al to ensure functions are compiled (or not!) to native code (we currently have module-level hot comments)
• @mustuse to ensure the return of a function is used
• @unused for variable declarations, although maybe _ prefix is fine for this
• @unroll for for loops to force unrolling to happen
• @checked for functions to enforce checking type annotations at runtime (not saying we end up doing this, but this came up in the past)