Unify and sweeten the syntax for attributes and macros using the @foo notation.

[pcwalton]

• Start Date: 2014-07-21
• RFC PR: (leave this empty)
• Rust Issue: (leave this empty)

Summary

Unify and sweeten the syntax for attributes and macros using the @foo notation.

Motivation

Currently, attributes and macros/syntax extensions are both conceptually macros: they are user-definable syntactic extensions that transform token trees to token trees. However, their syntaxes are quite different: at present, attributes use #[attr()], while macros use macro!(). By switching to a uniform syntax, @attr() and @macro(), we can emphasize their similarity and save syntactic space. At the same time, we reduce the verbosity of attributes and make them more like other languages by using the @ notation.

The ! and # notation take up syntactic space that we may want to use elsewhere. For example, RFC #204 suggests using ! for error assertion.

At least the following languages use @ notation for attributes: Java (annotations), Python (decorators), D, Dart (metadata), Scala (annotations), and Swift. Languages have generally chosen either @ or [] (brackets) to represent attributes; we cannot choose the latter because of ambiguity with array literals.

Julia uses @ for macros.

Objective-C uses @ to indicate special notation not in the C language. Since Objective-C is not a macro expander but is a full-fledged compiler, this is not directly analogous. But, in the author's opinion, the @ sigil has a similar feel in Objective-C.

Detailed design

The following syntactic changes occur:

• #[inline] → @inline
• #[inline(never)] → @inline(never)
• #[deprecated="May discolor some fabrics"] → @deprecated="May discolor some fabrics"
• println!("Hello {}", "Niko") → @println("Hello {}", "Niko")
• vec!["spam", "eggs", "bacon"] → @vec["spam", "eggs", "bacon"]
• bitflags! { flags Flags: u32 ... } → @bitflags { flags Flags: u32 ... }

Parsing is slightly complicated because, where an item is expected, the parser does not know whether an item macro or an attribute is next after parsing the leading @, identifier, and (. Therefore, the parser parses a series of parenthesis-delimited token trees in these cases, and looks at the next token following the ) to determine whether it parsed an item macro or an attribute. If the next token is ;, it considers what it just parsed an item. Otherwise, it reinterprets what it just parsed as an attribute.

The special rules to handle Unix-style #! can be simplified, because it may never be parsed as an attribute.

Drawbacks

• The beauty of @ vis-à-vis #/! is in the eye of the beholder.
• The complication of parsing increases the complexity of the language somewhat and may affect syntax highlighting. (However, this is mitigated to some degree because macros and attributes are already difficult to syntax highlight as a result of their free-form syntax.)

Alternatives

There are innumerable other syntaxes one could consider. Unadorned brackets for attributes, C#-style (e.g. [inline(never)]), does not seem possible to reconcile with our array syntax.

The impact of not doing this is that the current syntax will remain.

Unresolved questions

• @deprecated="foo" may be ugly. Should we do anything about this? One possibility is to switch to @deprecated("foo"), which is more consistent anyhow.

[bharrisau]

Unresolved question: IdentMacro? @macrorules newmacro() is different to @attribute function()?

[pcwalton]

That's a good point. It'd be @macro_rules macro { ... }, which has no ambiguities.

[bharrisau]

Sometimes there really feels like there should be a parser evaluator that can take a grammar and tell you k for LL-k and SLL-k, and other key metrics.

So not @attribute SomeStruct { ... }?

[sfackler]

How would the inner/outer attribute distinction be made?

I'm uneasy about macros and attributes having the same syntax. It complicates the parser and could also complicated a developer's visual "parsing" of the file.

[pcwalton]

The inner/outer distinction would be made the same way as today; e.g. @!inline for inner attributes.

[pcwalton]

@bharrisau @attribute struct SomeStruct { ... }.

[nikomatsakis]

@sfackler macros and attributes having the same syntax strikes me as a good thing, particular since decorators like @deriving are basically macros that are applied to item declarations (eventually, I'd like that to actually be literally the case -- that deriving would be supplied a token tree as input and would emit a token tree back out; but that's a discussion for another day).

[bharrisau]

I meant for let a = Foo { ... };, but you can't add an attribute there anyway.

The '[]' does feel a little redundant for the attributes as the brackets need to match anyway (the = case being the issue).

[sfackler]

@nikomatsakis I'm a bit worried about situations like this ending up being confusing:

@do_a_thing()

struct Foo {}

The RFC states that @do_a_thing() will be interpreted as an attribute attached to Foo since there's no semicolon, but the author may have intended for @do_a_thing to be a normal macro that expands to items or whatever. IMO it'll be a weird subtle thing that'll be easy to get wrong. Maybe good error reporting will make it not so bad.

EDIT: It could also be a bit weird that a normal item definition can't be followed by a semicolon, but a macro expanding to item definitions must be followed by a semicolon.

[gsingh93]

I like things the way they are now. Macros and attributes may both be user-definable syntax extensions, but that doesn't mean they're the same thing. That's why macros are called macros, and not attributes. I view attributes as metadata, and macros are not metadata to me. Because of this semantic difference, I'd like the syntax for macros and attributes to be different.

[pcwalton]

That's precisely what's confusing, IMO. Metadata to me means "data about data". But attributes are code: they actually describe a transformation, just like a macro. They are in fact precisely the same thing as a macro, just with different syntax.

[pcwalton]

@sfackler That's consistent with what we require for other items that use (), namely tuple structs.

[aturon]

@sfackler Note that the proposal matches the current setup for things like tuple structs versus other items.

[CloudiDust]

If the syntax are to be changed, I'd prefer @Foo() for attributes, and foo@() for macros. Easier to disambiguate for both the compiler and the programmer, I think.

They are sometimes related (so the notations also have @ in common) but still distinct.

[CloudiDust]

@pcwalton, under the hood, yes they are all code transformations, but they are used differently. Attributes are modifiers that "describe" some aspects of another piece of code, while macros are self-contained and don't describe anything else.

[pcwalton]

I understand that there can be differences, but I don't see that this warrants a separate sigil any more than than e.g. pure functions versus impure functions would deserve a separate sigil.

[sinistersnare]

would syntax like @deprecated="foo" be whitespace dependant?

could we do

@deprecated="this function sucks" fn my_fn() {}

if not, i vastly prefer @deprecated("foo")

@deprecated("this function sucks") fn my_fn() {}

Actually I prefer the latter anyways :)

+1 for this RFC

[CloudiDust]

@pcwalton

[CloudiDust]

Sorry my finger slipped, I agree that we don't need two sigils, but having some distinction is better.

If we see macros/attributes as compile-time functions, then macros are freestanding ones while attributes are "methods" with an implicit first arg that is the modified struct/enum/module/crate etc.

[Valloric]

This is going to turn into a bikeshed very quickly; before it does, let's point out that @ would be very consistent with what other languages use (and thus familiar to newcomers) which is a strict objective plus.

Let's try to refrain from "I like X over Y because X looks better" comments.

[andrew-d]

+1 for the @deprecated("foo") syntax, especially since it opens up future possibilities for stuff like @deprecated("foo", use_instead="bar"), which would be harder to parse without the brackets.

[CloudiDust]

@sinistersnare, I think @foo = "bar" fn ... looks like an assignment which is not properly delimited from a function definition. @foo("bar") is better and in sync with Java/Python etc.

@sfackler @pcwalton, I prefer changing #![foo] to @@foo, not @!foo. Sorry @Valloric, I do think @@foo looks better.

And one more reason why I think macros should be foo@(...) and attributes should be @foo(...) is that, attributes have a fixed syntax, but almost all bets are off inside the body of a macro call.

[CloudiDust]

@Valloric @pcwalton, besides "looks better", @@ also "frees ! further", which is one of the purpose of this RFC, and @!foo looks somewhat like "Warning! Dangerous attribute!" which is not the intention.

[CloudiDust]

! in Rust is also the not operator, and @!foo doesn't mean "not @foo" or "switch @foo off", so I think this is another point for @@.

[nikomatsakis]

@CloudiDust I expect that attributes will eventually permit almost anything inside their body, for what it's worth. (It'd certainly be useful to be able to provide arbitrary token-trees as input to @deriving in some cases, for example.)

[ftxqxd]

Isn’t @foo { ... } ambiguous, with RFC 40 accepted? It’s either an attribute decorating a block expression, or it’s a macro being invoked with curly braces. This applies to [] and () too. Would I have to use { @!foo ... } instead? And would that work with normal brackets ((@!foo 1 + 2))?

I’d rather we just keep what we have now, but change macros to use a leading #, mainly for ambiguity- and clarity-related reasons. But if people want to get rid of the brackets in attributes, I’d prefer using # to @, simply because it looks nicer, and # is used for similar-ish things in C[++].

[CloudiDust]

@nikomatsakis, that can be a great feature.

But it is still true that macros are "standalone" and many can be used in the context where (runtime) functions are expected, like vec@[...] and println@(...).

While attributes are not standalone. They "modify" some nearby piece of code (or the module/crate) and are not used like runtime functions. @@feature(phase) or @deriving are examples.

[pcwalton]

@P1start That's a good point, but inner attributes can still be used inside blocks.

We used to use # for macros and #println was very disliked.

[CloudiDust]

@P1start, FWIW, Rust used to have prefix # as the macro sigil, but it was later changed to suffix ! as many didn't like #. That's like the [...] vs <...> generics notation debate.

[CloudiDust]

@kballard, I think the problem is that macros and functions share the same syntax when they are actually different (they execute differently).

Lisp has simpler surface syntax rules, at the cost of increased difficulties when telling different things apart. That's why Clojure added "non-lispy" things that are "more syntax rules", and I like them.

That's also one of the reasons why I am against using @ for both macros and attributes in Rust.

[CloudiDust]

But then again, Lisps have simple cores and much of the essential functionality is implemented with macros. And requiring something like cond to bear a marker may be a non-starter, blessing some "core macros" to be marker-less also goes against the spirit of Lisps. :P

[jfager]

@kballard That ignores both that lisp macros can have non-s-expr forms (loop, format, read macros) and that other non-s-expr langs have sigil-free macros.

[huonw]

Also, macro'('A', ...) may seem less than ideal. Though I am not sure if there are any macros with chars as the first arguments.

This represents a very large ambiguity, as it would have to tokenise as IDENTIFIER("macro"), CHAR_LITERAL('('), IDENTIFIER("A"), ..., that is, the compiler would be seeing macro '(' A ',.

[mdinger]

@huonw Thanks for clarifying. I wasn't actually trying to champion that syntax. I was just mentioning it in case an implementer such as @pcwalton thought it might be useful.

[kennytm]

If macros and attributes share the same syntax, how could one know if I want #[cfg(test)] or cfg!(test)?

[Ericson2314]

+1 unify. +1 @deprecated("foo")

[brendanzab]

+1 @kennytm. Macros/attributes could potentially be allowed in more locations in the future, like attributes on block expressions. This would limit the places they could be used.

[nikomatsakis]

Relevant thoughts: http://smallcultfollowing.com/babysteps/blog/2014/09/11/attribute-and-macro-syntax/

[jfager]

I still don't understand the motivation for unification. Some attributes happen to be macros and they can share implementation details. So what? Even in a hypothetical unified future, everyone would still refer to 'attributes' and 'macros' as distinct things used for distinct purposes. If their similarity isn't general enough to obviate one term or the other, why should it be considered general enough to obviate one syntax or the other?

You say they are two sides of the same coin. But one side of a coin is called heads and the other tails, and they get different pictures. Distinctions are useful.

[CloudiDust]

@nikomatsakis I am now quite fond of the one solution proposed by Sodel_the_Vociferous in the comments of your blog post.

I expended on his comment a bit:

• Macros: @foo/@foo(..)/@foo[..]/@foo{..}.
• Outer attributes: @foo:/@foo(..):/@foo[..]:/@foo{..}:.
• Inner attributes: @!foo/@!foo(..)/@!foo[..]/@!foo{..}.

The colons at the end of outer attributes signify the fact that an outer attribute applies to the immediately following non-attribute entity.

We can also later support macro/outer attribute/inner attribute arrays: @[inline, cfg(..)]:.

[int-index]

Please, keep the hashtag form for attributes. It is nice.

[CloudiDust]

@nikomatsakis:

As the plan is to almost unify macros and attributes, and maybe allow macros to be used like attributes, we should indeed put macros and attributes in the same namespace, or at least forbid macros and attributes with identical names.

I think we can see @/@!/@: as "different calling syntax" of the same thing. The only difference is where the macro/attribute gets applied to.

Currently, macros can only be called with @, the "standalone syntax", and attributes can only be called with @! or @:, the "outer/inner decorator syntax".

But in the future some macros will be callable with @! and @:, and some attributes will be callable with @.

Anyway, "calling" an attribute feels weird, maybe we should just start calling them "decorators" and draw parallels with Python then:

A Python decorator is a function that gets called with the decorator syntax @foo and applied to the immediately following entity.

A Rust decorator is a compile-time function (in other words, macros or other compile time magic) that gets called with the decorator syntax @foo: and applied to the immediately following entity.

Python doesn't have a "compile time" and doesn't have macros, so it can simply use @ for decorators and run with it. But, Python does have two calling notations: normal function calling notation and decorator calling notation. It's just that the distinction is "with or without a sigil", not "with this or that (combination of) sigil(s)".

So it is fine for Rust to have three related but distinct calling notations for macros/outer decorators/inner decorators.

[MatejLach]

@nikomatsakis: After reading your blog post, I am leaning towards option 3, specifically because it will maintain a clear distinction between attributes and macros. Having both of them being represented the same will leave the distinction "on convention basis" and from my experience, that's not a good thing, especially when reading code, (remember, code is much more read than it's written).

[CloudiDust]

Using the @foo/@foo:/@!foo syntax with attribute/decorator arrays to rewrite my old example and we get:

@!feature(some_fancy_feature)
@![some_fancy_attr, another_fancy_attr]

@deprecated("This is old fashioned."):
@[repr(C), deriving(Show)]:
struct MyOldVector {
    x: f64,
    y: f64,
}

@deriving(Show):
enum MyVector {
    Vec2(f64, f64),
    Vec3(f64, f64, f64)
}

@inline:
fn create_random_vector() -> MyVector { ... }

@macro_rules here_be_magic { ... }

fn main() {
    match create_random_vector() {
        res @ Vec3(_, _, _) => @println("The result is a 3D-vector: {}.", res),
        @cold: Vec2(x, y) if @here_be_magic(x, y) == 6 => @println("Lucky!"),
        Vec2(_, _) => @println("The result is a 2D-vector {}.", res)
    }
}

The other parts look nicer than my original proposal (@foo/@:foo/@!foo), but @cold: makes visual parsing the match arms harder.

And in this regard, @[cold] may also be confused with an array pattern.

[Kimundi]

For what it's worth, a attribute syntax like @foo(): might be pedagogically useful because the colon indicates the direction/order in which multiple attributes expand:

@foo(...):
@bar(...):
struct Baz;

=>

@foo(..., @bar(..., struct Baz;));

[arcto]

👍 Two points for using @ for macros instead of the current !:

• At least in my eyes, it makes them a bit more visible -- which is good because macros may operate differently on arguments than a function call would, e.g. println!(foo) where foo is not consumed! For this reason it's good to make macros stand out a little bit, imho.
• It frees up the !-symbol which can then be given a role in error-handling, perhaps together with ?.

[kmcallister]

As someone who's written a lot of macro invocations I have a really strong "ughhhh" gut reaction to @foo(...) syntax. It just looks terrible to me. I'd get used to it but I don't want to :P I also prefer the current syntax for attributes but that one isn't a big deal.

using @ for macros instead of the current !… At least in my eyes, it makes them a bit more visible -- which is good because macros may operate differently on arguments than a function call would… For this reason it's good to make macros stand out a little bit, imho.

foo!(...) stands out plenty for me, especially with syntax highlighting. And a well-designed macro is going to have invocation syntax that makes sense in context. You shouldn't have to completely stop the flow of reading and pick apart the arguments one token tree at a time. Sure it's possible to write macros with that flaw but it's also possible to write ordinary functions with really ugly or convoluted calls. Macro hygiene prevents the most counter-intuitive behaviors, and we seem to be taking a hard line against unhygienic macros for the moment.

The benefits of freeing up ! are fairly compelling, and a prefix sigil for macros is potentially useful; it could simplify adding parser support for macros in more places. I just really dislike the @foo() option in particular.

We should also consider the interaction with some other bits of proposed syntax:

• qualified macro names: foo::bar!() vs @foo::bar()
• method macro sugar: foo.bar!() vs foo.@bar()

The first one seems like a wash (I mind the @ less on a qualified name for whatever reason) but the second one strengthens the case against @foo(), in my opinion.

[ftxqxd]

Hm, I’m starting to like this RFC more and more. (Although I still prefer #. :P)

But I can’t get over the ambiguities this introduces. For example, @foo(bar)(baz) could be parsed in not one, not two, not three, but four different ways (with spacing and added parentheses/semicolons for clarity):

// Calling a macro invocation
(@foo(bar))(baz)

// A statement macro followed by a parenthesised expression
@foo(bar);
(baz)

// An attribute with one parameter decorating a parenthesised expression
@foo(bar) (baz)

// An attribute with no parameters decorating a call expression
@foo (bar)(baz)

From the RFC itself, today’s semantics, and the previous discussion around ambiguities, I gather that the second interpretation would be how the parser would parse it, but any of the others could be the intended interpretation.

[nikomatsakis]

I think we can close this as "not gonna' happen".