Tracking issue for custom inner attributes

[alexcrichton]

Added in #54093 inner attributes are not allowed to be procedural macros right now, and this issue is tracking that feature of procedural macros!

Assistance in filling in this issue would be greatly appreciated!
UPDATE: #54726 (comment) below contains the detailed description.

[dhardy]

Summary:

error[E0658]: non-builtin inner attributes are unstable (see issue #54726)
 --> src/lib.rs:3:1
  |
3 | #![foo]
  | ^^^^^^^
  |
  = help: add #![feature(custom_inner_attributes)] to the crate attributes to enable

error[E0658]: The attribute `foo` is currently unknown to the compiler and may have meaning added to it in the future (see issue #29642)
 --> src/lib.rs:3:4
  |
3 | #![foo]
  |    ^^^
  |
  = help: add #![feature(custom_attribute)] to the crate attributes to enable

[petrochenkov]

(I'll fill in details about why this is unstable soon.)

[petrochenkov]

So, the primary open question about inner attributes is what scope they are resolved in.

For example:

use xxx::attr;

mod m {
    #![attr]

    use yyy::attr;
}

Does #![attr] refer to xxx::attr or yyy::attr?

Right now, #![attr] is resolved from outside of the module in the same way as outer attributes (with exception of the crate attributes maybe, not sure).

However, if something is inside a module (or block, etc) it's normally reasonable to expect that it's resolved from inside of that module as well.
IIRC, @nrc argued for applying this logic to attributes too (UPDATE: this happened in #41430).

With attributes such resolution from the inside causes problems though.

---

First of all, there's no "inside" before we expand all the macro attributes.

mod m {
    #![attr]
    
    /* items */
}

is just a token stream, even if it looks like module and items, it effectively doesn't exist in the module structure of the crate yet.

#![attr] can transform that token stream in any way, for example turn module into a block, or struct, or even into nothing, possibly invalidating our "inner" resolution for attr even if we somehow obtained it speculatively from the unexpanded mod m.

---

Possible solutions to this problem:

• Keep status quo and resolve inner attributes from the outside (i.e. treat them purely as syntactic sugar for the "true form" - outer attributes).
  This is the simplest and most straightforward solution.
  Some details need to be figured out for crate attributes though - what exactly is the "outer scope" for the crate root (apparently it's preludes/builtins, but some prelude names actually come from items in the root module) and whether compiler-injected stuff counts as input for macros in this position.

• Resolve from the inside, but prohibit macro attributes in inner attribute positions.
  This way we can be sure that the item to which the attribute is attached is fixed after all its outer attributes are expanded, so we can add it to the module structure of the crate immediately.
  Later, if the attribute does resolve to a macro, we can report an error.
  It turns out this is not entirely backward compatible, things like

  fn test() {
      #![test]
  
      /* body */
  }

  are already used in practice, even in the compiler itself.
  I expect the "no-macro" condition also being perceived as restrictive, given that some people want macros to work even at crate level (!) (Attribute macros invoked at crate root have issues #41430).

• Use the outer resolution for expanding #![attr], but then validate that the inner resolution after expansion gives the same result.
  This gives effectively the same result as status quo, but with future-proofing.
  I think this would be a great way forward, because it allows to remove the feature gate without having to decide on the outer vs inner question right now.

• Use the inner resolution before expansion for expanding #![attr], but then validate that the inner resolution after expansion gives the same result.
  This requires full-blown speculative eager expansion and we don't have it right now.
  We have some form of eager expansion in the compiler, but it's very ad-hoc, buggy and unhygienic, so it's suitable only for very simple stuff a la env!(stringify!(something)).

[petrochenkov]

Use the outer resolution for expanding #![attr], but then validate that the inner resolution after expansion gives the same result.

This variant is the way to go right now, I think.

[dhardy]

The caveat is that adding use yyy::attr; into the inner module may cause breakage. Overall though it seems a reasonable compromise.

[mehcode]

My use case for this feature would be to have a custom crate level attribute in a main.rs.

#![custom_attr_here]

fn main() {
 // [...]
}

There is no outer scope here to resolve that custom attribute.

[petrochenkov]

@mehcode
There's some outer scope even in this case

• built-in attributes / attribute macros (not custom)
• tool attributes, built-in or passed through command line in the future (#![rustfmt::skip])
• extern crates passed with --extern (#![my_crate::my_attr]).

For locally defined/imported attributes that's probably the hardest case though, it certainly requires eager expansion.

[macpp]

Sorry if I'm missing something important, but it seems to me that current behaviour of custom inner attributes is rather inconsistent. For example, if i have file named foo.rs with following content :

#![my_macro]

mod bar {
    #![my_macro]
    fn some_fn () {

    }
}

then first invocation of #![my_macro] receives token stream equivalent to mod foo; (without content of module) but second invocation receives tokent stream with module and full content :

mod bar {
    fn some_fn() { }
}

So custom inner attribute receives no module content if that module is placed in it's own file, but what's even stranger is that this is not always the case. If i add #![my_macro_crate::my_macro] at the top of main.rs then my_macro receives tokenStream with module and it's content, together with prelude import and fn main() .

Is there any chance to make this behavior more consistent? Current behavior could result in unwanted suprises when somebody decides for example to do refactoring and split modules to dedicated files.

[nhynes]

I tried adding a custom attribute using syn (after a small patch), but got the error

error: macro-expanded `extern crate` items cannot shadow names passed with `--extern`
thread 'rustc' panicked at 'internal error: entered unreachable code', src/libsyntax/ext/expand.rs:294:18

using rustc 1.35.0-nightly (f22dca0a1 2019-03-05) running on x86_64-unknown-linux-gnu.

lib.rs

#![feature(custom_inner_attributes, proc_macro_hygiene)]
#![my_attr]

and in the proc_macro crate

#[proc_macro_attribute]
pub fn my_attr(_args: TokenStream, input: TokenStream) -> TokenStream {
    let module = parse_macro_input!(input as syn::ItemMod);
    let content = module.content.unwrap().1; // workaround for mod having no name
    proc_macro::TokenStream::from(quote! { #(#content)* })
}

The error is coming from the extern crate std emitted during the addition of the prelude.

[petrochenkov]

@nhynes
This specific error about extern crate shadowing is too conservative in this case and can be fixed, but crate-level attribute macros are still generally unsupported.

The recommendation is to make a module and apply the attribute to it instead.

[gnzlbg]

This broke stdsimd, which uses #![rustfmt::skip] as an inner attribute. I thought inner tool attributes were ok, but since recently they fail (playground)[https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=e0cee9e1b9a9c8a7aba84b7cd700e184]:

error[E0658]: non-builtin inner attributes are unstable
 --> src/main.rs:1:1
  |
1 | #![rustfmt::skip]
  | ^^^^^^^^^^^^^^^^^
  |
  = note: for more information, see https://github.com/rust-lang/rust/issues/54726
  = help: add #![feature(custom_inner_attributes)] to the crate attributes to enable

[petrochenkov]

@gnzlbg
"Since recently" here is 1.29, this feature gate was introduced before stabilization of tool attributes.
So, inner tool attributes were never available on stable.

[gnzlbg]

Weird, this started erroring this week.

[petrochenkov]

@gnzlbg
I've just realized this is caused by #62133.

Previously #![feature(custom_inner_attributes)] was implicitly enabled by #![feature(custom_attributes)] or #![feature(rustc_attrs)], but the linked PR removed that because custom_attributes is being retired and rustc_attrs changed its meaning.
So, #![feature(custom_inner_attributes)] needs to be written explicitly now.

[Fishrock123]

I'd like this so I can expand a whole bunch of lint rules org-wide from one library attribute line. That'd be quite nicer than the current mess it gives us.

[afetisov]

So, the primary open question about inner attributes is what scope they are resolved in.

I don't know whether this was already implicitly finalized, but I haven't seen two other possibilities suggested;

1. Never resolve inner attributes within inner modules. All inner attributes must always be used with a fully qualified path. Perhaps the inner attributes wouldn't be common enough to make it onerous?
2. Introduce a separate "import inner attribute at crate level" syntax. For example, it could be a crate-level inner attribute similar to #[register_tool], e.g. #[register_attr(::path::to::attr)].

[joshlf]

Is there any way to work around this limitation at the crate root?

[sam0x17]

My use case for this is I am the author of a crate that allows for annotating a module with a cryptographic signature indicating that it has been audited (based on a hash of the underlying AST nodes of the module). Ideally we would only allow using this for top-level whole-file inner attributes, as for these you can safely assume that no unaudited items are being sideloaded into the module. Right now we have to do complex filtering of the module to ensure that every item referenced in the module is part of the audited footprint. With top-of-file inner attributes, this filtering is a lot simpler.

So would love to see this stabilized, with the ability to tell whether a particular attribute is "top-level" in a file or not. That would help us out a lot.

[Qix-]

Not sure if this should be a separate issue or not, or where it should live (let me know and I can create it elsewhere), but there seems to be a weird quirk with #![no_std] being emitted from a custom inner proc macro that is itself used otherwise legally.

Repro case: https://github.com/Qix-/repro-no_std-custom-inner-attr

Results in this error message:

   Compiling test-exe v0.1.0 (/src/qix-/repro-no_std-custom-inner-attr/test-exe)
error[E0658]: `#[prelude_import]` is for use by rustc only
 --> test-exe/src/main.rs:1:1
  |
1 | / #![feature(custom_inner_attributes)]
2 | | #![::custom_attr::add_no_std]
3 | |
4 | | fn main() {}
  | |____________^
  |
  = help: add `#![feature(prelude_import)]` to the crate attributes to enable

warning: unused import: `#![feature(custom_inner_attributes)]
         #![::custom_attr::add_no_std]

         fn main() {}`
 --> test-exe/src/main.rs:1:1
  |
1 | / #![feature(custom_inner_attributes)]
2 | | #![::custom_attr::add_no_std]
3 | |
4 | | fn main() {}
  | |____________^
  |
  = note: `#[warn(unused_imports)]` on by default

warning: unused `#[macro_use]` import
 --> test-exe/src/main.rs:1:1
  |
1 | / #![feature(custom_inner_attributes)]
2 | | #![::custom_attr::add_no_std]
3 | |
4 | | fn main() {}
  | |____________^

For more information about this error, try `rustc --explain E0658`.
warning: `test-exe` (bin "test-exe") generated 2 warnings
error: could not compile `test-exe` due to previous error; 2 warnings emitted

[DanielJoyce]

Yeah this is a rather sizable wart when you want to use an attribute macro in the root crate ( lib.rs ) because there is no way to decorate the crate itself outside of it. :/

[lukesneeringer]

Would it be possible to remove the feature gate if the inner attribute is at the crate root? It seems like the ambiguity doesn't actually apply there, and that's the key spot where there's no alternative / workaround.