Rollup merge of rust-lang#83366 - jyn514:stabilize-key-value-attrs, r…

…=petrochenkov

Stabilize extended_key_value_attributes

Closes rust-lang#44732. Closes rust-lang#78835. Closes rust-lang#82768 (by making it irrelevant).

 # Stabilization report

 ## Summary

This stabilizes using macro expansion in key-value attributes, like so:

 ```rust
 #[doc = include_str!("my_doc.md")]
 struct S;

 #[path = concat!(env!("OUT_DIR"), "/generated.rs")]
 mod m;
 ```

See Petrochenkov's excellent blog post [on internals](https://internals.rust-lang.org/t/macro-expansion-points-in-attributes/11455)
for alternatives that were considered and rejected ("why accept no more and no less?")

This has been available on nightly since 1.50 with no major issues.

## Notes

### Accepted syntax

The parser accepts arbitrary Rust expressions in this position, but any expression other than a macro invocation will ultimately lead to an error because it is not expected by the built-in expression forms (e.g., `#[doc]`).  Note that decorators and the like may be able to observe other expression forms.

### Expansion ordering

Expansion of macro expressions in "inert" attributes occurs after decorators have executed, analogously to macro expressions appearing in the function body or other parts of decorator input.

There is currently no way for decorators to accept macros in key-value position if macro expansion must be performed before the decorator executes (if the macro can simply be copied into the output for later expansion, that can work).

## Test cases

 - https://github.com/rust-lang/rust/blob/master/src/test/ui/attributes/key-value-expansion-on-mac.rs
 - https://github.com/rust-lang/rust/blob/master/src/test/rustdoc/external-doc.rs

The feature has also been dogfooded extensively in the compiler and
standard library:

- rust-lang#83329
- rust-lang#83230
- rust-lang#82641
- rust-lang#80534

## Implementation history

- Initial proposal: rust-lang#55414 (comment)
- Experiment to see how much code it would break: rust-lang#67121
- Preliminary work to restrict expansion that would conflict with this
feature: rust-lang#77271
- Initial implementation: rust-lang#78837
- Fix for an ICE: rust-lang#80563

## Unresolved Questions

~~rust-lang#83366 (comment) listed some concerns, but they have been resolved as of this final report.~~

 ## Additional Information

 There are two workarounds that have a similar effect for `#[doc]`
attributes on nightly. One is to emulate this behavior by using a limited version of this feature that was stabilized for historical reasons:

```rust
macro_rules! forward_inner_docs {
    ($e:expr => $i:item) => {
        #[doc = $e]
        $i
    };
}

forward_inner_docs!(include_str!("lib.rs") => struct S {});
```

This also works for other attributes (like `#[path = concat!(...)]`).
The other is to use `doc(include)`:

```rust
 #![feature(external_doc)]
 #[doc(include = "lib.rs")]
 struct S {}
```

The first works, but is non-trivial for people to discover, and
difficult to read and maintain. The second is a strange special-case for
a particular use of the macro. This generalizes it to work for any use
case, not just including files.

I plan to remove `doc(include)` when this is stabilized
(rust-lang#82539). The `forward_inner_docs`
workaround will still compile without warnings, but I expect it to be
used less once it's no longer necessary.

compiler/rustc_ast_passes/src/feature_gate.rs

@@ -712,10 +712,6 @@ pub fn check_crate(krate: &ast::Crate, sess: &Session) {
     gate_all!(const_trait_impl, "const trait impls are experimental");
     gate_all!(half_open_range_patterns, "half-open range patterns are unstable");
     gate_all!(inline_const, "inline-const is experimental");
-    gate_all!(
-        extended_key_value_attributes,
-        "arbitrary expressions in key-value attributes are unstable"
-    );
     gate_all!(
         const_generics_defaults,
         "default values for const generic parameters are experimental"

compiler/rustc_codegen_llvm/src/lib.rs

@@ -8,7 +8,7 @@
 #![feature(bool_to_option)]
 #![feature(const_cstr_unchecked)]
 #![feature(crate_visibility_modifier)]
-#![feature(extended_key_value_attributes)]
+#![cfg_attr(bootstrap, feature(extended_key_value_attributes))]
 #![feature(extern_types)]
 #![feature(in_band_lifetimes)]
 #![feature(iter_zip)]

compiler/rustc_errors/src/lib.rs

@@ -5,7 +5,7 @@
 #![doc(html_root_url = "https://doc.rust-lang.org/nightly/nightly-rustc/")]
 #![feature(crate_visibility_modifier)]
 #![feature(backtrace)]
-#![feature(extended_key_value_attributes)]
+#![cfg_attr(bootstrap, feature(extended_key_value_attributes))]
 #![feature(format_args_capture)]
 #![feature(iter_zip)]
 #![feature(nll)]

compiler/rustc_feature/src/accepted.rs

@@ -281,6 +281,8 @@ declare_features! (
     (accepted, or_patterns, "1.53.0", Some(54883), None),
     /// Allows defining identifiers beyond ASCII.
     (accepted, non_ascii_idents, "1.53.0", Some(55467), None),
+    /// Allows arbitrary expressions in key-value attributes at parse time.
+    (accepted, extended_key_value_attributes, "1.54.0", Some(78835), None),
 
     // -------------------------------------------------------------------------
     // feature-group-end: accepted features

compiler/rustc_feature/src/active.rs

@@ -601,9 +601,6 @@ declare_features! (
     /// Allows capturing disjoint fields in a closure/generator (RFC 2229).
     (active, capture_disjoint_fields, "1.49.0", Some(53488), None),
 
-    /// Allows arbitrary expressions in key-value attributes at parse time.
-    (active, extended_key_value_attributes, "1.50.0", Some(78835), None),
-
     /// Allows const generics to have default values (e.g. `struct Foo<const N: usize = 3>(...);`).
     (active, const_generics_defaults, "1.51.0", Some(44580), None),
 

compiler/rustc_hir/src/lib.rs

@@ -4,7 +4,7 @@
 
 #![feature(crate_visibility_modifier)]
 #![feature(const_panic)]
-#![feature(extended_key_value_attributes)]
+#![cfg_attr(bootstrap, feature(extended_key_value_attributes))]
 #![feature(in_band_lifetimes)]
 #![feature(once_cell)]
 #![cfg_attr(bootstrap, feature(or_patterns))]

compiler/rustc_parse/src/parser/mod.rs

@@ -1065,24 +1065,11 @@ impl<'a> Parser<'a> {
             } else if !delimited_only {
                 if self.eat(&token::Eq) {
                     let eq_span = self.prev_token.span;
-                    let mut is_interpolated_expr = false;
-                    if let token::Interpolated(nt) = &self.token.kind {
-                        if let token::NtExpr(..) = **nt {
-                            is_interpolated_expr = true;
-                        }
-                    }
 
                     // Collect tokens because they are used during lowering to HIR.
                     let expr = self.parse_expr_force_collect()?;
                     let span = expr.span;
 
-                    match &expr.kind {
-                        // Not gated to support things like `doc = $expr` that work on stable.
-                        _ if is_interpolated_expr => {}
-                        ExprKind::Lit(lit) if lit.kind.is_unsuffixed() => {}
-                        _ => self.sess.gated_spans.gate(sym::extended_key_value_attributes, span),
-                    }
-
                     let token_kind = token::Interpolated(Lrc::new(token::NtExpr(expr)));
                     MacArgs::Eq(eq_span, Token::new(token_kind, span))
                 } else {

library/core/src/lib.rs

@@ -113,7 +113,7 @@
 #![cfg_attr(bootstrap, feature(doc_spotlight))]
 #![cfg_attr(not(bootstrap), feature(doc_notable_trait))]
 #![feature(duration_consts_2)]
-#![feature(extended_key_value_attributes)]
+#![cfg_attr(bootstrap, feature(extended_key_value_attributes))]
 #![feature(extern_types)]
 #![feature(fundamental)]
 #![feature(intra_doc_pointers)]

library/std/src/lib.rs

@@ -268,7 +268,7 @@
 #![feature(exact_size_is_empty)]
 #![feature(exhaustive_patterns)]
 #![feature(extend_one)]
-#![feature(extended_key_value_attributes)]
+#![cfg_attr(bootstrap, feature(extended_key_value_attributes))]
 #![feature(fn_traits)]
 #![feature(format_args_nl)]
 #![feature(gen_future)]

src/doc/rustdoc/README.md

@@ -0,0 +1,5 @@
+# Rustdoc
+
+This is documentation for rustdoc itself, written in mdbook format.
+To build the book, use `x.py doc src/doc/rustdoc`.
+To run doctests, use `x.py test src/doc/rustdoc`.

src/doc/rustdoc/src/the-doc-attribute.md

@@ -35,6 +35,13 @@ Which can feel more flexible. Note that this would generate this:
 
 but given that docs are rendered via Markdown, it will remove these newlines.
 
+Another use case is for including external files as documentation:
+
+```rust,no_run
+#[doc = include_str!("../README.md")]
+# fn f() {}
+```
+
 The `doc` attribute has more options though! These don't involve the text of
 the output, but instead, various aspects of the presentation of the output.
 We've split them into two kinds below: attributes that are useful at the

src/test/rustdoc/external-doc.rs

@@ -1,5 +1,4 @@
 #![feature(external_doc)]
-#![feature(extended_key_value_attributes)]
 
 // @has external_doc/struct.CanHasDocs.html
 // @has - '//h1' 'External Docs'

src/test/ui/attributes/key-value-expansion-on-mac.rs

@@ -1,4 +1,3 @@
-#![feature(extended_key_value_attributes)]
 #![feature(rustc_attrs)]
 
 #[rustc_dummy = stringify!(a)] // OK

src/test/ui/attributes/key-value-expansion-on-mac.stderr

@@ -1,5 +1,5 @@
 error: unexpected token: `stringify!(b)`
-  --> $DIR/key-value-expansion-on-mac.rs:12:17
+  --> $DIR/key-value-expansion-on-mac.rs:11:17
    |
 LL | #[rustc_dummy = stringify!(b)]
    |                 ^^^^^^^^^^^^^

src/test/ui/suffixed-literal-meta.rs

@@ -1,4 +1,4 @@
-#![feature(rustc_attrs, extended_key_value_attributes)]
+#![feature(rustc_attrs)]
 
 #[rustc_dummy = 1usize] //~ ERROR: suffixed literals are not allowed in attributes
 #[rustc_dummy = 1u8] //~ ERROR: suffixed literals are not allowed in attributes