Tracking Issue for `macro_metavar_expr_concat` #124225

c410-f3r · 2024-04-21T13:18:10Z

This is a tracking issue for the experimental feature macro metavariable expression concat (RFC pending).
The feature gate for the issue is #![feature(macro_metavar_expr_concat)].

Provides a modern alternative to the concat_idents! macro.

#![feature(macro_metavar_expr_concat)]

macro_rules! many_idents {
    ($a:ident, $c:ident) => {
        const ${concat($a, B, $c, D)}: i32 = 1;
    };
}

fn main() {
    many_idents!(A, C);
    assert_eq!(ABCD, 1);
}

About experimental features

An experimental feature is one that has not yet had an RFC. The idea is to allow implementation work to proceed to better inform an upcoming RFC. Experimental features cannot be stabilized without first having an RFC. The existence of an experimental feature does not indicate lang team consensus that the feature is desirable, only that there is a problem that is worthy of being solved and that the idea has enough merit to consider exploring. See the lang team process page for more details.

About tracking issues

Tracking issues are used to record the overall progress of implementation.
They are also used as hubs connecting to other relevant issues, e.g., bugs or open design questions.
A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature.
Instead, open a dedicated issue for the specific matter and add the relevant feature gate label.

Steps / History

Implementation (Add a new concat metavar expr #118958)
Write an RFC
Adjust documentation (see instructions on rustc-dev-guide)
Formatting for new syntax has been added to the Style Guide (nightly-style-procedure)
Stabilization PR (see instructions on rustc-dev-guide)

Unresolved Questions

Should the syntax use commas? Add a new concat metavar expr #118958 (comment)
Should numeric literals be allowed? (yes) Add a new concat metavar expr #118958 (comment)
Which span should the resulting identifier have? Should it be related to the macro, the concatenated values, or customizable? Add a new concat metavar expr #118958 (comment)
What should be done with r#foo raw identifiers? Add a new concat metavar expr #118958 (comment)

References

Tracking Issue for RFC 3086: macro metavariable expressions #83527

The text was updated successfully, but these errors were encountered:

fmease · 2024-04-21T14:16:51Z

This still needs a T-lang shepherd, right?

c410-f3r · 2024-04-21T14:37:41Z

This still needs a T-lang shepherd, right?

Unfortunately I am not familiar or aware of this procedure.

fmease · 2024-04-21T15:09:07Z

All language features have to go through the RFC process eventually. However, the implementation can be done before an RFC has been written by following T-lang's experimental feature process which requires a T-lang member to support — shepherd — the venture. Only “small” language (not compiler) “fixes” don't need to follow the RFC process, they just need a T-lang FCP.

fmease · 2024-04-21T15:09:48Z

T-lang, this feature is seeking a shepherd, therefore I'm nominating this for you.

An implementation is already under way (#118958). This feature adds a new kind of macro metavariable expression (the latter (overarching) feature has been approved by the merged RFC 3086, see also its tracking issue and stabilization PR (proposed FCP, disposition: merge)). The feature represents a more potent alternative / potential successor to the unstable built-in macro concat_idents (the latter has stalled, blocked on design concerns, see its tracking issue).

Please consult the tracking issue description for more context.

joshtriplett · 2024-05-15T15:49:05Z

Happy to second this with my T-lang hat on.

@c410-f3r Good luck, and please reach out to #t-lang on Zulip if you have any questions. Feel free to @ me there if needed.

c410-f3r · 2024-05-15T17:26:33Z

Thank you @joshtriplett

Add a new concat metavar expr Revival of rust-lang#111930 Giving it another try now that rust-lang#117050 was merged. With the new rules, meta-variable expressions must be referenced with a dollar sign (`$`) and this can cause misunderstands with `$concat`. ```rust macro_rules! foo { ( $bar:ident ) => { const ${concat(VAR, bar)}: i32 = 1; }; } // Will produce `VARbar` instead of `VAR_123` foo!(_123); ``` In other words, forgetting the dollar symbol can produce undesired outputs. cc rust-lang#29599 cc rust-lang#124225

UserIsntAvailable · 2024-06-15T11:50:14Z

Sorry if this is the wrong place to discuss this, but should ${concat} provide its own case-conversion mechanism (just like paste! does), or should we instead provide a standalone ${case} metavar expr to handle this?

If we where to follow paste! syntax, then it would look like:

macro_rules! many_idents {
    ($a:ident, $c:ident) => {
        const ${concat($a:upper, B, $c:lower, D)}: i32 = 1;
    };
}

fn main() {
    many_idents!(a, C);
    assert_eq!(ABcD, 1);
}

Or with a future ${case} metavar expr:

macro_rules! many_idents {
    ($a:ident, $c:ident) => {
        const ${concat(${case($a, upper)}, B, ${case($c, lower)}, D)}: i32 = 1;
    };
}

fn main() {
    many_idents!(a, C);
    assert_eq!(ABcD, 1);
}

Which I think is way too verbose, but maybe I'm just too used to paste! simplicity. The advantage of ${case} over ${concat} would be that, currently, ${concat} requires 2 operands to be able to be used (this could technically be changed), so ${case} would allow:

use std::ops::Add;

pub struct Wrapper<T>(T);

macro_rules! impl_op {
    ($Op:ident => $Ty:ty) => {
        impl $Op<$Ty> for Wrapper<$Ty> {
            type Output = Self;

            // alternative syntax; `:` instead of `,`.
            fn ${case($Op:snake)}(self, rhs: $Ty) -> Self::Output {
                Self(self.0.${case($Op:snake)}(rhs))
            }
        }
    };
}

impl_op!(Add => u8);

With that said, maybe there is a world where both approaches are accepted, but I think there is probably less motivation to do that?

cc @c410-f3r @joshtriplett for opinions about this.

c410-f3r · 2024-06-15T16:36:45Z

Well, I personally don't mind discussing potential designs in a tracking issue.

${case}

The introduction of ${case} as a new metavariable expression that can be used with or without ${concat} looks worth pursuing and semantically correct but IFAICT, inputs currently only accept ad-hoc parameters. More specifically, ${concat} only accepts literal identifiers and variables.

// Same problem with multiples `${concat}`s

macro_rules! foo {
    ($a:ident, $b:ident, $c:ident) => {
        let ${concat(a, ${concat(b, c)})}: () = (); // ERROR !
    }
}

Perhaps the evaluation of nested metaravariables expressions will become a thing someday?

$ident:modifier

Looks feasible in regards to a possible implementation but semantically awkward IMO. I am not aware of something similar in the macro system.

If acceptable, then it should probably be restricted to a subset of operations.

macro_rules! foo {
    ($a:ident) => {
        // Is it OK?
        let $a:upper = ();
    }
}

These are just my opinions. A theoretical stabilization of ${concat} with or without case modifiers is fine to me.

UserIsntAvailable · 2024-06-16T11:41:46Z

The introduction of ${case} as a new metavariable expression that can be used with or without ${concat} looks worth pursuing and semantically correct but IFAICT, inputs currently only accept ad-hoc parameters. More specifically, ${concat} only accepts literal identifiers and variables.
// Same problem with multiples `${concat}`s

macro_rules! foo {
    ($a:ident, $b:ident, $c:ident) => {
        let ${concat(a, ${concat(b, c)})}: () = (); // ERROR !
    }
}
Perhaps the evaluation of nested metaravariables expressions will become a thing someday?

Oh I wasn't aware of that limitation. I imagine that something like that would be needed down the line. I don't much about the internals of the macro codebase, so I wouldn't know if this would be hard to implement.

As a side note, I wonder if something like ${concat($a, concat($b, $c)} would be able to be made non ambiguous; I would imagine this kind of analysis would be hard to pull off, since you can't no longer look for ${} to know that the next thing is a metavar expr identifier.

$ident:modifier

Looks feasible in regards to a possible implementation but semantically awkward IMO. I am not aware of something similar in the macro system.

If acceptable, then it should probably be restricted to a subset of operations.
macro_rules! foo {
    ($a:ident) => {
        // Is it OK?
        let $a:upper = ();
    }
}

Do note that if $ident:modifier is accepted as the final syntax, it would only work inside of ${concat} or ${case}, which would remove the ambiguity for let $a:upper = ();; identifiers can't use the : symbol, so let ${case($a:upper)} = (); is non ambiguous.

With that said, you made me realize that with this change some metavar exprs would have "special syntax" attached to them, which might not be the desired design, specially because most users would think about metavar exprs as functions rather than "fancier macros" (although accepting this change would contradict this).

Another option would be to allow a standalone syntax ${$ident:modifier} (without writing case), but I don't know if case-conversion is important enough to have its own syntax (I also don't know what other modifiers you could really add to these).

These are just my opinions. A theoretical stabilization of ${concat} with or without case modifiers is fine to me.

This is something that can go under "Future possibilities" on the final RFC. I don't particularly think that stabilization should be blocked on this, since we can always add it later. :)

Add a new concat metavar expr Revival of #111930 Giving it another try now that #117050 was merged. With the new rules, meta-variable expressions must be referenced with a dollar sign (`$`) and this can cause misunderstands with `$concat`. ```rust macro_rules! foo { ( $bar:ident ) => { const ${concat(VAR, bar)}: i32 = 1; }; } // Will produce `VARbar` instead of `VAR_123` foo!(_123); ``` In other words, forgetting the dollar symbol can produce undesired outputs. cc #29599 cc rust-lang/rust#124225

crumblingstatue · 2024-06-20T10:30:08Z

Is it a known issue that this doesn't seem to work with identifiers repeating at a depth?

#![feature(macro_metavar_expr_concat)]

macro_rules! many_idents {
    ($a:ident, $c:ident) => {
        const ${concat($a, B, $c, D)}: i32 = 1;
    };
}

// Paste implementation included for reference
macro_rules! many_idents_paste {
    ($a:ident, $c:ident) => {
        paste::paste! {
            const [<$a B $c D>]: i32 = 2;
        }
    };
}

macro_rules! many_idents_multi_metavar {
    ($($a:ident, $c:ident;)*) => {
        $(
            const ${concat($a, B, $c, D)}: i32 = 3;
        )*
    };
}

// Paste implementation included for reference
macro_rules! many_idents_multi_paste {
    ($($a:ident, $c:ident;)*) => {
        $(
            paste::paste! {
                const [<$a B $c D>]: i32 = 3;
            }
        )*
    };
}

fn main() {
    many_idents!(A, C);
    assert_eq!(ABCD, 1);
    many_idents_paste!(F, G);
    assert_eq!(FBGD, 2);
    many_idents_multi_paste! {
        H, I;
        J, K;
        L, M;
    }
    assert_eq!(HBID, 3);
    assert_eq!(JBKD, 3);
    assert_eq!(LBMD, 3);
    many_idents_multi_metavar! {
        N, O;
        P, Q;
        R, S;
    }
}

error: attempted to repeat an expression containing no syntax variables matched as repeating at this depth
  --> src/main.rs:19:10
   |
19 |           $(
   |  __________^
20 | |             const ${concat($a, B, $c, D)}: i32 = 3;
21 | |         )*
   | |_________^

The paste version works without errors.

I can go a bit further with ignore

macro_rules! many_idents_multi_metavar {
    ($($a:ident, $c:ident;)*) => {
        $(
            ${ignore($a)}
            ${ignore($c)}
            const ${concat($a, B, $c, D)}: i32 = 3;
        )*
    };
}

But then I get:

error: `${concat(..)}` currently only accepts identifiers or meta-variables as parameters
  --> src/main.rs:22:29
   |
22 |             const ${concat($a, B, $c, D)}: i32 = 3;
   |

This seems like a major limitation.

c410-f3r · 2024-06-21T22:52:59Z

Looks like a bug to me. I will try to investigate it in the following weeks

Add a new concat metavar expr Revival of #111930 Giving it another try now that #117050 was merged. With the new rules, meta-variable expressions must be referenced with a dollar sign (`$`) and this can cause misunderstands with `$concat`. ```rust macro_rules! foo { ( $bar:ident ) => { const ${concat(VAR, bar)}: i32 = 1; }; } // Will produce `VARbar` instead of `VAR_123` foo!(_123); ``` In other words, forgetting the dollar symbol can produce undesired outputs. cc #29599 cc rust-lang/rust#124225

[`macro_metavar_expr_concat`] Add support for literals Adds support for things like `${concat($variable, 123)}` or `${concat("hello", "_world")}` . cc rust-lang#124225

Rollup merge of rust-lang#126841 - c410-f3r:concat-again, r=petrochenkov [`macro_metavar_expr_concat`] Add support for literals Adds support for things like `${concat($variable, 123)}` or `${concat("hello", "_world")}` . cc rust-lang#124225

[`macro_metavar_expr_concat`] Add support for literals Adds support for literals in macro parameters. ```rust macro_rules! with_literal { ($literal:literal) => { const ${concat(FOO, $literal)}: i32 = 1; } } fn main() { with_literal!("_BAR"); assert_eq!(FOO_BAR, 1); } ``` cc rust-lang#124225 r? `@petrochenkov`

[`macro_metavar_expr_concat`] Add support for literals Adds support for literals in macro parameters. ```rust macro_rules! with_literal { ($literal:literal) => { const ${concat(FOO, $literal)}: i32 = 1; } } fn main() { with_literal!("_BAR"); assert_eq!(FOO_BAR, 1); } ``` cc rust-lang#124225 r? ``@petrochenkov``

tgross35 · 2024-07-18T04:57:07Z

It would be nice if there was some way to allow turning ${concat(c, $litstr)} into c"some string" for constructing C string literals when a Rust string is provided. That is, a replacement for cstr!. But I can't think of a good syntax here.

#127542 adds support for literals, but the above would get turned into (invalid) identifier csome string.

tgross35 · 2024-07-18T05:00:08Z

Regarding syntax, I would prefer a space-separated form like ${concat(get_ $item _mut)}, rather than the current comma-separated ${concat(get_, $item, _mut)}. That just makes it easier to read imo, more like format!, and also matches paste.

Rollup merge of rust-lang#127542 - c410-f3r:concat-again, r=petrochenkov [`macro_metavar_expr_concat`] Add support for literals Adds support for literals in macro parameters. ```rust macro_rules! with_literal { ($literal:literal) => { const ${concat(FOO, $literal)}: i32 = 1; } } fn main() { with_literal!("_BAR"); assert_eq!(FOO_BAR, 1); } ``` cc rust-lang#124225 r? ``@petrochenkov``

…gjubilee [`macro_metavar_expr_concat`] Dogfooding cc rust-lang#124225 Starts inner usage to test the robustness of the implementation.

Rollup merge of rust-lang#128491 - c410-f3r:unlock-rfc-2011, r=workingjubilee [`macro_metavar_expr_concat`] Dogfooding cc rust-lang#124225 Starts inner usage to test the robustness of the implementation.

safinaskar · 2024-10-13T04:32:38Z

🚀 I found a way to evaluate concat_idents (and concat and few other built-in macros) before evaluating other macro, which takes concat_idents as an argument! I. e. I found a way to evaluate a!(concat_idents!(...)) such way, that concat_idents evaluates before a. Answer is crate https://crates.io/crates/with_builtin_macros !!! Thanks, @danielhenrymantilla ! In other words, with_builtin_macros is paste, but not only for concat_idents, but also for concat and some other macros.

And in other words, with_builtin_macros allows one to achieve eager evaluation of macros in limited way.

Also, with_builtin_macros allows one to use concat_idents when defining new identifier.

Also, https://crates.io/crates/with_builtin_macros allows one to use concat_idents in stable Rust.

// (This code was not tested, may contain typos)

fn concat_idents!(a, b) () {} // Doesn't work

with_builtin_macros::with_eager_expansions! {
  fn #{ concat_idents!(a, b) } () {} // Works! Even on stable!
}

macro_rules! this_macro_accepts_ident {
  ($a:ident) => {}
}

// Doesn't work, because "this_macro_accepts_ident" evaluates before "concat_idents"
this_macro_accepts_ident!(concat_idents!(a, b));

with_builtin_macros::with_eager_expansions! {
  this_macro_accepts_ident!(#{ concat_idents!(a, b) }); // Works! Even on stable!
}

macro_rules! this_macro_accepts_literal {
  ($a:literal) => {}
}

// Doesn't work.
// Moreover, you cannot solve this problem using #[feature(macro_metavar_expr_concat)],
// because ${concat(...)} produces identifier, not string literal!!!
// Same applies to "paste"! "paste::paste!" deals with identifiers, not strings. So, with_builtin_macros is the only way!!!
this_macro_accepts_literal!(concat!("a", "b"));

with_builtin_macros::with_eager_expansions! {
  this_macro_accepts_literal!(#{ concat!("a", "b") }); // Works! Even on stable!
}

c410-f3r added the C-tracking-issue Category: A tracking issue for an RFC or an unstable feature. label Apr 21, 2024

fmease mentioned this issue Apr 21, 2024

Tracking issue for concat_idents #29599

Open

fmease added the I-lang-nominated Nominated for discussion during a lang team meeting. label Apr 21, 2024

junyang-zh mentioned this issue Apr 22, 2024

Improve shared locks (Arc<*Lock>) by allowing dynamically borrowed lock guards asterinas/asterinas#778

Merged

joshtriplett removed the I-lang-nominated Nominated for discussion during a lang team meeting. label May 15, 2024

tgross35 mentioned this issue May 22, 2024

Add a new concat metavar expr #118958

Merged

c410-f3r mentioned this issue Jun 22, 2024

[macro_metavar_expr_concat] Add support for literals #126841

Merged

c410-f3r mentioned this issue Jul 10, 2024

[macro_metavar_expr_concat] Add support for literals #127542

Merged

c410-f3r mentioned this issue Jul 14, 2024

[macro_metavar_expr_concat] Allow concat in repetitions #127720

Merged

Noratrieb mentioned this issue Jul 14, 2024

Macro metavar expr concat doesn't work with nested repetitions #127723

Open

c410-f3r mentioned this issue Aug 1, 2024

[macro_metavar_expr_concat] Dogfooding #128491

Merged

Alexendoo mentioned this issue Sep 6, 2024

Turn declare_clippy_lint into a declarative macro rust-lang/rust-clippy#13359

Merged

aschaeffer mentioned this issue Sep 14, 2024

Migrate to Rust Stable reactive-graph/reactive-graph#102

Closed

4 tasks

crumblingstatue mentioned this issue Oct 8, 2024

ICE in macro_metavar_expr_concat #131393

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tracking Issue for `macro_metavar_expr_concat` #124225

Tracking Issue for `macro_metavar_expr_concat` #124225

c410-f3r commented Apr 21, 2024 •

edited by tgross35

Loading

fmease commented Apr 21, 2024 •

edited

Loading

c410-f3r commented Apr 21, 2024

fmease commented Apr 21, 2024

fmease commented Apr 21, 2024 •

edited

Loading

joshtriplett commented May 15, 2024

c410-f3r commented May 15, 2024

UserIsntAvailable commented Jun 15, 2024

c410-f3r commented Jun 15, 2024

UserIsntAvailable commented Jun 16, 2024 •

edited

Loading

$ident:modifier

crumblingstatue commented Jun 20, 2024 •

edited

Loading

c410-f3r commented Jun 21, 2024

tgross35 commented Jul 18, 2024

tgross35 commented Jul 18, 2024 •

edited

Loading

safinaskar commented Oct 13, 2024 •

edited

Loading

Tracking Issue for macro_metavar_expr_concat #124225

Tracking Issue for macro_metavar_expr_concat #124225

Comments

c410-f3r commented Apr 21, 2024 • edited by tgross35 Loading

About experimental features

About tracking issues

Steps / History

Unresolved Questions

References

fmease commented Apr 21, 2024 • edited Loading

c410-f3r commented Apr 21, 2024

fmease commented Apr 21, 2024

fmease commented Apr 21, 2024 • edited Loading

joshtriplett commented May 15, 2024

c410-f3r commented May 15, 2024

UserIsntAvailable commented Jun 15, 2024

c410-f3r commented Jun 15, 2024

${case}

$ident:modifier

UserIsntAvailable commented Jun 16, 2024 • edited Loading

$ident:modifier

crumblingstatue commented Jun 20, 2024 • edited Loading

c410-f3r commented Jun 21, 2024

tgross35 commented Jul 18, 2024

tgross35 commented Jul 18, 2024 • edited Loading

safinaskar commented Oct 13, 2024 • edited Loading

Tracking Issue for `macro_metavar_expr_concat` #124225

Tracking Issue for `macro_metavar_expr_concat` #124225

c410-f3r commented Apr 21, 2024 •

edited by tgross35

Loading

fmease commented Apr 21, 2024 •

edited

Loading

fmease commented Apr 21, 2024 •

edited

Loading

UserIsntAvailable commented Jun 16, 2024 •

edited

Loading

crumblingstatue commented Jun 20, 2024 •

edited

Loading

tgross35 commented Jul 18, 2024 •

edited

Loading

safinaskar commented Oct 13, 2024 •

edited

Loading