Propose implicit named arguments for formatting macros

[davidhewitt]

Rendered

Proposes the ability to pass implicit named arguments to formatting macros, for example:

let person = "Charlie";
print!("Hello, {person}!");    // implicit named argument `person`

A follow up to internals discussion https://internals.rust-lang.org/t/println-use-named-arguments-from-scope/10633 and WIP PR rust-lang/rust#65338

[kennytm]

An alternative syntax is to clearly distinguish the interpolated variable from named arguments,

println!("Hello, {(person)}!");
//                ^      ^

and it allows all expressions can be naturally used

format_args!("Hello {(get_person())}")
//                   ^~~~~~~~~~~~~^

println!(r#"{("Hello"):~^33}\n{(self.person):~^33}"#);

println!("Hello, {(if self.foo { &self.person } else { &self.other_person })}");

println!(
    "variable-person = {(person1)}, named-person = {person2}, positional-person = {}",
    person3,
    person2=person2,
);

[CAD97]

and it allows all expressions can be naturally used

First: this is backwards compatible to add on top of the "single Ident special case". Most languages with string interpolation have a special case for single Ident, so we wouldn't be out of line to make that easier. It's also by far the most common case.

Second: allowing arbitrary expressions opens a whole separate can of worms around nesting. Just as a few edge case examples: "{("a")}", "{(\"a\")}", r#"{(")}")}"#. You can make rules for how these resolve (it'd have to be "standard string token first, then work on the content of the string literal"), but this adds a huge number of edge cases that just plain don't exist for the ""{name}" is a shortcut for "{name}", name=name" simple case.

So we definitely should stick to the simple, fairly obvious case that eliminates that stutter, and worry about generalizing it to more expressions later if desired.

[kennytm]

@CAD97

Most languages with string interpolation have a special case for single Ident

Half of the languages with string interpolation have special case for single ident.

Firstly, many of them do support interpolating arbitrary expressions, without any special case for single idents:

• Python's f-string f"{expr}"
• C#/VB's interpolated strings $"{expr}"
• JavaScript/TypeScript's template literals `${expr}`
• Swift's string interpolation "\(expr)"

Furthermore, when the string interpolation syntax is special-cased for single idents, it is often because there is no closing sigil, which is not the case for Rust

• Kotlin, Groovy, PHP: "$ident ${expr}"
• Ruby: "#@xxx #@@yyy #$zzz #{expr}" (note: local variables (no sigils) won't work, must use "#{ident}")
• Scala: s"$ident ${expr}"

Finally, string interpolation is a feature built into the language itself, which often live independently from the "format" facility, e.g. in Python you write f"{ident}" or "{}".format(ident), but "{ident}".format() is meaningless. This RFC fuses string formatting and interpolation, so the experiences from other languages are not reliable, e.g. they can't express format!("{a} {b}", b=123).

And hence I disagree with this criticism:

Second: allowing arbitrary expressions opens a whole separate can of worms around nesting. Just as a few edge case examples: "{("a")}", "{(\"a\")}", r#"{(")}")}"#. You can make rules for how these resolve (it'd have to be "standard string token first, then work on the content of the string literal"), but this adds a huge number of edge cases that just plain don't exist for the ""{name}" is a shortcut for "{name}", name=name" simple case.

"{("a")}" is simply syntax error. Remember we're talking about a macro format!("...") here, the format string literal is not special in the Rust tokenizer/parser, it is not an interpolated string. To the proc macro format_args!(), the format strings "{(\")}\")}" and r#"{(")}")}"# are equivalent. The format string parser could,

• If we see {,
  • If we see {, unescape as {
  • If we see an integer, it's a positional argument
  • If we see an identifier, it's a named argument
  • If we see : or }, it's an (implicit) positional argument
  • If we see (, step back one character then parse a Rust token tree
  • Otherwise, emit an error

I don't see "huge number of edge cases" syntactically. IMO it has less edge cases than the current RFC because when you see an ident, you don't need to determine whether {ident} is a named argument or interpolated variable!

And to clarify, introducing the "stutter" is the primary purpose of my comment above even if we reject interpolating arbitrary expressions. It's to ensure {ident} is always a named argument, which leads to

this is backwards compatible to add on top of the "single Ident special case".

Indeed we accepted this RFC as-is, {ident:?} can also be forward extended to {(expr):?}. But you'll need to consider what to do with {expr:?} — either reject all non-ident expressions (thus {self.prop:?} is invalid), or accept some expressions (which leads to "why is my particular expression not accepted?" and true edge cases involving numbers and : and {). So I'd avoid even allowing that in the first place.

---

It's also by far the most common case.

Yes. But stuff like self.stuff or path.display() is also quite common. Grepping format!, panic! and (e)print(ln)! (including docs) from https://github.com/BurntSushi/ripgrep we get

Type	Count
Single ident (e.g. format!("error: {}", err))	147
Properties (e.g. format!("value {}", self.x.y.z)	22
Other expressions (e.g. format!("{}", xyz[0].replace("\n", "\n\n")))	61
Expression with return branch (e.g. println!("{}", foo()?);)	2
Expression involving macros (e.g. format!("{} {}", $e, crate_version!()))	3

So non-single-ident covers 40% of all usages, which I won't brush them off simply as can-of-worms opener.

Also, when these expressions are interpolated, they are quite readable unlike the RFC's constructed example.

https://github.com/BurntSushi/ripgrep/blob/8892bf648cfec111e6e7ddd9f30e932b0371db68/tests/util.rs#L403-L414

// Current
            panic!("\n\n==========\n\
                    command failed but expected success!\
                    {}\
                    \n\ncommand: {:?}\
                    \ncwd: {}\
                    \n\nstatus: {}\
                    \n\nstdout: {}\
                    \n\nstderr: {}\
                    \n\n==========\n",
                   suggest, self.cmd, self.dir.dir.display(), o.status,
                   String::from_utf8_lossy(&o.stdout),
                   String::from_utf8_lossy(&o.stderr));

// Interpolated
            panic!("\n\n==========\n\
                    command failed but expected success!\
                    {(suggest)}\
                    \n\ncommand: {(self.cmd):?}\
                    \ncwd: {(self.dir.dir.display())}\
                    \n\nstatus: {(o.status)}\
                    \n\nstdout: {(String::from_utf8_lossy(&o.stdout))}\
                    \n\nstderr: {(String::from_utf8_lossy(&o.stdout))}\
                    \n\n==========\n");

https://github.com/BurntSushi/ripgrep/blob/8892bf648cfec111e6e7ddd9f30e932b0371db68/ignore/src/lib.rs#L287-L299

// Current
                write!(f, "{}", msgs.join("\n"))
...
                write!(f, "File system loop found: \
                           {} points to an ancestor {}",
                          child.display(), ancestor.display())

// Interpolated
                write!(f, r#"{(msgs.join("\n"))}"#)
...
                write!(f, "File system loop found: \
                           {(child.display())} points to an ancestor {(ancestor.display())}")

https://github.com/BurntSushi/ripgrep/blob/8892bf648cfec111e6e7ddd9f30e932b0371db68/src/search.rs#L244-L264

// Current
        write!(
            self.get_mut(),
            "
{matches} matches
{lines} matched lines
{searches_with_match} files contained matches
{searches} files searched
{bytes_printed} bytes printed
{bytes_searched} bytes searched
{search_time:0.6} seconds spent searching
{process_time:0.6} seconds
",
            matches = stats.matches(),
            lines = stats.matched_lines(),
            searches_with_match = stats.searches_with_match(),
            searches = stats.searches(),
            bytes_printed = stats.bytes_printed(),
            bytes_searched = stats.bytes_searched(),
            search_time = fractional_seconds(stats.elapsed()),
            process_time = fractional_seconds(total_duration)
        )

// Interpolated
        write!(
            self.get_mut(),
            "
{(stats.matches())} matches
{(stats.matched_lines())} matched lines
{(stats.searches_with_match())} files contained matches
{(stats.searches())} files searched
{(stats.bytes_printed())} bytes printed
{(stats.bytes_searched())} bytes searched
{(fractional_seconds(stats.elapsed())):0.6} seconds spent searching
{(fractional_seconds(total_duration)):0.6} seconds
")

[danielhenrymantilla]

As a comparison point, there is a substantial difference between the mentioned ::fstrings crate, featuring deliberately limited interpolation (c.f., danielhenrymantilla/fstrings-rs#1 (comment)), and the ::ifmt crate, featuring "arbitrary" interpolation.

In the linked ::fstrings PR, extending fstrings interpolation to field access is being considered, but nothing more (e.g., no method calls a priori). Imho this leads to a sweet point in letting people use interpolation while preventing abuse of the feature.

[davidhewitt]

I think @kennytm's interpolation proposal is reasonable, and I will add it to the alternatives section in the RFC shortly.

My initial observations about the proposed syntax for interpolation are that it's not actually any shorter than positional formatting arguments, although I would argue it reads easier (especially as the number of arguments scales):

println!("Hello, {(person)}!");
println!("Hello, {}!", person);

However, if the brackets are necessary to resolve parsing ambiguities then that is something we would have to accept. I do find iterpolation mechanisms convenient, though it can be difficult to judge how complex interpolations should be before they become hard-to-read / poor style.

I would like to contest this line though:

This RFC fuses string formatting and interpolation

The objective of this RFC is not to add interpolation to Rust's existing formatting macros. It is to add a shorthand to the macros to make them more ergonomic in a very common use case. I'd be glad to take a win on 60% of macro invocations even if this RFC doesn't improve the remaining 40%. This discussion appears to agree that interpolation can be added later in a backwards-compatible way (even if there are different arguments whether that would be desirable).

As pointed out by @petrochenkov in the WIP PR, we actually already have a similar shorthand in struct literal expressions for the special case of single identifiers:

struct Foo { bar: u8 }
let bar = 1u8;

let foo = Foo { bar: bar };
let foo = Foo { bar };        // This shorthand only accepts single identifiers

To make the original intention of the RFC stand out clearer, I will update the wording to state explicitly that adding interpolation to the macros was not the intended goal, as well as add the struct literal shorthand to the prior art section.

[danielhenrymantilla]

I like the nuances you've added.

Could you also state whether format!("{x}, {y}", y = 42) would be accepted or rejected? (it would avoid having let declarations used only for one "fstring" interpolation)

[davidhewitt]

Could you also state whether format!("{x}, {y}", y = 42) would be accepted or rejected? (it would avoid having let declarations used only for one "fstring" interpolation)

I would expect it would be accepted: it seems perfectly valid to me that one named parameter can be implicit and the other explicit.

[Centril]

Also, when these expressions are interpolated, they are quite readable unlike the RFC's constructed example.

I think there's value in separating the computing of results and how they are rendered (so that the former can be separated into functions eventually). With the {binding} case, I think a good balance is struck where things don't get too intermingled.

Also, given that 60% of the cases in the data in question were for single-ident cases, it seems to me that {binding} is right to optimize for those cases whereas {{binding}} optimizes for the minority.

The rustc compiler has a lot of diagnostics so it is probably a better data set to dig into if you want more of those.

All in all, I think this RFC finds a well-struck balance.

[tomwhoiscontrary]

Would this involve any changes to the language syntax, or can this be implemented entirely in a macro?

I realise that format_args! is not currently implemented as a true macro, but i think it's a good idea to maintain the fiction that it is, or at least could be. format_args! shouldn't do anything a user-written macro can't.

[davidhewitt]

Would this involve any changes to the language syntax, or can this be implemented entirely in a macro?

The fstrings crate implements a PoC of this functionality using proc_macro (and proc_macro_hack, I believe). So this can indeed be implemented in a user-written macro.

Also, the test PR I wrote needed just a tiny adjustment to the internals of format_args! without any changes anywhere else.

[joshtriplett]

I'd love to have this. I find myself writing name=name, another_name=another_name all the time.

I fully support the rationale in this RFC to only support single identifiers, and defer any more complex interpolation. I do think we'll want dotted fields in the future, but I added a comment about one bit of complexity even that level of extended interpolation would produce.

[tmccombs]

I have a couple of reasons to not support arbitrary expressions without require parenthesis ( "{(expr)}"):

1. There is ambiguity between an integer expression and number indicating a positional argument
2. There could potentially be ambiguity, or at least confusing with format arguments, since the colon is used both in rust syntax and as the delimiter before format arguments.

[davidhewitt]

I have a couple of reasons to not support arbitrary expressions without require parenthesis ("{(expr)}"):

Fully agree with your arguments. For anything beyond {ident}, I agree {(expr)} is much better than {expr}.

[danielhenrymantilla]

A new version of ::fstrings has just been released, that supports interpolation of field accesses / dotted.paths (but not after the :, which is currently just left as is).

It can be useful for those wanting to get a feeling of what the ergonomic improvements this RFC and a future one could provide.

[tmccombs]

Another possible alternative: No special case for identifiers, but allow arbitrary expressions within parenthesis. So, {ident} means, use the keyword argument named ident and {(ident)} means use the variable ident, and {(expr)} works for any expression.

[rfcbot]

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

The RFC will be merged soon.

[davidhewitt]

Very excited to see this given a chance! I'm happy to do the implementation work for this going forward.

[davidhewitt]

Checklist for the tracking issue:

Steps:

• Implementation for format_args! behind format_implicit_args feature (name to be bikeshed?)
• Implement alternative solution for format_args!(concat!(...)) case that doesn't risk spurious macro hygiene
• Implementation for panic!. Behind feature gate and / or edition switch (see unresolved question)
• Stabilize (may depend on new edition)
• Documentation

Unresolved Questions:

• Solution for format_args!(concat!(...)) - perhaps try out a new unstable macro as per this comment
• Final design of the panic! solution - perhaps based off this comment

[nikomatsakis]

Huzzah! The @rust-lang/lang and @rust-lang/libs teams have decided to accept this RFC.

To track further discussion, subscribe to the tracking issue here:

rust-lang/rust#67984

[Arnavion]

The RFC doesn't document how it interacts with raw idents, so for posterity:

let r#type = 5;

// Before this feature
assert_eq!("5", format!("{}", r#type));
assert_eq!("6", format!("{type}", type = 6)); // Named explicit does not need to be raw.
assert_eq!("5", format!("{type}", type = r#type));
assert_eq!("5", format!("{type}", r#type = r#type)); // Named explicit can be raw and maps to cooked version in the format string.

println!("{type}", type = type); // Error. RHS is an invalid expr.
println!("{r#type}", r#type = r#type); // Error. Named explicit can be raw but the format string can only use the cooked version.

// New after this feature
assert_eq!("5", format!("{type}")); // Format string references cooked ident and value is from raw ident.
                                    // ie compiler is smart enough to treat this as `type = r#type` and not `type = type`

println!("{r#type}"); // Still an error. Format string can only use the cooked version.
                      // So `"{type}"` is the *only* way to write this.