proposal: Go 2: Printf unspecific string interpolation evaluating to string and list of expressions

[Cookie04DE]

This proposal is meant to be an alternative for, not a replacement of #50554.
#50554 attempts to find a mechanism that integrates directly into Printf style functions. This proposal aims for a broader scope which breaks this compatibility but provides greater value overall. It also tries to integrate the previous discussion and suggested improvements.
Like #50554 it tries to maintain the separation between the spec and the standard library.

Note that this necessitates the use of different functions. Since I don't have a good name for them yet, I just add New to their name. So fmt.Printf becomes fmt.PrintNewf and log.Fatalf becomes log.FatalNewf. That's just for demonstration purposes though and these need better names.

Author background

• Would you consider yourself a novice, intermediate, or experienced Go programmer?
  Experienced.
• What other languages do you have experience with?
  Java, Kotlin, Rust, C

Related proposals

• Has this idea, or one like it, been proposed before?
  proposal: Go 2: string interpolation #34174 attempts to provide string interpolation at the language level (the specific formatting of each type will be recorded in the language spec)
  proposal: Go 2: string interpolation evaluating to string and list of expressions #50554 attempts to provide a feature directly compatible with Printf style functions to aid in readability.
  • If so, how does this proposal differ?
    Unlike proposal: Go 2: string interpolation #34174 this proposal does not specify the formatting of each type and lets the user decide (even applications other than string formatting are possible; more on that later).
    Unlike proposal: Go 2: string interpolation evaluating to string and list of expressions #50554 the language level functionality is not directly compatible with Printf style functions but aims to provide a broader, more useful functionality.
• Does this affect error handling?
  No.
• Is this about generics?
  No.

Proposal

• What is the proposed change?
  Please refer to the section after next.
• Who does this proposal help, and why?
  Traditional C style format strings are easy, robust and well known, but they suffer from one main problem: It is hard to read and modify them.
  Consider the following:

fmt.Printf("Hello %s, as requested here is your notification that %s is back in stock now (%d). If you wish to purchase now you may click this: %s.\nIf you no longer wish to receive these notifications please click here: %s\n", name, product, buyURL, stockCount, unsubscribeURL)

Did you catch that buyURL and stockCount were accidentally swapped?
It is also difficult to modify it correctly. If you want to change the position of an expression you need to move the formatting verb (%v for example) and the expression in the list so the order is correct again. Or if you want to delete an expression you need to delete the marker in the string and find the correct expression in the list and delete that too.
The problem becomes worse the bigger the format string.

This is how the (corrected) example would look like with the new syntax:

fmt.PrintNewf(%"Hello {name}, as requested here is your notification that {product} is back in stock now ({stockCount:d}). If you wish to purchase now you may click this: {buyURL}.\nIf you no longer wish to receive these notifications please click here: {unsubscribeURL}\n")

The proposed changes could also be used for other purposes.

Example using current syntax:

package main

import (
	"context"
	"log"

	"github.com/jackc/pgx/v4"
)

func main() {
	db, err := pgx.Connect(context.Background(), "some connect url")
	if err != nil {
		log.Fatalf("Error connecting to database: %s", err)
	}
	const userID, userName = 15, "harry"
	var age int
	// this creates a prepared statement (injection safe) on the database server and executes it with the data provided (15 and "harry"); $1 is 15 and $2 is "harry"
	if err = db.QueryRow(context.Background(), "SELECT age FROM user WHERE id=$1 AND username=$2", userID, userName).Scan(&age); err != nil {
		log.Fatalf("Error getting age for user %s (id: %d): %s", userName, userID, err)
	}
	log.Printf("The user with the username %s and the id %d is %d years old!", userName, userID, age)
}

This could be rewritten into this:

package main

import (
	"context"
	"log"

	"github.com/jackc/pgx/v4"
)

func main() {
	db, err := pgx.Connect(context.Background(), "some connect url")
	if err != nil {
		log.FatalNewf(%"Error connecting to database: {err}")
	}
	const userID, userName = 15, "harry"
	var age int
	// this creates a prepared statement (injection safe) on the database server and executes it with the data provided (15 and "harry"); The positional parameters ($1 and $2) are inserted automatically by the function
	if err = db.QueryNewRow(context.Background(), %"SELECT age FROM user WHERE id={userID} AND username={userName}").Scan(&age); err != nil {
		log.FatalNewf(%"Error getting age for user {userName} (id: {userID}): {err}")
	}
	log.PrintNewf("The user with the username {userName} and the id {userID} is {age} years old!")
}

• Please describe as precisely as possible the change to the language.
  I propose to add the following builtin type:

type fsArgument struct {
	Pos        int
	Value      any
	Expression string
	Tag        string
}

Furthermore I propose to add two types of literals. I will refer to them as "format string" (fs for short) and "raw format string" (rfs for short) henceforth.
An fs has the following form:

%"Hello {name} (age {age:d}), enter \"\{\" to exit this prompt!\n> "

It behaves very similar to string literals with the exception that it has to be prefixed with % and may contain Go expressions contained within curly brackets ({ and }).
The brackets have to match up, so there may be no closed curly brackets without corresponding opening brackets and the other way around.

If a literal open curly bracket or a closed curly bracket is desired the following two escape sequences are valid inside an fs: \{ and \}. They are only valid in the string portion of an fs. Using them anywhere else results in a syntax error.

The curly brackets may also contain a tag that is separated from the expression with a colon (:). The tag is optional but if a colon is present it may not be empty.

The fs above evaluates to the following expression:

"Hello  (age ), enter \"{\" to exit this prompt!\n> ", []fsArgument{
		{
			Pos:        6,       // the byte offset inside the string
			Value:      "Peter", // the value of the expression (the variable name contained the string "Peter")
			Expression: "name",  // the text of the expression as the compiler sees it
			Tag:        "",      // there is no tag
		},
		{
			Pos:        12,
			Value:      25,
			Expression: "age", // the text of the expression as the compiler sees it
			Tag:        "d",   // the text of the tag (the text after the colon until the closed curly bracket)
		},
	}

The returned string contains everything besides the curly brackets and their contents.
The expressions are returned inside the slice of fsArgument's. The length of this slice may be 0 but the slice may not be nil.
The expressions are evaluated in the order they appear in the fs.

Similarly how fs's mirror string literals rfs's mirror raw string literals. rfs's may span multiple lines and contain expressions and tags the same way fs's do. They can however not use escape sequences the same way raw string literals can't. The same example above but using a rfs looks like this:

%`Hello {name} (age {age:d}), enter "{"{"}" to exit this prompt!
> `

"{"{"}" is a little confusing due to the lack of syntax highlighting so I will explain it going from the outside inwards:

• "" are literal double quotes
• {} introduce an fs argument
• "" is a string literal
• { is an open curly bracket inside the string literal

This is necessary to recreate the original fs example exactly and because the escape sequence for a literal open curly bracket is not usable in an rfs.

It evaluates to the following:

`Hello  (age ), enter "" to exit this prompt!
	> `, []fsArgument{
		{ // identical to the fs example
			Pos:        6,
			Value:      "Peter",
			Expression: "name",
			Tag:        "",
		},
		{
			Pos:        12,
			Value:      25,
			Expression: "age",
			Tag:        "d",
		},
		{ // this is where the lack of escape sequences makes a difference
			Pos:        22,
			Value:      "{",     // this is the open curly bracket we want to insert into the double quotes to tell the user how to exit the prompt
			Expression: "\"{\"", // this is the string literal evaluating to the above
			Tag:        "",      // no tag present
		},
	}

• What would change in the language spec?
  It would include the new builtin type and the two new literal's. It is not necessary however to specify the way every type get's formatted or even mention the fmt package.
• Please also describe the change informally, as in a class teaching Go.
  C style formatting is easy and well known but it can get awkward really quick and become difficult to read and maintain. Since there is a large gap between where you insert a value and actually provide it you have to constantly switch between the actual string and the argument list. To solve this you can use Go's format strings. They provide the values in the string itself so you can easily see where a value is inserted, what is is and can easily change it's location.
• Is this change backward compatible?
  Not entirely. Since a new builtin type is added there might be conflict if there is already is another type with the same name.
  • Breaking the Go 1 compatibility guarantee is a large cost and requires a large benefit.
    Show example code before and after the change.
  • Before

    fmt.Printf("Hello %s, as requested here is your notification that %s is back in stock now (%d). If you wish to purchase now you may click this: %s.\nIf you no longer wish to receive these notifications please click here: %s\n", name, product, stockCount, purchaseURL, unsubscribeURL)

  • After

    fmt.SomeNewF(%"Hello {name}, as requested here is your notification that {product} is back in stock now ({stockCount}). If you wish to purchase now you may click this: {purchaseURL}.\nIf you no longer wish to receive these notifications please click here: {unsubscribeURL}\n")

• Orthogonality: how does this change interact or overlap with existing features?
  It provides an alternative to format strings (which is a library feature) but this is a change on the language level and can support a broader use case.
• Is the goal of this change a performance improvement?
  No.

Costs

• Would this change make Go easier or harder to learn, and why?
  Harder, since it's a new feature and every feature adds complexity and learn time.
  However it might make it easier for programmers used to a language with string interpolation to pick up Go since they can use a similar feature instead of needing to learn C style format strings.
• What is the cost of this proposal? (Every language change has a cost).
  The compiler needs to be able to recognize the new syntax and compile it appropriately. The tooling needs to recognize it as well to provide syntax highlighting, formatting and auto completion.
• How many tools (such as vet, gopls, gofmt, goimports, etc.) would be affected?
  All of them need to recognize the new syntax. gofmt in particular needs to be modified to format the expressions inside the fs's.
• What is the compile time cost?
  The compiler needs to be able to recognize the new syntax and compile it correctly. This should however not be a huge cost.
• What is the run time cost?
  There is no additional run time cost. The change is cosmetic.
• Can you describe a possible implementation?
  The compiler already includes most components to implement the new syntax. It needs new "context"'s for fs's and rfs's to recognize the two new escape sequences and check the expressions inside the curly brackets. It also needs to expand them to their full form.
• Do you have a prototype? (This is not required.)
  Not yet.

[au-phiware]

I don't understand the need for the "New" methods, you can do this:

• Before

fmt.Printf("Hello %s, as requested here is your notification that %s is back in stock now (%d). If you wish to purchase now you may click this: %s.\nIf you no longer wish to receive these notifications please click here: %s\n", name, product, stockCount, purchaseURL, unsubscribeURL)

• After

fmt.Print(%"Hello {name}, as requested here is your notification that {product} is back in stock now ({stockCount}). If you wish to purchase now you may click this: {purchaseURL}.\nIf you no longer wish to receive these notifications please click here: {unsubscribeURL}\n")

[hellojukay]

This is not necessary, it will make the programming style confusing and will increase the complexity of understanding the program

[Cookie04DE]

@au-phiware For this proposal we need a different function, since Printf takes string, any... or any... for Print but %"..." evaluates to string, []fsArgument. If you want a Printf compatible proposal, please see #50554
@hellojukay I respect your opinion but I disagree. C style format strings are fine for small strings but get progressively harder to read and maintain the longer they are.
I think readability and maintainability is a corner stone of Go so I wanted to do something about that.
If you agree with that but think my solution is bad please let me know. I'd love to hear an alternative solution.

[ianlancetaylor]

Thanks. This seems quite similar to #34174 and #50554, different only in details. I don't think we need to keep multiple similar issues open on this topic. Closing as a dup.

[Cookie04DE]

I disagree that this proposal is a duplicate of #50554 because this proposal is incompatible with Printf style functions, in contrast to #50554, while adding the ability to do something similar to python's f"{variable=}" due to the fact that it includes the text of the expression and is generally more versatile and useful.
If you insist that #50554 is a duplicate, I would kindly ask you to close the former and reopen this proposal, since comments on #50554 indicate, at least to me, that support for this proposal is more likely than for the former.

[ianlancetaylor]

@Cookie04DE I don't think it helps us to have multiple different discussions about approaches to string interpolation. That's hard to follow and hard to remember all the places to look. I think it's more useful to have a single issue that discusses different approaches in the hopes of building consensus. Thanks.