proposal: Go 2: deferred code blocks

[83naff]

What version of Go are you using (go version)?

$ go version
go version go1.14.2 linux/amd64

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/damien/.cache/go-build"
GOENV="/home/damien/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/damien/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build816314905=/tmp/go-build -gno-record-gcc-switches"

Proposal: Deferred code blocks

Currently in Go, we can defer function calls like this:

defer f()

However, the problem with this is that we are unable to capture the return values of f and use them, so what someone like myself ends up doing is this:

defer func() { 
   result := f()
   // do something with result
}()

Although it works, it's not exactly the best way to go about it; we could call f() at the end of the function without using defer then use the results as required.

What I thus propose is that we introduce deferred code blocks which will have this syntax:

defer {
   // do this at the end of the enclosing function
}

What this will do is execute the entire code block at the end of the function in which 'defer' is used in. Each line will in the code block will then be run sequentially. To give a more realistic example of where this would be useful:

f, err := os.Create("file.txt")

if err != nil {
   panic(err)
}

defer {
   err := f.Close()

   if err != nil {
      panic(err)
   }
}

As opposed to the current way of going about this:

f, err := os.Create("file.txt")

if err != nil {
   panic(err)
}

defer func() {
   err := f.Close()

   if err != nil {
      panic(err)
   }
}()

In some respect, you could argue that it's merely cosmetic since all the defer { } is doing is making the otherwise defer func () {}() look neater.

Thoughts?

[mvdan]

So this proposal is simply to avoid func() and ()? Personally, it doesn't seem worth the high cost that all language syntax changes come with. Also, defer has a nice consistency now, just like go, where it's always followed by a function call. You would break that consistency, and thus break lots of tools too.

[83naff]

@mvdan That's not correct I'm afraid since introducing defer { } doesn't change the current nature of defer as it is. We're only augmenting what is already provided, not diminishing it. As it stands, the defer { } is only a shortcut for defer func () {} () which allows sequential statements of code to have its execution deferred to the end of the function.

[mvdan]

Yes, you're adding more ways to write Go code. Which, as I explained, has a high cost - any static analysis tools that currently examine defer statements would need to be updated, for example. This is why the language spec is almost never touched, and even less so for syntax changes.

[mvdan]

Also, please fill https://github.com/golang/proposal/blob/master/go2-language-changes.md.

[83naff]

Having considered the above document before raising this proposal, I still believe this is a worthy addition to Go. As it stands, this proposal is not:

1. Asking for the current 'defer' mechanism to be changed
2. Preventing people from using defer the usual way

Whilst I appreciate your concern for the integrity of the language spec, it remains an important issue in any language, not just Go, for the spec to always evolve according to the user base whilst maintaining compatibility with previous versions. Although, if you are still sceptical about this proposal, @mvdan, then your opinions are duly noted.

[ianlancetaylor]

For language change proposals, please fill out the template at https://go.googlesource.com/proposal/+/refs/heads/master/go2-language-changes.md .

When you are done, please reply to the issue with @gopherbot please remove label WaitingForInfo.

Thanks!

[pjebs]

I believe Swift's defer operates as proposed.

If you want to try it out you can fork my package: https://github.com/rocketlaunchr/igo and make a few adjustments and quickly implement what you are proposing.

[83naff]

Here's the template that I've been asked to produce:

Would you consider yourself a novice, intermediate, or experienced Go programmer?
Intermediate.

What other languages do you have experience with?
I also have experience with C++, Java, Python and Perl

Would this change make Go easier or harder to learn, and why?
It will neither make Golang easier nor harder as extending the defer to encapsulate code will not break the existing arrangements in place for 'defer' as it stands. Although, those who might want to use the new proposal if it gets implemented might find it easier to work with than chaining defer statements. By issuing code inside one defer statement, it becomes more legible and obvious in the order the statements will execute.

Has this idea, or one like it, been proposed before?
Not to my knowledge

If so, how does this proposal differ?
N/A

Who does this proposal help, and why?
It would benefit anyone who uses defer statements. With this extension, it will allow the programmer to easily check for errors within the deferred codeblocks rather than wrapping each statement in a function and then checking for error messages whilst each line is deferred.

What is the proposed change?
Please see my first post above. In effect, it is augmenting the functionality of the 'defer' to wrap code intended for execution at the end of the enclosing function.

Please describe as precisely as possible the change to the language.
Please see my first post.

What would change in the language spec?
Only that the defer statement, as well as being able to be used for one line of code, can now wrap code within braces for which each line in the braces will be executed at the end of the enclosing function.

Please also describe the change informally, as in a class teaching Go.
Hello, young padowans! Did you know you can also wrap code within a defer statement too? Instead of writing multiple defer statements, you can now include them within the braces like so:

defer {
   // line 1
   // line 2
}

Doing it this way means you can now do the error checking inside the defer statement, without having to encapsulate that line in a function call! How awesome is that?

Is this change backward compatible?
Breaking the Go 1 compatibility guarantee is a large cost and requires a large benefit.
This change is indeed backwards compatible as it does not break the current usage of defer; it only augments it to optionally include braces where code resides in.

Show example code before and after the change.
Please see my first post.

What is the cost of this proposal? (Every language change has a cost).
I'm not sure what this means, sorry.

How many tools (such as vet, gopls, gofmt, goimports, etc.) would be affected?
Gopls is mostly concerned with the symbols within a language which include variables, functions and objects to name a few. I'm not quite sure about gopls implementation as I've not used it, but if it performs checks on 'defer' then it would also need to extend those checks to cover the proposed usage of defer as well.

Gofmt would need to be amended to format code within the braces accordingly so that it is indented.

Goimports is unaffected.

With go vet, this proposal if implemented can mean defer codeblocks can be empty (defer { }) so we should add a check to ensure there are no empty defer codeblocks.

What is the compile time cost? What is the run time cost?
As it is a shorthand of writing defer func () {}(), the runtime and compile time cost should be the same as though you're writing the aforementioned.

Can you describe a possible implementation?
Not this early on, no.

Do you have a prototype? (This is not required.)
@pjebs I have taken a look at your repository, and I'll definitely take a look at using it to build something once time allows. Thanks for the tip :-)

How would the language spec change?
This is a repeat of an earlier question I answered. Ignored.

Orthogonality: how does this change interact or overlap with existing features?
As it is still early-days, it's unclear how this change will overlap with existing features. Briefly though, it only affects the functionality surrounding the current usage of 'defer'. Where features are concerned with that, those would be affected by this change. Naturally, it is not aiming at breaking the existing usage, but solely augmenting it.

Is the goal of this change a performance improvement?
If so, what quantifiable improvement should we expect?
Potentially as it would spare having to create a function and then deferring its call. It also means we can catch errors sooner and handling the errors sooner which will halt execution earlier. This in effect is a performance improvement.

How would we measure it?
We can benchmark the current arrangements with 'defer' with the augmented proposal. We'll compare the benchmarks of the current 'defer' before the change and then after the change. For the latter, we'll also include benchmarks for the proposed defer as well.

Does this affect error handling?
If so, how does this differ from previous error handling proposals?
It does as this means we can now check for errors within the deferred codeblock instead of ignoring the error completely. It won't change the fundamental way Golang checks for errors, but it will certainly make it easier for us to check for them when using 'defer'!

Is this about generics?
If so, how does this differ from the the current design draft and the previous generics proposals?
No, it is not about generics.

[randall77]

What happens if a return is in the code block?

func f() int {
    defer {
        return 7
    }
    return 5
}

That's not semantically equivalent to

func f() int {
    defer func() {
        return 7
    }()
    return 5
}

The latter is obviously wrong. Is the former wrong? Or does that return cause 7 to override 5?

Not sure if this is a ding for or against this proposal. It's definitely a weird semantic corner. But maybe, just maybe, if we allow it we can get rid of the must-name-return-values-to-update-them-with-defer problem.

A similar situation applies to goto. What if there is a goto to outside the defer block? Is that allowed? In this case, though, I think it is clearer that this is just an error.

[83naff]

@randall77 Good points there. In order to understand the use case of the proposed defer, scope here is also important. The defer function (defer func() { // return }()) will mean that any return will only affect the scope of the inner function the defer is caling. But in defer code blocks (defer { // return } ) it means that the outer function will return. This is because the proposed defer is encapsulating a sequence of statements and if a return is found then it would be the equivalent of a return statement being called at the end of the function in which the proposed defer appears.

[randall77]

With the different return semantics, this is no longer just a cosmetic proposal.

[josharian]

But in defer code blocks (defer { // return } ) it means that the outer function will return.

To confirm, do other pending defers for the outer function then still get run?

---

What happens with nested defers? Are they associated with the outer function? We've never had a situation before where you can modify a function's defers while the defers are running. This might pose some implementation challenges.

Concretely, this program (https://play.golang.org/p/sVcNwWjb9Cy) prints 0, 1, 2, 3:

func main() {
	defer func() {
		println(3)
	}()
	defer func() {
		println(1)
		defer func() {
			println(2)
		}()
	}()
	println(0)
}

What does this program do?

func main() {
	defer {
		println(3)
	}
	defer {
		println(1)
		defer {
			println(2)
		}
	}
	println(0)
}

[pjebs]

From Swift 5.2 reference manual:

Defer Statement
A defer statement is used for executing code just before transferring program control outside of the scope that the defer statement appears in.

A defer statement has the following form:

defer {
    statements
}

The statements within the defer statement are executed no matter how program control is transferred. This means that a defer statement can be used, for example, to perform manual resource management such as closing file descriptors, and to perform actions that need to happen even if an error is thrown.

If multiple defer statements appear in the same scope, the order they appear is the reverse of the order they are executed. Executing the last defer statement in a given scope first means that statements inside that last defer statement can refer to resources that will be cleaned up by other defer statements.

func f() {
    defer { print("First defer") }
    defer { print("Second defer") }
    print("End of function")
}
f()
// Prints "End of function"
// Prints "Second defer"
// Prints "First defer"

The statements in the defer statement can’t transfer program control outside of the defer statement.

[pjebs]

What you probably want is not a simple conversion of defer {} to defer func() { } (), which gets run just before the function returns, but rather something that runs before the immediately containing scope finishes.

This then becomes not syntax sugar but a large change to the Go language.
It also encompasses such proposals as:

• Per-loop defer statements: Per-loop defer statements #3978
• proposal: Go 2: allow defer to have its own scope: proposal: Go 2: allow defer to have its own scope #30130 and
• proposal: Go 2: change defer to execute once out-of-scope: proposal: Go 2: change defer to execute once out-of-scope #27762

[fabriciogpp]

What if you'd able to defer statements?

Given your example:

f, err := os.Create("file.txt")

if err != nil {
   panic(err)
}

defer func() {
   err := f.Close()

   if err != nil {
      panic(err)
   }
}()

It could be:

f, err := os.Create("file.txt")

if err != nil {
    panic(err)
}

defer if err := f.Close(); err != nil {
    panic(err)
}

[ianlancetaylor]

The basic idea here is syntactic sugar around the existing defer statement. But as others have pointed out there are complexities if the new defer block uses a return or a goto out of the block. Also, the emoji voting is not in favor. Therefore, this is a likely decline. Leaving open for three weeks for final comments.

[ianlancetaylor]

No further comments.