-
Notifications
You must be signed in to change notification settings - Fork 17.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cmd/go: separate default GODEBUGs from go language version #65573
Comments
Thanks for opening this. Giving the main module the final say over compatibility switches in a usable way makes a lot of sense to me (just like it has the final say over dependency versions via From my perspective (maintaining multiple Kubernetes release branches and patch releases with a large dependency tree), this proposal would restore the usability of the Go 1.21 backwards-compatibility improvements, and would remove a lot of uncertainty and friction from the approach we're recommending (for components to update toolchain minor versions and do security-related dependency bumps as needed on maintenance branches). |
What about unit tests? Will they also apply the Also, I think the merging rules can get tricky here. For example I say Also, should the |
My read is that they would behave the same as they do for the existing defaulting based on the go.mod
I think this would be equivalent to expanding the |
I agree that this is a pain point and don’t have objections to this proposal. That said, it seems to me that the premise of this issue generalizes to any change tied to the For example, suppose I am keeping Unless I am forgetting something, my options are:
That last option is technically a workaround, but it is even more painful than the “ I don’t think that for loop semantics specifically have enough pain to justify a specific fix, I’m just wondering if we should think about approaches to address the more general problem. |
@prattmic raises an important point. The question is whether the more general problem of backdating the language semantics is or will be an actual problem. We have made many public commitments about the loop semantics change being an one-time exception justified by a severe cost-benefit imbalance in favor of benefits, not a prelude to more invasive language changes. If we introduced a line that controlled the language semantics separate from the go line, that would make the more invasive language changes that much more tempting. I'm inclined to focus this on godebug specifically to avoid that temptation, like Odysseus tying himself to the mast. |
This proposal has been added to the active column of the proposals project |
If a library was written against go 1.23, so it does not clone loop variables, but go debug is set to go 1.21, will it introduce bugs in calls to the library code? |
Hi @hherman1, I think part of the answer might be that there is no So as far as I understand this proposal, I think the answer would be no, it would not introduce bugs in your scenario, and something like |
Yes, @thepudds is right (as usual 😄) about the loopvars: that's not godebug-controlled. |
Have all remaining concerns about this proposal been addressed? The proposal is in the top comment: #65573 (comment). |
🤔 I guess it’s expected that libraries written against go 1.N should work with godebug settings from 1.N+1 then? If so my concerns are addressed |
With the new knobs in place, I am wondering whether there would be a general policy/guidance on how to set the versions in different places what could be applied to most of the projects so that owners can easily follow, or it has to be analyzed case by case by project owners in order to set it correctly? |
I think adding However, I don't think |
Right now, we cannot park a maintenance branch at an older configuration even if we fully understand what it means. That's a problem.
This is a good thing to decide and clarify. I see three possibilities if a godebug setting is removed (after the promised 2+ years) and is still requested via
Expansion doesn't seem like what we want... it loses intent and the expanded list becomes incomplete/incorrect in the future when a new minor adds new compatibility switches. |
I would vote for a build error in order to force a maintenance activity to update the |
Why can't you just add the requisite
I don't think it would be incomplete or incorrect. For example, let's say you are upgrading from 1.21 to 1.22. I'm thinking you could just run some command of the form "inject all godebug directives into go.mod to emulate 1.21 in 1.22". Then later if your upgrade from 1.22 to 1.23, you'd run the command "inject all godebug directives into go.mod to emulate 1.22 in 1.23". The key here would be that if a particular godebug setting already appears in the go.mod, then the injection should leave it as-is. In other words, the go.mod file just explicitly says what the end result of this proposal would have been. |
Doing that for multiple (sometimes dozens-to-hundreds) of main packages on multiple release branches and revisiting/updating that on every go minor version bump on the release branch is a lot of churn for a branch that should be seeing minimal changes.
The intent is lost and future go versions would not honor it unless I took action to reassert that intent. That seems problematic and accident-prone. |
Sure, I agree it's not a pleasant process, as I mentioned above. But I think that is addressed by moving the
"The compatibility settings for 1.20 under 1.22" ought to be equivalent to "the compatibility settings for 1.20 under 1.21" + "the compatibility settings for 1.21 under 1.22", with the former taking precedence. So I don't understand what the concern here is, or how it would be error prone. Can you elaborate? |
There's multiple steps required:
If any of those are forgotten or done incorrectly, the system defaults to not doing what I want. Requiring ongoing active modifications to keep compatibility seems burdensome. It's also much harder to review a changing set of godebug lines to make sure they correctly match the desired defaults for the current builder version, than to establish a single |
Fair, but it also seems undesirable that either I have to include both |
The default= feature is very important and should not be removed. The motivating context is that you have a file that says
but you need to bring in a dependency module that says 'go 1.24' (say) while preserving the godebug configuration you were already using. To do that, you change your go.mod to say:
This also explains the meaning: 'default=go1.22' works exactly as well as 'go 1.22' was working. If a godebug that changed behavior in go1.23 was retired in go1.24, then both 'godebug default=go1.22' would silently ignore that setting, just as the 'go 1.22' line would. Degrading gracefully over time for these general statement was an explicit design decision for the 'go' lines, and the 'godebug default=' lines should match. That said, the hypothetical is a policy violation since we guarantee godebugs to last for at least 2 years (4 releases), so to observe the degraded behavior you would have to have a larger skew between your current toolchain and the godebug, like:
I think we all agree that's not something maintained programs are going to encounter. The rationale for unmaintained programs is we might as well keep them building as best we can. |
Have all remaining concerns about this proposal been addressed? The proposal is in the top comment: #65573 (comment). |
Based on the discussion above, this proposal seems like a likely accept. The proposal is in the top comment: #65573 (comment). |
No change in consensus, so accepted. 🎉 The proposal is in the top comment: #65573 (comment). |
Change https://go.dev/cl/584300 mentions this issue: |
Change https://go.dev/cl/584476 mentions this issue: |
Change https://go.dev/cl/584475 mentions this issue: |
Change https://go.dev/cl/584218 mentions this issue: |
For golang/go#65573 Change-Id: I5c1be8833f70b0b5a7257bd5216fa6a89bd2665f Reviewed-on: https://go-review.googlesource.com/c/mod/+/584300 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Russ Cox <rsc@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
go get golang.org/x/mod@c0bdc7bd go mod tidy go mod vendor Pulls in CL 584300. For #65573. Change-Id: Ia8ec86e2ee049b911fcf09d57f83972786b0470d Reviewed-on: https://go-review.googlesource.com/c/go/+/584475 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Russ Cox <rsc@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org>
Excellent to see this land for go1.23, this will help a lot 🎉 Now that the implementation size / scope is well understood, I wondered if any thought had been given to this bit:
We're seeing significant numbers of modules bumping go.mod to go 1.21+, many of which only release security fixes at HEAD. We're already building with Go 1.21 and 1.22, but it's becoming difficult to hold our maintenance branches |
For golang/go#65573. Change-Id: Ie6b92586d203dd921d86084da84edc4706143fd8 Reviewed-on: https://go-review.googlesource.com/c/website/+/584218 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Russ Cox <rsc@golang.org> Reviewed-by: Sam Thanawalla <samthanawalla@google.com> Reviewed-by: Michael Matloob <matloob@golang.org>
Answer back from @rsc
|
Go 1.21 introduced a formalization of how we handle compatible-but-breaking changes, defining configuration knobs called GODEBUG settings that let users control whether or when these changes happen in their specific programs. The current docs are https://go.dev/doc/godebug, and the proposal was #56986. The default settings are taken from the “go” version line in the go.mod file for the main package, and main package Go source files can override with
//go:debug key=value
lines.Go 1.21 also introduced additional forward compatibility rules, among them that the “go” line in any module must be the max of all the “go” lines in its dependencies. Among other things, this means that when deciding whether the current toolchain is new enough to build a module at all, only the top-level “go” line needs to be consulted. This is documented at https://go.dev/doc/toolchain, and the proposal was #57001.
These two interact in an unfortunate way: a maintenance version M of a particular program wants to lock in the GODEBUG semantics from the release they started with, which they can do by setting the “go” line even as they update to newer toolchains. But when they update to newer versions of dependencies, if those dependencies have updated to newer “go” lines, that will force using a newer “go” line in the top-level module, which changes the default GODEBUG settings. If this happens for one dependency, it can be forked and replaced. For a module with a large dependency tree, it may well happen with many dependencies, at which point fork+replace does not scale.
Another possible workaround is to add //go:debug lines to every package main in the module. This is reasonable except that there is no way to predict which ones to add. The point is to say “be like Go 1.X” for some X, but when the module updates to a “go 1.(X+1)” line, the GODEBUG settings from Go 1.(X+1) become relevant, and those can't be predicted when cutting the maintenance branch.
I propose two changes to allow separation of the default GODEBUGs from the “go” line in the go.mod.
First, add a new meta-setting “default=go1.X” that means “set everything the way Go 1.X was”, the same as the “go 1.X” line means. This would let you have a module that says “go 1.23” but a main program that says
//go:debug default=go1.21
. In terms of this new meta-setting, the “go 1.X” line in go.mod essentially implies//go:debug default=go1.X
in each main package. The meta-setting can be used in $GODEBUG as well, of course.Second, add a new “godebug” line to go.mod, respected only in the current module (like toolchain, replace, and exclude are only respected in the current module), as well as to go.work (again like those). It takes a single k=v argument, same as a
//go:debug
source line, but it applies to all the package main in the module. So you can writeSetting multiple GODEBUGs is done with multiple debug lines (just as it is done with multiple
//go:debug
source lines), and like everything else in the go.mod or go.work file, godebug lines can be factored:The precedence order would be: run-time $GODEBUG wins, then //go:debug lines in package main, then go.work or go.mod (go.work when using a workspace, go.mod otherwise).
If this proposal is accepted for Go 1.23, we may want to consider whether to backport either just the first step or both steps to Go 1.22 and possibly Go 1.21. Go 1.22 in particular has a lot of new GODEBUG settings. That's good: we are paying close attention to compatibility. But it's also a bit difficult to cope with in modules that need a “go 1.22” line due to updated dependencies but also want Go 1.21 semantics from their GODEBUG settings. Being able to write
//go:debug default=go1.21
could be worth backporting there. Go 1.21 has fewer relevant settings: basically just panicnil. But our typical rule is that if a fix is deemed critical to backport, we backport it to all supported releases.Note that backporting the godebug directive in go.mod does not cause any compatibility concerns for older go commands seeing these in dependencies: unknown directives are always ignored in dependency go.mods, for precisely this reason.
The text was updated successfully, but these errors were encountered: