Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to Go 1.21 #2297

Merged
merged 2 commits into from
Apr 22, 2024
Merged

Update to Go 1.21 #2297

merged 2 commits into from
Apr 22, 2024

Conversation

mtrmac
Copy link
Contributor

@mtrmac mtrmac commented Apr 15, 2024

… primarily because some dependencies started to require it: if we ever needed to quickly update a dependency for a vulnerability fix, we might have to update to Go 1.21 on a short notice, or fork the dependency.

Cc: @jnovy @lsm5 @cevich @TomSweeneyRedHat

go.mod Outdated
@@ -1,7 +1,7 @@
module github.com/containers/skopeo

// Minimum required golang version
go 1.20 // ***** ATTENTION WARNING CAUTION DANGER ******
go 1.21 // ***** ATTENTION WARNING CAUTION DANGER ******
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding is that along with this, we also need toolchain 1.21 as well? Please double-check, and also please remove the big warning comment since it no longer applies.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My reading of “Go toolchain selection” in https://go.dev/doc/toolchain is that it is not compulsory, and effectively defaults to the go directive.

I don’t mind adding 1.21.0 here … and the comment records that as the original plan, so sure.


Do you want that comment removed entirely, or maybe kept up to the “ golang version consistency is desireable.” point?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I understand the situation (could easily be wrong), it's a complex interaction between Renovate proposed package update PRs and our CI systems.

Without the toolchain directive, and without any go version restrictions to Renovate, on Non-Fedora platforms (Debian, Mac, Windows), go will "auto-update" to the in-use version of go for the package update in question (as proposed by Renovate). Meaning for Renovate PRs, builds in CI might not be uniform across all platforms. In other words, Windows might use golang 5.4.3 for updated package FOO 0.2 while Fedora uses golang 1.21 for FOO 0.2 I'm assuming this is a bad/unexpected thing.

OTOH, setting a toolchain forces the maximum go version, and both go and Renovate will honor that for non-Fedora platforms. Meaning builds across all platforms will be exactly the same (for all shared packages).

Note: The only thing making Fedora special in these matters is they set some magic envar by default which disables the auto-update behavior.

@Luap99 please double-check my understanding, when you have a chance.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The way I read the paragraph linked above, toolchain is the minimum Go version; it triggers unwanted upgrades (but not downgrades).


… and IIRC the effect with Renovate included is that at the times where Renovate adds a toolchain directive, it adds one that matches Renovate’s default toolchain.

… and that behavior we don’t want, you’re right.

I’ll add the toolchain directives tomorrow. (The renovate hard-coded version will, I think, orthogonally, need bumping to 1.21.)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes I think we should have the toolchain there so the normal go commands can be used without adding the toolchain later, especially so renovate can be used properly.

I do however think that we need to make sure the toolchain is only ever set to the same value as the go line so we do not force update users unnecessary.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and yes toolchain is the minimum version, not maximum.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added toolchain 1.21.0, and updated the comment a bit.

I’m very open to suggestions about the comment.

Copy link
Member

@cevich cevich Apr 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the comment just needs to say:

Warning: Ensure the "go" and "toolchain" versions match exactly to prevent unwanted auto-updates

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@cevich
Copy link
Member

cevich commented Apr 15, 2024

After this and containers/image#2377 merge, do you have plans for buildah and podman? Reason is, we're currently blocking all updates that require anything past 1.20 with https://github.com/containers/automation/pull/187/files#diff-4d5df5c9d8fa729092ddd28691af9c033a34c9fba7434c06501ae5f875e7b1a6R115-R132

Meaning, that will either need to be removed, or relocated to the buildah and podman Renovate config overrides.

@mtrmac
Copy link
Contributor Author

mtrmac commented Apr 15, 2024

Reason is, we're currently blocking all updates that require anything past 1.20

Oh, I completely forgot about that. At least I figured the Cc: list out :)

After c/image updates in containers/image#2377 , consumer repos will be forced to update shortly in order to consume c/image. So (assuming we can safely trigger the update), I think bumping that to Go 1.21 for all repos would be appropriate.

@mtrmac
Copy link
Contributor Author

mtrmac commented Apr 15, 2024

(Marking as draft to make sure we have a decision the Renovate configuration first; I understand Renovate would immediately start failing otherwise.)

@mtrmac mtrmac marked this pull request as draft April 15, 2024 19:46
@cevich
Copy link
Member

cevich commented Apr 15, 2024

understand Renovate would immediately start failing otherwise.)

I don't believe it will fail, it will simply (and mostly silently) refuse to open any module update PRs which also require a bump of go 1.21.

@mtrmac
Copy link
Contributor Author

mtrmac commented Apr 15, 2024

IIRC from the F38 situation, the Go 1.20 toolchain refuses to work on go 1.21 repos. I could well be misremembering.

Either way, “Renovate silently stops filing PRs” is the worst kind of failure mode for me.

@Luap99
Copy link
Member

Luap99 commented Apr 16, 2024

IIRC from the F38 situation, the Go 1.20 toolchain refuses to work on go 1.21 repos. I could well be misremembering.

No it will just fail to compile if the repo actually uses newer features, only starting with 1.21 go will enforce the minimum go version stated in go.mod. So we should never set a newer go version than our supported distros use.
And f38 was updated to go 1.21 so we should be good even on repos where we still test on f38.

Copy link

Ephemeral COPR build failed. @containers/packit-build please check.

@mtrmac
Copy link
Contributor Author

mtrmac commented Apr 16, 2024

Ephemeral COPR build failed. @containers/packit-build please check.

vendor/github.com/containers/libtrust/ec_key_openssl.go:23:15: undefined: ecdsa.HashSign

The build log does contain mentions of /usr/lib/golang/src/vendor/github.com/golang-fips/openssl/v2 , so the FIPS patch does seem to be present.

Best I can trace this, golang-fips/go@56ac3db has removed the ecdsa.HashSign function from the FIPS-patched Go standard library.

@derekparker, is the above correct, and if so, what is the recommended migration path? (I seem to vaguely remember that the upstream ecsa.Sign did not work in FIPS mode. Is that no longer the case, and should we now always call it?) And is there a single code implementation that now works in all Go versions, or is that Go version-dependent, so that we need to continue using HashSign for older Go versions?

@derekparker
Copy link

derekparker commented Apr 16, 2024

Hey,

Yes it is correct, that particular symbol is removed in the openssl@v2 version. The upgrade path is to use this equivalent symbol instead: https://github.com/golang-fips/openssl/blob/9dbc52b7d686ce75e707d18bd997964a3570f554/ecdsa.go#L112. So the new symbol name will be HashSignECDSA.

@cevich
Copy link
Member

cevich commented Apr 16, 2024

@mtrmac @Luap99 Just to be sure I'm keeping-up: Before this (and the c/image) PR merges, I need to open PRs for Buildah and Podman that disable Renovate updates past go 1.20. After those merge, I need another PR to disable the global Renovate go 1.20 limitation. Then this (and the c/image) PRs are safe to merge.

Somehow that sounds overcomplicated 😞

@Luap99
Copy link
Member

Luap99 commented Apr 17, 2024

@mtrmac @Luap99 Just to be sure I'm keeping-up: Before this (and the c/image) PR merges, I need to open PRs for Buildah and Podman that disable Renovate updates past go 1.20. After those merge, I need another PR to disable the global Renovate go 1.20 limitation. Then this (and the c/image) PRs are safe to merge.

Somehow that sounds overcomplicated 😞

I see no reason why this couldn't be merged before updating the renovate config. Sure vendor PRs that need a newer golang will be broken but so are they today so there is really no difference regardless of merge order. Ideally these changes are merged sooner than later given the vast amount of deps that need go 1.21 so we can unlock many updates.

@mtrmac
Copy link
Contributor Author

mtrmac commented Apr 17, 2024

@mtrmac @Luap99 Just to be sure I'm keeping-up: Before this (and the c/image) PR merges, I need to open PRs for Buildah and Podman that disable Renovate updates past go 1.20. After those merge, I need another PR to disable the global Renovate go 1.20 limitation.

I think that once this merges, Buildah and Podman need to be structurally ready to accept 1.21 (because they will get it on any c/image bump).

So I was thinking just bump Renovate globally to 1.21 and that’s all. Probably pay close attention to the toolchain lines if they were added to by Renovate.

Alternatively, we could line up PRs to bump to 1.21 with a toolchain we like for all projects, and merge them all immediately followed by a Renovate config update.

@Luap99
Copy link
Member

Luap99 commented Apr 17, 2024

So I was thinking just bump Renovate globally to 1.21 and that’s all. Probably pay close attention to the toolchain lines if they were added to by Renovate.

Yeah that should happen, and even if someone somewhere merges a PR with a to new toolchain directive we can always manually revert it back to an older version so no big problem.

@cevich
Copy link
Member

cevich commented Apr 17, 2024

Great, this sounds like a problem that can be solved easily by a short e-mail 😁

@cevich
Copy link
Member

cevich commented Apr 17, 2024

Ref: containers/automation#189

@cevich
Copy link
Member

cevich commented Apr 17, 2024

PR merged and e-mail sent.

@mtrmac mtrmac marked this pull request as ready for review April 17, 2024 19:44
mtrmac added 2 commits April 18, 2024 18:09
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Not "maps" because maps.Keys is not included.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
@mtrmac
Copy link
Contributor Author

mtrmac commented Apr 18, 2024

Ephemeral COPR build failed. @containers/packit-build please check.

vendor/github.com/containers/libtrust/ec_key_openssl.go:23:15: undefined: ecdsa.HashSign

Dealing with this continues in #2305 .

Copy link
Member

@Luap99 Luap99 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@TomSweeneyRedHat
Copy link
Member

LGTM
once the tests are happy

@mtrmac
Copy link
Contributor Author

mtrmac commented Apr 19, 2024

(I think those test failures are orthogonal, but it does make a lot of sense to fix those failures before introducing more changes.)

@lsm5
Copy link
Member

lsm5 commented Apr 22, 2024

we can ignore the 2 centos-stream failures for now. To use TMT tests, we'll have to switch to epel- targets anyway as in containers/podman#22432 , which I will do in another PR.

@mtrmac mtrmac merged commit d4e8198 into containers:main Apr 22, 2024
22 of 24 checks passed
@mtrmac mtrmac deleted the go1.21 branch April 22, 2024 18:18
@stale-locking-app stale-locking-app bot locked as resolved and limited conversation to collaborators Jul 22, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants