performance drop with GO111MODULE="on"

[BenTheElder]

export GO111MODULE="on" and then run any of the code generators (so far defaulter-gen, deepcopy-gen, conversion-gen) for a pretty drastic increase in time to run vs export GO111MODULE="off" first.

I've been converting https://sigs.k8s.io/kind over to modules fully, and observed this performance issue, I've not yet had a chance to dig into why this occurs, but ensuring that module mode is set to off while calling the code generators seems to dramatically improve peformance.

[markmandel]

This might be because go mod will download all the dependencies of the project, rather than looking at the vendor directory (if it hasn't pulled them down already?). May want to try with -mod=vendor enabled?

Just a thought.

[BenTheElder]

I don't think that's it but I'm not 100% certain. The performance difference can be observed with a populated vendor and module cache just between running GO111MODULE="on" <code-generator> and GO111MODULE="off" <code-generator>. The code generator code performance seems to be tied to this.

So far a viable work around to embrace modules is:

• build the code generators while module mode is enabled (so you can install the version your module tracks)
• go mod vendor to populate vendor locally from your module cache
• disable module mode export GO111MODULE="off"
• fake a GOPATH with a tempdir and a symlink to the repo
• run the generators
• cleanup the fake gopath

This is a bit hacky, but allows everything to use modules and doesn't have the big performance penalty.

Implementation here:
https://github.com/kubernetes-sigs/kind/blob/10642f530782963ed23b3d4a14587950bbacaa63/hack/update-generated.sh

[vincepri]

@BenTheElder Can you try with GO111MODULE="on" GOFLAGS="-mod=vendor" <code-generator> as well? We're about to hit this in cluster-api soon :)

[BenTheElder]

@vincepri feel free to give it a shot in the kind repo, we're still using the workaround detailed above for now. I might not get to this for a bit myself 😅
The fake gopath part is pretty trivial and either way we need the rest of the wrapper to populate vendor etc. if it's going to use vendor instead of the module cache

[vincepri]

Oops, sorry I missed the other comment above yours which suggested the same. I wonder how this will perform with the latest build of code-generator, which should be using tools/packages to retrieve the modules.

I'll update when I can test in CAPI or kind.

[nikhita]

latest build of code-generator, which should be using tools/packages to retrieve the modules.

@vincepri do you have a link to an issue/PR about this? Afaik gengo, the main generation tool behind code-generator, would need to move to using tools/packages first and I don't think that's going to happen soon-ish because that would result in breaking lots of existing interfaces.

cc @sttts

[vincepri]

@nikhita I think I might have talked too soon 😄. We should definitely get this on the roadmap, maybe with a feature flag and documentation to opt in?

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[nikhita]

/remove-lifecycle stale

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[maelvls]

/remove-lifecycle stale

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[BenTheElder]

/lifecycle frozen
this issue is not going away on it's own and it will probably bite us in the future, FYI @liggitt @dims (be aware of this when touching k/k build scripts...)

[BenTheElder]

FWIW i'm no longer using the vendor / GOPATH workaround in kind, but I've also changed a lot of things:

• less code
• less dependencies
• less use of generators
• go1.15rc1
  ...

[ialidzhikov]

/kind bug
/priority important-soon
/sig api-machinery

[MadhavJivrajani]

Could this change landing in Go 1.18 help here? 🤔
golang/go#44435

[BenTheElder]

I don't think so. That issue seems to relate to populating dependencies in the module cache more quickly. Module mode slows the generators significantly independently of that step.

[thockin]

/close

[k8s-ci-robot]

@thockin: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[takirala]

any context around how/why this is closed ? Seems like there was no activity for three years and it was closed - is that it or it was "fixed" somehow ?

[thockin]

An answer in parts.

1. Yes, module mode is slower because Go's libs internally call exec("go", "list", "-json") and then parse the results.
2. It's not as much slower now as it was when this was filed.
3. Continuing to support non-modules mode in k/k was a burden, which we have decided not to carry. We stopped running codegen on every make and we adopted go workspaces.

IOW, Go's packages and related libs could get faster, but we no longer consider it "a problem".

[takirala]

thank you for clarification!