Run test-go-mod on local modules

[annasong20]

This PR comments out the test-go-mod prow-presubmit job.

This PR changes test-go-mod so that go mod tidy is run with local module dependencies instead of the latest released modules. This makes the CI much friendlier to changes that across multiple modules.

The failed job as a result of cross-module changes in #4959 motivated this change.

[annasong20]

/cc @natasha41575

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: annasong20, yuwenma
Once this PR has been reviewed and has the lgtm label, please assign knverey for approval by writing /assign @knverey in a comment. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[yuwenma]

/lgtm

cc @koba1t who made the go workspace improvements for kustomize.

[annasong20]

/cc @koba1t

[k8s-ci-robot]

This PR has multiple commits, and the default merge method is: merge.
You can request commits to be squashed using the label: tide/merge-method-squash

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-ci-robot]

New changes are detected. LGTM label has been removed.

[annasong20]

/label tide/merge-method-squash

[annasong20]

The test-go-mod rule should now add replace statements, run go mod tidy, then remove said replace statements on all modules.

I left my concerns below.

[annasong20]

I'd like to move this into Makefile to avoid re-calculating it for each module, but I don't know how to export variables in Makefile so that they're accessible here.

[annasong20]

I'm assuming go list -m returns the modules in the same order on every call.

I've considered 2 alternatives:

• go list -m -json returns all information, including name and directory path, for all modules, but I'd have to parse the json output
• fetch the module name as I navigate to each module with hack/for-each-module.sh; however, go list -m lists all modules in the workspace regardless of the working directory, so the best I could come up with was parsing the go.mod file for the module name

[annasong20]

I tried inlining the scripts in replace.sh and dropReplace.sh, but couldn't figure out how to escape the relevant characters.

[KnVerey]

I think calling a script from here is better than inlining the details anyway. :)

I'd suggest making a single script we can invoke from here like ./hack/for-each-module.sh "\$$KUSTOMIZE_ROOT/hack/go-mod-tidy-kust-dev.sh" and then adding a bunch of comments in that file to explain the steps and why we're doing them. Or to be more flexible, we can make a script that itself takes a command to run in the context of the "pinned" module, invoked like: ./hack/for-each-module.sh "\$$KUSTOMIZE_ROOT/hack/with-pinned-kust-dev.sh 'go mod tidy -v'".

[annasong20]

I tried using $$KUSTOMIZE_ROOT instead of $$(pwd), but couldn't figure out how to use the exported hack/for-each-module.sh variable here.

[KnVerey]

I think "\$$KUSTOMIZE_ROOT/hack/... should work!

[natasha41575]

Closing because #5011 will merge soon