cmd/go: run constantly fails and suggests 'go mod tidy'

[myitcv]

What version of Go are you using (go version)?

$ go version
go version devel +eaa97fbf20 Wed Sep 16 03:02:13 2020 +0000 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/myitcv/.cache/go-build"
GOENV="/home/myitcv/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/myitcv/gostuff/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/myitcv/gostuff"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/myitcv/gos"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/myitcv/gos/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/myitcv/gostuff/src/github.com/myitcv/playground/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build826187771=/tmp/go-build -gno-record-gcc-switches"

What did you do?

exec go mod tidy
exec go run blah.com

-- blah/go.mod --
module blah.com

go 1.16
-- blah/main.go --
package main

func main() {
}
-- go.mod --
module mod.com

go 1.16

replace blah.com => ./blah
-- go.sum --
-- main.go --
package main

func main() {
}

What did you expect to see?

Success

What did you see instead?

go: found blah.com in blah.com v0.0.0-00010101000000-000000000000
go: updates to go.mod needed; try 'go mod tidy' first

cc @bcmills @jayconrod

[myitcv]

Hmm this doesn't even appear to be specific to directory replace directives.

[bcmills]

This is almost certainly a consequence of CL 251881 / #40728, and (except for the details of the error message) it's mostly working as designed.

That said, we should at least give a better error message when the arguments include one or more packages outside the main module (for which go mod tidy won't necessarily do any good). For packages outside the main module, go get would suffice as an alternative, but for individual source files there isn't really a good alternative to -mod=mod at the moment.

CC @matloob

[myitcv]

(except for the details of the error message)

As discussed offline, the main problem here is with my bad repro 😄

The repro above does not include a // +build tools-esque file that _ requires blah.com, despite the fact that I thought it did in my local testing. Adding such a file ensures a require directive exists for blah.com with a v0.0.0-00010101000000-000000000000 version and all is good.

Thanks for taking the time to patiently discover the real cause (me)!

[jayconrod]

I'll take a look at this. There are some changes to go.mod that don't cause errors in -mod=readonly: things like adding a go directive to go.mod, formatting, or changing // indirect comments. Adding a null requirement for a replacement should probably be one of those (actually i thought it already was).

[bcmills]

The null requirement can actually be significant, depending on what the replacement's go.mod file says about other dependencies. (For example, it can change the output of go list -m all for other modules — and will continue to do so even once we have lazy loading.)

[jayconrod]

@bcmills Just to confirm what you're saying, the two files below are not equivalent:

module mod.com

go 1.16

replace blah.com => ./blah

module mod.com

go 1.16

require blah.com v0.0.0-00010101000000-000000000000

replace blah.com => ./blah

If we don't load a package from blah.com, we won't add the null requirement or read the go.mod file from the replacement. That go.mod could require higher versions of other dependencies.

So it's correct that we report an error here, but the recommendation to run go mod tidy isn't helpful. go mod tidy doesn't fix this because packages in blah.com aren't in all.

[toothrot]

@jayconrod @bcmills What are the next steps on fixing this, given that it is a release blocker?

[jayconrod]

I plan to work on this before the freeze. Should just involve fixing the error message.

[gopherbot]

Change https://golang.org/cl/258220 mentions this issue: cmd/go/internal/modload: allow 'go get' to use replaced versions

[gopherbot]

Change https://golang.org/cl/263146 mentions this issue: cmd/go: change error message for missing import with unused replacement

[bcmills]

As of CL 258220, the error message is now:

$ go run blah.com
no required module provides package blah.com; try 'go get -d blah.com' to add it

If the replace directive specified some explicit version other than the latest one, that suggestion might fetch a version other than the intended one, but it's still much better than suggesting go mod tidy.

[bcmills]

The go mod tidy suggestion from the original also suggests to me that there is an error in our inAll computation, possibly related to #37376.

[gopherbot]

Change https://golang.org/cl/263266 mentions this issue: cmd/go/internal/modload: fix sort condition in (*replacementRepo).Versions