cmd/go: only generate a new 'go.mod' file during 'go mod init'

[sunliver]

What version of Go are you using (go version)?

$ go version
go version go1.11.4 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GOARCH="amd64"
GOBIN="/Users/sunliver/go/bin"
GOCACHE="/Users/sunliver/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/sunliver/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.11.4/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.11.4/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/r3/fxzm7n8s0497vkmvnj0qygkr0000gp/T/go-build192454760=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

1. export GO111MODULE=on
2. mkdir -p ~/go/project/src/github.com/sunliver/pkg1 && cd ~/go/project/src/github.com/sunliver/pkg1 && touch Gopkg.lock Gopkg.toml
~/go is default module GOPATH
   the project under module GOPATH is required
3. run go env GOMOD you will get
   /Users/sunliver/go/project/src/github.com/sunliver/pkg1/go.mod
   though the go.mod file doesn't exist

further:

1. export GO111MODULE=on && export GOPATH=~/go/project
2. create simple hello.go under pkg1 and run go build
   you will get a go.mod file from Gopkg.lock

What did you expect to see?

GOMOD left unset

What did you see instead?

/Users/sunliver/go/project/src/github.com/sunliver/pkg1/go.mod

Though building legacy GOPATH project when GO111MODULE is on is confusing, I hope at least GOMOD left unset and go build will not auto generate the go.mod file. Maybe give a warning or error instead.

[odeke-em]

Thank you for filing this issue @sunliver and welcome to the Go project!

So if I recall right, enabling/setting GO111MODULE=on in your environment right before a build opts you into using go modules and hence that generated go.mod file. I don't see how we could use go modules without generating those files.

I'll page some experts to chime in too @bcmills @rsc

[sunliver]

Yes. And go env GOMOD gets the go.mod though it does not exist. The none exist go.mod is confusing.
My opinion is maybe give me a warning or error if I trying to do so.

[mvdan]

This is expected; if you don't want the go.mod file, build in the GOPATH mode with GO111MODULE=auto or GO111MODULE=off.

What exactly are you trying to achieve? A Go module build requires a go.mod file, so you can't really be building a module without having a go.mod file somwhere on disk. If you really want to build a module, you should keep that file.

As for why GOMOD points at a non-existing file, I'm not sure why that happens. Perhaps @bcmills can clarify.

[sunliver]

This is expected; if you don't want the go.mod file, build in the GOPATH mode with GO111MODULE=auto or GO111MODULE=off.

What exactly are you trying to achieve? A Go module build requires a go.mod file, so you can't really be building a module without having a go.mod file somwhere on disk. If you really want to build a module, you should keep that file.

Yes, a Go module build requires a go.mod file. So if you set GO111MODULE=on and try to build a module, you will get go: cannot find main module; see 'go help modules'. But if you have Gopkg.lock and Gopkg.toml and you're under default module path (I've only test dep.), go build will create a go.mod file and try to build it. At most time, the module name is not what you are expected. (Maybe you want A/B, but the generated go.mod is module A.)

[mvdan]

I see what you mean. I think the go tool automatically converts what it can over to modules, leaving it to the developer to do go mod init module/path (or to edit go.mod) if they need something else.

[bcmills]

Within GOPATH, the inferred module name should be the relative path from GOPATH/src to the directory containing the converted Gopkg.lock.

• The existence of Gopkg.lock and Gopkg.toml indicates the module root, since those files are generally only found in the topmost directory of a project.

• The module path is an import-path prefix for the packages it contains, so the inferred module path gives the same package paths in module mode that you were already using in GOPATH mode.

Those two facts tell the go command everything it needs to know to figure out where the go.mod file ought to be, and since it knows where that file ought to be and what module path it ought to have, it goes ahead and creates it.

[bcmills]

My only major concern here is the interaction with #24250: if you intended to be outside of any module, but are actually inside some repository where we can infer both a name and a module root, then creating a go.mod file is really not the right thing to do.

But I'm skeptical that “inside some repository where we can infer both a name and a module root” is actually at all likely to occur.

[rsc]

In general it's incredibly helpful to create the go.mod and thereby incorporate all the version pinning etc in the existing package manager files. This made projects that hadn't yet converted to modules but were already using an existing package manager work out of the box, which is really great. We don't want to make that process more onerous without good cause. The example in the report is working as intended.

But Bryan points out that:

1. Suppose the example had been in a git repo at a v2 or later tag not written using semantic import versions.
2. The created go.mod has a 'module x' (not x/v2), which works fine locally.
3. Checking in and releasing a new v2.5.0 (say), with go.mod saying 'module x' will make go get clients stop seeing the sequence as +incompatible.
4. Those clients will now break.

Creating the go.mod automatically leads to this result (along with the user explicitly adding and checking in the generated go.mod without understanding semantic import versioning).

One defense would be to check 'git describe' etc when creating the go.mod, to diagnose when the module path needs a v2 and warn the user (with a link to a good doc). But we don't want this happening during 'go build' the way that auto-go.mod-creation does. 'go build' should avoid running any VCS commands ever.

So one option would be to make 'go build' no longer auto-generate go.mod and instead print a good message leading to 'go mod init'. Then the explicit 'go mod init' could reasonably invoke VCS commands to double-check its work. That would help us help the user in the sequence above.

It would be sad to lose the auto-go.mod in 'go build' but if we make the 'go build' output very clear about what should happen next ('go mod init'), maybe only just a very little bit sad.

Sounds OK to try for Go 1.13, early.

[gopherbot]

Change https://golang.org/cl/162699 mentions this issue: cmd/go: only generate a go.mod file during 'go mod init'

[bcmills]

The remaining work I had intended to do for this issue is now covered by #31549, so closing.