cmd/go: "go.mod has post-v0 module path ... at revision" with GOPROXY=direct, but not with proxy or Go 1.12

[zikaeroh]

What version of Go are you using (go version)?

$ go version
go version devel +e831719 Fri Jul 12 17:31:29 2019 +0000 linux/amd64

Does this issue reproduce with the latest release?

No on 1.12.7, yes on tip.

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/jake/nobackup/gotip_home/cache/go-build"
GOENV="/home/jake/nobackup/gotip_home/config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/jake/nobackup/gotip_home/gopath"
GOPRIVATE=""
GOPROXY="direct"
GOROOT="/home/jake/sdk/gotip"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/jake/sdk/gotip/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/jake/toolsbug2/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build276222493=/tmp/go-build -gno-record-gcc-switches"

What did you do?

go.mod:

module toolsbug2

go 1.13

main.go:

package main

func main() {}

tools.go:

// +build tools

package tools

import (
	_ "github.com/maxbrunsfeld/counterfeiter/v6"
)

Set GOPROXY=direct and run go mod tidy.

What did you expect to see?

go.mod populated with github.com/maxbrunsfeld/counterfeiter/v6 at version v6.2.1.

What did you see instead?

go mod tidy does not modify go.mod. That's likely due to #27063, so I do go build instead and see:

go: finding github.com/maxbrunsfeld/counterfeiter latest
go: downloading github.com/maxbrunsfeld/counterfeiter v0.0.0-20190712002401-c570f95f2c71
build toolsbug2: cannot load github.com/maxbrunsfeld/counterfeiter/v6: go.mod has post-v0 module path "github.com/maxbrunsfeld/counterfeiter/v6" at revision c570f95f2c71

If GOPROXY is the default (https://proxy.golang.org,direct), or I use Go 1.12.7, there aren't any issues.

Note that this is a boiled down version of a real setup which uses the tools.go pattern, but it doesn't matter where the import is for this to happen.

EDIT: Forgot you can't import a main package without the +build tools to ignore the failure, so I moved this back to a tools.go version. This makes my message from go build not actually happen since the tag isn't set, but the message probably shows up in some other command that doesn't ignore errors.

[zikaeroh]

Interestingly, go.sum gets populated with:

github.com/maxbrunsfeld/counterfeiter/v6 v6.2.1 h1:s0HwWQiNYF+YpoOncE8OxHVYG3YShNiRG8iuPDiSDWM=

[zikaeroh]

The first time I tried to bisect this, I got an unrelated CL, but after repeating and removing my modcache every step, I got to CL 177958.

Bisected by making a git repo with my setup and staging everything, then running gotip mod tidy; git --no-pager diff go.mod and seeing if v6.2.1 showed up or not (then clean all unstaged changes and repeat).

[bcmills]

Thanks for the clear reproducer — and for bisecting the CL. Should be an easy fix, I expect: we're probably missing a “not found” error classification on the post-v0 error.

CC @jayconrod

[bcmills]

(And, BTW, the “other command that doesn't ignore errors” that we usually use for testing is go list, which does reproduce the problem nicely.)

[gopherbot]

Change https://golang.org/cl/186237 mentions this issue: cmd/go/internal/modfetch: consult the go.mod file unconditionally

[zikaeroh]

Yep, I knew go list would work, I just didn't go try because it gets annoying with the build tags and I already had the message... 🙂

Your CL works for me, thanks.

[zikaeroh]

Actually, scratch that, this is causing issues for dependencies I already have:

go get: upgrading gotest.tools@v2.3.0+incompatible: gotest.tools@v2.3.0+incompatible: invalid version: +incompatible suffix not allowed: module contains a go.mod file, so semantic import versioning is required
go get: upgrading github.com/cenkalti/backoff@v2.2.0+incompatible: github.com/cenkalti/backoff@v2.2.0+incompatible: invalid version: +incompatible suffix not allowed: module contains a go.mod file, so semantic import versioning is required

With a gotip get -d -t -u ./...; gotip mod tidy (where my gotip tree has the CL checked out).

I don't disagree that those deps are "bad" in that they are v2+ versions with +incompatible and also a go.mod, but I'm guessing that specific fix will break more people than without, at least in its current form.

[bcmills]

One other (minor) hitch: it turns out that a couple of our test modules (namely, github.com/rsc/vgotest1, vcs-test.golang.org/hg/vgotest1.hg, and vcs-test.golang.org/git/querytest.git/v3) have module-path mismatches and/or ambiguous go.mod files at various points that render them unusable.

[bcmills]

@zikaeroh, the validation for +incompatible suffixes is due to the fix for #32695. We know that it will expose a few broken/mistagged modules, but they can be worked around explicitly using a replace directive. See https://tip.golang.org/doc/go1.13#version-validation.

[bcmills]

(See also cenkalti/backoff#76 and gotestyourself/gotest.tools#156.)

[zikaeroh]

@bcmills I should I should further explain, then. This is what I observe with that CL merged:

1. Start with a clean modcache.
2. Run gotip mod tidy depending on gotest.tools, backoff, go-redis, etc, with the default GOPROXY (so https://proxy.golang.org,direct).
   • I end up with the "bad" revisions mentioned before, but everything builds fine.
3. Run gotip clean -modcache, then set GOPROXY=direct.
4. Run gotip get -d -t -u ./... && gotip mod tidy.

At step 4, things break, as the proxy has given me revisions which gotip tells me are wrong. My initial guess would be that #32805 would cut these from the proxy, but this all doesn't sit too well with me, where some server can give me something that will cause bad things to happen later on based on what happened in my dev environment...

I'm happy to open another issue to not broaden this one, as my original error case was fixed (but it almost feels like it just got moved somewhere else).

[bcmills]

That's #31428.