cmd/go: 'go mod vendor' gets internal/ modules

[estroz]

What version of Go are you using (go version)?

$ go version
go version go1.12.1 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/estroz/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/estroz/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build997648725=/tmp/go-build -gno-record-gcc-switches"

What did you do?

$ go get -d github.com/operator-framework/operator-sdk
$ cd $GOPATH/src/github.com/operator-framework/operator-sdk
$ GO111MODULE=on go mod vendor

What did you expect to see?

go mod vendor does not treat bitbucket.org/ww/goautoneg as a module.

What did you see instead?

$ GO111MODULE=on go mod vendor
...
go: finding gopkg.in/natefinch/lumberjack.v2 v2.0.0-20170531160350-a96e63847dc3
go: finding k8s.io/kubernetes v1.11.7-beta.0.0.20181219023948-b875d52ea96d
go: bitbucket.org/ww/goautoneg@v0.0.0-20120707110453-75cd24fc2f2c: hg clone -U https://bitbucket.org/ww/goautoneg . in /home/estroz/go/pkg/mod/cache/vcs/59c2185b80ea440a7c3b8c5eff3d8abb68c53dea1f20f615370c924c4150b27f: exec: "hg": executable file not found in $PATH
go: finding golang.org/x/net v0.0.0-20180724234803-3673e40ba225
go: finding github.com/ugorji/go v1.1.1
...
$ grep -R 'goautoneg' vendor/
vendor/modules.txt:github.com/prometheus/common/internal/bitbucket.org/ww/goautoneg
vendor/github.com/prometheus/common/expfmt/encode.go:	"github.com/prometheus/common/internal/bitbucket.org/ww/goautoneg"
vendor/github.com/prometheus/common/expfmt/encode.go:	for _, ac := range goautoneg.ParseAccept(h.Get(hdrAccept)) {
vendor/github.com/prometheus/common/internal/bitbucket.org/ww/goautoneg/README.txt:package goautoneg
vendor/github.com/prometheus/common/internal/bitbucket.org/ww/goautoneg/README.txt:import "bitbucket.org/ww/goautoneg"
vendor/github.com/prometheus/common/internal/bitbucket.org/ww/goautoneg/autoneg.go:package goautoneg
vendor/go.opencensus.io/Gopkg.lock:    "internal/bitbucket.org/ww/goautoneg",

Per the grep output, any module that imports goautoneg does so through an internal directory.

[jayconrod]

The module bitbucket.org/ww/goautoneg is actually required, not from github.com/prometheus/common. The Go command isn't adequately explaining why it's getting pulled in though.

I'm working on a patch to improve error output in cases like this. With CL 166984 (fixing #30661), here's the output:

go: github.com/operator-framework/operator-lifecycle-manager@v0.0.0-20190128024246-5eb7ae5bdb7a
	-> github.com/operator-framework/operator-registry@v1.0.4
	-> github.com/operator-framework/operator-lifecycle-manager@v0.0.0-20190105193533-81104ffdc4fb
	-> bitbucket.org/ww/goautoneg@v0.0.0-20120707110453-75cd24fc2f2c: hg clone -U https://bitbucket.org/ww/goautoneg . in /var/folders/rq/x0692kqj6ml8cvrhcqh5bswc008xj1/T/tmp.3bFPwkWK/pkg/mod/cache/vcs/59c2185b80ea440a7c3b8c5eff3d8abb68c53dea1f20f615370c924c4150b27f: exec: "hg": executable file not found in $PATH

The go.mod file for github.com/operator-framework/operator-lifecycle-manager at commit 81104ffdc4fb requires bitbucket.org/ww/goautoneg v0.0.0-20120707110453-75cd24fc2f2c.