cmd/go: long delay before error for missing replaced module

[alltom]

What version of Go are you using (go version)?

$ go version
go version go1.13.4 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/tom/Library/Caches/go-build"
GOENV="/Users/tom/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/tom/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.13.4/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.13.4/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/y8/11jfs3dj5q1dkwc3styvmc740000gn/T/go-build204589310=/tmp/go-build -gno-record-gcc-switches -fno-common"
GOROOT/bin/go version: go version go1.13.4 darwin/amd64
GOROOT/bin/go tool compile -V: compile version go1.13.4
uname -v: Darwin Kernel Version 19.0.0: Thu Oct 17 16:17:15 PDT 2019; root:xnu-6153.41.3~29/RELEASE_X86_64
ProductName:	Mac OS X
ProductVersion:	10.15.1
BuildVersion:	19B88
lldb --version: lldb-1100.0.30.6
Apple Swift version 5.1.2 (swiftlang-1100.0.278 clang-1100.0.33.9)

What did you do?

Create three local modules a, b, and c, where a depends on b, and b depends on c, as in #33478.

What did you expect to see?

Successful build.

What did you see instead?

go build a.go appears to hang, but it's actually waiting 30 seconds for a doomed network request to fetch module c.

I understand the motivation (assuming that my comment here is correct: #33478 (comment)) However, I'm working on modules locally that may never be published, so I feel like there are two bugs wrt the local development workflow:

1. go build appears to hang for 30 seconds. It took a while for me to figure out what was going on because I kept getting impatient and killing the process before seeing an error message. Adding -v doesn't help.
2. When developing local modules, the refusal to respect replace in dependencies means that all of my modules need to add replace statements for the entire transitive closure of their dependencies. If I forget one and that dependency has not been published, the build will silently hang for 30 seconds before failing. If I forget one that has been published at least once before, the build will silently use the published version instead of my local version. Neither is great.

Thanks!

[OneOfOne]

Personally I disable modules for local dev, I'm pro-modules but it's extremely annoying at times.

[ghost]

(Warning: I am learning Go so please be skeptical of my assertions.)

It appears that the Go build process uses 3 separate cache facilities: GOPROXY, GOCACHE and $GOPATH/pkg.

1. During active development/modifications/testing, GOPROXY is perhaps unhelpful: use GOPROXY="direct".

2. For initial development/testing, it may reduce timesuck to delete/rename go.mod files and work using local copies of dependencies. Use "go get" to download and install the correct dependencies. Later, after dust settles, turn Modules and GOPROXY back on...

3. Use "go clean" to clear local build/test caches.

4. (Somehow) audit the date/timestamp of each package being tested to make sure it matches what was intended.

5. It appears that "go install github.com/userid/reponame/pkg" (a pathname relative to $GOPATH) is not triggering Modules=On behavior but "go install" inside the package directory will use Modules=On behavior. This may be a "feature" or a bug, but it may be useful with Go1.13.4.

[dmitshur]

/cc @bcmills @jayconrod

[jayconrod]

go build appears to hang for 30 seconds. It took a while for me to figure out what was going on because I kept getting impatient and killing the process before seeing an error message. Adding -v doesn't help.

Is there anything different than what you posed in #33478? Was there any output at all (even at the end of 30 seconds)? Is there anything unusual about your network setup (proxy or firewall)?

I couldn't reproduce this with go 1.13.4. I ran go build a.go in the a directory. It failed quickly with:

go: github.com/kellengreen/test/b@v0.0.0 requires
	github.com/kellengreen/test/c@v0.0.0: reading github.com/kellengreen/test/c/c/go.mod at revision c/v0.0.0: unknown revision c/v0.0.0

Did you see any output at all? There's generally some chatter about "finding", "downloading", and "extracting" modules, though we've trimmed that back a bit as it's generally a bit much.

[alltom]

Here's my minimal repro: https://github.com/alltom/gobug-35902 I've tripped the bug on a few different networks, so I don't think it's them. I do, however, run Encrypt.me, although it's supposed to be disabled on those networks. Let me know if it still doesn't repro for you and I'll try harder to disable it.

FWIW, the title you gave this bug addresses item 1 in my original report, but no longer addresses item 2. Silently using an unexpected version of my code (on my laptop, I always want the source that is on disk) is at least at confusing and frustrating as the other bug.

[jayconrod]

@alltom Thanks for putting together that repo. I was able to reproduce this.

The reason the go command is going out to the network is because there's no replacement for alltom.com/a. If that's added, as below, the build completes quickly:

module alltom.com/c

go 1.13

replace alltom.com/a => ../a
replace alltom.com/b => ../b

Since there's no requirement or replacement for alltom.com/a, the go command sends a GET request to https://alltom.com/a?go-get=1 to find the repository root. This is what times out. curl runs into the same issue.

The -x flag prints each outgoing GET request, so that's probably the most useful tool at the moment. We could probably do a better job of printing requests that are taking a long time though.

[jayconrod]

@alltom About the retitling: by design, replace directives only work in the main module. They're mostly intended to be used as a workaround for upstream build issues and for local development and debugging. Making replace work transitively would be more confusing than not, and it would make modules less composable in general.

If you find yourself using replace very frequently, you might see if gohack helps.

[bcmills]

I would like to make the go command print a useful summary of the pending requests when it exits via SIGINT, although I think @rsc has expressed some reservations about that approach in the past. We can talk to him for more detail.

[bcmills]

Regarding item 2, the found messages added in CL 204777 may already do what you need. (If you don't mind, try a build from source at head and let us know what still needs improvement!)

[alltom]

@jayconrod I understand the concern with transitive replacements, but I believe that it's important for us to find a mitigation for "all of my modules need to add replace statements for the entire transitive closure of their dependencies". Would a global replace break the dependency search? It seems like it would help solve this problem, but I'm unused to thinking about the ramifications of changes to Go's algorithm.

I'm developing several command-line tools right now, consisting of about 15 modules. There are dependency chains like tool → plugin lib → networking lib. Yesterday I made a breaking change to fix a bug in "networking lib" and to figure out if the fix was adequate for all the tools, I had to add several replace statements to every single tool and intermediate library in order to build and test them individually, then back them all out again in order to submit the code to my private Github.

With a GOROOT, the only steps would have been building and testing. I haven't used a GOROOT because the only thing they all have in common is that I happen to be working on them at the same time.

gohack seems to solve a different problem, where I'm editing a dependency but only testing the change in one dependent.

@bcmills I will try to give it a shot. I've never tried installing a second Go before. Please tell me it's painless. :)

[bcmills]

@alltom, the painless way is go get golang.org/dl/gotip. 🙂

[bcmills]

Yesterday I made a breaking change to fix a bug in "networking lib" and to figure out if the fix was adequate for all the tools, I had to add several replace statements to every single tool and intermediate library in order to build and test them individually, then back them all out again in order to submit the code to my private Github.

Ah, for that workflow I would recommend doing the testing “in reverse”: do all of your work in the “networking lib” module, and use replace statements to map in all of the things that depend on it while you iterate. Then you only have to back out replace directives from the networking lib's go.mod file.

But this is getting a bit off-topic: #27542 is the main bug for editing interdependent modules.

[alltom]

Awesome, it really was painless. The change in gotip fixes my first issue.

And I'm happy to let the second issue in my report be duped to #27542 since the discussion over there is great.

Thanks!