cmd/go: allow replacement modules to alias other active modules

[mwf]

What version of Go are you using (go version)?

Go tip:
go version devel +f2131f6e0c Wed Aug 8 21:37:36 2018 +0000 darwin/amd64

Does this issue reproduce with the latest release?

Yes (it is not reproduced with go version go1.11beta2 darwin/amd64)

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/ikorolev/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/var/folders/_b/d1934m9s587_8t_6ngv3hnc00000gp/T/tmp.cqU8g8OM/gopath"
GOPROXY=""
GORACE=""
GOROOT="/Users/ikorolev/.gvm/gos/go1.11beta3"
GOTMPDIR=""
GOTOOLDIR="/Users/ikorolev/.gvm/gos/go1.11beta3/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/var/folders/_b/d1934m9s587_8t_6ngv3hnc00000gp/T/tmp.cqU8g8OM/vgo-a-user/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/_b/d1934m9s587_8t_6ngv3hnc00000gp/T/go-build138999780=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Sorry, no standalone reproduction, since issue is connected with repository forking

Assume we have a repository A: https://github.com/mwf/vgo-a with the only feature:

package a

var A = "A"

Than we have a fork1 https://github.com/mwf/vgo-a-fork1, adding a feature B :

package a

var B = "B is a new feature in a-fork1"

Unfortunately fork1 will never be merged to the upstream, just because a author don't like this feature.

It's important to note, that both a and a-fork1 don't have go.mod, they are too conservative for that 😄

Then we got a happy user, using both projects in his repo.
go.mod:

module github.com/mwf/vgo-a-user

require (
	github.com/mwf/vgo-a v0.1.0
	github.com/mwf/vgo-a-fork1 v0.2.0
)

main.go

package main

import (
	"fmt"

	"github.com/mwf/vgo-a"
	a_fork "github.com/mwf/vgo-a-fork1"
)

func main() {
	fmt.Printf("A: %q\n", a.A)
	fmt.Printf("B: %q\n", a_fork.B)
}

All just works fine:

$ go run .
A: "A"
B: "B is a new feature in a-fork1"

Here appears fork2 https://github.com/mwf/vgo-a-fork2, forked from fork1, and fixing some bugs both in the upstream and in fork1.

We use the fork2 with replace in our main repo: https://github.com/mwf/vgo-a-user/blob/master/go.mod

module github.com/mwf/vgo-a-user

require (
	github.com/mwf/vgo-a v0.1.0
	github.com/mwf/vgo-a-fork1 v0.2.0
)

replace github.com/mwf/vgo-a => github.com/mwf/vgo-a-fork2 v0.2.1

replace github.com/mwf/vgo-a-fork1 => github.com/mwf/vgo-a-fork2 v0.2.1

What did you expect to see?

Building this with go1.11beta2 works just fine:

cd `mktemp -d`
git clone git@github.com:mwf/vgo-a-user.git .
go version && go run .

Output:

go version go1.11beta2 darwin/amd64
go: finding github.com/mwf/vgo-a-fork2 v0.2.1
go: downloading github.com/mwf/vgo-a-fork2 v0.2.1
go: finding github.com/mwf/vgo-a v0.1.0
go: finding github.com/mwf/vgo-a-fork1 v0.2.0
A: "A, fixed in a-fork2"
B: "B, fixed in a-fork2"

What did you see instead?

Building with the tip (and beta3) returns an error:

cd `mktemp -d`
git clone git@github.com:mwf/vgo-a-user.git .
go version && go run .

Output:

go version devel +f2131f6e0c Wed Aug 8 21:37:36 2018 +0000 darwin/amd64
go: finding github.com/mwf/vgo-a-fork2 v0.2.1
go: downloading github.com/mwf/vgo-a-fork2 v0.2.1
go: github.com/mwf/vgo-a-fork1@v0.2.0 used for two different module paths (github.com/mwf/vgo-a and github.com/mwf/vgo-a-fork1)

More comments

I understand that this case is very specific and arguable - this should not ever happen ideally, but we have the real case here:
https://github.com/utrack/clay/blob/master/integration/binding_with_body_and_response/go.mod

There is a little workaround, to define go.mod at fork2 and make a replace upstream -> fork2_with_go.mod, but it's too dirty :)

replace github.com/mwf/vgo-a => github.com/mwf/vgo-a-fork2 v0.3.0 // version with go.mod
replace github.com/mwf/vgo-a-fork1 => github.com/mwf/vgo-a-fork2 v0.2.1 // no go.mod

It works with tip and beta3:

$ go version && go run .
go version devel +f2131f6e0c Wed Aug 8 21:37:36 2018 +0000 darwin/amd64
A: "A, fixed in a-fork2"
B: "B, fixed in a-fork2"

If you decide that the case is too specific and crazy, and you'd like to close as "Won't fix" - then I assume we should change the error string, because it's confusing now:

go: github.com/mwf/vgo-a-fork1@v0.2.0 used for two different module paths (github.com/mwf/vgo-a and github.com/mwf/vgo-a-fork1)

It should look like this:

go: github.com/mwf/vgo-a-fork2@v0.2.1 used for two different module paths (github.com/mwf/vgo-a and github.com/mwf/vgo-a-fork1)

because it's github.com/mwf/vgo-a-fork2 who's to blame for the error.

[mwf]

And this crazy stuff could happen only in non-go.mod repositories, because you can't use both upstream and fork at the same time if they have go.mod inside:

module github.com/mwf/vgo-a-user

require (
    github.com/mwf/vgo-a-mod v0.1.0
    github.com/mwf/vgo-a-mod-fork1 v0.2.0
)

$ go run .
go: finding github.com/mwf/vgo-a-mod-fork1 v0.2.0
go: github.com/mwf/vgo-a-mod-fork1@v0.2.0: parsing go.mod: unexpected module path "github.com/mwf/vgo-a-mod"
go: finding github.com/mwf/vgo-a-mod v0.1.0
go: error loading module requirements

You are forced either to use your fork as replace argument, or to change the module in fork's go.mod and never assume it as a fork again :) And this is for the best 👍

So no one will ever hit such an issue in the brave new world of go modules 😄

[bcmills]

Ideally, I think the long-term solution will be to treat replacements as rewriting the import paths rather than the source code, to allow for precisely this kind of fork-unification behavior.

(For an example of how this can go wrong otherwise, see the code in #26607.)

[mwf]

@bcmills you set "Go1.12" milestone for the issue.

Maybe we could at least fix the error string in Go1.11 to eliminate the confusion?

If you decide that the case is too specific and crazy, and you'd like to close as "Won't fix" - then I assume we should change the error string, because it's confusing now:

go: github.com/mwf/vgo-a-fork1@v0.2.0 used for two different module paths (github.com/mwf/vgo-a and github.com/mwf/vgo-a-fork1)

It should look like this:

go: github.com/mwf/vgo-a-fork2@v0.2.1 used for two different module paths (github.com/mwf/vgo-a and github.com/mwf/vgo-a-fork1)

because it's github.com/mwf/vgo-a-fork2 who's to blame for the error.

[bcmills]

Oh, yeah, that's an easy fix.

[gopherbot]

Change https://golang.org/cl/128878 mentions this issue: cmd/go/internal/modload: emit correct module in duplication error

[F21]

I am also seeing this problem when using go mod why. Since testcontainers/testcontainer-go was renamed to testcontainers/testcontainers-go recently, modules break for me since it is being used by one of my dependencies (not sure how deep).

I added:

replace github.com/testcontainers/testcontainer-go => github.com/testcontainers/testcontainers-go v0.0.0-20190108154635-47c0da630f72

to go.mod in order to get go get -u and go mod tidy to work correctly.

However, if I run go mod why github.com/testcontainers/testcontainers-go, I get:

go: finding github.com/testcontainers/testcontainers-go latest
go: github.com/testcontainers/testcontainers-go@v0.0.0-20190108154635-47c0da630f72 used for two different module paths (github.com/testcontainers/testcontainer-go and github.com/testcontainers/testcontainers-go)

[integrii]

I just want to leave a note here to say that I've spent the last two hours trying to get a package in the github.com/docker/docker module to build, but can't because of this fix not being in yet.

go: github.com/sirupsen/logrus@v1.8.1 used for two different module paths (github.com/Sirupsen/logrus and github.com/sirupsen/logrus)

Specifically, the replace line below for go.mod below is not possible in go 1.18, and that means I can not get my build to work no matter what I do.

replace github.com/Sirupsen/logrus v1.8.1 => github.com/sirupsen/logrus v1.8.1

[ianlancetaylor]

@bcmills @matloob This issue is marked for 1.19. It's just been rolling forward in milestones. Should it move to Backlog?

[bcmills]

We've been doing active design work for this issue this week. It's not ready for the Backlog quite yet.

[ianlancetaylor]

This is still open and just rolling forward. @matloob should we move this to backlog? Thanks.

[matloob]

Yes, let's move this to backlog. We would like to fix this issue at one point but it's unlikely to be done in 1.24.