cmd/go: resolve local paths to modules in subdirectory replacements

[jayconrod]

What version of Go are you using (go version)?

$ go version
go version go1.16beta1 darwin/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/jayconrod/Library/Caches/go-build"
GOENV="/Users/jayconrod/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/jayconrod/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/jayconrod/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/opt/go/installed"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/opt/go/installed/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.16beta1"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/jayconrod/Code/test3/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/rq/x0692kqj6ml8cvrhcqh5bswc008xj1/T/go-build3849412287=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Run go list or any other module-aware command on an absolute or relative path (starting with ./) that points to a subdirectory inside a separate module which provides a replacement.

go list ./sub
go list $(pwd)/sub

-- go.mod --
module example.com/m

go 1.16

require example.com/m/sub v0.0.0

replace example.com/m/sub v0.0.0 => ./sub
-- sub/go.mod --
module example.com/m/sub

go 1.16
-- sub/sub.go --
package sub

What did you expect to see?

go list should succeed and print the package path. In the example above, it should print example.com/m/sub.

What did you see instead?

go list fails with:

main module (example.com/m) does not contain package example.com/m/sub

It looks like the go command resolves the path argument to a directory within the module before checking whether it might be part of a replacement module in another directory.

Other notes

This issue is related to the thread go list across multiple modules on golang-nuts.

[gopherbot]

Change https://golang.org/cl/335269 mentions this issue: cmd/go: expand local directory in replace directive

[hyangah]

Is this proposal to cover only the case where the module in the subdirectory is mentioned in the main module? Or is it going to allow to run go commands with subdirectories whose module aren't listed in the main module? For example, the current x/tools repo has golang.org/x/tools/gopls module in the gopls directory and is not a dependency of golang.org/x/tools. Will this change allow go list ./gopls from the the repo root to work?

[jayconrod]

Will this change allow go list ./gopls from the the repo root to work?

No, that shouldn't work, though an error message probably ought to say why. After this is fixed, go list should (still) only be able to load packages from modules in the build list.

[iwdgo]

On a recent tip, go list seems to behave as expected:
The directory ...\43733 has the structure described in the issue.
The sub-directory sub is a sub-module in the context of the main module and a package in a context outside the module. Replacing the required sub-module by a local package looks like a mistake from the module perspective.

GOPATH\Issues\43733>gotip version
go version devel go1.18-95d0657670 Fri Nov 12 16:58:34 2021 +0000 windows/amd64

GOPATH\Issues\43733>set GO111MODULE
Environment variable GO111MODULE not defined

GOPATH\Issues\43733>gotip list
no Go files in GOPATH\Issues\43733

GOPATH\Google\Issues\43733>gotip list -m
example.com/m

example.com/m is a module as a go.mod is present in its main directory.

GOPATH\Google\Issues\43733>gotip list -m ...
example.com/m
example.com/m/sub v0.0.0 => ./sub

example.com/m/sub is a sub-module in this context. It is replaced as a package in the local sub-directory.

GOPATH\Google\Issues\43733>gotip list -m ./sub
go: cannot use relative path ./sub to specify module

GOPATH\Google\Issues\43733>gotip list ./sub
main module (example.com/m) does not contain package example.com/m/sub

Errors seem correct as the sub directory is a package in this context and cannot be listed as a module.
When listing packages,, the main module does not contain the package example.com/m/sub but a module as go.mod is present. So there is an error. The main module requires a package which is in fact a sub-module.

GOPATH\Google\Issues\43733>gotip list ...
archive/tar
(all golang packages)
example.com/m/sub

The included directory is seen as a package.

No change in results when go.mod is set to 1.16 or 1.17.

[ianlancetaylor]

This is in the Go1.18 milestone. Is it likely to happen for 1.18? Thanks.

[ianlancetaylor]

@bcmills This is in the 1.18 milestone; time to move to 1.19? Thanks.

[bcmills]

As of Go 1.18 this is also relevant to paths that could refer to packages in other modules in the same workspace.

The interaction between this and ... patterns could be a bit confusing, though. In the original example, ./sub would resolve to a package in a different module, but would ./... enumerate that package? (It doesn't today.)

[iwdgo]

Using go1.18rc1 and setting go.mod version to 1.18, the behavior of the test of the issue evolves but seems in line with expectations.

The warning below should be considered as correct as the sub-directory is a module (go.mod is present) and a not a package.

>go list ./...
go: warning: "./..." matched no packages

When expecting all of them:

>go list ...
archive/tar
... (abridged as all packages of go source are listed)
example.com/m/sub