[release-branch.go1.9] cmd/go: add minimal module-awareness for legac…

…y operation

We want authors to be able to publish code that works with both
the current standard go command and the planned new go command
support for modules. If authors have tagged their code v2 or later,
semantic import versioning means the import paths must include a
v2 path element after the path prefix naming the module.
One option for making this convention compatible with original go get
is to move code into a v2 subdirectory of the root.
That makes sense for some authors, but many authors would prefer
not to move all the code into a v2 subdirectory for a transition and
then move it back up once we everyone has a module-aware go command.

Instead, this CL teaches the old (non-module-aware) go command
a tiny amount about modules and their import paths, to expand
the options for authors who want to publish compatible packages.
If an author has a v2 of a package, say my/thing/v2/sub/pkg,
in the my/thing repo's sub/pkg subdirectory (no v2 in the file system path),
then old go get continues to import that package as my/thing/sub/pkg.
But when go get is processing code in any module (code in a tree with
a go.mod file) and encounters a path like my/thing/v2/sub/pkg,
it will check to see if my/thing/go.mod says "module my/thing/v2".
If so, the go command will read the import my/thing/v2/sub/pkg
as if it said my/thing/sub/pkg, which is the correct "old" import path
for the package in question.

This CL will be back-ported to Go 1.10 and Go 1.9 as well.

Once users have updated to the latest Go point releases containing
this new logic, authors will be able to update to using modules
within their own repos, including using semantic import paths
with vN path elements, and old go get will still be able to consume
those repositories.

This CL also makes "go get" ignore meta go-import lines using
the new "mod" VCS type. This allows a package to specify both
a "mod" type and a "git" type, to present more efficient module
access to module-aware go but still present a Git repo to the old
"go get".

Fixes #24751.
Fixes #25069.

This backport to Go 1.9 also had to pick up p.Internal.RawImports
from CL 74750 and CL 74356 and use it to prepare an updated
set of -importmap arguments for the compiler. (The old code only
understood vendor-related rewriting of import paths.)

Backport fixes #25140.

Change-Id: I378955613a0d63834d4f50f121f4db7e4d87dc0a
Reviewed-on: https://go-review.googlesource.com/115298
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Andrew Bonventre <andybons@golang.org>

src/cmd/go/internal/get/discovery.go

@@ -55,6 +55,13 @@ func parseMetaGoImports(r io.Reader) (imports []metaImport, err error) {
 			continue
 		}
 		if f := strings.Fields(attrValue(e.Attr, "content")); len(f) == 3 {
+			// Ignore VCS type "mod", which is new Go modules.
+			// This code is for old go get and must ignore the new mod lines.
+			// Otherwise matchGoImport will complain about two
+			// different metaImport lines for the same Prefix.
+			if f[1] == "mod" {
+				continue
+			}
 			imports = append(imports, metaImport{
 				Prefix:   f[0],
 				VCS:      f[1],

src/cmd/go/internal/get/get.go

@@ -210,7 +210,7 @@ var downloadRootCache = map[string]bool{}
 // download runs the download half of the get command
 // for the package named by the argument.
 func download(arg string, parent *load.Package, stk *load.ImportStack, mode int) {
-	if mode&load.UseVendor != 0 {
+	if mode&load.ResolveImport != 0 {
 		// Caller is responsible for expanding vendor paths.
 		panic("internal error: download mode has useVendor set")
 	}

src/cmd/go/internal/get/pkg_test.go

@@ -47,6 +47,20 @@ var parseMetaGoImportsTests = []struct {
 			{"baz/quux", "git", "http://github.com/rsc/baz/quux"},
 		},
 	},
+	{
+		`<meta name="go-import" content="foo/bar git https://github.com/rsc/foo/bar">
+		<meta name="go-import" content="foo/bar mod http://github.com/rsc/baz/quux">`,
+		[]metaImport{
+			{"foo/bar", "git", "https://github.com/rsc/foo/bar"},
+		},
+	},
+	{
+		`<meta name="go-import" content="foo/bar mod http://github.com/rsc/baz/quux">
+		<meta name="go-import" content="foo/bar git https://github.com/rsc/foo/bar">`,
+		[]metaImport{
+			{"foo/bar", "git", "https://github.com/rsc/foo/bar"},
+		},
+	},
 	{
 		`<head>
 		<meta name="go-import" content="foo/bar git https://github.com/rsc/foo/bar">

src/cmd/go/internal/list/list.go

@@ -201,8 +201,8 @@ func runList(cmd *base.Command, args []string) {
 
 	for _, pkg := range loadpkgs(args) {
 		// Show vendor-expanded paths in listing
-		pkg.TestImports = pkg.Vendored(pkg.TestImports)
-		pkg.XTestImports = pkg.Vendored(pkg.XTestImports)
+		pkg.TestImports = pkg.Resolve(pkg.TestImports)
+		pkg.XTestImports = pkg.Resolve(pkg.XTestImports)
 
 		do(&pkg.PackagePublic)
 	}