cmd/go: go get -d fails with "no go-import meta tag"

[robcapo]

Hello, I've noticed that occasionally our CI builds have been failing during go get. It doesn't seem to be one package that's causing this to happen, and the issue usually resolves itself quickly, but the flakiness is causing alerts in our environments. I was asked to create a new issue in https://github.com/golang/go/issues/27622#issuecomment-420312891

Two examples follow:

package golang.org/x/crypto/pbkdf2: unrecognized import path "golang.org/x/crypto/pbkdf2" (fetch https://golang.org/x/crypto?go-get=1: no go-import meta tag)

package golang.org/x/net/html/charset: unrecognized import path "golang.org/x/net/html/charset" (parse https://golang.org/x/net/html/charset?go-get=1: no go-import meta tags ())

What version of Go are you using (go version)?

go version go1.9 linux/amd64

Does this issue reproduce with the latest release?

It's hard to say because it's flaky. I'm currently updating our CI servers to 1.10.

What operating system and processor architecture are you using (go env)?

GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/rcapo/go"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build837121565=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"

What did you do?

Ran go get -d ./... 2>&1 as part of a jenkins job.

What did you expect to see?

Nothing. Successful update

What did you see instead?

Errors above (both are golang.org/x/ packages)

package golang.org/x/...: unrecognized import path "golang.org/x/..." (parse https://golang.org/x/...?go-get=1: no go-import meta tags ())

[bcmills]

occasionally our CI builds have been failing

Are you running multiple go get operations concurrently (#26794)?

Does this occur only for golang.org/x paths, or for arbitrary modules? (Is it possible that the golang.org servers are just flaky?)

[bcmills]

go version go1.9 linux/amd64

Note that go1.9 is not longer supported per the release policy. Can you reproduce the failures on 1.10 or 1.11?

[robcapo]

It's only happened for golang.org/x paths so far. Only running one go get at a time.

I think it may just be flakiness on the servers.

[robcapo]

Sure, I'm updating go on the build servers now. I'll report back if I notice the issue again.

[lshulyay]

I just saw this issue on my CI, on Go 1.11, using docker image golang:1.11-alpine. A rerun seems to "fix" it.

package golang.org/x/crypto/pbkdf2: unrecognized import path "golang.org/x/crypto/pbkdf2" (parse https://golang.org/x/crypto/pbkdf2?go-get=1: no go-import meta tags ())

The full log from the job below:

Running with gitlab-runner 11.3.0-rc1 (861c565e)
on docker-auto-scale ed2dce3a
Using Docker executor with image golang:1.11-alpine ...
Pulling docker image golang:1.11-alpine ...
Using docker image sha256:233ed4ed14bff3bd26934c973cc89ed04078ac95143fa741c3fbe1602fe20df9 for golang:1.11-alpine ...
Running on runner-ed2dce3a-project-3575820-concurrent-0 via runner-ed2dce3a-srm-1537729575-628cda11...
Cloning repository...
Cloning into '/builds/drakonka/gosnaillife'...
Checking out e1d1bcce as master...
Skipping Git submodules setup
$ apk add --no-cache git build-base
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/community/x86_64/APKINDEX.tar.gz
(1/25) Installing binutils (2.30-r5)
(2/25) Installing libmagic (5.32-r0)
(3/25) Installing file (5.32-r0)
(4/25) Installing gmp (6.1.2-r1)
(5/25) Installing isl (0.18-r0)
(6/25) Installing libgomp (6.4.0-r8)
(7/25) Installing libatomic (6.4.0-r8)
(8/25) Installing pkgconf (1.5.3-r0)
(9/25) Installing libgcc (6.4.0-r8)
(10/25) Installing mpfr3 (3.1.5-r1)
(11/25) Installing mpc1 (1.0.3-r1)
(12/25) Installing libstdc++ (6.4.0-r8)
(13/25) Installing gcc (6.4.0-r8)
(14/25) Installing musl-dev (1.1.19-r10)
(15/25) Installing libc-dev (0.7.1-r0)
(16/25) Installing g++ (6.4.0-r8)
(17/25) Installing make (4.2.1-r2)
(18/25) Installing fortify-headers (0.9-r0)
(19/25) Installing build-base (0.5-r1)
(20/25) Installing nghttp2-libs (1.32.0-r0)
(21/25) Installing libssh2 (1.8.0-r3)
(22/25) Installing libcurl (7.61.1-r0)
(23/25) Installing expat (2.2.5-r0)
(24/25) Installing pcre2 (10.31-r0)
(25/25) Installing git (2.18.0-r0)
Executing busybox-1.28.4-r1.trigger
OK: 180 MiB in 39 packages
$ apk add --no-cache bash
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/community/x86_64/APKINDEX.tar.gz
(1/5) Installing ncurses-terminfo-base (6.1_p20180818-r1)
(2/5) Installing ncurses-terminfo (6.1_p20180818-r1)
(3/5) Installing ncurses-libs (6.1_p20180818-r1)
(4/5) Installing readline (7.0.003-r0)
(5/5) Installing bash (4.4.19-r1)
Executing bash-4.4.19-r1.post-install
Executing busybox-1.28.4-r1.trigger
OK: 189 MiB in 44 packages
$ mkdir -p $GOPATH/src/$(dirname $REPO_NAME)
$ cp -a /$CI_PROJECT_DIR/. $GOPATH/src/$REPO_NAME
$ cd $GOPATH/src/$REPO_NAME
$ go version
go version go1.11 linux/amd64
$ go get ./...
package golang.org/x/crypto/pbkdf2: unrecognized import path "golang.org/x/crypto/pbkdf2" (parse https://golang.org/x/crypto/pbkdf2?go-get=1: no go-import meta tags ())
ERROR: Job failed: exit code 1

[bcmills]

I believe that a fix to #28194 will also address this symptom. Closing as duplicate.

[willsr]

Seeing the same issue via bitbucket pipelines on all "golang.org/x/.." dependencies

[agnivade]

Please subscribe to #30619.