cmd/go: "go test" seems to now require git due to -buildvcs

[mvdan]

go version go1.18 linux/amd64

I have a CI job that runs go test ./... in a git clone, but inside a Docker image where git isn't installed. This worked fine in Go 1.17.x, but breaks with 1.18 unless I add -buildvcs=false.

Reproducer:

#!/bin/bash

docker run -i golang:1.18-alpine <<-SCRIPT

	set -ex

	go version

	apk update
	apk add git

	git clone --depth=1 https://github.com/mvdan/sh
	cd sh

	apk del git

	go test ./...

SCRIPT

Output:

[...]
+ go test ./...
go: downloading github.com/google/renameio v1.0.1
go: downloading github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e
go: downloading golang.org/x/term v0.0.0-20210927222741-03fcf44c2211
go: downloading mvdan.cc/editorconfig v0.2.0
go: downloading github.com/rogpeppe/go-internal v1.8.1
go: downloading golang.org/x/sync v0.0.0-20210220032951-036812b2e83c
go: downloading golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e
go: downloading github.com/creack/pty v1.1.17
go: downloading github.com/frankban/quicktest v1.14.0
go: downloading github.com/google/go-cmp v0.5.6
go: downloading gopkg.in/errgo.v2 v2.1.0
go: downloading github.com/kr/pretty v0.3.0
go: downloading github.com/kr/text v0.2.0
go: missing Git command. See https://golang.org/s/gogetcmd
error obtaining VCS status: exec: "git": executable file not found in $PATH
	Use -buildvcs=false to disable VCS stamping.
error obtaining VCS status: exec: "git": executable file not found in $PATH
	Use -buildvcs=false to disable VCS stamping.

I tried to reproduce this with a dummy module and no dependencies, even creating a dummy git repository, but all of that worked. So I imagine it has something to do with the dependencies. Note that the dependencies here are all downloaded from GOPROXY, so no VCS is needed.

cc @bcmills @matloob

[mvdan]

Another theory: ./... matches some main packages, like ./cmd/shfmt. Perhaps they are being built and then buildvcs kicks in?

[rsc]

I suppose if git is not found we should proceed without the vcs info.
I think this should only trigger if we are in a tree with a .git directory, so one option would be to delete the .git directory.

Either way, we should set -buildvcs=false during builds in go test that will be thrown away, same as we set -ldflags=-w.

[ocket8888]

It's required in general by go build, it seems, whenever go detects that it's in a git repository. That's breaking my automation build systems that don't have git installed (because they didn't need to). I assume it'll probably break some others.

[mvdan]

@ocket8888 This issue is about go test; I think you want #51748, which is about go build. They are different because VCS build stamping shouldn't be needed for running the tests.

Either way, we should set -buildvcs=false during builds in go test that will be thrown away, same as we set -ldflags=-w.

I'd agree with that. go test ./... shouldn't produce any output binaries for the user, so I can't see why -buildvcs=true should ever be part of the equation.

I've also narrowed down the cause to a package main. I imagine that go test builds all packages it needs, and for a main package, -buildvcs kicks in as per the default. So, presumably, swapping the default for go test to false should be enough.

$ cat repro.sh
#!/bin/bash

docker run -i golang:1.18-alpine <<-SCRIPT

	set -ex

	go version

	mkdir test
	cd test
	go mod init test
	mkdir .git

	cat >main.go <<-EOF
		package main

		func main() {}
	EOF

	go test

SCRIPT
$ bash repro.sh 
+ go version
go version go1.18 linux/amd64
+ mkdir test
+ cd test
+ go mod init test
go: creating new go.mod: module test
+ mkdir .git
+ cat
+ go test
go: missing Git command. See https://golang.org/s/gogetcmd
error obtaining VCS status: exec: "git": executable file not found in $PATH
	Use -buildvcs=false to disable VCS stamping.

[bcmills]

@gopherbot, please backport to Go 1.18. Invoking git for each go test of a package main is surprising and wastes users' time and battery power.

[gopherbot]

Backport issue(s) opened: #51767 (for 1.18).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://go.dev/wiki/MinorReleases.

[gopherbot]

Change https://go.dev/cl/393894 mentions this issue: cmd/go: avoid stamping VCS metadata in test binaries

[gopherbot]

Change https://go.dev/cl/393878 mentions this issue: [release-branch.go1.18] cmd/go: avoid stamping VCS metadata in test binaries

[ruchi-dhore-tw]

Is this issue still there?

I am on latest go version:
go version go1.18 darwin/amd64

Yet, I am still getting:

go: missing Git command. See https://golang.org/s/gogetcmd
error obtaining VCS status: exec: "git": executable file not found in $PATH
Use -buildvcs=false to disable VCS stamping.

I tried providing -buildvcs=false in following command, in that case as well I am getting above errors and below kafka errors:
go test ./... -race -buildvcs=false -covermode=atomic -coverprofile=coverage.out

Errors of kafka:

github.com/confluentinc/confluent-kafka-go/kafka
/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/bin/ld: /go/pkg/mod/github.com/confluentinc/confluent-kafka-go@v1.8.2/kafka/librdkafka_vendor/librdkafka_glibc_linux.a(rdkafka_txnmgr.o): in function `rd_kafka_txn_set_fatal_error':
much more such ones..

[DasSkelett]

Is this issue still there?

I am on latest go version: go version go1.18 darwin/amd64

Yes of course, Go 1.18.1 hasn't been released yet.

tried providing -buildvcs=false in following command, in that case as well I am getting above errors and below kafka errors:

That's something different, but it's missing important lines from the output to know what's wrong. You should open a new issue with the complete output.

[ruchi-dhore-tw]

@mvdan adding -buildvcs=false is not working for me [I am getting same errors the one you posted in thread], am i missing anything in the command? I am running this command inside a docker container.
go test -buildvcs=false ./... -race -covermode=atomic -coverprofile=coverage.out

[orsinium]

adding -buildvcs=false is not working for me

Are you on Go 1.18? Are you sure you've added it to the command that failed? If you have go build invoked, it needs the flag too. The following patch solved the issue for me: life4/gweb@2090dd2