Coverage does not work for Go tests

[JoelSpeed]

We've been trying to unify our tooling to use the Gingko cli to run all of our test suites as we are migrating towards it for as many of our tests as we can.

As part of this, we are expecting to be able to include cover profiles from the tests, even if there are some Go tests still to be run.

The issue is, if you include --cover --coverprofile=<something>, we get to the following line to start the go test:

ginkgo/ginkgo/internal/run.go

Line 34 in 143d208

suite = runGoTest(suite, cliConfig, goFlagsConfig)

Which then calls GenerateGoTestRunArgs:

ginkgo/ginkgo/internal/run.go

Line 66 in 143d208

args, err := types.GenerateGoTestRunArgs(goFlagsConfig)

Which for the Go test, will now return --test.coverprofile=<something>. Issue is, when running the Go test, Ginkgo runs it from the directory of the suite, so the coverprofile ends up in the test suite directory with the value of the Ginkgo coverprofile as its name.

Next up, is the clean up phase, FinalizeProfilesAndReportsForSuites which observes the suite and says, yep, this should have a coverprofile.

So it calculates the location of the file to pick the coverage up from:

ginkgo/ginkgo/internal/profiles_and_reports.go

Line 38 in 143d208

coverProfiles = append(coverProfiles, AbsPathForGeneratedAsset(goFlagsConfig.CoverProfile, suite, cliConfig, 0))

Which is a combination of the suite package name and the output directory - which does not align with where the file actually is.

So I see two solutions:

• Fix the finalize to detect a Go suite and pick the file up from the Go suite
• Fix the Go test args to put the coverprofile in the location expected by the finalize

Have I followed this right? Any thoughts on a preferred direction?

An example of the error output:

ginkgo run failed
  could not finalize profiles:
  Unable to read coverage file /logs/artifacts/pkg_cloud_azure_actuators_machine_test-unit-coverage.out:
  open /logs/artifacts/pkg_cloud_azure_actuators_machine_test-unit-coverage.out: no such file or directory

[onsi]

hey! sorry this isn't Just Working™ - i've managed to reproduce it locally and will take a look and circle back today/tomorrow.

[JoelSpeed]

I'm happy to work on a patch if you're open to that, would just like a little guidance on which of the suggested paths to take

[onsi]

that would be great! my instinct is to fix the go test args that are being generated and not to change the finalize. i think the main things to check are that things work for the various combinations of -output-dir (or no output-dir) and (-keep-separate-coverprofile)

thanks!

[JoelSpeed]

Looks like when running serial this problem is being solved by

ginkgo/ginkgo/internal/run.go

Lines 85 to 87 in 49fab7a

	if goFlagsConfig.Cover {
	goFlagsConfig.CoverProfile = AbsPathForGeneratedAsset(goFlagsConfig.CoverProfile, suite, cliConfig, 0)
	}

So I've raised a PR which basically does the same for the go test version. Manually tested it with various combinations and it seems to work as I'd expect

[onsi]

hey thanks for this! i've pulled the PR in and will need to cut a release next - i'd like to play with the changes a bit first and see if there's an integration test i could backfill as i clearly don't have enough coverage for standard go tests.