Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

-shuffle seed should be displayed in the summary #255

Closed
kevinawoo opened this issue Jun 16, 2022 · 1 comment · Fixed by #256
Closed

-shuffle seed should be displayed in the summary #255

kevinawoo opened this issue Jun 16, 2022 · 1 comment · Fixed by #256
Labels
enhancement New feature or request

Comments

@kevinawoo
Copy link

kevinawoo commented Jun 16, 2022

When using go test -shuffle=on ./..., this will print out a shuffle seed

Example output if there's a failure:

ok  	github.com/kevinawoo/test/pkg/example	0.686s
-test.shuffle 1655405871045329000
--- FAIL: TestExample (0.00s)
panic: runtime error: index out of range [99] with length 1 [recovered]
	panic: runtime error: index out of range [99] with length 1

The seed 1655405871045329000 allows us to reproduce the same test shuffle to reproduce failures.

go test -json -shuffle=on ./... output does show the shuffle seed, example:

{"Time":"2022-06-16T11:52:52.124788-07:00","Action":"output","Package":"github.com/kevinawoo/test/pkg/example","Output":"-test.shuffle 1655405871045329000\n"}

However when we run it go run gotest.tools/gotestsum@latest -- -shuffle=on ./..., the shuffle seed doesn't get printed out.

@dnephin
Copy link
Member

dnephin commented Jun 16, 2022

Thank you for opening this issue! It looks like there will need to be special handling for a few of the formats.

  • testname - does print the -test.shuffle line, but it's not clear exactly which package the line is associated with. Maybe it should only be printed on failure.
  • pkgname (the default format) - does not print it on failure
  • standard-quiet - prints it all the time, but shouldn't

The rest seem to do the right thing.

Since the output is associated with the package (not a specific test), I think we'll have to handle it similar to cached, and coverage. Store it on Package, and print it as part of the package output.

@dnephin dnephin added the enhancement New feature or request label Jun 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants