bench: BenchmarkAlterTableDropColumn fails: got 21.142857142857142, expected 22

[andreimatei]

Do make bench PKG=./pkg/bench/rttanalysis TESTFLAGS="-v" BENCHES=BenchmarkAlterTableDropColumn and see that multiple benchmarks fail. They seem to fail in the good direction - less roundtrips.
I think other benchmarks in the rttanalysis pkg are affected too.

I think this bit me in CI; CI runs the benchmarks with -benchtime=1ms. That seems to usually work, but I guess you can get unlucky.

[ajwerner]

I think this bit me in CI; CI runs the benchmarks with -benchtime=1ms. That seems to usually work, but I guess you can get unlucky.

This test is actually a bit meta, the go test runs itself in bench mode.

cockroach/pkg/bench/rttanalysis/validate_benchmark_data.go

Lines 80 to 84 in 6a06d56

	flags := []string{
	"--test.run=^$",
	"--test.bench=" + b,
	"--test.benchtime=1x",
	}

[andreimatei]

TIL benchtime=1x

[ajwerner]

Stale.