roachtest: import/tpch/nodes=8 failed

[cockroach-teamcity]

roachtest.import/tpch/nodes=8 failed with artifacts on release-22.1 @ d3b519c2b635a6e82be32c44f96df085ee9c7044:

The test failed on branch=release-22.1, cloud=gce:
test artifacts and logs in: /artifacts/import/tpch/nodes=8/run_1
	cluster.go:1604,import.go:215,test_runner.go:883: cluster.PutE: put /go/src/github.com/cockroachdb/cockroach/bin/cockroach failed: ~ scp -r -C -o StrictHostKeyChecking=no -i /home/roach/.ssh/id_rsa -i /home/roach/.ssh/google_compute_engine ubuntu@34.139.207.70:./cockroach ubuntu@34.148.160.129:./cockroach: exit status 1
		(1) attached stack trace
		  -- stack trace:
		  | main.(*clusterImpl).PutE
		  | 	main/pkg/cmd/roachtest/cluster.go:1618
		  | main.(*clusterImpl).Put
		  | 	main/pkg/cmd/roachtest/cluster.go:1603
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerImportTPCH.func1
		  | 	github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/import.go:215
		  | main.(*testRunner).runTest.func2
		  | 	main/pkg/cmd/roachtest/test_runner.go:883
		Wraps: (2) cluster.PutE
		Wraps: (3) attached stack trace
		  -- stack trace:
		  | github.com/cockroachdb/cockroach/pkg/roachprod/install.(*SyncedCluster).Put
		  | 	github.com/cockroachdb/cockroach/pkg/roachprod/install/cluster_synced.go:1595
		  | github.com/cockroachdb/cockroach/pkg/roachprod.Put
		  | 	github.com/cockroachdb/cockroach/pkg/roachprod/roachprod.go:818
		  | main.(*clusterImpl).PutE
		  | 	main/pkg/cmd/roachtest/cluster.go:1618
		  | main.(*clusterImpl).Put
		  | 	main/pkg/cmd/roachtest/cluster.go:1603
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerImportTPCH.func1
		  | 	github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/import.go:215
		  | [...repeated from below...]
		Wraps: (4) put /go/src/github.com/cockroachdb/cockroach/bin/cockroach failed
		Wraps: (5) attached stack trace
		  -- stack trace:
		  | github.com/cockroachdb/cockroach/pkg/roachprod/install.(*SyncedCluster).scp
		  | 	github.com/cockroachdb/cockroach/pkg/roachprod/install/cluster_synced.go:2062
		  | github.com/cockroachdb/cockroach/pkg/roachprod/install.(*SyncedCluster).Put.func3
		  | 	github.com/cockroachdb/cockroach/pkg/roachprod/install/cluster_synced.go:1504
		  | runtime.goexit
		  | 	GOROOT/src/runtime/asm_amd64.s:1581
		Wraps: (6) ~ scp -r -C -o StrictHostKeyChecking=no -i /home/roach/.ssh/id_rsa -i /home/roach/.ssh/google_compute_engine ubuntu@34.139.207.70:./cockroach ubuntu@34.148.160.129:./cockroach
		  | client_loop: send disconnect: Broken pipe
		  | lost connection
		Wraps: (7) exit status 1
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *withstack.withStack (6) *errutil.withPrefix (7) *exec.ExitError

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

• roachtest: import/tpch/nodes=8 failed #90021 roachtest: import/tpch/nodes=8 failed [C-test-failure O-roachtest O-robot T-disaster-recovery branch-release-22.2]

/cc @cockroachdb/disaster-recovery

This test on roachdash | Improve this report!

Jira issue: CRDB-21531

[blathers-crl]

cc @cockroachdb/test-eng

[msbutler]

this looks like a test infra flake. the actual roachtest never started. Start up failed on cluster.Pute, which i believe is used to put some stuff onto the roachprod cluster before the test begins.

[msbutler]

this is potentially an instance of #91771, but i'll let test-eng triage.

[smg260]

Yes, looks like a flake which will be mitigated somewhat by SCP retries