kv/kvserver: TestStoreRangeSplitRaceUninitializedRHS failed [skipped]

[cockroach-teamcity]

kv/kvserver.TestStoreRangeSplitRaceUninitializedRHS failed with artifacts on master @ 11fbbca584e3bd774ebee3ee94c49ca30a096b26:

=== RUN   TestStoreRangeSplitRaceUninitializedRHS
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/33e1d369c27b9c01b2b6009c561815a3/logTestStoreRangeSplitRaceUninitializedRHS2440527177
    test_log_scope.go:79: use -show-logs to present logs inline
    client_split_test.go:2172: split at key "a\x87\xf8" failed: fetching current range descriptor value: replica unavailable: (n2,s2):2 unable to serve request to r5:{/Systemtse-a\x87\xf9?} [(n1,s1):1, (n2,s2):2, next=3, gen=9]: closed timestamp: 1679039201.857084936,0 (2023-03-17 07:46:41); raft status: {"id":"2","term":43,"vote":"1","commit":613,"lead":"0","raftState":"StatePreCandidate","applied":613,"progress":{},"leadtransferee":"0"}: encountered poisoned latch /Local/Range/System/"tse"/RangeDescriptor@0,0
    panic.go:522: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/33e1d369c27b9c01b2b6009c561815a3/logTestStoreRangeSplitRaceUninitializedRHS2440527177
--- FAIL: TestStoreRangeSplitRaceUninitializedRHS (323.67s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

/cc @cockroachdb/kv

This test on roachdash | Improve this report!

Jira issue: CRDB-25542

[blathers-crl]

cc @cockroachdb/replication

[kvoli]

I recently re enabled this test xref #66480.

I didn't stress or stress race under a deadlock build which could be why it didnt flake over a few k runs. I also don't have a good feel for whether this test is still valuable +@erikgrinaker .

[erikgrinaker]

We'll have a look, thanks.

[cockroach-teamcity]

kv/kvserver.TestStoreRangeSplitRaceUninitializedRHS failed with artifacts on master @ 0a72a49eb6e4ce5823b53d02125441c3da2e94f5:

=== RUN   TestStoreRangeSplitRaceUninitializedRHS
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/33e1d369c27b9c01b2b6009c561815a3/logTestStoreRangeSplitRaceUninitializedRHS1222101082
    test_log_scope.go:79: use -show-logs to present logs inline
    client_split_test.go:2172: split at key "a\x87\xf9" failed: replica unavailable: (n2,s2):2 unable to serve request to r5:{/Systemtse-a\x87\xfa?} [(n1,s1):1, (n2,s2):2, next=3, gen=8]: closed timestamp: 1679125881.690391559,0 (2023-03-18 07:51:21); raft status: {"id":"2","term":36,"vote":"1","commit":717,"lead":"0","raftState":"StatePreCandidate","applied":717,"progress":{},"leadtransferee":"0"}: encountered poisoned latch /Local/Range/System/"tse"/Transaction/"2252b810-dc66-4593-81be-45ccff818af1"@0,0
    panic.go:522: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/33e1d369c27b9c01b2b6009c561815a3/logTestStoreRangeSplitRaceUninitializedRHS1222101082
--- FAIL: TestStoreRangeSplitRaceUninitializedRHS (387.51s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

[cockroach-teamcity]

kv/kvserver.TestStoreRangeSplitRaceUninitializedRHS failed with artifacts on master @ 53dbb86acb1d48309530181b94838faf937084d3:

=== RUN   TestStoreRangeSplitRaceUninitializedRHS
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/33e1d369c27b9c01b2b6009c561815a3/logTestStoreRangeSplitRaceUninitializedRHS418021715
    test_log_scope.go:79: use -show-logs to present logs inline
    client_split_test.go:2173: split at key "a\x87\xfa" failed: replica unavailable: (n2,s2):2 unable to serve request to r5:{/Systemtse-a\x87\xfb?} [(n1,s1):1, (n2,s2):2, next=3, gen=7]: closed timestamp: 1679216257.200848665,0 (2023-03-19 08:57:37); raft status: {"id":"2","term":24,"vote":"1","commit":243,"lead":"0","raftState":"StatePreCandidate","applied":243,"progress":{},"leadtransferee":"0"}: encountered poisoned latch /Local/Range/System/"tse"/Transaction/"cacf971e-9741-4c92-9759-5a5fcd8c7067"@0,0
    panic.go:540: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/33e1d369c27b9c01b2b6009c561815a3/logTestStoreRangeSplitRaceUninitializedRHS418021715
--- FAIL: TestStoreRangeSplitRaceUninitializedRHS (178.99s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

[cockroach-teamcity]

kv/kvserver.TestStoreRangeSplitRaceUninitializedRHS failed with artifacts on master @ 53dbb86acb1d48309530181b94838faf937084d3:

=== RUN   TestStoreRangeSplitRaceUninitializedRHS
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/33e1d369c27b9c01b2b6009c561815a3/logTestStoreRangeSplitRaceUninitializedRHS3929842575
    test_log_scope.go:79: use -show-logs to present logs inline
    client_split_test.go:2173: split at key "a\x87\xfa" failed: replica unavailable: (n2,s2):2 unable to serve request to r5:{/Systemtse-a\x87\xfb?} [(n1,s1):1, (n2,s2):2, next=3, gen=7]: closed timestamp: 1679214942.308604545,0 (2023-03-19 08:35:42); raft status: {"id":"2","term":45,"vote":"1","commit":796,"lead":"0","raftState":"StateFollower","applied":796,"progress":{},"leadtransferee":"0"}: encountered poisoned latch /Local/Range/System/"tse"/Transaction/"a35af954-37a8-45a1-9a49-d0bf05f0fdba"@0,0
    panic.go:522: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/33e1d369c27b9c01b2b6009c561815a3/logTestStoreRangeSplitRaceUninitializedRHS3929842575
--- FAIL: TestStoreRangeSplitRaceUninitializedRHS (292.99s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

[nicktrav]

I'm going to re-open this to track the skipped test.

[blathers-crl]

cc @cockroachdb/replication

[erikgrinaker]

It's still running, just not under deadlock/race. We have lots of these integration tests that have implicit timing dependencies because we're spinning up actual in-memory clusters, and some of them just don't work at all under deadlock/race because these are too slow, and they never will.

[nicktrav]

Ah cool. Thanks for the context.