Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

teamcity: failed test: TestStoreRangeMergeWatcher #31224

Closed
cockroach-teamcity opened this issue Oct 10, 2018 · 1 comment
Closed

teamcity: failed test: TestStoreRangeMergeWatcher #31224

cockroach-teamcity opened this issue Oct 10, 2018 · 1 comment
Labels
C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot.
Milestone
19.1

Comments

@cockroach-teamcity
Copy link
Member

The following tests appear to have failed on master (test): TestStoreRangeMergeWatcher, TestStoreRangeMergeWatcher/inject-failures=true, TestStoreRangeMergeWatcher/inject-failures=false

You may want to check for open issues.

#957198:

TestStoreRangeMergeWatcher/inject-failures=false
...snapshot to (n2,s2):?: kv pairs: 49, log entries: 6, rate-limit: 2.0 MiB/sec, 1ms
I181010 21:08:37.113589 47682 storage/replica_raftstorage.go:803  [s2,r1/?:{-}] applying preemptive snapshot at index 16 (id=e4eca666, encoded size=8342, 1 rocksdb batches, 6 log entries)
I181010 21:08:37.114314 47682 storage/replica_raftstorage.go:809  [s2,r1/?:/M{in-ax}] applied preemptive snapshot in 1ms [clear=0ms batch=0ms entries=0ms commit=0ms]
I181010 21:08:37.114735 47419 storage/replica_command.go:812  [s1,r1/1:/M{in-ax}] change replicas (ADD_REPLICA (n2,s2):2): read existing descriptor r1:/M{in-ax} [(n1,s1):1, next=2, gen=0]
I181010 21:08:37.121031 47419 storage/replica.go:3899  [s1,r1/1:/M{in-ax},txn=0a070d76] proposing ADD_REPLICA((n2,s2):2): updated=[(n1,s1):1 (n2,s2):2] next=3
I181010 21:08:37.122890 47419 storage/store_snapshot.go:615  [s1,r1/1:/M{in-ax}] sending preemptive snapshot c2a7e106 at applied index 18
I181010 21:08:37.125137 47419 storage/store_snapshot.go:657  [s1,r1/1:/M{in-ax}] streamed snapshot to (n3,s3):?: kv pairs: 52, log entries: 8, rate-limit: 2.0 MiB/sec, 3ms
I181010 21:08:37.126234 47816 storage/replica_raftstorage.go:803  [s3,r1/?:{-}] applying preemptive snapshot at index 18 (id=c2a7e106, encoded size=9284, 1 rocksdb batches, 8 log entries)
I181010 21:08:37.127228 47816 storage/replica_raftstorage.go:809  [s3,r1/?:/M{in-ax}] applied preemptive snapshot in 1ms [clear=0ms batch=0ms entries=1ms commit=0ms]
I181010 21:08:37.127915 47419 storage/replica_command.go:812  [s1,r1/1:/M{in-ax}] change replicas (ADD_REPLICA (n3,s3):3): read existing descriptor r1:/M{in-ax} [(n1,s1):1, (n2,s2):2, next=3, gen=0]
I181010 21:08:37.132623 47419 storage/replica.go:3899  [s1,r1/1:/M{in-ax},txn=7244fa08] proposing ADD_REPLICA((n3,s3):3): updated=[(n1,s1):1 (n2,s2):2 (n3,s3):3] next=4
I181010 21:08:37.280946 47419 storage/replica_command.go:298  [s1,r1/1:/M{in-ax}] initiating a split of this range at key "b" [r2]
E181010 21:08:37.293162 47578 storage/queue.go:791  [raftsnapshot,s1,r2/1:{b-/Max}] snapshot failed: (n3,s3):3: remote couldn't accept Raft snapshot c6a67e52 at applied index 11 with error: [n3,s3],r2: cannot apply snapshot: snapshot intersects existing range [n3,s3,r1/3:/M{in-ax}]
I181010 21:08:37.501783 47822 storage/store_snapshot.go:615  [raftsnapshot,s1,r2/1:{b-/Max}] sending Raft snapshot 20d907d2 at applied index 13
I181010 21:08:37.502206 47822 storage/store_snapshot.go:657  [raftsnapshot,s1,r2/1:{b-/Max}] streamed snapshot to (n3,s3):3: kv pairs: 42, log entries: 3, rate-limit: 8.0 MiB/sec, 10ms
I181010 21:08:37.503100 47825 storage/replica_raftstorage.go:803  [s3,r2/3:{b-/Max}] applying Raft snapshot at index 13 (id=20d907d2, encoded size=7785, 1 rocksdb batches, 3 log entries)
I181010 21:08:37.503850 47825 storage/replica_raftstorage.go:809  [s3,r2/3:{b-/Max}] applied Raft snapshot in 1ms [clear=0ms batch=0ms entries=0ms commit=0ms]
I181010 21:08:37.505546 47419 storage/replica_command.go:430  [s1,r1/1:{/Min-b}] initiating a merge of r2:{b-/Max} [(n1,s1):1, (n2,s2):2, (n3,s3):3, next=4, gen=0] into this range
I181010 21:08:37.514223 47465 storage/store.go:2744  [s1,r1/1:{/Min-b},txn=e200b868] removing replica r2/1
I181010 21:08:37.514364 47628 storage/store.go:2744  [s2,r1/2:{/Min-b}] removing replica r2/2
I181010 21:08:37.515820 47714 storage/store.go:2744  [s3,r1/3:{/Min-b}] removing replica r2/3
W181010 21:08:37.517828 47573 storage/raft_transport.go:281  unable to accept Raft message from (n2,s2):2: no handler registered for (n1,s1):1
W181010 21:08:37.517916 47573 storage/raft_transport.go:281  unable to accept Raft message from (n2,s2):2: no handler registered for (n1,s1):1
W181010 21:08:37.517939 47573 storage/raft_transport.go:281  unable to accept Raft message from (n3,s3):3: no handler registered for (n1,s1):1
W181010 21:08:37.518302 47819 storage/raft_transport.go:583  while processing outgoing Raft queue to node 1: rpc error: code = Unavailable desc = transport is closing:



TestStoreRangeMergeWatcher
--- FAIL: testrace/TestStoreRangeMergeWatcher (3.070s)
Test ended in panic.




TestStoreRangeMergeWatcher/inject-failures=true
...til/stop/stopper.go:199 +0xe9
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:192 +0xad

goroutine 47952 [semacquire]:
sync.runtime_notifyListWait(0xc4212208d0, 0x77d4)
	/usr/local/go/src/runtime/sema.go:510 +0x10b
sync.(*Cond).Wait(0xc4212208c0)
	/usr/local/go/src/sync/cond.go:56 +0x80
github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc421620f00, 0x2c3d680, 0xc42173e840)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:196 +0x7c
github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x2c3d680, 0xc42173e840)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:165 +0x3e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc4217c0880, 0xc421819320, 0xc4217c0870)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:199 +0xe9
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:192 +0xad

goroutine 48002 [chan receive, 7 minutes]:
github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func1(0x2c3d680, 0xc421de4090)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:155 +0x49
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc4203afbd0, 0xc421020510, 0xc4204a8980)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:199 +0xe9
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:192 +0xad

goroutine 47999 [select]:
github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/transport.(*http2Client).keepalive(0xc420e46b40)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/transport/http2_client.go:1176 +0x137
created by github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/transport.newHTTP2Client
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/transport/http2_client.go:309 +0xd02

goroutine 47916 [semacquire]:
sync.runtime_notifyListWait(0xc4212208d0, 0x77cd)
	/usr/local/go/src/runtime/sema.go:510 +0x10b
sync.(*Cond).Wait(0xc4212208c0)
	/usr/local/go/src/sync/cond.go:56 +0x80
github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc421620f00, 0x2c3d680, 0xc42099da70)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:196 +0x7c
github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x2c3d680, 0xc42099da70)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:165 +0x3e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc4217c0400, 0xc421819320, 0xc4217c03f0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:199 +0xe9
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:192 +0xad

goroutine 48061 [semacquire]:
sync.runtime_notifyListWait(0xc42165b010, 0x5bb0)
	/usr/local/go/src/runtime/sema.go:510 +0x10b
sync.(*Cond).Wait(0xc42165b000)
	/usr/local/go/src/sync/cond.go:56 +0x80
github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc4217be600, 0x2c3d680, 0xc4214523c0)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:196 +0x7c
github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x2c3d680, 0xc4214523c0)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:165 +0x3e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc4213b5140, 0xc420c6fb90, 0xc4213b5130)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:199 +0xe9
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:192 +0xad



TestStoreRangeMergeWatcher
--- FAIL: test/TestStoreRangeMergeWatcher (0.000s)
Test ended in panic.

Please assign, take a look and update the issue accordingly.
@cockroach-teamcity cockroach-teamcity added this to the 2.2 milestone Oct 10, 2018
@cockroach-teamcity cockroach-teamcity added C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. labels Oct 10, 2018
@tbg
Copy link
Member

tbg commented Oct 10, 2018

About to be fixed in #31215

@tbg tbg closed this as completed Oct 10, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot.
Projects
None yet
Milestone
19.1
Development

No branches or pull requests
2 participants
@tbg @cockroach-teamcity