kv/kvclient/kvstreamer: TestStreamerBudgetErrorInEnqueue failed

[cockroach-teamcity]

kv/kvclient/kvstreamer.TestStreamerBudgetErrorInEnqueue failed with artifacts on master @ c3d71ac887844bef174abb6dab2a4e1ce9270ab7:

=== RUN   TestStreamerBudgetErrorInEnqueue/single_key_exceeds_limit
POTENTIAL DEADLOCK: Inconsistent locking. saw this ordering in one goroutine:
happened before
streamer.go:393 kvstreamer.(*Streamer).Enqueue { s.mu.Lock() } <<<<<
streamer.go:392 kvstreamer.(*Streamer).Enqueue { // acquisitions once pipelining is implemented. }
streamer_test.go:244 kvstreamer.TestStreamerBudgetErrorInEnqueue.func3 { reqs[0] = makeGetRequest(limitBytes + 1) }

happened after
budget.go:91 kvstreamer.(*budget).consume { b.mu.Lock() } <<<<<
budget.go:90 kvstreamer.(*budget).consume { func (b *budget) consume(ctx context.Context, bytes int64, allowDebt bool) error { }
streamer.go:510 kvstreamer.(*Streamer).Enqueue { allowDebt := len(reqs) == 1 }
streamer_test.go:244 kvstreamer.TestStreamerBudgetErrorInEnqueue.func3 { reqs[0] = makeGetRequest(limitBytes + 1) }

in another goroutine: happened before
streamer.go:758 kvstreamer.(*workerCoordinator).issueRequestsForAsyncProcessing { w.s.budget.mu.Lock() } <<<<<
streamer.go:757 kvstreamer.(*workerCoordinator).issueRequestsForAsyncProcessing { }() }
streamer.go:713 kvstreamer.(*workerCoordinator).mainLoop { err := w.issueRequestsForAsyncProcessing(ctx, requestsToServe, avgResponseSize) }
../../../util/stop/stopper.go:488 stop.(*Stopper).RunAsyncTaskEx.func2 { f(ctx) }

happened after
streamer.go:602 kvstreamer.(*Streamer).getNumRequestsInFlight { s.mu.Lock() } <<<<<
streamer.go:601 kvstreamer.(*Streamer).getNumRequestsInFlight { func (s *Streamer) getNumRequestsInFlight() int { }
streamer.go:760 kvstreamer.(*workerCoordinator).issueRequestsForAsyncProcessing {  }
streamer.go:713 kvstreamer.(*workerCoordinator).mainLoop { err := w.issueRequestsForAsyncProcessing(ctx, requestsToServe, avgResponseSize) }
../../../util/stop/stopper.go:488 stop.(*Stopper).RunAsyncTaskEx.func2 { f(ctx) }

Other goroutines holding locks:
goroutine 130808 lock 0xc001017420
streamer.go:758 kvstreamer.(*workerCoordinator).issueRequestsForAsyncProcessing { w.s.budget.mu.Lock() } <<<<<
streamer.go:757 kvstreamer.(*workerCoordinator).issueRequestsForAsyncProcessing { }() }
streamer.go:713 kvstreamer.(*workerCoordinator).mainLoop { err := w.issueRequestsForAsyncProcessing(ctx, requestsToServe, avgResponseSize) }
../../../util/stop/stopper.go:488 stop.(*Stopper).RunAsyncTaskEx.func2 { f(ctx) }





ERROR: exit status 2

1 runs completed, 1 failures, over 1s
context canceled

Help

See also: How To Investigate a Go Test Failure (internal)
Parameters in this failure:

• TAGS=deadlock

• GOFLAGS=-parallel=4

/cc @cockroachdb/kv

This test on roachdash | Improve this report!

[erikgrinaker]

cc @yuzefovich