DataRace in IndexMergeReaderExec #31229

guo-shaoge · 2021-12-31T08:46:36Z

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

cd $working_dir/tidb/planner/core && go test . -run TestT -check.f testIntegrationSuite.TestIndexMergeWithCorrelatedColumns -race -count 100

2. What did you expect to see? (Required)

runs ok.

3. What did you see instead (Required)

==================
WARNING: DATA RACE
Write at 0x00c005dc57e0 by goroutine 182:
  github.com/pingcap/tidb/executor.(*IndexMergeReaderExecutor).startPartialIndexWorker.func1.1()
      /go/tidb/executor/index_merge_reader.go:291 +0x444
  github.com/pingcap/tidb/util.WithRecovery()
      /go/tidb/util/misc.go:100 +0x77
  github.com/pingcap/tidb/executor.(*IndexMergeReaderExecutor).startPartialIndexWorker.func1()
      /go/tidb/executor/index_merge_reader.go:278 +0x290
Previous write at 0x00c005dc57e0 by goroutine 176:
  github.com/pingcap/tidb/executor.(*IndexMergeReaderExecutor).startPartialIndexWorker()
      /go/tidb/executor/index_merge_reader.go:359 +0x5a5
  github.com/pingcap/tidb/executor.(*IndexMergeReaderExecutor).startWorkers()
      /go/tidb/executor/index_merge_reader.go:214 +0x1cb
  github.com/pingcap/tidb/executor.(*IndexMergeReaderExecutor).Next()
      /go/tidb/executor/index_merge_reader.go:629 +0x8e
  github.com/pingcap/tidb/executor.Next()
      /go/tidb/executor/executor.go:286 +0x58f
  github.com/pingcap/tidb/executor.(*StreamAggExec).consumeCurGroupRowsAndFetchChild()
      /go/tidb/executor/aggregate.go:1367 +0x13e
  github.com/pingcap/tidb/executor.(*StreamAggExec).consumeOneGroup()
      /go/tidb/executor/aggregate.go:1295 +0xee
  github.com/pingcap/tidb/executor.(*StreamAggExec).Next()
      /go/tidb/executor/aggregate.go:1284 +0x15d
  github.com/pingcap/tidb/executor.Next()
      /go/tidb/executor/executor.go:286 +0x58f
  github.com/pingcap/tidb/executor.(*NestedLoopApplyExec).fetchAllInners()
      /go/tidb/executor/join.go:979 +0x6bd
  github.com/pingcap/tidb/executor.(*NestedLoopApplyExec).Next()
      /go/tidb/executor/join.go:1032 +0x54d
  github.com/pingcap/tidb/executor.Next()
      /go/tidb/executor/executor.go:286 +0x58f
  github.com/pingcap/tidb/executor.(*projectionInputFetcher).run()
      /go/tidb/executor/projection.go:381 +0x468
  github.com/pingcap/tidb/executor.(*ProjectionExec).prepare·dwrap·115()
      /go/tidb/executor/projection.go:272 +0x58
Goroutine 182 (running) created at:
  github.com/pingcap/tidb/executor.(*IndexMergeReaderExecutor).startPartialIndexWorker()
      /go/tidb/executor/index_merge_reader.go:275 +0x598
  github.com/pingcap/tidb/executor.(*IndexMergeReaderExecutor).startWorkers()
      /go/tidb/executor/index_merge_reader.go:214 +0x1cb
  github.com/pingcap/tidb/executor.(*IndexMergeReaderExecutor).Next()
      /go/tidb/executor/index_merge_reader.go:629 +0x8e
  github.com/pingcap/tidb/executor.Next()
      /go/tidb/executor/executor.go:286 +0x58f
  github.com/pingcap/tidb/executor.(*StreamAggExec).consumeCurGroupRowsAndFetchChild()
      /go/tidb/executor/aggregate.go:1367 +0x13e
  github.com/pingcap/tidb/executor.(*StreamAggExec).consumeOneGroup()
      /go/tidb/executor/aggregate.go:1295 +0xee
  github.com/pingcap/tidb/executor.(*StreamAggExec).Next()
      /go/tidb/executor/aggregate.go:1284 +0x15d
  github.com/pingcap/tidb/executor.Next()
      /go/tidb/executor/executor.go:286 +0x58f
  github.com/pingcap/tidb/executor.(*NestedLoopApplyExec).fetchAllInners()
      /go/tidb/executor/join.go:979 +0x6bd
  github.com/pingcap/tidb/executor.(*NestedLoopApplyExec).Next()
      /go/tidb/executor/join.go:1032 +0x54d
  github.com/pingcap/tidb/executor.Next()
      /go/tidb/executor/executor.go:286 +0x58f
  github.com/pingcap/tidb/executor.(*projectionInputFetcher).run()
      /go/tidb/executor/projection.go:381 +0x468
  github.com/pingcap/tidb/executor.(*ProjectionExec).prepare·dwrap·115()
      /go/tidb/executor/projection.go:272 +0x58
Goroutine 176 (running) created at:
  github.com/pingcap/tidb/executor.(*ProjectionExec).prepare()
      /go/tidb/executor/projection.go:272 +0xc08
  github.com/pingcap/tidb/executor.(*ProjectionExec).parallelExecute()
      /go/tidb/executor/projection.go:209 +0xca
  github.com/pingcap/tidb/executor.(*ProjectionExec).Next()
      /go/tidb/executor/projection.go:182 +0xc4
  github.com/pingcap/tidb/executor.Next()
      /go/tidb/executor/executor.go:286 +0x58f
  github.com/pingcap/tidb/executor.(*SortExec).fetchRowChunks()
      /go/tidb/executor/sort.go:195 +0xf73
  github.com/pingcap/tidb/executor.(*SortExec).Next()
      /go/tidb/executor/sort.go:113 +0xc4
  github.com/pingcap/tidb/executor.Next()
      /go/tidb/executor/executor.go:286 +0x58f
  github.com/pingcap/tidb/executor.(*recordSet).Next()
      /go/tidb/executor/adapter.go:149 +0x13b
  github.com/pingcap/tidb/session.(*execStmtResult).Next()
      <autogenerated>:1 +0x7b
  github.com/pingcap/tidb/session.GetRows4Test()
      /go/tidb/session/tidb.go:325 +0xea
  github.com/pingcap/tidb/session.ResultSetToStringSlice()
      /go/tidb/session/tidb.go:343 +0xb4
  github.com/pingcap/tidb/util/testkit.(*TestKit).ResultSetToResultWithCtx()
      /go/tidb/util/testkit/testkit.go:418 +0xbd
  github.com/pingcap/tidb/util/testkit.(*TestKit).ResultSetToResult()
      /go/tidb/util/testkit/testkit.go:413 +0x418
  github.com/pingcap/tidb/util/testkit.(*TestKit).MustQuery()
      /go/tidb/util/testkit/testkit.go:356 +0x420
  github.com/pingcap/tidb/planner/core_test.(*testIntegrationSuite).TestIndexMergeWithCorrelatedColumns()
      /go/tidb/planner/core/integration_test.go:5149 +0x4fb
  runtime.call16()
      /usr/local/go/src/runtime/asm_amd64.s:625 +0x48
  reflect.Value.Call()
      /usr/local/go/src/reflect/value.go:339 +0xd7
  github.com/pingcap/check.(*suiteRunner).forkTest.func1()
      /home/prow/go/pkg/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:850 +0xa54
  github.com/pingcap/check.(*suiteRunner).forkCall.func1()
      /home/prow/go/pkg/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:739 +0x12b
==================

4. What is your TiDB version? (Required)

master

The text was updated successfully, but these errors were encountered:

close #31229

github-actions · 2021-12-31T09:42:03Z

Please check whether the issue should be labeled with 'affects-x.y' or 'fixes-x.y.z', and then remove 'needs-more-info' label.

* topsql: make topsql enable only be controlled by pub/sub sink (pingcap#31209) * ddl: support batch create table (pingcap#28763) * executor: fix data race in IndexMergeReaderExec (pingcap#31230) close pingcap#31229 * server: filter the EOF error for normal closed at handshake (pingcap#31081) close pingcap#31063 * expression: change date add function return type (pingcap#28133) close pingcap#27573 * support create interval partition Signed-off-by: crazycs520 <crazycs520@gmail.com> * support create interval partition (support int/timestamp partition key) Signed-off-by: crazycs520 <crazycs520@gmail.com> * parser: support alter table partitions move engine statement Signed-off-by: crazycs520 <crazycs520@gmail.com> * support ddl operation Signed-off-by: crazycs520 <crazycs520@gmail.com> * support interval partition manager Signed-off-by: crazycs520 <crazycs520@gmail.com> * support interval partition manager handle job framwork Signed-off-by: crazycs520 <crazycs520@gmail.com> * support auto create interval partition when insert meet no partition suitable error Signed-off-by: crazycs520 <crazycs520@gmail.com> * fix bug Signed-off-by: crazycs520 <crazycs520@gmail.com> * fix cancel job and load old job then continue to do Signed-off-by: crazycs520 <crazycs520@gmail.com> * make partition readonly work(not allow to insert/update/delete) Signed-off-by: crazycs520 <crazycs520@gmail.com> * add begin,end time in tables Signed-off-by: crazycs520 <crazycs520@gmail.com> * tiny fix for auto create interval partition in concurrent case Signed-off-by: crazycs520 <crazycs520@gmail.com> * init Signed-off-by: crazycs520 <crazycs520@gmail.com> * init Signed-off-by: crazycs520 <crazycs520@gmail.com> * todo: remove flag Signed-off-by: crazycs520 <crazycs520@gmail.com> * fix dumpling Signed-off-by: crazycs520 <crazycs520@gmail.com> * remove data in aws s3 when drop/truncate table/partition Signed-off-by: crazycs520 <crazycs520@gmail.com> * make hello world work Signed-off-by: crazycs520 <crazycs520@gmail.com> * remove debug info Signed-off-by: crazycs520 <crazycs520@gmail.com> Co-authored-by: xhe <xw897002528@gmail.com> Co-authored-by: guo-shaoge <shaoge1994@163.com> Co-authored-by: knull-cn <hu__haifeng@163.com> Co-authored-by: Meng Xin <tregoldmeng@gmail.com>

close #31229

guo-shaoge added the type/bug The issue is confirmed as a bug. label Dec 31, 2021

guo-shaoge self-assigned this Dec 31, 2021

guo-shaoge mentioned this issue Dec 31, 2021

executor: fix data race in IndexMergeReaderExec #31230

Merged

12 tasks

ti-chi-bot closed this as completed in #31230 Dec 31, 2021

ti-chi-bot pushed a commit that referenced this issue Dec 31, 2021

executor: fix data race in IndexMergeReaderExec (#31230)

ceed548

close #31229

github-actions bot added the needs-more-info label Dec 31, 2021

ti-srebot mentioned this issue Dec 31, 2021

executor: fix data race in IndexMergeReaderExec (#31230) #31235

Merged

12 tasks

djshow832 added severity/major affects-5.4 This bug affects the 5.4.x(LTS) versions. labels Jan 4, 2022

ti-chi-bot pushed a commit that referenced this issue Jan 6, 2022

executor: fix data race in IndexMergeReaderExec (#31230) (#31235)

e2ab8a5

close #31229

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DataRace in IndexMergeReaderExec #31229

DataRace in IndexMergeReaderExec #31229

guo-shaoge commented Dec 31, 2021

github-actions bot commented Dec 31, 2021

DataRace in IndexMergeReaderExec #31229

DataRace in IndexMergeReaderExec #31229

Comments

guo-shaoge commented Dec 31, 2021

Bug Report

1. Minimal reproduce step (Required)

2. What did you expect to see? (Required)

3. What did you see instead (Required)

4. What is your TiDB version? (Required)

github-actions bot commented Dec 31, 2021