Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DATA RACE in the ResetContextOfStmt #31062

Closed
hawkingrei opened this issue Dec 27, 2021 · 5 comments · Fixed by #31334
Closed

DATA RACE in the ResetContextOfStmt #31062

hawkingrei opened this issue Dec 27, 2021 · 5 comments · Fixed by #31334
Assignees
@XuHuaiyu
Labels
affects-5.2 This bug affects 5.2.x versions. severity/major sig/execution SIG execution type/bug The issue is confirmed as a bug.

Comments

@hawkingrei
Copy link
Member

hawkingrei commented Dec 27, 2021 •
edited
Loading

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

 WARNING: DATA RACE
Write at 0x00c0754fc898 by goroutine 83:
  github.com/pingcap/tidb/executor.ResetContextOfStmt()
      /go/tidb/executor/executor.go:1860 +0x297e
  github.com/pingcap/tidb/session.(*session).ExecuteStmt()
      /go/tidb/session/session.go:1650 +0x3c4
  github.com/pingcap/tidb/util/testkit.(*TestKit).Exec()
      /go/tidb/util/testkit/testkit.go:199 +0x7ed
  github.com/pingcap/tidb/util/testkit.(*TestKit).MustExec()
      /go/tidb/util/testkit/testkit.go:245 +0x99
  github.com/pingcap/tidb/executor_test.(*testSerialSuite).TestIssue28650()
      /go/tidb/executor/executor_test.go:9498 +0x56c
  github.com/pingcap/tidb/util/memory.(*Tracker).Consume()
      /go/tidb/util/memory/tracker.go:344 +0x192
  github.com/pingcap/tidb/planner/core.(*indexJoinBuildHelper).buildTemplateRange()
      /go/tidb/planner/core/exhaust_physical_plans.go:1576 +0x9af
  github.com/pingcap/tidb/planner/core.(*indexJoinBuildHelper).analyzeLookUpFilters()
      /go/tidb/planner/core/exhaust_physical_plans.go:1426 +0x626
  github.com/pingcap/tidb/planner/core.(*LogicalJoin).getIndexJoinBuildHelper()
      /go/tidb/planner/core/exhaust_physical_plans.go:715 +0x25b
  github.com/pingcap/tidb/planner/core.(*LogicalJoin).buildIndexJoinInner2IndexScan()
      /go/tidb/planner/core/exhaust_physical_plans.go:832 +0xd2
  github.com/pingcap/tidb/planner/core.(*LogicalJoin).getIndexJoinByOuterIdx()
      /go/tidb/planner/core/exhaust_physical_plans.go:705 +0x764
  github.com/pingcap/tidb/planner/core.(*LogicalJoin).tryToGetIndexJoin()
      /go/tidb/planner/core/exhaust_physical_plans.go:1713 +0x544
  github.com/pingcap/tidb/planner/core.(*LogicalJoin).exhaustPhysicalPlans()
      /go/tidb/planner/core/exhaust_physical_plans.go:1821 +0x8c7
  github.com/pingcap/tidb/planner/core.(*baseLogicalPlan).findBestTask()
      /go/tidb/planner/core/find_best_task.go:317 +0x565
  github.com/pingcap/tidb/planner/core.(*LogicalJoin).findBestTask()
      <autogenerated>:1 +0x5e
  github.com/pingcap/tidb/planner/core.(*baseLogicalPlan).enumeratePhysicalPlans4Task()
      /go/tidb/planner/core/find_best_task.go:220 +0x818
  github.com/pingcap/tidb/planner/core.(*baseLogicalPlan).findBestTask()
      /go/tidb/planner/core/find_best_task.go:356 +0x845
  github.com/pingcap/tidb/planner/core.(*LogicalAggregation).findBestTask()
      <autogenerated>:1 +0x5e
  github.com/pingcap/tidb/planner/core.(*baseLogicalPlan).enumeratePhysicalPlans4Task()
      /go/tidb/planner/core/find_best_task.go:220 +0x818
  github.com/pingcap/tidb/planner/core.(*baseLogicalPlan).findBestTask()
      /go/tidb/planner/core/find_best_task.go:356 +0x845
  github.com/pingcap/tidb/planner/core.(*LogicalProjection).findBestTask()
      <autogenerated>:1 +0x5e
  github.com/pingcap/tidb/planner/core.physicalOptimize()
      /go/tidb/planner/core/optimizer.go:453 +0x1b4
  github.com/pingcap/tidb/planner/core.DoOptimize()
      /go/tidb/planner/core/optimizer.go:281 +0x1fb
  github.com/pingcap/tidb/planner.optimize()
      /go/tidb/planner/optimize.go:389 +0xb48
  github.com/pingcap/tidb/planner.Optimize()
      /go/tidb/planner/optimize.go:219 +0xa13
  github.com/pingcap/tidb/planner/core.(*PlanBuilder).buildExplain()
      /go/tidb/planner/core/planbuilder.go:4150 +0x211
  github.com/pingcap/tidb/planner/core.(*PlanBuilder).Build()
      /go/tidb/planner/core/planbuilder.go:693 +0xbc8
  github.com/pingcap/tidb/planner.optimize()
      /go/tidb/planner/optimize.go:336 +0x484
  github.com/pingcap/tidb/planner.Optimize()
      /go/tidb/planner/optimize.go:219 +0xa13
  github.com/pingcap/tidb/executor.(*Compiler).Compile()
      /go/tidb/executor/compiler.go:77 +0x704
  github.com/pingcap/tidb/session.(*session).ExecuteStmt()
      /go/tidb/session/session.go:1677 +0x757
  github.com/pingcap/tidb/util/testkit.(*TestKit).Exec()
      /go/tidb/util/testkit/testkit.go:199 +0x7ed
  github.com/pingcap/tidb/util/testkit.(*TestKit).MustExec()
      /go/tidb/util/testkit/testkit.go:245 +0x99
  github.com/pingcap/tidb/executor_test.(*testSerialSuite).TestIssue28650.func4()
      /go/tidb/executor/executor_test.go:9506 +0xc7
  github.com/pingcap/tidb/executor_test.(*testSerialSuite).TestIssue28650()
      /go/tidb/executor/executor_test.go:9507 +0x589
  runtime.call16()
      /usr/local/go/src/runtime/asm_amd64.s:625 +0x48
  reflect.Value.Call()
      /usr/local/go/src/reflect/value.go:339 +0xd7
  github.com/pingcap/check.(*suiteRunner).forkTest.func1()
      /home/prow/go/pkg/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:850 +0xa54
  github.com/pingcap/check.(*suiteRunner).forkCall.func1()
      /home/prow/go/pkg/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:739 +0x12b
Previous read at 0x00c0754fc898 by goroutine 796:
  github.com/pingcap/tidb/executor.(*indexHashJoinInnerWorker).buildHashTableForOuterResult()
      /go/tidb/executor/index_lookup_hash_join.go:577 +0x844
  github.com/pingcap/tidb/executor.(*indexHashJoinInnerWorker).handleTask.func3()
      /go/tidb/executor/index_lookup_hash_join.go:623 +0x7a
  github.com/pingcap/tidb/util.WithRecovery()
      /go/tidb/util/misc.go:100 +0x77
  github.com/pingcap/tidb/executor.(*indexHashJoinInnerWorker).handleTask·dwrap·52()
      /go/tidb/executor/index_lookup_hash_join.go:623 +0x47
Goroutine 83 (running) created at:
  github.com/pingcap/check.(*suiteRunner).forkCall()
      /home/prow/go/pkg/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:734 +0x5e4
  github.com/pingcap/check.(*suiteRunner).forkTest()
      /home/prow/go/pkg/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:832 +0x164
  github.com/pingcap/check.(*suiteRunner).doRun()
      /home/prow/go/pkg/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:666 +0x1fa
  github.com/pingcap/check.(*suiteRunner).run()
      /home/prow/go/pkg/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:696 +0xe5
  github.com/pingcap/check.Run()
      /home/prow/go/pkg/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/run.go:150 +0x49
  github.com/pingcap/check.RunAll()
      /home/prow/go/pkg/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/run.go:136 +0x1184
  github.com/pingcap/check.TestingT()
      /home/prow/go/pkg/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/run.go:99 +0x664
  github.com/pingcap/tidb/executor_test.TestT()
      /go/tidb/executor/executor_test.go:101 +0x69
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:1259 +0x22f
  testing.(*T).Run·dwrap·21()
      /usr/local/go/src/testing/testing.go:1306 +0x47
Goroutine 796 (finished) created at:
  github.com/pingcap/tidb/executor.(*indexHashJoinInnerWorker).handleTask()
      /go/tidb/executor/index_lookup_hash_join.go:623 +0x4f8
  github.com/pingcap/tidb/executor.(*indexHashJoinInnerWorker).run()
      /go/tidb/executor/index_lookup_hash_join.go:504 +0x396
  github.com/pingcap/tidb/executor.(*IndexNestedLoopHashJoin).startWorkers.func2()
      /go/tidb/executor/index_lookup_hash_join.go:198 +0x84
  github.com/pingcap/tidb/util.WithRecovery()
      /go/tidb/util/misc.go:100 +0x77
  github.com/pingcap/tidb/executor.(*IndexNestedLoopHashJoin).startWorkers·dwrap·47()
      /go/tidb/executor/index_lookup_hash_join.go:198 +0x47

2. What did you expect to see? (Required)

3. What did you see instead (Required)

4. What is your TiDB version? (Required)

master
@hawkingrei hawkingrei added the type/bug The issue is confirmed as a bug. label Dec 27, 2021
@tiancaiamao
Copy link
Contributor

What's the version? @hawkingrei

@tiancaiamao
Copy link
Contributor

This looks like when compiling the next SQL, the previous query doesn't fully exit.
Then indexHashJoinInnerWorker background goroutine is still actively building the executor and conflict with the next SQL compilation.
PTAL @XuHuaiyu

@hawkingrei
Copy link
Member Author

What's the version? @hawkingrei

master. It is found in CI.

@yudongusa
Copy link

@XuHuaiyu assign this to you first.

@XuHuaiyu XuHuaiyu mentioned this issue Jan 5, 2022
Merged
ti-chi-bot pushed a commit that referenced this issue Jan 6, 2022
@XuHuaiyu
executor: make sure buildHashTable worker exits before IndexHashJoin …
40c0567 
…finish (#31334)

close #31062
@github-actions
Copy link

github-actions bot commented Jan 6, 2022

Please check whether the issue should be labeled with 'affects-x.y' or 'fixes-x.y.z', and then remove 'needs-more-info' label.

This was referenced Jan 6, 2022
Closed
Merged
Merged
Merged
Merged
ti-chi-bot pushed a commit that referenced this issue Jan 7, 2022
@ti-srebot
executor: make sure buildHashTable worker exits before IndexHashJoin …
9fb7634 
…finish (#31334) (#31420)

close #31062
ti-chi-bot pushed a commit that referenced this issue Feb 9, 2022
@ti-srebot
executor: make sure buildHashTable worker exits before IndexHashJoin …
269c040 
…finish (#31334) (#31417)

close #31062
ti-chi-bot pushed a commit that referenced this issue Feb 22, 2022
@ti-srebot
executor: make sure buildHashTable worker exits before IndexHashJoin …
95ad27d 
…finish (#31334) (#31419)

close #31062
@winoros winoros added the affects-5.2 This bug affects 5.2.x versions. label Apr 12, 2022
ti-chi-bot pushed a commit that referenced this issue Apr 12, 2022
@ti-srebot
executor: make sure buildHashTable worker exits before IndexHashJoin …
6ce9030 
…finish (#31334) (#31418)

close #31062
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@XuHuaiyu XuHuaiyu

Labels
affects-5.2 This bug affects 5.2.x versions. severity/major sig/execution SIG execution type/bug The issue is confirmed as a bug.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

executor: make sure buildHashTable worker exits before IndexHashJoin finish XuHuaiyu/tidb
6 participants
@tiancaiamao @hawkingrei @winoros @XuHuaiyu @aytrack @yudongusa