executor: reorg codes for hashtable in HashJoinExec #11937

SunRunAway · 2019-08-29T13:28:39Z

What problem does this PR solve?

Part of #11607
It reorgs codes for the hash table and introduces hashRowContainer. So that In further pull request, I can implement spillOutToDisk feature for hashRowContainer

What is changed and how it works?

reorg codes for hash table in HashJoinExec
make a suitable maxEntrySliceLen, it's currently too large(8k) for OLTP queries.
add unit test for rowHashMap.
try to replace FNV64 to another hash function. (abandon, FNV64 is the fastest.)
initialize the size of rowHashMap from the estimated row count for better performance.

Check List

Tests

Unit test

Code changes

None

Side effects

None

Related changes

None

Release note

None

Benchmark

name                                                                     old time/op    new time/op    delta
HashJoinExec/(rows:100000,_concurency:4,_joinKeyIdx:_[0_1])-12              760ms ± 6%     781ms ± 7%     ~     (p=0.421 n=5+5)
HashJoinExec/(rows:100000,_concurency:4,_joinKeyIdx:_[0])-12                168ms ± 5%     166ms ± 2%     ~     (p=0.841 n=5+5)
BuildHashTableForList/(rows:100000,_concurency:4,_joinKeyIdx:_[0_1])-12     512ms ± 0%     544ms ± 1%   +6.19%  (p=0.016 n=4+5)
BuildHashTableForList/(rows:100000,_concurency:4,_joinKeyIdx:_[0])-12      12.2ms ± 2%    12.5ms ± 8%     ~     (p=0.548 n=5+5)
BuildHashTableForList/(rows:10,_concurency:4,_joinKeyIdx:_[0])-12          2.46µs ± 2%    2.94µs ± 0%  +19.46%  (p=0.008 n=5+5)

name                                                                     old alloc/op   new alloc/op   delta
HashJoinExec/(rows:100000,_concurency:4,_joinKeyIdx:_[0_1])-12              305MB ± 0%     304MB ± 0%   -0.28%  (p=0.008 n=5+5)
HashJoinExec/(rows:100000,_concurency:4,_joinKeyIdx:_[0])-12                305MB ± 0%     304MB ± 0%   -0.28%  (p=0.008 n=5+5)
BuildHashTableForList/(rows:100000,_concurency:4,_joinKeyIdx:_[0_1])-12    7.97MB ± 0%    7.11MB ± 0%  -10.80%  (p=0.008 n=5+5)
BuildHashTableForList/(rows:100000,_concurency:4,_joinKeyIdx:_[0])-12      7.97MB ± 0%    7.11MB ± 0%  -10.80%  (p=0.008 n=5+5)
BuildHashTableForList/(rows:10,_concurency:4,_joinKeyIdx:_[0])-12          1.62kB ± 0%    1.96kB ± 0%  +20.74%  (p=0.008 n=5+5)

name                                                                     old allocs/op  new allocs/op  delta
HashJoinExec/(rows:100000,_concurency:4,_joinKeyIdx:_[0_1])-12               306k ± 0%      306k ± 0%   -0.08%  (p=0.008 n=5+5)
HashJoinExec/(rows:100000,_concurency:4,_joinKeyIdx:_[0])-12                 306k ± 0%      306k ± 0%   -0.09%  (p=0.008 n=5+5)
BuildHashTableForList/(rows:100000,_concurency:4,_joinKeyIdx:_[0_1])-12     4.07k ± 2%     3.80k ± 1%   -6.56%  (p=0.008 n=5+5)
BuildHashTableForList/(rows:100000,_concurency:4,_joinKeyIdx:_[0])-12       4.04k ± 0%     3.79k ± 0%   -6.41%  (p=0.008 n=5+5)
BuildHashTableForList/(rows:10,_concurency:4,_joinKeyIdx:_[0])-12            9.00 ± 0%     14.00 ± 0%  +55.56%  (p=0.008 n=5+5)

old.txt
new.txt

SunRunAway · 2019-08-29T13:30:12Z

/run-all-tests

codecov · 2019-08-29T14:01:14Z

Codecov Report

Merging #11937 into master will not change coverage.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             master     #11937   +/-   ##
===========================================
  Coverage   81.4498%   81.4498%           
===========================================
  Files           444        444           
  Lines         95622      95622           
===========================================
  Hits          77884      77884           
  Misses        12238      12238           
  Partials       5500       5500

SunRunAway · 2019-08-29T14:01:24Z

/run-all-tests

…TP queries.

qw4990 · 2019-09-02T08:48:46Z

executor/hash_table.go

 type rowHashMap struct {
 	entryStore entryStore
 	hashTable  map[uint64]entryAddr
 	length     int
 }

 // newRowHashMap creates a new rowHashMap.
-func newRowHashMap() *rowHashMap {
+func newRowHashMapWithStatCount(statCount int) *rowHashMap {


statCount -> initCap?

XuHuaiyu

The current rowContainer suppose that we always use the inner table to build hash table and use outer row to probe.
Should we also handle the case that using the outer table to build the hash table?

XuHuaiyu · 2019-09-02T08:59:44Z

executor/join.go

-
-	h := fnv.New64()
-	chkIdx := uint32(0)
+	statCount := int(e.innerStatsCount / statCountDivisor)


Can we put line 518~524 into newHashRowContainer?

XuHuaiyu · 2019-09-02T09:10:28Z

executor/hash_table.go

+		if err != nil {
+			return errors.Trace(err)
+		}
+		if hasNull {


Should we record the rowIdx of which hasNull is true?
If we use the outer table to build hashtable, we need to know the null-value lines.

I'll adjust it in further PR to involve hashRowContainer into index join.

XuHuaiyu · 2019-09-02T09:11:14Z

executor/hash_table.go

+// in multiple goroutines while each goroutine should keep its own
+// h and buf.
+func (c *hashRowContainer) GetMatchedRows(probeRow chunk.Row, joinKeysTypes []*types.FieldType, keyColIdx []int, h hash.Hash64, buf []byte) (matched []chunk.Row, hasNull bool, err error) {
+


useless line

XuHuaiyu · 2019-09-02T09:13:40Z

executor/hash_table.go

+	}
+	innerPtrs := c.hashTable.Get(key)
+	if len(innerPtrs) == 0 {
+		hasNull = true


Why we set hashNull to true?

It means no inner rows matched by the outer row in hashRowContainer so we need call onMissMatch.
The same logic in old code is at https://github.com/pingcap/tidb/blob/304619a/executor/join.go#L417 and https://github.com/pingcap/tidb/blob/304619a/executor/join.go#L434

I've addressed this by removing the ambiguous isNull return value, and use the size of matched rows to determine whether to call onMissMatch.

XuHuaiyu · 2019-09-02T09:13:47Z

executor/hash_table.go

+		matched = append(matched, matchedRow)
+	}
+	if len(matched) == 0 { // TODO(fengliyuan): add test case
+		hasNull = true


SunRunAway · 2019-09-02T10:15:54Z

/run-all-tests

SunRunAway · 2019-09-02T10:32:58Z

All comments are addressed.
PTAL @zz-jason, @qw4990, @XuHuaiyu

zz-jason

LGTM

XuHuaiyu · 2019-09-03T03:29:39Z

executor/hash_table.go

+
+// matchJoinKey checks if join keys of buildRow and probeRow are logically equal.
+func (c *hashRowContainer) matchJoinKey(buildRow, probeRow chunk.Row, probeHCtx *hashContext) (ok bool, err error) {
+


This line is useless.

XuHuaiyu · 2019-09-03T03:33:26Z

executor/hash_table.go

+)
+
+const (
+	// estCountMaxFactor defines the factor of maxStatCount with maxChunkSize.


What does maxStatCount mean?

What does estCountMax mean?

XuHuaiyu · 2019-09-03T03:34:44Z

executor/hash_table.go

+	// Set this threshold to prevent innerEstCount being too large and causing a performance regression.
+	estCountMaxFactor = 10 * 1024
+
+	// estCountMinFactor defines the factor of statCountMin with maxChunkSize.


XuHuaiyu · 2019-09-03T04:20:53Z

executor/hash_table.go

+	// Set this threshold to prevent innerEstCount being too large and causing a performance and memory regression.
+	estCountMaxFactor = 10 * 1024
+
+	// estCountMinFactor defines the factor of statCountMin with maxChunkSize.


s/ statCountMin/ estCountMin ?

XuHuaiyu

LGTM

sre-bot · 2019-09-03T05:27:37Z

/run-all-tests

sre-bot · 2019-09-03T05:33:08Z

/run-all-tests

SunRunAway mentioned this pull request Aug 29, 2019

Consider using a disk-based hash table for hash join avoiding OOM #11607

Open

14 tasks

SunRunAway added sig/execution SIG execution type/new-feature type/enhancement The issue or PR belongs to an enhancement. status/WIP labels Aug 29, 2019

executor: reorg code for hashtable in HashJoinExec

6cf30e5

SunRunAway force-pushed the disk-join-issue11607-reorg branch from a1df811 to 6cf30e5 Compare August 29, 2019 14:01

fix benchmark

f980b13

SunRunAway force-pushed the disk-join-issue11607-reorg branch from 87d6fb3 to f980b13 Compare August 30, 2019 04:03

make a suitable maxEntrySliceLen, it's currently too large(8k) for OL…

8387cb3

…TP queries.

SunRunAway force-pushed the disk-join-issue11607-reorg branch from a57388d to 2b4808e Compare August 30, 2019 09:42

add unit test for rowHashMap.

63a0c3f

SunRunAway force-pushed the disk-join-issue11607-reorg branch from 2b4808e to 63a0c3f Compare August 30, 2019 09:52

SunRunAway added 4 commits August 30, 2019 19:14

initialize the size of rowHashMap from the estimated row count.

a086d7f

licence

f0e04d6

add TODO

0f66252

fix benchmark

8a5087b

SunRunAway requested review from XuHuaiyu, qw4990, zz-jason and wshwsh12 and removed request for XuHuaiyu August 30, 2019 11:45

SunRunAway added status/PTAL and removed status/WIP type/new-feature labels Aug 30, 2019

statCountMinFactor

8def643

qw4990 reviewed Sep 2, 2019

View reviewed changes

XuHuaiyu reviewed Sep 2, 2019

View reviewed changes

address comments

8ae2de4

SunRunAway requested review from zz-jason, qw4990 and XuHuaiyu September 2, 2019 10:33

SunRunAway added 2 commits September 2, 2019 18:47

remove the ambiguous return value hasNull in GetMatchedRows

ef97c5f

add comment for estCount

633143d

zz-jason reviewed Sep 2, 2019

View reviewed changes

zz-jason added the status/LGT1 Indicates that a PR has LGTM 1. label Sep 2, 2019

SunRunAway mentioned this pull request Sep 3, 2019

executor: support index nested loop hash join #8661

Merged

XuHuaiyu reviewed Sep 3, 2019

View reviewed changes

address comments

a3cafe8

XuHuaiyu reviewed Sep 3, 2019

View reviewed changes

SunRunAway added 2 commits September 3, 2019 12:41

fix

b16e09b

fix

a18a423

XuHuaiyu approved these changes Sep 3, 2019

View reviewed changes

XuHuaiyu added the status/can-merge Indicates a PR has been approved by a committer. label Sep 3, 2019

Merge branch 'master' into disk-join-issue11607-reorg

a5e1273

zz-jason added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. status/PTAL labels Sep 3, 2019

Merge branch 'master' into disk-join-issue11607-reorg

10b0a3e

XuHuaiyu merged commit 1ff620d into pingcap:master Sep 3, 2019

SunRunAway deleted the disk-join-issue11607-reorg branch September 3, 2019 05:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

executor: reorg codes for hashtable in HashJoinExec #11937

executor: reorg codes for hashtable in HashJoinExec #11937

SunRunAway commented Aug 29, 2019 •

edited

Loading

SunRunAway commented Aug 29, 2019

codecov bot commented Aug 29, 2019 •

edited

Loading

SunRunAway commented Aug 29, 2019

qw4990 Sep 2, 2019

XuHuaiyu left a comment

XuHuaiyu Sep 2, 2019

XuHuaiyu Sep 2, 2019

SunRunAway Sep 2, 2019 •

edited

Loading

XuHuaiyu Sep 2, 2019

XuHuaiyu Sep 2, 2019

SunRunAway Sep 2, 2019 •

edited

Loading

SunRunAway Sep 2, 2019 •

edited

Loading

XuHuaiyu Sep 2, 2019

SunRunAway commented Sep 2, 2019

SunRunAway commented Sep 2, 2019

zz-jason left a comment

XuHuaiyu Sep 3, 2019

XuHuaiyu Sep 3, 2019

XuHuaiyu Sep 3, 2019

XuHuaiyu Sep 3, 2019

XuHuaiyu left a comment

sre-bot commented Sep 3, 2019

sre-bot commented Sep 3, 2019


		// matchJoinKey checks if join keys of buildRow and probeRow are logically equal.
		func (c hashRowContainer) matchJoinKey(buildRow, probeRow chunk.Row, probeHCtx hashContext) (ok bool, err error) {

executor: reorg codes for hashtable in HashJoinExec #11937

executor: reorg codes for hashtable in HashJoinExec #11937

Conversation

SunRunAway commented Aug 29, 2019 • edited Loading

What problem does this PR solve?

What is changed and how it works?

Check List

Benchmark

SunRunAway commented Aug 29, 2019

codecov bot commented Aug 29, 2019 • edited Loading

Codecov Report

SunRunAway commented Aug 29, 2019

Choose a reason for hiding this comment

XuHuaiyu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SunRunAway Sep 2, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SunRunAway Sep 2, 2019 • edited Loading

Choose a reason for hiding this comment

SunRunAway Sep 2, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SunRunAway commented Sep 2, 2019

SunRunAway commented Sep 2, 2019

zz-jason left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

XuHuaiyu left a comment

Choose a reason for hiding this comment

sre-bot commented Sep 3, 2019

sre-bot commented Sep 3, 2019

SunRunAway commented Aug 29, 2019 •

edited

Loading

codecov bot commented Aug 29, 2019 •

edited

Loading

SunRunAway Sep 2, 2019 •

edited

Loading

SunRunAway Sep 2, 2019 •

edited

Loading

SunRunAway Sep 2, 2019 •

edited

Loading