Optimize locatePartition() function and improve the write performance on partitioned table

[tiancaiamao]

Development Task

Here is a flamegraph captured during load data into a partitioned table.

As we can see, the write operation is heavy and the locatePartition function is not optimized.

[tiancaiamao]

(pprof) top30
Showing nodes accounting for 39.80s, 72.10% of 55.20s total
Dropped 730 nodes (cum <= 0.28s)
Showing top 30 nodes out of 203
      flat  flat%   sum%        cum   cum%
    11.89s 21.54% 21.54%     21.83s 39.55%  runtime.scanobject
     4.51s  8.17% 29.71%      5.02s  9.09%  runtime.findObject
        3s  5.43% 35.14%         3s  5.43%  runtime.memclrNoHeapPointers
     2.33s  4.22% 39.37%     13.82s 25.04%  runtime.mallocgc
     1.94s  3.51% 42.88%      2.24s  4.06%  runtime.heapBitsSetType
     1.41s  2.55% 45.43%      1.47s  2.66%  runtime.pageIndexOf
     1.29s  2.34% 47.77%      1.29s  2.34%  runtime.futex
     1.09s  1.97% 49.75%      1.09s  1.97%  runtime.nextFreeFast
     0.99s  1.79% 51.54%      0.99s  1.79%  runtime.duffcopy
     0.98s  1.78% 53.32%      0.98s  1.78%  runtime.markBits.isMarked
     0.95s  1.72% 55.04%     21.69s 39.29%  runtime.gcDrain
     0.86s  1.56% 56.59%      3.66s  6.63%  runtime.greyobject
     0.85s  1.54% 58.13%      0.85s  1.54%  cmpbody
     0.79s  1.43% 59.57%      0.79s  1.43%  runtime.memmove
     0.73s  1.32% 60.89%      2.10s  3.80%  runtime.sweepone
     0.69s  1.25% 62.14%      0.69s  1.25%  runtime.heapBits.bits
     0.67s  1.21% 63.35%      0.75s  1.36%  syscall.Syscall
     0.63s  1.14% 64.49%      0.63s  1.14%  github.com/pingcap/tidb/kv/memdb.(*node).nodeLen
     0.63s  1.14% 65.63%      0.63s  1.14%  runtime.madvise
     0.40s  0.72% 66.36%      7.44s 13.48%  runtime.makeslice
     0.39s  0.71% 67.07%      0.44s   0.8%  runtime.heapBits.next
     0.38s  0.69% 67.75%      0.39s  0.71%  runtime.markrootSpans
     0.38s  0.69% 68.44%      0.43s  0.78%  runtime.spanOf
     0.32s  0.58% 69.02%      0.37s  0.67%  runtime.unlock
     0.31s  0.56% 69.58%      0.37s  0.67%  runtime.(*treapNode).updateInvariants
     0.30s  0.54% 70.13%      0.40s  0.72%  runtime.heapBitsForAddr
     0.28s  0.51% 70.63%      0.60s  1.09%  github.com/pingcap/tidb/table.CastValue
     0.28s  0.51% 71.14%      8.61s 15.60%  github.com/pingcap/tidb/util/chunk.MutRowFromDatums
     0.28s  0.51% 71.65%      0.28s  0.51%  runtime.arenaIndex
     0.25s  0.45% 72.10%      0.38s  0.69%  runtime.lock

(pprof) top -cum 30
Showing nodes accounting for 21.20s, 38.41% of 55.20s total
Dropped 730 nodes (cum <= 0.28s)
Showing top 30 nodes out of 203
      flat  flat%   sum%        cum   cum%
     0.01s 0.018% 0.018%     28.83s 52.23%  runtime.systemstack
         0     0% 0.018%     22.59s 40.92%  github.com/pingcap/tidb/executor.(*LoadDataInfo).CommitOneTask
         0     0% 0.018%     22.59s 40.92%  github.com/pingcap/tidb/executor.(*LoadDataInfo).CommitWork
         0     0% 0.018%     22.59s 40.92%  github.com/pingcap/tidb/server.(*Server).onConn
         0     0% 0.018%     22.59s 40.92%  github.com/pingcap/tidb/server.(*clientConn).Run
         0     0% 0.018%     22.59s 40.92%  github.com/pingcap/tidb/server.(*clientConn).dispatch
         0     0% 0.018%     22.59s 40.92%  github.com/pingcap/tidb/server.(*clientConn).handleLoadData
         0     0% 0.018%     22.59s 40.92%  github.com/pingcap/tidb/server.(*clientConn).handleQuery
     0.03s 0.054% 0.072%     21.83s 39.55%  runtime.gcBgMarkWorker
    11.89s 21.54% 21.61%     21.83s 39.55%  runtime.scanobject
         0     0% 21.61%     21.70s 39.31%  runtime.gcBgMarkWorker.func2
     0.95s  1.72% 23.33%     21.69s 39.29%  runtime.gcDrain
     0.08s  0.14% 23.48%     17.89s 32.41%  github.com/pingcap/tidb/executor.(*InsertValues).batchCheckAndInsert
         0     0% 23.48%     17.89s 32.41%  github.com/pingcap/tidb/executor.(*LoadDataInfo).CheckAndInsertOneBatch
     2.33s  4.22% 27.70%     13.82s 25.04%  runtime.mallocgc
     0.02s 0.036% 27.74%      9.78s 17.72%  sort.Search
     0.03s 0.054% 27.79%      9.67s 17.52%  github.com/pingcap/tidb/table/tables.(*partitionedTable).locatePartition
     0.02s 0.036% 27.83%      9.64s 17.46%  github.com/pingcap/tidb/table/tables.(*partitionedTable).locateRangePartition
     0.11s   0.2% 28.03%      9.62s 17.43%  github.com/pingcap/tidb/table/tables.(*partitionedTable).locateRangePartition.func1
     0.01s 0.018% 28.04%      9.19s 16.65%  github.com/pingcap/tidb/executor.(*LoadDataInfo).addRecordLD
     0.01s 0.018% 28.06%      9.18s 16.63%  github.com/pingcap/tidb/executor.(*InsertValues).addRecord
     0.02s 0.036% 28.10%      9.17s 16.61%  github.com/pingcap/tidb/executor.(*InsertValues).addRecordWithAutoIDHint
     0.01s 0.018% 28.12%      8.97s 16.25%  github.com/pingcap/tidb/table/tables.(*partitionedTable).AddRecord
     0.28s  0.51% 28.62%      8.61s 15.60%  github.com/pingcap/tidb/util/chunk.MutRowFromDatums
     0.40s  0.72% 29.35%      7.44s 13.48%  runtime.makeslice
         0     0% 29.35%      6.56s 11.88%  github.com/pingcap/tidb/executor.getKeysNeedCheck
     0.21s  0.38% 29.73%      6.55s 11.87%  github.com/pingcap/tidb/executor.getKeysNeedCheckOneRow
     0.11s   0.2% 29.93%      5.58s 10.11%  github.com/pingcap/tidb/util/chunk.makeMutRowColumn
     0.17s  0.31% 30.24%      5.39s  9.76%  runtime.newobject
     4.51s  8.17% 38.41%      5.02s  9.09%  runtime.findObject

[tiancaiamao]

Expression accepts chunk.Row, but AddRecord provides []Datum.
To avoid the []Datum to chunk.Row conversion, a heavy refactoring is required.

We need to refactor the all write path code to use chunk.Chunk, this takes a lot of work.