*: add memory tracker for InsertExec and ReplaceExec #14179

XuHuaiyu · 2019-12-23T06:33:08Z

What problem does this PR solve?

Before this commit, the memory usage of InsertExec and ReplaceExec is not traced.
For example,
we set the MemQuotaQuery in config.go to 32bytes.

We run the following SQLs, and check the tidb.log

tidb> show create table t;
+-------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table | Create Table                                                                                                                                                               |
+-------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| t     | CREATE TABLE `t` (
  `a` bigint(20) DEFAULT NULL,
  `b` bigint(20) DEFAULT NULL,
  `c` bigint(20) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin |
+-------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

tidb> insert into t values(1,1,1);
Query OK, 1 row affected (0.00 sec)

tidb> insert into t select * from t;
Query OK, 1 row affected (0.00 sec)
Records: 1  Duplicates: 0  Warnings: 0

From the tidb.log, we can ONLY got the out of memory message of the second sql, because that the second sql will read data from a TableReader, and the memory usage of the TableReader is already traced

[2019/12/23 17:09:57.907 +08:00] [WARN] [expensivequery.go:167] [expensive_query] [cost_time=0.000231972s] [stats=t:pseudo] [conn_id=1] [user=root] [database=test] [table_ids="[50]"] [txn_start_ts=413425257128132608] [mem_max="163 Bytes (163 Bytes)"] [sql="insert into t select * from t"]

After this commit, we can get the out-of-memory message of the 2 SQLs:

[2019/12/23 17:18:39.380 +08:00] [WARN] [expensivequery.go:167] [expensive_query] [cost_time=0.000015217s] [conn_id=1] [user=root] [database=test] [txn_start_ts=0] [mem_max="168 Bytes (168 Bytes)"] [sql="insert into t values(1,1,1)"]
[2019/12/23 17:18:45.029 +08:00] [WARN] [expensivequery.go:167] [expensive_query] [cost_time=0.000194098s] [stats=t:pseudo] [conn_id=1] [user=root] [database=test] [table_ids="[52]"] [txn_start_ts=413425395310002176] [mem_max="163 Bytes (163 Bytes)"] [sql="insert into t select * from t"]

What is changed and how it works?

Check List

Tests

Integration test

Code changes

Has exported function/method change
Has exported variable/fields change
Has interface methods change

Side effects

N/A

Related changes

N/A

Release note

N/A

…rite-memory

XuHuaiyu · 2019-12-24T03:16:07Z

/run-unit-test

XuHuaiyu · 2019-12-25T08:07:32Z

/re-run

executor/insert_common.go

lysu · 2019-12-25T12:36:04Z

session/txn.go

@@ -455,6 +461,9 @@ func (s *session) StmtCommit() error {
 		st.doNotCommit = err
 		return err
 	}
+	if memTracker != nil {
+		memTracker.Consume(int64(st.Transaction.Size()))


for situation like:

begin; insert1 insert2 commit;

insert1 will add 1k to transaction buffer, insert2 will add 1k to transaction buffer.

so when insert2 step in here will +3k at here..

it's hard to track transaction memory in stmt level if one transaction have multiple statements. - -

lysu · 2019-12-25T12:40:56Z

session/txn.go

@@ -431,7 +437,7 @@ func (s *session) getTxnFuture(ctx context.Context) *txnFuture {
 }

 // StmtCommit implements the sessionctx.Context interface.
-func (s *session) StmtCommit() error {
+func (s *session) StmtCommit(memTracker *memory.Tracker) error {
 	defer s.txn.cleanup()


if we follow code logic, this defer maybe need memTracker.Consume(-int64(txn.Size())), before this line we have put all data from stmt buffer into transaction buffer, and this line will cleanup stmt buffer

lysu · 2019-12-25T12:52:26Z

executor/replace.go

@@ -202,6 +207,7 @@ func (e *ReplaceExec) exec(ctx context.Context, newRows [][]types.Datum) error {
 			return err
 		}
 	}
+	memTracker.Consume(int64(txn.Size()))


this also have potential be duplicate add consume amount if user use batch insert(although we are not recommend use batch insert)

exec be executed for each batch, so second batch will see first batch's memory still in stmt buffer

XuHuaiyu · 2019-12-26T07:03:08Z

/run-all-tests

XuHuaiyu · 2019-12-26T07:06:38Z

/run-all-tests

jackysp · 2019-12-26T07:14:30Z

session/txn.go

@@ -431,9 +437,13 @@ func (s *session) getTxnFuture(ctx context.Context) *txnFuture {
 }

 // StmtCommit implements the sessionctx.Context interface.
-func (s *session) StmtCommit() error {
-	defer s.txn.cleanup()
+func (s *session) StmtCommit(memTracker *memory.Tracker) error {


The statement is finished and the memory is allocated when StmtCommit, I think it is meanless. Consume it at

tidb/session/txn.go

Line 260 in d666ed4

return st.buf.Set(k, v)

.

~~But stmtBuffer can be GCed after be written to txnBuffer?~~

XuHuaiyu · 2019-12-26T07:22:36Z

/run-unit-test

…into trace-write-memory

XuHuaiyu · 2019-12-26T09:02:41Z

/run-unit-test

executor/insert.go

XuHuaiyu · 2019-12-27T07:53:39Z

/run-unit-test

executor/insert_common.go

tiancaiamao · 2019-12-27T13:08:11Z

executor/insert_common.go

@@ -220,6 +232,8 @@ func insertRows(ctx context.Context, base insertCommon) (err error) {
 		}
 		rows = append(rows, row)
 		if batchInsert && e.rowCount%uint64(batchSize) == 0 {
+			memUsageOfRows = types.EstimatedMemUsage(rows[0], len(rows))
+			memTracker.Consume(memUsageOfRows)


For LoadDataExec, here is nil pointer??

LoadDataExec will not touch here.

…rite-memory

XuHuaiyu · 2019-12-30T03:28:20Z

/run-all-tests

tiancaiamao · 2019-12-30T03:37:05Z

LGTM

lysu

LGTM

XuHuaiyu added sig/execution SIG execution type/enhancement The issue or PR belongs to an enhancement. status/WIP labels Dec 23, 2019

XuHuaiyu added 4 commits December 23, 2019 17:17

*: add memory tracker for InsertExec and ReplaceExec

e4bffdb

test

99a6574

fix ci

c6f9e29

fix ci

1bb2640

XuHuaiyu force-pushed the trace-write-memory branch from fb6ab45 to 1bb2640 Compare December 24, 2019 03:07

XuHuaiyu added 3 commits December 24, 2019 11:08

Merge branch 'master' of https://github.com/pingcap/tidb into trace-w…

a9f2e5b

…rite-memory

remove useless code

ddfbbe1

refine

04d4dad

remove useless code

554f22c

XuHuaiyu requested a review from jackysp December 24, 2019 08:21

lysu self-requested a review December 24, 2019 08:24

add some tests

d430227

XuHuaiyu removed the status/WIP label Dec 24, 2019

XuHuaiyu marked this pull request as ready for review December 24, 2019 09:02

XuHuaiyu requested a review from a team as a code owner December 24, 2019 09:02

ghost requested review from francis0407 and alivxxx and removed request for a team December 24, 2019 09:02

XuHuaiyu added 4 commits December 25, 2019 11:19

refine code

1f6ed6e

remove useless code

13edb1f

add test

b55a29e

fix ci

dbc820c

XuHuaiyu added 2 commits December 25, 2019 16:46

test

0b61a15

fix ci

4797740

lysu reviewed Dec 25, 2019

View reviewed changes

XuHuaiyu force-pushed the trace-write-memory branch from 927ab7c to ab3258d Compare December 26, 2019 05:59

fix ci

46e698d

XuHuaiyu force-pushed the trace-write-memory branch from ab3258d to 46e698d Compare December 26, 2019 06:55

Merge branch 'master' into trace-write-memory

80926b3

jackysp reviewed Dec 26, 2019

View reviewed changes

XuHuaiyu added 2 commits December 26, 2019 16:03

address comment

3fb96e9

Merge branch 'trace-write-memory' of https://github.com/XuHuaiyu/tidb …

d69dd2c

…into trace-write-memory

tiancaiamao reviewed Dec 27, 2019

View reviewed changes

executor/insert.go Outdated Show resolved Hide resolved

XuHuaiyu added 4 commits December 27, 2019 11:58

address comment

6419d09

address comment

1df692f

address comment

7b9ed07

fix ci

438ee6c

tiancaiamao reviewed Dec 27, 2019

View reviewed changes

XuHuaiyu added 4 commits December 30, 2019 11:15

address comment

0fd2084

address comment

0b63c28

Merge branch 'master' of https://github.com/pingcap/tidb into trace-w…

4859fa3

…rite-memory

remove useless code

7625390

tiancaiamao added the status/LGT1 Indicates that a PR has LGTM 1. label Dec 30, 2019

lysu approved these changes Dec 30, 2019

View reviewed changes

lysu added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Dec 30, 2019

XuHuaiyu merged commit c1bc9ff into pingcap:master Dec 30, 2019

XuHuaiyu deleted the trace-write-memory branch December 30, 2019 03:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

*: add memory tracker for InsertExec and ReplaceExec #14179

*: add memory tracker for InsertExec and ReplaceExec #14179

XuHuaiyu commented Dec 23, 2019 •

edited

Loading

XuHuaiyu commented Dec 24, 2019

XuHuaiyu commented Dec 25, 2019

lysu Dec 25, 2019

lysu Dec 25, 2019

lysu Dec 25, 2019 •

edited

Loading

XuHuaiyu commented Dec 26, 2019

XuHuaiyu commented Dec 26, 2019

jackysp Dec 26, 2019 •

edited

Loading

XuHuaiyu Dec 26, 2019 •

edited

Loading

XuHuaiyu commented Dec 26, 2019

XuHuaiyu commented Dec 26, 2019

XuHuaiyu commented Dec 27, 2019

tiancaiamao Dec 27, 2019

XuHuaiyu Dec 30, 2019

XuHuaiyu commented Dec 30, 2019

tiancaiamao commented Dec 30, 2019

lysu left a comment

*: add memory tracker for InsertExec and ReplaceExec #14179

*: add memory tracker for InsertExec and ReplaceExec #14179

Conversation

XuHuaiyu commented Dec 23, 2019 • edited Loading

What problem does this PR solve?

What is changed and how it works?

Check List

XuHuaiyu commented Dec 24, 2019

XuHuaiyu commented Dec 25, 2019

lysu Dec 25, 2019

Choose a reason for hiding this comment

lysu Dec 25, 2019

Choose a reason for hiding this comment

lysu Dec 25, 2019 • edited Loading

Choose a reason for hiding this comment

XuHuaiyu commented Dec 26, 2019

XuHuaiyu commented Dec 26, 2019

jackysp Dec 26, 2019 • edited Loading

Choose a reason for hiding this comment

XuHuaiyu Dec 26, 2019 • edited Loading

Choose a reason for hiding this comment

XuHuaiyu commented Dec 26, 2019

XuHuaiyu commented Dec 26, 2019

XuHuaiyu commented Dec 27, 2019

tiancaiamao Dec 27, 2019

Choose a reason for hiding this comment

XuHuaiyu Dec 30, 2019

Choose a reason for hiding this comment

XuHuaiyu commented Dec 30, 2019

tiancaiamao commented Dec 30, 2019

lysu left a comment

Choose a reason for hiding this comment

XuHuaiyu commented Dec 23, 2019 •

edited

Loading

lysu Dec 25, 2019 •

edited

Loading

jackysp Dec 26, 2019 •

edited

Loading

XuHuaiyu Dec 26, 2019 •

edited

Loading