benchmark: add lightning import for TPC-C #2169

jackysp · 2020-01-08T03:57:15Z

Signed-off-by: Shuaipeng Yu jackysp@gmail.com

What is changed, added or deleted?

Add lightning import for TPC-C.

What is the related PR or file link(s)?

Which version does your change affect?

dev, 3.0, 3.1

Signed-off-by: Shuaipeng Yu <jackysp@gmail.com>

jackysp · 2020-01-08T03:57:52Z

I'll update 3.0 and 3.1 when it is ready to merge.

kennytm

Rest LGTM

dev/benchmark/how-to-run-tpcc.md

kennytm · 2020-01-08T05:08:32Z

dev/benchmark/how-to-run-tpcc.md

+* 在 Importer 目录下创建一个 log 文件夹
+* 在 Importer 目录下执行 `scripts/start_importer.sh`，启动 Importer
+* 在 Lightning 目录下创建一个 log 文件夹
+* 在 Lightning 目录下执行 `scripts/start_lightning.sh`，开始导入数据


The log folders should be created automatically by the shell scripts 🤔

It doesn't.

\cc @liubo0127

All directorys should be created automatically. I'll check it.

Fixed in pingcap/tidb-ansible#1103

Co-Authored-By: kennytm <kennytm@gmail.com>

jackysp · 2020-01-08T05:15:15Z

dev/benchmark/how-to-run-tpcc.md

+
+```yaml
+tikv_importer:
+    addr: "172.16.5.34:13323" # 这里指向 importer 的地址，需要将上面 inventory.ini 里指定的再写一次


Could we improve this? I mean it doesn't need to set IP:port once again. @kennytm @liubo0127

It should be configured automatically.

I have tested, and it can be configured automatically.

port does not set automatically.

Ok, it's actually set automatically, but it doesn't match the custom port.

yikeke

Some small editing advice on improving the language. Rest LGTM.

dev/benchmark/how-to-run-tpcc.md

Co-Authored-By: Keke Yi <40977455+yikeke@users.noreply.github.com>

yikeke

Language LGTM

Signed-off-by: Shuaipeng Yu <jackysp@gmail.com>

jackysp · 2020-01-10T05:41:40Z

All comments addressed, and apply to v3.0 and v3.1, PTAL @yikeke

yikeke

LGTM

claassistantio · 2020-01-10T05:56:34Z

All committers have signed the CLA.

sre-bot · 2020-01-10T05:57:45Z

/run-all-tests

jackysp · 2020-01-10T06:05:32Z

... the bot blocks CLA.

yikeke · 2020-01-10T06:07:24Z

... the bot blocks CLA.

I asked the bot's manager @you06 to sign it. Wait a moment.

you06 · 2020-01-10T07:22:39Z

Facing a bot-login issue, we're trying to solve this.

july2993 · 2020-03-21T10:31:44Z

dev/benchmark/how-to-run-tpcc.md

+LS1 ansible_host=172.16.5.34 deploy_dir=/data2/ls1 tidb_lightning_pprof_port=23323 data_source_dir=/home/user/csv
+```
+
+##### 修改 conf/tidb-lightning.yml


not toml? @kennytm

YAML is correct, Ansible uses YAML as template. 😓 @july2993

I methinks this is the config file lightning use directly 😓

jackysp · 2020-03-24T12:05:05Z

The issue found in this PR is fixed by pingcap/tidb-ansible#1105, pingcap/tidb-ansible#1104 and pingcap/tidb-ansible#1103

add lightning import

da16cda

Signed-off-by: Shuaipeng Yu <jackysp@gmail.com>

jackysp requested review from yikeke, IANTHEREAL and kennytm January 8, 2020 04:01

kennytm reviewed Jan 8, 2020

View reviewed changes

jackysp and others added 2 commits January 8, 2020 13:12

Update dev/benchmark/how-to-run-tpcc.md

3163e68

Co-Authored-By: kennytm <kennytm@gmail.com>

Apply suggestions from code review

3de3525

Co-Authored-By: kennytm <kennytm@gmail.com>

jackysp commented Jan 8, 2020

View reviewed changes

lilin90 added size/large Changes of a large size. translation/doing This PR’s assignee is translating this PR. labels Jan 8, 2020

lilin90 assigned yikeke Jan 8, 2020

yikeke reviewed Jan 9, 2020

View reviewed changes

Apply suggestions from code review

11391c9

Co-Authored-By: Keke Yi <40977455+yikeke@users.noreply.github.com>

yikeke reviewed Jan 9, 2020

View reviewed changes

jackysp added 2 commits January 10, 2020 13:32

remove fix steps

a47d1da

Signed-off-by: Shuaipeng Yu <jackysp@gmail.com>

apply to v3.0 and v3.1

06c6a4a

Signed-off-by: Shuaipeng Yu <jackysp@gmail.com>

yikeke approved these changes Jan 10, 2020

View reviewed changes

yikeke added the status/can-merge Indicates a PR has been approved by a committer. label Jan 10, 2020

Merge branch 'master' into tpcc_lightning

fd24a5e

sre-bot merged commit 0b052a7 into pingcap:master Jan 10, 2020

yikeke assigned toutdesuite and unassigned yikeke Mar 4, 2020

yikeke added priority/P1 The issue has P1 priority. and removed priority/P1 The issue has P1 priority. labels Mar 4, 2020

jackysp deleted the tpcc_lightning branch March 18, 2020 06:16

toutdesuite mentioned this pull request Mar 19, 2020

benchmark: add lightning import for TPC-C pingcap/docs#2057

Merged

4 tasks

toutdesuite added translation/done This PR has been translated from English into Chinese and updated to pingcap/docs-cn in a PR. and removed translation/doing This PR’s assignee is translating this PR. labels Mar 19, 2020

july2993 reviewed Mar 21, 2020

View reviewed changes

This was referenced Mar 24, 2020

benchmark: add lightning import for TPC-C (#2057) pingcap/docs#2084

Merged

benchmark: add lightning import for TPC-C (#2057) pingcap/docs#2085

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

benchmark: add lightning import for TPC-C #2169

benchmark: add lightning import for TPC-C #2169

jackysp commented Jan 8, 2020

jackysp commented Jan 8, 2020

kennytm left a comment

kennytm Jan 8, 2020 •

edited

Loading

jackysp Jan 8, 2020

jackysp Jan 8, 2020

liubo0127 Jan 9, 2020

liubo0127 Jan 9, 2020

jackysp Jan 10, 2020

jackysp Jan 8, 2020

This comment was marked as off-topic.

liubo0127 Jan 9, 2020

liubo0127 Jan 9, 2020

jackysp Jan 10, 2020

jackysp Jan 10, 2020

liubo0127 Jan 10, 2020

liubo0127 Jan 10, 2020

jackysp Jan 10, 2020

yikeke left a comment

yikeke left a comment

jackysp commented Jan 10, 2020

yikeke left a comment

claassistantio commented Jan 10, 2020 •

edited

Loading

sre-bot commented Jan 10, 2020

jackysp commented Jan 10, 2020

yikeke commented Jan 10, 2020 •

edited

Loading

you06 commented Jan 10, 2020

july2993 Mar 21, 2020

kennytm Mar 22, 2020

july2993 Mar 23, 2020

jackysp commented Mar 24, 2020

benchmark: add lightning import for TPC-C #2169

benchmark: add lightning import for TPC-C #2169

Conversation

jackysp commented Jan 8, 2020

What is changed, added or deleted?

What is the related PR or file link(s)?

Which version does your change affect?

jackysp commented Jan 8, 2020

kennytm left a comment

Choose a reason for hiding this comment

kennytm Jan 8, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

This comment was marked as off-topic.

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yikeke left a comment

Choose a reason for hiding this comment

yikeke left a comment

Choose a reason for hiding this comment

jackysp commented Jan 10, 2020

yikeke left a comment

Choose a reason for hiding this comment

claassistantio commented Jan 10, 2020 • edited Loading

sre-bot commented Jan 10, 2020

jackysp commented Jan 10, 2020

yikeke commented Jan 10, 2020 • edited Loading

you06 commented Jan 10, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jackysp commented Mar 24, 2020

kennytm Jan 8, 2020 •

edited

Loading

claassistantio commented Jan 10, 2020 •

edited

Loading

yikeke commented Jan 10, 2020 •

edited

Loading