Skip to content
This repository has been archived by the owner on Nov 24, 2023. It is now read-only.

set suitable default value for process, source and task config items #1130

Closed
csuzhangxc opened this issue Oct 9, 2020 · 2 comments · Fixed by #1156
Closed

set suitable default value for process, source and task config items #1130

csuzhangxc opened this issue Oct 9, 2020 · 2 comments · Fixed by #1156
Assignees
Labels
severity/major type/bug This issue is a bug report

Comments

@csuzhangxc
Copy link
Member

Bug Report

Please answer these questions before submitting your issue. Thanks!

  1. What did you do? If possible, provide a recipe for reproducing the error.

  2. What did you expect to see?

  3. What did you see instead?

we do not set a suitable value for some config items (like statement-size, this may dump too large txn to make TiDB OOM)

  1. Versions of the cluster

    • DM version (run dmctl -V or dm-worker -V or dm-master -V):

      DM v2.0.0-rc.2
      
@csuzhangxc csuzhangxc added type/bug This issue is a bug report severity/major labels Oct 9, 2020
@csuzhangxc
Copy link
Member Author

dumpling has set the default value of statement-size in pingcap/dumpling#160

@lance6716
Copy link
Collaborator

lance6716 commented Oct 10, 2020

some default config which maybe dangerous

use dumpling default value:

  • if mydumper in config is not null but lack some items:
    • FileSize is UnspecifiedSize, but seems one big file could have multiple insert statements, will check if it's fine
    • Rows is UnspecifiedSize, don't split table every Rows into multiple files, seems fine because we didn't set it before
    • SessionParams is empty, maybe we could make use of it in future
    • DumpEmptyDatabase, NoViews

overwrite dumpling to DM specified default:

  • EscapeBackslash is true (what's this)

task config:

  • CleanDumpFile is true

source config:

  • didn't call Verify

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
severity/major type/bug This issue is a bug report
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants