Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

*: record previous statement when commit is slow #11908

Merged
merged 5 commits into from
Sep 3, 2019
Merged

*: record previous statement when commit is slow #11908

merged 5 commits into from
Sep 3, 2019

Conversation

jackysp
Copy link
Member

@jackysp jackysp commented Aug 28, 2019

Signed-off-by: Shuaipeng Yu jackysp@gmail.com

What problem does this PR solve?

If commit is slow, the user is hard to know which statement was executed in the slow log.

What is changed and how it works?

Add the previous statement for commit if commit is slow.

Check List

Tests

  • Unit test

Code changes

  • Has exported function/method change

Related changes

  • Need to cherry-pick to the release branch

Release note

  • Write release note for bug-fix or new feature.
    Record the previous statement in the slow log when commit is slow.

@jackysp
*: record last statement when commit is slow
6d4a45b 
Signed-off-by: Shuaipeng Yu <jackysp@gmail.com>
@jackysp jackysp added type/enhancement The issue or PR belongs to an enhancement. needs-cherry-pick-2.1 labels Aug 28, 2019
@jackysp
Copy link
Member Author

jackysp commented Aug 29, 2019

/run-check_dev_2

imtbkcat
imtbkcat reviewed Aug 29, 2019
View reviewed changes
Copy link

@imtbkcat imtbkcat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@codecov
Copy link

codecov bot commented Aug 29, 2019 •
edited
Loading

Codecov Report

Merging #11908 into master will not change coverage.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             master     #11908   +/-   ##
===========================================
  Coverage   81.5251%   81.5251%           
===========================================
  Files           445        445           
  Lines         96071      96071           
===========================================
  Hits          78322      78322           
  Misses        12231      12231           
  Partials       5518       5518

crazycs520
crazycs520 reviewed Aug 29, 2019
View reviewed changes
infoschema/slow_log.go Outdated Show resolved Hide resolved
crazycs520
crazycs520 reviewed Aug 29, 2019
View reviewed changes
infoschema/slow_log.go Outdated
@@ -87,6 +88,7 @@ func parseSlowLogFile(tz *time.Location, filePath string) ([][]types.Datum, erro
func ParseSlowLog(tz *time.Location, reader *bufio.Reader) ([][]types.Datum, error) {
var rows [][]types.Datum
startFlag := false
prevStmtPrefix := variable.SlowLogPrevStmt + variable.SlowLogSpaceMarkStr
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make this to be a global const .

lysu
lysu reviewed Aug 29, 2019
View reviewed changes
Copy link
Contributor

@lysu lysu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jackysp
address comments
8e9e414 
Signed-off-by: Shuaipeng Yu <jackysp@gmail.com>
zz-jason
zz-jason reviewed Aug 29, 2019
View reviewed changes
executor/adapter.go
@@ -712,38 +719,28 @@ func (a *ExecStmt) LogSlowQuery(txnTS uint64, succ bool) {
copTaskInfo := sessVars.StmtCtx.CopTasksDetails()
statsInfos := plannercore.GetStatsInfo(a.Plan)
memMax := sessVars.StmtCtx.MemTracker.MaxConsumed()
_, digest := sessVars.StmtCtx.SQLDigest()
slowItems := &variable.SlowQueryLogItems{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's better to be put inside the if...else... branch? If the log level is higher than debug, we can save an object allocation?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it will return at https://github.com/pingcap/tidb/pull/11908/files#diff-350127760839dbfd52d23927f7ff2d95R706 if the log level is higher than debug.

executor/adapter.go
Succ: succ,
}
if _, ok := a.StmtNode.(*ast.CommitStmt); ok {
slowItems.PrevStmt = FormatSQL(sessVars.PrevStmt, sessVars)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PrevStmt only records the last executed statement, it's not enough to find which sql in the transaction is slow.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is all what we could do :)

@jackysp
Copy link
Member Author

jackysp commented Aug 29, 2019

PTAL @crazycs520 @zz-jason

crazycs520
crazycs520 reviewed Aug 30, 2019
View reviewed changes
Copy link
Contributor

@crazycs520 crazycs520 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

zz-jason
zz-jason approved these changes Sep 3, 2019
View reviewed changes
Copy link
Member

@zz-jason zz-jason left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zz-jason zz-jason added status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. labels Sep 3, 2019
@sre-bot
Copy link
Contributor

sre-bot commented Sep 3, 2019

/run-all-tests

@sre-bot sre-bot merged commit 994e14e into pingcap:master Sep 3, 2019
@sre-bot
Copy link
Contributor

sre-bot commented Sep 3, 2019

cherry pick to release-2.1 failed

@sre-bot
Copy link
Contributor

sre-bot commented Sep 3, 2019

cherry pick to release-3.0 failed

jackysp added a commit to jackysp/tidb that referenced this pull request Sep 12, 2019
@jackysp
*: record previous statement when commit is slow (pingcap#11908)
544535b
@jackysp jackysp mentioned this pull request Sep 12, 2019
Merged
sre-bot pushed a commit that referenced this pull request Sep 12, 2019
@jackysp @sre-bot
*: record previous statement when commit is slow (#11908) (#12180)
2a33209
jackysp added a commit to jackysp/tidb that referenced this pull request Oct 15, 2019
@jackysp
*: record previous statement when commit is slow (pingcap#11908)
ecfbb79
@jackysp jackysp mentioned this pull request Oct 15, 2019
Merged
sre-bot pushed a commit that referenced this pull request Oct 16, 2019
@jackysp @sre-bot
*: record previous statement when commit is slow (#11908) (#12724)
c1c80d6
jackysp added a commit to jackysp/tidb that referenced this pull request Nov 7, 2019
@jackysp
*: record previous statement when commit is slow (pingcap#11908) (pin…
855796d 
…gcap#12180)

Conflicts:
session/tidb.go
sessionctx/variable/session.go
sre-bot pushed a commit that referenced this pull request Nov 7, 2019
@jackysp @sre-bot
*: record previous statement when commit is slow (#11908) (#12180) (#…
d2ec98f 
…13232)
@jackysp jackysp deleted the last_stmt branch February 27, 2020 13:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@crazycs520 crazycs520 crazycs520 left review comments

@imtbkcat imtbkcat imtbkcat left review comments

@lysu lysu lysu left review comments

@zz-jason zz-jason zz-jason approved these changes

Assignees
No one assigned
Labels
status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@jackysp @sre-bot @lysu @zz-jason @imtbkcat @crazycs520