planner,executor,parser: support the plan cache for prepared insert/delete/update statements #8107

dbjoa · 2018-10-30T07:15:25Z

What problem does this PR solve?

Support the plan cache for the prepared insert/delete/update statements

What is changed and how it works?

Extend rebuildRange() to support the insert/update/delete plans
Allow prepared insert/update/delete statements to be cacheable

Check List

Tests

Unit test

Code changes

Has exported function/method change

Side effects

No

Related changes

Need to be included in the release note

sre-bot · 2018-10-30T07:15:29Z

Hi contributor, thanks for your PR.

This patch needs to be approved by someone of admins. They should reply with "/ok-to-test" to accept this PR for running test automatically.

dbjoa · 2018-10-30T07:15:42Z

/run-all-tests

shenli · 2018-10-30T07:24:08Z

@dbjoa Thanks!

dbjoa · 2018-10-30T08:05:28Z

/run-common-test

planner/core/common_plans.go

eurekaka · 2018-10-30T13:21:05Z

vendor/github.com/pingcap/parser/ast/dml.go

-	node, ok := n.Table.Accept(v)
-	if !ok {
-		return n, false
+	if n.Table != nil {


Could you please give an example where n.Table is nil in INSERT?

I don't know a real example. The code is to pass the test case below .
https://github.com/pingcap/tidb/blob/6fbb42d22bdfdbe4822174c44430b64f7946b73f/planner/core/cacheable_checker_test.go#L41-L42

I see, then this code change is unreasonable IMHO. By the way, I learned that since we have moved parser to a separate repository, we need to file a PR in https://github.com/pingcap/parser if we want to make changes, and then make update in tidb repo.

eurekaka · 2018-10-30T14:32:46Z

The failed common-test can be simplified to this test case:

import java.sql.*;

class TiDBTestConnector {
	public static void main(String args[]){
		try{
				ResultSet rs;
				Class.forName("com.mysql.jdbc.Driver");
				Connection con=DriverManager.getConnection("jdbc:mysql://127.0.0.1:4000/test?useServerPrepStmts=true","root","");
				//createTable("testExecuteLargeBatch", "(id BIGINT AUTO_INCREMENT PRIMARY KEY, n INT)");
				PreparedStatement stmt = con.prepareStatement("INSERT INTO testExecuteLargeBatch (n) VALUES (?)", Statement.RETURN_GENERATED_KEYS);
				stmt.setInt(1, 1);
				stmt.addBatch();
				stmt.setInt(1, 2);
				stmt.addBatch();
				stmt.setInt(1, 3);
				stmt.addBatch();
				stmt.setInt(1, 4);
				stmt.addBatch();
				stmt.addBatch("INSERT INTO testExecuteLargeBatch (n) VALUES (5), (6), (7)");
				stmt.setString(1, "eight");
				stmt.addBatch();
				stmt.addBatch("INSERT INTO testExecuteLargeBatch (n) VALUES (9), (10)");
				try {
					stmt.executeLargeBatch();
					System.out.println("error, should raise exception");
				} catch (BatchUpdateException e) {
					String err = "Incorrect int value: 'eight' for column 'n' at row 1";
					if (!err.equals(e.getMessage())) {
						System.out.println("wrong error message: " + e.getMessage());
					}
				}
				System.out.println("finished");
				con.close();
		}catch(Exception e) { System.out.println(e); }
	}
}

when plan cache is enabled, the error message returned is wrong: Incorrect int value: '4' for column 'n' at row 1, you can use this for diagnosing.

dbjoa · 2018-10-30T14:36:48Z

@eurekaka, Thank you for the test case. I'll come back after fixing the failure.

dbjoa · 2018-10-31T03:30:24Z

@eurekaka, The updated PR should fix the test failure.

dbjoa · 2018-10-31T03:30:40Z

/run-all-tests

eurekaka

LGTM, except for the parser part mentioned. @zz-jason @winoros @lamxTyler PTAL.

dbjoa · 2018-10-31T05:56:36Z

@eurekaka, I've made the parser PR #5.

zimulala · 2018-11-01T08:50:47Z

planner/core/common_plans.go

@@ -192,6 +193,7 @@ func (e *Execute) getPhysicalPlan(ctx sessionctx.Context, is infoschema.InfoSche
 			if err != nil {
 				return nil, errors.Trace(err)
 			}
+			log.Errorf("hit: %+v", ToString(plan))


Why here use the level of “Error”？

Thank you for the comment. The PR should remove the debugging code. I'll delete the line from the updated PR.

zimulala · 2018-11-01T08:52:57Z

planner/core/cacheable_checker.go

@@ -21,7 +21,12 @@ import (

 // Cacheable checks whether the input ast is cacheable.
 func Cacheable(node ast.Node) bool {
-	if _, isSelect := node.(*ast.SelectStmt); !isSelect {
+


Could we remove this blank line?

Thank you for the comment. I'll delete the line from the updated PR.

…elete/update statements

dbjoa · 2018-11-02T05:14:00Z

/run-all-tests

dbjoa · 2018-11-02T05:33:52Z

@eurekaka, @zimulala PTAL

eurekaka

LGTM

XuHuaiyu

Really great work! Rest LGTM

expression/util.go

XuHuaiyu

LGTM

…ert-delete

shenli added the contribution This PR is from a community contributor. label Oct 30, 2018

eurekaka added sig/planner SIG: Planner type/enhancement The issue or PR belongs to an enhancement. labels Oct 30, 2018

eurekaka reviewed Oct 30, 2018

View reviewed changes

planner/core/common_plans.go Outdated Show resolved Hide resolved

planner/core/common_plans.go Show resolved Hide resolved

eurekaka reviewed Oct 30, 2018

View reviewed changes

eurekaka added the status/all tests passed label Oct 31, 2018

eurekaka reviewed Oct 31, 2018

View reviewed changes

dbjoa mentioned this pull request Oct 31, 2018

ast: fix the panics of testing the TiDB PR #8017 pingcap/parser#5

Closed

zimulala reviewed Nov 1, 2018

View reviewed changes

dbjoa changed the title ~~planner,executor,parser: support the plan cache for prepared insert/delete/update statements~~ [WIP] planner,executor,parser: support the plan cache for prepared insert/delete/update statements Nov 1, 2018

planner,executor,parser: support the prepared plan cache for insert/d…

79eb3d0

…elete/update statements

dbjoa changed the title ~~[WIP] planner,executor,parser: support the plan cache for prepared insert/delete/update statements~~ planner,executor,parser: support the plan cache for prepared insert/delete/update statements Nov 2, 2018

eurekaka reviewed Nov 2, 2018

View reviewed changes

eurekaka added the status/LGT1 Indicates that a PR has LGTM 1. label Nov 2, 2018

XuHuaiyu reviewed Nov 5, 2018

View reviewed changes

expression/util.go Show resolved Hide resolved

XuHuaiyu approved these changes Nov 5, 2018

View reviewed changes

Merge branch 'master' into support-prepared-plan-cache-for-update-ins…

0f5b517

…ert-delete

XuHuaiyu added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Nov 5, 2018

ngaut merged commit 34c3d4c into pingcap:master Nov 5, 2018

dbjoa mentioned this pull request Dec 11, 2018

types: restore the field length of a string typed parameter value to the original one #8656

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

planner,executor,parser: support the plan cache for prepared insert/delete/update statements #8107

planner,executor,parser: support the plan cache for prepared insert/delete/update statements #8107

dbjoa commented Oct 30, 2018

sre-bot commented Oct 30, 2018

dbjoa commented Oct 30, 2018

shenli commented Oct 30, 2018

dbjoa commented Oct 30, 2018

eurekaka Oct 30, 2018

dbjoa Oct 30, 2018

eurekaka Oct 30, 2018

eurekaka commented Oct 30, 2018

dbjoa commented Oct 30, 2018

dbjoa commented Oct 31, 2018

dbjoa commented Oct 31, 2018

eurekaka left a comment

dbjoa commented Oct 31, 2018

zimulala Nov 1, 2018

dbjoa Nov 1, 2018

zimulala Nov 1, 2018

dbjoa Nov 1, 2018

dbjoa commented Nov 2, 2018

dbjoa commented Nov 2, 2018

eurekaka left a comment

XuHuaiyu left a comment

XuHuaiyu left a comment

planner,executor,parser: support the plan cache for prepared insert/delete/update statements #8107

planner,executor,parser: support the plan cache for prepared insert/delete/update statements #8107

Conversation

dbjoa commented Oct 30, 2018

What problem does this PR solve?

What is changed and how it works?

Check List

sre-bot commented Oct 30, 2018

dbjoa commented Oct 30, 2018

shenli commented Oct 30, 2018

dbjoa commented Oct 30, 2018

eurekaka Oct 30, 2018

Choose a reason for hiding this comment

dbjoa Oct 30, 2018

Choose a reason for hiding this comment

eurekaka Oct 30, 2018

Choose a reason for hiding this comment

eurekaka commented Oct 30, 2018

dbjoa commented Oct 30, 2018

dbjoa commented Oct 31, 2018

dbjoa commented Oct 31, 2018

eurekaka left a comment

Choose a reason for hiding this comment

dbjoa commented Oct 31, 2018

zimulala Nov 1, 2018

Choose a reason for hiding this comment

dbjoa Nov 1, 2018

Choose a reason for hiding this comment

zimulala Nov 1, 2018

Choose a reason for hiding this comment

dbjoa Nov 1, 2018

Choose a reason for hiding this comment

dbjoa commented Nov 2, 2018

dbjoa commented Nov 2, 2018

eurekaka left a comment

Choose a reason for hiding this comment

XuHuaiyu left a comment

Choose a reason for hiding this comment

XuHuaiyu left a comment

Choose a reason for hiding this comment