server: fix a unstable test TestInitConnect

[tiancaiamao]

What problem does this PR solve?

Issue Number: close #30249

Problem Summary:

[2021-11-30T02:29:26.743Z] --- FAIL: TestInitConnect (1.36s)
[2021-11-30T02:29:26.743Z]     dbtestkit.go:72:
[2021-11-30T02:29:26.743Z]              Error Trace:    dbtestkit.go:72
[2021-11-30T02:29:26.743Z]                                                      server_test.go:1939
[2021-11-30T02:29:26.743Z]                                                      server_test.go:117
[2021-11-30T02:29:26.743Z]                                                      server_test.go:1938
[2021-11-30T02:29:26.743Z]                                                      tidb_test.go:1107
[2021-11-30T02:29:26.743Z]              Error:          Received unexpected error:
[2021-11-30T02:29:26.743Z]                              commands out of sync. Did you run multiple statements at once?
[2021-11-30T02:29:26.743Z]              Test:           TestInitConnect
[2021-11-30T02:29:26.743Z]              Messages:       sql:SET GLOBAL init_connect="insert into test.ts VALUES (NOW());SET @a=1;", args:[]
[2021-11-30T02:29:26.743Z]     server_test.go:112:
[2021-11-30T02:29:26.743Z]              Error Trace:    server_test.go:112
[2021-11-30T02:29:26.743Z]                                                      panic.go:613
[2021-11-30T02:29:26.743Z]                                                      testing.go:741
[2021-11-30T02:29:26.743Z]                                                      dbtestkit.go:72
[2021-11-30T02:29:26.743Z]                                                      server_test.go:1939
[2021-11-30T02:29:26.743Z]                                                      server_test.go:117
[2021-11-30T02:29:26.743Z]                                                      server_test.go:1938
[2021-11-30T02:29:26.743Z]                                                      tidb_test.go:1107
[2021-11-30T02:29:26.743Z]              Error:          Received unexpected error:
[2021-11-30T02:29:26.743Z]                              bad connection
[2021-11-30T02:29:26.743Z]              Test:           TestInitConnect

What is changed and how it works?

commands out of sync. Did you run multiple statements at once?

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No code

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

None

[ti-chi-bot]

[REVIEW NOTIFICATION]

This pull request has been approved by:

• djshow832
• xhebox

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

[hawkingrei]

/run-check_dev

[djshow832]

Error 1064: You have an error in your SQL syntax; check the manual that corresponds to your TiDB version for the right syntax to use line 1 column 60 near ""insert into test.ts VALUES (NOW());"

[bb7133]

I suspect there is something wrong with the parallel execution of test case:

> commands out of sync. Did you run multiple statements at once?

Which looks similiar to #30286

[tiancaiamao]

/hold

[tiancaiamao]

The change seems to be incorrect.

The sql should be written as SET GLOBAL init_connect="insert into test.ts VALUES (NOW());SET @a=1;"
This is a feature introduced in #23713, tidb will check the init_connect variable and execute the corresponding SQL for new incomming connection.

And the PR is merged in Mar 31 ... for a long I never see such fail in CI,
so I believe it might be related to the restruct of the test case #29535

There isn't much information in the error log, if it happens again I'll take a close look.

[djshow832]

I suspect there is something wrong with the parallel execution of test case:

> commands out of sync. Did you run multiple statements at once?

Which looks similiar to #30286

Agreed. Before migrating to testify, both TestGetIndexMVCC and TestInitConnect ran serially, but now they run in parallel and this kind of error happens frequently.

[bb7133]

Agreed. Before migrating to testify, both TestGetIndexMVCC and TestInitConnect ran serially.

Looks that we have to change them back.

[bb7133]

@tiancaiamao Could you try to clean up all the wrongly added Parallel() functions like:

https://github.com/pingcap/tidb/pull/29535/files#diff-40acdfe302a05645257d8fc93ba27dbecaeb828d2477fd44ff9ec8d89a5c9d6bR1101

[djshow832]

But I still have not figured out why running in parallel could cause this problem. Each Server runs separately with different ports in each tidbTestSuite and should not disturb other servers. :-(

commands out of sync. Did you run multiple statements at once? is thrown from the go-sql-driver and is not caused by running multiple statements most of the time, according to the discussion in go-sql-driver/mysql#314 .

[mjonss]

Could it be that the error really is 'bad connection'? I.e. there were some issue when connecting, which did not get handled properly?

[2021-11-30T02:29:26.743Z]              Error:          Received unexpected error:
[2021-11-30T02:29:26.743Z]                              bad connection

[mjonss]

I would suggest to add a db.Ping() right after the sql.Open() in runTests, to be sure the connection is working (Open does only open/create the handle, it does not actually connect yet).

[djshow832]

There is already a PR #30306