TiDB loses connection when executing transaction

[JZuming]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

Setup the environment:

tidb/bin/tidb-server &
mysql -h "127.0.0.1" -u root -P 4000 -D testdb < mysql_bk.sql

mysql_bk.sql:
mysql_bk.sql.txt

Testcase 1

mysql -h "127.0.0.1" -u root -P 4000 -D testdb
mysql> insert into t_p2n_bb values (2, 35);
mysql> delete from t_p2n_bb where exists (select 1 from t_aqzphd);

Testcase 2

mysql -h "127.0.0.1" -u root -P 4000 -D testdb
mysql> start transaction;
mysql> insert into t_p2n_bb values (2, 35);
mysql> delete from t_p2n_bb where exists (select 1 from t_aqzphd);
mysql> commit;

2. What did you expect to see? (Required)

Two testcases are both executed normally.

3. What did you see instead (Required)

Testcase 1 was executed normally

Query OK, 1 row affected (0.00 sec)

Query OK, 0 rows affected (0.01 sec)

But testcase 2 made the connection lost

Query OK, 1 row affected (0.00 sec)

ERROR 1105 (HY000): close of nil channel
ERROR 2013 (HY000): Lost connection to MySQL server during query

4. What is your TiDB version? (Required)

Release Version: v5.4.0-alpha-133-g20b9a4d8c
Edition: Community
Git Commit Hash: 20b9a4d8ca32449c5003963eb7d382633c3ed25e
Git Branch: master
UTC Build Time: 2021-11-17 08:53:20
GoVersion: go1.16
Race Enabled: false
TiKV Min Version: v3.0.0-60965b006877ca7234adaced7890d7b029ed1306
Check Table Before Drop: false

[JZuming]

The log of the server:

[2021/11/30 14:48:11.149 +00:00] [ERROR] [conn.go:971] ["connection running loop panic"] [conn=3] [lastSQL="delete from t_p2n_bb where exists (select 1 from t_aqzphd)"] [err="close of nil channel"] [stack="goroutine 475 [running]:\ngit.luolix.top/pingcap/tidb/server.(*clientConn).Run.func1(0x419b610, 0xc010623830, 0xc0101ce640)\n\t/home/zuming/tidb/server/conn.go:969 +0xf5\npanic(0x37f9300, 0x410ca40)\n\t/usr/local/go/src/runtime/panic.go:965 +0x1b9\ngit.luolix.top/pingcap/tidb/executor.(*ShuffleExec).Close(0xc010f89d40, 0x0, 0x0)\n\t/home/zuming/tidb/executor/shuffle.go:145 +0x4cc\ngit.luolix.top/pingcap/tidb/executor.(*SortExec).Close(0xc0111326c0, 0x0, 0xc0110d58f0)\n\t/home/zuming/tidb/executor/sort.go:81 +0xee\ngit.luolix.top/pingcap/tidb/executor.(*baseExecutor).Close(0xc011130300, 0x107, 0x419c1e0)\n\t/home/zuming/tidb/executor/executor.go:181 +0x7e\ngit.luolix.top/pingcap/tidb/executor.(*ProjectionExec).Close(0xc011130300, 0x0, 0x0)\n\t/home/zuming/tidb/executor/projection.go:329 +0x138\ngit.luolix.top/pingcap/tidb/executor.(*baseExecutor).Close(0xc010765b00, 0x36e36c0, 0x5bc25a8)\n\t/home/zuming/tidb/executor/executor.go:181 +0x7e\ngit.luolix.top/pingcap/tidb/executor.(*HashJoinExec).Close(0xc010765b00, 0xc0110c6fb0, 0xc0111329c0)\n\t/home/zuming/tidb/executor/join.go:153 +0x334\ngit.luolix.top/pingcap/tidb/parser/terror.Call(0xc00ffbefe0)\n\t/home/zuming/tidb/parser/terror/terror.go:298 +0x3f\ngit.luolix.top/pingcap/tidb/executor.init.1.func1(0x419b610, 0xc010ed9a70, 0x41dd2f0, 0xc00fbff6c0, 0x41da898, 0xc010dc8380, 0x41e5498, 0xc00fc07c00, 0x0, 0x0, ...)\n\t/home/zuming/tidb/executor/executor.go:1197 +0x4a5\ngit.luolix.top/pingcap/tidb/planner/core.(*expressionRewriter).handleExistSubquery(0xc010eb9790, 0x419b610, 0xc010ed9a70, 0xc0105f0a20, 0x0, 0x0, 0x11dea00)\n\t/home/zuming/tidb/planner/core/expression_rewriter.go:817 +0x9ae\ngit.luolix.top/pingcap/tidb/planner/core.(*expressionRewriter).Enter(0xc010eb9790, 0x41b3140, 0xc0105f0a20, 0xc00ffbf2c0, 0x288c1c9, 0xc0109f2280)\n\t/home/zuming/tidb/planner/core/expression_rewriter.go:373 +0x908\ngit.luolix.top/pingcap/tidb/parser/ast.(*ExistsSubqueryExpr).Accept(0xc0105f0a20, 0x4177540, 0xc010eb9790, 0x41e29b0, 0xc0105ca8c0, 0xc00ffbf388)\n\t/home/zuming/tidb/parser/ast/expressions.go:669 +0x4b\ngit.luolix.top/pingcap/tidb/planner/core.(*PlanBuilder).rewriteExprNode(0xc00098e000, 0xc010eb9790, 0x41cad18, 0xc0105f0a20, 0xc0105ca800, 0x0, 0x0, 0x0, 0x0, 0x0, ...)\n\t/home/zuming/tidb/planner/core/expression_rewriter.go:199 +0x9c\ngit.luolix.top/pingcap/tidb/planner/core.(*PlanBuilder).rewriteWithPreprocess(0xc00098e000, 0x419b610, 0xc010ed9a70, 0x41cad18, 0xc0105f0a20, 0x41e29b0, 0xc0105ca8c0, 0x0, 0x0, 0x0, ...)\n\t/home/zuming/tidb/planner/core/expression_rewriter.go:145 +0x14c\ngit.luolix.top/pingcap/tidb/planner/core.(*PlanBuilder).rewrite(0xc00098e000, 0x419b610, 0xc010ed9a70, 0x41cad18, 0xc0105f0a20, 0x41e29b0, 0xc0105ca8c0, 0x0, 0x0, 0x0, ...)\n\t/home/zuming/tidb/planner/core/expression_rewriter.go:113 +0xb9\ngit.luolix.top/pingcap/tidb/planner/core.(*PlanBuilder).buildSelection(0xc00098e000, 0x419b610, 0xc010ed9a70, 0x41e29b0, 0xc0105ca8c0, 0x41cad18, 0xc0105f0a20, 0x0, 0x0, 0x278ea25, ...)\n\t/home/zuming/tidb/planner/core/logical_plan_builder.go:975 +0xb6a\ngit.luolix.top/pingcap/tidb/planner/core.(*PlanBuilder).buildDelete(0xc00098e000, 0x419b610, 0xc010ed9a70, 0xc010069000, 0x0, 0x0, 0x0, 0x0)\n\t/home/zuming/tidb/planner/core/logical_plan_builder.go:5002 +0x2865\ngit.luolix.top/pingcap/tidb/planner/core.(*PlanBuilder).Build(0xc00098e000, 0x419b610, 0xc010ed9a70, 0x41b2d50, 0xc010069000, 0xc010ed9cb0, 0xc00098e000, 0xc010068580, 0x1)\n\t/home/zuming/tidb/planner/core/planbuilder.go:689 +0x6e5\ngit.luolix.top/pingcap/tidb/planner.optimize(0x419b610, 0xc010ed9a70, 0x41e5498, 0xc00fc07c00, 0x41b2d50, 0xc010069000, 0x41da898, 0xc010dc8380, 0x0, 0x0, ...)\n\t/home/zuming/tidb/planner/optimize.go:337 +0x2ba\ngit.luolix.top/pingcap/tidb/planner.Optimize(0x419b610, 0xc010ed9a70, 0x41e5498, 0xc00fc07c00, 0x41b2d50, 0xc010069000, 0x41da898, 0xc010dc8380, 0x0, 0x0, ...)\n\t/home/zuming/tidb/planner/optimize.go:221 +0x1334\ngit.luolix.top/pingcap/tidb/executor.(*Compiler).Compile(0xc00ffc0be0, 0x419b610, 0xc010ed9a70, 0x41b91e8, 0xc010069000, 0x0, 0x0, 0x0)\n\t/home/zuming/tidb/executor/compiler.go:66 +0x3bc\ngit.luolix.top/pingcap/tidb/session.(*session).ExecuteStmt(0xc00fc07c00, 0x419b610, 0xc010ed9a70, 0x41b91e8, 0x"]

[XuHuaiyu]

The panic happens because of that ShuffleExec.Open returns error, and the member attributes are NOT initialized successfully. Thus the ShuffleExec.Close triggers the nil pointer panic

[XuHuaiyu]

Another problem is that, why the error is returned in the transaction.
Error stack:

errors@v0.11.5-0.20211009033009-93128226aaa3/errors.go:174
github.com/pingcap/errors.(*Error).GenWithStackByArgs
	/Users/xuhuaiyu/Development/GOPATH/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20211009033009-93128226aaa3/normalize.go:159
github.com/pingcap/tidb/types.getValidFloatPrefix
	/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/types/convert.go:720
github.com/pingcap/tidb/types.StrToFloat
	/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/types/convert.go:528
github.com/pingcap/tidb/expression.(*builtinCastStringAsRealSig).evalReal
	/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/expression/builtin_cast.go:1229
github.com/pingcap/tidb/expression.(*ScalarFunction).EvalReal
	/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/expression/scalar_function.go:402
github.com/pingcap/tidb/expression.(*builtinRealIsTrueSig).evalInt
	/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/expression/builtin_op.go:535
github.com/pingcap/tidb/expression.(*ScalarFunction).EvalInt
	/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/expression/scalar_function.go:397
github.com/pingcap/tidb/expression.(*builtinLogicOrSig).evalInt
	/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/expression/builtin_op.go:155
github.com/pingcap/tidb/expression.(*ScalarFunction).EvalInt
	/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/expression/scalar_function.go:397
github.com/pingcap/tidb/expression.(*builtinCastIntAsStringSig).evalString
	/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/expression/builtin_cast.go:550
github.com/pingcap/tidb/expression.(*ScalarFunction).EvalString
	/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/expression/scalar_function.go:412
github.com/pingcap/tidb/expression.(*builtinLikeSig).evalInt
	/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/expression/builtin_like.go:79
github.com/pingcap/tidb/expression.(*ScalarFunction).EvalInt
	/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/expression/scalar_function.go:397
github.com/pingcap/tidb/expression.(*builtinUnaryNotIntSig).evalInt
	/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/expression/builtin_op.go:801
github.com/pingcap/tidb/expression.(*ScalarFunction).EvalInt
	/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/expression/scalar_function.go:397
github.com/pingcap/tidb/expression.(*ScalarFunction).Eval
	/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/expression/scalar_function.go:350
github.com/pingcap/tidb/expression.EvalBool
	/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/expression/expression.go:243
github.com/pingcap/tidb/executor.(*memTableReader).getMemRows.func1
	/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/executor/mem_reader.go:215
github.com/pingcap/tidb/executor.iterTxnMemBuffer
	/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/executor/mem_reader.go:352
github.com/pingcap/tidb/executor.(*memTableReader).getMemRows
	/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/executor/mem_reader.go:208
github.com/pingcap/tidb/executor.(*UnionScanExec).open
	/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/executor/union_scan.go:99
github.com/pingcap/tidb/executor.(*UnionScanExec).Open
	/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/executor/union_scan.go:69
github.com/pingcap/tidb/executor.(*baseExecutor).Open
	/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/executor/executor.go:169
github.com/pingcap/tidb/executor.(*HashJoinExec).Open
	/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/executor/join.go:182
github.com/pingcap/tidb/executor.(*baseExecutor).Open
	/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/executor/executor.go:169
github.com/pingcap/tidb/executor.(*ProjectionExec).Open
	/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/executor/projection.go:87
github.com/pingcap/tidb/executor.(*SortExec).Open
	/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/executor/sort.go:97
github.com/pingcap/tidb/executor.(*baseExecutor).Open
	/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/executor/executor.go:169
github.com/pingcap/tidb/executor.(*ProjectionExec).Open
	/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/executor/projection.go:87
github.com/pingcap/tidb/executor.(*baseExecutor).Open
	/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/executor/executor.go:169
github.com/pingcap/tidb/executor.(*ProjectionExec).Open
	/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/executor/projection.go:87

[XuHuaiyu]

Test in MySQL, the sql without transaction should also return an error.
So, the problem becomes why the sql without txn does not return an error

[github-actions]

Please check whether the issue should be labeled with 'affects-x.y' or 'fixes-x.y.z', and then remove 'needs-more-info' label.