When PD node down, can not query TIKV_REGION_STATUS

[yilongrong]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

1. use TiUP create a v5.4.0 tidb cluster (PD must not on same host)
2. tiup cluster stop one PD node
3. select * from TIKV_REGION_STATUS meet error

2. What did you expect to see? (Required)

select * from TIKV_REGION_STATUS should normal.

3. What did you see instead (Required)

ERROR 1105 (HY000): Get "http://172.xx.xx.162:18279/pd/api/v1/regions": dial tcp 172.xx.xx.162:18279: connect: connection refused

tidb.log erro stack:
[2022/06/24 10:27:47.896 +08:00] [INFO] [conn.go:1115] ["command dispatched failed"] [conn=7] [connInfo="id:7, addr:172.xxx.xx.136:55676 status:10, collation:utf8_general_ci, user:root"] [command=Query] [status="inTxn:0, autocommit:1"] [sql="select * from TIKV_REGION_STATUS"] [txn_mode=PESSIMISTIC] [err="Get \"http://172.xxx.xx.162:18279/pd/api/v1/regions\": dial tcp 172.xxx.xx.162:18279: connect: connection refused\ngit.luolix.top/pingcap/errors.AddStack\n\t/nfs/cache/mod/github.com/pingcap/errors@v0.11.5-0.20211224045212-9687c2b0f87c/errors.go:174\ngit.luolix.top/pingcap/errors.Trace\n\t/nfs/cache/mod/github.com/pingcap/errors@v0.11.5-0.20211224045212-9687c2b0f87c/juju_adaptor.go:15\ngit.luolix.top/pingcap/tidb/store/helper.(*Helper).requestPD\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/store/helper/helper.go:813\ngit.luolix.top/pingcap/tidb/store/helper.(*Helper).GetRegionsInfo\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/store/helper/helper.go:771\ngit.luolix.top/pingcap/tidb/executor.(*memtableRetriever).setDataForTiKVRegionStatus\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/infoschema_reader.go:1449\ngit.luolix.top/pingcap/tidb/executor.(*memtableRetriever).retrieve\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/infoschema_reader.go:141\ngit.luolix.top/pingcap/tidb/executor.(*MemTableReaderExec).Next\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/memtable_reader.go:118\ngit.luolix.top/pingcap/tidb/executor.Next\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/executor.go:286\ngit.luolix.top/pingcap/tidb/executor.(*recordSet).Next\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/adapter.go:149\ngit.luolix.top/pingcap/tidb/server.(*tidbResultSet).Next\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/driver_tidb.go:312\ngit.luolix.top/pingcap/tidb/server.(*clientConn).writeChunks\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:2165\ngit.luolix.top/pingcap/tidb/server.(*clientConn).writeResultset\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:2116\ngit.luolix.top/pingcap/tidb/server.(*clientConn).handleStmt\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:1994\ngit.luolix.top/pingcap/tidb/server.(*clientConn).handleQuery\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:1841\ngit.luolix.top/pingcap/tidb/server.(*clientConn).dispatch\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:1336\ngit.luolix.top/pingcap/tidb/server.(*clientConn).Run\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:1091\ngit.luolix.top/pingcap/tidb/server.(*Server).onConn\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/server.go:548\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1371"]

4. What is your TiDB version? (Required)

v5.4.0

[tangenta]

@yilongrong Why do you think this is a bug? The implementation shows that PD is required to access tikv_region_status.

[yilongrong]

The TiDB cluster has 3 PD nodes. When a PD node is stopped, pd functions still normally
I think select * from TIKV_REGION_STATUS should be displayed normally.
The stopped PD node should be ignored.
Please also check https://asktug.com/t/topic/694309

[tangenta]

@yilongrong OK I get it. This issue is confirmed to be duplicated with #26303. It has been fixed in v5.2 and later versions.

[tangenta]

This issue should be tracked in #35319.