add index blocked after injection pd leader io delay

[Lily2025]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

1、tidb_enable_dist_task='off'
2、run sysbench
3、add index for one table
4、injection pd leader io delay

logs:
endless-ha-test-add-index-tps-7553489-1-251.zip

2. What did you expect to see? (Required)

add index can success

3. What did you see instead (Required)

add index blocked after injection pd leader io delay

the status of ddl job is not synced after 1h0m0s (now: 2024-09-06 12:33:26, jobId: 611, job type: add index /* ingest /, state: running)
operatorLogs:
[2024-09-06 11:33:15] ###### start adding index
ALTER TABLE sbtest1 ADD INDEX index_test_1725593595755(c)
[2024-09-06 11:33:15] ###### wait for ddl job finish
[2024-09-06 12:33:26] ###### wait for ddl job finish timeout(1h0m0s)
select job_id, db_name, table_name, job_type, create_time, start_time, end_time, state, query from information_schema.ddl_jobs where query = 'ALTER TABLE sbtest1 ADD INDEX index_test_1725593595755(c)'
jobId: 611, job type: add index / ingest */, state: running

4. What is your TiDB version? (Required)

./tidb-server -V
Release Version: v8.4.0-alpha
Edition: Community
Git Commit Hash: 9c7f4eb
Git Branch: heads/refs/tags/v8.4.0-alpha
UTC Build Time: 2024-09-05 11:51:24
GoVersion: go1.21.10
Race Enabled: false
Check Table Before Drop: false
Store: unistore
2024-09-06T11:13:31.856+0800

[Lily2025]

/assign tangenta

[tangenta]

Introduced by #54149