The efficiency of auto analyze is low with auto_analyze_priority_queue and infoSchemaV2 open in million tables scenario #54086
Labels
affects-8.5
This bug affects the 8.5.x(LTS) versions.
feature/developing
the related feature is in development
may-affects-5.4
This bug maybe affects 5.4.x versions.
may-affects-6.1
may-affects-6.5
may-affects-7.1
may-affects-7.5
may-affects-8.1
severity/major
sig/planner
SIG: Planner
type/bug
The issue is confirmed as a bug.
Bug Report
Please answer these questions before submitting your issue. Thanks!
1. Minimal reproduce step (Required)
tidb_schema_cache_size 2GB
create 100k schemas and about 15 tables each of schema. 1.5 million tables totally
insert data for above tables
It takes about 10 minutes to execute a single analyze task on average when auto_analyze_priority_queue and infoSchemaV2 open. While 20 analyze tasks are executed per minute when auto_analyze_priority_queue or infoSchemaV2 close.
Currently, the automatic statistics collection task priority queue feature retrieves all table information each time it performs an
analyze
, selecting one from them to execute. If some tables are not in theinfoschema
cache, this process is quite time-consuming.2. What did you expect to see? (Required)
3. What did you see instead (Required)
4. What is your TiDB version? (Required)
Release Version: v8.2.0-alpha
Edition: Community
Git Commit Hash: 6cf8776
Git Branch: heads/refs/tags/v8.2.0-alpha
UTC Build Time: 2024-06-13 11:49:48
GoVersion: go1.21.10
Race Enabled: false
Check Table Before Drop: false
Store: tikv
The text was updated successfully, but these errors were encountered: