Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[YSQL] Collect catalog tables mutation counts #22104

Closed
1 task done
yifanguan opened this issue Apr 23, 2024 · 2 comments
Closed
1 task done

[YSQL] Collect catalog tables mutation counts #22104

yifanguan opened this issue Apr 23, 2024 · 2 comments
Assignees
@yifanguan
Labels
area/ysql Yugabyte SQL (YSQL) kind/new-feature This is a request for a completely new feature priority/medium Medium priority issue

Comments

@yifanguan
Copy link
Contributor

yifanguan commented Apr 23, 2024 •
edited by jira bot
Loading

Jira Link: DB-11037

Description

For auto-analyze service, we should make sure we collect catalogs' table mutation counts from DDL statements.
Accurate stats on catalogs help query optimizer chooses optimal plan for system queries.

Issue Type

kind/new-feature

Warning: Please confirm that this issue does not contain any sensitive information

  • I confirm this issue does not contain any sensitive information.
@yifanguan yifanguan added area/ysql Yugabyte SQL (YSQL) status/awaiting-triage Issue awaiting triage labels Apr 23, 2024
@yugabyte-ci yugabyte-ci added kind/new-feature This is a request for a completely new feature priority/medium Medium priority issue labels Apr 23, 2024
@yugabyte-ci yugabyte-ci removed the status/awaiting-triage Issue awaiting triage label Jun 25, 2024
@yifanguan
Copy link
Contributor Author

Currently, we collect mutations from direct DMLs on catalog tables, but not from DDLs.
For example, a simple CREATE TABLE does not increase the mutation count of pg_class.

yifanguan added a commit that referenced this issue Sep 11, 2024
@yifanguan
[#22104,#23506] YSQL: auto analyze service collects mutation counts f…
ee639f4 
…rom DDL and skips indexes

Summary:
This diff enhances the following two aspects of auto analyze service:
  - For DDLs inserting/updating/deleting entries of catalog tables, auto analyze service should track mutations count of them.
  - Auto analyze service should skip increasing mutations count for indexes because only tables should be analyzed by auto analyze service. Otherwise, indexes are analyzed redundantly (one analyze introduced by its base table, and one analyze introduced by itself). Also, PG doesn't support analyzing non-tables.
Jira: DB-11037, DB-12428

Test Plan:
./yb_build.sh --cxx-test pgwrapper_pg_auto_analyze-test --gtest_filter PgAutoAnalyzeTest.CheckDDLMutationsCount
./yb_build.sh --cxx-test pgwrapper_pg_auto_analyze-test --gtest_filter PgAutoAnalyzeTest.CheckIndexMutationsCount

Reviewers: mihnea, pjain

Reviewed By: pjain

Subscribers: svc_phabricator, ybase, yql

Differential Revision: https://phorge.dev.yugabyte.com/D37432
@yifanguan
Copy link
Contributor Author

issue is resolved by commit ee639f4

jasonyb pushed a commit that referenced this issue Sep 12, 2024
@yugabyte-ci @jaki
[BACKPORT pg15-cherrypicks] all: Bulk port from master - 99
730b16d 
Summary:
 aec9a66 [doc][xcluster] Truncate limitation (#23833)
 fe8890d [PLAT-13984] Change default metric graph points count from 100 to 250 + make runtime configurable.
 7d9b57b [#23869] YSQL: Fix one type of ddl atomicity stress test flakiness
 7c1bca8 [PLAT-14052][PLAT-15237] :Add advanced date-time option,Restrict CP for K8s
 afce6ad [PLAT-14158] Support multiple transactional xCluster configs per universe on YBA UI
 33342b3 [PLAT-15101] Add runtime config to turn on/off rollN for k8s
 361a99a [#23809] CDCSDK: Filter out records corresponding to index tables in colocated DBs
 8bbdf66 [docs] changed date (#23885)
 ee639f4 [#22104,#23506] YSQL: auto analyze service collects mutation counts from DDL and skips indexes
 3dbf6da Delete architecture/design/multi-region-xcluster-async-replication.md
 e013578 [#23864] DocDB: Move cluster_uuid to server common flags
 2b6a2d3 [PLAT-15062][PLAT-15071] Support DB scoped on UI and display schema change mode information
 8260075 [PLAT-15079] Treat dropped on target tables as unconfigured but preselected
 f5169ca DocDB: Follow redirects for callhome, and fix URL
 eb61ef6 [PLAT-15228] Update package installation command for YBA
 e791c40 [#18822] YSQL: Add serialization/deserialization mechanism for update optimization metadata
 e69d8cb [doc] Backups and DDLs (#23840)
 e72ae64 [PLAT-14552]filter support bundle core files by date
 8796c83 [PLAT-15274]: BackFill Pitr params for DR config and add not-null constraints in DB.
 dc9cc67 Fixed NPM test:ci build error
 2e5ebef [PLAT-15287]: Add PITR Column and Restore Window Information to Backup List
 d053e45 [#238989]yugabyted: Node doesn't join using `--join` flag
 da4da45 [#23399] DocDB: Fix StopActiveTxnsPriorTo from using local variables in the call back

Test Plan: Jenkins: rebase: pg15-cherrypicks

Reviewers: jason, tfoucher

Differential Revision: https://phorge.dev.yugabyte.com/D38008
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@yifanguan yifanguan

Labels
area/ysql Yugabyte SQL (YSQL) kind/new-feature This is a request for a completely new feature priority/medium Medium priority issue
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@yugabyte-ci @yifanguan