-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[YSQL] Support backup for pre-split multi-tablet range tables #4873
Comments
The idea is simple:
|
Example of the range table: https://docs.yugabyte.com/latest/api/ysql/the-sql-language/statements/ddl_create_table/
|
Commits b14485a and 96beb9e add generic YSQL backup/restore handling for any partitioning mismatch. Even if the In theory, this fix should extend to range partitioned tables. However, since the repartition takes time (and locks on master), it may be worth adding a Also, @yifanguan, note that the first commit I mention adds a disabled test |
AS requested schema DDL attached to ticket |
…ations Summary: After 96beb9e, restoring a backup will re-partition relations to match the splits from their actual tablet snapshots if they differ from the pre-created ones. As a result, even if the split information from the ysql_dump file is missing or not accurate, restore should go through correctly. Therefore, as a stop-gap fix for backup-restore make the error when dumping a multi-tablet range-split relation (table or index) just a warning instead. This diff also fixes an issue in TestYSQLRangeSplitConstraint index-data validation code. Previously that data read was actually using the table instead of the index. Fully supporting range-split tables in ysql_dump (SPLIT AT clause) is still needed for regular ysql_dump export flow and may also improve performance of restore. So this will be addressed in a follow-up commit. Test Plan: YBBackupTest.TestYSQLRangeSplitConstraint org.yb.pgsql.TestYsqlDump Reviewers: yguan, jason Reviewed By: yguan, jason Subscribers: yql Differential Revision: https://phabricator.dev.yugabyte.com/D15700
…range-split relations Summary: In this backport diff, formatted string's type field for `yb_table_properties.num_tablets` is changed from `PRIu64` to `%u` because its type is `uint32_t` instead of `uint64_t` on this backport branch. The expected output files of test: TestYsqlDump are also changed accordingly based on branch 2.12. This test passed on Jenkins. After 96beb9e, restoring a backup will re-partition relations to match the splits from their actual tablet snapshots if they differ from the pre-created ones. As a result, even if the split information from the ysql_dump file is missing or not accurate, restore should go through correctly. Therefore, as a stop-gap fix for backup-restore make the error when dumping a multi-tablet range-split relation (table or index) just a warning instead. This diff also fixes an issue in TestYSQLRangeSplitConstraint index-data validation code. Previously that data read was actually using the table instead of the index. Fully supporting range-split tables in ysql_dump (SPLIT AT clause) is still needed for regular ysql_dump export flow and may also improve performance of restore. So this will be addressed in a follow-up commit. Original Commit: 3156825 Original Differential Revision: https://phabricator.dev.yugabyte.com/D15700 Test Plan: YBBackupTest.TestYSQLRangeSplitConstraint org.yb.pgsql.TestYsqlDump Reviewers: mihnea, jason Reviewed By: jason Subscribers: yql Differential Revision: https://phabricator.dev.yugabyte.com/D15758
…plit relations Summary: After 96beb9e, restoring a backup will re-partition relations to match the splits from their actual tablet snapshots if they differ from the pre-created ones. As a result, even if the split information from the ysql_dump file is missing or not accurate, restore should go through correctly. Therefore, as a stop-gap fix for backup-restore make the error when dumping a multi-tablet range-split relation (table or index) just a warning instead. This diff also fixes an issue in TestYSQLRangeSplitConstraint index-data validation code. Previously that data read was actually using the table instead of the index. Fully supporting range-split tables in ysql_dump (SPLIT AT clause) is still needed for regular ysql_dump export flow and may also improve performance of restore. So this will be addressed in a follow-up commit. Test Plan: YBBackupTest.TestYSQLRangeSplitConstraint org.yb.pgsql.TestYsqlDump Reviewers: yguan, jason Reviewed By: yguan, jason Subscribers: yql Differential Revision: https://phabricator.dev.yugabyte.com/D15700
…ange-split relations Summary: In this backport diff, for `pg_dump.c` and `ruleutils.c`, formatted string's type field for yb_table_properties.num_tablets is changed from `PRIu64` to `%u` because its type is `uint32_t` instead of `uint64_t` on this backport branch. The expected output files of test: `TestYsqlDump` are also changed accordingly based on branch 2.8. Changes to `yb_ysql_dump_verifier.out` is ignored because there is no test file: `yb_ysql_dump_verifier.sql` on branch 2.8. After 96beb9e, restoring a backup will re-partition relations to match the splits from their actual tablet snapshots if they differ from the pre-created ones. As a result, even if the split information from the ysql_dump file is missing or not accurate, restore should go through correctly. Therefore, as a stop-gap fix for backup-restore make the error when dumping a multi-tablet range-split relation (table or index) just a warning instead. This diff also fixes an issue in TestYSQLRangeSplitConstraint index-data validation code. Previously that data read was actually using the table instead of the index. Fully supporting range-split tables in ysql_dump (SPLIT AT clause) is still needed for regular ysql_dump export flow and may also improve performance of restore. So this will be addressed in a follow-up commit. Original Commit: 3156825 Original Differential Revision: https://phabricator.dev.yugabyte.com/D15700 Test Plan: YBBackupTest.TestYSQLRangeSplitConstraint org.yb.pgsql.TestYsqlDump Reviewers: mihnea, jason Reviewed By: jason Subscribers: yql Differential Revision: https://phabricator.dev.yugabyte.com/D15940
Closing as this is solved for backups by 3156825. |
Jira Link: DB-2111
Currently
ysql_dump
being used for distributed backups only addsSPLIT INTO
clause (or tablet partition information) for hash tables. We should add support for pre-split range tables.The text was updated successfully, but these errors were encountered: