[Backup]Support content, exclude and whole database in backup #5314
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Change-Id: Ib1b7649a0657d4295d1b6f23b06d4b6de12efdea
EmmyMiao87
added
area/backup
morningman
changed the title
morningman
reviewed
Jan 28, 2021
fe/fe-core/src/main/java/org/apache/doris/analysis/AbstractBackupTableRefClause.java
Outdated
Show resolved
Hide resolved
| AbstractBackupTableRefClause abstractBackupTableRefClause = stmt.getAbstractBackupTableRefClause(); | ||
| if (abstractBackupTableRefClause == null) { | ||
| tableNames = db.getTableNamesWithLock(); | ||
| } else if (abstractBackupTableRefClause != null && abstractBackupTableRefClause.isExclude()) { |
There was a problem hiding this comment.
Suggested change
| } else if (abstractBackupTableRefClause != null && abstractBackupTableRefClause.isExclude()) { | |
| } else if (abstractBackupTableRefClause.isExclude()) { |
| } else if (abstractBackupTableRefClause != null && abstractBackupTableRefClause.isExclude()) { | ||
| tableNames = db.getTableNamesWithLock(); | ||
| for (TableRef tableRef : abstractBackupTableRefClause.getTableRefList()) { | ||
| tableNames.remove(tableRef.getName().getTbl()); |
There was a problem hiding this comment.
Do we need to check if there "excluded" tables exist in database?
eg:
Suggested change
| tableNames.remove(tableRef.getName().getTbl()); | |
| if (!tableNames.remove(tableRef.getName().getTbl())) { | |
| throw exception() | |
| } |
There was a problem hiding this comment.
I think it can be checked but it is not necessary to report an error.Just write a log.
| } | ||
| } | ||
| List<TableRef> tblRefs = Lists.newArrayList(); | ||
| if (tableNames.isEmpty()) { |
There was a problem hiding this comment.
There may be no table in database at all, and in this case, the tableNames is empty, and abstractBackupTableRefClause is null. So NPE will thrown.
| @@ -204,6 +278,7 @@ public String getOrginNameByAlias(String alias) { | |||
| public String database; | |||
| @SerializedName("backup_time") | |||
| public long backupTime; | |||
| public BackupContent content; | |||
There was a problem hiding this comment.
No need @SerializedName?
There was a problem hiding this comment.
No, it will be serialized by default, but some serialized names and attribute names are different.
Closed
EmmyMiao87
added a commit
to EmmyMiao87/incubator-doris
that referenced
this pull request
Apr 21, 2021
…#5314) This PR support following functions: 1. Support content properties in backup stmt. It means user can backup only metadata or meta+data which use content [METADATA_ONLY| ALL]attribute to distinguish. 2. Support exclude some tables in backup and restore stmt. This means that some very large and unimportant tables can be excluded when the entire database is backed up. 3. Support backup and restore whole database instead of declaring each table name in the backup and restore statement. The backup and restore api has changed as following: ``` BACKUP SNAPSHOT [db_name].{snapshot_name} TO 'repo_name' [ON|EXCLUDE ( 'table_name' [partition (p1,...)] )] [properties ( "content" = "metadata_only|all" )] RESTORE SNAPSHOT [db_name].{snapshot_name} TO 'repo_name' [EXCLUDE|ON ( 'table_name' [partition (p1,...)] )] [properties ( )] ``` Change-Id: I6ae7deb9fa4135294a6cfd3472c86e8ec9ce413a
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Proposed changes
This PR support following functions:
The backup and restore api has changed as following:
Types of changes
What types of changes does your code introduce to Doris?
Put an
xin the boxes that applyChecklist
Put an
xin the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.Further comments
I will continue to update the documentation, please review my code first.