Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*: support parsing BACKUP and RESTORE statements #746

Merged
merged 6 commits into from
Mar 16, 2020

Conversation

kennytm
Copy link
Contributor

@kennytm kennytm commented Feb 17, 2020

What problem does this PR solve?

Added the BACKUP, RESTORE and SHOW BACKUP/RESTORE statements to parser.

BACKUP DATABASE * TO 'local:///tmp/archive01/';
BACKUP DATABASE `a` TO 'local:///tmp/archive02/';
BACKUP TABLE `a`.`b` TO 'local:///tmp/archive03/';
RESTORE DATABASE * FROM 'local:///tmp/archive01/';
RESTORE DATABASE `a` FROM 'local:///tmp/archive02/';
RESTORE TABLE `a`.`b` FROM 'local:///tmp/archive03/';
BACKUP DATABASE * FULL TO 'local:///tmp/archive04/';
BACKUP DATABASE * INCREMENTAL UNTIL TIMESTAMP '2020-02-02 14:15:16' TO 'local:///tmp/archive05/';
BACKUP DATABASE * INCREMENTAL UNTIL TIMESTAMP_ORACLE 1234567890 TO 'local:///tmp/archive06/';

BACKUP DATABASE * TO 's3://my_bucket/path' 
    RATE_LIMIT = 100 MB/SECOND 
    SNAPSHOT = 2 HOUR AGO 
    CONCURRENCY = 32 
    S3_REGION = 'us-west-1' 
    S3_USE_ACCELERATE_ENDPOINT = 1;

SHOW BACKUP;
SHOW RESTORE;
SHOW BACKUP LIKE 'b01234';
SHOW BACKUP WHERE status <> 'running';

These are needed for integrating BR into TiDB (PR will come later).

(Note: because the new unresolved keyword S3_USE_ACCELERATE_ENDPOINT is longer than all existing tokens, forcing gofmt to replace the entire token map, I've taken advantage of this to sort the entire tokenMap)

What is changed and how it works?

Check List

Tests

  • Unit test

Code changes

Side effects

Related changes

  • Need to update the documentation
  • Need to be included in the release note

@kennytm kennytm added type/enhancement New feature or request status/PTAL labels Feb 17, 2020
@kennytm kennytm requested a review from a team February 17, 2020 21:08
@ghost ghost requested review from tangenta and removed request for a team February 17, 2020 21:08
@kennytm kennytm force-pushed the kennytm/backup-restore-statements branch from e62fb52 to f9e078c Compare February 17, 2020 21:24
@codecov
Copy link

codecov bot commented Feb 17, 2020

Codecov Report

Merging #746 into master will increase coverage by 0.21%.
The diff coverage is 81.42%.

@@            Coverage Diff             @@
##           master     #746      +/-   ##
==========================================
+ Coverage   77.82%   78.03%   +0.21%     
==========================================
  Files          40       40              
  Lines       14421    14632     +211     
==========================================
+ Hits        11223    11418     +195     
- Misses       2514     2535      +21     
+ Partials      684      679       -5

Copy link
Contributor

@tangenta tangenta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

}
| "RATE_LIMIT" EqOpt LengthNum "MB" '/' "SECOND"
{
// TODO: check overflow?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How can it overflow ❔

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tangenta RATE_LIMIT = 9223372036854775807 MB/SECOND... admittedly this is not a real issue 😅

{"backup database * to 'noop://' rate_limit 500 MB/second snapshot 5 minute ago", true, "BACKUP DATABASE * TO 'noop://' RATE_LIMIT = 500 MB/SECOND SNAPSHOT = 300000000 MICROSECOND AGO"},
{"restore table g from 'noop://' concurrency 40 checksum 0 online 1", true, "RESTORE TABLE `g` FROM 'noop://' CONCURRENCY = 40 CHECKSUM = 0 ONLINE = 1"},
{
// FIXME: should we really include the access key in the Restore() text???
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we also handle SetPwdStmt.Restore()?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😱. Perhaps deal with this later (e.g. add a WriteSecureString() method). This needs to be considered together with how TiDB uses Restore().

@tangenta tangenta added the status/LGT1 LGT1 label Feb 18, 2020
@tiancaiamao
Copy link
Collaborator

LGTM
With this feature implemented, preparing some test cases can be much faster.

ast/misc.go Show resolved Hide resolved
@IANTHEREAL
Copy link

@kennytm please resolve conflicts, it seems to be merged soon

@kennytm
Copy link
Contributor Author

kennytm commented Mar 15, 2020

PTAL again @tangenta @tiancaiamao thanks.

Copy link
Contributor

@tangenta tangenta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tiancaiamao
Copy link
Collaborator

LGTM

@kennytm kennytm merged commit c3235db into master Mar 16, 2020
@kennytm kennytm deleted the kennytm/backup-restore-statements branch March 16, 2020 08:09
tiancaiamao pushed a commit to tiancaiamao/parser that referenced this pull request Apr 27, 2021
* *: support parsing BACKUP and RESTORE statements

* parser: add SNAPSHOT = 'str' option to BACKUP statement
lyonzhi pushed a commit to lyonzhi/parser that referenced this pull request Apr 25, 2024
* *: support parsing BACKUP and RESTORE statements

* parser: add SNAPSHOT = 'str' option to BACKUP statement
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status/LGT1 LGT1 type/enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants