Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement checkpoint #1069

Merged
merged 18 commits into from
Jan 22, 2021
Merged

Implement checkpoint #1069

merged 18 commits into from
Jan 22, 2021

Conversation

lucklove
Copy link
Member

What problem does this PR solve?

Close #1026

What is changed and how it works?

Implement a cache struct (CheckPointExecutor) overlay Executor, which writes the results of Executor to auth log and read them back when in checkpoint mode:

If user specify the --checkpoint flag,the CheckPointExecutor will read all checkpoints from the specified auth log and use them as the cache. After that, when there is a command should be execute in remote, it will check if it's exists in the cache, if so, use the cached result directly, otherwise, call real Executor and records it's result to current auth log.

Check List

Tests

  • Unit test

Side effects

  • Increased code complexity

Related changes

  • Need to update the documentation

Release notes:

NONE

@lucklove lucklove added the status/need-doc Indicates that we should update document before merge a PR. label Jan 14, 2021
@lucklove lucklove added this to the v1.4.0 milestone Jan 14, 2021
@ti-chi-bot ti-chi-bot requested a review from lonng January 14, 2021 10:27
@ti-chi-bot ti-chi-bot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Jan 14, 2021
@codecov-io
Copy link

codecov-io commented Jan 14, 2021

Codecov Report

Merging #1069 (d3850c1) into master (d4881ba) will increase coverage by 0.13%.
The diff coverage is 64.81%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1069      +/-   ##
==========================================
+ Coverage   56.41%   56.54%   +0.13%     
==========================================
  Files         278      283       +5     
  Lines       19696    19970     +274     
==========================================
+ Hits        11111    11293     +182     
- Misses       6862     6928      +66     
- Partials     1723     1749      +26     
Flag Coverage Δ
cluster 45.04% <63.50%> (+0.41%) ⬆️
dm 24.78% <41.78%> (+0.43%) ⬆️
integrate 51.00% <63.00%> (+0.24%) ⬆️
playground 20.91% <ø> (ø)
tiup 16.41% <3.48%> (-0.27%) ⬇️
unittest 22.73% <19.85%> (+0.30%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
components/dm/command/import.go 19.04% <0.00%> (-0.31%) ⬇️
components/dm/command/scale_in.go 60.60% <0.00%> (ø)
components/dm/task/update_dm_meta.go 85.29% <ø> (ø)
pkg/cluster/ansible/config.go 0.00% <0.00%> (ø)
pkg/cluster/ansible/service.go 3.62% <0.00%> (-0.02%) ⬇️
pkg/cluster/audit/audit.go 63.46% <ø> (ø)
pkg/cluster/ctxt/event_bus.go 71.42% <ø> (ø)
pkg/cluster/executor/local.go 55.71% <ø> (-0.63%) ⬇️
pkg/cluster/manager/deploy.go 70.35% <0.00%> (ø)
pkg/cluster/manager/scale_in.go 59.52% <0.00%> (ø)
... and 84 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d4881ba...d3850c1. Read the comment docs.

@lucklove
Copy link
Member Author

/hold

@ti-chi-bot ti-chi-bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 15, 2021
@ti-chi-bot ti-chi-bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jan 19, 2021
@lucklove
Copy link
Member Author

/hold cancel

@ti-chi-bot ti-chi-bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 19, 2021
components/cluster/command/root.go Outdated Show resolved Hide resolved
components/dm/command/root.go Outdated Show resolved Hide resolved
pkg/checkpoint/checkpoint.go Outdated Show resolved Hide resolved
pkg/cluster/operation/action.go Show resolved Hide resolved
pkg/cluster/operation/operation.go Outdated Show resolved Hide resolved
pkg/cluster/operation/telemetry.go Show resolved Hide resolved
pkg/queue/any_queue.go Outdated Show resolved Hide resolved
pkg/queue/any_queue.go Show resolved Hide resolved
lucklove and others added 3 commits January 21, 2021 14:45
Co-authored-by: Allen Zhong <allen@moe.cat>
Co-authored-by: Allen Zhong <allen@moe.cat>
@AstroProfundis
Copy link
Contributor

/lgtm

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Jan 21, 2021
@AstroProfundis
Copy link
Contributor

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: fdc4771

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Jan 22, 2021
@ti-chi-bot
Copy link
Member

@lucklove: Your PR has out-of-dated, I have automatically updated it for you.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT1 Indicates that a PR has LGTM 1. status/need-doc Indicates that we should update document before merge a PR.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

support break off restart upgrade
4 participants