cluster: support evict leaders on stop and restore on start

[AstroProfundis]

What problem does this PR solve?

Close #1781

What is changed and how it works?

• Add a hidden --evict-leaders argument to tiup-cluster stop command, to evict store leaders from an instance and wait it to complete, ignored for non-store instances. Only use it with -N argument to stop specific instance, if there is no place for store leaders to transfer, the operation may fail.
• Add a hidden --restore-leaders argument to tiup-cluster start command, to delete the store leader evict scheduler of an instance, so that leaders are allowed to transfer back to it. If the instance was stopped with --evict-leaders and started without --restore-leaders, the instance will be up but no leader could be transferred to it, in that case, run start again with --restore-leaders or use pd-ctl to delete the store leader evict scheduler.

Also comment some unused code in pkg/cluster/task/action.go, we're calling those functions direct in manager, not via a task, maybe we need to also refactor those part later to make things easier.

Check List

Tests

• Manual test (add detailed scripts or steps below)

Code changes

• Has exported function/method change

Side effects

• Increased code complexity

[ti-chi-bot]

[REVIEW NOTIFICATION]

This pull request has been approved by:

• srstack

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

[codecov-commenter]

Codecov Report

Merging #1789 (6e48a8e) into master (5b18753) will decrease coverage by 5.90%.
The diff coverage is 62.14%.

❗ Current head 6e48a8e differs from pull request most recent head 836dcda. Consider uploading reports for the commit 836dcda to get more accurate results

@@            Coverage Diff             @@
##           master    #1789      +/-   ##
==========================================
- Coverage   53.08%   47.18%   -5.90%     
==========================================
  Files         303      303              
  Lines       34587    34829     +242     
==========================================
- Hits        18358    16431    -1927     
- Misses      14077    16436    +2359     
+ Partials     2152     1962     -190     

Flag	Coverage Δ
cluster	32.09% <60.24%> (-9.24%)	⬇️
dm	24.83% <53.75%> (+0.01%)	⬆️
playground	24.95% <0.00%> (+12.33%)	⬆️
tiup	14.40% <ø> (-0.02%)	⬇️
unittest	21.30% <23.08%> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
components/cluster/command/audit.go	41.38% <ø> (-18.00%)	⬇️
components/dm/command/audit.go	62.07% <ø> (+2.69%)	⬆️
pkg/cluster/audit/audit.go	37.81% <ø> (-1.03%)	⬇️
pkg/cluster/manager/cleanup.go	0.00% <0.00%> (-57.92%)	⬇️
pkg/cluster/operation/operation.go	74.19% <ø> (-6.45%)	⬇️
pkg/cluster/task/action.go	46.30% <0.00%> (-9.26%)	⬇️
pkg/cluster/operation/action.go	60.69% <39.39%> (-4.25%)	⬇️
pkg/cluster/manager/basic.go	48.29% <44.44%> (-6.30%)	⬇️
components/cluster/command/stop.go	52.00% <83.33%> (-18.00%)	⬇️
components/dm/command/scale_in.go	60.71% <88.89%> (+1.31%)	⬆️
... and 92 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5b18753...836dcda. Read the comment docs.

[AstroProfundis]

/merge

[ti-chi-bot]

This pull request has been accepted and is ready to merge.

Commit hash: 836dcda