Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*: refactor scheduler test and move scheduler register #5957

Merged
merged 7 commits into from
Feb 14, 2023

Conversation

lhy1024
Copy link
Contributor

@lhy1024 lhy1024 commented Feb 9, 2023

Signed-off-by: lhy1024 admin@liudos.us

What problem does this PR solve?

Issue Number: Close #5956

What is changed and how does it work?

  1. refactor scheduler test by adding newTestCluster
  2. move scheduler register and reduce unnecessary register

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)

Release note

None.

Signed-off-by: lhy1024 <admin@liudos.us>
@ti-chi-bot
Copy link
Member

ti-chi-bot commented Feb 9, 2023

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • nolouch
  • rleungx

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@codecov
Copy link

codecov bot commented Feb 9, 2023

Codecov Report

Base: 75.15% // Head: 75.11% // Decreases project coverage by -0.05% ⚠️

Coverage data is based on head (097d873) compared to base (5e07877).
Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5957      +/-   ##
==========================================
- Coverage   75.15%   75.11%   -0.05%     
==========================================
  Files         362      362              
  Lines       36280    36281       +1     
==========================================
- Hits        27267    27251      -16     
- Misses       6641     6656      +15     
- Partials     2372     2374       +2     
Flag Coverage Δ
unittests 75.11% <100.00%> (-0.05%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
server/testutil.go 90.62% <100.00%> (ø)
tests/cluster.go 79.27% <100.00%> (+0.06%) ⬆️
pkg/mcs/discovery/register.go 53.48% <0.00%> (-18.61%) ⬇️
client/resource_group/controller/limiter.go 61.25% <0.00%> (-6.25%) ⬇️
pkg/tso/local_allocator.go 64.86% <0.00%> (-4.06%) ⬇️
server/schedulers/shuffle_hot_region.go 56.66% <0.00%> (-3.34%) ⬇️
pkg/election/leadership.go 75.25% <0.00%> (-3.10%) ⬇️
pkg/election/lease.go 84.05% <0.00%> (-2.90%) ⬇️
pkg/encryption/key_manager.go 70.11% <0.00%> (-1.60%) ⬇️
pkg/tso/allocator_manager.go 63.38% <0.00%> (-0.81%) ⬇️
... and 11 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

Signed-off-by: lhy1024 <admin@liudos.us>
@lhy1024
Copy link
Contributor Author

lhy1024 commented Feb 9, 2023

except for #5953

# testing all pkgs...
CGO_ENABLED=1 go test -tags tso_function_test,deadlock -timeout 20m -race -cover github.com/tikv/pd/pkg/audit github.com/tikv/pd/pkg/autoscaling github.com/tikv/pd/pkg/btree github.com/tikv/pd/pkg/cache github.com/tikv/pd/pkg/cgroup github.com/tikv/pd/pkg/codec github.com/tikv/pd/pkg/core github.com/tikv/pd/pkg/core/rangetree github.com/tikv/pd/pkg/core/storelimit github.com/tikv/pd/pkg/dashboard/adapter github.com/tikv/pd/pkg/election github.com/tikv/pd/pkg/encryption github.com/tikv/pd/pkg/errs github.com/tikv/pd/pkg/gctuner github.com/tikv/pd/pkg/id github.com/tikv/pd/pkg/mcs/discovery github.com/tikv/pd/pkg/mcs/resource_manager/server github.com/tikv/pd/pkg/memory github.com/tikv/pd/pkg/mock/mockhbstream github.com/tikv/pd/pkg/movingaverage github.com/tikv/pd/pkg/progress github.com/tikv/pd/pkg/ratelimit github.com/tikv/pd/pkg/slice github.com/tikv/pd/pkg/storage github.com/tikv/pd/pkg/storage/endpoint github.com/tikv/pd/pkg/storage/kv github.com/tikv/pd/pkg/systimemon github.com/tikv/pd/pkg/utils/apiutil github.com/tikv/pd/pkg/utils/assertutil github.com/tikv/pd/pkg/utils/etcdutil github.com/tikv/pd/pkg/utils/grpcutil github.com/tikv/pd/pkg/utils/jsonutil github.com/tikv/pd/pkg/utils/keyutil github.com/tikv/pd/pkg/utils/logutil github.com/tikv/pd/pkg/utils/metricutil github.com/tikv/pd/pkg/utils/netutil github.com/tikv/pd/pkg/utils/reflectutil github.com/tikv/pd/pkg/utils/requestutil github.com/tikv/pd/pkg/utils/syncutil github.com/tikv/pd/pkg/utils/typeutil github.com/tikv/pd/server github.com/tikv/pd/server/api github.com/tikv/pd/server/cluster github.com/tikv/pd/server/config github.com/tikv/pd/server/gc github.com/tikv/pd/server/join github.com/tikv/pd/server/keyspace github.com/tikv/pd/server/region_syncer github.com/tikv/pd/server/replication github.com/tikv/pd/server/schedule github.com/tikv/pd/server/schedule/checker github.com/tikv/pd/server/schedule/filter github.com/tikv/pd/server/schedule/labeler github.com/tikv/pd/server/schedule/operator github.com/tikv/pd/server/schedule/placement github.com/tikv/pd/server/schedule/rangelist github.com/tikv/pd/server/schedulers github.com/tikv/pd/server/statistics github.com/tikv/pd/server/statistics/buckets github.com/tikv/pd/tests/autoscaling github.com/tikv/pd/tests/compatibility github.com/tikv/pd/tests/dashboard github.com/tikv/pd/tests/pdbackup github.com/tikv/pd/tests/pdctl github.com/tikv/pd/tests/pdctl/cluster github.com/tikv/pd/tests/pdctl/completion github.com/tikv/pd/tests/pdctl/config github.com/tikv/pd/tests/pdctl/health github.com/tikv/pd/tests/pdctl/hot github.com/tikv/pd/tests/pdctl/label github.com/tikv/pd/tests/pdctl/log github.com/tikv/pd/tests/pdctl/member github.com/tikv/pd/tests/pdctl/operator github.com/tikv/pd/tests/pdctl/region github.com/tikv/pd/tests/pdctl/scheduler github.com/tikv/pd/tests/pdctl/store github.com/tikv/pd/tests/pdctl/tso github.com/tikv/pd/tests/pdctl/unsafe github.com/tikv/pd/tests/registry github.com/tikv/pd/tests/server github.com/tikv/pd/tests/server/api github.com/tikv/pd/tests/server/apiv2/handlers github.com/tikv/pd/tests/server/cluster github.com/tikv/pd/tests/server/config github.com/tikv/pd/tests/server/id github.com/tikv/pd/tests/server/join github.com/tikv/pd/tests/server/join/join_fail github.com/tikv/pd/tests/server/keyspace github.com/tikv/pd/tests/server/member github.com/tikv/pd/tests/server/region_syncer github.com/tikv/pd/tests/server/storage github.com/tikv/pd/tests/server/watch github.com/tikv/pd/tools/pd-analysis/analysis github.com/tikv/pd/tools/pd-backup/pdbackup github.com/tikv/pd/tools/pd-ctl/pdctl github.com/tikv/pd/tools/pd-simulator/simulator/simutil || { $(find $PWD/ -type d | grep -vE "\.git" | xargs failpoint-ctl disable); exit 1; }
ok  	github.com/tikv/pd/pkg/audit	(cached)	coverage: 90.0% of statements
ok  	github.com/tikv/pd/pkg/autoscaling	(cached)	coverage: 46.0% of statements
ok  	github.com/tikv/pd/pkg/btree	(cached)	coverage: 90.3% of statements
ok  	github.com/tikv/pd/pkg/cache	(cached)	coverage: 70.8% of statements
--- FAIL: TestGetCgroupCPU (0.00s)
    cgroup_cpu_test.go:45: 
        	Error Trace:	/home/lhy1024/pd/pkg/cgroup/cgroup_cpu_test.go:45
        	Error:      	Received unexpected error:
        	            	no cpu controller detected
        	            	github.com/tikv/pd/pkg/cgroup.getCgroupCPUHelper
        	            		/home/lhy1024/pd/pkg/cgroup/cgroup_cpu.go:33
        	            	github.com/tikv/pd/pkg/cgroup.GetCgroupCPU
        	            		/home/lhy1024/pd/pkg/cgroup/cgroup_cpu_linux.go:28
        	            	github.com/tikv/pd/pkg/cgroup.TestGetCgroupCPU
        	            		/home/lhy1024/pd/pkg/cgroup/cgroup_cpu_test.go:44
        	            	testing.tRunner
        	            		/usr/local/go/src/testing/testing.go:1446
        	            	runtime.goexit
        	            		/usr/local/go/src/runtime/asm_amd64.s:1594
        	Test:       	TestGetCgroupCPU
[2023/02/09 19:17:12.326 +08:00] [WARN] [cgroup_memory.go:84] ["no cgroup memory controller detected"]
[2023/02/09 19:17:12.341 +08:00] [WARN] [cgroup_memory.go:55] ["no cgroup memory controller detected"]
[2023/02/09 19:17:12.357 +08:00] [WARN] [cgroup_memory.go:115] ["no cgroup memory controller detected"]
FAIL
coverage: 76.5% of statements
FAIL	github.com/tikv/pd/pkg/cgroup	0.121s
ok  	github.com/tikv/pd/pkg/codec	(cached)	coverage: 63.0% of statements
ok  	github.com/tikv/pd/pkg/core	(cached)	coverage: 39.4% of statements
ok  	github.com/tikv/pd/pkg/core/rangetree	(cached)	coverage: 58.5% of statements
ok  	github.com/tikv/pd/pkg/core/storelimit	(cached)	coverage: 72.2% of statements
ok  	github.com/tikv/pd/pkg/dashboard/adapter	(cached)	coverage: 22.1% of statements
ok  	github.com/tikv/pd/pkg/election	16.594s	coverage: 63.0% of statements
ok  	github.com/tikv/pd/pkg/encryption	18.303s	coverage: 73.6% of statements
ok  	github.com/tikv/pd/pkg/errs	(cached)	coverage: 85.7% of statements
ok  	github.com/tikv/pd/pkg/gctuner	(cached)	coverage: 76.9% of statements
ok  	github.com/tikv/pd/pkg/id	2.377s	coverage: 72.3% of statements
ok  	github.com/tikv/pd/pkg/mcs/discovery	8.148s	coverage: 63.3% of statements
ok  	github.com/tikv/pd/pkg/mcs/resource_manager/server	(cached)	coverage: 29.7% of statements
ok  	github.com/tikv/pd/pkg/memory	(cached)	coverage: 32.6% of statements [no tests to run]
ok  	github.com/tikv/pd/pkg/mock/mockhbstream	(cached)	coverage: 85.7% of statements
ok  	github.com/tikv/pd/pkg/movingaverage	(cached)	coverage: 88.8% of statements
ok  	github.com/tikv/pd/pkg/progress	(cached)	coverage: 88.5% of statements
ok  	github.com/tikv/pd/pkg/ratelimit	(cached)	coverage: 95.4% of statements
ok  	github.com/tikv/pd/pkg/slice	(cached)	coverage: 66.7% of statements
ok  	github.com/tikv/pd/pkg/storage	2.026s	coverage: 47.2% of statements
ok  	github.com/tikv/pd/pkg/storage/endpoint	(cached)	coverage: 3.2% of statements
ok  	github.com/tikv/pd/pkg/storage/kv	1.289s	coverage: 71.9% of statements
ok  	github.com/tikv/pd/pkg/systimemon	(cached)	coverage: 100.0% of statements
ok  	github.com/tikv/pd/pkg/utils/apiutil	(cached)	coverage: 20.5% of statements
ok  	github.com/tikv/pd/pkg/utils/assertutil	(cached)	coverage: 36.4% of statements
ok  	github.com/tikv/pd/pkg/utils/etcdutil	6.571s	coverage: 60.9% of statements
ok  	github.com/tikv/pd/pkg/utils/grpcutil	(cached)	coverage: 43.9% of statements
ok  	github.com/tikv/pd/pkg/utils/jsonutil	(cached)	coverage: 80.0% of statements
ok  	github.com/tikv/pd/pkg/utils/keyutil	(cached)	coverage: 14.3% of statements
ok  	github.com/tikv/pd/pkg/utils/logutil	(cached)	coverage: 64.0% of statements
ok  	github.com/tikv/pd/pkg/utils/metricutil	(cached)	coverage: 89.7% of statements
ok  	github.com/tikv/pd/pkg/utils/netutil	(cached)	coverage: 85.0% of statements
ok  	github.com/tikv/pd/pkg/utils/reflectutil	(cached)	coverage: 90.3% of statements
ok  	github.com/tikv/pd/pkg/utils/requestutil	(cached)	coverage: 30.0% of statements
ok  	github.com/tikv/pd/pkg/utils/syncutil	(cached)	coverage: 87.0% of statements
ok  	github.com/tikv/pd/pkg/utils/typeutil	(cached)	coverage: 77.1% of statements
ok  	github.com/tikv/pd/server	21.224s	coverage: 15.1% of statements
ok  	github.com/tikv/pd/server/api	217.952s	coverage: 63.2% of statements
ok  	github.com/tikv/pd/server/cluster	(cached)	coverage: 58.6% of statements
ok  	github.com/tikv/pd/server/config	(cached)	coverage: 47.3% of statements
ok  	github.com/tikv/pd/server/gc	(cached)	coverage: 95.8% of statements
ok  	github.com/tikv/pd/server/join	0.680s	coverage: 4.7% of statements
ok  	github.com/tikv/pd/server/keyspace	(cached)	coverage: 71.7% of statements
ok  	github.com/tikv/pd/server/region_syncer	(cached)	coverage: 29.7% of statements
ok  	github.com/tikv/pd/server/replication	(cached)	coverage: 79.3% of statements
ok  	github.com/tikv/pd/server/schedule	(cached)	coverage: 68.9% of statements
ok  	github.com/tikv/pd/server/schedule/checker	(cached)	coverage: 73.6% of statements
ok  	github.com/tikv/pd/server/schedule/filter	(cached)	coverage: 59.2% of statements
ok  	github.com/tikv/pd/server/schedule/labeler	(cached)	coverage: 86.6% of statements
ok  	github.com/tikv/pd/server/schedule/operator	(cached)	coverage: 72.2% of statements
ok  	github.com/tikv/pd/server/schedule/placement	(cached)	coverage: 77.2% of statements
ok  	github.com/tikv/pd/server/schedule/rangelist	(cached)	coverage: 100.0% of statements
ok  	github.com/tikv/pd/server/schedulers	73.113s	coverage: 69.4% of statements
ok  	github.com/tikv/pd/server/statistics	(cached)	coverage: 52.7% of statements
ok  	github.com/tikv/pd/server/statistics/buckets	(cached)	coverage: 94.0% of statements
ok  	github.com/tikv/pd/tests/autoscaling	3.801s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/compatibility	10.354s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/dashboard	22.906s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/pdbackup	3.327s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/pdctl	1.621s	coverage: 11.6% of statements
ok  	github.com/tikv/pd/tests/pdctl/cluster	1.943s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/pdctl/completion	(cached)	coverage: [no statements]
ok  	github.com/tikv/pd/tests/pdctl/config	16.250s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/pdctl/health	7.719s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/pdctl/hot	15.296s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/pdctl/label	3.490s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/pdctl/log	15.229s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/pdctl/member	11.877s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/pdctl/operator	4.696s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/pdctl/region	6.312s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/pdctl/scheduler	7.440s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/pdctl/store	11.774s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/pdctl/tso	(cached)	coverage: [no statements]
ok  	github.com/tikv/pd/tests/pdctl/unsafe	4.262s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/registry	3.142s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/server	44.518s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/server/api	97.398s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/server/apiv2/handlers	9.787s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/server/cluster	112.219s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/server/config	12.164s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/server/id	36.214s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/server/join	77.010s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/server/join/join_fail	4.888s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/server/keyspace	54.111s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/server/member	134.116s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/server/region_syncer	60.309s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/server/storage	12.981s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/server/watch	30.133s	coverage: [no statements]
ok  	github.com/tikv/pd/tools/pd-analysis/analysis	0.273s	coverage: 75.5% of statements
ok  	github.com/tikv/pd/tools/pd-backup/pdbackup	1.603s	coverage: 77.8% of statements
ok  	github.com/tikv/pd/tools/pd-ctl/pdctl	(cached)	coverage: 12.1% of statements
ok  	github.com/tikv/pd/tools/pd-simulator/simulator/simutil	(cached)	coverage: 85.5% of statements
FAIL

Signed-off-by: lhy1024 <admin@liudos.us>
Signed-off-by: lhy1024 <admin@liudos.us>
server/schedulers/utils.go Outdated Show resolved Hide resolved
Copy link
Member

@rleungx rleungx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rest LGTM

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Feb 13, 2023
Signed-off-by: lhy1024 <admin@liudos.us>
Signed-off-by: lhy1024 <admin@liudos.us>
@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Feb 14, 2023
@lhy1024
Copy link
Contributor Author

lhy1024 commented Feb 14, 2023

/merge

@ti-chi-bot
Copy link
Member

@lhy1024: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 9468cb6

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Feb 14, 2023
@ti-chi-bot
Copy link
Member

@lhy1024: Your PR was out of date, I have automatically updated it for you.

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot ti-chi-bot merged commit 58dbc3f into tikv:master Feb 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-note-none status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

scheduler: move init function
4 participants