-
Notifications
You must be signed in to change notification settings - Fork 24
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Add basic testing large syncs support #1862
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think scheduler_fuzz
is a good name for this. I did the exercise of looking at the code without looking at the issue first, and it made no sense to me what the point of this scheduler was 🤔
It's more of a load test scheduler 🤔 It doesn't do fuzzing.
I think it would pay to add a small comment in the constructor of what problem it solves.
💯 Happy to rename it. I don't like the fuzz name as well (though it might do some fuzzing in the future). I'll rename |
52c560e
to
19bd9fa
Compare
Done in 08bb4a0 |
🤖 I have created a release *beep* *boop* --- ## [4.59.0](v4.58.1...v4.59.0) (2024-08-08) ### Features * Add basic testing large syncs support ([#1862](#1862)) ([40a0095](40a0095)) ### Bug Fixes * **deps:** Update module github.com/cloudquery/cloudquery-api-go to v1.12.7 ([#1860](#1860)) ([25ed3d2](25ed3d2)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
Summary
A very crude and simple way to do https://github.com/cloudquery/cloudquery-issues/issues/1846.
Adds a new hidden
fuzztest scheduler that only multiplies the clients (at the moment).The code is based on the shuffle scheduler then adds then duplicates client based on the multiplier.
See example:
This has a couple of downsides/tradeoffs
plugin-sdk/scheduler/metrics.go
Line 144 in 25ed3d2
However I think this is still useful if we want to artificially make a sync large (e.g. simulate a sync on many AWS accounts)
Use the following steps to ensure your PR is ready to be reviewed
go fmt
to format your code 🖊golangci-lint run
🚨 (install golangci-lint here)