-
Notifications
You must be signed in to change notification settings - Fork 477
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(core/bench): add fs, monoiofs and compfs benchmark #5095
base: main
Are you sure you want to change the base?
Conversation
Thank you very much for your work! I have noticed that Monoio performs well on single reads but not as effectively on concurrent ones. Could this be related to our thread-per-core design? Are there any plans to improve it? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about calling this benchmark fs_alike
? By the way, why do we need separate benchmark suites?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This benchmark puts services in a group, criterion can generate summary graphs of benchmarks that shows difference between them.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm thinking of designing a way to benchmark multiple services, like OPENDAL_TEST=fs,monoiofs,compfs
. But it looks like requires some refactor to our current test infrastructure.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm thinking of designing a way to benchmark multiple services, like
OPENDAL_TEST=fs,monoiofs,compfs
. But it looks like requires some refactor to our current test infrastructure.
Yep, this makes more sense to me.
@@ -0,0 +1,13 @@ | |||
# OpenDAL services fs vs. monoiofs vs. compfs |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could support more FS interfaces.
|
||
This benchmark compares the performance of OpenDAL services fs, monoiofs and compfs. | ||
|
||
## Goal |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi, this isn't the benchmark goal. We shouldn't make assumptions about it. In fact, monoiofs can't outperform tokio-based fs in every case.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In fact, monoiofs can't outperform tokio-based fs in every case.
Actually, monoiofs is expected to be slower since it is currently single threaded. I'm trying to express the final goal we wish to achieve. Did I misunderstood the point of this section?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did I misunderstood the point of this section?
Our goal is to measure the performance of various fs services so users can choose for themselves, rather than trying to prove that monoiofs is faster than tokio fs. Please note that OpenDAL does not declare a winner; we simply offer options.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for pointing out that!
Monoiofs is currently single threaded ( |
Also, PositionWrite is not implemented for monoiofs. |
Part of #4552.
This PR introduces a benchmark that compares the performance of OpenDAL services fs, monoiofs (and compfs, but unfortunately commented out since it is still a work in progress and did not finish the benchmark).
Concurrent benchmarks uses
.chunk(size).concurrent(parallel)
rather than polling several independent io tasks (which is the waybench/ops
benchmarks). Not sure which one simulates real world better.Full benchmark result is as follows. I'm going to write a progress report of monoiofs along with a brief analysis of the result on the mailing list, so stay tuned. 😋
Benchmark result