-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Benchmark parameters for Citrine runs #78
Comments
I would cut few of the thread counts to reduce the number of runs. t=4,6,8,10,12,13,14,15,16,17,18,20,22,24,26,28,30 |
BaselineI consider this the baseline:
Benchmarks focused on batchingbatch by deferring receives
this should perform worse than batch receives on poll thread
continue inline without batchingIt is also interesting to run the continuations inline, without batching, to differentiate between effect of batching and continuation.
note: iouring doesn't have a mode that disables batching. Benchmarks focused on scheduling
I'm not sure if we should include For
|
I think the point is to have a proper comparison between AIO / no AIO.
Doesn't this affect only ThreadPool schedulers? Isn't it waste of time to run all the rest? |
I consider this one interesting too. It's important to note that
For max performance, you should also set |
IoQueue is also a ThreadPool scheduler. Maybe these could be left out for
|
@adamsitnik @antonfirsov do we want to run middleware json/platform json (#32)? I'm fine using middleware, but maybe you have a specific preference for platform? Note that the pipelined plaintext will suffer from |
I think that we should use middleware and compare it with current middleware implementation which is around 750k RPS (link to PowerBI) |
@tmds @adamsitnik you can check the results here: The grouping should be straightforward, but if it's not I'm happy to answer questions. On several places, there are multiple versions of the same diagram with different series enabled/disabled. Red lines in table are for missing or outlier data. @tmds does this help getting insights? Is there anything unexpected to you? Anything else we should run? |
@tmds as we discussed, I extended the ThreadPool scheduling benchmarks with |
Thanks Anton! The effect being mostly being at lower |
@antonfirsov this is the combination we discussed that would be interesting also to benchmark on Citrine:
|
Anton, can you also run these benchmarks?
|
Also sharing the doc here: @tmds added the 2 graphs you requested |
Thank you Anton! |
We're missing
It's an interesting point on the graph (should be best for |
Let's define the parameters we want to use for extensive Citrine runs (or at least for the first one).
The machines are mine for Friday (big thanks to Sebastien!), but my naive approach to try all combinations of the major parameters is defining too many jobs, even for a whole day run. Are some of these combinations worthless to include even in a comprehensive analysis?
I'm using the syntax of my new tool for #74 to define the benchmakrs
DefaultTransport to get a baseline:
LinuxTransport for our information:
epoll combinations
Normally I would run epoll with all possible combinations of important parameters, but the following definition would mean ~500 benchmark executions. Would be nice to reduce it.
ThreadPool with
COMPlus_ThreadPool_UnfairSemaphoreSpinLimit=0
I would set
c=true
here:io_uring combinations
@tmds @adamsitnik anything missing? Which combos should I cut off?
The text was updated successfully, but these errors were encountered: