Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Remote Store] Performance test setup for 2.3 release #3148

Closed
sachinpkale opened this issue May 3, 2022 · 5 comments
Closed

[Remote Store] Performance test setup for 2.3 release #3148

sachinpkale opened this issue May 3, 2022 · 5 comments
Labels
distributed framework enhancement Enhancement or improvement to existing feature or request Storage:Durability Issues and PRs related to the durability framework

Comments

@sachinpkale
Copy link
Member

Describe the solution you'd like
A performance test setup integrated with opensearch-benchmark which will run and report performance numbers on each commit. This will enable us to catch any performance regressions up-front.
@sachinpkale sachinpkale added enhancement Enhancement or improvement to existing feature or request untriaged labels May 3, 2022
This was referenced May 3, 2022
Open
Merged
@kartg kartg added distributed framework Storage:Durability Issues and PRs related to the durability framework and removed untriaged labels May 3, 2022
@sachinpkale
Copy link
Member Author

Setup

  • Tool used: opensearch-benchmark
  • OpenSearch Benchmark Test: nyc_taxis
  • Cluster
    • 3 node cluster with each node - r5.2xlarge
  • Benchmark node - r5.2xlarge
  • Non-optimized implementation of Remote Segment Store
  • Remote translog is not supported
  • GitHub Commit ID: sachinpkale@6c11fd6

Observations

  • Indexing latency and throughput are impacted
  • The quick first change would be to cache list of files in remote segment store in memory
Metric Task Without Remote Storage With Remote Storage Diff Unit
Cumulative indexing time of primary shards   254.623 299.414 44.7913 min
Min cumulative indexing time across primary shard   254.623 299.414 44.7913 min
Median cumulative indexing time across primary shard   254.623 299.414 44.7913 min
Max cumulative indexing time across primary shard   254.623 299.414 44.7913 min
Cumulative indexing throttle time of primary shards   0 0 0 min
Min cumulative indexing throttle time across primary shard   0 0 0 min
Median cumulative indexing throttle time across primary shard   0 0 0 min
Max cumulative indexing throttle time across primary shard   0 0 0 min
Cumulative merge time of primary shards   95.0004 106.406 11.4051 min
Cumulative merge count of primary shards   221 201 -20  
Min cumulative merge time across primary shard   95.0004 106.406 11.4051 min
Median cumulative merge time across primary shard   95.0004 106.406 11.4051 min
Max cumulative merge time across primary shard   95.0004 106.406 11.4051 min
Cumulative merge throttle time of primary shards   3.72203 4.29547 0.57343 min
Min cumulative merge throttle time across primary shard   3.72203 4.29547 0.57343 min
Median cumulative merge throttle time across primary shard   3.72203 4.29547 0.57343 min
Max cumulative merge throttle time across primary shard   3.72203 4.29547 0.57343 min
Cumulative refresh time of primary shards   2.32588 1.73848 -0.5874 min
Cumulative refresh count of primary shards   119 84 -35  
Min cumulative refresh time across primary shard   2.32588 1.73848 -0.5874 min
Median cumulative refresh time across primary shard   2.32588 1.73848 -0.5874 min
Max cumulative refresh time across primary shard   2.32588 1.73848 -0.5874 min
Cumulative flush time of primary shards   5.48385 18.8994 13.4155 min
Cumulative flush count of primary shards   31 29 -2  
Min cumulative flush time across primary shard   5.48385 18.8994 13.4155 min
Median cumulative flush time across primary shard   5.48385 18.8994 13.4155 min
Max cumulative flush time across primary shard   5.48385 18.8994 13.4155 min
Total Young Gen GC time   190.253 199.536 9.283 s
Total Young Gen GC count   20376 21046 670  
Total Old Gen GC time   0 0 0 s
Total Old Gen GC count   0 0 0  
Store size   23.4558 23.4502 -0.00567 GB
Translog size   5.12E-08 5.12E-08 0 GB
Heap used for segments   0 0 0 MB
Heap used for doc values   0 0 0 MB
Heap used for terms   0 0 0 MB
Heap used for norms   0 0 0 MB
Heap used for points   0 0 0 MB
Heap used for stored fields   0 0 0 MB
Segment count   27 27 0  
Min Throughput index 66289.9 58847.9 -7442.06 docs/s
Mean Throughput index 69768.5 61950.5 -7818.04 docs/s
Median Throughput index 68763 61035.3 -7727.71 docs/s
Max Throughput index 75814.3 69285.9 -6528.36 docs/s
50th percentile latency index 1032.17 1190.8 158.637 ms
90th percentile latency index 1608.38 1801.82 193.443 ms
99th percentile latency index 2900.14 3346.61 446.468 ms
99.9th percentile latency index 4062.04 5071.4 1009.36 ms
99.99th percentile latency index 5209.35 5894.15 684.797 ms
100th percentile latency index 6027.82 6071.15 43.3313 ms
50th percentile service time index 1032.17 1190.8 158.637 ms
90th percentile service time index 1608.38 1801.82 193.443 ms
99th percentile service time index 2900.14 3346.61 446.468 ms
99.9th percentile service time index 4062.04 5071.4 1009.36 ms
99.99th percentile service time index 5209.35 5894.15 684.797 ms
100th percentile service time index 6027.82 6071.15 43.3313 ms
error rate index 0 0 0 %
Min Throughput wait-until-merges-finish 0.00676 0.00766 0.00089 ops/s
Mean Throughput wait-until-merges-finish 0.00676 0.00766 0.00089 ops/s
Median Throughput wait-until-merges-finish 0.00676 0.00766 0.00089 ops/s
Max Throughput wait-until-merges-finish 0.00676 0.00766 0.00089 ops/s
100th percentile latency wait-until-merges-finish 147827 130610 -17216.5 ms
100th percentile service time wait-until-merges-finish 147827 130610 -17216.5 ms
error rate wait-until-merges-finish 0 0 0 %
Min Throughput default 3.01439 3.01761 0.00322 ops/s
Mean Throughput default 3.02343 3.02867 0.00525 ops/s
Median Throughput default 3.02131 3.02614 0.00483 ops/s
Max Throughput default 3.04132 3.0506 0.00928 ops/s
50th percentile latency default 6.64658 6.5142 -0.13238 ms
90th percentile latency default 7.39876 7.28572 -0.11304 ms
99th percentile latency default 7.85555 7.86881 0.01326 ms
100th percentile latency default 7.88995 9.53959 1.64964 ms
50th percentile service time default 5.54675 5.48905 -0.0577 ms
90th percentile service time default 5.97729 5.96132 -0.01597 ms
99th percentile service time default 6.52644 6.41277 -0.11366 ms
100th percentile service time default 7.12074 8.73328 1.61253 ms
error rate default 0 0 0 %
Min Throughput range 0.70218 0.70223 5.00E-05 ops/s
Mean Throughput range 0.70358 0.70366 7.00E-05 ops/s
Median Throughput range 0.70326 0.70333 6.00E-05 ops/s
Max Throughput range 0.70646 0.70659 0.00013 ops/s
50th percentile latency range 632.281 652.068 19.7872 ms
90th percentile latency range 634.706 659.184 24.4778 ms
99th percentile latency range 639.423 661.863 22.4397 ms
100th percentile latency range 642.279 700.012 57.7326 ms
50th percentile service time range 630.697 650.13 19.4331 ms
90th percentile service time range 632.97 657.669 24.6985 ms
99th percentile service time range 637.634 659.641 22.0069 ms
100th percentile service time range 640.541 698.549 58.0088 ms
error rate range 0 0 0 %
Min Throughput distance_amount_agg 2.0121 2.01191 -0.00019 ops/s
Mean Throughput distance_amount_agg 2.0199 2.0196 -0.0003 ops/s
Median Throughput distance_amount_agg 2.01807 2.0178 -0.00026 ops/s
Max Throughput distance_amount_agg 2.03572 2.03525 -0.00047 ops/s
50th percentile latency distance_amount_agg 5.32663 5.26678 -0.05985 ms
90th percentile latency distance_amount_agg 6.02059 5.69067 -0.32991 ms
99th percentile latency distance_amount_agg 6.59205 6.07883 -0.51321 ms
100th percentile latency distance_amount_agg 6.89191 6.35985 -0.53206 ms
50th percentile service time distance_amount_agg 4.11577 3.8171 -0.29868 ms
90th percentile service time distance_amount_agg 4.61674 4.2864 -0.33034 ms
99th percentile service time distance_amount_agg 5.11496 4.51019 -0.60477 ms
100th percentile service time distance_amount_agg 5.71144 4.7327 -0.97874 ms
error rate distance_amount_agg 0 0 0 %
Min Throughput autohisto_agg 1.49802 1.49609 -0.00193 ops/s
Mean Throughput autohisto_agg 1.49889 1.49782 -0.00107 ops/s
Median Throughput autohisto_agg 1.49899 1.49802 -0.00098 ops/s
Max Throughput autohisto_agg 1.49933 1.49866 -0.00067 ops/s
50th percentile latency autohisto_agg 495.737 493.814 -1.92287 ms
90th percentile latency autohisto_agg 505.387 503.649 -1.73756 ms
99th percentile latency autohisto_agg 516.338 508.96 -7.37807 ms
100th percentile latency autohisto_agg 516.401 509.453 -6.94745 ms
50th percentile service time autohisto_agg 494.473 492.376 -2.09646 ms
90th percentile service time autohisto_agg 504.339 502.783 -1.55646 ms
99th percentile service time autohisto_agg 514.02 507.674 -6.34651 ms
100th percentile service time autohisto_agg 515.307 508.029 -7.27823 ms
error rate autohisto_agg 0 0 0 %
Min Throughput date_histogram_agg 1.5019 1.50175 -0.00015 ops/s
Mean Throughput date_histogram_agg 1.50309 1.50284 -0.00025 ops/s
Median Throughput date_histogram_agg 1.50283 1.5026 -0.00023 ops/s
Max Throughput date_histogram_agg 1.50549 1.50505 -0.00044 ops/s
50th percentile latency date_histogram_agg 545.45 529.173 -16.2769 ms
90th percentile latency date_histogram_agg 566.344 543.815 -22.5293 ms
99th percentile latency date_histogram_agg 576.84 561.054 -15.7868 ms
100th percentile latency date_histogram_agg 577.66 564.438 -13.2222 ms
50th percentile service time date_histogram_agg 544.481 528.019 -16.4615 ms
90th percentile service time date_histogram_agg 565.533 542.919 -22.614 ms
99th percentile service time date_histogram_agg 575.487 559.642 -15.8453 ms
100th percentile service time date_histogram_agg 577.184 563.161 -14.0223 ms
error rate date_histogram_agg 0 0 0 %

@sachinpkale
Copy link
Member Author

sachinpkale commented May 24, 2022 •
edited
Loading

Setup

  • Tool used: opensearch-benchmark
  • OpenSearch Benchmark Test: nyc_taxis
  • Cluster
    • 3 node cluster with each node - r5.2xlarge
  • Benchmark node - r5.2xlarge
  • Non-optimized implementation of Remote Segment Store
  • Remote translog is not supported
  • GitHub Commit ID: sachinpkale@e202720

Observations

  • Indexing performance improved after caching file list from remote store in memory.
  • Deleting merged away segment files would still be impacting the performance. Deleting files in async flow should improve the performance further.
Metric Task Baseline Contender Diff Unit
Cumulative indexing time of primary shards 253.075 296.512 43.4374 min
Min cumulative indexing time across primary shard 253.075 296.512 43.4374 min
Median cumulative indexing time across primary shard 253.075 296.512 43.4374 min
Max cumulative indexing time across primary shard 253.075 296.512 43.4374 min
Cumulative indexing throttle time of primary shards 0 0 0 min
Min cumulative indexing throttle time across primary shard 0 0 0 min
Median cumulative indexing throttle time across primary shard 0 0 0 min
Max cumulative indexing throttle time across primary shard 0 0 0 min
Cumulative merge time of primary shards 96.9922 106.939 9.94725 min
Cumulative merge count of primary shards 220 209 -11
Min cumulative merge time across primary shard 96.9922 106.939 9.94725 min
Median cumulative merge time across primary shard 96.9922 106.939 9.94725 min
Max cumulative merge time across primary shard 96.9922 106.939 9.94725 min
Cumulative merge throttle time of primary shards 2.98208 3.13542 0.15333 min
Min cumulative merge throttle time across primary shard 2.98208 3.13542 0.15333 min
Median cumulative merge throttle time across primary shard 2.98208 3.13542 0.15333 min
Max cumulative merge throttle time across primary shard 2.98208 3.13542 0.15333 min
Cumulative refresh time of primary shards 2.20817 1.89697 -0.3112 min
Cumulative refresh count of primary shards 115 89 -26
Min cumulative refresh time across primary shard 2.20817 1.89697 -0.3112 min
Median cumulative refresh time across primary shard 2.20817 1.89697 -0.3112 min
Max cumulative refresh time across primary shard 2.20817 1.89697 -0.3112 min
Cumulative flush time of primary shards 5.76538 16.8177 11.0523 min
Cumulative flush count of primary shards 29 31 2
Min cumulative flush time across primary shard 5.76538 16.8177 11.0523 min
Median cumulative flush time across primary shard 5.76538 16.8177 11.0523 min
Max cumulative flush time across primary shard 5.76538 16.8177 11.0523 min
Total Young Gen GC time 159.031 166.386 7.355 s
Total Young Gen GC count 20326 20779 453
Total Old Gen GC time 0 0 0 s
Total Old Gen GC count 0 0 0
Store size 24.1613 24.2019 0.04053 GB
Translog size 5.12227e-08 5.12227e-08 0 GB
Heap used for segments 0 0 0 MB
Heap used for doc values 0 0 0 MB
Heap used for terms 0 0 0 MB
Heap used for norms 0 0 0 MB
Heap used for points 0 0 0 MB
Heap used for stored fields 0 0 0 MB
Segment count 28 35 7
Min Throughput index 66079.6 61734.6 -4345.04 docs/s
Mean Throughput index 69666.3 64964.3 -4702.02 docs/s
Median Throughput index 69013.3 64018.2 -4995.13 docs/s
Max Throughput index 76897.9 71298.7 -5599.25 docs/s
50th percentile latency index 1033.19 1129.71 96.5144 ms
90th percentile latency index 1667.44 1482.01 -185.43 ms
99th percentile latency index 2896.64 3201.48 304.845 ms
99.9th percentile latency index 4122.8 4927.24 804.441 ms
99.99th percentile latency index 5132.79 6154.82 1022.03 ms
100th percentile latency index 5735.43 6515.6 780.167 ms
50th percentile service time index 1033.19 1129.71 96.5144 ms
90th percentile service time index 1667.44 1482.01 -185.43 ms
99th percentile service time index 2896.64 3201.48 304.845 ms
99.9th percentile service time index 4122.8 4927.24 804.441 ms
99.99th percentile service time index 5132.79 6154.82 1022.03 ms
100th percentile service time index 5735.43 6515.6 780.167 ms
error rate index 0 0 0 %
Min Throughput wait-until-merges-finish 0.0150139 0.00747963 -0.00753 ops/s
Mean Throughput wait-until-merges-finish 0.0150139 0.00747963 -0.00753 ops/s
Median Throughput wait-until-merges-finish 0.0150139 0.00747963 -0.00753 ops/s
Max Throughput wait-until-merges-finish 0.0150139 0.00747963 -0.00753 ops/s
100th percentile latency wait-until-merges-finish 66604.5 133696 67091.4 ms
100th percentile service time wait-until-merges-finish 66604.5 133696 67091.4 ms
error rate wait-until-merges-finish 0 0 0 %
Min Throughput default 3.01679 3.01453 -0.00226 ops/s
Mean Throughput default 3.02734 3.0237 -0.00365 ops/s
Median Throughput default 3.02487 3.02161 -0.00325 ops/s
Max Throughput default 3.04834 3.04171 -0.00663 ops/s
50th percentile latency default 6.12262 6.66216 0.53954 ms
90th percentile latency default 6.86559 7.45249 0.5869 ms
99th percentile latency default 7.24896 8.57816 1.32919 ms
100th percentile latency default 8.1294 9.3935 1.2641 ms
50th percentile service time default 5.00421 5.56704 0.56283 ms
90th percentile service time default 5.49765 6.28737 0.78972 ms
99th percentile service time default 5.69171 7.81069 2.11898 ms
100th percentile service time default 6.52109 8.65448 2.13339 ms
error rate default 0 0 0 %
Min Throughput range 0.703279 0.703397 0.00012 ops/s
Mean Throughput range 0.705392 0.705586 0.00019 ops/s
Median Throughput range 0.704907 0.705082 0.00018 ops/s
Max Throughput range 0.709732 0.710085 0.00035 ops/s
50th percentile latency range 214.534 220.999 6.46424 ms
90th percentile latency range 217.882 224.429 6.54726 ms
99th percentile latency range 224.645 230.943 6.29858 ms
100th percentile latency range 226.75 242.2 15.4494 ms
50th percentile service time range 212.492 218.8 6.30862 ms
90th percentile service time range 215.803 222.259 6.45583 ms
99th percentile service time range 222.555 229.087 6.53183 ms
100th percentile service time range 224.196 240.289 16.0937 ms
error rate range 0 0 0 %
Min Throughput distance_amount_agg 2.01198 2.01214 0.00015 ops/s
Mean Throughput distance_amount_agg 2.01971 2.01998 0.00027 ops/s
Median Throughput distance_amount_agg 2.01793 2.01817 0.00024 ops/s
Max Throughput distance_amount_agg 2.03539 2.03587 0.00048 ops/s
50th percentile latency distance_amount_agg 5.03569 4.91612 -0.11957 ms
90th percentile latency distance_amount_agg 5.71255 5.54775 -0.1648 ms
99th percentile latency distance_amount_agg 5.93613 5.78433 -0.1518 ms
100th percentile latency distance_amount_agg 6.25026 5.96179 -0.28846 ms
50th percentile service time distance_amount_agg 3.81376 3.71955 -0.0942 ms
90th percentile service time distance_amount_agg 4.10394 4.03015 -0.07378 ms
99th percentile service time distance_amount_agg 4.54176 4.32433 -0.21743 ms
100th percentile service time distance_amount_agg 4.63936 4.44204 -0.19731 ms
error rate distance_amount_agg 0 0 0 %
Min Throughput autohisto_agg 1.50038 1.49878 -0.0016 ops/s
Mean Throughput autohisto_agg 1.50062 1.49932 -0.0013 ops/s
Median Throughput autohisto_agg 1.50057 1.49938 -0.00119 ops/s
Max Throughput autohisto_agg 1.5011 1.49958 -0.00151 ops/s
50th percentile latency autohisto_agg 485.995 489.358 3.36348 ms
90th percentile latency autohisto_agg 495.982 497.994 2.01226 ms
99th percentile latency autohisto_agg 509.317 506.771 -2.5464 ms
100th percentile latency autohisto_agg 531.214 512.137 -19.0766 ms
50th percentile service time autohisto_agg 484.821 488.241 3.42085 ms
90th percentile service time autohisto_agg 495.449 497.329 1.8802 ms
99th percentile service time autohisto_agg 508.706 505.455 -3.25075 ms
100th percentile service time autohisto_agg 529.869 511.489 -18.38 ms
error rate autohisto_agg 0 0 0 %
Min Throughput date_histogram_agg 1.502 1.50179 -0.00022 ops/s
Mean Throughput date_histogram_agg 1.50324 1.50291 -0.00034 ops/s
Median Throughput date_histogram_agg 1.50296 1.50265 -0.00031 ops/s
Max Throughput date_histogram_agg 1.50573 1.50515 -0.00058 ops/s
50th percentile latency date_histogram_agg 509.46 531.019 21.5592 ms
90th percentile latency date_histogram_agg 518.483 540.611 22.1279 ms
99th percentile latency date_histogram_agg 524.308 551.966 27.658 ms
100th percentile latency date_histogram_agg 528.937 551.975 23.0384 ms
50th percentile service time date_histogram_agg 508.487 530.154 21.6669 ms
90th percentile service time date_histogram_agg 517.694 539.466 21.772 ms
99th percentile service time date_histogram_agg 523.51 549.977 26.4667 ms
100th percentile service time date_histogram_agg 528.18 550.989 22.8089 ms
error rate date_histogram_agg 0 0 0 %

@sachinpkale
Copy link
Member Author

Setup

  • Tool used: opensearch-benchmark
  • OpenSearch Benchmark Test: nyc_taxis
  • Cluster
    • 3 node cluster with each node - r5.2xlarge
  • Benchmark node - r5.2xlarge
  • Non-optimized implementation of Remote Segment Store
  • Remote translog is not supported
  • GitHub Commit ID: sachinpkale@f0b87be
Metric Task Baseline Contender Diff Unit
Cumulative indexing time of primary shards 253.075 260.082 7.00707 min
Min cumulative indexing time across primary shard 253.075 260.082 7.00707 min
Median cumulative indexing time across primary shard 253.075 260.082 7.00707 min
Max cumulative indexing time across primary shard 253.075 260.082 7.00707 min
Cumulative indexing throttle time of primary shards 0 0 0 min
Min cumulative indexing throttle time across primary shard 0 0 0 min
Median cumulative indexing throttle time across primary shard 0 0 0 min
Max cumulative indexing throttle time across primary shard 0 0 0 min
Cumulative merge time of primary shards 96.9922 94.8396 -2.15252 min
Cumulative merge count of primary shards 220 199 -21
Min cumulative merge time across primary shard 96.9922 94.8396 -2.15252 min
Median cumulative merge time across primary shard 96.9922 94.8396 -2.15252 min
Max cumulative merge time across primary shard 96.9922 94.8396 -2.15252 min
Cumulative merge throttle time of primary shards 2.98208 4.35395 1.37187 min
Min cumulative merge throttle time across primary shard 2.98208 4.35395 1.37187 min
Median cumulative merge throttle time across primary shard 2.98208 4.35395 1.37187 min
Max cumulative merge throttle time across primary shard 2.98208 4.35395 1.37187 min
Cumulative refresh time of primary shards 2.20817 1.7818 -0.42637 min
Cumulative refresh count of primary shards 115 85 -30
Min cumulative refresh time across primary shard 2.20817 1.7818 -0.42637 min
Median cumulative refresh time across primary shard 2.20817 1.7818 -0.42637 min
Max cumulative refresh time across primary shard 2.20817 1.7818 -0.42637 min
Cumulative flush time of primary shards 5.76538 19.56 13.7946 min
Cumulative flush count of primary shards 29 30 1
Min cumulative flush time across primary shard 5.76538 19.56 13.7946 min
Median cumulative flush time across primary shard 5.76538 19.56 13.7946 min
Max cumulative flush time across primary shard 5.76538 19.56 13.7946 min
Total Young Gen GC time 159.031 162.47 3.439 s
Total Young Gen GC count 20326 20650 324
Total Old Gen GC time 0 0 0 s
Total Old Gen GC count 0 0 0
Store size 24.1613 24.1172 -0.04412 GB
Translog size 5.12227e-08 5.12227e-08 0 GB
Heap used for segments 0 0 0 MB
Heap used for doc values 0 0 0 MB
Heap used for terms 0 0 0 MB
Heap used for norms 0 0 0 MB
Heap used for points 0 0 0 MB
Heap used for stored fields 0 0 0 MB
Segment count 28 33 5
Min Throughput index 66079.6 65194.3 -885.307 docs/s
Mean Throughput index 69666.3 68502.7 -1163.65 docs/s
Median Throughput index 69013.3 68155.4 -857.93 docs/s
Max Throughput index 76897.9 74238.5 -2659.46 docs/s
50th percentile latency index 1033.19 1051.14 17.9466 ms
90th percentile latency index 1667.44 1580.05 -87.3923 ms
99th percentile latency index 2896.64 2965.8 69.1632 ms
99.9th percentile latency index 4122.8 4369.49 246.69 ms
99.99th percentile latency index 5132.79 4981.3 -151.49 ms
100th percentile latency index 5735.43 4995.51 -739.921 ms
50th percentile service time index 1033.19 1051.14 17.9466 ms
90th percentile service time index 1667.44 1580.05 -87.3923 ms
99th percentile service time index 2896.64 2965.8 69.1632 ms
99.9th percentile service time index 4122.8 4369.49 246.69 ms
99.99th percentile service time index 5132.79 4981.3 -151.49 ms
100th percentile service time index 5735.43 4995.51 -739.921 ms
error rate index 0 0 0 %
Min Throughput wait-until-merges-finish 0.0150139 0.0127025 -0.00231 ops/s
Mean Throughput wait-until-merges-finish 0.0150139 0.0127025 -0.00231 ops/s
Median Throughput wait-until-merges-finish 0.0150139 0.0127025 -0.00231 ops/s
Max Throughput wait-until-merges-finish 0.0150139 0.0127025 -0.00231 ops/s
100th percentile latency wait-until-merges-finish 66604.5 78724.1 12119.6 ms
100th percentile service time wait-until-merges-finish 66604.5 78724.1 12119.6 ms
error rate wait-until-merges-finish 0 0 0 %
Min Throughput default 3.01679 3.01471 -0.00208 ops/s
Mean Throughput default 3.02734 3.02397 -0.00337 ops/s
Median Throughput default 3.02487 3.02186 -0.00301 ops/s
Max Throughput default 3.04834 3.04224 -0.0061 ops/s
50th percentile latency default 6.12262 6.58541 0.46279 ms
90th percentile latency default 6.86559 7.29519 0.4296 ms
99th percentile latency default 7.24896 8.20452 0.95556 ms
100th percentile latency default 8.1294 23.4195 15.2901 ms
50th percentile service time default 5.00421 5.48601 0.4818 ms
90th percentile service time default 5.49765 5.94323 0.44558 ms
99th percentile service time default 5.69171 6.81174 1.12004 ms
100th percentile service time default 6.52109 22.3202 15.7991 ms
error rate default 0 0 0 %
Min Throughput range 0.703279 0.703563 0.00028 ops/s
Mean Throughput range 0.705392 0.705859 0.00047 ops/s
Median Throughput range 0.704907 0.705331 0.00042 ops/s
Max Throughput range 0.709732 0.710582 0.00085 ops/s
50th percentile latency range 214.534 212.203 -2.33104 ms
90th percentile latency range 217.882 216.04 -1.84187 ms
99th percentile latency range 224.645 228.639 3.99436 ms
100th percentile latency range 226.75 234.584 7.83404 ms
50th percentile service time range 212.492 210.379 -2.11235 ms
90th percentile service time range 215.803 214.227 -1.57559 ms
99th percentile service time range 222.555 226.6 4.04481 ms
100th percentile service time range 224.196 232.426 8.23077 ms
error rate range 0 0 0 %
Min Throughput distance_amount_agg 2.01198 2.01214 0.00016 ops/s
Mean Throughput distance_amount_agg 2.01971 2.01997 0.00026 ops/s
Median Throughput distance_amount_agg 2.01793 2.01815 0.00022 ops/s
Max Throughput distance_amount_agg 2.03539 2.03593 0.00054 ops/s
50th percentile latency distance_amount_agg 5.03569 5.24919 0.2135 ms
90th percentile latency distance_amount_agg 5.71255 5.81049 0.09794 ms
99th percentile latency distance_amount_agg 5.93613 6.19458 0.25845 ms
100th percentile latency distance_amount_agg 6.25026 6.57474 0.32449 ms
50th percentile service time distance_amount_agg 3.81376 3.91622 0.10246 ms
90th percentile service time distance_amount_agg 4.10394 4.30092 0.19698 ms
99th percentile service time distance_amount_agg 4.54176 4.54479 0.00303 ms
100th percentile service time distance_amount_agg 4.63936 4.7822 0.14285 ms
error rate distance_amount_agg 0 0 0 %
Min Throughput autohisto_agg 1.50038 1.5001 -0.00028 ops/s
Mean Throughput autohisto_agg 1.50062 1.50016 -0.00046 ops/s
Median Throughput autohisto_agg 1.50057 1.50015 -0.00043 ops/s
Max Throughput autohisto_agg 1.5011 1.50029 -0.00081 ops/s
50th percentile latency autohisto_agg 485.995 470.427 -15.5683 ms
90th percentile latency autohisto_agg 495.982 478.445 -17.5367 ms
99th percentile latency autohisto_agg 509.317 490.694 -18.6232 ms
100th percentile latency autohisto_agg 531.214 490.75 -40.4635 ms
50th percentile service time autohisto_agg 484.821 469.195 -15.6257 ms
90th percentile service time autohisto_agg 495.449 477.491 -17.9581 ms
99th percentile service time autohisto_agg 508.706 489.195 -19.5114 ms
100th percentile service time autohisto_agg 529.869 489.813 -40.0559 ms
error rate autohisto_agg 0 0 0 %
Min Throughput date_histogram_agg 1.502 1.50224 0.00024 ops/s
Mean Throughput date_histogram_agg 1.50324 1.50364 0.00039 ops/s
Median Throughput date_histogram_agg 1.50296 1.50332 0.00036 ops/s
Max Throughput date_histogram_agg 1.50573 1.50644 0.00071 ops/s
50th percentile latency date_histogram_agg 509.46 496.996 -12.4634 ms
90th percentile latency date_histogram_agg 518.483 506.363 -12.1203 ms
99th percentile latency date_histogram_agg 524.308 521.004 -3.30358 ms
100th percentile latency date_histogram_agg 528.937 530.718 1.78078 ms
50th percentile service time date_histogram_agg 508.487 495.929 -12.5581 ms
90th percentile service time date_histogram_agg 517.694 505.394 -12.2998 ms
99th percentile service time date_histogram_agg 523.51 520.072 -3.43827 ms
100th percentile service time date_histogram_agg 528.18 529.666 1.48557 ms
error rate date_histogram_agg 0 0 0 %

@sachinpkale
Copy link
Member Author

Setup

  • Tool used: opensearch-benchmark
  • OpenSearch Benchmark Test: nyc_taxis
  • Cluster
    • 3 node cluster with each node - r5.2xlarge
  • Benchmark node - r5.2xlarge
  • Branch - 2.x
  • GitHub Commit ID: 32258cf9cf960e227db3e8b95cb4059caf29d495
Metric Task Baseline Contender Diff Unit
Cumulative indexing time of primary shards 275.189 286.681 11.4926 min
Min cumulative indexing time across primary shard 275.189 286.681 11.4926 min
Median cumulative indexing time across primary shard 275.189 286.681 11.4926 min
Max cumulative indexing time across primary shard 275.189 286.681 11.4926 min
Cumulative indexing throttle time of primary shards 0 0 0 min
Min cumulative indexing throttle time across primary shard 0 0 0 min
Median cumulative indexing throttle time across primary shard 0 0 0 min
Max cumulative indexing throttle time across primary shard 0 0 0 min
Cumulative merge time of primary shards 106.01 107.627 1.61695 min
Cumulative merge count of primary shards 222 206 -16
Min cumulative merge time across primary shard 106.01 107.627 1.61695 min
Median cumulative merge time across primary shard 106.01 107.627 1.61695 min
Max cumulative merge time across primary shard 106.01 107.627 1.61695 min
Cumulative merge throttle time of primary shards 5.14645 7.402 2.25555 min
Min cumulative merge throttle time across primary shard 5.14645 7.402 2.25555 min
Median cumulative merge throttle time across primary shard 5.14645 7.402 2.25555 min
Max cumulative merge throttle time across primary shard 5.14645 7.402 2.25555 min
Cumulative refresh time of primary shards 3.13458 2.72237 -0.41222 min
Cumulative refresh count of primary shards 139 113 -26
Min cumulative refresh time across primary shard 3.13458 2.72237 -0.41222 min
Median cumulative refresh time across primary shard 3.13458 2.72237 -0.41222 min
Max cumulative refresh time across primary shard 3.13458 2.72237 -0.41222 min
Cumulative flush time of primary shards 4.59315 16.957 12.3638 min
Cumulative flush count of primary shards 30 31 1
Min cumulative flush time across primary shard 4.59315 16.957 12.3638 min
Median cumulative flush time across primary shard 4.59315 16.957 12.3638 min
Max cumulative flush time across primary shard 4.59315 16.957 12.3638 min
Total Young Gen GC time 223.997 226.263 2.266 s
Total Young Gen GC count 31256 31463 207
Total Old Gen GC time 0 0 0 s
Total Old Gen GC count 0 0 0
Store size 29.9324 30.2096 0.27723 GB
Translog size 5.12227e-08 5.12227e-08 0 GB
Heap used for segments 0 0 0 MB
Heap used for doc values 0 0 0 MB
Heap used for terms 0 0 0 MB
Heap used for norms 0 0 0 MB
Heap used for points 0 0 0 MB
Heap used for stored fields 0 0 0 MB
Segment count 28 32 4
Min Throughput index 55831.2 53973.5 -1857.77 docs/s
Mean Throughput index 59637.9 58278.3 -1359.65 docs/s
Median Throughput index 58406 57127.2 -1278.79 docs/s
Max Throughput index 66284.7 65012.2 -1272.5 docs/s
50th percentile latency index 1266.39 1312.58 46.1916 ms
90th percentile latency index 1733.09 1790.96 57.8737 ms
99th percentile latency index 3429.09 3542.78 113.688 ms
99.9th percentile latency index 4664.53 4526.22 -138.307 ms
99.99th percentile latency index 5804.83 5812.06 7.23372 ms
100th percentile latency index 6813.85 6221.47 -592.376 ms
50th percentile service time index 1266.39 1312.58 46.1916 ms
90th percentile service time index 1733.09 1790.96 57.8737 ms
99th percentile service time index 3429.09 3542.78 113.688 ms
99.9th percentile service time index 4664.53 4526.22 -138.307 ms
99.99th percentile service time index 5804.83 5812.06 7.23372 ms
100th percentile service time index 6813.85 6221.47 -592.376 ms
error rate index 0 0 0 %
Min Throughput wait-until-merges-finish 0.00801719 0.0208435 0.01283 ops/s
Mean Throughput wait-until-merges-finish 0.00801719 0.0208435 0.01283 ops/s
Median Throughput wait-until-merges-finish 0.00801719 0.0208435 0.01283 ops/s
Max Throughput wait-until-merges-finish 0.00801719 0.0208435 0.01283 ops/s
100th percentile latency wait-until-merges-finish 124732 47976 -76755.6 ms
100th percentile service time wait-until-merges-finish 124732 47976 -76755.6 ms
error rate wait-until-merges-finish 0 0 0 %
Min Throughput default 3.01314 3.01569 0.00255 ops/s
Mean Throughput default 3.02139 3.02561 0.00422 ops/s
Median Throughput default 3.0195 3.02332 0.00382 ops/s
Max Throughput default 3.03766 3.04513 0.00747 ops/s
50th percentile latency default 7.75753 7.23946 -0.51807 ms
90th percentile latency default 9.85089 9.03646 -0.81443 ms
99th percentile latency default 10.2704 10.411 0.14062 ms
100th percentile latency default 10.2715 10.7091 0.43754 ms
50th percentile service time default 6.57339 5.9749 -0.59849 ms
90th percentile service time default 8.53501 8.03566 -0.49935 ms
99th percentile service time default 9.06355 9.14273 0.07918 ms
100th percentile service time default 9.207 9.48836 0.28136 ms
error rate default 0 0 0 %
Min Throughput range 0.702633 0.702882 0.00025 ops/s
Mean Throughput range 0.704324 0.704733 0.00041 ops/s
Median Throughput range 0.703935 0.704307 0.00037 ops/s
Max Throughput range 0.707797 0.708531 0.00073 ops/s
50th percentile latency range 370.18 283.882 -86.2972 ms
90th percentile latency range 395.581 299.782 -95.7983 ms
99th percentile latency range 430.759 312.857 -117.902 ms
100th percentile latency range 444.906 320.408 -124.498 ms
50th percentile service time range 368.185 281.428 -86.7576 ms
90th percentile service time range 393.774 297.845 -95.9291 ms
99th percentile service time range 428.358 310.804 -117.555 ms
100th percentile service time range 442.984 317.373 -125.611 ms
error rate range 0 0 0 %
Min Throughput distance_amount_agg 2.01194 2.01212 0.00019 ops/s
Mean Throughput distance_amount_agg 2.01962 2.01993 0.00031 ops/s
Median Throughput distance_amount_agg 2.01783 2.01811 0.00029 ops/s
Max Throughput distance_amount_agg 2.03529 2.03581 0.00053 ops/s
50th percentile latency distance_amount_agg 5.27528 5.29516 0.01988 ms
90th percentile latency distance_amount_agg 5.82886 5.90282 0.07396 ms
99th percentile latency distance_amount_agg 7.12287 6.32954 -0.79333 ms
100th percentile latency distance_amount_agg 7.13448 6.66176 -0.47272 ms
50th percentile service time distance_amount_agg 4.00613 3.97121 -0.03491 ms
90th percentile service time distance_amount_agg 4.36864 4.4508 0.08216 ms
99th percentile service time distance_amount_agg 5.76243 5.08476 -0.67767 ms
100th percentile service time distance_amount_agg 5.8741 5.15127 -0.72283 ms
error rate distance_amount_agg 0 0 0 %
Min Throughput autohisto_agg 1.49949 1.50016 0.00066 ops/s
Mean Throughput autohisto_agg 1.49975 1.50026 0.0005 ops/s
Median Throughput autohisto_agg 1.49978 1.50023 0.00045 ops/s
Max Throughput autohisto_agg 1.49985 1.50046 0.00061 ops/s
50th percentile latency autohisto_agg 540.717 492.289 -48.4284 ms
90th percentile latency autohisto_agg 552.677 505.539 -47.1373 ms
99th percentile latency autohisto_agg 563.944 524.723 -39.2215 ms
100th percentile latency autohisto_agg 572.177 534.844 -37.3331 ms
50th percentile service time autohisto_agg 539.733 490.876 -48.8573 ms
90th percentile service time autohisto_agg 551.737 504.766 -46.9719 ms
99th percentile service time autohisto_agg 562.965 523.733 -39.2318 ms
100th percentile service time autohisto_agg 571.129 533.689 -37.4404 ms
error rate autohisto_agg 0 0 0 %
Min Throughput date_histogram_agg 1.50134 1.50176 0.00043 ops/s
Mean Throughput date_histogram_agg 1.50217 1.50287 0.00069 ops/s
Median Throughput date_histogram_agg 1.50197 1.50261 0.00063 ops/s
Max Throughput date_histogram_agg 1.50383 1.50508 0.00125 ops/s
50th percentile latency date_histogram_agg 548.604 518.739 -29.8654 ms
90th percentile latency date_histogram_agg 561.162 530.808 -30.3538 ms
99th percentile latency date_histogram_agg 580.846 536.375 -44.4707 ms
100th percentile latency date_histogram_agg 581.092 540.434 -40.6584 ms
50th percentile service time date_histogram_agg 547.694 517.559 -30.1353 ms
90th percentile service time date_histogram_agg 560.438 529.282 -31.1564 ms
99th percentile service time date_histogram_agg 579.805 535.513 -44.2919 ms
100th percentile service time date_histogram_agg 580.587 539.957 -40.6297 ms
error rate date_histogram_agg 0 0 0 %

@sachinpkale sachinpkale changed the title [Remote Store] Performance test setup [Remote Store] Performance test setup for 2.3 release Sep 8, 2022
@sachinpkale
Copy link
Member Author

Setup

  • Tool used: opensearch-benchmark
  • OpenSearch Benchmark Test: http_logs
  • Cluster
    • 3 node cluster with each node - r5.2xlarge
  • Benchmark node - r5.2xlarge
  • Branch - 2.x
  • GitHub Commit ID: 32258cf9cf960e227db3e8b95cb4059caf29d495
Metric Task Baseline Contender Diff Unit
Cumulative indexing time of primary shards 132.052 132.052 0 min
Min cumulative indexing time across primary shard 0 0 0 min
Median cumulative indexing time across primary shard 1.15807 1.15807 0 min
Max cumulative indexing time across primary shard 22.9152 22.9152 0 min
Cumulative indexing throttle time of primary shards 0 0 0 min
Min cumulative indexing throttle time across primary shard 0 0 0 min
Median cumulative indexing throttle time across primary shard 0 0 0 min
Max cumulative indexing throttle time across primary shard 0 0 0 min
Cumulative merge time of primary shards 97.4242 97.4242 0 min
Cumulative merge count of primary shards 1584 1584 0
Min cumulative merge time across primary shard 0 0 0 min
Median cumulative merge time across primary shard 0.34745 0.34745 0 min
Max cumulative merge time across primary shard 18.8673 18.8673 0 min
Cumulative merge throttle time of primary shards 37.3667 37.3667 0 min
Min cumulative merge throttle time across primary shard 0 0 0 min
Median cumulative merge throttle time across primary shard 0 0 0 min
Max cumulative merge throttle time across primary shard 9.58227 9.58227 0 min
Cumulative refresh time of primary shards 13.0921 13.0921 0 min
Cumulative refresh count of primary shards 3472 3472 0
Min cumulative refresh time across primary shard 0 0 0 min
Median cumulative refresh time across primary shard 0.115675 0.115675 0 min
Max cumulative refresh time across primary shard 2.01725 2.01725 0 min
Cumulative flush time of primary shards 2.66023 2.66023 0 min
Cumulative flush count of primary shards 111 111 0
Min cumulative flush time across primary shard 0 0 0 min
Median cumulative flush time across primary shard 0.00118333 0.00118333 0 min
Max cumulative flush time across primary shard 0.64785 0.64785 0 min
Total Young Gen GC time 40.439 40.439 0 s
Total Young Gen GC count 10022 10022 0
Total Old Gen GC time 0 0 0 s
Total Old Gen GC count 0 0 0
Store size 18.8249 18.8249 0 GB
Translog size 2.04891e-06 2.04891e-06 0 GB
Heap used for segments 0 0 0 MB
Heap used for doc values 0 0 0 MB
Heap used for terms 0 0 0 MB
Heap used for norms 0 0 0 MB
Heap used for points 0 0 0 MB
Heap used for stored fields 0 0 0 MB
Segment count 35 35 0
Min Throughput index-append 293118 293118 0 docs/s
Mean Throughput index-append 301576 301576 0 docs/s
Median Throughput index-append 299820 299820 0 docs/s
Max Throughput index-append 313648 313648 0 docs/s
50th percentile latency index-append 101.514 101.514 0 ms
90th percentile latency index-append 190.459 190.459 0 ms
99th percentile latency index-append 389.89 389.89 0 ms
99.9th percentile latency index-append 475.846 475.846 0 ms
99.99th percentile latency index-append 569.517 569.517 0 ms
100th percentile latency index-append 626.968 626.968 0 ms
50th percentile service time index-append 101.514 101.514 0 ms
90th percentile service time index-append 190.459 190.459 0 ms
99th percentile service time index-append 389.89 389.89 0 ms
99.9th percentile service time index-append 475.846 475.846 0 ms
99.99th percentile service time index-append 569.517 569.517 0 ms
100th percentile service time index-append 626.968 626.968 0 ms
error rate index-append 0 0 0 %
Min Throughput wait-until-merges-finish 0.00329353 0.00329353 0 ops/s
Mean Throughput wait-until-merges-finish 0.00329353 0.00329353 0 ops/s
Median Throughput wait-until-merges-finish 0.00329353 0.00329353 0 ops/s
Max Throughput wait-until-merges-finish 0.00329353 0.00329353 0 ops/s
100th percentile latency wait-until-merges-finish 303625 303625 0 ms
100th percentile service time wait-until-merges-finish 303625 303625 0 ms
error rate wait-until-merges-finish 0 0 0 %
Min Throughput default 8.00149 8.00149 0 ops/s
Mean Throughput default 8.00167 8.00167 0 ops/s
Median Throughput default 8.00169 8.00169 0 ops/s
Max Throughput default 8.00179 8.00179 0 ops/s
50th percentile latency default 5.01027 5.01027 0 ms
90th percentile latency default 5.38617 5.38617 0 ms
99th percentile latency default 5.90608 5.90608 0 ms
100th percentile latency default 9.21095 9.21095 0 ms
50th percentile service time default 4.05074 4.05074 0 ms
90th percentile service time default 4.26605 4.26605 0 ms
99th percentile service time default 4.51007 4.51007 0 ms
100th percentile service time default 8.30794 8.30794 0 ms
error rate default 0 0 0 %
Min Throughput term 49.8947 49.8947 0 ops/s
Mean Throughput term 49.8987 49.8987 0 ops/s
Median Throughput term 49.8987 49.8987 0 ops/s
Max Throughput term 49.9027 49.9027 0 ops/s
50th percentile latency term 5.37422 5.37422 0 ms
90th percentile latency term 6.16806 6.16806 0 ms
99th percentile latency term 9.00793 9.00793 0 ms
100th percentile latency term 10.0282 10.0282 0 ms
50th percentile service time term 4.68759 4.68759 0 ms
90th percentile service time term 5.06819 5.06819 0 ms
99th percentile service time term 8.23352 8.23352 0 ms
100th percentile service time term 9.57217 9.57217 0 ms
error rate term 0 0 0 %
Min Throughput range 1.0047 1.0047 0 ops/s
Mean Throughput range 1.00651 1.00651 0 ops/s
Median Throughput range 1.00625 1.00625 0 ops/s
Max Throughput range 1.00935 1.00935 0 ops/s
50th percentile latency range 9.37687 9.37687 0 ms
90th percentile latency range 11.0545 11.0545 0 ms
99th percentile latency range 14.2521 14.2521 0 ms
100th percentile latency range 14.4589 14.4589 0 ms
50th percentile service time range 7.44582 7.44582 0 ms
90th percentile service time range 9.33296 9.33296 0 ms
99th percentile service time range 12.2837 12.2837 0 ms
100th percentile service time range 13.0274 13.0274 0 ms
error rate range 0 0 0 %
Min Throughput 200s-in-range 32.7725 32.7725 0 ops/s
Mean Throughput 200s-in-range 32.7849 32.7849 0 ops/s
Median Throughput 200s-in-range 32.7851 32.7851 0 ops/s
Max Throughput 200s-in-range 32.7972 32.7972 0 ops/s
50th percentile latency 200s-in-range 5.80845 5.80845 0 ms
90th percentile latency 200s-in-range 6.31589 6.31589 0 ms
99th percentile latency 200s-in-range 8.29093 8.29093 0 ms
100th percentile latency 200s-in-range 8.92723 8.92723 0 ms
50th percentile service time 200s-in-range 5.0347 5.0347 0 ms
90th percentile service time 200s-in-range 5.43639 5.43639 0 ms
99th percentile service time 200s-in-range 7.64143 7.64143 0 ms
100th percentile service time 200s-in-range 7.91627 7.91627 0 ms
error rate 200s-in-range 0 0 0 %
Min Throughput 400s-in-range 50.0104 50.0104 0 ops/s
Mean Throughput 400s-in-range 50.011 50.011 0 ops/s
Median Throughput 400s-in-range 50.011 50.011 0 ops/s
Max Throughput 400s-in-range 50.0115 50.0115 0 ops/s
50th percentile latency 400s-in-range 4.25956 4.25956 0 ms
90th percentile latency 400s-in-range 4.78588 4.78588 0 ms
99th percentile latency 400s-in-range 5.04351 5.04351 0 ms
100th percentile latency 400s-in-range 7.84099 7.84099 0 ms
50th percentile service time 400s-in-range 3.55511 3.55511 0 ms
90th percentile service time 400s-in-range 3.78818 3.78818 0 ms
99th percentile service time 400s-in-range 4.10627 4.10627 0 ms
100th percentile service time 400s-in-range 7.07565 7.07565 0 ms
error rate 400s-in-range 0 0 0 %
Min Throughput hourly_agg 0.20061 0.20061 0 ops/s
Mean Throughput hourly_agg 0.200843 0.200843 0 ops/s
Median Throughput hourly_agg 0.20081 0.20081 0 ops/s
Max Throughput hourly_agg 0.20121 0.20121 0 ops/s
50th percentile latency hourly_agg 1724.8 1724.8 0 ms
90th percentile latency hourly_agg 1762.52 1762.52 0 ms
99th percentile latency hourly_agg 1786.21 1786.21 0 ms
100th percentile latency hourly_agg 1812.62 1812.62 0 ms
50th percentile service time hourly_agg 1721.65 1721.65 0 ms
90th percentile service time hourly_agg 1760.51 1760.51 0 ms
99th percentile service time hourly_agg 1784.07 1784.07 0 ms
100th percentile service time hourly_agg 1810.91 1810.91 0 ms
error rate hourly_agg 0 0 0 %
Min Throughput scroll 24.7667 24.7667 0 pages/s
Mean Throughput scroll 25.021 25.021 0 pages/s
Median Throughput scroll 25.0218 25.0218 0 pages/s
Max Throughput scroll 25.0433 25.0433 0 pages/s
50th percentile latency scroll 495.845 495.845 0 ms
90th percentile latency scroll 508.122 508.122 0 ms
99th percentile latency scroll 2065.53 2065.53 0 ms
100th percentile latency scroll 3078.64 3078.64 0 ms
50th percentile service time scroll 493.921 493.921 0 ms
90th percentile service time scroll 504.582 504.582 0 ms
99th percentile service time scroll 732.363 732.363 0 ms
100th percentile service time scroll 3076.89 3076.89 0 ms
error rate scroll 0 0 0 %
Min Throughput desc_sort_timestamp 0.500819 0.500819 0 ops/s
Mean Throughput desc_sort_timestamp 0.500995 0.500995 0 ops/s
Median Throughput desc_sort_timestamp 0.500982 0.500982 0 ops/s
Max Throughput desc_sort_timestamp 0.501224 0.501224 0 ops/s
50th percentile latency desc_sort_timestamp 905.569 905.569 0 ms
90th percentile latency desc_sort_timestamp 944.914 944.914 0 ms
99th percentile latency desc_sort_timestamp 974.091 974.091 0 ms
100th percentile latency desc_sort_timestamp 979.53 979.53 0 ms
50th percentile service time desc_sort_timestamp 903.647 903.647 0 ms
90th percentile service time desc_sort_timestamp 942.995 942.995 0 ms
99th percentile service time desc_sort_timestamp 972.585 972.585 0 ms
100th percentile service time desc_sort_timestamp 977.826 977.826 0 ms
error rate desc_sort_timestamp 0 0 0 %
Min Throughput asc_sort_timestamp 0.499726 0.499726 0 ops/s
Mean Throughput asc_sort_timestamp 0.501919 0.501919 0 ops/s
Median Throughput asc_sort_timestamp 0.501904 0.501904 0 ops/s
Max Throughput asc_sort_timestamp 0.50239 0.50239 0 ops/s
50th percentile latency asc_sort_timestamp 85.3886 85.3886 0 ms
90th percentile latency asc_sort_timestamp 99.9578 99.9578 0 ms
99th percentile latency asc_sort_timestamp 2281.42 2281.42 0 ms
100th percentile latency asc_sort_timestamp 4188.07 4188.07 0 ms
50th percentile service time asc_sort_timestamp 82.8252 82.8252 0 ms
90th percentile service time asc_sort_timestamp 96.2524 96.2524 0 ms
99th percentile service time asc_sort_timestamp 144.666 144.666 0 ms
100th percentile service time asc_sort_timestamp 4186 4186 0 ms
error rate asc_sort_timestamp 0 0 0 %
Min Throughput desc_sort_with_after_timestamp 0.501826 0.501826 0 ops/s
Mean Throughput desc_sort_with_after_timestamp 0.504785 0.504785 0 ops/s
Median Throughput desc_sort_with_after_timestamp 0.503332 0.503332 0 ops/s
Max Throughput desc_sort_with_after_timestamp 0.518919 0.518919 0 ops/s
50th percentile latency desc_sort_with_after_timestamp 1168.92 1168.92 0 ms
90th percentile latency desc_sort_with_after_timestamp 1184.02 1184.02 0 ms
99th percentile latency desc_sort_with_after_timestamp 1229.99 1229.99 0 ms
100th percentile latency desc_sort_with_after_timestamp 1274.25 1274.25 0 ms
50th percentile service time desc_sort_with_after_timestamp 1166.52 1166.52 0 ms
90th percentile service time desc_sort_with_after_timestamp 1182.72 1182.72 0 ms
99th percentile service time desc_sort_with_after_timestamp 1227.07 1227.07 0 ms
100th percentile service time desc_sort_with_after_timestamp 1272.33 1272.33 0 ms
error rate desc_sort_with_after_timestamp 0 0 0 %
Min Throughput asc_sort_with_after_timestamp 0.502428 0.502428 0 ops/s
Mean Throughput asc_sort_with_after_timestamp 0.506379 0.506379 0 ops/s
Median Throughput asc_sort_with_after_timestamp 0.504428 0.504428 0 ops/s
Max Throughput asc_sort_with_after_timestamp 0.525395 0.525395 0 ops/s
50th percentile latency asc_sort_with_after_timestamp 937.226 937.226 0 ms
90th percentile latency asc_sort_with_after_timestamp 963.721 963.721 0 ms
99th percentile latency asc_sort_with_after_timestamp 982.259 982.259 0 ms
100th percentile latency asc_sort_with_after_timestamp 1000.22 1000.22 0 ms
50th percentile service time asc_sort_with_after_timestamp 935.637 935.637 0 ms
90th percentile service time asc_sort_with_after_timestamp 961.928 961.928 0 ms
99th percentile service time asc_sort_with_after_timestamp 979.948 979.948 0 ms
100th percentile service time asc_sort_with_after_timestamp 998.734 998.734 0 ms
error rate asc_sort_with_after_timestamp 0 0 0 %
Min Throughput wait-until-merges-1-seg-finish 83.477 83.477 0 ops/s
Mean Throughput wait-until-merges-1-seg-finish 83.477 83.477 0 ops/s
Median Throughput wait-until-merges-1-seg-finish 83.477 83.477 0 ops/s
Max Throughput wait-until-merges-1-seg-finish 83.477 83.477 0 ops/s
100th percentile latency wait-until-merges-1-seg-finish 11.3196 11.3196 0 ms
100th percentile service time wait-until-merges-1-seg-finish 11.3196 11.3196 0 ms
error rate wait-until-merges-1-seg-finish 0 0 0 %
Min Throughput desc-sort-timestamp-after-force-merge-1-seg 1.10214 1.10214 0 ops/s
Mean Throughput desc-sort-timestamp-after-force-merge-1-seg 1.10297 1.10297 0 ops/s
Median Throughput desc-sort-timestamp-after-force-merge-1-seg 1.10297 1.10297 0 ops/s
Max Throughput desc-sort-timestamp-after-force-merge-1-seg 1.10393 1.10393 0 ops/s
50th percentile latency desc-sort-timestamp-after-force-merge-1-seg 102469 102469 0 ms
90th percentile latency desc-sort-timestamp-after-force-merge-1-seg 118470 118470 0 ms
99th percentile latency desc-sort-timestamp-after-force-merge-1-seg 122027 122027 0 ms
100th percentile latency desc-sort-timestamp-after-force-merge-1-seg 122450 122450 0 ms
50th percentile service time desc-sort-timestamp-after-force-merge-1-seg 898.261 898.261 0 ms
90th percentile service time desc-sort-timestamp-after-force-merge-1-seg 939.41 939.41 0 ms
99th percentile service time desc-sort-timestamp-after-force-merge-1-seg 993.814 993.814 0 ms
100th percentile service time desc-sort-timestamp-after-force-merge-1-seg 1104.74 1104.74 0 ms
error rate desc-sort-timestamp-after-force-merge-1-seg 0 0 0 %
Min Throughput asc-sort-timestamp-after-force-merge-1-seg 2.00556 2.00556 0 ops/s
Mean Throughput asc-sort-timestamp-after-force-merge-1-seg 2.00674 2.00674 0 ops/s
Median Throughput asc-sort-timestamp-after-force-merge-1-seg 2.00666 2.00666 0 ops/s
Max Throughput asc-sort-timestamp-after-force-merge-1-seg 2.00828 2.00828 0 ops/s
50th percentile latency asc-sort-timestamp-after-force-merge-1-seg 82.6848 82.6848 0 ms
90th percentile latency asc-sort-timestamp-after-force-merge-1-seg 92.5229 92.5229 0 ms
99th percentile latency asc-sort-timestamp-after-force-merge-1-seg 98.1609 98.1609 0 ms
100th percentile latency asc-sort-timestamp-after-force-merge-1-seg 98.6338 98.6338 0 ms
50th percentile service time asc-sort-timestamp-after-force-merge-1-seg 81.1041 81.1041 0 ms
90th percentile service time asc-sort-timestamp-after-force-merge-1-seg 91.3304 91.3304 0 ms
99th percentile service time asc-sort-timestamp-after-force-merge-1-seg 96.9806 96.9806 0 ms
100th percentile service time asc-sort-timestamp-after-force-merge-1-seg 97.0005 97.0005 0 ms
error rate asc-sort-timestamp-after-force-merge-1-seg 0 0 0 %
Min Throughput desc-sort-with-after-timestamp-after-force-merge-1-seg 0.501714 0.501714 0 ops/s
Mean Throughput desc-sort-with-after-timestamp-after-force-merge-1-seg 0.504485 0.504485 0 ops/s
Median Throughput desc-sort-with-after-timestamp-after-force-merge-1-seg 0.503126 0.503126 0 ops/s
Max Throughput desc-sort-with-after-timestamp-after-force-merge-1-seg 0.517677 0.517677 0 ops/s
50th percentile latency desc-sort-with-after-timestamp-after-force-merge-1-seg 1244.57 1244.57 0 ms
90th percentile latency desc-sort-with-after-timestamp-after-force-merge-1-seg 1274.46 1274.46 0 ms
99th percentile latency desc-sort-with-after-timestamp-after-force-merge-1-seg 1363.05 1363.05 0 ms
100th percentile latency desc-sort-with-after-timestamp-after-force-merge-1-seg 1388.51 1388.51 0 ms
50th percentile service time desc-sort-with-after-timestamp-after-force-merge-1-seg 1242.67 1242.67 0 ms
90th percentile service time desc-sort-with-after-timestamp-after-force-merge-1-seg 1273.25 1273.25 0 ms
99th percentile service time desc-sort-with-after-timestamp-after-force-merge-1-seg 1361.55 1361.55 0 ms
100th percentile service time desc-sort-with-after-timestamp-after-force-merge-1-seg 1387.09 1387.09 0 ms
error rate desc-sort-with-after-timestamp-after-force-merge-1-seg 0 0 0 %
Min Throughput asc-sort-with-after-timestamp-after-force-merge-1-seg 0.502217 0.502217 0 ops/s
Mean Throughput asc-sort-with-after-timestamp-after-force-merge-1-seg 0.505823 0.505823 0 ops/s
Median Throughput asc-sort-with-after-timestamp-after-force-merge-1-seg 0.504046 0.504046 0 ops/s
Max Throughput asc-sort-with-after-timestamp-after-force-merge-1-seg 0.523112 0.523112 0 ops/s
50th percentile latency asc-sort-with-after-timestamp-after-force-merge-1-seg 1029.79 1029.79 0 ms
90th percentile latency asc-sort-with-after-timestamp-after-force-merge-1-seg 1050.97 1050.97 0 ms
99th percentile latency asc-sort-with-after-timestamp-after-force-merge-1-seg 1080.42 1080.42 0 ms
100th percentile latency asc-sort-with-after-timestamp-after-force-merge-1-seg 1087.54 1087.54 0 ms
50th percentile service time asc-sort-with-after-timestamp-after-force-merge-1-seg 1027.87 1027.87 0 ms
90th percentile service time asc-sort-with-after-timestamp-after-force-merge-1-seg 1048.44 1048.44 0 ms
99th percentile service time asc-sort-with-after-timestamp-after-force-merge-1-seg 1077.71 1077.71 0 ms
100th percentile service time asc-sort-with-after-timestamp-after-force-merge-1-seg 1084.46 1084.46 0 ms
error rate asc-sort-with-after-timestamp-after-force-merge-1-seg 0 0 0 %
Metric Task Baseline Contender Diff Unit
--------------------------------------------------------------: -------------------------------------------------------: ------------: ------------: ---------: --------:
Cumulative indexing time of primary shards 132.052 129.919 -2.1322 min
Min cumulative indexing time across primary shard 0 0 0 min
Median cumulative indexing time across primary shard 1.15807 1.1908 0.03272 min
Max cumulative indexing time across primary shard 22.9152 22.9782 0.06305 min
Cumulative indexing throttle time of primary shards 0 0.1302 0.1302 min
Min cumulative indexing throttle time across primary shard 0 0 0 min
Median cumulative indexing throttle time across primary shard 0 0 0 min
Max cumulative indexing throttle time across primary shard 0 0.1302 0.1302 min
Cumulative merge time of primary shards 97.4242 76.5158 -20.9083 min
Cumulative merge count of primary shards 1584 657 -927
Min cumulative merge time across primary shard 0 0 0 min
Median cumulative merge time across primary shard 0.34745 0.290858 -0.05659 min
Max cumulative merge time across primary shard 18.8673 15.0272 -3.84007 min
Cumulative merge throttle time of primary shards 37.3667 33.5271 -3.83963 min
Min cumulative merge throttle time across primary shard 0 0 0 min
Median cumulative merge throttle time across primary shard 0 0.032925 0.03293 min
Max cumulative merge throttle time across primary shard 9.58227 7.32488 -2.25738 min
Cumulative refresh time of primary shards 13.0921 11.5128 -1.57933 min
Cumulative refresh count of primary shards 3472 1665 -1807
Min cumulative refresh time across primary shard 0 0 0 min
Median cumulative refresh time across primary shard 0.115675 0.111717 -0.00396 min
Max cumulative refresh time across primary shard 2.01725 1.74898 -0.26827 min
Cumulative flush time of primary shards 2.66023 8.02617 5.36593 min
Cumulative flush count of primary shards 111 115 4
Min cumulative flush time across primary shard 0 0 0 min
Median cumulative flush time across primary shard 0.00118333 0.00348333 0.0023 min
Max cumulative flush time across primary shard 0.64785 1.69793 1.05008 min
Total Young Gen GC time 40.439 43.518 3.079 s
Total Young Gen GC count 10022 10946 924
Total Old Gen GC time 0 0 0 s
Total Old Gen GC count 0 0 0
Store size 18.8249 18.8316 0.00674 GB
Translog size 2.04891e-06 2.04891e-06 0 GB
Heap used for segments 0 0 0 MB
Heap used for doc values 0 0 0 MB
Heap used for terms 0 0 0 MB
Heap used for norms 0 0 0 MB
Heap used for points 0 0 0 MB
Heap used for stored fields 0 0 0 MB
Segment count 35 35 0
Min Throughput index-append 293118 286077 -7040.36 docs/s
Mean Throughput index-append 301576 297520 -4055.84 docs/s
Median Throughput index-append 299820 296974 -2846.01 docs/s
Max Throughput index-append 313648 307119 -6528.3 docs/s
50th percentile latency index-append 101.514 95.3194 -6.19431 ms
90th percentile latency index-append 190.459 158.315 -32.144 ms
99th percentile latency index-append 389.89 661.29 271.4 ms
99.9th percentile latency index-append 475.846 1300.61 824.763 ms
99.99th percentile latency index-append 569.517 1858.35 1288.83 ms
100th percentile latency index-append 626.968 2407.84 1780.87 ms
50th percentile service time index-append 101.514 95.3194 -6.19431 ms
90th percentile service time index-append 190.459 158.315 -32.144 ms
99th percentile service time index-append 389.89 661.29 271.4 ms
99.9th percentile service time index-append 475.846 1300.61 824.763 ms
99.99th percentile service time index-append 569.517 1858.35 1288.83 ms
100th percentile service time index-append 626.968 2407.84 1780.87 ms
error rate index-append 0 0 0 %
Min Throughput wait-until-merges-finish 0.00329353 0.0262604 0.02297 ops/s
Mean Throughput wait-until-merges-finish 0.00329353 0.0262604 0.02297 ops/s
Median Throughput wait-until-merges-finish 0.00329353 0.0262604 0.02297 ops/s
Max Throughput wait-until-merges-finish 0.00329353 0.0262604 0.02297 ops/s
100th percentile latency wait-until-merges-finish 303625 38079.7 -265545 ms
100th percentile service time wait-until-merges-finish 303625 38079.7 -265545 ms
error rate wait-until-merges-finish 0 0 0 %
Min Throughput default 8.00149 8.00039 -0.0011 ops/s
Mean Throughput default 8.00167 8.00048 -0.00119 ops/s
Median Throughput default 8.00169 8.00048 -0.0012 ops/s
Max Throughput default 8.00179 8.00056 -0.00123 ops/s
50th percentile latency default 5.01027 5.00027 -0.00999 ms
90th percentile latency default 5.38617 5.44592 0.05975 ms
99th percentile latency default 5.90608 5.81678 -0.0893 ms
100th percentile latency default 9.21095 6.1959 -3.01504 ms
50th percentile service time default 4.05074 4.07387 0.02313 ms
90th percentile service time default 4.26605 4.30322 0.03717 ms
99th percentile service time default 4.51007 4.53482 0.02474 ms
100th percentile service time default 8.30794 4.53601 -3.77193 ms
error rate default 0 0 0 %
Min Throughput term 49.8947 49.9288 0.03415 ops/s
Mean Throughput term 49.8987 49.9317 0.03299 ops/s
Median Throughput term 49.8987 49.9317 0.03299 ops/s
Max Throughput term 49.9027 49.9346 0.03183 ops/s
50th percentile latency term 5.37422 5.36566 -0.00856 ms
90th percentile latency term 6.16806 5.86259 -0.30546 ms
99th percentile latency term 9.00793 9.75104 0.74311 ms
100th percentile latency term 10.0282 10.5292 0.501 ms
50th percentile service time term 4.68759 4.49411 -0.19347 ms
90th percentile service time term 5.06819 4.77322 -0.29497 ms
99th percentile service time term 8.23352 9.00561 0.77209 ms
100th percentile service time term 9.57217 9.45684 -0.11533 ms
error rate term 0 0 0 %
Min Throughput range 1.0047 1.00435 -0.00035 ops/s
Mean Throughput range 1.00651 1.00602 -0.00049 ops/s
Median Throughput range 1.00625 1.00578 -0.00047 ops/s
Max Throughput range 1.00935 1.00864 -0.00071 ops/s
50th percentile latency range 9.37687 8.75575 -0.62112 ms
90th percentile latency range 11.0545 10.2652 -0.78926 ms
99th percentile latency range 14.2521 13.7256 -0.52654 ms
100th percentile latency range 14.4589 14.4831 0.02421 ms
50th percentile service time range 7.44582 7.12708 -0.31874 ms
90th percentile service time range 9.33296 8.79053 -0.54243 ms
99th percentile service time range 12.2837 11.5455 -0.73824 ms
100th percentile service time range 13.0274 12.7159 -0.31156 ms
error rate range 0 0 0 %
Min Throughput 200s-in-range 32.7725 32.9535 0.18103 ops/s
Mean Throughput 200s-in-range 32.7849 32.956 0.17105 ops/s
Median Throughput 200s-in-range 32.7851 32.9561 0.17106 ops/s
Max Throughput 200s-in-range 32.7972 32.9583 0.16107 ops/s
50th percentile latency 200s-in-range 5.80845 5.5175 -0.29095 ms
90th percentile latency 200s-in-range 6.31589 6.11654 -0.19935 ms
99th percentile latency 200s-in-range 8.29093 11.2653 2.97441 ms
100th percentile latency 200s-in-range 8.92723 15.7465 6.81924 ms
50th percentile service time 200s-in-range 5.0347 4.56976 -0.46494 ms
90th percentile service time 200s-in-range 5.43639 4.97877 -0.45761 ms
99th percentile service time 200s-in-range 7.64143 10.8236 3.18215 ms
100th percentile service time 200s-in-range 7.91627 14.5724 6.65614 ms
error rate 200s-in-range 0 0 0 %
Min Throughput 400s-in-range 50.0104 49.998 -0.01249 ops/s
Mean Throughput 400s-in-range 50.011 49.9989 -0.01209 ops/s
Median Throughput 400s-in-range 50.011 49.9989 -0.01209 ops/s
Max Throughput 400s-in-range 50.0115 49.9998 -0.01169 ops/s
50th percentile latency 400s-in-range 4.25956 4.4095 0.14994 ms
90th percentile latency 400s-in-range 4.78588 4.89455 0.10867 ms
99th percentile latency 400s-in-range 5.04351 7.57248 2.52897 ms
100th percentile latency 400s-in-range 7.84099 8.26936 0.42837 ms
50th percentile service time 400s-in-range 3.55511 3.59765 0.04254 ms
90th percentile service time 400s-in-range 3.78818 3.79303 0.00485 ms
99th percentile service time 400s-in-range 4.10627 6.2953 2.18903 ms
100th percentile service time 400s-in-range 7.07565 7.90822 0.83256 ms
error rate 400s-in-range 0 0 0 %
Min Throughput hourly_agg 0.20061 0.200571 -4e-05 ops/s
Mean Throughput hourly_agg 0.200843 0.200789 -5e-05 ops/s
Median Throughput hourly_agg 0.20081 0.200759 -5e-05 ops/s
Max Throughput hourly_agg 0.20121 0.201133 -8e-05 ops/s
50th percentile latency hourly_agg 1724.8 1789.06 64.2622 ms
90th percentile latency hourly_agg 1762.52 1832.86 70.3438 ms
99th percentile latency hourly_agg 1786.21 1864.49 78.2779 ms
100th percentile latency hourly_agg 1812.62 1957.56 144.936 ms
50th percentile service time hourly_agg 1721.65 1785.76 64.107 ms
90th percentile service time hourly_agg 1760.51 1830.65 70.1318 ms
99th percentile service time hourly_agg 1784.07 1863.41 79.3331 ms
100th percentile service time hourly_agg 1810.91 1956.42 145.512 ms
error rate hourly_agg 0 0 0 %
Min Throughput scroll 24.7667 24.9684 0.2017 pages/s
Mean Throughput scroll 25.021 25.0236 0.00255 pages/s
Median Throughput scroll 25.0218 25.0215 -0.00028 pages/s
Max Throughput scroll 25.0433 25.0432 -0.00014 pages/s
50th percentile latency scroll 495.845 497.46 1.61492 ms
90th percentile latency scroll 508.122 508.81 0.68777 ms
99th percentile latency scroll 2065.53 737.906 -1327.62 ms
100th percentile latency scroll 3078.64 1410.13 -1668.51 ms
50th percentile service time scroll 493.921 495.638 1.71698 ms
90th percentile service time scroll 504.582 506.773 2.19097 ms
99th percentile service time scroll 732.363 530.614 -201.748 ms
100th percentile service time scroll 3076.89 1407.97 -1668.92 ms
error rate scroll 0 0 0 %
Min Throughput desc_sort_timestamp 0.500819 0.500855 4e-05 ops/s
Mean Throughput desc_sort_timestamp 0.500995 0.501039 4e-05 ops/s
Median Throughput desc_sort_timestamp 0.500982 0.501025 4e-05 ops/s
Max Throughput desc_sort_timestamp 0.501224 0.501279 5e-05 ops/s
50th percentile latency desc_sort_timestamp 905.569 887.631 -17.9379 ms
90th percentile latency desc_sort_timestamp 944.914 902.602 -42.3117 ms
99th percentile latency desc_sort_timestamp 974.091 973.07 -1.02077 ms
100th percentile latency desc_sort_timestamp 979.53 980.668 1.1375 ms
50th percentile service time desc_sort_timestamp 903.647 885.814 -17.8326 ms
90th percentile service time desc_sort_timestamp 942.995 900.976 -42.0192 ms
99th percentile service time desc_sort_timestamp 972.585 971.159 -1.42632 ms
100th percentile service time desc_sort_timestamp 977.826 978.675 0.84858 ms
error rate desc_sort_timestamp 0 0 0 %
Min Throughput asc_sort_timestamp 0.499726 0.501595 0.00187 ops/s
Mean Throughput asc_sort_timestamp 0.501919 0.501938 2e-05 ops/s
Median Throughput asc_sort_timestamp 0.501904 0.501913 1e-05 ops/s
Max Throughput asc_sort_timestamp 0.50239 0.502386 -0 ops/s
50th percentile latency asc_sort_timestamp 85.3886 83.2714 -2.11718 ms
90th percentile latency asc_sort_timestamp 99.9578 89.857 -10.1009 ms
99th percentile latency asc_sort_timestamp 2281.42 99.0244 -2182.4 ms
100th percentile latency asc_sort_timestamp 4188.07 102.269 -4085.8 ms
50th percentile service time asc_sort_timestamp 82.8252 80.5578 -2.2674 ms
90th percentile service time asc_sort_timestamp 96.2524 87.6067 -8.64571 ms
99th percentile service time asc_sort_timestamp 144.666 95.9738 -48.6923 ms
100th percentile service time asc_sort_timestamp 4186 100.165 -4085.83 ms
error rate asc_sort_timestamp 0 0 0 %
Min Throughput desc_sort_with_after_timestamp 0.501826 0.501581 -0.00024 ops/s
Mean Throughput desc_sort_with_after_timestamp 0.504785 0.504133 -0.00065 ops/s
Median Throughput desc_sort_with_after_timestamp 0.503332 0.502881 -0.00045 ops/s
Max Throughput desc_sort_with_after_timestamp 0.518919 0.516282 -0.00264 ops/s
50th percentile latency desc_sort_with_after_timestamp 1168.92 1265.95 97.0292 ms
90th percentile latency desc_sort_with_after_timestamp 1184.02 1281.05 97.0329 ms
99th percentile latency desc_sort_with_after_timestamp 1229.99 1329.96 99.9676 ms
100th percentile latency desc_sort_with_after_timestamp 1274.25 1468.32 194.064 ms
50th percentile service time desc_sort_with_after_timestamp 1166.52 1264.27 97.747 ms
90th percentile service time desc_sort_with_after_timestamp 1182.72 1279.8 97.078 ms
99th percentile service time desc_sort_with_after_timestamp 1227.07 1328.74 101.671 ms
100th percentile service time desc_sort_with_after_timestamp 1272.33 1466.68 194.358 ms
error rate desc_sort_with_after_timestamp 0 0 0 %
Min Throughput asc_sort_with_after_timestamp 0.502428 0.502262 -0.00017 ops/s
Mean Throughput asc_sort_with_after_timestamp 0.506379 0.505939 -0.00044 ops/s
Median Throughput asc_sort_with_after_timestamp 0.504428 0.504129 -0.0003 ops/s
Max Throughput asc_sort_with_after_timestamp 0.525395 0.52358 -0.00182 ops/s
50th percentile latency asc_sort_with_after_timestamp 937.226 1001.68 64.453 ms
90th percentile latency asc_sort_with_after_timestamp 963.721 1024.4 60.6779 ms
99th percentile latency asc_sort_with_after_timestamp 982.259 1046.68 64.4232 ms
100th percentile latency asc_sort_with_after_timestamp 1000.22 1080.14 79.9225 ms
50th percentile service time asc_sort_with_after_timestamp 935.637 999.842 64.2053 ms
90th percentile service time asc_sort_with_after_timestamp 961.928 1022.7 60.7741 ms
99th percentile service time asc_sort_with_after_timestamp 979.948 1045.03 65.081 ms
100th percentile service time asc_sort_with_after_timestamp 998.734 1078.37 79.6313 ms
error rate asc_sort_with_after_timestamp 0 0 0 %
Min Throughput wait-until-merges-1-seg-finish 83.477 88.8963 5.41923 ops/s
Mean Throughput wait-until-merges-1-seg-finish 83.477 88.8963 5.41923 ops/s
Median Throughput wait-until-merges-1-seg-finish 83.477 88.8963 5.41923 ops/s
Max Throughput wait-until-merges-1-seg-finish 83.477 88.8963 5.41923 ops/s
100th percentile latency wait-until-merges-1-seg-finish 11.3196 10.8113 -0.50825 ms
100th percentile service time wait-until-merges-1-seg-finish 11.3196 10.8113 -0.50825 ms
error rate wait-until-merges-1-seg-finish 0 0 0 %
Min Throughput desc-sort-timestamp-after-force-merge-1-seg 1.10214 1.13812 0.03598 ops/s
Mean Throughput desc-sort-timestamp-after-force-merge-1-seg 1.10297 1.13898 0.03602 ops/s
Median Throughput desc-sort-timestamp-after-force-merge-1-seg 1.10297 1.13898 0.03601 ops/s
Max Throughput desc-sort-timestamp-after-force-merge-1-seg 1.10393 1.13978 0.03585 ops/s
50th percentile latency desc-sort-timestamp-after-force-merge-1-seg 102469 95138.5 -7330.87 ms
90th percentile latency desc-sort-timestamp-after-force-merge-1-seg 118470 110222 -8247.93 ms
99th percentile latency desc-sort-timestamp-after-force-merge-1-seg 122027 113348 -8678.58 ms
100th percentile latency desc-sort-timestamp-after-force-merge-1-seg 122450 113722 -8728.12 ms
50th percentile service time desc-sort-timestamp-after-force-merge-1-seg 898.261 864.516 -33.7451 ms
90th percentile service time desc-sort-timestamp-after-force-merge-1-seg 939.41 922.126 -17.2845 ms
99th percentile service time desc-sort-timestamp-after-force-merge-1-seg 993.814 980.896 -12.9179 ms
100th percentile service time desc-sort-timestamp-after-force-merge-1-seg 1104.74 990.016 -114.728 ms
error rate desc-sort-timestamp-after-force-merge-1-seg 0 0 0 %
Min Throughput asc-sort-timestamp-after-force-merge-1-seg 2.00556 2.00561 5e-05 ops/s
Mean Throughput asc-sort-timestamp-after-force-merge-1-seg 2.00674 2.0068 6e-05 ops/s
Median Throughput asc-sort-timestamp-after-force-merge-1-seg 2.00666 2.00671 6e-05 ops/s
Max Throughput asc-sort-timestamp-after-force-merge-1-seg 2.00828 2.00835 7e-05 ops/s
50th percentile latency asc-sort-timestamp-after-force-merge-1-seg 82.6848 83.1094 0.42462 ms
90th percentile latency asc-sort-timestamp-after-force-merge-1-seg 92.5229 93.9711 1.44813 ms
99th percentile latency asc-sort-timestamp-after-force-merge-1-seg 98.1609 102.72 4.55922 ms
100th percentile latency asc-sort-timestamp-after-force-merge-1-seg 98.6338 103.397 4.76297 ms
50th percentile service time asc-sort-timestamp-after-force-merge-1-seg 81.1041 81.3608 0.25675 ms
90th percentile service time asc-sort-timestamp-after-force-merge-1-seg 91.3304 92.8983 1.56797 ms
99th percentile service time asc-sort-timestamp-after-force-merge-1-seg 96.9806 100.542 3.5609 ms
100th percentile service time asc-sort-timestamp-after-force-merge-1-seg 97.0005 101.419 4.41801 ms
error rate asc-sort-timestamp-after-force-merge-1-seg 0 0 0 %
Min Throughput desc-sort-with-after-timestamp-after-force-merge-1-seg 0.501714 0.501385 -0.00033 ops/s
Mean Throughput desc-sort-with-after-timestamp-after-force-merge-1-seg 0.504485 0.503614 -0.00087 ops/s
Median Throughput desc-sort-with-after-timestamp-after-force-merge-1-seg 0.503126 0.502524 -0.0006 ops/s
Max Throughput desc-sort-with-after-timestamp-after-force-merge-1-seg 0.517677 0.514206 -0.00347 ops/s
50th percentile latency desc-sort-with-after-timestamp-after-force-merge-1-seg 1244.57 1304.22 59.6528 ms
90th percentile latency desc-sort-with-after-timestamp-after-force-merge-1-seg 1274.46 1339.52 65.0636 ms
99th percentile latency desc-sort-with-after-timestamp-after-force-merge-1-seg 1363.05 1399.85 36.8024 ms
100th percentile latency desc-sort-with-after-timestamp-after-force-merge-1-seg 1388.51 1468.63 80.1218 ms
50th percentile service time desc-sort-with-after-timestamp-after-force-merge-1-seg 1242.67 1302.31 59.6447 ms
90th percentile service time desc-sort-with-after-timestamp-after-force-merge-1-seg 1273.25 1336.71 63.4593 ms
99th percentile service time desc-sort-with-after-timestamp-after-force-merge-1-seg 1361.55 1398.44 36.883 ms
100th percentile service time desc-sort-with-after-timestamp-after-force-merge-1-seg 1387.09 1467.37 80.2806 ms
error rate desc-sort-with-after-timestamp-after-force-merge-1-seg 0 0 0 %
Min Throughput asc-sort-with-after-timestamp-after-force-merge-1-seg 0.502217 0.50186 -0.00036 ops/s
Mean Throughput asc-sort-with-after-timestamp-after-force-merge-1-seg 0.505823 0.504875 -0.00095 ops/s
Median Throughput asc-sort-with-after-timestamp-after-force-merge-1-seg 0.504046 0.503398 -0.00065 ops/s
Max Throughput asc-sort-with-after-timestamp-after-force-merge-1-seg 0.523112 0.519269 -0.00384 ops/s
50th percentile latency asc-sort-with-after-timestamp-after-force-merge-1-seg 1029.79 1077.89 48.0973 ms
90th percentile latency asc-sort-with-after-timestamp-after-force-merge-1-seg 1050.97 1110.61 59.6473 ms
99th percentile latency asc-sort-with-after-timestamp-after-force-merge-1-seg 1080.42 1186.89 106.465 ms
100th percentile latency asc-sort-with-after-timestamp-after-force-merge-1-seg 1087.54 1199.35 111.813 ms
50th percentile service time asc-sort-with-after-timestamp-after-force-merge-1-seg 1027.87 1075.95 48.084 ms
90th percentile service time asc-sort-with-after-timestamp-after-force-merge-1-seg 1048.44 1108.52 60.0835 ms
99th percentile service time asc-sort-with-after-timestamp-after-force-merge-1-seg 1077.71 1183.87 106.155 ms
100th percentile service time asc-sort-with-after-timestamp-after-force-merge-1-seg 1084.46 1197.97 113.512 ms
error rate asc-sort-with-after-timestamp-after-force-merge-1-seg 0 0 0 %

@Bukhtawar Bukhtawar mentioned this issue Sep 8, 2022
Closed
17 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
distributed framework enhancement Enhancement or improvement to existing feature or request Storage:Durability Issues and PRs related to the durability framework
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@sachinpkale @kartg