Add micro benchmarks. #537

dblock · 2023-10-13T14:36:17Z

Description

Adds a set of benchmarks that are easy to run and iterate with.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: dblock <dblock@amazon.com>

…ghput. Signed-off-by: dblock <dblock@amazon.com>

Signed-off-by: dblock <dblock@amazon.com>

codecov · 2023-10-13T14:38:48Z

Codecov Report

Merging #537 (d65095d) into main (7a638cd) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #537   +/-   ##
=======================================
  Coverage   70.70%   70.70%           
=======================================
  Files          83       83           
  Lines        7857     7857           
=======================================
  Hits         5555     5555           
  Misses       2302     2302

saimedhi

Code tested and working.

saimedhi · 2023-10-13T17:46:11Z

README.md

@@ -26,7 +26,7 @@ For more information, see [opensearch.org](https://opensearch.org/) and the [API

 ## User Guide

-To get started with the OpenSearch Python Client, see [User Guide](https://github.com/opensearch-project/opensearch-py/blob/main/USER_GUIDE.md).
+To get started with the OpenSearch Python Client, see [User Guide](https://github.com/opensearch-project/opensearch-py/blob/main/USER_GUIDE.md). This repository also contains [working samples](samples) and [benchmarks](benchmarks).


@dblock, Please give complete paths for samples and benchmarks like it is for user_guide. To prevent link checker failure.

Changed it for this PR, however that feels incorrect because when you're on a branch, you end up navigating to main which may have content that has evolved, or even code moved which then will break the link.

@dblock, I agree

I opened #538. Note that even with the full link it doesn't pass checking because that URL won't exist until the code is merged.

saimedhi · 2023-10-13T17:50:07Z

benchmarks/README.md

+│                sync vs. async (8) │ 3.198   │ 3.198   │ 3.198   │ 0.966 (3.3x)    │ 0.966 (3.3x)    │ 0.966 (3.3x)    │
+└───────────────────────────────────┴─────────┴─────────┴─────────┴─────────────────┴─────────────────┴─────────────────┘
+```
+


Could you please put the commands within a single code block for easier copying? The output can be placed in a separate block or outside of the code block. Thank you!

saimedhi · 2023-10-13T17:51:33Z

benchmarks/pyproject.toml

+[tool.poetry]
+name = "package"
+version = "0.1.0"
+description = "OpenSearch samples."


Should this be "OpenSearch benchmarks."?

Signed-off-by: dblock <dblock@amazon.com>

Signed-off-by: Djcarrillo6 <djcarrillo6@yahoo.com> Updated CHANGELOG Signed-off-by: Djcarrillo6 <djcarrillo6@yahoo.com> Updated CHANGELOG & link to sample. Signed-off-by: Djcarrillo6 <djcarrillo6@yahoo.com> updated changelog (opensearch-project#522) Signed-off-by: saimedhi <saimedhi@amazon.com> Bump version to 2.3.2 (opensearch-project#524) Signed-off-by: saimedhi <saimedhi@amazon.com> Fix: typos. (opensearch-project#526) * Fix: typo. Signed-off-by: dblock <dblock@amazon.com> * Fix: typo. Signed-off-by: dblock <dblock@amazon.com> * Fixed its. Signed-off-by: dblock <dblock@amazon.com> * Added Visual Code settings to .gitignore. Signed-off-by: dblock <dblock@amazon.com> * Added loop type for async client. Signed-off-by: dblock <dblock@amazon.com> --------- Signed-off-by: dblock <dblock@amazon.com> Modified generator to generate api deprecation warnings (opensearch-project#527) Signed-off-by: saimedhi <saimedhi@amazon.com> Generate cat client from API specs (opensearch-project#529) Signed-off-by: saimedhi <saimedhi@amazon.com> Generate cluster client from API specs (opensearch-project#530) Signed-off-by: saimedhi <saimedhi@amazon.com> Added new guide & sample module for using index templates. (opensearch-project#531) Added index_template guide and sample Signed-off-by: Djcarrillo6 <djcarrillo6@yahoo.com> Removed EOL Python3.5 & bumped urllib3 version to patch security vulnerability (opensearch-project#533) Updated CHANGELOG with pull # Updated CHANGELOG with pull # Updated CHANGELOG removed section. Updated CHANGELOG removed section again Signed-off-by: Djcarrillo6 <djcarrillo6@yahoo.com> Align pool_maxsize for different connection pool implementations. (opensearch-project#535) * Align pool_maxsize for different connection pool implementations. Signed-off-by: dblock <dblock@amazon.com> * Document connection classes and settings. Signed-off-by: dblock <dblock@amazon.com> * Undo change in async for backwards compatibility. Signed-off-by: dblock <dblock@amazon.com> * Fix: typo. Signed-off-by: dblock <dblock@amazon.com> --------- Signed-off-by: dblock <dblock@amazon.com> Add micro benchmarks. (opensearch-project#537) * Align pool_maxsize for different connection pool implementations. Signed-off-by: dblock <dblock@amazon.com> * Added benchmarks. Signed-off-by: dblock <dblock@amazon.com> * Multi-threaded vs. async benchmarks. Signed-off-by: dblock <dblock@amazon.com> * Set pool size to the number of threads. Signed-off-by: dblock <dblock@amazon.com> * Added sync/async benchmark. Signed-off-by: dblock <dblock@amazon.com> * Report client-side latency. Signed-off-by: dblock <dblock@amazon.com> * Various updates to benchmarks, demonstrating threading improves throughput. Signed-off-by: dblock <dblock@amazon.com> * Bench info. Signed-off-by: dblock <dblock@amazon.com> * Fixup format. Signed-off-by: dblock <dblock@amazon.com> * Undo async maxsize. Signed-off-by: dblock <dblock@amazon.com> * Moved benchmarks folder. Signed-off-by: dblock <dblock@amazon.com> * Updated documentation and project description. Signed-off-by: dblock <dblock@amazon.com> --------- Signed-off-by: dblock <dblock@amazon.com>

* Align pool_maxsize for different connection pool implementations. Signed-off-by: dblock <dblock@amazon.com> * Added benchmarks. Signed-off-by: dblock <dblock@amazon.com> * Multi-threaded vs. async benchmarks. Signed-off-by: dblock <dblock@amazon.com> * Set pool size to the number of threads. Signed-off-by: dblock <dblock@amazon.com> * Added sync/async benchmark. Signed-off-by: dblock <dblock@amazon.com> * Report client-side latency. Signed-off-by: dblock <dblock@amazon.com> * Various updates to benchmarks, demonstrating threading improves throughput. Signed-off-by: dblock <dblock@amazon.com> * Bench info. Signed-off-by: dblock <dblock@amazon.com> * Fixup format. Signed-off-by: dblock <dblock@amazon.com> * Undo async maxsize. Signed-off-by: dblock <dblock@amazon.com> * Moved benchmarks folder. Signed-off-by: dblock <dblock@amazon.com> * Updated documentation and project description. Signed-off-by: dblock <dblock@amazon.com> --------- Signed-off-by: dblock <dblock@amazon.com> Signed-off-by: roma2023 <romasaparhan19@gmail.com>

dblock added 10 commits October 13, 2023 10:22

Align pool_maxsize for different connection pool implementations.

bfffb78

Signed-off-by: dblock <dblock@amazon.com>

Added benchmarks.

6da4eea

Signed-off-by: dblock <dblock@amazon.com>

Multi-threaded vs. async benchmarks.

39f8830

Signed-off-by: dblock <dblock@amazon.com>

Set pool size to the number of threads.

c7e388d

Signed-off-by: dblock <dblock@amazon.com>

Added sync/async benchmark.

1f0b464

Signed-off-by: dblock <dblock@amazon.com>

Report client-side latency.

be14447

Signed-off-by: dblock <dblock@amazon.com>

Various updates to benchmarks, demonstrating threading improves throu…

5a4584a

…ghput. Signed-off-by: dblock <dblock@amazon.com>

Bench info.

8d242f8

Signed-off-by: dblock <dblock@amazon.com>

Fixup format.

4f22f3c

Signed-off-by: dblock <dblock@amazon.com>

Undo async maxsize.

fcfd6a4

Signed-off-by: dblock <dblock@amazon.com>

dblock marked this pull request as ready for review October 13, 2023 14:36

dblock requested review from VachaShah, harshavamsi, axeoman, deztructor, Shephalimittal, saimedhi and florianvazelle as code owners October 13, 2023 14:36

dblock mentioned this pull request Oct 13, 2023

Improve performance in sync client #446

Open

Moved benchmarks folder.

e2ccdd0

Signed-off-by: dblock <dblock@amazon.com>

dblock force-pushed the benchmarks branch from f551c7f to e2ccdd0 Compare October 13, 2023 14:38

saimedhi requested changes Oct 13, 2023

View reviewed changes

Updated documentation and project description.

d65095d

Signed-off-by: dblock <dblock@amazon.com>

saimedhi approved these changes Oct 13, 2023

View reviewed changes

saimedhi merged commit af263a0 into opensearch-project:main Oct 13, 2023
52 of 53 checks passed

dblock deleted the benchmarks branch October 16, 2023 13:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add micro benchmarks. #537

Add micro benchmarks. #537

dblock commented Oct 13, 2023

codecov bot commented Oct 13, 2023 •

edited

Loading

saimedhi left a comment

saimedhi Oct 13, 2023

dblock Oct 13, 2023

saimedhi Oct 13, 2023

dblock Oct 13, 2023

saimedhi Oct 13, 2023

dblock Oct 13, 2023

saimedhi Oct 13, 2023

dblock Oct 13, 2023

Add micro benchmarks. #537

Add micro benchmarks. #537

Conversation

dblock commented Oct 13, 2023

Description

codecov bot commented Oct 13, 2023 • edited Loading

Codecov Report

saimedhi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Oct 13, 2023 •

edited

Loading