Added new guide & sample module for using index templates. #531

Djcarrillo6 · 2023-10-11T04:28:46Z

Description:

Added a new guide for using index templates with the Python client.

The key sections of the guide include:
• Setup: Instructions for setting up a connection to an OpenSearch server running on the local machine.
• Index Template API Actions: Details on how to interact with the Index Template API to create and use index templates that match certain patterns. It demonstrates scenarios of using single and multiple templates, and how template priority affects the settings applied.
• Composable Index Templates: Information about the newer type of template, composable index templates, which allow for composing multiple component templates into a final one.
• Get an Index Template: Guidance on how to retrieve specifications of a particular index template.
• Delete an Index Template: Steps to delete a specified index template.
• Cleanup: Demonstrates how to delete created indices and templates for clean-up

Issues Resolved

Part of #351

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here

Djcarrillo6 · 2023-10-11T05:44:59Z

@saimedhi I'm from the OSCI & this is my first time contributing to this repo. I was originally assigned to the .NET client, however after seeing you comment in the #osci-client channel regarding there being many good first issues open on the Python client repo, I requested & received assignment on this issue.

I believe my commit only has the four added/modified files associated with the new guide, however this test is referencing a package vulnerability even though I didn't modify anything directly outside of the files in my commit. Can you advise me on how to resolve the issue that caused the WhiteSource Security Check to fail?

dblock

Looks good!

Some nits that you may or may not want to fix.

Let's link the working sample from the guide?

dblock · 2023-10-11T14:46:10Z

guides/index_template.md

+## Setup
+
+Assuming you have OpenSearch running locally on port 9200, you can create a client instance
+with the following code:


No need to line-wrap in markdown.

I have removed the line-wrap from the MD file, and pushed the changes.

dblock · 2023-10-11T14:46:47Z

guides/index_template.md

+```python
+from opensearchpy import OpenSearch
+client = OpenSearch(
+    hosts=['https://admin:admin@localhost:9200'],


Separate auth, http_auth=..., it's not a great practice to include that in the URL.

I have added the http_auth param and removed "admin:admin" from the hosts param. This change has also been pushed.

codecov · 2023-10-11T16:06:45Z

Codecov Report

Merging #531 (b450075) into main (ce835fb) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #531   +/-   ##
=======================================
  Coverage   70.72%   70.72%           
=======================================
  Files          83       83           
  Lines        7852     7852           
=======================================
  Hits         5553     5553           
  Misses       2299     2299

Djcarrillo6 · 2023-10-11T16:08:33Z

@dblock I have made the changes you suggested. Is there anything else you'd like me to address?

saimedhi · 2023-10-11T16:13:18Z

@saimedhi I'm from the OSCI & this is my first time contributing to this repo. I was originally assigned to the .NET client, however after seeing you comment in the #osci-client channel regarding there being many good first issues open on the Python client repo, I requested & received assignment on this issue.

I believe my commit only has the four added/modified files associated with the new guide, however this test is referencing a package vulnerability even though I didn't modify anything directly outside of the files in my commit. Can you advise me on how to resolve the issue that caused the WhiteSource Security Check to fail?

Hello @Djcarrillo6,

The WhiteSource Security Check failure isn't related to this PR. However, you can work on that issue next if you prefer. We currently face an issue with the urllib3 version used by opensearch-py. To upgrade to the latest urllib3 version, we must remove end-of-life Python versions from this repository.

Relevant issues: #430

saimedhi

Code tested and working

saimedhi · 2023-10-11T16:23:34Z

CHANGELOG.md

@@ -125,6 +125,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 - Compatibility with OpenSearch 2.1.0 - 2.4.1 ([#257](https://github.com/opensearch-project/opensearch-py/pull/257))
 - Added explicit parameters for `AIOHttpConnection` and `AsyncTransport` ([#276](https://github.com/opensearch-project/opensearch-py/pull/276))
 - Added support for a custom signing service name for AWS SigV4 ([#268](https://github.com/opensearch-project/opensearch-py/pull/268))
+- Added new guide for using index templates with the client ([#351](https://github.com/opensearch-project/opensearch-py/issues/351))


@Djcarrillo6 Please write your PR details instead of issue details in the unreleased - added section

@saimedhi this has been fixed and pushed.

saimedhi · 2023-10-11T16:32:08Z

USER_GUIDE.md

@@ -153,6 +153,7 @@ print(response)
 - [Search](guides/search.md)
 - [Point in Time](guides/point_in_time.md)
 - [Using a Proxy](guides/proxy.md)
+- [Using Index Templates](guides/index_template.md)


I think only Index Templates is enough :)

@saimedhi This has been fixed and pushed.

saimedhi · 2023-10-11T16:49:30Z

CHANGELOG.md

@@ -125,6 +125,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 - Compatibility with OpenSearch 2.1.0 - 2.4.1 ([#257](https://github.com/opensearch-project/opensearch-py/pull/257))
 - Added explicit parameters for `AIOHttpConnection` and `AsyncTransport` ([#276](https://github.com/opensearch-project/opensearch-py/pull/276))
 - Added support for a custom signing service name for AWS SigV4 ([#268](https://github.com/opensearch-project/opensearch-py/pull/268))
+- Added new guide for using index templates with the client ([#351](https://github.com/opensearch-project/opensearch-py/pull/531))


Please change 351 to 531

saimedhi · 2023-10-11T16:55:38Z

@Djcarrillo6, sorry 1 final change required as mentioned above please add changelog entry in unreleased added section. Not in 2.0.1

Signed-off-by: Djcarrillo6 <djcarrillo6@yahoo.com> Fixed line wrapping & connect params in index_template.md Signed-off-by: Djcarrillo6 <djcarrillo6@yahoo.com> Linked sample code to index_template.md file Signed-off-by: Djcarrillo6 <djcarrillo6@yahoo.com> Fixed broken link in to sample in index_template.md Signed-off-by: Djcarrillo6 <djcarrillo6@yahoo.com> Added http_auth param to the index_template_sample.py module Signed-off-by: Djcarrillo6 <djcarrillo6@yahoo.com> Fixed MD files. Signed-off-by: Djcarrillo6 <djcarrillo6@yahoo.com> Fixed MD files again Signed-off-by: Djcarrillo6 <djcarrillo6@yahoo.com> Fixed MD files again 2 Signed-off-by: Djcarrillo6 <djcarrillo6@yahoo.com>

Djcarrillo6 · 2023-10-11T16:59:26Z

@saimedhi Moved my CHANGELOG entry to unreleased section

saimedhi · 2023-10-11T17:06:05Z

@Djcarrillo6,

If you're interested in fixing the detected security vulnerability, it's a straightforward process. To bump urllib3 to the latest version, simply remove Python version 3.5 from noxfile.py and .github/workflows/test.yml. Before that just comment on this issue #430 saying removing end of life python version because of security vulnerability.
You can follow this PR for guidance on the urllib3 upgrade.

Thank you.

Signed-off-by: Djcarrillo6 <djcarrillo6@yahoo.com> Updated CHANGELOG Signed-off-by: Djcarrillo6 <djcarrillo6@yahoo.com> Updated CHANGELOG & link to sample. Signed-off-by: Djcarrillo6 <djcarrillo6@yahoo.com> updated changelog (opensearch-project#522) Signed-off-by: saimedhi <saimedhi@amazon.com> Bump version to 2.3.2 (opensearch-project#524) Signed-off-by: saimedhi <saimedhi@amazon.com> Fix: typos. (opensearch-project#526) * Fix: typo. Signed-off-by: dblock <dblock@amazon.com> * Fix: typo. Signed-off-by: dblock <dblock@amazon.com> * Fixed its. Signed-off-by: dblock <dblock@amazon.com> * Added Visual Code settings to .gitignore. Signed-off-by: dblock <dblock@amazon.com> * Added loop type for async client. Signed-off-by: dblock <dblock@amazon.com> --------- Signed-off-by: dblock <dblock@amazon.com> Modified generator to generate api deprecation warnings (opensearch-project#527) Signed-off-by: saimedhi <saimedhi@amazon.com> Generate cat client from API specs (opensearch-project#529) Signed-off-by: saimedhi <saimedhi@amazon.com> Generate cluster client from API specs (opensearch-project#530) Signed-off-by: saimedhi <saimedhi@amazon.com> Added new guide & sample module for using index templates. (opensearch-project#531) Added index_template guide and sample Signed-off-by: Djcarrillo6 <djcarrillo6@yahoo.com> Removed EOL Python3.5 & bumped urllib3 version to patch security vulnerability (opensearch-project#533) Updated CHANGELOG with pull # Updated CHANGELOG with pull # Updated CHANGELOG removed section. Updated CHANGELOG removed section again Signed-off-by: Djcarrillo6 <djcarrillo6@yahoo.com> Align pool_maxsize for different connection pool implementations. (opensearch-project#535) * Align pool_maxsize for different connection pool implementations. Signed-off-by: dblock <dblock@amazon.com> * Document connection classes and settings. Signed-off-by: dblock <dblock@amazon.com> * Undo change in async for backwards compatibility. Signed-off-by: dblock <dblock@amazon.com> * Fix: typo. Signed-off-by: dblock <dblock@amazon.com> --------- Signed-off-by: dblock <dblock@amazon.com> Add micro benchmarks. (opensearch-project#537) * Align pool_maxsize for different connection pool implementations. Signed-off-by: dblock <dblock@amazon.com> * Added benchmarks. Signed-off-by: dblock <dblock@amazon.com> * Multi-threaded vs. async benchmarks. Signed-off-by: dblock <dblock@amazon.com> * Set pool size to the number of threads. Signed-off-by: dblock <dblock@amazon.com> * Added sync/async benchmark. Signed-off-by: dblock <dblock@amazon.com> * Report client-side latency. Signed-off-by: dblock <dblock@amazon.com> * Various updates to benchmarks, demonstrating threading improves throughput. Signed-off-by: dblock <dblock@amazon.com> * Bench info. Signed-off-by: dblock <dblock@amazon.com> * Fixup format. Signed-off-by: dblock <dblock@amazon.com> * Undo async maxsize. Signed-off-by: dblock <dblock@amazon.com> * Moved benchmarks folder. Signed-off-by: dblock <dblock@amazon.com> * Updated documentation and project description. Signed-off-by: dblock <dblock@amazon.com> --------- Signed-off-by: dblock <dblock@amazon.com>

…h-project#531) Added index_template guide and sample Signed-off-by: Djcarrillo6 <djcarrillo6@yahoo.com>

Djcarrillo6 requested review from VachaShah, dblock, harshavamsi, axeoman, deztructor, Shephalimittal, saimedhi and florianvazelle as code owners October 11, 2023 04:28

dblock previously approved these changes Oct 11, 2023

View reviewed changes

Djcarrillo6 dismissed dblock’s stale review via 3806db6 October 11, 2023 16:04

Djcarrillo6 force-pushed the guide_index_template branch from 8c447a2 to 3806db6 Compare October 11, 2023 16:04

saimedhi added the OSCI-2023 label Oct 11, 2023

Djcarrillo6 force-pushed the guide_index_template branch 2 times, most recently from 1cc7a23 to ba3d76a Compare October 11, 2023 16:29

saimedhi requested changes Oct 11, 2023

View reviewed changes

Djcarrillo6 force-pushed the guide_index_template branch from ba3d76a to ab024cf Compare October 11, 2023 16:47

saimedhi reviewed Oct 11, 2023

View reviewed changes

Djcarrillo6 force-pushed the guide_index_template branch from ab024cf to 9305a0e Compare October 11, 2023 16:51

Djcarrillo6 force-pushed the guide_index_template branch from 9305a0e to b450075 Compare October 11, 2023 16:58

saimedhi approved these changes Oct 11, 2023

View reviewed changes

saimedhi merged commit 84ac172 into opensearch-project:main Oct 11, 2023
53 of 54 checks passed

roma2023 pushed a commit to roma2023/opensearch-py that referenced this pull request Dec 28, 2023

Added new guide & sample module for using index templates. (opensearc…

781c0ff

…h-project#531) Added index_template guide and sample Signed-off-by: Djcarrillo6 <djcarrillo6@yahoo.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added new guide & sample module for using index templates. #531

Added new guide & sample module for using index templates. #531

Djcarrillo6 commented Oct 11, 2023

Djcarrillo6 commented Oct 11, 2023

dblock left a comment

dblock Oct 11, 2023

Djcarrillo6 Oct 11, 2023

dblock Oct 11, 2023

Djcarrillo6 Oct 11, 2023

codecov bot commented Oct 11, 2023 •

edited

Loading

Djcarrillo6 commented Oct 11, 2023 •

edited

Loading

saimedhi commented Oct 11, 2023

saimedhi left a comment

saimedhi Oct 11, 2023

Djcarrillo6 Oct 11, 2023

saimedhi Oct 11, 2023

Djcarrillo6 Oct 11, 2023

saimedhi Oct 11, 2023

saimedhi commented Oct 11, 2023

Djcarrillo6 commented Oct 11, 2023

saimedhi commented Oct 11, 2023

Added new guide & sample module for using index templates. #531

Added new guide & sample module for using index templates. #531

Conversation

Djcarrillo6 commented Oct 11, 2023

Description:

Issues Resolved

Djcarrillo6 commented Oct 11, 2023

dblock left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Oct 11, 2023 • edited Loading

Codecov Report

Djcarrillo6 commented Oct 11, 2023 • edited Loading

saimedhi commented Oct 11, 2023

saimedhi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

saimedhi commented Oct 11, 2023

Djcarrillo6 commented Oct 11, 2023

saimedhi commented Oct 11, 2023

codecov bot commented Oct 11, 2023 •

edited

Loading

Djcarrillo6 commented Oct 11, 2023 •

edited

Loading