Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added new guide & sample module for using index templates. #531

Merged
merged 1 commit into from
Oct 11, 2023
Merged

Added new guide & sample module for using index templates. #531

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
### Added
- Added generating imports and headers to API generator ([#467](https://github.com/opensearch-project/opensearch-py/pull/467))
- Added point-in-time APIs (create_pit, delete_pit, delete_all_pits, get_all_pits) and Security Client APIs (health and update_audit_configuration) ([#502](https://github.com/opensearch-project/opensearch-py/pull/502))
- Added new guide for using index templates with the client ([#531](https://github.com/opensearch-project/opensearch-py/pull/531))
### Changed
- Generate `tasks` client from API specs ([#508](https://github.com/opensearch-project/opensearch-py/pull/508))
- Generate `ingest` client from API specs ([#513](https://github.com/opensearch-project/opensearch-py/pull/513))
Expand Down
1 change: 1 addition & 0 deletions USER_GUIDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,7 @@ print(response)
- [Search](guides/search.md)
- [Point in Time](guides/point_in_time.md)
- [Using a Proxy](guides/proxy.md)
- [Index Templates](guides/index_template.md)

## Plugins

Expand Down
184 changes: 184 additions & 0 deletions guides/index_template.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,184 @@
# Index Template
Index templates are a convenient way to define settings, mappings, and aliases for one or more indices when they are created. In this guide, you'll learn how to create an index template and apply it to an index.

## Setup

Assuming you have OpenSearch running locally on port 9200, you can create a client instance with the following code:
```python
from opensearchpy import OpenSearch
client = OpenSearch(
hosts=['https://localhost:9200'],
use_ssl=True,
verify_certs=False,
http_auth=('admin', 'admin')
)
```

## Index Template API Actions

### Create an Index Template
You can create an index template to define default settings and mappings for indices of certain patterns. The following example creates an index template named `books` with default settings and mappings for indices of the `books-*` pattern:

```python
client.indices.put_index_template(
name='books',
body={
'index_patterns': ['books-*'],
'template': {
'settings': {
'index': {
'number_of_shards': 3,
'number_of_replicas': 0
}
},
'mappings': {
'properties': {
'title': { 'type': 'text' },
'author': { 'type': 'text' },
'published_on': { 'type': 'date' },
'pages': { 'type': 'integer' }
}
}
}
}
)
```

Now, when you create an index that matches the `books-*` pattern, OpenSearch will automatically apply the template's settings and mappings to the index. Let's create an index named `books-nonfiction` and verify that its settings and mappings match those of the template:

```python
client.indices.create(index='books-nonfiction')
print(client.indices.get(index='books-nonfiction'))
```

### Multiple Index Templates
If multiple index templates match the index's name, OpenSearch will apply the template with the highest priority. The following example creates two index templates named `books-*` and `books-fiction-*` with different settings:

```python
client.indices.put_index_template(
name='books',
body={
'index_patterns': ['books-*'],
'priority': 0, # default priority
'template': {
'settings': {
'index': {
'number_of_shards': 3,
'number_of_replicas': 0
}
}
}
}
)

client.indices.put_index_template(
name='books-fiction',
body={
'index_patterns': ['books-fiction-*'],
'priority': 1, # higher priority than the `books` template
'template': {
'settings': {
'index': {
'number_of_shards': 1,
'number_of_replicas': 1
}
}
}
}
)
```

When we create an index named `books-fiction-romance`, OpenSearch will apply the `books-fiction-*` template's settings to the index:

```python
client.indices.create(index='books-fiction-romance')
print(client.indices.get(index='books-fiction-romance'))
```

### Composable Index Templates
Composable index templates are a new type of index template that allow you to define multiple component templates and compose them into a final template. The following example creates a component template named `books_mappings` with default mappings for indices of the `books-*` and `books-fiction-*` patterns:

```python
client.cluster.put_component_template(
name='books_mappings',
body={
'template': {
'mappings': {
'properties': {
'title': { 'type': 'text' },
'author': { 'type': 'text' },
'published_on': { 'type': 'date' },
'pages': { 'type': 'integer' }
}
}
}
}
)

client.indices.put_index_template(
name='books',
body={
'index_patterns': ['books-*'],
'composed_of': ['books_mappings'], # use the `books_mappings` component template
'priority': 0,
'template': {
'settings': {
'index': {
'number_of_shards': 3,
'number_of_replicas': 0
}
}
}
}
)

client.indices.put_index_template(
name='books',
body={
'index_patterns': ['books-*'],
'composed_of': ['books_mappings'], # use the `books_mappings` component template
'priority': 1,
'template': {
'settings': {
'index': {
'number_of_shards': 1,
'number_of_replicas': 1
}
}
}
}
)
```

When we create an index named `books-fiction-horror`, OpenSearch will apply the `books-fiction-*` template's settings, and `books_mappings` template mappings to the index:

```python
client.indices.create(index='books-fiction-horror')
print(client.indices.get(index='books-fiction-horror'))
```

### Get an Index Template
You can get an index template with the `get_index_template` API action:

```python
print(client.indices.get_index_template(name='books'))
```

### Delete an Index Template
You can delete an index template with the `delete_template` API action:

```python
client.indices.delete_index_template(name='books')
```

## Cleanup
Let's delete all resources created in this guide:

```python
client.indices.delete(index='books-*')
client.indices.delete_index_template(name='books-fiction')
client.cluster.delete_component_template(name='books_mappings')
```

# Sample Code
See [index_template_sample.py](/samples/index_template/index_template_sample.py) for a working sample of the concepts in this guide.
143 changes: 143 additions & 0 deletions samples/index_template/index_template_sample.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
from opensearchpy import OpenSearch

# Create a client instance
client = OpenSearch(
hosts=['https://localhost:9200'],
use_ssl=True,
verify_certs=False,
http_auth=('admin', 'admin')
)

# You can create an index template to define default settings and mappings for indices of certain patterns. The following example creates an index template named `books` with default settings and mappings for indices of the `books-*` pattern:
client.indices.put_index_template(
name='books',
body={
'index_patterns': ['books-*'],
'priority': 1,
'template': {
'settings': {
'index': {
'number_of_shards': 3,
'number_of_replicas': 0
}
},
'mappings': {
'properties': {
'title': { 'type': 'text' },
'author': { 'type': 'text' },
'published_on': { 'type': 'date' },
'pages': { 'type': 'integer' }
}
}
}
}
)

# Now, when you create an index that matches the `books-*` pattern, OpenSearch will automatically apply the template's settings and mappings to the index. Let's create an index named books-nonfiction and verify that its settings and mappings match those of the template:
client.indices.create(index='books-nonfiction')
print(client.indices.get(index='books-nonfiction'))

# If multiple index templates match the index's name, OpenSearch will apply the template with the highest `priority`. The following example creates two index templates named `books-*` and `books-fiction-*` with different settings:
client.indices.put_index_template(
name='books',
body={
'index_patterns': ['books-*'],
'priority': 1,
'template': {
'settings': {
'index': {
'number_of_shards': 3,
'number_of_replicas': 0
}
}
}
}
)

client.indices.put_index_template(
name='books-fiction',
body={
'index_patterns': ['books-fiction-*'],
'priority': 2,
'template': {
'settings': {
'index': {
'number_of_shards': 1,
'number_of_replicas': 1
}
}
}
}
)

# # Test multiple index templates
client.indices.create(index='books-fiction-romance')
print(client.indices.get(index='books-fiction-romance'))


# Composable index templates are a new type of index template that allow you to define multiple component templates and compose them into a final template. The following example creates a component template named `books_mappings` with default mappings for indices of the `books-*` and `books-fiction-*` patterns:
client.cluster.put_component_template(
name='books_mappings',
body={
'template': {
'mappings': {
'properties': {
'title': { 'type': 'text' },
'author': { 'type': 'text' },
'published_on': { 'type': 'date' },
'pages': { 'type': 'integer' }
}
}
}
}
)

client.indices.put_index_template(
name='books',
body={
'index_patterns': ['books-*'],
'composed_of': ['books_mappings'],
'priority': 4,
'template': {
'settings': {
'index': {
'number_of_shards': 3,
'number_of_replicas': 0
}
}
}
}
)

client.indices.put_index_template(
name='books-fiction',
body={
'index_patterns': ['books-fiction-*'],
'composed_of': ['books_mappings'],
'priority': 5,
'template': {
'settings': {
'index': {
'number_of_shards': 1,
'number_of_replicas': 1
}
}
}
}
)


# Test composable index templates
client.indices.create(index='books-fiction-horror')
print(client.indices.get(index='books-fiction-horror'))

# Get an index template
print(client.indices.get_index_template(name='books'))

# Delete an index template
client.indices.delete_index_template(name='books')

# Cleanup
client.indices.delete(index='books-*')
client.indices.delete_index_template(name='books-fiction')
client.cluster.delete_component_template(name='books_mappings')
Loading