Skip to content

Commit

Permalink
Add support for component template APIs (#411) (#436)
Browse files Browse the repository at this point in the history
* Generate `cluster.put_component_template`

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Implement high-level DeleteComponentTemplate

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Implement high-level ComponentTemplateExists

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Fix license

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Remove old put_component_template

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Implement high-level GetComponentTemplate

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add tests for ComponentTemplateExists

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add tests for DeleteComponentTemplate

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add tests for GetComponentTemplate

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Implement high-level PutComponentTemplate

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add component template CRUD tests

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add naming convention exception for ComponentTemplateExists

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Update guide

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Formatting

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Add changelog entry

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Fix license header

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Fix namespaces

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

* Fix license header

Signed-off-by: Thomas Farr <tsfarr@amazon.com>

---------

Signed-off-by: Thomas Farr <tsfarr@amazon.com>
(cherry picked from commit 7eedc00)
  • Loading branch information
Xtansia authored Nov 21, 2023
1 parent e25c8a4 commit 40923b4
Show file tree
Hide file tree
Showing 31 changed files with 2,201 additions and 193 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
## [Unreleased]
### Added
- Added support for point-in-time search and associated APIs ([#405](https://github.com/opensearch-project/opensearch-net/pull/405))
- Added support for the component template APIs ([#411](https://github.com/opensearch-project/opensearch-net/pull/411))

### Dependencies
- Bumps `FSharp.Data` from 6.2.0 to 6.3.0
Expand Down
205 changes: 205 additions & 0 deletions guides/index-template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,205 @@
# Index Template
Index templates allow you to define default settings, mappings, and aliases for one or more indices during their creation. This guide will teach you how to create index templates and apply them to indices using the OpenSearch .NET client.

## Setup
**At the time of writing the API methods related to composable templates do not yet exist in the high-level client, as such this guide makes use of their low-level counterparts.**


Assuming you have OpenSearch running locally on port 9200, you can create a client instance with the following code:

```csharp
using OpenSearch.Client;
using OpenSearch.Net;

var node = new Uri("https://localhost:9200");
var config = new ConnectionSettings(node)
.ServerCertificateValidationCallback(CertificateValidations.AllowAll)
.BasicAuthentication("admin", "admin");

var client = new OpenSearchClient(config);;
```


## Index Template API Actions


### Create an Index Template
You can create an index template to define default settings and mappings for indices of certain patterns. The following example creates an index template named `books` with default settings and mappings for indices of the `books-*` pattern:

```csharp
client.LowLevel.Indices.PutTemplateV2ForAll<VoidResponse>("books", PostData.Serializable(new
{
index_patterns = new[] { "books-*" },
priority = 0,
template = new
{
settings = new
{
index = new
{
number_of_shards = 3,
number_of_replicas = 0
}
},
mappings = new
{
properties = new
{
title = new { type = "text" },
author = new { type = "text" },
published_on = new { type = "date" },
pages = new { type = "integer" }
}
}
}
}));
```

Now, when you create an index that matches the `books-*` pattern, OpenSearch will automatically apply the template's settings and mappings to the index. Let's create an index named `books-nonfiction` and verify that its settings and mappings match those of the template:

```csharp
client.Indices.Create("books-nonfiction");
var getResponse = client.Indices.Get("books-nonfiction");
Console.WriteLine(getResponse.Indices["books-nonfiction"].Mappings.Properties["pages"].Type); // integer
```


### Multiple Index Templates

```csharp
var createResponseOne = client.LowLevel.Indices.PutTemplateV2ForAll<VoidResponse>("books", PostData.Serializable(new
{
index_patterns = new[] { "books-*" },
priority = 0,
template = new
{
settings = new
{
index = new
{
number_of_shards = 3,
number_of_replicas = 0
}
}
}
}));

client.LowLevel.Indices.PutTemplateV2ForAll<VoidResponse>("books-fiction", PostData.Serializable(new
{
index_patterns = new[] { "books-fiction-*" },
priority = 1, // higher priority than the `books` template
template = new
{
settings = new
{
index = new
{
number_of_shards = 1,
number_of_replicas = 1
}
}
}
}));
```

When we create an index named `books-fiction-romance`, OpenSearch will apply the `books-fiction-*` template's settings to the index:

```csharp
client.Indices.Create("books-fiction-romance");
var getResponse = client.Indices.Get("books-fiction-romance");
Console.WriteLine(getResponse.Indices["books-fiction-romance"].Settings.NumberOfShards); // 1
```


### Composable Index Templates
Composable index templates are a new type of index template that allow you to define multiple component templates and compose them into a final template. The following example creates a component template named `books_mappings` with default mappings for indices of the `books-*` and `books-fiction-*` patterns:

```csharp
// Create a component template
client.Cluster.PutComponentTemplate("books_mappings", ct => ct
.Template(t => t
.Map(m => m
.Properties(p => p
.Text(tp => tp
.Name("title"))
.Text(tp => tp
.Name("author"))
.Date(d => d
.Name("published_on"))
.Number(n => n
.Name("pages")
.Type(NumberType.Integer))))));

// Create an index template for "books"
var createBooksTemplateResponse = client.LowLevel.Indices.PutTemplateV2ForAll<VoidResponse>("books", PostData.Serializable(new
{
index_patterns = new[] { "books-*" },
composed_of = new[] { "books_mappings" },
priority = 0,
template = new
{
settings = new
{
index = new
{
number_of_shards = 3,
number_of_replicas = 0
}
}
}
}));

// Create an index template for "books-fiction"
var createBooksFictionTemplateResponse = client.LowLevel.Indices.PutTemplateV2ForAll<VoidResponse>("books-fiction", PostData.Serializable(new
{
index_patterns = new[] { "books-fiction-*" },
composed_of = new[] { "books_mappings" },
priority = 1,
template = new
{
settings = new
{
index = new
{
number_of_shards = 1,
number_of_replicas = 1
}
}
}
}));
```

When we create an index named `books-fiction-horror`, OpenSearch will apply the `books-fiction-*` template's settings, and `books_mappings` template mappings to the index:

```csharp
client.Indices.Create("books-fiction-horror");
var getResponse = client.Indices.Get("books-fiction-horror");
Console.WriteLine(getResponse.Indices["books-fiction-horror"].Settings.NumberOfShards); // 1
Console.WriteLine(getResponse.Indices["books-fiction-horror"].Mappings.Properties["pages"].Type); // integer
```

### Get an Index Template
You can get an index template with the `GetTemplateV2ForAll` API action. The following example gets the `books` index template:

```csharp
var getResponse = client.LowLevel.Indices.GetTemplateV2ForAll<StringResponse>("books").Body;
Console.WriteLine($"Get response: {getResponse}"); // Get response: {"books":{"order":0,"index_patterns":["books-*"],"settings":{"index":{"number_of_shards":"3","number_of_replicas":"0"}},"mappings":{},"aliases":{}}}
```

### Delete an Index Template
You can delete an index template with the `DeleteTemplateV2ForAll` API action. The following example deletes the `books` index template:

```csharp
var deleteResponse = client.LowLevel.Indices.DeleteTemplateV2ForAll<VoidResponse>("books");
Console.WriteLine($"Delete response: {deleteResponse}"); // Delete response: {"acknowledged":true}
```


## Cleanup
Let's delete all resources created in this guide:

```csharp
client.Indices.Delete("books-");
client.LowLevel.Indices.DeleteTemplateV2ForAll("books-fiction");
client.Cluster.DeleteComponentTemplate("books_mappings");
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
/* SPDX-License-Identifier: Apache-2.0
*
* The OpenSearch Contributors require contributions made to
* this file be licensed under the Apache-2.0 license or a
* compatible open source license.
*/

using System;
using System.Collections.Generic;
using System.Runtime.Serialization;
using OpenSearch.Net.Utf8Json;

namespace OpenSearch.Client;

[ReadAs(typeof(ComponentTemplate))]
public interface IComponentTemplate
{
[DataMember(Name = "template")]
ITemplate Template { get; set; }

[DataMember(Name = "version")]
long? Version { get; set; }

[DataMember(Name = "_meta")]
[JsonFormatter(typeof(VerbatimDictionaryInterfaceKeysFormatter<string, object>))]
IDictionary<string, object> Meta { get; set; }
}

public class ComponentTemplate : IComponentTemplate
{
public ITemplate Template { get; set; }
public long? Version { get; set; }
public IDictionary<string, object> Meta { get; set; }
}

[ReadAs(typeof(Template))]
public interface ITemplate
{
[DataMember(Name = "aliases")]
IAliases Aliases { get; set; }

[DataMember(Name = "mappings")]
ITypeMapping Mappings { get; set; }

[DataMember(Name = "settings")]
IIndexSettings Settings { get; set; }
}

public class Template : ITemplate
{
public IAliases Aliases { get; set; }
public ITypeMapping Mappings { get; set; }
public IIndexSettings Settings { get; set; }
}

public class TemplateDescriptor : DescriptorBase<TemplateDescriptor, ITemplate>, ITemplate
{
IAliases ITemplate.Aliases { get; set; }
ITypeMapping ITemplate.Mappings { get; set; }
IIndexSettings ITemplate.Settings { get; set; }

public TemplateDescriptor Aliases(Func<AliasesDescriptor, IPromise<IAliases>> aliasDescriptor) =>
Assign(aliasDescriptor, (a, v) => a.Aliases = v?.Invoke(new AliasesDescriptor())?.Value);

public TemplateDescriptor Map<T>(Func<TypeMappingDescriptor<T>, ITypeMapping> selector) where T : class =>
Assign(selector, (a, v) => a.Mappings = v?.Invoke(new TypeMappingDescriptor<T>()));

public TemplateDescriptor Map(Func<TypeMappingDescriptor<object>, ITypeMapping> selector) =>
Assign(selector, (a, v) => a.Mappings = v?.Invoke(new TypeMappingDescriptor<object>()));

public TemplateDescriptor Settings(Func<IndexSettingsDescriptor, IPromise<IIndexSettings>> settingsSelector) =>
Assign(settingsSelector, (a, v) => a.Settings = v?.Invoke(new IndexSettingsDescriptor())?.Value);
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
/* SPDX-License-Identifier: Apache-2.0
*
* The OpenSearch Contributors require contributions made to
* this file be licensed under the Apache-2.0 license or a
* compatible open source license.
*/

namespace OpenSearch.Client;

[MapsApi("cluster.exists_component_template")]
public partial interface IComponentTemplateExistsRequest { }

public partial class ComponentTemplateExistsRequest { }

public partial class ComponentTemplateExistsDescriptor { }
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
/* SPDX-License-Identifier: Apache-2.0
*
* The OpenSearch Contributors require contributions made to
* this file be licensed under the Apache-2.0 license or a
* compatible open source license.
*/

namespace OpenSearch.Client;

[MapsApi("cluster.delete_component_template")]
public partial interface IDeleteComponentTemplateRequest { }

public partial class DeleteComponentTemplateRequest { }

public partial class DeleteComponentTemplateDescriptor { }
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
/* SPDX-License-Identifier: Apache-2.0
*
* The OpenSearch Contributors require contributions made to
* this file be licensed under the Apache-2.0 license or a
* compatible open source license.
*/

using System.Runtime.Serialization;

namespace OpenSearch.Client;

[DataContract]
public class DeleteComponentTemplateResponse : AcknowledgedResponseBase { }
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
/* SPDX-License-Identifier: Apache-2.0
*
* The OpenSearch Contributors require contributions made to
* this file be licensed under the Apache-2.0 license or a
* compatible open source license.
*/

namespace OpenSearch.Client;

[MapsApi("cluster.get_component_template")]
public partial interface IGetComponentTemplateRequest { }

public partial class GetComponentTemplateRequest { }

public partial class GetComponentTemplateDescriptor { }
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
/* SPDX-License-Identifier: Apache-2.0
*
* The OpenSearch Contributors require contributions made to
* this file be licensed under the Apache-2.0 license or a
* compatible open source license.
*/

using System.Collections.Generic;
using System.Runtime.Serialization;

namespace OpenSearch.Client;

[DataContract]
public class GetComponentTemplateResponse : ResponseBase
{
[DataMember(Name = "component_templates")]
public IReadOnlyCollection<NamedComponentTemplate> ComponentTemplates { get; internal set; }
}

[DataContract]
public class NamedComponentTemplate
{
[DataMember(Name = "name")]
public string Name { get; internal set; }

[DataMember(Name = "component_template")]
public ComponentTemplate ComponentTemplate { get; internal set; }
}
Loading

0 comments on commit 40923b4

Please sign in to comment.