Skip to content

Commit

Permalink
Rename elasticsearch to @searchable & opensearch in v2 transformer (a…
Browse files Browse the repository at this point in the history
…ws-amplify#7775)

* refactor(graphql-searchable-transformer): change Elasticsearch refs to @searchable and OpenSearch

* refactor(graphql-transformer-core): change Elasticsearch refs to @searchable and OpenSearch

* refactor(graphql-transformer-interfaces): change Elasticsearch refs to @searchable and OpenSearch

* refactor(graphql-model-transformer): change Elasticsearch refs to @searchable and OpenSearch

* refactor(graphql-mapping-template): add OpenSearch template that extends from Elasticsearch

* refactor(graphql-mapping-template): create Searchable template and extend for Elasticsearch template

* refactor(graphql-searchable-transformer): use Searchable mapping template

* refactor(graphql-searchable-transformer): update additional refs to Elasticsearch in python script

* refactor(graphql-transformer-common): add OpenSearch to resource constants file

* refactor(graphql-searchable-transformer): swap out Elasticsearch constants with OpenSearch

* refactor(graphql-transformer-core): fix typo

* refactor(graphql-transformer-core): change AMAZON_OPENSEARCH back to AMAZON_ELASTICSEARCH

* refactor(graphql-searchable-transformer): change AMAZON_OPENSEARCH back to AMAZON_ELASTICSEARCH

* refactor(graphql-transformer-interfaces): change AMAZON_OPENSEARCH back to AMAZON_ELASTICSEARCH

* Update packages/amplify-graphql-searchable-transformer/streaming-lambda/python_streaming_function.py

Co-authored-by: Colin Ihrig <cjihrig@gmail.com>

* refactor(graphql-transformer-core): fix rebase

* refactor(graphql-searchable-transformer): change ES_ env vars to OPENSEARCH_

* refactor(graphql-transformer-core): rename elastic search to searchble in code comments

Co-authored-by: Colin Ihrig <cjihrig@gmail.com>
  • Loading branch information
2 people authored and akshbhu committed Aug 15, 2021
1 parent 2c9d5e1 commit 0812c16
Show file tree
Hide file tree
Showing 23 changed files with 410 additions and 384 deletions.
20 changes: 10 additions & 10 deletions packages/amplify-graphql-model-transformer/Readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
Object types that are annotated with `@model` are top level entities in the
generated API. Objects annotated with `@model` are stored in DynamoDB and are
capable of being protected via `@auth`, related to other objects via `@connection`,
and streamed into Elasticsearch via `@searchable`.
and streamed via `@searchable`.

#### Definition

Expand Down Expand Up @@ -298,13 +298,13 @@ your queries efficient but this also comes with additional cost.

### @searchable

The `@searchable` directive handles streaming the data of an `@model` object type to
Elasticsearch and configures search resolvers that search that information.
The `@searchable` directive handles streaming the data of an `@model` object type
and configures search resolvers that search that information.

#### Definition

```graphql
# Streams data from dynamodb into elasticsearch and exposes search capabilities.
# Streams data from dynamodb into opensearch and exposes search capabilities.
directive @searchable(queries: SearchableQueryMap) on OBJECT
input SearchableQueryMap {
search: String
Expand Down Expand Up @@ -339,19 +339,19 @@ type Comment @model {
}
```

And then pass the schema to an instance of the `GraphQLTransform` class with the DynamoDB, Elasticsearch, and Connection transformers enabled:
And then pass the schema to an instance of the `GraphQLTransform` class with the DynamoDB, `@searchable`, and Connection transformers enabled:

```javascript
import GraphQLTransform from 'graphql-transformer-core';
import AppSyncDynamoDBTransformer from 'graphql-dynamodb-transformer';
import AppSyncElasticsearchTransformer from 'graphql-elasticsearch-transformer';
import SearchableModelTransformer from 'amplify-graphql-searchable-transformer';
import AppSyncConnectionTransformer from 'graphql-connection-transformer';
import AppSyncAuthTransformer from 'graphql-auth-transformer';

const transformer = new GraphQLTransform({
transformers: [
new AppSyncDynamoDBTransformer(),
new AppSyncElasticsearchTransformer(),
new SearchableModelTransformer(),
new AppSyncAuthTransformer(),
new AppSyncConnectionTransformer(),
],
Expand All @@ -363,7 +363,7 @@ console.log('Application creation successfully started. It may take a few minute

The `GraphQLTransform` class implements a single `transform()` function that when invoked parses the document, walks the AST, and when a directive such as **@model** is found invokes any relevant transformers.
In this case the transformers were defined for you but the code is structured to make writing custom transformers as simple as possible.
The output of the above code is a full CloudFormation document that defines DynamoDB tables, an Elasticsearch cluster, a lambda function to stream from DynamoDB -> Elasticsearch,
The output of the above code is a full CloudFormation document that defines DynamoDB tables, an OpenSearch cluster, a lambda function to stream from DynamoDB -> OpenSearch,
an AppSync API, AppSync data sources, CRUD resolvers (create, update, delete, get, list, search), resolvers that implement connections between types stored in different DynamoDB tables,
a number of minimally scoped IAM roles,

Expand All @@ -379,9 +379,9 @@ The package contains the core of the library and acts as the entry point to the

This package implements a number of directives that deal with DynamoDB. Out of the box, this implements the **@model** and **connection** directives.

**graphql-elasticsearch-transformer**
**amplify-graphql-searchable-transformer**

This package implements any directives that deal with Elasticsearch. Out of the box, this implements the **@searchable** directive.
This package implements any directives that deal with OpenSearch. Out of the box, this implements the **@searchable** directive.

**graphql-auth-transformer**

Expand Down
6 changes: 3 additions & 3 deletions packages/amplify-graphql-searchable-transformer/Readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,13 @@

### @searchable

The `@searchable` directive handles streaming the data of an `@model` object type to
Elasticsearch and configures search resolvers that search that information.
The `@searchable` directive handles streaming the data of an `@model` object type
and configures search resolvers that search that information.

#### Definition

```graphql
# Streams data from dynamodb into elasticsearch and exposes search capabilities.
# Streams data from dynamodb into opensearch and exposes search capabilities.
directive @searchable(queries: SearchableQueryMap) on OBJECT
input SearchableQueryMap {
search: String
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -204,7 +204,7 @@ test('it generates expected resources', () => {
ApiId: {
Ref: anything(),
},
Name: 'ElasticSearchDataSource',
Name: 'OpenSearchDataSource',
Type: 'AMAZON_ELASTICSEARCH',
ElasticsearchConfig: {
AwsRegion: {
Expand All @@ -214,7 +214,7 @@ test('it generates expected resources', () => {
'Fn::Split': [
':',
{
'Fn::GetAtt': ['ElasticSearchDomain', 'Arn'],
'Fn::GetAtt': ['OpenSearchDomain', 'Arn'],
},
],
},
Expand All @@ -226,14 +226,14 @@ test('it generates expected resources', () => {
[
'https://',
{
'Fn::GetAtt': ['ElasticSearchDomain', 'DomainEndpoint'],
'Fn::GetAtt': ['OpenSearchDomain', 'DomainEndpoint'],
},
],
],
},
},
ServiceRoleArn: {
'Fn::GetAtt': ['ElasticSearchAccessIAMRoleAAA3FF0B', 'Arn'],
'Fn::GetAtt': ['OpenSearchAccessIAMRole6A1D9CC5', 'Arn'],
},
}),
);
Expand Down Expand Up @@ -263,7 +263,7 @@ test('it generates expected resources', () => {
},
'"))\n$util.qr($ctx.stash.put("endpoint", "https://',
{
'Fn::GetAtt': ['ElasticSearchDomain', 'DomainEndpoint'],
'Fn::GetAtt': ['OpenSearchDomain', 'DomainEndpoint'],
},
'"))\n$util.toJson({})',
],
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,15 @@ import { CfnOutput } from '@aws-cdk/core';
import { ResourceConstants } from 'graphql-transformer-common';

export const createStackOutputs = (stack: Construct, endpoint: string, apiId: string, arn: string): void => {
const { ElasticsearchDomainArn, ElasticsearchDomainEndpoint } = ResourceConstants.OUTPUTS;
new CfnOutput(stack, ElasticsearchDomainArn, {
const { OpenSearchDomainArn, OpenSearchDomainEndpoint } = ResourceConstants.OUTPUTS;
new CfnOutput(stack, OpenSearchDomainArn, {
value: arn,
description: 'Elasticsearch instance Domain ARN.',
exportName: Fn.join(':', [apiId, 'GetAtt', 'Elasticsearch', 'DomainArn']).toString(),
description: 'OpenSearch instance Domain ARN.',
exportName: Fn.join(':', [apiId, 'GetAtt', 'OpenSearch', 'DomainArn']).toString(),
});
new CfnOutput(stack, ElasticsearchDomainEndpoint, {
new CfnOutput(stack, OpenSearchDomainEndpoint, {
value: 'https://' + endpoint,
description: 'Elasticsearch instance Domain Endpoint.',
exportName: Fn.join(':', [apiId, 'GetAtt', 'Elasticsearch', 'DomainEndpoint']).toString(),
description: 'OpenSearch instance Domain Endpoint.',
exportName: Fn.join(':', [apiId, 'GetAtt', 'OpenSearch', 'DomainEndpoint']).toString(),
});
};
Original file line number Diff line number Diff line change
Expand Up @@ -3,101 +3,101 @@ import { CfnParameter, Stack } from '@aws-cdk/core';

export function createParametersStack(stack: Stack): Map<string, CfnParameter> {
const {
ElasticsearchAccessIAMRoleName,
ElasticsearchStreamingLambdaHandlerName,
ElasticsearchStreamingLambdaRuntime,
ElasticsearchStreamingFunctionName,
ElasticsearchStreamBatchSize,
ElasticsearchStreamMaximumBatchingWindowInSeconds,
ElasticsearchStreamingIAMRoleName,
ElasticsearchDebugStreamingLambda,
ElasticsearchInstanceCount,
ElasticsearchInstanceType,
ElasticsearchEBSVolumeGB,
OpenSearchAccessIAMRoleName,
OpenSearchStreamingLambdaHandlerName,
OpenSearchStreamingLambdaRuntime,
OpenSearchStreamingFunctionName,
OpenSearchStreamBatchSize,
OpenSearchStreamMaximumBatchingWindowInSeconds,
OpenSearchStreamingIAMRoleName,
OpenSearchDebugStreamingLambda,
OpenSearchInstanceCount,
OpenSearchInstanceType,
OpenSearchEBSVolumeGB,
} = ResourceConstants.PARAMETERS;
return new Map<string, CfnParameter>([
[
ElasticsearchAccessIAMRoleName,
new CfnParameter(stack, ElasticsearchAccessIAMRoleName, {
description: 'The name of the IAM role assumed by AppSync for Elasticsearch.',
default: 'AppSyncElasticsearchRole',
OpenSearchAccessIAMRoleName,
new CfnParameter(stack, OpenSearchAccessIAMRoleName, {
description: 'The name of the IAM role assumed by AppSync for OpenSearch.',
default: 'AppSyncOpenSearchRole',
}),
],

[
ElasticsearchStreamingLambdaHandlerName,
new CfnParameter(stack, ElasticsearchStreamingLambdaHandlerName, {
OpenSearchStreamingLambdaHandlerName,
new CfnParameter(stack, OpenSearchStreamingLambdaHandlerName, {
description: 'The name of the lambda handler.',
default: 'python_streaming_function.lambda_handler',
}),
],

[
ElasticsearchStreamingLambdaRuntime,
new CfnParameter(stack, ElasticsearchStreamingLambdaRuntime, {
OpenSearchStreamingLambdaRuntime,
new CfnParameter(stack, OpenSearchStreamingLambdaRuntime, {
description: `The lambda runtime \
(https://docs.aws.amazon.com/lambda/latest/dg/API_CreateFunction.html#SSS-CreateFunction-request-Runtime)`,
default: 'python3.6',
}),
],

[
ElasticsearchStreamingFunctionName,
new CfnParameter(stack, ElasticsearchStreamingFunctionName, {
OpenSearchStreamingFunctionName,
new CfnParameter(stack, OpenSearchStreamingFunctionName, {
description: 'The name of the streaming lambda function.',
default: 'DdbToEsFn',
}),
],

[
ElasticsearchStreamBatchSize,
new CfnParameter(stack, ElasticsearchStreamBatchSize, {
description: 'The maximum number of records to stream to Elasticsearch per batch.',
OpenSearchStreamBatchSize,
new CfnParameter(stack, OpenSearchStreamBatchSize, {
description: 'The maximum number of records to stream to OpenSearch per batch.',
type: 'Number',
default: 100,
}),
],

[
ElasticsearchStreamMaximumBatchingWindowInSeconds,
new CfnParameter(stack, ElasticsearchStreamMaximumBatchingWindowInSeconds, {
OpenSearchStreamMaximumBatchingWindowInSeconds,
new CfnParameter(stack, OpenSearchStreamMaximumBatchingWindowInSeconds, {
description: 'The maximum amount of time in seconds to wait for DynamoDB stream records before sending to streaming lambda.',
type: 'Number',
default: 1,
}),
],

[
ElasticsearchAccessIAMRoleName,
new CfnParameter(stack, ElasticsearchStreamingIAMRoleName, {
OpenSearchAccessIAMRoleName,
new CfnParameter(stack, OpenSearchStreamingIAMRoleName, {
description: 'The name of the streaming lambda function IAM role.',
default: 'SearchableLambdaIAMRole',
}),
],

[
ElasticsearchDebugStreamingLambda,
new CfnParameter(stack, ElasticsearchDebugStreamingLambda, {
description: 'Enable debug logs for the Dynamo -> ES streaming lambda.',
OpenSearchDebugStreamingLambda,
new CfnParameter(stack, OpenSearchDebugStreamingLambda, {
description: 'Enable debug logs for the Dynamo -> OpenSearch streaming lambda.',
default: 1,
type: 'Number',
allowedValues: ['0', '1'],
}),
],

[
ElasticsearchInstanceCount,
new CfnParameter(stack, ElasticsearchInstanceCount, {
description: 'The number of instances to launch into the Elasticsearch domain.',
OpenSearchInstanceCount,
new CfnParameter(stack, OpenSearchInstanceCount, {
description: 'The number of instances to launch into the OpenSearch domain.',
default: 1,
type: 'Number',
}),
],

[
ElasticsearchInstanceType,
new CfnParameter(stack, ElasticsearchInstanceType, {
description: 'The type of instance to launch into the Elasticsearch domain.',
OpenSearchInstanceType,
new CfnParameter(stack, OpenSearchInstanceType, {
description: 'The type of instance to launch into the OpenSearch domain.',
default: 't2.small.elasticsearch',
allowedValues: [
't2.small.elasticsearch',
Expand Down Expand Up @@ -140,8 +140,8 @@ export function createParametersStack(stack: Stack): Map<string, CfnParameter> {
],

[
ElasticsearchEBSVolumeGB,
new CfnParameter(stack, ElasticsearchEBSVolumeGB, {
OpenSearchEBSVolumeGB,
new CfnParameter(stack, OpenSearchEBSVolumeGB, {
description: 'The size in GB of the EBS volumes that contain our data.',
default: 10,
type: 'Number',
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -5,23 +5,23 @@ import { ResourceConstants } from 'graphql-transformer-common';
import assert from 'assert';
import { Stack } from '@aws-cdk/core';

export const createEsDataSource = (
export const createSearchableDataSource = (
stack: Stack,
graphqlApiProvider: GraphQLAPIProvider,
domainEndpoint: string,
role: IRole,
region?: string,
): BaseDataSource => {
const { ElasticsearchDataSourceLogicalID } = ResourceConstants.RESOURCES;
const { OpenSearchDataSourceLogicalID } = ResourceConstants.RESOURCES;
assert(region);
const dsEndpoint = 'https://' + domainEndpoint;
return graphqlApiProvider.host.addElasticSearchDataSource(
ElasticsearchDataSourceLogicalID,
return graphqlApiProvider.host.addSearchableDataSource(
OpenSearchDataSourceLogicalID,
region,
dsEndpoint,
{
serviceRole: role,
name: ElasticsearchDataSourceLogicalID,
name: OpenSearchDataSourceLogicalID,
},
stack,
);
Expand Down
Loading

0 comments on commit 0812c16

Please sign in to comment.