-
Notifications
You must be signed in to change notification settings - Fork 249
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(aws-sns-sqs): New aws-sns-sqs pattern implementation #48
Merged
hnishar
merged 8 commits into
awslabs:master
from
danielmatuki:danielmatuki/aws-sns-sqs-construct
Aug 28, 2020
Merged
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
ccd1d9c
feat(aws-sns-sqs): New aws-sns-sqs pattern implementation
danielmatuki c3d902c
Merge branch 'master' into danielmatuki/aws-sns-sqs-construct
danielmatuki 409fdb3
fix(aws-sns-sqs): Update aws-sns-sqs package configuration to utilize…
danielmatuki 3e9cb3d
fix(aws-sns-sqs): Update managed KMS key integration test to utilize …
danielmatuki 26c4ec6
Merge branch 'master' into danielmatuki/aws-sns-sqs-construct
danielmatuki d7ca31a
Merge branch 'master' into danielmatuki/aws-sns-sqs-construct
danielmatuki a2ead48
fix(aws-sns-sqs): code review changes for new aws-sns-sqs construct
danielmatuki dc27900
fix(aws-sns-sqs): code review changes for new aws-sns-sqs construct
danielmatuki File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
5 changes: 5 additions & 0 deletions
5
source/patterns/@aws-solutions-constructs/aws-sns-sqs/.eslintignore
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
lib/*.js | ||
test/*.js | ||
*.d.ts | ||
coverage | ||
test/lambda/index.js |
15 changes: 15 additions & 0 deletions
15
source/patterns/@aws-solutions-constructs/aws-sns-sqs/.gitignore
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
lib/*.js | ||
test/*.js | ||
*.js.map | ||
*.d.ts | ||
node_modules | ||
*.generated.ts | ||
dist | ||
.jsii | ||
|
||
.LAST_BUILD | ||
.nyc_output | ||
coverage | ||
.nycrc | ||
.LAST_PACKAGE | ||
*.snk |
21 changes: 21 additions & 0 deletions
21
source/patterns/@aws-solutions-constructs/aws-sns-sqs/.npmignore
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
# Exclude typescript source and config | ||
*.ts | ||
tsconfig.json | ||
coverage | ||
.nyc_output | ||
*.tgz | ||
*.snk | ||
*.tsbuildinfo | ||
|
||
# Include javascript files and typescript declarations | ||
!*.js | ||
!*.d.ts | ||
|
||
# Exclude jsii outdir | ||
dist | ||
|
||
# Include .jsii | ||
!.jsii | ||
|
||
# Include .jsii | ||
!.jsii |
93 changes: 93 additions & 0 deletions
93
source/patterns/@aws-solutions-constructs/aws-sns-sqs/README.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,93 @@ | ||
# aws-sns-sqs module | ||
<!--BEGIN STABILITY BANNER--> | ||
|
||
--- | ||
|
||
![Stability: Experimental](https://img.shields.io/badge/stability-Experimental-important.svg?style=for-the-badge) | ||
|
||
> All classes are under active development and subject to non-backward compatible changes or removal in any | ||
> future version. These are not subject to the [Semantic Versioning](https://semver.org/) model. | ||
> This means that while you may use them, you may need to update your source code when upgrading to a newer version of this package. | ||
|
||
--- | ||
<!--END STABILITY BANNER--> | ||
|
||
| **Reference Documentation**:| <span style="font-weight: normal">https://docs.aws.amazon.com/solutions/latest/constructs/</span>| | ||
|:-------------|:-------------| | ||
<div style="height:8px"></div> | ||
|
||
| **Language** | **Package** | | ||
|:-------------|-----------------| | ||
|![Python Logo](https://docs.aws.amazon.com/cdk/api/latest/img/python32.png) Python|`aws_solutions_constructs.aws_sns_sqs`| | ||
|![Typescript Logo](https://docs.aws.amazon.com/cdk/api/latest/img/typescript32.png) Typescript|`@aws-solutions-constructs/aws-sns-sqs`| | ||
|![Java Logo](https://docs.aws.amazon.com/cdk/api/latest/img/java32.png) Java|`software.amazon.awsconstructs.services.snssqs`| | ||
|
||
This AWS Solutions Construct implements an Amazon SNS topic connected to an Amazon SQS queue. | ||
|
||
Here is a minimal deployable pattern definition: | ||
|
||
``` javascript | ||
const { SnsToSqs } = require('@aws-solutions-constructs/aws-sns-sqs'); | ||
|
||
const props: SnsToSqsProps = {}; | ||
|
||
new SnsToSqs(stack, 'SnsToSqsPattern', props); | ||
|
||
``` | ||
|
||
## Initializer | ||
|
||
``` text | ||
new SnsToSqs(scope: Construct, id: string, props: SnsToSqsProps); | ||
``` | ||
|
||
_Parameters_ | ||
|
||
* scope [`Construct`](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_core.Construct.html) | ||
* id `string` | ||
* props [`SnsToSqsProps`](#pattern-construct-props) | ||
|
||
## Pattern Construct Props | ||
|
||
| **Name** | **Type** | **Description** | | ||
|:-------------|:----------------|-----------------| | ||
|existingTopicObj?|[`sns.Topic`](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-sns.Topic.html)|An optional, existing SNS topic to be used instead of the default topic. If an existing topic is provided, the `topicProps` property will be ignored.| | ||
|topicProps?|[`sns.TopicProps`](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-sns.TopicProps.html)|Optional user provided properties to override the default properties for the SNS topic.| | ||
|existingQueueObj?|[`sqs.Queue`](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-sqs.Queue.html)|An optional, existing SQS queue to be used instead of the default queue. If an existing queue is provided, the `queueProps` property will be ignored.| | ||
|queueProps?|[`sqs.QueueProps`](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-sqs.QueueProps.html)|Optional user provided properties to override the default properties for the SQS queue.| | ||
|deployDeadLetterQueue?|`boolean`|Whether to create a secondary queue to be used as a dead letter queue. Defaults to true.| | ||
|deadLetterQueueProps?|[`sqs.QueueProps`](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-sqs.QueueProps.html)|Optional user-provided props to override the default props for the dead letter SQS queue.| | ||
|maxReceiveCount?|`number`|The number of times a message can be unsuccessfully dequeued before being moved to the dead letter queue. Defaults to 15.| | ||
|enableEncryptionWithCustomerManagedKey?|`boolean`|Use a KMS Key, either managed by this CDK app, or imported. If importing an encryption key, it must be specified in the encryptionKey property for this construct.| | ||
|encryptionKey?|[`kms.Key`](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-kms.Key.html)|An optional, imported encryption key to encrypt the SQS queue, and SNS Topic.| | ||
|encryptionKeyProps?|[`kms.KeyProps`](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-kms.KeyProps.html)|An optional, user provided properties to override the default properties for the KMS encryption key.| | ||
|
||
## Pattern Properties | ||
|
||
| **Name** | **Type** | **Description** | | ||
|:-------------|:----------------|-----------------| | ||
|snsTopic|[`sns.Topic`](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-sns.Topic.html)|Returns an instance of the SNS topic created by the pattern.| | ||
|encryptionKey|[`kms.Key`](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-kms.Key.html)|Returns an instance of kms.Key used for the SQS queue, and SNS Topic.| | ||
|sqsQueue|[`sqs.Queue`](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-sqs.Queue.html)|Returns an instance of the SQS queue created by the pattern.| | ||
|deadLetterQueue?|[`sqs.Queue`](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-sqs.Queue.html)|Returns an instance of the dead-letter SQS queue created by the pattern.| | ||
|
||
## Default settings | ||
|
||
Out of the box implementation of the Construct without any override will set the following defaults: | ||
|
||
### Amazon SNS Topic | ||
* Configure least privilege access permissions for SNS Topic | ||
* Enable server-side encryption for SNS Topic using Customer managed KMS Key | ||
* Enforce encryption of data in transit | ||
|
||
### Amazon SQS Queue | ||
* Configure least privilege access permissions for SQS Queue | ||
* Deploy SQS dead-letter queue for the source SQS Queue | ||
* Enable server-side encryption for SQS Queue using Customer managed KMS Key | ||
* Enforce encryption of data in transit | ||
|
||
## Architecture | ||
![Architecture Diagram](architecture.png) | ||
|
||
*** | ||
© Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. |
Binary file added
BIN
+168 KB
source/patterns/@aws-solutions-constructs/aws-sns-sqs/architecture.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
168 changes: 168 additions & 0 deletions
168
source/patterns/@aws-solutions-constructs/aws-sns-sqs/lib/index.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,168 @@ | ||
/** | ||
* Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved. | ||
* | ||
* Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance | ||
* with the License. A copy of the License is located at | ||
* | ||
* http://www.apache.org/licenses/LICENSE-2.0 | ||
* | ||
* or in the 'license' file accompanying this file. This file is distributed on an 'AS IS' BASIS, WITHOUT WARRANTIES | ||
* OR CONDITIONS OF ANY KIND, express or implied. See the License for the specific language governing permissions | ||
* and limitations under the License. | ||
*/ | ||
|
||
// Imports | ||
import * as sqs from '@aws-cdk/aws-sqs'; | ||
import * as sns from '@aws-cdk/aws-sns'; | ||
import * as subscriptions from '@aws-cdk/aws-sns-subscriptions'; | ||
import * as kms from '@aws-cdk/aws-kms'; | ||
import * as iam from '@aws-cdk/aws-iam'; | ||
import * as defaults from '@aws-solutions-constructs/core'; | ||
import { Construct } from '@aws-cdk/core'; | ||
import {buildEncryptionKey} from "@aws-solutions-constructs/core"; | ||
|
||
/** | ||
* @summary The properties for the SnsToSqs class. | ||
*/ | ||
export interface SnsToSqsProps { | ||
/** | ||
* Existing instance of SNS topic object, if this is set then topicProps is ignored. | ||
* | ||
* @default - Default props are used | ||
*/ | ||
readonly existingTopicObj?: sns.Topic, | ||
/** | ||
* Optional user provided properties to override the default properties for the SNS topic. | ||
* | ||
* @default - Default properties are used. | ||
*/ | ||
readonly topicProps?: sns.TopicProps, | ||
/** | ||
* Existing instance of SQS queue object, if this is set then queueProps is ignored. | ||
* | ||
* @default - Default props are used | ||
*/ | ||
readonly existingQueueObj?: sqs.Queue, | ||
/** | ||
* Optional user provided properties | ||
* | ||
* @default - Default props are used | ||
*/ | ||
readonly queueProps?: sqs.QueueProps, | ||
/** | ||
* Optional user provided properties for the dead letter queue | ||
* | ||
* @default - Default props are used | ||
*/ | ||
readonly deadLetterQueueProps?: sqs.QueueProps, | ||
/** | ||
* Whether to deploy a secondary queue to be used as a dead letter queue. | ||
* | ||
* @default - true. | ||
*/ | ||
readonly deployDeadLetterQueue?: boolean, | ||
/** | ||
* The number of times a message can be unsuccessfully dequeued before being moved to the dead-letter queue. | ||
* | ||
* @default - required field if deployDeadLetterQueue=true. | ||
*/ | ||
readonly maxReceiveCount?: number | ||
/** | ||
* Use a KMS Key, either managed by this CDK app, or imported. If importing an encryption key, it must be specified in | ||
* the encryptionKey property for this construct. | ||
* | ||
* @default - true (encryption enabled, managed by this CDK app). | ||
*/ | ||
readonly enableEncryptionWithCustomerManagedKey?: boolean | ||
/** | ||
* An optional, imported encryption key to encrypt the SQS queue, and SNS Topic. | ||
* | ||
* @default - not specified. | ||
*/ | ||
readonly encryptionKey?: kms.Key | ||
/** | ||
* Optional user-provided props to override the default props for the encryption key. | ||
* | ||
* @default - Default props are used. | ||
*/ | ||
readonly encryptionKeyProps?: kms.KeyProps | ||
} | ||
|
||
/** | ||
* @summary The SnsToSqs class. | ||
*/ | ||
export class SnsToSqs extends Construct { | ||
public readonly snsTopic: sns.Topic; | ||
public readonly encryptionKey?: kms.Key; | ||
public readonly sqsQueue: sqs.Queue; | ||
public readonly deadLetterQueue?: sqs.DeadLetterQueue; | ||
|
||
/** | ||
* @summary Constructs a new instance of the SnsToSqs class. | ||
* @param {cdk.App} scope - represents the scope for all the resources. | ||
* @param {string} id - this is a a scope-unique id. | ||
* @param {SnsToSqsProps} props - user provided props for the construct. | ||
* @since 1.61.0 | ||
* @access public | ||
*/ | ||
constructor(scope: Construct, id: string, props: SnsToSqsProps) { | ||
super(scope, id); | ||
|
||
// Setup the dead letter queue, if applicable | ||
if (props.deployDeadLetterQueue || props.deployDeadLetterQueue === undefined) { | ||
const dlq: sqs.Queue = defaults.buildQueue(this, 'deadLetterQueue', { | ||
queueProps: props.deadLetterQueueProps | ||
}); | ||
this.deadLetterQueue = defaults.buildDeadLetterQueue({ | ||
deadLetterQueue: dlq, | ||
maxReceiveCount: props.maxReceiveCount | ||
}); | ||
} | ||
|
||
let enableEncryptionParam:boolean | undefined = props.enableEncryptionWithCustomerManagedKey; | ||
let encryptionKeyParam:kms.Key | undefined = props.encryptionKey; | ||
|
||
if (props.enableEncryptionWithCustomerManagedKey === undefined || | ||
props.enableEncryptionWithCustomerManagedKey === true) { | ||
enableEncryptionParam = true; | ||
// Create the encryptionKey if none was provided | ||
if (!props.encryptionKey) { | ||
encryptionKeyParam = buildEncryptionKey(scope, { | ||
encryptionKeyProps: props.encryptionKeyProps | ||
}); | ||
} | ||
} | ||
// Setup the SNS topic | ||
if (!props.existingTopicObj) { | ||
// If an existingTopicObj was not specified create new topic | ||
[this.snsTopic, this.encryptionKey] = defaults.buildTopic(this, { | ||
topicProps: props.topicProps, | ||
enableEncryption: enableEncryptionParam, | ||
encryptionKey: encryptionKeyParam | ||
}); | ||
} else { | ||
// If an existingTopicObj was specified utilize the provided topic | ||
this.snsTopic = props.existingTopicObj; | ||
} | ||
|
||
// Setup the queue | ||
this.sqsQueue = defaults.buildQueue(this, 'queue', { | ||
existingQueueObj: props.existingQueueObj, | ||
queueProps: props.queueProps, | ||
deadLetterQueue: this.deadLetterQueue, | ||
enableEncryption: enableEncryptionParam, | ||
encryptionKey: encryptionKeyParam | ||
}); | ||
|
||
// Setup the SQS queue subscription to the SNS topic | ||
this.snsTopic.addSubscription(new subscriptions.SqsSubscription(this.sqsQueue)); | ||
|
||
// Grant SNS service access to the SQS queue encryption key | ||
if (this.sqsQueue.encryptionMasterKey) { | ||
this.sqsQueue.encryptionMasterKey.grant(new iam.ServicePrincipal("sns.amazonaws.com"), | ||
'kms:Decrypt', | ||
'kms:GenerateDataKey*', | ||
); | ||
} | ||
} | ||
} |
85 changes: 85 additions & 0 deletions
85
source/patterns/@aws-solutions-constructs/aws-sns-sqs/package.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,85 @@ | ||
{ | ||
"name": "@aws-solutions-constructs/aws-sns-sqs", | ||
"version": "1.61.0", | ||
"description": "CDK constructs for defining an interaction between an Amazon SNS topic and an Amazon SQS queue.", | ||
"main": "lib/index.js", | ||
"types": "lib/index.d.ts", | ||
"repository": { | ||
"type": "git", | ||
"url": "https://github.com/awslabs/aws-solutions-constructs.git", | ||
"directory": "source/patterns/@aws-solutions-constructs/aws-sns-sqs" | ||
}, | ||
"author": { | ||
"name": "Amazon Web Services", | ||
"url": "https://aws.amazon.com", | ||
"organization": true | ||
}, | ||
"license": "Apache-2.0", | ||
"scripts": { | ||
"build": "tsc -b .", | ||
"lint": "eslint -c ../eslintrc.yml --ext=.js,.ts . && tslint --project .", | ||
"lint-fix": "eslint -c ../eslintrc.yml --ext=.js,.ts --fix .", | ||
"test": "jest --coverage", | ||
"clean": "tsc -b --clean", | ||
"watch": "tsc -b -w", | ||
"integ": "cdk-integ", | ||
"integ-assert": "cdk-integ-assert", | ||
"integ-no-clean": "cdk-integ --no-clean", | ||
"jsii": "jsii", | ||
"jsii-pacmak": "jsii-pacmak", | ||
"build+lint+test": "npm run jsii && npm run lint && npm test && npm run integ-assert", | ||
"snapshot-update": "npm run jsii && npm test -- -u && npm run integ-assert" | ||
}, | ||
"jsii": { | ||
"outdir": "dist", | ||
"targets": { | ||
"java": { | ||
"package": "software.amazon.awsconstructs.services.snssqs", | ||
"maven": { | ||
"groupId": "software.amazon.awsconstructs", | ||
"artifactId": "snssqs" | ||
} | ||
}, | ||
"dotnet": { | ||
"namespace": "Amazon.Constructs.AWS.SnsSqs", | ||
"packageId": "Amazon.Constructs.AWS.SnsSqs", | ||
"signAssembly": true, | ||
"iconUrl": "https://raw.githubusercontent.com/aws/aws-cdk/master/logo/default-256-dark.png" | ||
}, | ||
"python": { | ||
"distName": "aws-solutions-constructs.aws-sns-sqs", | ||
"module": "aws_solutions_constructs.aws_sns_sqs" | ||
} | ||
} | ||
}, | ||
"dependencies": { | ||
"@aws-cdk/aws-iam": "~1.61.0", | ||
"@aws-cdk/aws-sns": "~1.61.0", | ||
"@aws-cdk/aws-sqs": "~1.61.0", | ||
"@aws-cdk/aws-sns-subscriptions": "~1.61.0", | ||
"@aws-cdk/aws-kms": "~1.61.0", | ||
"@aws-cdk/core": "~1.61.0", | ||
"@aws-solutions-constructs/core": "~1.61.0", | ||
"constructs": "^3.0.4" | ||
}, | ||
"devDependencies": { | ||
"@aws-cdk/assert": "~1.61.0", | ||
"@types/jest": "^24.0.23", | ||
"@types/node": "^10.3.0" | ||
}, | ||
"jest": { | ||
"moduleFileExtensions": [ | ||
"js" | ||
] | ||
}, | ||
"peerDependencies": { | ||
"@aws-cdk/aws-iam": "~1.61.0", | ||
"@aws-cdk/aws-sns": "~1.61.0", | ||
"@aws-cdk/aws-sqs": "~1.61.0", | ||
"@aws-cdk/aws-sns-subscriptions": "~1.61.0", | ||
"@aws-cdk/aws-kms": "~1.61.0", | ||
"@aws-cdk/core": "~1.61.0", | ||
"@aws-solutions-constructs/core": "~1.61.0", | ||
"constructs": "^3.0.4" | ||
} | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One more optional argument for user-provided encryptionKeyProps?
readonly encryptionKeyProps?: kms.KeyProps