From 94279f35e4f5ef961e0ba8528e34a8fccb9ef3fe Mon Sep 17 00:00:00 2001 From: Mitch Lloyd Date: Tue, 9 Feb 2021 15:14:42 -0800 Subject: [PATCH] feat(aws-kinesisanalyticsv2): L2 construct for Flink applications (#12464) Opened for https://github.com/aws/aws-cdk/issues/12407. Notes for review: 1. Flink and SQL applications share almost no properties so having separate ka.FlinkApplication and ka.SqlApplication constructs seems correct. I can't see why these would even share an abstract base class. 1. I'm trying to focus on shipping the Flink construct before SQL since they are so different and I haven't used an SQL application. 1. I unested lots of the configuration for discoverability. The Cfn naming is verbose (usually with prefixes) so collisions are unlikely. [This](https://github.com/aws-samples/amazon-kinesis-analytics-streaming-etl/blob/master/cdk/lib/streaming-etl.ts#L100) is a pretty good example of using CDK today to build a Flink app. Running List of Open Questions ------------------------------- 1. ~aws-kinesisanalytics exports both `aws-kinesisanalytics` and `aws-kinesisanalyticsv2` generated code. How should we resolve this? Currently I've exported `aws-kinesisanalyticsv2` from `aws-kinesisanalyticsv2` and haven't changed the other package.~ Kept this package isolated from aws-kinesisanalytics at the expense of duplicate generated code. 2. ~I'm not confident with the use cases for the `fromAttributes` factory. I'd prefer to leave this factory method out in this initial PR if possible, but I'm also open to comments about what use cases this should handle.~ 3. ~All logging options could be flattened into FlinkApplicationProps (e.g. logRentention, logGroupName, logStreamName, logEncryptionKey...). My first thought was to provide a new `ka.Logging` type that can be passed to customize the logGroup and logStream. There may be some other middle ground that could let users pass in a logGroup and then a logStream.~ Went with the flat list of props added to FlinkApplicationProps 4. ~When I run `yarn build` I get this error even though I've already tagged the class.~ Was missing a `gen` yarn script entry. 5. ~Should this module become `aws-flink-application` or `aws-kinesisanalytics-flink` to side step the confusion of having seemingly unrelated constructs in the same module at the expense of clashing with CloudFormation?~ New module name: `aws-kinesisanalytics-flink`. Todo ---- - [x] Inline documentation - [x] Add property groups - [x] checkpointEnabled - [x] minPauseBetweenCheckpoints - [x] logLevel - [x] metricsLevel - [x] autoscalingEnabled - [x] parallelism - [x] parallelismPerKpu - [x] snapshotsEnabled - [x] Figure out fromAttributes approach - [x] Add log stream - [x] Decide on logging options approach - [x] Add metrics access - [x] Validate application name - [x] Integration tests ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license* --- .../aws-kinesisanalytics-flink/.eslintrc.js | 3 + .../aws-kinesisanalytics-flink/.gitignore | 19 + .../aws-kinesisanalytics-flink/.npmignore | 27 + .../aws-kinesisanalytics-flink/LICENSE | 201 ++++++ .../aws-kinesisanalytics-flink/NOTICE | 2 + .../aws-kinesisanalytics-flink/README.md | 74 +++ .../aws-kinesisanalytics-flink/jest.config.js | 2 + .../lib/application-code.ts | 139 ++++ .../lib/application.ts | 341 ++++++++++ .../aws-kinesisanalytics-flink/lib/index.ts | 4 + .../lib/private/environment-properties.ts | 16 + .../flink-application-configuration.ts | 78 +++ .../lib/private/validation.ts | 54 ++ .../aws-kinesisanalytics-flink/lib/types.ts | 67 ++ .../aws-kinesisanalytics-flink/package.json | 115 ++++ .../test/application.test.ts | 604 ++++++++++++++++++ .../test/code-asset/WordCount.jar | Bin 0 -> 15192 bytes ...ication-code-from-bucket.lit.expected.json | 295 +++++++++ .../integ.application-code-from-bucket.lit.ts | 22 + .../test/integ.application.lit.expected.json | 295 +++++++++ .../test/integ.application.lit.ts | 15 + .../@aws-cdk/aws-kinesisanalytics/README.md | 7 + packages/aws-cdk-lib/package.json | 1 + packages/decdk/package.json | 1 + packages/monocdk/package.json | 1 + 25 files changed, 2383 insertions(+) create mode 100644 packages/@aws-cdk/aws-kinesisanalytics-flink/.eslintrc.js create mode 100644 packages/@aws-cdk/aws-kinesisanalytics-flink/.gitignore create mode 100644 packages/@aws-cdk/aws-kinesisanalytics-flink/.npmignore create mode 100644 packages/@aws-cdk/aws-kinesisanalytics-flink/LICENSE create mode 100644 packages/@aws-cdk/aws-kinesisanalytics-flink/NOTICE create mode 100644 packages/@aws-cdk/aws-kinesisanalytics-flink/README.md create mode 100644 packages/@aws-cdk/aws-kinesisanalytics-flink/jest.config.js create mode 100644 packages/@aws-cdk/aws-kinesisanalytics-flink/lib/application-code.ts create mode 100644 packages/@aws-cdk/aws-kinesisanalytics-flink/lib/application.ts create mode 100644 packages/@aws-cdk/aws-kinesisanalytics-flink/lib/index.ts create mode 100644 packages/@aws-cdk/aws-kinesisanalytics-flink/lib/private/environment-properties.ts create mode 100644 packages/@aws-cdk/aws-kinesisanalytics-flink/lib/private/flink-application-configuration.ts create mode 100644 packages/@aws-cdk/aws-kinesisanalytics-flink/lib/private/validation.ts create mode 100644 packages/@aws-cdk/aws-kinesisanalytics-flink/lib/types.ts create mode 100644 packages/@aws-cdk/aws-kinesisanalytics-flink/package.json create mode 100644 packages/@aws-cdk/aws-kinesisanalytics-flink/test/application.test.ts create mode 100644 packages/@aws-cdk/aws-kinesisanalytics-flink/test/code-asset/WordCount.jar create mode 100644 packages/@aws-cdk/aws-kinesisanalytics-flink/test/integ.application-code-from-bucket.lit.expected.json create mode 100644 packages/@aws-cdk/aws-kinesisanalytics-flink/test/integ.application-code-from-bucket.lit.ts create mode 100644 packages/@aws-cdk/aws-kinesisanalytics-flink/test/integ.application.lit.expected.json create mode 100644 packages/@aws-cdk/aws-kinesisanalytics-flink/test/integ.application.lit.ts diff --git a/packages/@aws-cdk/aws-kinesisanalytics-flink/.eslintrc.js b/packages/@aws-cdk/aws-kinesisanalytics-flink/.eslintrc.js new file mode 100644 index 0000000000000..61dd8dd001f63 --- /dev/null +++ b/packages/@aws-cdk/aws-kinesisanalytics-flink/.eslintrc.js @@ -0,0 +1,3 @@ +const baseConfig = require('cdk-build-tools/config/eslintrc'); +baseConfig.parserOptions.project = __dirname + '/tsconfig.json'; +module.exports = baseConfig; diff --git a/packages/@aws-cdk/aws-kinesisanalytics-flink/.gitignore b/packages/@aws-cdk/aws-kinesisanalytics-flink/.gitignore new file mode 100644 index 0000000000000..d8a8561d50885 --- /dev/null +++ b/packages/@aws-cdk/aws-kinesisanalytics-flink/.gitignore @@ -0,0 +1,19 @@ +*.js +*.js.map +*.d.ts +tsconfig.json +node_modules +*.generated.ts +dist +.jsii + +.LAST_BUILD +.nyc_output +coverage +nyc.config.js +.LAST_PACKAGE +*.snk +!.eslintrc.js +!jest.config.js + +junit.xml \ No newline at end of file diff --git a/packages/@aws-cdk/aws-kinesisanalytics-flink/.npmignore b/packages/@aws-cdk/aws-kinesisanalytics-flink/.npmignore new file mode 100644 index 0000000000000..63ab95621c764 --- /dev/null +++ b/packages/@aws-cdk/aws-kinesisanalytics-flink/.npmignore @@ -0,0 +1,27 @@ +# Don't include original .ts files when doing `npm pack` +*.ts +!*.d.ts +coverage +.nyc_output +*.tgz + +dist +.LAST_PACKAGE +.LAST_BUILD +!*.js + +# Include .jsii +!.jsii + +*.snk + +*.tsbuildinfo + +tsconfig.json +.eslintrc.js +jest.config.js + +# exclude cdk artifacts +**/cdk.out +junit.xml +test/ \ No newline at end of file diff --git a/packages/@aws-cdk/aws-kinesisanalytics-flink/LICENSE b/packages/@aws-cdk/aws-kinesisanalytics-flink/LICENSE new file mode 100644 index 0000000000000..28e4bdcec77ec --- /dev/null +++ b/packages/@aws-cdk/aws-kinesisanalytics-flink/LICENSE @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright 2018-2021 Amazon.com, Inc. or its affiliates. All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/packages/@aws-cdk/aws-kinesisanalytics-flink/NOTICE b/packages/@aws-cdk/aws-kinesisanalytics-flink/NOTICE new file mode 100644 index 0000000000000..5fc3826926b5b --- /dev/null +++ b/packages/@aws-cdk/aws-kinesisanalytics-flink/NOTICE @@ -0,0 +1,2 @@ +AWS Cloud Development Kit (AWS CDK) +Copyright 2018-2021 Amazon.com, Inc. or its affiliates. All Rights Reserved. diff --git a/packages/@aws-cdk/aws-kinesisanalytics-flink/README.md b/packages/@aws-cdk/aws-kinesisanalytics-flink/README.md new file mode 100644 index 0000000000000..7dc9b1a088b16 --- /dev/null +++ b/packages/@aws-cdk/aws-kinesisanalytics-flink/README.md @@ -0,0 +1,74 @@ +# Kinesis Analytics Flink + + +--- + +![cdk-constructs: Experimental](https://img.shields.io/badge/cdk--constructs-experimental-important.svg?style=for-the-badge) + +> The APIs of higher level constructs in this module are experimental and under active development. +> They are subject to non-backward compatible changes or removal in any future version. These are +> not subject to the [Semantic Versioning](https://semver.org/) model and breaking changes will be +> announced in the release notes. This means that while you may use them, you may need to update +> your source code when upgrading to a newer version of this package. + +--- + + + +This package provides constructs for creating Kinesis Analytics Flink +applications. To learn more about using using managed Flink applications, see +the [AWS developer +guide](https://docs.aws.amazon.com/kinesisanalytics/latest/java/what-is.html). + +## Creating Flink Applications + +To create a new Flink application, use the `Application` construct: + +[simple flink application](test/integ.application.lit.ts) + +The `code` property can use `fromAsset` as shown above to reference a local jar +file in s3 or `fromBucket` to reference a file in s3. + +[flink application using code from bucket](test/integ.application-code-from-bucket.lit.ts) + +The `propertyGroups` property provides a way of passing arbitrary runtime +properties to your Flink application. You can use the +aws-kinesisanalytics-runtime library to [retrieve these +properties](https://docs.aws.amazon.com/kinesisanalytics/latest/java/how-properties.html#how-properties-access). + +```ts +import * as flink from '@aws-cdk/aws-kinesisanalytics-flink'; + +const flinkApp = new flink.Application(this, 'Application', { + // ... + propertyGroups: { + FlinkApplicationProperties: { + inputStreamName: 'my-input-kinesis-stream', + outputStreamName: 'my-output-kinesis-stream', + }, + }, +}); +``` + +Flink applications also have specific configuration for passing parameters +when the Flink job starts. These include parameters for checkpointing, +snapshotting, monitoring, and parallelism. + +```ts +import * as logs from '@aws-cdk/aws-logs'; + +const flinkApp = new flink.Application(this, 'Application', { + code: flink.ApplicationCode.fromBucket(bucket, 'my-app.jar'), + runtime: file.Runtime.FLINK_1_11, + checkpointingEnabled: true, // default is true + checkpointInterval: cdk.Duration.seconds(30), // default is 1 minute + minPausesBetweenCheckpoints: cdk.Duration.seconds(10), // default is 5 seconds + logLevel: flink.LogLevel.ERROR, // default is INFO + metricsLevel: flink.MetricsLevel.PARALLELISM, // default is APPLICATION + autoScalingEnabled: false, // default is true + parallelism: 32, // default is 1 + parallelismPerKpu: 2, // default is 1 + snapshotsEnabled: false, // default is true + logGroup: new logs.LogGroup(this, 'LogGroup'), // by default, a new LogGroup will be created +}); +``` diff --git a/packages/@aws-cdk/aws-kinesisanalytics-flink/jest.config.js b/packages/@aws-cdk/aws-kinesisanalytics-flink/jest.config.js new file mode 100644 index 0000000000000..54e28beb9798b --- /dev/null +++ b/packages/@aws-cdk/aws-kinesisanalytics-flink/jest.config.js @@ -0,0 +1,2 @@ +const baseConfig = require('cdk-build-tools/config/jest.config'); +module.exports = baseConfig; diff --git a/packages/@aws-cdk/aws-kinesisanalytics-flink/lib/application-code.ts b/packages/@aws-cdk/aws-kinesisanalytics-flink/lib/application-code.ts new file mode 100644 index 0000000000000..c7d7b49180086 --- /dev/null +++ b/packages/@aws-cdk/aws-kinesisanalytics-flink/lib/application-code.ts @@ -0,0 +1,139 @@ +import * as ka from '@aws-cdk/aws-kinesisanalytics'; +import * as s3 from '@aws-cdk/aws-s3'; +import * as s3_assets from '@aws-cdk/aws-s3-assets'; +import { Construct } from '@aws-cdk/core'; + +/** + * The return type of {@link ApplicationCode.bind}. This represents + * CloudFormation configuration and an s3 bucket holding the Flink application + * JAR file. + */ +export interface ApplicationCodeConfig { + /** + * Low-level Cloudformation ApplicationConfigurationProperty + */ + readonly applicationCodeConfigurationProperty: ka.CfnApplicationV2.ApplicationConfigurationProperty; + + /** + * S3 Bucket that stores the Flink application code + */ + readonly bucket: s3.IBucket; +} + +/** + * Code configuration providing the location to a Flink application JAR file. + */ +export abstract class ApplicationCode { + /** + * Reference code from an S3 bucket. + * + * @param bucket - an s3 bucket + * @param fileKey - a key pointing to a Flink JAR file + * @param objectVersion - an optional version string for the provided fileKey + */ + public static fromBucket(bucket: s3.IBucket, fileKey: string, objectVersion?: string): ApplicationCode { + return new BucketApplicationCode({ + bucket, + fileKey, + objectVersion, + }); + } + + /** + * Reference code from a local directory containing a Flink JAR file. + * + * @param path - a local directory path + * @parm options - standard s3 AssetOptions + */ + public static fromAsset(path: string, options?: s3_assets.AssetOptions): ApplicationCode { + return new AssetApplicationCode(path, options); + } + + /** + * A method to lazily bind asset resources to the parent FlinkApplication. + */ + public abstract bind(scope: Construct): ApplicationCodeConfig; +} + +interface BucketApplicationCodeProps { + readonly bucket: s3.IBucket; + readonly fileKey: string; + readonly objectVersion?: string; +} + +class BucketApplicationCode extends ApplicationCode { + public readonly bucket?: s3.IBucket; + public readonly fileKey: string; + public readonly objectVersion?: string; + + constructor(props: BucketApplicationCodeProps) { + super(); + this.bucket = props.bucket; + this.fileKey = props.fileKey; + this.objectVersion = props.objectVersion; + } + + public bind(_scope: Construct): ApplicationCodeConfig { + return { + applicationCodeConfigurationProperty: { + applicationCodeConfiguration: { + codeContent: { + s3ContentLocation: { + bucketArn: this.bucket!.bucketArn, + fileKey: this.fileKey, + objectVersion: this.objectVersion, + }, + }, + codeContentType: 'ZIPFILE', + }, + }, + bucket: this.bucket!, + }; + } +} + +class AssetApplicationCode extends ApplicationCode { + private readonly path: string; + private readonly options?: s3_assets.AssetOptions; + private _asset?: s3_assets.Asset; + + constructor(path: string, options?: s3_assets.AssetOptions) { + super(); + this.path = path; + this.options = options; + } + + public bind(scope: Construct): ApplicationCodeConfig { + this._asset = new s3_assets.Asset(scope, 'Code', { + path: this.path, + ...this.options, + }); + + if (!this._asset.isZipArchive) { + throw new Error(`Asset must be a .zip file or a directory (${this.path})`); + } + + return { + applicationCodeConfigurationProperty: { + applicationCodeConfiguration: { + codeContent: { + s3ContentLocation: { + bucketArn: this._asset.bucket.bucketArn, + fileKey: this._asset.s3ObjectKey, + }, + }, + codeContentType: 'ZIPFILE', + }, + }, + bucket: this._asset.bucket, + }; + } + + get asset(): s3_assets.Asset | undefined { + return this._asset; + } + + get bucket(): s3.IBucket | undefined { + return this._asset?.bucket; + } +} diff --git a/packages/@aws-cdk/aws-kinesisanalytics-flink/lib/application.ts b/packages/@aws-cdk/aws-kinesisanalytics-flink/lib/application.ts new file mode 100644 index 0000000000000..1bcfa0fc4cbbf --- /dev/null +++ b/packages/@aws-cdk/aws-kinesisanalytics-flink/lib/application.ts @@ -0,0 +1,341 @@ +import * as iam from '@aws-cdk/aws-iam'; +import * as ka from '@aws-cdk/aws-kinesisanalytics'; +import * as logs from '@aws-cdk/aws-logs'; +import * as core from '@aws-cdk/core'; +import { Construct } from 'constructs'; +import { ApplicationCode } from './application-code'; +import { environmentProperties } from './private/environment-properties'; +import { flinkApplicationConfiguration } from './private/flink-application-configuration'; +import { validateFlinkApplicationProps as validateApplicationProps } from './private/validation'; +import { LogLevel, MetricsLevel, PropertyGroups, Runtime } from './types'; + +/** + * An interface expressing the public properties on both an imported and + * CDK-created Flink application. + */ +export interface IApplication extends core.IResource, iam.IGrantable { + /** + * The application ARN. + * + * @attribute + */ + readonly applicationArn: string; + + /** + * The name of the Flink application. + * + * @attribute + */ + readonly applicationName: string; + + /** + * The application IAM role. + */ + readonly role?: iam.IRole; + + /** + * Convenience method for adding a policy statement to the application role. + */ + addToRolePolicy(policyStatement: iam.PolicyStatement): boolean; +} + +/** + * Implements the functionality shared between CDK created and imported + * IApplications. + */ +abstract class ApplicationBase extends core.Resource implements IApplication { + public abstract readonly applicationArn: string; + public abstract readonly applicationName: string; + public abstract readonly role?: iam.IRole; + + // Implement iam.IGrantable interface + public abstract readonly grantPrincipal: iam.IPrincipal; + + /** Implement the convenience {@link IApplication.addToPrincipalPolicy} method. */ + public addToRolePolicy(policyStatement: iam.PolicyStatement): boolean { + if (this.role) { + this.role.addToPrincipalPolicy(policyStatement); + return true; + } + + return false; + } +} + +/** + * Props for creating an Application construct. + */ +export interface ApplicationProps { + /** + * A name for your Application that is unique to an AWS account. + * + * @default - CloudFormation-generated name + */ + readonly applicationName?: string; + + /** + * The Flink version to use for this application. + */ + readonly runtime: Runtime; + + /** + * The Flink code asset to run. + */ + readonly code: ApplicationCode; + + /** + * Whether checkpointing is enabled while your application runs. + * + * @default true + */ + readonly checkpointingEnabled?: boolean; + + /** + * The interval between checkpoints. + * + * @default 1 minute + */ + readonly checkpointInterval?: core.Duration; + + /** + * The minimum amount of time in to wait after a checkpoint finishes to start + * a new checkpoint. + * + * @default 5 seconds + */ + readonly minPauseBetweenCheckpoints?: core.Duration; + + /** + * The level of log verbosity from the Flink application. + * + * @default FlinkLogLevel.INFO + */ + readonly logLevel?: LogLevel; + + /** + * Describes the granularity of the CloudWatch metrics for an application. + * Use caution with Parallelism level metrics. Parallelism granularity logs + * metrics for each parallel thread and can quickly become expensive when + * parallelism is high (e.g. > 64). + * + * @default MetricsLevel.APPLICATION + */ + readonly metricsLevel?: MetricsLevel; + + /** + * Whether the Kinesis Data Analytics service can increase the parallelism of + * the application in response to resource usage. + * + * @default true + */ + readonly autoScalingEnabled?: boolean; + + /** + * The initial parallelism for the application. Kinesis Data Analytics can + * stop the app, increase the parallelism, and start the app again if + * autoScalingEnabled is true (the default value). + * + * @default 1 + */ + readonly parallelism?: number; + + /** + * The Flink parallelism allowed per Kinesis Processing Unit (KPU). + * + * @default 1 + */ + readonly parallelismPerKpu?: number + + /** + * Determines if Flink snapshots are enabled. + * + * @default true + */ + readonly snapshotsEnabled?: boolean; + + /** + * Configuration PropertyGroups. You can use these property groups to pass + * arbitrary runtime configuration values to your Flink app. + * + * @default No property group configuration provided to the Flink app + */ + readonly propertyGroups?: PropertyGroups; + + /** + * A role to use to grant permissions to your application. Prefer omitting + * this property and using the default role. + * + * @default - a new Role will be created + */ + readonly role?: iam.IRole; + + /** + * Provide a RemovalPolicy to override the default. + * + * @default RemovalPolicy.DESTROY + */ + readonly removalPolicy?: core.RemovalPolicy; + + /** + * The log group to send log entries to. + * + * @default CDK's default LogGroup + */ + readonly logGroup?: logs.ILogGroup; +} + +/** + * An imported Flink application. + */ +class Import extends ApplicationBase { + public readonly grantPrincipal: iam.IPrincipal; + public readonly role?: iam.IRole; + public readonly applicationName: string; + public readonly applicationArn: string; + + constructor(scope: Construct, id: string, attrs: { applicationArn: string, applicationName: string }) { + super(scope, id); + + // Imported applications have no associated role or grantPrincipal + this.grantPrincipal = new iam.UnknownPrincipal({ resource: this }); + this.role = undefined; + + this.applicationArn = attrs.applicationArn; + this.applicationName = attrs.applicationName; + } +} + +/** + * The L2 construct for Flink Kinesis Data Applications. + * + * @resource AWS::KinesisAnalyticsV2::Application + * + * @experimental + */ +export class Application extends ApplicationBase { + /** + * Import an existing Flink application defined outside of CDK code by + * applicationName. + */ + public static fromApplicationName(scope: Construct, id: string, applicationName: string): IApplication { + const applicationArn = core.Stack.of(scope).formatArn(applicationArnComponents(applicationName)); + + return new Import(scope, id, { applicationArn, applicationName }); + } + + /** + * Import an existing application defined outside of CDK code by + * applicationArn. + */ + public static fromApplicationArn(scope: Construct, id: string, applicationArn: string): IApplication { + const applicationName = core.Stack.of(scope).parseArn(applicationArn).resourceName; + if (!applicationName) { + throw new Error(`applicationArn for fromApplicationArn (${applicationArn}) must include resource name`); + } + + return new Import(scope, id, { applicationArn, applicationName }); + } + + public readonly applicationArn: string; + public readonly applicationName: string; + + // Role must be optional for JSII compatibility + public readonly role?: iam.IRole; + + public readonly grantPrincipal: iam.IPrincipal; + + constructor(scope: Construct, id: string, props: ApplicationProps) { + super(scope, id, { physicalName: props.applicationName }); + validateApplicationProps(props); + + this.role = props.role ?? new iam.Role(this, 'Role', { + assumedBy: new iam.ServicePrincipal('kinesisanalytics.amazonaws.com'), + }); + this.grantPrincipal = this.role; + + // Permit metric publishing to CloudWatch + this.role.addToPrincipalPolicy(new iam.PolicyStatement({ + actions: ['cloudwatch:PutMetricData'], + resources: ['*'], + })); + + const code = props.code.bind(this); + code.bucket.grantRead(this); + + const resource = new ka.CfnApplicationV2(this, 'Resource', { + runtimeEnvironment: props.runtime.value, + serviceExecutionRole: this.role.roleArn, + applicationConfiguration: { + ...code.applicationCodeConfigurationProperty, + environmentProperties: environmentProperties(props.propertyGroups), + flinkApplicationConfiguration: flinkApplicationConfiguration({ + checkpointingEnabled: props.checkpointingEnabled, + checkpointInterval: props.checkpointInterval, + minPauseBetweenCheckpoints: props.minPauseBetweenCheckpoints, + logLevel: props.logLevel, + metricsLevel: props.metricsLevel, + autoScalingEnabled: props.autoScalingEnabled, + parallelism: props.parallelism, + parallelismPerKpu: props.parallelismPerKpu, + }), + applicationSnapshotConfiguration: { + snapshotsEnabled: props.snapshotsEnabled ?? true, + }, + }, + }); + resource.node.addDependency(this.role); + + const logGroup = props.logGroup ?? new logs.LogGroup(this, 'LogGroup'); + const logStream = new logs.LogStream(this, 'LogStream', { logGroup }); + + /* Permit logging */ + + this.role.addToPrincipalPolicy(new iam.PolicyStatement({ + actions: ['logs:DescribeLogGroups'], + resources: [ + core.Stack.of(this).formatArn({ + service: 'logs', + resource: 'log-group', + sep: ':', + resourceName: '*', + }), + ], + })); + + this.role.addToPrincipalPolicy(new iam.PolicyStatement({ + actions: ['logs:DescribeLogStreams'], + resources: [logGroup.logGroupArn], + })); + + const logStreamArn = `arn:${core.Aws.PARTITION}:logs:${core.Aws.REGION}:${core.Aws.ACCOUNT_ID}:log-group:${logGroup.logGroupName}:log-stream:${logStream.logStreamName}`; + this.role.addToPrincipalPolicy(new iam.PolicyStatement({ + actions: ['logs:PutLogEvents'], + resources: [logStreamArn], + })); + + new ka.CfnApplicationCloudWatchLoggingOptionV2(this, 'LoggingOption', { + applicationName: resource.ref, + cloudWatchLoggingOption: { + logStreamArn, + }, + }); + + this.applicationName = this.getResourceNameAttribute(resource.ref); + this.applicationArn = this.getResourceArnAttribute( + core.Stack.of(this).formatArn(applicationArnComponents(resource.ref)), + applicationArnComponents(this.physicalName), + ); + + resource.applyRemovalPolicy(props.removalPolicy, { + default: core.RemovalPolicy.DESTROY, + }); + } +} + +function applicationArnComponents(resourceName: string): core.ArnComponents { + return { + service: 'kinesisanalytics', + resource: 'application', + resourceName, + }; +} diff --git a/packages/@aws-cdk/aws-kinesisanalytics-flink/lib/index.ts b/packages/@aws-cdk/aws-kinesisanalytics-flink/lib/index.ts new file mode 100644 index 0000000000000..6e2bea43a0ebf --- /dev/null +++ b/packages/@aws-cdk/aws-kinesisanalytics-flink/lib/index.ts @@ -0,0 +1,4 @@ +export * from './application'; +export * from './application-code'; +export * from './types'; + diff --git a/packages/@aws-cdk/aws-kinesisanalytics-flink/lib/private/environment-properties.ts b/packages/@aws-cdk/aws-kinesisanalytics-flink/lib/private/environment-properties.ts new file mode 100644 index 0000000000000..d13a0e870e23e --- /dev/null +++ b/packages/@aws-cdk/aws-kinesisanalytics-flink/lib/private/environment-properties.ts @@ -0,0 +1,16 @@ +import * as ka from '@aws-cdk/aws-kinesisanalytics'; +import { PropertyGroups } from '../types'; + +export function environmentProperties(propertyGroups?: PropertyGroups): ka.CfnApplicationV2.EnvironmentPropertiesProperty | undefined { + const entries = Object.entries(propertyGroups ?? {}); + if (entries.length === 0) { + return; + } + + return { + propertyGroups: entries.map(([id, map]) => ({ + propertyGroupId: id, + propertyMap: map, + })), + }; +} diff --git a/packages/@aws-cdk/aws-kinesisanalytics-flink/lib/private/flink-application-configuration.ts b/packages/@aws-cdk/aws-kinesisanalytics-flink/lib/private/flink-application-configuration.ts new file mode 100644 index 0000000000000..c0a93edac20d6 --- /dev/null +++ b/packages/@aws-cdk/aws-kinesisanalytics-flink/lib/private/flink-application-configuration.ts @@ -0,0 +1,78 @@ +import * as core from '@aws-cdk/core'; +import { LogLevel, MetricsLevel } from '../types'; + +interface FlinkApplicationConfiguration extends + CheckpointConfiguration, + MonitoringConfiguration, + ParallelismConfiguration {} + +interface CheckpointConfiguration { + checkpointingEnabled?: boolean; + checkpointInterval?: core.Duration; + minPauseBetweenCheckpoints?: core.Duration; +} + +interface MonitoringConfiguration { + logLevel?: LogLevel; + metricsLevel?: MetricsLevel; +} + +interface ParallelismConfiguration { + autoScalingEnabled?: boolean; + parallelism?: number; + parallelismPerKpu?: number; +} + +/** + * Build the nested Cfn FlinkApplicationConfiguration object. This function + * doesn't return empty config objects, returning the minimal config needed to + * express the supplied properties. + * + * This function also handles the quirky configType: 'CUSTOM' setting required + * whenever config in one of the nested groupings. + */ +export function flinkApplicationConfiguration(config: FlinkApplicationConfiguration) { + const checkpointConfiguration = configFor({ + checkpointingEnabled: config.checkpointingEnabled, + checkpointInterval: config.checkpointInterval?.toMilliseconds(), + minPauseBetweenCheckpoints: config.minPauseBetweenCheckpoints?.toMilliseconds(), + }); + + const monitoringConfiguration = configFor({ + logLevel: config.logLevel, + metricsLevel: config.metricsLevel, + }); + + const parallelismConfiguration = configFor({ + autoScalingEnabled: config.autoScalingEnabled, + parallelism: config.parallelism, + parallelismPerKpu: config.parallelismPerKpu, + }); + + const applicationConfiguration = { + checkpointConfiguration, + monitoringConfiguration, + parallelismConfiguration, + }; + + if (isEmptyObj(applicationConfiguration)) { + return; + } + + return applicationConfiguration; +} + +function configFor(config: {[key: string]: unknown}) { + if (isEmptyObj(config)) { + return; + } + + return { + ...config, + configurationType: 'CUSTOM', + }; +} + +function isEmptyObj(obj: {[key: string]: unknown}) { + return Object.values(obj).every(v => v === undefined); +} diff --git a/packages/@aws-cdk/aws-kinesisanalytics-flink/lib/private/validation.ts b/packages/@aws-cdk/aws-kinesisanalytics-flink/lib/private/validation.ts new file mode 100644 index 0000000000000..b0f94f56daf77 --- /dev/null +++ b/packages/@aws-cdk/aws-kinesisanalytics-flink/lib/private/validation.ts @@ -0,0 +1,54 @@ +import * as core from '@aws-cdk/core'; + +interface ValidatedProps { + applicationName?: string; + parallelism?: number; + parallelismPerKpu?: number; +} + +/** + * Early validation for the props used to create FlinkApplications. + */ +export function validateFlinkApplicationProps(props: ValidatedProps) { + validateApplicationName(props.applicationName); + validateParallelism(props.parallelism); + validateParallelismPerKpu(props.parallelismPerKpu); +} + +function validateApplicationName(applicationName?: string) { + if (applicationName === undefined || core.Token.isUnresolved(applicationName)) { + return; + } + + if (applicationName.length === 0) { + throw new Error('applicationName cannot be empty. It must contain at least 1 character.'); + } + + if (!/^[a-zA-Z0-9_.-]+$/.test(applicationName)) { + throw new Error(`applicationName may only contain letters, numbers, underscores, hyphens, and periods. Name: ${applicationName}`); + } + + if (applicationName.length > 128) { + throw new Error(`applicationName max length is 128. Name: ${applicationName} is ${applicationName.length} characters.`); + } +} + +function validateParallelism(parallelism?: number) { + if (parallelism === undefined || core.Token.isUnresolved(parallelism)) { + return; + } + + if (parallelism < 1) { + throw new Error('parallelism must be at least 1'); + } +} + +function validateParallelismPerKpu(parallelismPerKpu?: number) { + if (parallelismPerKpu === undefined || core.Token.isUnresolved(parallelismPerKpu)) { + return; + } + + if (parallelismPerKpu < 1) { + throw new Error('parallelismPerKpu must be at least 1'); + } +} diff --git a/packages/@aws-cdk/aws-kinesisanalytics-flink/lib/types.ts b/packages/@aws-cdk/aws-kinesisanalytics-flink/lib/types.ts new file mode 100644 index 0000000000000..b92d55cb48518 --- /dev/null +++ b/packages/@aws-cdk/aws-kinesisanalytics-flink/lib/types.ts @@ -0,0 +1,67 @@ +/** + * Available log levels for Flink applications. + */ +export enum LogLevel { + /** Debug level logging */ + DEBUG = 'DEBUG', + + /** Info level logging */ + INFO = 'INFO', + + /** Warn level logging */ + WARN = 'WARN', + + /** Error level logging */ + ERROR = 'ERROR', +} + +/** + * Granularity of metrics sent to CloudWatch. + */ +export enum MetricsLevel { + /** Application sends the least metrics to CloudWatch */ + APPLICATION = 'APPLICATION', + + /** Task includes task-level metrics sent to CloudWatch */ + TASK = 'TASK', + + /** Operator includes task-level and operator-level metrics sent to CloudWatch */ + OPERATOR = 'OPERATOR', + + /** Send all metrics including metrics per task thread */ + PARALLELISM = 'PARALLELISM', +} + +/** + * Interface for building AWS::KinesisAnalyticsV2::Application PropertyGroup + * configuration. + */ +export interface PropertyGroups { + readonly [propertyId: string]: {[mapKey: string]: string}; +} + +/** + * Available Flink runtimes for Kinesis Analytics. + */ +export class Runtime { + /** Flink Version 1.6 */ + public static readonly FLINK_1_6 = Runtime.of('FLINK-1_6'); + + /** Flink Version 1.8 */ + public static readonly FLINK_1_8 = Runtime.of('FLINK-1_8'); + + /** Flink Version 1.11 */ + public static readonly FLINK_1_11 = Runtime.of('FLINK-1_11'); + + /** Create a new Runtime with with an arbitrary Flink version string */ + public static of(value: string) { + return new Runtime(value); + } + + /** The Cfn string that represents a version of Flink */ + public readonly value: string; + + private constructor(value: string) { + this.value = value; + } +} diff --git a/packages/@aws-cdk/aws-kinesisanalytics-flink/package.json b/packages/@aws-cdk/aws-kinesisanalytics-flink/package.json new file mode 100644 index 0000000000000..80dbdfd4aeb76 --- /dev/null +++ b/packages/@aws-cdk/aws-kinesisanalytics-flink/package.json @@ -0,0 +1,115 @@ +{ + "name": "@aws-cdk/aws-kinesisanalytics-flink", + "version": "0.0.0", + "description": "A CDK Construct Library for Kinesis Analytics Flink applications", + "main": "lib/index.js", + "types": "lib/index.d.ts", + "jsii": { + "outdir": "dist", + "targets": { + "java": { + "package": "software.amazon.awscdk.services.kinesis.analytics.flink", + "maven": { + "groupId": "software.amazon.awscdk", + "artifactId": "kinesisanalytics-flink" + } + }, + "dotnet": { + "namespace": "Amazon.CDK.AWS.KinesisAnalyticsFlink", + "packageId": "Amazon.CDK.AWS.KinesisAnalyticsFlink", + "iconUrl": "https://raw.githubusercontent.com/aws/aws-cdk/master/logo/default-256-dark.png" + }, + "python": { + "distName": "aws-cdk.aws-kinesisanalytics-flink", + "module": "aws_cdk.aws_kinesisanalytics_flink", + "classifiers": [ + "Framework :: AWS CDK", + "Framework :: AWS CDK :: 1" + ] + } + }, + "projectReferences": true + }, + "repository": { + "type": "git", + "url": "https://github.com/aws/aws-cdk.git", + "directory": "packages/@aws-cdk/aws-kinesisanalytics-flink" + }, + "scripts": { + "build": "cdk-build", + "watch": "cdk-watch", + "lint": "cdk-lint", + "test": "cdk-test", + "integ": "cdk-integ", + "pkglint": "pkglint -f", + "package": "cdk-package", + "awslint": "cdk-awslint", + "build+test": "yarn build && yarn test", + "build+test+package": "yarn build+test && yarn package", + "compat": "cdk-compat", + "rosetta:extract": "yarn --silent jsii-rosetta extract" + }, + "keywords": [ + "aws", + "cdk", + "kinesis", + "analytics", + "kinesisanalytcs", + "flink" + ], + "author": { + "name": "Amazon Web Services", + "url": "https://aws.amazon.com", + "organization": true + }, + "license": "Apache-2.0", + "devDependencies": { + "@aws-cdk/assert": "0.0.0", + "cdk-build-tools": "0.0.0", + "cdk-integ-tools": "0.0.0", + "jest": "^26.6.3", + "pkglint": "0.0.0" + }, + "dependencies": { + "@aws-cdk/assets": "0.0.0", + "@aws-cdk/core": "0.0.0", + "@aws-cdk/aws-iam": "0.0.0", + "@aws-cdk/aws-kinesisanalytics": "0.0.0", + "@aws-cdk/aws-kms": "0.0.0", + "@aws-cdk/aws-logs": "0.0.0", + "@aws-cdk/aws-s3": "0.0.0", + "@aws-cdk/aws-s3-assets": "0.0.0", + "constructs": "^3.2.0" + }, + "homepage": "https://github.com/aws/aws-cdk", + "peerDependencies": { + "@aws-cdk/assets": "0.0.0", + "@aws-cdk/core": "0.0.0", + "@aws-cdk/aws-iam": "0.0.0", + "@aws-cdk/aws-kinesisanalytics": "0.0.0", + "@aws-cdk/aws-kms": "0.0.0", + "@aws-cdk/aws-logs": "0.0.0", + "@aws-cdk/aws-s3": "0.0.0", + "@aws-cdk/aws-s3-assets": "0.0.0", + "constructs": "^3.2.0" + }, + "engines": { + "node": ">= 10.13.0 <13 || >=13.7.0" + }, + "awslint": { + "exclude": [ + "props-physical-name:@aws-cdk/aws-kinesisanalytics-flink.ApplicationProps" + ] + }, + "stability": "experimental", + "maturity": "experimental", + "awscdkio": { + "announce": false + }, + "cdk-build": { + "jest": true, + "env": { + "AWSLINT_BASE_CONSTRUCT": true + } + } +} diff --git a/packages/@aws-cdk/aws-kinesisanalytics-flink/test/application.test.ts b/packages/@aws-cdk/aws-kinesisanalytics-flink/test/application.test.ts new file mode 100644 index 0000000000000..6dc35fadd44b6 --- /dev/null +++ b/packages/@aws-cdk/aws-kinesisanalytics-flink/test/application.test.ts @@ -0,0 +1,604 @@ +import { arrayWith, objectLike, ResourcePart } from '@aws-cdk/assert'; +import '@aws-cdk/assert/jest'; +import * as iam from '@aws-cdk/aws-iam'; +import * as logs from '@aws-cdk/aws-logs'; +import * as s3 from '@aws-cdk/aws-s3'; +import * as core from '@aws-cdk/core'; +import * as path from 'path'; +import * as flink from '../lib'; + +describe('Application', () => { + let stack: core.Stack; + let bucket: s3.Bucket; + let requiredProps: { + runtime: flink.Runtime; + code: flink.ApplicationCode; + }; + + beforeEach(() => { + stack = new core.Stack(); + bucket = new s3.Bucket(stack, 'CodeBucket'); + requiredProps = { + runtime: flink.Runtime.FLINK_1_11, + code: flink.ApplicationCode.fromBucket(bucket, 'my-app.jar'), + }; + }); + + test('default Flink Application', () => { + new flink.Application(stack, 'FlinkApplication', { + runtime: flink.Runtime.FLINK_1_11, + code: flink.ApplicationCode.fromBucket(bucket, 'my-app.jar'), + }); + + expect(stack).toHaveResource('AWS::KinesisAnalyticsV2::Application', { + RuntimeEnvironment: 'FLINK-1_11', + ServiceExecutionRole: { + 'Fn::GetAtt': [ + 'FlinkApplicationRole2F7BCBF6', + 'Arn', + ], + }, + ApplicationConfiguration: { + ApplicationCodeConfiguration: { + CodeContent: { + S3ContentLocation: { + BucketARN: stack.resolve(bucket.bucketArn), + FileKey: 'my-app.jar', + }, + }, + CodeContentType: 'ZIPFILE', + }, + ApplicationSnapshotConfiguration: { + SnapshotsEnabled: true, + }, + }, + }); + + expect(stack).toHaveResourceLike('AWS::KinesisAnalyticsV2::Application', { + DeletionPolicy: 'Delete', + }, ResourcePart.CompleteDefinition); + + expect(stack).toHaveResource('AWS::IAM::Role', { + AssumeRolePolicyDocument: { + Statement: [{ + Action: 'sts:AssumeRole', + Effect: 'Allow', + Principal: { + Service: 'kinesisanalytics.amazonaws.com', + }, + }], + Version: '2012-10-17', + }, + }); + + expect(stack).toHaveResourceLike('AWS::IAM::Policy', { + PolicyDocument: { + Statement: arrayWith( + { Action: 'cloudwatch:PutMetricData', Effect: 'Allow', Resource: '*' }, + { + Action: 'logs:DescribeLogGroups', + Effect: 'Allow', + Resource: { + // looks like arn:aws:logs:us-east-1:123456789012:log-group:*, + 'Fn::Join': ['', [ + 'arn:', + { Ref: 'AWS::Partition' }, + ':logs:', + { Ref: 'AWS::Region' }, + ':', + { Ref: 'AWS::AccountId' }, + ':log-group:*', + ]], + }, + }, + { + Action: 'logs:DescribeLogStreams', + Effect: 'Allow', + Resource: { + // looks like: arn:aws:logs:us-east-1:123456789012:log-group:my-log-group:*, + 'Fn::GetAtt': ['FlinkApplicationLogGroup7739479C', 'Arn'], + }, + }, + { + Action: 'logs:PutLogEvents', + Effect: 'Allow', + Resource: { + 'Fn::Join': ['', [ + 'arn:', + { Ref: 'AWS::Partition' }, + ':logs:', + { Ref: 'AWS::Region' }, + ':', + { Ref: 'AWS::AccountId' }, + ':log-group:', + { Ref: 'FlinkApplicationLogGroup7739479C' }, + ':log-stream:', + { Ref: 'FlinkApplicationLogStreamB633AF32' }, + ]], + }, + }, + ), + }, + }); + }); + + test('providing a custom role', () => { + new flink.Application(stack, 'FlinkApplication', { + ...requiredProps, + role: new iam.Role(stack, 'CustomRole', { + assumedBy: new iam.ServicePrincipal('custom-principal'), + }), + }); + + expect(stack).toHaveResource('AWS::IAM::Role', { + AssumeRolePolicyDocument: { + Statement: [ + { + Action: 'sts:AssumeRole', + Effect: 'Allow', + Principal: { + Service: 'custom-principal.amazonaws.com', + }, + }, + ], + Version: '2012-10-17', + }, + }); + }); + + test('addToPrincipalPolicy', () => { + const app = new flink.Application(stack, 'FlinkApplication', { + ...requiredProps, + }); + + app.addToRolePolicy(new iam.PolicyStatement({ + actions: ['custom:action'], + resources: ['*'], + })); + + expect(stack).toHaveResourceLike('AWS::IAM::Policy', { + PolicyDocument: { + Statement: arrayWith( + objectLike({ Action: 'custom:action', Effect: 'Allow', Resource: '*' }), + ), + }, + }); + }); + + test('providing a custom runtime', () => { + new flink.Application(stack, 'FlinkApplication', { + ...requiredProps, + runtime: flink.Runtime.of('custom'), + }); + + expect(stack).toHaveResourceLike('AWS::KinesisAnalyticsV2::Application', { + RuntimeEnvironment: 'custom', + }); + }); + + test('providing a custom removal policy', () => { + new flink.Application(stack, 'FlinkApplication', { + ...requiredProps, + removalPolicy: core.RemovalPolicy.RETAIN, + }); + + expect(stack).toHaveResourceLike('AWS::KinesisAnalyticsV2::Application', { + DeletionPolicy: 'Retain', + }, ResourcePart.CompleteDefinition); + }); + + test('granting permissions to resources', () => { + const app = new flink.Application(stack, 'FlinkApplication', { + ...requiredProps, + }); + + const dataBucket = new s3.Bucket(stack, 'DataBucket'); + dataBucket.grantRead(app); + + expect(stack).toHaveResourceLike('AWS::IAM::Policy', { + PolicyDocument: { + Version: '2012-10-17', + Statement: arrayWith( + objectLike({ Action: ['s3:GetObject*', 's3:GetBucket*', 's3:List*'] }), + ), + }, + }); + }); + + test('using an asset for code', () => { + const code = flink.ApplicationCode.fromAsset(path.join(__dirname, 'code-asset')); + new flink.Application(stack, 'FlinkApplication', { + ...requiredProps, + code, + }); + const assetRef = 'AssetParameters8be9e0b5f53d41e9a3b1d51c9572c65f24f8170a7188d0ed57fb7d571de4d577S3BucketEBA17A67'; + const versionKeyRef = 'AssetParameters8be9e0b5f53d41e9a3b1d51c9572c65f24f8170a7188d0ed57fb7d571de4d577S3VersionKey5922697E'; + + expect(stack).toHaveResourceLike('AWS::KinesisAnalyticsV2::Application', { + ApplicationConfiguration: { + ApplicationCodeConfiguration: { + CodeContent: { + S3ContentLocation: { + BucketARN: { + 'Fn::Join': ['', [ + 'arn:', + { Ref: 'AWS::Partition' }, + ':s3:::', + { Ref: assetRef }, + ]], + }, + FileKey: { + 'Fn::Join': ['', [ + { 'Fn::Select': [0, { 'Fn::Split': ['||', { Ref: versionKeyRef }] }] }, + { 'Fn::Select': [1, { 'Fn::Split': ['||', { Ref: versionKeyRef }] }] }, + ]], + }, + }, + }, + CodeContentType: 'ZIPFILE', + }, + }, + }); + }); + + test('adding property groups', () => { + new flink.Application(stack, 'FlinkApplication', { + ...requiredProps, + propertyGroups: { + FlinkApplicationProperties: { + SomeProperty: 'SomeValue', + }, + }, + }); + + expect(stack).toHaveResourceLike('AWS::KinesisAnalyticsV2::Application', { + ApplicationConfiguration: { + EnvironmentProperties: { + PropertyGroups: [ + { + PropertyGroupId: 'FlinkApplicationProperties', + PropertyMap: { + SomeProperty: 'SomeValue', + }, + }, + ], + }, + }, + }); + }); + + test('checkpointEnabled setting', () => { + new flink.Application(stack, 'FlinkApplication', { + ...requiredProps, + checkpointingEnabled: false, + }); + + expect(stack).toHaveResourceLike('AWS::KinesisAnalyticsV2::Application', { + ApplicationConfiguration: { + FlinkApplicationConfiguration: { + CheckpointConfiguration: { + ConfigurationType: 'CUSTOM', + CheckpointingEnabled: false, + }, + }, + }, + }); + }); + + test('checkpointInterval setting', () => { + new flink.Application(stack, 'FlinkApplication', { + ...requiredProps, + checkpointInterval: core.Duration.minutes(5), + }); + + expect(stack).toHaveResourceLike('AWS::KinesisAnalyticsV2::Application', { + ApplicationConfiguration: { + FlinkApplicationConfiguration: { + CheckpointConfiguration: { + ConfigurationType: 'CUSTOM', + CheckpointInterval: 300_000, + }, + }, + }, + }); + }); + + test('minPauseBetweenCheckpoints setting', () => { + new flink.Application(stack, 'FlinkApplication', { + ...requiredProps, + minPauseBetweenCheckpoints: core.Duration.seconds(10), + }); + + expect(stack).toHaveResourceLike('AWS::KinesisAnalyticsV2::Application', { + ApplicationConfiguration: { + FlinkApplicationConfiguration: { + CheckpointConfiguration: { + ConfigurationType: 'CUSTOM', + MinPauseBetweenCheckpoints: 10_000, + }, + }, + }, + }); + }); + + test('logLevel setting', () => { + new flink.Application(stack, 'FlinkApplication', { + ...requiredProps, + logLevel: flink.LogLevel.DEBUG, + }); + + expect(stack).toHaveResourceLike('AWS::KinesisAnalyticsV2::Application', { + ApplicationConfiguration: { + FlinkApplicationConfiguration: { + MonitoringConfiguration: { + ConfigurationType: 'CUSTOM', + LogLevel: 'DEBUG', + }, + }, + }, + }); + }); + + test('metricsLevel setting', () => { + new flink.Application(stack, 'FlinkApplication', { + ...requiredProps, + metricsLevel: flink.MetricsLevel.PARALLELISM, + }); + + expect(stack).toHaveResourceLike('AWS::KinesisAnalyticsV2::Application', { + ApplicationConfiguration: { + FlinkApplicationConfiguration: { + MonitoringConfiguration: { + ConfigurationType: 'CUSTOM', + MetricsLevel: 'PARALLELISM', + }, + }, + }, + }); + }); + + test('autoscalingEnabled setting', () => { + new flink.Application(stack, 'FlinkApplication', { + ...requiredProps, + autoScalingEnabled: false, + }); + + expect(stack).toHaveResourceLike('AWS::KinesisAnalyticsV2::Application', { + ApplicationConfiguration: { + FlinkApplicationConfiguration: { + ParallelismConfiguration: { + ConfigurationType: 'CUSTOM', + AutoScalingEnabled: false, + }, + }, + }, + }); + }); + + test('parallelism setting', () => { + new flink.Application(stack, 'FlinkApplication', { + ...requiredProps, + parallelism: 2, + }); + + expect(stack).toHaveResourceLike('AWS::KinesisAnalyticsV2::Application', { + ApplicationConfiguration: { + FlinkApplicationConfiguration: { + ParallelismConfiguration: { + ConfigurationType: 'CUSTOM', + Parallelism: 2, + }, + }, + }, + }); + }); + + test('parallelismPerKpu setting', () => { + new flink.Application(stack, 'FlinkApplication', { + ...requiredProps, + parallelismPerKpu: 2, + }); + + expect(stack).toHaveResourceLike('AWS::KinesisAnalyticsV2::Application', { + ApplicationConfiguration: { + FlinkApplicationConfiguration: { + ParallelismConfiguration: { + ConfigurationType: 'CUSTOM', + ParallelismPerKPU: 2, + }, + }, + }, + }); + }); + + test('snapshotsEnabled setting', () => { + new flink.Application(stack, 'FlinkApplication', { + ...requiredProps, + snapshotsEnabled: false, + }); + + expect(stack).toHaveResourceLike('AWS::KinesisAnalyticsV2::Application', { + ApplicationConfiguration: { + ApplicationSnapshotConfiguration: { + SnapshotsEnabled: false, + }, + }, + }); + }); + + test('default logging option', () => { + new flink.Application(stack, 'FlinkApplication', { + ...requiredProps, + snapshotsEnabled: false, + }); + + expect(stack).toHaveResource('AWS::KinesisAnalyticsV2::ApplicationCloudWatchLoggingOption', { + ApplicationName: { + Ref: 'FlinkApplicationC5836815', + }, + CloudWatchLoggingOption: { + LogStreamARN: { + 'Fn::Join': [ + '', + [ + 'arn:', + { + Ref: 'AWS::Partition', + }, + ':logs:', + { + Ref: 'AWS::Region', + }, + ':', + { + Ref: 'AWS::AccountId', + }, + ':log-group:', + { + Ref: 'FlinkApplicationLogGroup7739479C', + }, + ':log-stream:', + { + Ref: 'FlinkApplicationLogStreamB633AF32', + }, + ], + ], + }, + }, + }); + + expect(stack).toHaveResource('AWS::Logs::LogGroup', { + Properties: { + RetentionInDays: 731, + }, + UpdateReplacePolicy: 'Retain', + DeletionPolicy: 'Retain', + }, ResourcePart.CompleteDefinition); + + expect(stack).toHaveResource('AWS::Logs::LogStream', { + UpdateReplacePolicy: 'Retain', + DeletionPolicy: 'Retain', + }, ResourcePart.CompleteDefinition); + }); + + test('logGroup setting', () => { + new flink.Application(stack, 'FlinkApplication', { + ...requiredProps, + logGroup: new logs.LogGroup(stack, 'LogGroup', { + logGroupName: 'custom', + }), + }); + + expect(stack).toHaveResource('AWS::Logs::LogGroup', { + LogGroupName: 'custom', + }); + }); + + test('validating applicationName', () => { + // Expect no error with valid name + new flink.Application(stack, 'ValidString', { + ...requiredProps, + applicationName: 'my-VALID.app_name', + }); + + // Expect no error with ref + new flink.Application(stack, 'ValidRef', { + ...requiredProps, + applicationName: new core.CfnParameter(stack, 'Parameter').valueAsString, + }); + + expect(() => { + new flink.Application(stack, 'Empty', { + ...requiredProps, + applicationName: '', + }); + }).toThrow(/cannot be empty/); + + expect(() => { + new flink.Application(stack, 'InvalidCharacters', { + ...requiredProps, + applicationName: '!!!', + }); + }).toThrow(/may only contain letters, numbers, underscores, hyphens, and periods/); + + expect(() => { + new flink.Application(stack, 'TooLong', { + ...requiredProps, + applicationName: 'a'.repeat(129), + }); + }).toThrow(/max length is 128/); + }); + + test('validating parallelism', () => { + // Expect no error with valid value + new flink.Application(stack, 'ValidNumber', { + ...requiredProps, + parallelism: 32, + }); + + // Expect no error with ref + new flink.Application(stack, 'ValidRef', { + ...requiredProps, + parallelism: new core.CfnParameter(stack, 'Parameter', { + type: 'Number', + }).valueAsNumber, + }); + + expect(() => { + new flink.Application(stack, 'TooSmall', { + ...requiredProps, + parallelism: 0, + }); + }).toThrow(/must be at least 1/); + }); + + test('validating parallelismPerKpu', () => { + // Expect no error with valid value + new flink.Application(stack, 'ValidNumber', { + ...requiredProps, + parallelismPerKpu: 10, + }); + + // Expect no error with ref + new flink.Application(stack, 'ValidRef', { + ...requiredProps, + parallelismPerKpu: new core.CfnParameter(stack, 'Parameter', { + type: 'Number', + }).valueAsNumber, + }); + + expect(() => { + new flink.Application(stack, 'TooSmall', { + ...requiredProps, + parallelismPerKpu: 0, + }); + }).toThrow(/must be at least 1/); + }); + + test('fromFlinkApplicationName', () => { + const flinkApp = flink.Application.fromApplicationName(stack, 'Imported', 'my-app'); + + expect(flinkApp.applicationName).toEqual('my-app'); + expect(stack.resolve(flinkApp.applicationArn)).toEqual({ + 'Fn::Join': ['', [ + 'arn:', + { Ref: 'AWS::Partition' }, + ':kinesisanalytics:', + { Ref: 'AWS::Region' }, + ':', + { Ref: 'AWS::AccountId' }, + ':application/my-app', + ]], + }); + expect(flinkApp.addToRolePolicy(new iam.PolicyStatement())).toBe(false); + }); + + test('fromFlinkApplicationArn', () => { + const arn = 'arn:aws:kinesisanalytics:us-west-2:012345678901:application/my-app'; + const flinkApp = flink.Application.fromApplicationArn(stack, 'Imported', arn); + + expect(flinkApp.applicationName).toEqual('my-app'); + expect(flinkApp.applicationArn).toEqual(arn); + expect(flinkApp.addToRolePolicy(new iam.PolicyStatement())).toBe(false); + }); +}); diff --git a/packages/@aws-cdk/aws-kinesisanalytics-flink/test/code-asset/WordCount.jar b/packages/@aws-cdk/aws-kinesisanalytics-flink/test/code-asset/WordCount.jar new file mode 100644 index 0000000000000000000000000000000000000000..9c533e6fea60771f319c0f4923cdbf728bdc72e0 GIT binary patch literal 15192 zcmb_@19)Uxvv$X}Z9AFRw#|ucPc*S@Pdu?EwlT47PBNL;`ZM=n&d2-Rd;ir>_ul<< z)my99?zPsg+Vv_(gMdN!747){A>n5^GvM^=ghmAlE=h5bNP{MHD7%dyfy)`Dju%4m8JKP@_LTr;QQVkKB zlak36(;6H)4>VqLK#KV~zx5R_KI?s@=G+1^f;Z+@*Ht%lOj3c{21^ z`zM$#`3B!x+9oXu(N)GSPmz^CpW&xRmz6_&g=pS27YH9g;q08iJ@ttAjv=QbAFzlR zN6qmJue&UsN6K{iG^lhl@9L@e5%zu=C`CY=m-oVJ*wS}$KzY)jz zXW~xIjwS{+7PkMcL!|#+(Zt=r#@^b*>ED>9{|^z|>>Q1a>|AV}|HOg>n7#eix>jI$ zj6eW@X9xg5{C|w4{)Q_26O~lO&dS8r!o$Ro-pJa($*DqhOc_NI^_5k!$N>{wBtXpV zxhE(vCqJ|pR53DVDXLmkK}-9%5G#a{IZblx3UD+>{R)Gi*Y@3nq3KvnGn@Aj;qJXV z@_Tt;bkcf-@+6f`_NtA+Q*Y zP3!>mb>R+><^@FlVmM5hc;NibLT)fYwfpbg$!!XZeTNOTI=Bf$!A&(DC11mHv z-ZaBwnKMIJEVL&_hSdj{K-L8!dl0&4!OA_R1)Y_$p~AZhS|(g;OB!9>^Eu{L2{)mJ zM60PgymPjYI?24E4DD+XtZ6#y#DOI58yJjlx`&`}c)V0ldGD<1@Cz6cC`r<>y+Cd- zm~0MnRU?8_ar5SiX*L<75lt*TB&4xE5?gRjl{|4RdmtZ6jA!CVo5r7tDTgW2RTL^$ z#ZHjUET3UcoVgU}<+&ZcH}5oazpy)FH-CkISU%G+s$%CW-5ln)V}eEqy!u*^gHSeP z1c%bMG4KU*5;Ewd*I}9)zOsishw6uTd`;>j6a{2aG_c$@s)@G;4O+ch@XbRgzh58t z^d_zur_cW30cH+AN&9!R@PsJo_~Yz?ItUz#nAk>8MQ_Ge{D(;{u9q@<4-j3x@fd?p z>tnr%EiW0_s{xOzj%_Z#VK1sS6G@YZJQObcY}z8>rmn1- z<#0!nj8M6!^#IiIi1p$>qB|!jNU&!h-}0z3$LD!EGiLYk<%tGK_w(uP6l^=Y*eT0{ z?^H-6pP_TW6mot>B%i5s#FUMg8|Iox-RL2H{$nJc!IkgS=SV)ItJtZ7$PWFwoHxR;H*C<*1WyOcav1iDU)^2wU-W`8RUgs(hQ;;oF3HCfm-jpp|1FJS&iT>-zq z%`Wa$q$@N+GjOv~8*fpx+^&LsY31YT}58IKrK6!vC-Z*O6S&g#dpjWUJ5~fHrCjyC~?RZrZ{O`v~!}A zQN;=Y34|6mD{WeX#&dD823Bz_EDaKPnl>VVHkwl^@Gq?WfC)8{i&4i0X|zC_-W96CV>1b?pSe(&fP;maRMb*R9U}-9Iq#ukS?u<{ zI!+6=ZltI+Vw=Ap1^Anl6}P%_o)X)baaw70HBK!s|E@hz>=(R2kON5&rh_Pm zR_TErx@6wu;9Gh+KScZ*l$9wDIT5r+asDwRs)4-HX2`+krOuy=iul*>v!D=+5#C>Sl-1jCLGP!y0oB5zT+Pzl6+I*ylY4HG$!nkcRSb>>y=m zSzX2t|KSEp^nFKPGTo#R1wOp5NP-DvIi5o{Ri9_rg|3pvi#kZQIboE$YOxifiAY#WzsKTm6pkK`b{G80xqcMNM#d)KU zYk7gSR1uvWLV?Z{K*l@~*Cp+iU+B^c$AaIJH8)+(s>dh$B|{b#}zsFt_j{> zn#qNEaDDnj!_IOsJj<+iJT#~_X-=-+=#dRmS@fguqeAfcCDWatuP<--WsX|8Ia#@4e#b!3 z1ZJ!i*a^yqyzhRu7P>anFjQ}(t59KUBp73kV{BSS$nE9y_P8Qu>S*7ojm{jJU<>OB zf-ra39{OcDsPlXexWRsIz|Mbz*NI>8f|^ecG=#{j`lJ}{T?6KiuN5U2esX26}OAVUK&a-O#R-$3Q#p*_2g~? zr5Cb7x_6AWir?k@L5b=IpO#P<&zX zR2(^*Gr(jgt}(V)?8j6<^-Hx^D)}XaGagzw?m-4v-?7v}Sjus{5lY_Sxu6hHg7VKh4-E3|ydd&!Nr7Yn5vwxTOcrp~!Jl=}Kd&scUTH zlf@Th!=E-NpQmQkWZFZRMLEt3AVG~)gHo5;nJ<-`O!pLtWrrpZTLq{T4^_uwiD&a0 zW{hf;MjXhG_ezh%Ye`AldoD>_*A*bSYBEO~6!lhM%`g@3q7UbCzgupQ4X2!~&;`aB zH_J{Gp(ZiWhRSLXUCk=u^@z-~DYMX*J~}@1G#^o&pkS=zOm9Ti8!VwvPT$=YPoNI= z$Y)hrZZ{FuNQtU4w*GJ?3*&=tKS9aSSEpj1DoKrKzKk-HfP7KsPZAkc+LS#X7BgIJH-$Xaq}H!cjPD$1aAK$x zNb;=W5L0fjPMAJ6mE|GRQn~2N4z?dD2#mKIr#^gG8BQNbKaI|ysZlSXFM5r+g_S2p zUmW3TVH)g%{{@v8rG}*>wPQW@Trqv5olCOJkW39p8tqg6GpI^G$WSRqxDj-88Hb-T zHGZEkHD3$^rMpHy$sIA0`Z*a-={A#>W{!1JGIfXl46yV73O6C)f-Db*!-A-kpoDXqmAYoJj~j&u9LAaJIuN0<$C?z8g@nrS~M#EwB0n z%ddV0gVxzsg_-L*kn2P36SYx%N9RLP?E!_{bH&U$u&p9zPn&CJiDC3IebS+# zGo?UoKpPJ`!yMv$1s>A8(3E6LbyZudqEh>=u>OK@mhL~i&|_vmwt zs9F$qL$f-;I&(?FgyZ`nl>?i4YCLQ^ilBwshFw)hp0$3fz}g5?T6MP!Cf>@j#k=u? zVJES(W(^z3Z%ixUIJUB+z^l>MbxU$MICzg*+1MPVJHdvWP29Idc)Wrbix+&nbNOuF zt5--Iq23(U)Jes2zN*3Jlfzw%F#P8#q6bH)DLn90E1qA`C5G z38te0pv)@TKjWgZKfZUe&rz(n<(R*v(}@K~s!V$@ZT#YqZY_61CJva`(gBQsfmY&H z2t}>4Sd6qt=R!=KPI)feK!4}im}BmQ#{dSqzjCKv7HjCKVPp=TNeldrI+ij7lQ_i8 z`Mn{IqRR31460SR+|z3M#Foga)tM~ZxmKY0p1c#CUMe!Ll}^M>drTw4?fveTiCyeT zwc8sBzc1OhE8;ns5^O9R?b_Bjp5tdypM<9-Iwnt^x4Q-=M_UQB6HMVxO`kbCjhfD6 zgZt^(0<4}bO42TRndVCRoo^}N{S;o-x|ekw_~JvSOf2NcMY-xv&-jMXgq^WO*xjr* zOo73&U_xGM`N3nbGXoSIlPyLb(Rx|3pS=9-3n~?&nDR{!rtp?s|)s8U*U^O*V1A06uLd_gO zA*nuEKWG)5o3n=rt&L&4mGMb5&OC#Fc3fyFB$WG{Ks-#Ky=vMG7s$_BgVqcMl)qu_ zA#Zt*(LvBe5gla(Uf;6f6jiDb!viLcmb@O1!>xr)!xeT94}x(A(!YCHQzmXlB^o9= zhd9E*rk;Yy`%zKF)L{u~6i}P`ZDbqly5qikL|4{7Y5gilGmx@a2Rc<0D2X&pd4$dzg| zJ3;OreQc{i@T*aHO$h9pKy9d>?Md(rinGa60^QJvbfOUFmDS=vQ5KZ)K%W$))VdD0 zqvcXFlL4CE^+1`klewucE2XJ}*BM#55u%DK+yp_H* z(X2Xdphm;pAhO?^bF-?MK%IpU0Zux=CcyXsNjTF8o;j3jMC{N?=?gjiuO58!z)k(&gso0SAdBZEnuNyq$w+vui-FC1I0sx>#3jiR#-QfM1 zl$Mbc7L`*LUDMKbS{Xp|xvg4sEK;@rFUopnfpZX6vm0C?Y$4KYOP&xyR!!IlUOhxl z82)_Co6_!AVC?pwCR@ZR`F-ij!9~_-Wl()BwA$LOkI)?CfXFn>S5E)XjRDdHHOZmh-%mKDn?=NG4xRRv@O^*+Of_( ziL#3ArB%bm1j|*2g*t%m=Ob6L?&MOj`OMwN6OGDRwlo-wAhHMl9L3)xDOC|}W z<%_kc$uQ6bq9<5^YX=98S8f4KQ~^zZt<=&rv)O2*-d#IgSMjZl^fhPk*kI7ny|>xj zjP~19OanV2({Sn3R<}3{gNc~9!?w`-3#BjW4S-wCGi2&q?iV*Aa1tpfxxs@r)R!MIXSu2K?#H?+(xqmT8|-|j0v+?oz||es7IDe?%I@^`TC(tt zyPNA%1`BSGeS=;I!JSR;YOV*ss&Ntwvf!?mi+-R|01d@73va^6N46aqoi44Xlq}tS zy^z;Qa)cIa#bmS(UhY6gNOg1YXh7}SC%9ld{a6%0&3mM?^VQl=RFL21@Wr+4R4dAZ z_O7gC(O(0#>x!oB-$`ns;g+}7Vhy9l2IjE@m^+s_xSi#*TvrdCIo1(z6apWItda=o z!SX8vK2=@2)?INo6)IwRty;f;<=U6Fe20I+YcjH?Q*_g}_s&Ak~R{ ziN<|FzP9?{?i@Wh+R*s(#%i3f8rnlbg4zt*!0R<15h8s;@dv0^!&ifv zt2e$mruYGISovZD(k$iz$kh!|11!9eV2$@ zEo0{b(VF@2gc<`PH_UWZC$j=?2XR&EI&3$aGQ zyb7KNwo{N$(E=;C01#~g)F!j{W63fO1x>_c8Gpw+zblnBb=%_N!emZwbUV#d4}IYga->>BEmN?uKivgeOF9uVDj1Y z%ur0Ad&#?!LH)pQn!-FiiaJ#zQ<3oTjY&5sLZ4*6Q!3E!T{Vz=4^a>yh%=ZFBle%O z2D?cno|wD;{w^ry&V|QysB1zgd(}5I=47hK5aO#fY9ES%n5YE2)WKr`sq;}I()Qa_ysCrkDv2`tmB#ZsKh3b~q5iqSqa{;v=`XlRXeyF%SQ3PRzT915q{F?)ASZLm-85L+RX ziDN&hIBmRc1?R5#qs1EW4xB?xlX095!j*&32<6;F1q#G$_>SvX;(70ciLN*9z2i4z zirY)Tn@Du(L3f=bV1ZbZ0$qCrIo0W6otbOy3}Mj1=58u;J(iZ z-2)3Mj$w0&;z0=awFLOwe*{vD#9L%Sg5XiSDUa1B&xDoj;)8De+F@6{_70CDt2}aX zPn8Czq=E6v<6jH!gIx1~>Jec-rf5PHn5GA>_odvfq9aDta2E5l4^M0NnxGjZpzJ zT`gaZ3Ehzv!JQ-@ykF-+g8;P@tq5ta0Ab70oo}Xr-wbL&V$B9Tq=-wD@At}HLoB~y zO1~vfs2Qh4#4G__p+(5dr;eMLNua<_AJKSjt|Ic=~gJD7Uy-; z8qrG@u8NX3#}Cqu&&=32tHz!eo8*cX-}9?K9^^*%Bl_P6x?xhR^0rue1~y?%Q=(|! z(qReLWPTerY51ZFLaKq}?lj+{>;}NbBcNa5uuTQd>w=Y@t2vv}C8Sk@{-7Ke2L(Mv zRG7h!6GJ$D=mcAGmy(K;Zx$=~RW%rrP6dmHJUxMB9KOW7h+1u z%u7>4o?;~CGr>d1{**;l6sPqbvlFRqaAs^Wyawg_#B~PACFnj#HgA;KM8QGhW8+*& zm1Z-SJ3fd@$=JOQym$gox)hn+@y)VmXDS^v{Uu2zn;uJvBe&pY9o?-G_i@RX zUe;I-mG*qp{_b`WI$8ia*B;bCiUQrx0V2L@mgfUKh@)EAVrdOxlO-YpK>zXT&mF@Q z1&bSusUQ4hSY(F!FfA2g@tosjkx_n4JCjxMh&r@kx;h)GYig6m2@LxDU`TZ!E>mot z%b?i|(+m3~tC`u5X-PM(WWrpzpsu7ulZR(2e0QcfRf-g$alGh*q+h~HsU%u-IgZ(| zNFh@+xl!6;39bt;N{E;ASac`UrXFjJ06PW&S8-l+W)Z$YzLL?SH1Zdic#aJY+%H4c zsYy{a*|@r7MKRp?2#%>oHW%M6KhC6oNEfv<1?nYEZBg60iCd@|^^Mp*ftyQ-+A4yb zag{#7OGZ)}i&B8kEJb|}YJhD_e~!b{x(P(3!PFq4efW*?41rI$m_#|Ibd)gLmmRyW zw4%gr5l_W6u7Q<5JeGhA}wSz4>7MxsrqIXwX z$15b)J?4Ih@FaMYa6tjvTIPD)tzM*vy+ zv~X-!S`te%+1sXQVovr)saxw%0VF!?g<^ zm-hVy2z0%HlWln1o2}<Yie0@2Hja=l6#}m04hGCChvXhEx?nRF+SVn4YQ#ifg z$@@i5>sJqh=_gV&K7dVrvL)oz0QZ`ktVVGTl;g&!56qy2%poGca<*JC*MDEwLc49_ z#)P#uFg#(l^Xwl6Jbs1KP9!A$)y4rI>6ES~8KlE+^U(Fl%ZJfJ?kfbmiT?<;EY4#X zpjoZKkvnT^=jcpIz=N~v8v1np%Zix zQN@7v&W>10Ij2}8);-SJ2Q$;Aba`0%>~<6ct*siIUZ7UiqUB@qPk285dDahl+lh#r z-J#(NO6>W2mZTXT-;tYhiQV^Q5!=AJHpfo7acCg?0vtj?pWP7;O!^F3%JJW_Lh~=z za6H62>1awzj4-?pYpVzeV#D!w1FE%GzhceQ6r#|>RVBUwy5V1<$g zyR})Sm0c^AqVB*7;@Dp+RlGv82XpYB;Zfa6PtNAz-ys@(VH0#SV2k=Bgk<+2s5)RY zrAlj)CzYY2a~ep{DBwJQ$%vw_!uTEL5(`S^xVT-Nv#cu)GG}8adM;7rE4JUeZpv@c zG8Aihq$2VOd9<)kGMJv^x-5J~Jx(17Hb>?eQgf4)XVCfsb7@n}b#z3EgbP#QR9WIy zGd5|Ta6!n0)uYF#5fme4n$Pz)hsC%if#_nbv4@}dA_gMa#B@1)tJKvs)F;}(#XA{n z?Y83f_&0K3EVKZj9%oeyyH!87>c^lsMAm*GnG2KHAU=NJW2G3A@XTeVaM=3Yq zxGIoHWVF{1VQxwg?$D&QR($mpdMy@Q7MDKL&+%Q87;LsdMe4fOL&|#vSx6qd)1CIc zo^~tTi2|aKx;=Qia!$-?lWhiwyGPY0A&-D;@y7Db=^Z|a(sHx(VEMRCL~8mozq|3h zy<1A+D-uXhosOqFo=Nw$d$zelqywJ(K4ad!N_2vplYgm5N-swA&oqbYgT<{_iAJ;+ zfXbnD9<;l;w~*XE{&A&qiSY{bx1l+b#VBmZTS(sg_I``de}(39@+xn^xpyQtd=De6 zfZL}Y64)z>B3sBQm^zGV>=}^)heR$=Tlt{AuUG45SNdo4Jk^6M5%Y&1R$ONL>RPJ1 zRJXaC`>9=JJ^5T@7q*SD7~=Nv@>lB>3~@;zgpWu8GV?Z=38_3ZEsT%WH^(>9aQsWs z#sLLquR#!H*pDsrg(_EL)RNNP<$%pU(+U4rZ}m5QwlQ!uvHi2i;2(tv|LWjx!f(}F z^uN_=(f@ZPSbYD1_J3i}{ZUj!_h*6DduDnjrayZI37D>+QrZW;ZzF;Q04yZ|0MY+5 zw!NJVy}OO|iH^423J1DR^hdCTLtz{drRi_Lq6^nu8RQyfIG;n;T=z?I$d*ls74_o7 zjbI-(ZewSIS~%3SFyyMWQHw?64!n5Wq{L62eF%aMqwNKaZaO@-@!gONd~g-mK5jho zaJsF#emr&a_~`Vuj2xkmOZXZ`?)v9+BR`%^aAXgo?Sp0wLcvsn3LWn0d%Zw+lWkF& z7N9tx_A~a!r0qJVOJG96aRwe_#)dLd=>!f0nlI;<4WC`Xj+skv0il^9RqH7>`$DPY zsE*oA!5(fB=7dwhBbhKFT0r8-?WdO60A}{mXYPga-23%E!?b|Pq0|SQe*hQ8EKMi+ zZih!f&H7gD5o(aF-97g1|iHuNkq=v5Y?MmllcTj?*lBm;cU`#Au?Op zH=;Z7T5pLDZ@9Ap^iUy4WlkEapsw1cQ`74}Raur!adEHqdE}CNa}ZEjw!g&Iu4JdI zDSly0aSrx|N_nUh{=sw@7Mg9gro=L%FvIU4uh&Y!%h0NJ%K>Gu;I@mB9MN%n(>4v* zykOSPg3#d7_XQQtIk97SLFR+6;Vy)|tk1Z3&;yz?;vjx@5(RA+HlxD2;~a;)=HcxR z9q8vnMJIYkEu}G=S7{o&YfGRwFdgf%w|eMM*{` zG$JtJ;6#Oo1D7OTPg-tbj~7b@bZO$lrC)tVy}#V}PTqx`K!>K1Zf=$EN@so?uA`R@ z%r>n%qEqR#c7K_CYf0cEoPE?C7SS!ylVGivT&U52ugxeP)RG+X3vJED_R@({ElN!n z5L6e#vo_KfEyBukRUlzxqK`IR3b>gf)AqOEUXv&alyMz~QBqgyB7BVv!08 z%9Lf5OPcODY5;_ohD=a~oh+rgQWB1y#{=imT}(a0IC~v0C90tsE>Rvt8dp-mc#EL| zTn1{%j-e~WtmLwYG^s22j2(_y!I(fTja($TucxrrwGl9Y6j`Y2t)O^aOSnN176Jnng#DvG1;I{%X z-=5yjpx_k5Vr5aR;zogRAmnq0#}R=*@Pdn@n$#5$j$##T+Grj^5~GHM3s`?ZO~7>m zW!%NyRRT_yoQE}qK%}{oVaj}_di~K{`9QP7fw~{NrAa&Hh#@DyTO~xTDwai~@})o8 zDR(?fN|Yrt@QfiDbBtqlGD+)WcwwhN5j4dLSZc77lbY1^oiAn76ZG*St(2qulL$JS zO;o{KKd`S*igIpvuy^xEz=rA(2DM!p9Hz5;MXv1GEbxV!Mk_~6J%QIr}QD%HnxymncJ#0_8=n4J_a%M@x7EdCr(Vx_v4lCjm6dx`()v4949qh z8_&Y`2|tln6j5&QwH!Oev(1d1w-;4{z_1l_S8z0RjZwVdZqjM6?-QSpI*UOy#%bi^ z(b#nRZI53J;G@brgU53zSD{(kO-{`19{UDomy061AKa`cqb#Pa$onItJ4kHd92J zC%Y8RS3c`=aE?~C(9il_&?-2WR>IQ!rB~!I_|jPMUB+JBuNTZx5tpiFitg1xP%7;#j#eeOg);E&U3IJsW$W)4er}e5T#_``OPEmNz99U9}VBJjYj#d&x>uyxEWS7k@bpR9bMC6ph@aMW*QyUjd%Gm{eGW3(fXl!3xwP=~?3#7RFuX*iLXvkZi+cfPxz3)8ZK^p<8VA zd`Gw}P5gDS70x%?-)tAqj@^Z45;F#yo%KF4%$COX1byz_^ER9-=b1|sZ(`;mjJiX& zRV3+!bqPs-o}%QOGO}h5iUes@Z9-w9)-2mI()LNyioT z)pqh;gBYz5@o|H~h31Hs?B!f*FF8wTr$4>19ScxqF6rtGJB#7vDEhom;C9wVc|e^X z|Mvm<$rupg@wShKrW`YE3M<#Xc z)RC}IlpiR!%u$~gzI7GkEY-(}r`FEQ=i=>d&&$*k8rZBa2ax#{!JSse5Ls={WPN_x ziUtE0$Ho=14D;b;%3Ga}ZLGkkYQEj%GLT@H;4*Bs^j4J(iiPo(K$I(D@pD$;5)$&x z4586oWs=@YLS~7g8XAO^g;ILg!;PGSIO5+s0^=3(ey^e_{iz{uz?xNedz2C*DrAfL zhK-1oY`KL1m{%&YWo& ze#_|-Qdq)*U4C{7{Osw*>+`P5wU29BjrE$e$9{-2R1ZG0Rq1 zr+HdI$`=jH9ewM>HQ=pjCWt#KvriO;{DJxEgC>+h{&%T8z&Yr|C(m zN4MBbAshy=z_W^b&Sws1Cs&s5uwcc_7QPLawE4hTXvczWMI3)oGtRKp+_NnVmxm~) zD`7C%DCECBAULj*>?l^j*GuZ3JF0!+5;yXnsrIO)gu6vRXcz7UE!~*Tg4m~-vC7uN zvO(j!SY(2{Qr{(C#Nnc}x=^_n7?4=OP@3MrSAF6jgvo?t54YQf=m{(OMk~=o`ZO30 z0f+{@bqO=ky1M?%W|Q6);rSL=6wsL+4gSSg2_^W zj(MhEFKL|QhRX^@H2;7R`4Q(aO1iz>n)LLC4NPa7K0}Vx1g5s3onB)K-Ryh3y4Bv` zAkQU{sjWqHZ3yu)NO5e8V{ z5l+s- zo0CAms38BE74_!nTSEk#2>v|&`8fKQ+COJW{S^M$3j8SEKL1qv3uo#V!aqizKgmPC zHAKMcs}j&3RHA>~)t_XdUmDK8Yy3$q`qzp-$wj{uv;J1`k2v>VtN$b${Zb$OXX<}O zwr@f0zXtkAKKca&_C_}Q$)5V%f#3S^3+Rui`QIY`BqRMo6#owq|4L2zqqqN@ob(5J zfnUP^82isk)W4@H{ktLh$)xzL{rp#Gga2nk^vj{&c@=-_)!#EK{xDJCSN^{<@TY6P zcox5$`sY==ziigMB{}|+>;B8BKh6DC)B9Ib|E!+mm#HN5zs$?ut4sb~`|nd;zqA?2 z|Dyd{0_>OaKThbMxu@US&r7%aSLHu4QGZ;Pe^e&?L-%bNdVNKDTTuR7PyS_Q>R+S% z%uW5)exk|$2WbCYrs@w{|2$XqhqVH~vj4NK-!oUgMjG This module is part of the [AWS Cloud Development Kit](https://github.com/aws/aws-cdk) project. + +## Kinesis Analytics Flink + +The `aws-kinesisanalytics-flink` package provides constructs for building Flink applications. + + * [Github](https://github.com/aws/aws-cdk/tree/master/packages/%40aws-cdk/aws-kinesisanalytics-flink) + * [CDK Docs](https://docs.aws.amazon.com/cdk/api/latest/docs/aws-kinesisanalytics-flink.html) diff --git a/packages/aws-cdk-lib/package.json b/packages/aws-cdk-lib/package.json index 10849cc784a4a..fda319d7104db 100644 --- a/packages/aws-cdk-lib/package.json +++ b/packages/aws-cdk-lib/package.json @@ -202,6 +202,7 @@ "@aws-cdk/aws-kendra": "0.0.0", "@aws-cdk/aws-kinesis": "0.0.0", "@aws-cdk/aws-kinesisanalytics": "0.0.0", + "@aws-cdk/aws-kinesisanalytics-flink": "0.0.0", "@aws-cdk/aws-kinesisfirehose": "0.0.0", "@aws-cdk/aws-kms": "0.0.0", "@aws-cdk/aws-lakeformation": "0.0.0", diff --git a/packages/decdk/package.json b/packages/decdk/package.json index a5c184e65f1d5..e030318880208 100644 --- a/packages/decdk/package.json +++ b/packages/decdk/package.json @@ -128,6 +128,7 @@ "@aws-cdk/aws-kendra": "0.0.0", "@aws-cdk/aws-kinesis": "0.0.0", "@aws-cdk/aws-kinesisanalytics": "0.0.0", + "@aws-cdk/aws-kinesisanalytics-flink": "0.0.0", "@aws-cdk/aws-kinesisfirehose": "0.0.0", "@aws-cdk/aws-kms": "0.0.0", "@aws-cdk/aws-lakeformation": "0.0.0", diff --git a/packages/monocdk/package.json b/packages/monocdk/package.json index a02fee1b78939..f06d5a236d5e3 100644 --- a/packages/monocdk/package.json +++ b/packages/monocdk/package.json @@ -207,6 +207,7 @@ "@aws-cdk/aws-kendra": "0.0.0", "@aws-cdk/aws-kinesis": "0.0.0", "@aws-cdk/aws-kinesisanalytics": "0.0.0", + "@aws-cdk/aws-kinesisanalytics-flink": "0.0.0", "@aws-cdk/aws-kinesisfirehose": "0.0.0", "@aws-cdk/aws-kms": "0.0.0", "@aws-cdk/aws-lakeformation": "0.0.0",