Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(pipes-targets): add SageMaker #30696

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions packages/@aws-cdk/aws-pipes-targets-alpha/.eslintrc.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,6 @@ const baseConfig = require('@aws-cdk/cdk-build-tools/config/eslintrc');
baseConfig.parserOptions.project = __dirname + '/tsconfig.json';

baseConfig.rules['import/no-extraneous-dependencies'] = ['error', { devDependencies: true, peerDependencies: true }];
baseConfig.rules['@aws-cdk/invalid-cfn-imports'] = 'off';

module.exports = baseConfig;
16 changes: 16 additions & 0 deletions packages/@aws-cdk/aws-pipes-targets-alpha/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -171,3 +171,19 @@ const pipe = new pipes.Pipe(this, 'Pipe', {
target: pipeTarget
});
```

### Amazon SageMaker Pipeline

A SageMaker pipeline can be used as a target for a pipe. The pipeline will receive the (enriched/filtered) source payload.

```ts
declare const sourceQueue: sqs.Queue;
declare const targetPipeline: sagemaker.IPipeline;

const pipelineTarget = new targets.SageMakerTarget(targetPipeline, {});

const pipe = new pipes.Pipe(this, 'Pipe', {
source: new SomeSource(sourceQueue),
target: pipelineTarget,
});
```
1 change: 1 addition & 0 deletions packages/@aws-cdk/aws-pipes-targets-alpha/lib/index.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
export * from './lambda';
export * from './sagemaker';
export * from './sqs';
export * from './stepfunctions';
68 changes: 68 additions & 0 deletions packages/@aws-cdk/aws-pipes-targets-alpha/lib/sagemaker.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
import { IInputTransformation, IPipe, ITarget, TargetConfig } from '@aws-cdk/aws-pipes-alpha';
import { IRole } from 'aws-cdk-lib/aws-iam';
import { IPipeline } from 'aws-cdk-lib/aws-sagemaker';

/**
* SageMaker target properties.
*/
export interface SageMakerTargetParameters {
/**
* The input transformation to apply to the message before sending it to the target.
*
* @see https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-pipes-pipe-pipetargetparameters.html#cfn-pipes-pipe-pipetargetparameters-inputtemplate
* @default none
*/
readonly inputTransformation?: IInputTransformation;

/**
* List of parameter names and values for SageMaker Model Building Pipeline execution.
msambol marked this conversation as resolved.
Show resolved Hide resolved
*
* The Name/Value pairs are passed to start execution of the pipeline.
*
* @see https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-pipes-pipe-pipetargetsagemakerpipelineparameters.html#cfn-pipes-pipe-pipetargetsagemakerpipelineparameters-pipelineparameterlist
* @default - none
*/
readonly pipelineParameterList?: Record<string, string>;
msambol marked this conversation as resolved.
Show resolved Hide resolved
}

/**
* A EventBridge Pipes target that sends messages to an SageMaker.
*/
export class SageMakerTarget implements ITarget {
private pipeline: IPipeline;
private sagemakerParameters?: SageMakerTargetParameters;
private pipelineParameterList?: Record<string, string>;
public readonly targetArn: string;

constructor(pipeline: IPipeline, parameters?: SageMakerTargetParameters) {
this.pipeline = pipeline;
this.targetArn = pipeline.pipelineArn;
this.sagemakerParameters = parameters;
this.pipelineParameterList = this.sagemakerParameters?.pipelineParameterList;
}

grantPush(grantee: IRole): void {
this.pipeline.grantStartPipelineExecution(grantee);
}

bind(pipe: IPipe): TargetConfig {
if (!this.sagemakerParameters) {
return {
targetParameters: {},
};
}

return {
targetParameters: {
inputTemplate: this.sagemakerParameters.inputTransformation?.bind(pipe).inputTemplate,
sageMakerPipelineParameters: {
pipelineParameterList: this.pipelineParameterList ?
Object.entries(this.pipelineParameterList).map(([key, value]) => ({
name: key,
value: value,
})) : undefined,
},
},
};
}
}
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
// Fixture with packages imported, but nothing else
import * as cdk from 'aws-cdk-lib';
import * as sagemaker from 'aws-cdk-lib/aws-sagemaker';
import * as sqs from 'aws-cdk-lib/aws-sqs';
import * as sfn from 'aws-cdk-lib/aws-stepfunctions';
import * as lambda from 'aws-cdk-lib/aws-lambda';
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
// Jest Snapshot v1, https://goo.gl/fbAQLP

exports[`SageMaker should grant pipe role push access 1`] = `
{
"MyPipeRoleCBC8E9AB": {
"Properties": {
"AssumeRolePolicyDocument": {
"Statement": [
{
"Action": "sts:AssumeRole",
"Effect": "Allow",
"Principal": {
"Service": "pipes.amazonaws.com",
},
},
],
"Version": "2012-10-17",
},
},
"Type": "AWS::IAM::Role",
},
}
`;

exports[`SageMaker should grant pipe role push access 2`] = `
{
"MyPipeRoleDefaultPolicy31387C20": {
"Properties": {
"PolicyDocument": {
"Statement": [
{
"Action": "sagemaker:StartPipelineExecution",
"Effect": "Allow",
"Resource": "MyPipeline",
},
],
"Version": "2012-10-17",
},
"PolicyName": "MyPipeRoleDefaultPolicy31387C20",
"Roles": [
{
"Ref": "MyPipeRoleCBC8E9AB",
},
],
},
"Type": "AWS::IAM::Policy",
},
}
`;

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading