Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(scheduler-targets-alpha): KinesisStreamPutRecord Target #27845

Merged
merged 23 commits into from
Dec 4, 2023

Conversation

go-to-k
Copy link
Contributor

@go-to-k go-to-k commented Nov 5, 2023

This PR adds KinesisStreamPutRecord Target for EventBridge Scheduler.

Closes #27451.


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@aws-cdk-automation aws-cdk-automation requested a review from a team November 5, 2023 15:03
@github-actions github-actions bot added star-contributor [Pilot] contributed between 25-49 PRs to the CDK effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. p2 labels Nov 5, 2023
Copy link
Collaborator

@aws-cdk-automation aws-cdk-automation left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed add Clarification Request to a comment.

@go-to-k go-to-k marked this pull request as ready for review November 6, 2023 04:47
@go-to-k go-to-k marked this pull request as draft November 6, 2023 04:49
@aws-cdk-automation aws-cdk-automation dismissed their stale review November 6, 2023 04:49

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

@go-to-k go-to-k marked this pull request as ready for review November 6, 2023 04:51
@aws-cdk-automation aws-cdk-automation added the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Nov 6, 2023
Copy link
Contributor

@lpizzinidev lpizzinidev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall, thanks 👍
The unit tests on imported streams will require a fix on the importing function.

/**
* The shard to which EventBridge Scheduler sends the event.
*
* A length of `partitionKey` must be between 1 and 256.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* A length of `partitionKey` must be between 1 and 256.
* The length must be between 1 and 256.

Comment on lines 295 to 333
test('throws when stream is imported from different account', () => {
const stack2 = new Stack(app, 'Stack2', {
env: {
region: 'us-east-1',
account: '234567890123',
},
});

const importedStream = kinesis.Stream.fromStreamArn(stack2, 'ImportedStream', 'arn:aws:kinesis:us-east-1:234567890123:stream/Foo');
const streamTarget = new KinesisStreamPutRecord(importedStream, {
partitionKey: 'key',
});

expect(() =>
new Schedule(stack, 'MyScheduleDummy', {
schedule: expr,
target: streamTarget,
})).toThrow(/Both the schedule and the stream must be in the same account/);
});

test('throws when stream is imported from different region', () => {
const stack2 = new Stack(app, 'Stack2', {
env: {
region: 'us-west-2',
account: '123456789012',
},
});

const importedStream = kinesis.Stream.fromStreamArn(stack2, 'ImportedStream', 'arn:aws:kinesis:us-west-2:123456789012:stream/Foo');
const streamTarget = new KinesisStreamPutRecord(importedStream, {
partitionKey: 'key',
});

expect(() =>
new Schedule(stack, 'MyScheduleDummy', {
schedule: expr,
target: streamTarget,
})).toThrow(/Both the schedule and the stream must be in the same region/);
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
test('throws when stream is imported from different account', () => {
const stack2 = new Stack(app, 'Stack2', {
env: {
region: 'us-east-1',
account: '234567890123',
},
});
const importedStream = kinesis.Stream.fromStreamArn(stack2, 'ImportedStream', 'arn:aws:kinesis:us-east-1:234567890123:stream/Foo');
const streamTarget = new KinesisStreamPutRecord(importedStream, {
partitionKey: 'key',
});
expect(() =>
new Schedule(stack, 'MyScheduleDummy', {
schedule: expr,
target: streamTarget,
})).toThrow(/Both the schedule and the stream must be in the same account/);
});
test('throws when stream is imported from different region', () => {
const stack2 = new Stack(app, 'Stack2', {
env: {
region: 'us-west-2',
account: '123456789012',
},
});
const importedStream = kinesis.Stream.fromStreamArn(stack2, 'ImportedStream', 'arn:aws:kinesis:us-west-2:123456789012:stream/Foo');
const streamTarget = new KinesisStreamPutRecord(importedStream, {
partitionKey: 'key',
});
expect(() =>
new Schedule(stack, 'MyScheduleDummy', {
schedule: expr,
target: streamTarget,
})).toThrow(/Both the schedule and the stream must be in the same region/);
});
test.each([
['account', 'arn:aws: kinesis:us-east-1:999999999999:stream/Foo', /Both the schedule and the stream must be in the same account./],
['region', 'arn:aws: kinesis:eu-central-1:123456789012:stream/Foo', /Both the schedule and the stream must be in the same region./],
])('throws when Kinesis Data Stream is imported from different %s', (_, arn: string, expectedError: RegExp) => {
const importedStream = kinesis.Stream.fromStreamArn(stack, 'ImportedStream', arn);
const streamTarget = new KinesisStreamPutRecord(importedStream, {
partitionKey: 'key',
});
expect(() =>
new Schedule(stack, 'MyScheduleDummy', {
schedule: expr,
target: target,
})).toThrow(expectedError);
});

fromStreamArn should use the same stack.
fromStreamAttributes will require to solve the environment for the imported stream ARN:

  public static fromStreamAttributes(scope: Construct, id: string, attrs: StreamAttributes): IStream {
    class Import extends StreamBase {
      public readonly streamArn = attrs.streamArn;
      public readonly streamName = Stack.of(scope).splitArn(attrs.streamArn, ArnFormat.SLASH_RESOURCE_NAME).resourceName!;
      public readonly encryptionKey = attrs.encryptionKey;
    }

    return new Import(scope, id, {
      environmentFromArn: attrs.streamArn,
    });
  }

Also, can you please add unit tests to verify that the environment gets resolved correctly when using fromStreamArn and fromStreamAttributes after this change?

@aws-cdk-automation aws-cdk-automation removed the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Nov 9, 2023
@go-to-k
Copy link
Contributor Author

go-to-k commented Nov 9, 2023

@lpizzinidev

I just changed! Please check them.

Copy link
Contributor

@lpizzinidev lpizzinidev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks 👍

@aws-cdk-automation aws-cdk-automation added the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Nov 9, 2023
@aws-cdk-automation
Copy link
Collaborator

This PR cannot be merged because it has conflicts. Please resolve them. The PR will be considered stale and closed if it remains in an unmergeable state.

@vinayak-kukreja vinayak-kukreja self-assigned this Nov 30, 2023
@vinayak-kukreja vinayak-kukreja changed the title feat(scheduler-targets): KinesisStreamPutRecord Target feat(scheduler-targets-alpha): KinesisStreamPutRecord Target Nov 30, 2023
Signed-off-by: Vinayak Kukreja <vinakuk@amazon.com>
Copy link
Contributor

@vinayak-kukreja vinayak-kukreja left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @go-to-k , thank you for this contribution.

Copy link
Contributor

mergify bot commented Nov 30, 2023

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation aws-cdk-automation removed the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Nov 30, 2023
Copy link
Contributor

mergify bot commented Nov 30, 2023

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@go-to-k
Copy link
Contributor Author

go-to-k commented Dec 1, 2023

@vinayak-kukreja

Thanks for your approval! I fixed conflicts. Please approve again.

I fixed the other PR the same way, but when one of them gets merged, the other one gets conflicts again. If that happens, I will fix it again.

@aws-cdk-automation aws-cdk-automation added the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Dec 1, 2023
@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: d47f958
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Copy link
Contributor

mergify bot commented Dec 4, 2023

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@mergify mergify bot merged commit 47a09b5 into aws:main Dec 4, 2023
10 checks passed
chenjane-dev pushed a commit to chenjane-dev/aws-cdk that referenced this pull request Dec 5, 2023
…7845)

This PR adds KinesisStreamPutRecord Target for EventBridge Scheduler.

Closes aws#27451.

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. p2 pr/needs-maintainer-review This PR needs a review from a Core Team Member star-contributor [Pilot] contributed between 25-49 PRs to the CDK
Projects
None yet
Development

Successfully merging this pull request may close these issues.

(aws-scheduler-targets-alpha): Add KinesisStreamPutRecord Target
4 participants