Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(pipes-targets): add API destination #30756

Merged
merged 21 commits into from
Sep 27, 2024
Merged

Conversation

msambol
Copy link
Contributor

@msambol msambol commented Jul 5, 2024

Add EventBridge API destination as a Pipes target.

CloudFormation groups EventBridge API destination with API Gateway REST API
as PipeTargetHttpParameters, but I think separating them here similar to aws-event-targets
makes more sense, as API Gateway requires stage, path, and method (see here).

@aws-cdk-automation aws-cdk-automation requested a review from a team July 5, 2024 00:03
@github-actions github-actions bot added distinguished-contributor [Pilot] contributed 50+ PRs to the CDK p2 labels Jul 5, 2024
@msambol msambol changed the title feat(pipes-targets): add Http endpoints feat(pipes-targets): add http endpoints Jul 5, 2024
@msambol msambol changed the title feat(pipes-targets): add http endpoints feat(pipes-targets): add API destination Jul 5, 2024
@aws-cdk-automation aws-cdk-automation added the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Jul 5, 2024
Copy link
Contributor

@nmussy nmussy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly integ coverage, LGTM overall

});

const connection = new cdk.aws_events.Connection(stack, 'MyConnection', {
authorization: cdk.aws_events.Authorization.apiKey('x-api-key', secret.secretValue),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm also curious to see what happens when you also set the same header in ApiDestination.headerParameters. It's either ignored or overrides the authorization, but I think we should probably warn the user either way

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comment below with screenshot.

@aws-cdk-automation aws-cdk-automation removed the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Jul 6, 2024
@msambol
Copy link
Contributor Author

msambol commented Jul 6, 2024

@nmussy Here's the result from the Lambda logs. The x-api-key defined in Connection has precedence over the one defined in headerParameters.

Screenshot 2024-07-06 at 10 35 18 AM

* The headers to send as part of the request invoking the EventBridge API destination.
*
* The headers are merged with the headers from the API destination.
* If there are conflicts, the headers from the API destination take precedence.
Copy link
Contributor Author

@msambol msambol Jul 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nmussy I added this note about precedence. From the logs we can see that x-api-key is abc123, not apiKeyFromHeaderParams.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure whether this is enough to indicate to users which value will take precedence. Is there somewhere we can print a warning when there are two identical headers provided, which explicitly lets the user know that the value is being set twice, and which of those values it's using?

Copy link
Contributor Author

@msambol msambol Sep 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@redwheeler3 jfyi. I think it's worth mentioning in Docs and/or Console about precedence regarding conflicting headers.

@pahud pahud added the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Jul 22, 2024
@aws-cdk-automation aws-cdk-automation removed the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Jul 22, 2024
@Leo10Gama Leo10Gama self-assigned this Sep 19, 2024
Copy link
Member

@Leo10Gama Leo10Gama left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution, and sorry for the delay! Overall seems good to me, just a few things to note before moving forward with these changes.

* The headers to send as part of the request invoking the EventBridge API destination.
*
* The headers are merged with the headers from the API destination.
* If there are conflicts, the headers from the API destination take precedence.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure whether this is enough to indicate to users which value will take precedence. Is there somewhere we can print a warning when there are two identical headers provided, which explicitly lets the user know that the value is being set twice, and which of those values it's using?

@msambol
Copy link
Contributor Author

msambol commented Sep 20, 2024

I'm not sure whether this is enough to indicate to users which value will take precedence. Is there somewhere we can print a warning when there are two identical headers provided, which explicitly lets the user know that the value is being set twice, and which of those values it's using?

I'm not quite sure how to do this. There could be any number of custom headers, no?

@Leo10Gama
Copy link
Member

I'm not sure whether this is enough to indicate to users which value will take precedence. Is there somewhere we can print a warning when there are two identical headers provided, which explicitly lets the user know that the value is being set twice, and which of those values it's using?

I'm not quite sure how to do this. There could be any number of custom headers, no?

I believe that's the case, yeah. I was thinking some check like iterating through the header keys to see which overlap, like turning them into a set and taking the intersection type of operation. Then if we know the Connection key has precedence, like your logs show, we print a warning along the lines of:

"Key <so-and-so> is defined in both Connection [connection] and ApiDestinationTarget [api destination target]. Value <my value> from Connection will be used."

@mergify mergify bot dismissed Leo10Gama’s stale review September 20, 2024 21:51

Pull request has been modified.

@msambol
Copy link
Contributor Author

msambol commented Sep 20, 2024

I'm not sure whether this is enough to indicate to users which value will take precedence. Is there somewhere we can print a warning when there are two identical headers provided, which explicitly lets the user know that the value is being set twice, and which of those values it's using?

I'm not quite sure how to do this. There could be any number of custom headers, no?

I believe that's the case, yeah. I was thinking some check like iterating through the header keys to see which overlap, like turning them into a set and taking the intersection type of operation. Then if we know the Connection key has precedence, like your logs show, we print a warning along the lines of:

"Key <so-and-so> is defined in both Connection [connection] and ApiDestinationTarget [api destination target]. Value <my value> from Connection will be used."

I'd prefer to do this in a follow-up PR. What do you think? I think the logic could get messy/complicated. I think the documentation note is sufficient for now.

@Leo10Gama
Copy link
Member

I'd prefer to do this in a follow-up PR. What do you think? I think the logic could get messy/complicated. I think the documentation note is sufficient for now.

I do think we should include these changes in this PR, unless there are any other extra changes you wanted to add that might make the logic more confusing to follow. But ideally, including them in this PR means the logic is bundled together from the get-go.

Copy link
Member

@Leo10Gama Leo10Gama left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the changes! Aside from my comment above regarding including the warning, just one more minor nit in the README.

packages/@aws-cdk/aws-pipes-targets-alpha/README.md Outdated Show resolved Hide resolved
@msambol
Copy link
Contributor Author

msambol commented Sep 23, 2024

I'd prefer to do this in a follow-up PR. What do you think? I think the logic could get messy/complicated. I think the documentation note is sufficient for now.

I do think we should include these changes in this PR, unless there are any other extra changes you wanted to add that might make the logic more confusing to follow. But ideally, including them in this PR means the logic is bundled together from the get-go.

I'm a little confused on what this would look like. An ApiDestination doesn't have headers associated with it. In my integration test example, the Connection has an API key (x-api-key). Happy to add the warning but I need some direction as there is nothing to iterate through on ApiDestination ?

@mergify mergify bot dismissed Leo10Gama’s stale review September 23, 2024 22:52

Pull request has been modified.

@Leo10Gama
Copy link
Member

I'd prefer to do this in a follow-up PR. What do you think? I think the logic could get messy/complicated. I think the documentation note is sufficient for now.

I do think we should include these changes in this PR, unless there are any other extra changes you wanted to add that might make the logic more confusing to follow. But ideally, including them in this PR means the logic is bundled together from the get-go.

I'm a little confused on what this would look like. An ApiDestination doesn't have headers associated with it. In my integration test example, the Connection has an API key (x-api-key). Happy to add the warning but I need some direction as there is nothing to iterate through on ApiDestination ?

If I'm understanding the code correctly, the area of conflict is between ApiDestination.headerParameters and the API key, so we'd iterate on the header parameters. We should be able to compare the headers since the target has access to the Connection via ApiDestinationTarget.destination.connection.

@msambol
Copy link
Contributor Author

msambol commented Sep 25, 2024

I'd prefer to do this in a follow-up PR. What do you think? I think the logic could get messy/complicated. I think the documentation note is sufficient for now.

I do think we should include these changes in this PR, unless there are any other extra changes you wanted to add that might make the logic more confusing to follow. But ideally, including them in this PR means the logic is bundled together from the get-go.

I'm a little confused on what this would look like. An ApiDestination doesn't have headers associated with it. In my integration test example, the Connection has an API key (x-api-key). Happy to add the warning but I need some direction as there is nothing to iterate through on ApiDestination ?

If I'm understanding the code correctly, the area of conflict is between ApiDestination.headerParameters and the API key, so we'd iterate on the header parameters. We should be able to compare the headers since the target has access to the Connection via ApiDestinationTarget.destination.connection.

I think I would need to change

private destination: IApiDestination;

to

private destination: ApiDestination;

Excuse my naïveté. Good with that?

@msambol
Copy link
Contributor Author

msambol commented Sep 27, 2024

@moelasmar updated with your feedback—thanks!

Copy link
Member

@Leo10Gama Leo10Gama left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems all good to me, thanks for the quick responses on feedback! Just adding in one quick typehint that I missed before, and will wait to make sure the tests pass.

@msambol
Copy link
Contributor Author

msambol commented Sep 27, 2024

@mergify update

Copy link
Contributor

mergify bot commented Sep 27, 2024

update

✅ Branch has been successfully updated

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: b0b1827
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Copy link
Contributor

mergify bot commented Sep 27, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@mergify mergify bot merged commit 5e08c98 into aws:main Sep 27, 2024
9 checks passed
@msambol msambol deleted the pipes-targets-http branch September 27, 2024 23:47
Copy link

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 27, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
distinguished-contributor [Pilot] contributed 50+ PRs to the CDK p2
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants