Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add S3 region configuration to security mapping #18838

Merged
merged 2 commits into from
Feb 28, 2024

Conversation

kekwan
Copy link
Contributor

@kekwan kekwan commented Aug 28, 2023

Description

  • Add ability to configure S3 regions dynamically per-bucket via security mapping feature

Additional context and related issues

  • add S3 endpoint configuration to security mapping #3869 provides undocumented capability to dynamically update s3 endpoints but this is not sufficient for use-cases that have cross-region S3 access. For use-cases like cross-region VPC endpoints, you need to set the corresponding region also.

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(X) Release notes are required, with the following suggested text:

# Section
* Add S3 region configuration to security mapping

@cla-bot cla-bot bot added the cla-signed label Aug 28, 2023
@kekwan kekwan marked this pull request as ready for review August 28, 2023 22:19
@github-actions github-actions bot added the docs label Aug 28, 2023
@kekwan kekwan force-pushed the s3-region-per-bucket-oss branch from 8700b33 to 4e309d8 Compare August 28, 2023 23:23
Copy link
Member

@hashhar hashhar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

using the security mapping feature to change endpoint and region is a misuse IMO.

The correct solution is to use different catalogs IMO.

@electrum your opinion?

@kekwan
Copy link
Contributor Author

kekwan commented Aug 29, 2023

using the security mapping feature to change endpoint and region is a misuse IMO.

The correct solution is to use different catalogs IMO.

@electrum your opinion?

I do agree that would be ideal. But I think there are valid use-cases for this feature.

For example, our user has tables with data from many different buckets each needing their own VPC endpoints. Using a single catalog gives us a unified querying experience and doesn't require us enforcing users to make sure they're using the correct catalog.

@electrum
Copy link
Member

electrum commented Sep 2, 2023

I think this is reasonable. The feature is really “flexible S3 configuration” and not just security.

@kekwan kekwan requested a review from hashhar September 5, 2023 17:14
@kekwan kekwan requested a review from electrum September 12, 2023 18:35
Copy link

This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua

@github-actions github-actions bot added the stale label Jan 11, 2024
@mosabua
Copy link
Member

mosabua commented Jan 11, 2024

👋 @kekwan could you rebase this PR so we can continue review and get to merge potentially.

@github-actions github-actions bot removed the stale label Jan 12, 2024
Copy link

github-actions bot commented Feb 2, 2024

This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua

@github-actions github-actions bot added the stale label Feb 2, 2024
Copy link

Closing this pull request, as it has been stale for six weeks. Feel free to re-open at any time.

@github-actions github-actions bot closed this Feb 28, 2024
@electrum electrum reopened this Feb 28, 2024
@electrum electrum merged commit ea8bdb2 into trinodb:master Feb 28, 2024
61 of 62 checks passed
@github-actions github-actions bot added this to the 440 milestone Feb 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

4 participants