Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(config,cache/s3proxy): add s3 addressing style support #662

Conversation

wuhuizuo
Copy link
Contributor

@wuhuizuo wuhuizuo commented May 24, 2023

Some S3 providers do not support auto bucket lookup type, need to set it with path type or dns type.

Here I added a support to set it in s3_proxy filed in configuration to make it reachable.

@wuhuizuo wuhuizuo force-pushed the feature/s3-proxy-addressing-style-support branch from 6202029 to 8c78612 Compare May 24, 2023 12:25
@wuhuizuo wuhuizuo closed this May 25, 2023
@wuhuizuo wuhuizuo reopened this May 25, 2023
@wuhuizuo
Copy link
Contributor Author

/reopen to rerun the golang ci lint job

Copy link
Collaborator

@mostynb mostynb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution. Sorry it took me a few days to look at this.

cache/s3proxy/s3proxy.go Show resolved Hide resolved
@wuhuizuo
Copy link
Contributor Author

wuhuizuo commented Jun 3, 2023

Thanks for the contribution. Sorry it took me a few days to look at this.

I fixed it.

Copy link
Collaborator

@mostynb mostynb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feature looks good to me, but I think we should use string flags instead, for improved readability.

I had to do some background reading to understand the use case. This blog post was useful, sharing it here for anyone reading through this PR later: https://aws.amazon.com/blogs/aws/amazon-s3-path-deprecation-plan-the-rest-of-the-story/

utils/flags/flags.go Outdated Show resolved Hide resolved
@wuhuizuo wuhuizuo force-pushed the feature/s3-proxy-addressing-style-support branch from c743668 to 5fbf4c4 Compare June 18, 2023 07:42
@wuhuizuo wuhuizuo force-pushed the feature/s3-proxy-addressing-style-support branch from 5fbf4c4 to 67dbda2 Compare June 18, 2023 07:45
@wuhuizuo wuhuizuo requested a review from mostynb June 18, 2023 07:45
config/proxy.go Outdated
Comment on lines 91 to 92
// also when not found the type, return "auto" type.
return valMap[typeStr]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this function should probably check if typeStr is in the map, and return an error if it isn't. And this error checking should be done in validateConfig() in config/config.go

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we only need to check it in validateConfig().

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also I refactored it again.

utils/flags/flags.go Outdated Show resolved Hide resolved
README.md Show resolved Hide resolved
@wuhuizuo wuhuizuo force-pushed the feature/s3-proxy-addressing-style-support branch from 49bc4de to dff949c Compare June 20, 2023 10:59
@mostynb
Copy link
Collaborator

mostynb commented Jun 20, 2023

Thanks for following through with these changes- I made a few minor tweaks and landed this on the master branch.

@mostynb mostynb closed this Jun 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants