Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for newer dot region AWS S3 virtual host style paths #263

Closed
picatz opened this issue Jul 19, 2020 · 4 comments · Fixed by #283
Closed

Add support for newer dot region AWS S3 virtual host style paths #263

picatz opened this issue Jul 19, 2020 · 4 comments · Fixed by #283
Labels

Comments

@picatz
Copy link
Contributor

picatz commented Jul 19, 2020

Following up on hashicorp/nomad#8454, the current tests don't include the newer virtual hosted style syntax, and doesn't seem to be supported. This is causing valid S3 paths to fail for Nomad's artifact stanza, and is a pain to debug / manually spot.

The S3 detector implementation should be adjusted to support the newer virtual host style paths using the dot region syntax.

Example

Newer virtual host style with S3 dot region endpoint structure:
https://legacy-player-resources.s3.eu-west-2.amazonaws.com/...
                                                               ☝️

Older virtual host style with S3 dash region endpoint structure:
https://legacy-player-resources.s3-eu-west-2.amazonaws.com/...
                                                               ☝️

The older dash-style that is supported is actually discouraged in the S3 documentation:

We recommend that you do not use this endpoint structure in your requests.

Phasing out S3 Path Style

To add to the complications, the other "path style" that is also supported will be soon phased out:

Support for the path-style model continues for buckets created on or before September 30, 2020. Buckets created after that date must be referenced using the virtual-hosted model.

@picatz picatz added the bug label Jul 19, 2020
@scalp42
Copy link

scalp42 commented Oct 7, 2020

@picatz any timeline on this? I believe it's now broken with new buckets

@picatz
Copy link
Contributor Author

picatz commented Nov 12, 2020

@scalp42 sorry I don't have a timeline for this, yet!

Need to review/merge #283, cut a release, and get the updates in the required projects (like hashicorp/nomad#9050)

@picatz
Copy link
Contributor Author

picatz commented Nov 12, 2020

Fix released in v1.5.1

@scalp42
Copy link

scalp42 commented Nov 13, 2020

awesome! thanks @picatz

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants