Support S3 Express One Zone

[tustvold]

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

AWS recently announced support for S3 Express One Zone - https://aws.amazon.com/blogs/aws/new-amazon-s3-express-one-zone-high-performance-storage-class/

Describe the solution you'd like

Documentation is currently incomplete, but once sufficient information is available we should add support for this

Describe alternatives you've considered

Additional context

[Xuanwo]

Some experience:

• endpoint is a bit tricky that doesn’t follow the same pattern with other s3 bucket.
• Most API works the same except ListObjects must use / as delimiter.

I believe we don’t need to do any code changes. Only documentation need to add some content for the detailed behavior.

[changhiskhan]

Some experience:

• endpoint is a bit tricky that doesn’t follow the same pattern with other s3 bucket.
• Most API works the same except ListObjects must use / as delimiter.

I believe we don’t need to do any code changes. Only documentation need to add some content for the detailed behavior.

what is the convention for the endpoint?

[tustvold]

I thought it was just using virtual hosted style buckets, which we already support? We also don't support delimiters other than /. The major changes concern authentication.

[changhiskhan]

yeah, when i try things out as is i get some error coming from CreateSession

[Xuanwo]

I thought it was just using virtual hosted style buckets, which we already support?

Yep, we already support it.

The endpoint will be something like https://s3express-use1-az6.us-east-1.amazonaws.com.

We also don't support delimiters other than /.

It's more strict than that: the path to list must be endswith /. Otherwise, we will get error like:

{ code: "InvalidRequest", message: "This bucket does not support a prefix that does not end in a delimiter. Specify a prefix path ending with a delimiter and try again.", resource: "", request_id: "0033eada6b00018c5b6963200408d9d6e63702f8" }

[tustvold]

It's more strict than that: the path to list must be endswith /. Otherwise, we will get error like:

We always add / to the path prefix we provide, so as to ensure listing behaves in a similar manner to filesystems

[Xuanwo]

We always add / to the path prefix we provide, so as to ensure listing behaves in a similar manner to filesystems

Oh, seems I misunderstood something here. So object_store doesn't accept input like foo/bar? I used to think it's valid for list and list_with_delimiter to list keys like for/bar_xyz.

[tustvold]

used to think it's valid for list and list_with_delimiter to list keys like for/bar_xyz

No, list_with_delimiter should prepend a trailing / to the prefix provided and therefore bar would list bar/a but not bar_foo

[tustvold]

label_issue.py automatically added labels {'object-store'} from #5268