-
Notifications
You must be signed in to change notification settings - Fork 78
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make region
optional for S3 instructions
#5603
Conversation
extensions/s3/src/main/java/io/deephaven/extensions/s3/S3Instructions.java
Outdated
Show resolved
Hide resolved
extensions/s3/src/main/java/io/deephaven/extensions/s3/S3SeekableChannelProviderPlugin.java
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My comments are the same as Larry's.
extensions/s3/src/main/java/io/deephaven/extensions/s3/S3Instructions.java
Outdated
Show resolved
Hide resolved
extensions/s3/src/main/java/io/deephaven/extensions/s3/S3AsyncClientFactory.java
Outdated
Show resolved
Hide resolved
extensions/s3/src/main/java/io/deephaven/extensions/s3/S3SeekableChannelProviderPlugin.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Caught up
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Python changes LGTM
Labels indicate documentation is required. Issues for documentation have been opened: Community: deephaven/deephaven-docs-community#234 |
Before this change,
region
was a mandatory parameter for reading parquet files from S3.After this change, if
region
is not provided, it will be picked up by the AWS sdk using the following provider chain:Reference: https://sdk.amazonaws.com/java/api/2.0.0-preview-11/software/amazon/awssdk/awscore/client/builder/AwsClientBuilder.html
Also, since
region
was the only mandatory parameter in S3 instructions, after this change, reading from S3 will be allowed without passing any S3 instructions. For example, code like the following python example will work, as long as the region, access key and secret key are set properly using properties or config files: