Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't read S3-stored datafile without the dataverse.files.s3-bucket-name JVM option present #5065

Closed
landreev opened this issue Sep 17, 2018 · 4 comments

Comments

@landreev
Copy link
Contributor

The JVM option dataverse.files.s3-bucket-name is required when using S3 and storing new files; it tells the S3 driver what "bucket" to put the file into.
If you switch back to the filesystem, or swift driver, that's where any new files will be stored. But we're still supposed to be able to read any old S3 files.
It appears that this is no longer working if the JVM option above is no longer present. Which seems unnecessary, because the bucket name is actually embedded in the storageidentifier that's saved in the database.
For example, "s3://iqssqa:165a6268a5e-fb97b82e022d" - "iqssqa" is the bucket where the file lives.

(This is probably one of the least important bugs/issues we have; it's just weird, and the failures are somewhat confusing...)

@poikilotherm
Copy link
Contributor

As I am currently looking into the S3 stuff anyway (see #4690), maybe I can try to fix this, too.

@pdurbin
Copy link
Member

pdurbin commented Sep 19, 2018

@poikilotherm sure, while you're in there you could at least try to reproduce the bug and report any findings here. Thanks!

@qqmyers
Copy link
Member

qqmyers commented Oct 6, 2018

FWIW - Looks like S3IOAccess just reads the jvm option and fails if it is not set. If it is set, for file reads, the code does use the bucketname written in the file storage identifier.

@pdurbin
Copy link
Member

pdurbin commented Dec 1, 2023

I just checked with @landreev and he said it's safe to close this.

@pdurbin pdurbin closed this as completed Dec 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants