Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for s3 without tagging (e.g., s3 api on top of swift) #10022

Closed
ErykKul opened this issue Oct 18, 2023 · 2 comments · Fixed by #10029
Closed

Support for s3 without tagging (e.g., s3 api on top of swift) #10022

ErykKul opened this issue Oct 18, 2023 · 2 comments · Fixed by #10029
Labels
Feature: File Upload & Handling Type: Feature a feature request User Role: Sysadmin Installs, upgrades, and configures the system, connects via ssh
Milestone

Comments

@ErykKul
Copy link
Collaborator

ErykKul commented Oct 18, 2023

What steps does it take to reproduce the issue?
When using an s3 implementation that does not support tagging, e.g., s3 api of the openstack swift, you cannot use the upload-redirect feature. The uploads are prevented with "501: Not implemented" error.

When does this issue occur?
An error is thrown when uploading a file with upload-redirect feature turned on. This happens only on a Dataverse instance using an s3 implementation that does not support tagging. I have tested it with s3 on top of the openstack swift.

Which page(s) does it occurs on?
File upload

What happens?
"501: Not implemented" error.

To whom does it occur (all users, curators, superusers)?
All users

What did you expect to happen?
I expect the upload to succeed.

Which version of Dataverse are you using?
Latest.

Any related open or closed issues to this bug report?
#9130
Since the solution to this problem is removing the tag header (I am working on a pull request for this issue that would add an option to disable tagging for uploads), a cleanup mechanism is needed to removed partially uploaded files, etc., since the temp tag would not be there anymore. That mechanism is already implemented in #9132 as a solution to the linked issue.

Screenshots:

image

@ErykKul
Copy link
Collaborator Author

ErykKul commented Oct 18, 2023

I am working on a pull request for this.

@pdurbin
Copy link
Member

pdurbin commented Oct 18, 2023

@ErykKul hmm, I wonder if we can join forces. I'm currently working on S3 tests here:

@poikilotherm and I are talking about that issue at https://dataverse.zulipchat.com/#narrow/stream/375812-containers/topic/testing.20S3/near/396903562 and you're welcome to join!

Right now I'm testing with LocalStack, by the way.

@pdurbin pdurbin added Type: Feature a feature request Feature: File Upload & Handling User Role: Sysadmin Installs, upgrades, and configures the system, connects via ssh labels Nov 12, 2023
@pdurbin pdurbin assigned pdurbin and unassigned pdurbin Apr 16, 2024
pdurbin added a commit to ErykKul/dataverse that referenced this issue Apr 16, 2024
…S#10022

Conflicts:
src/main/java/edu/harvard/iq/dataverse/settings/JvmSettings.java
pdurbin added a commit to ErykKul/dataverse that referenced this issue Apr 16, 2024
pdurbin added a commit to ErykKul/dataverse that referenced this issue Apr 17, 2024
@pdurbin pdurbin added this to the 6.3 milestone Apr 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature: File Upload & Handling Type: Feature a feature request User Role: Sysadmin Installs, upgrades, and configures the system, connects via ssh
Projects
Status: High priority
Development

Successfully merging a pull request may close this issue.

2 participants