Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: upload: --existing=overwrite-metadata to upload fresh metadata even if file already there #355

Closed
wants to merge 4 commits into from

Conversation

yarikoptic
Copy link
Member

@yarikoptic yarikoptic commented Feb 1, 2021

To facilitate cases where we want to ask users to reupload while using newer dandi-cli which would extract metadata differently

Closes #105

TODOs:

  • Development safeguard (DANDI_UPLOAD_SAME_MUST_EXIST env var) for paranoids so if we operate under assumption that we have up to date local data files, we do not mistakenly reupload (outdated) data.
  • test on sample cases to verify correct operation (see e.g. incomplete info for a file #349 as one of the target use cases)

@codecov
Copy link

codecov bot commented Feb 1, 2021

Codecov Report

Merging #355 (91b2af7) into master (53da9e9) will decrease coverage by 0.34%.
The diff coverage is 42.85%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #355      +/-   ##
==========================================
- Coverage   82.15%   81.81%   -0.35%     
==========================================
  Files          55       55              
  Lines        5454     5482      +28     
==========================================
+ Hits         4481     4485       +4     
- Misses        973      997      +24     
Flag Coverage Δ
unittests 81.81% <42.85%> (-0.35%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
dandi/cli/cmd_upload.py 88.88% <ø> (ø)
dandi/upload.py 66.66% <42.85%> (-3.19%) ⬇️
dandi/dandiapi.py 85.22% <0.00%> (-0.99%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 53da9e9...91b2af7. Read the comment docs.

@yarikoptic yarikoptic marked this pull request as ready for review February 1, 2021 18:58
…ven if file already there

To facilitate cases where we want to ask users to reupload while using newer
dandi-cli which would extract metadata differently
I decided to let directory be created if needed... minor
@yarikoptic
Copy link
Member Author

"tested" in real life, managed to delete file with some version before this one... so I better wait to have done more testing or if I add some unittest to guarantee that there is no side effects... and we might not need it in the long run with a new API server since if data blob is the same - we should not reupload it.

@yarikoptic
Copy link
Member Author

I will close it for now ... can resurrect if the need arises again and would need to do it also for dandi-api interface

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

upload: only metadata
2 participants