♻️🐛 storage provides presigned download links for files in folders

[GitHK]

What do these changes do?

Pre signed download links would not be generated for files placed inside a file_meta_data entry which is a folder.

• 🐛 pre signed links are now generated for all existing objects in S3 if they are placed inside a directory.
• ♻️ checking the existence of files in S3 is done via the Object HEAD request, since it is more reliable, less impactful on S3 and less likely to break when changing the implementation of _list_objects_v2_paginated.

Related issue/s

• required by ✨ Link assets folder from service to front-end #4572
• Project home page osparc-issues#979

How to test

DevOps Checklist

[codecov]

Codecov Report

Merging #4573 (eff1787) into master (82edbc1) will decrease coverage by 9.7%.
The diff coverage is 97.8%.

@@            Coverage Diff            @@
##           master   #4573      +/-   ##
=========================================
- Coverage    86.7%   77.0%    -9.7%     
=========================================
  Files        1023     663     -360     
  Lines       43790   30815   -12975     
  Branches     1013     533     -480     
=========================================
- Hits        37991   23755   -14236     
- Misses       5572    6936    +1364     
+ Partials      227     124     -103     

Flag	Coverage Δ
integrationtests	66.6% <ø> (ø)
unittests	75.3% <97.8%> (-9.0%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed	Coverage Δ
...vices/storage/src/simcore_service_storage/utils.py	100.0% <ø> (ø)
...rage/src/simcore_service_storage/simcore_s3_dsm.py	94.7% <95.0%> (+0.1%)	⬆️
...s/storage/src/simcore_service_storage/s3_client.py	98.7% <100.0%> (+<0.1%)	⬆️
...rc/simcore_service_storage/simcore_s3_dsm_utils.py	100.0% <100.0%> (ø)

... and 533 files with indirect coverage changes

[sanderegg]

Thanks for the changes!
might be worth that we look at it together on Monday?

[sanderegg]

Ok I don't want to hinder the progress on this any further. but I think you should refactor a bit the test so that it is maintainable (you are repeating a test that already exists, and I would suggest to separate the use cases instead of putting them altogether, your call).

[codeclimate]

Code Climate has analyzed commit eff1787 and detected 0 issues on this pull request.

View more on Code Climate.

[sonarqubecloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.1% Duplication