🐛 Attempt to fix 400 RequestTimeout when uploading to S3 ⚠️

[GitHK]

What do these changes do?

This is an enhancement based on #4483

Attempts to fix a long running issue when uploading files to S3, sometimes the AWS S3 backend will reply with 400 ReuestTimeout.

When receiving such a reply the entire multipart upload is stopped and another one will be started once more. New multipart upload links will be requested.

The retry is only applied to the above mentioned case. A specific error was created and isolated to better be propagated and handled.
To disable this feature set DIRECTOR_V2_NODE_PORTS_400_REQUEST_TIMEOUT_ATTEMPTS to 0. No retry will happen and the error will just bubble up as it previously did, it will now show as AWSS3400RequestTimeOutError.

Devops ⚠️

No actions are required from devops. Be aware about the following:

• added new env var DIRECTOR_V2_NODE_PORTS_400_REQUEST_TIMEOUT_ATTEMPTS to the director-v2 which will be sent to the dynamic-sidecar. Inside it, nodeports captures the env var and uses it for the retry.

Related issue/s

• S3TransferError 400 BadRequest with reason RequestTimeout #3531
• closes ♻️⚗️Add (another) retry for multipart file upload #4483

How to test

DevOps Checklist

[codecov]

Codecov Report

Merging #4996 (4fcb191) into master (910c1a1) will decrease coverage by 0.1%.
The diff coverage is 97.8%.

@@            Coverage Diff            @@
##           master   #4996      +/-   ##
=========================================
- Coverage    87.1%   87.0%    -0.1%     
=========================================
  Files        1243     907     -336     
  Lines       51132   40099   -11033     
  Branches     1081     198     -883     
=========================================
- Hits        44549   34921    -9628     
+ Misses       6345    5129    -1216     
+ Partials      238      49     -189     

Flag	Coverage Δ
integrationtests	63.6% <91.4%> (-0.1%)	⬇️
unittests	84.4% <72.3%> (-0.6%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
...dk/src/simcore_sdk/node_ports_common/exceptions.py	69.8% <100.0%> (+1.2%)	⬆️
...k/src/simcore_sdk/node_ports_common/filemanager.py	84.6% <100.0%> (+1.1%)	⬆️
...-sdk/src/simcore_sdk/node_ports_common/settings.py	100.0% <100.0%> (ø)
...2/src/simcore_service_director_v2/core/settings.py	98.8% <100.0%> (+<0.1%)	⬆️
...es/dynamic_sidecar/docker_service_specs/sidecar.py	88.1% <ø> (ø)
...src/simcore_sdk/node_ports_common/file_io_utils.py	88.4% <96.2%> (-0.2%)	⬇️

... and 341 files with indirect coverage changes

[sanderegg]

Please check how you raise your exception.. there is dead code

[matusdrobuliak66]

👍

[sanderegg]

thanks! just think of updating the description of your PR

[codeclimate]

Code Climate has analyzed commit 4fcb191 and detected 0 issues on this pull request.

View more on Code Climate.

[sonarqubecloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.1% Duplication