✨Autoscaling: use label instead of draining machine

[sanderegg]

What do these changes do?

It seems that the performance of starting a dynamic-sidecar on an autoscaled node is reduced when the docker node is first transitioning from drain to active state.
This PR changes how the autoscaling service treats "drained" nodes.

Before:

• drained nodes would be set to the drained state as defined in docker terminology (docker node in state drained)

After:

• drained nodes are maintained as active in the docker terminology but a docker node label is added
• the docker node label is named osparc-services-ready
• when the node is "drained", osparc-services-ready=false and when the node is "active" it is set as osparc-services-ready=true
• an ENV variable allows to control the behavior on the autoscaling service, AUTOSCALING_LABELIZE_DRAINED_NODES it is set currently by default to False so we can test the results in AWS-master. Therefore changes in osparc-config might come later if this proves fruitful.

Related issue/s

• fixes Autoscaling: use labels instead of draining nodes #5339

How to test

Dev Checklist

• No ENV changes or I properly updated ENV (read the instruction)

DevOps Checklist

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (734e40e) 87.3% compared to head (60941f1) 87.3%.

Additional details and impacted files

@@          Coverage Diff           @@
##           master   #5340   +/-   ##
======================================
  Coverage    87.3%   87.3%           
======================================
  Files        1321    1321           
  Lines       54146   54167   +21     
  Branches     1172    1172           
======================================
+ Hits        47278   47307   +29     
+ Misses       6618    6610    -8     
  Partials      250     250           

Flag	Coverage Δ
integrationtests	64.9% <ø> (-0.1%)	⬇️
unittests	85.1% <100.0%> (+<0.1%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
...g/src/simcore_service_autoscaling/core/settings.py	97.9% <100.0%> (+<0.1%)	⬆️
...e_service_autoscaling/modules/auto_scaling_core.py	94.2% <100.0%> (+<0.1%)	⬆️
...vice_autoscaling/modules/auto_scaling_mode_base.py	81.4% <100.0%> (-0.4%)	⬇️
...scaling/modules/auto_scaling_mode_computational.py	91.0% <100.0%> (ø)
...e_autoscaling/modules/auto_scaling_mode_dynamic.py	100.0% <100.0%> (ø)
.../simcore_service_autoscaling/utils/utils_docker.py	99.0% <100.0%> (+<0.1%)	⬆️

... and 3 files with indirect coverage changes

[pcrespov]

thx!

[matusdrobuliak66]

Thanks for the changes! Q: Now other containers such as exporter will still keep running correct?

[YuryHrytsuk]

Q: do we need to update some director ENV to make sure dynamic sidecar has proper node label constraints?

[GitHK]

🎉

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

[YuryHrytsuk]

🚀 🚀 🚀