🐛 aborting tasks fails

[sanderegg]

What do these changes do?

Cancelling tasks seems to have been failing since a couple of PRs went in.
First, the python 3.8 PR changed how asyncio.loop.call_soon is to used.
Then, when the redis database was splitted in 3 separate DBs (one redis instance with 3 databases) instead of 1. the director-v2 celery client was not fixed accordingly.

This has lead to the following:

1. Cancelling a pipeline would cancel the tasks AFTER the one currently running correctly. BUT the currently running tasks would still run until completion instead of being stopped.
2. When a sidecar is shutdown, the currently running task would not be correctly cancelled.

Both of these issues are now fixed.

Related issue/s

How to test

make build-devel up-devel

then start a pipeline with 1sleeper that lasts more than 2 seconds... cancel it. it should instantly be cancelled.

Checklist

[codecov]

Codecov Report

Merging #2449 (f8eee49) into master (424d857) will decrease coverage by 0.0%.
The diff coverage is 53.8%.

@@           Coverage Diff            @@
##           master   #2449     +/-   ##
========================================
- Coverage    76.0%   75.9%   -0.1%     
========================================
  Files         574     574             
  Lines       21658   21667      +9     
  Branches     2078    2082      +4     
========================================
- Hits        16465   16464      -1     
- Misses       4639    4648      +9     
- Partials      554     555      +1     

Flag	Coverage Δ
integrationtests	65.5% <47.8%> (-0.2%)	⬇️
unittests	69.9% <50.0%> (-0.1%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...ices/sidecar/src/simcore_service_sidecar/celery.py	0.0% <0.0%> (ø)
...r/src/simcore_service_sidecar/celery_task_utils.py	66.6% <ø> (+13.7%)	⬆️
...vices/sidecar/src/simcore_service_sidecar/utils.py	64.7% <12.5%> (-15.3%)	⬇️
...sidecar/src/simcore_service_sidecar/celery_task.py	38.8% <50.0%> (ø)
...ervices/sidecar/src/simcore_service_sidecar/cli.py	65.8% <50.0%> (-1.7%)	⬇️
...es/settings-library/src/settings_library/celery.py	88.8% <83.3%> (+2.2%)	⬆️
...es/sidecar/src/simcore_service_sidecar/rabbitmq.py	88.0% <88.8%> (-2.0%)	⬇️
..._director_v2/modules/dynamic_sidecar/docker_api.py	85.6% <0.0%> (+0.6%)	⬆️

[mguidon]

Thanks for fixing!

[GitHK]

Looks OK, just some proposals to be verified.