♻️ Maintenance/refactors servicelib

[pcrespov]

What do these changes do?

• adds extra aiohttp to separate dependencies exclusive to aiohttp-services
• can install and test using minimal installation (e.g. for fastapi-based services) or w/ aiohttp extra (for aiohttp-based services)

In detail

• adds extra requirements aiohttp (affects requirements/* and setup.py)
• moves aiohttp-specific modules inside src/servicelib/aiohttp
• moves aiohttp-specific tests inside tests/aiohttp
• changes in Makefile

Related issue/s

How to test

make devenv
source .venv/bin/activate
cd packages/services-library

# installs normally (i.e. no aiohttp extras)
make install-dev
make test-dev

# install with aiohttp extras
make "install-dev[aiohttp]"
make "test[aiohttp]"

Checklist

• CI two tests: with aiohttp extras and without aiohttp extras
• adapt services using servicelib:
  • aiohttp-based: web-server and storage - install as service-library[aiohttp]
  • fastapi-based: - install as service-library
• increase version of servicelib
• simcore-sdk now only depends on service-library
• run make reqs on all involved dependencies to make sure it works correctly

[codecov]

Codecov Report

Merging #2516 (186167a) into master (7382c4e) will decrease coverage by 0.0%.
The diff coverage is 97.9%.

@@           Coverage Diff            @@
##           master   #2516     +/-   ##
========================================
- Coverage    76.6%   76.6%   -0.1%     
========================================
  Files         609     610      +1     
  Lines       23388   23397      +9     
  Branches     2294    2294             
========================================
+ Hits        17926   17928      +2     
- Misses       4857    4865      +8     
+ Partials      605     604      -1     

Flag	Coverage Δ
integrationtests	66.5% <98.3%> (+<0.1%)	⬆️
unittests	70.6% <97.9%> (-0.1%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...vice-library/src/servicelib/aiohttp/application.py	0.0% <ø> (ø)
...library/src/servicelib/aiohttp/application_keys.py	100.0% <ø> (ø)
...ibrary/src/servicelib/aiohttp/application_setup.py	81.9% <ø> (ø)
...e-library/src/servicelib/aiohttp/client_session.py	0.0% <0.0%> (ø)
...rary/src/servicelib/aiohttp/config_schema_utils.py	0.0% <ø> (ø)
...ervice-library/src/servicelib/aiohttp/incidents.py	84.6% <ø> (ø)
...library/src/servicelib/aiohttp/jsonschema_specs.py	0.0% <ø> (ø)
...ry/src/servicelib/aiohttp/jsonschema_validation.py	0.0% <ø> (ø)
...library/src/servicelib/aiohttp/monitor_services.py	0.0% <ø> (ø)
...rvice-library/src/servicelib/aiohttp/monitoring.py	0.0% <ø> (ø)
... and 125 more

[GitHK]

@pcrespov while trying to make this PR work, it was not possible to do so.

• simcore-sdk depends on servicelib[aiohttp] (aiopg_utils)
• simcore-sdk will also be present in dynamic-sidecar (♻️✨adding nodeports support to dynamic-sidecar #2509), but instead of servicelib[aiohttp] the service should install servicelib (without extra aiohttp dependencies).

I propose closing this PR.

In alternative I'd create a new library called utilslib containing general utility functions, to be shared among other packages and services:

• for my future use case there are are 2 functions which need to be shared by servicelib, director-v2, simcore-sdk, dynamic-sidecar.
• I would also add a test to avoid check that this new library installs other simcore packages (avoids future issues).

FYI: @sanderegg @mguidon

[GitHK]

After a discussione with @pcrespov, we decided to break dependency between simcore-sdk and servicelib.

• will copy functions from the apiopg_utils
• archiving_utils will go into a separate very small focused library called archivinglib.

[GitHK]

In the end, I've managed to refactor some code and avoid adding additional libraries

[GitHK]

Think I've managed to address everything.

@pcrespov one more thing does not convince me. In the webserver I did not change anything inside requirements. But I've managed to fix all the tests and to use the "new import paths". I think you mentioned something about the code still being there even after installing servicelib without the aiohttp part. That would explain why it works.

[pcrespov]

Think I've managed to address everything.

@pcrespov one more thing does not convince me. In the webserver I did not change anything inside requirements. But I've managed to fix all the tests and to use the "new import paths". I think you mentioned something about the code still being there even after installing servicelib without the aiohttp part. That would explain why it works.

@GitHK is there any reason why you add the _aiohttp.in requirements in the webserver? Most of the dependencies in servicelib[aiohttp] are already included in web-server installer and the test coverage is not 100%.
I did some refactoring and added it

[GitHK]

👍 I think it looks all good now.

[mguidon]

Cleaning Lady was here...