🎨 Reduces response time of catalog/services listing entrypoint

[pcrespov]

What do these changes do?

This PR analyses and provides a temporary solution for the incident #5267 that reports very long delays in GET /catalog/services entrypoint. In addition the front-end can at time spawn up to 17 parallel calls to this entrypoint point (probably due to some retry mechanism)

We are well aware that the original design does not scale with the amount of services. A proper resolution of this issue therefore requires a redesign that should incorporate at least pagination and lighter item objects with bounded fields.

For the moment we decided to go with a strategic TTL cache that will reduce the overhead of postprocessing every service item in the webserver. Note that the webserver does not just forward the service object provided by the catalog service but also computes and adds some extra information (e.g. units) to it. Using a profiler reveals that replace_service_inputs incurs a considerable computation overhead. Therefore, with the last increase in the number of services this has caused the large delays reported in #5267.

This is the benchark results and profiler before the cache was added

and this is after

Finally here we can see several calls to the entrypoint. After the first, subsequent calls take much less time

Regarding the empty lists, I also noticed that this comes directly in the response of the catalog service (and not from the webserver). Probably is due to an issue of the caches on the multiple replicas there. Nonetheless this point has not been addressed here

Details

• ⬆️ New cachetools to cache sync functions (we have aiocache to cache async functions`
• ⬆️ Adds pytest-benchmark to benchmark the replace_service_inputs
• ⬆️ Adds msgpack mainly to remove log warning message of some libraries (e.g. aiocache) that uses this as a default if it is available.

Related issue/s

• Analyses and provides a fix for Backend replies with empty list of services #5267

How to test

Driving test services/web/server/tests/unit/isolated/test_catalog_models.py

Dev Checklist

• No ENV changes or I properly updated ENV (read the instruction)

DevOps

[codecov]

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (4fa4917) 87.2% compared to head (508a986) 88.7%.

Additional details and impacted files

@@           Coverage Diff            @@
##           master   #5273     +/-   ##
========================================
+ Coverage    87.2%   88.7%   +1.4%     
========================================
  Files        1308    1213     -95     
  Lines       53550   49548   -4002     
  Branches     1170    1024    -146     
========================================
- Hits        46749   43960   -2789     
+ Misses       6552    5367   -1185     
+ Partials      249     221     -28     

Flag	Coverage Δ
integrationtests	65.2% <61.5%> (-0.1%)	⬇️
unittests	87.1% <92.3%> (+2.0%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
...rver/src/simcore_service_webserver/catalog/_api.py	42.3% <ø> (ø)
...r/src/simcore_service_webserver/catalog/_models.py	64.9% <100.0%> (-26.4%)	⬇️
...er/src/simcore_service_webserver/catalog/client.py	51.1% <0.0%> (ø)

... and 113 files with indirect coverage changes

[sonarqubecloud]

Quality Gate passed

Kudos, no new issues were introduced!

0 New issues
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

[matusdrobuliak66]

👍