♻️ fixes flaky test_time_overhead_on_handlers_of_auth_decorators

[pcrespov]

What do these changes do?

test_time_overhead_on_handlers_of_auth_decorators would run sequentially two requests (w/ and w/o auth) and compare their times to make sure the overhead of auth_decorators was bound.

@sanderegg correctly pointed out that this test was not well defined since in the CI multiple tests are run in parallel and we cannot control how busy the machines are . Therefore even if in the original test we do a relative comparison, we cannot guarantee that each of the individual runs were done in "equivalent" conditions.

To resolve this issue, this test has been refactored to run multiple times each request and compute use the median of each of the runs do the comparison ( Note that the mean is strongly affected by outliers and is not necessarily a typical example of the data points. For a more robust, although less efficient, measure of central tendency we use media. Quote from 1)

NOTE: I also tried to use pytest-benchmark to do this test I had many troubles with the event loop (benchrmark needs a sync callable) and also was not clear to me how to compare the two different runs.

Related issue/s

• Fixes flaky test introduced in ✨ New product-bound login: user must have access to target product 🚨 #5262

How to test

cd services/web/server
make install-dev
pytest -vv --pdb tests/unit/isolated/test_security_api.py

Two driving tests:

• test_number_of_db_calls_on_handlers_of_auth_decorators ensures max. number of calls to the db from decorators
• test_time_overhead_on_handlers_of_auth_decorators benchmarks class w/ and w/o authz and compares time overhead.

Dev Checklist

• No ENV changes or I properly updated ENV (read the instruction)

DevOps

None

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (fc33561) 87.3% compared to head (35351ee) 76.8%.

Additional details and impacted files

@@            Coverage Diff            @@
##           master   #5279      +/-   ##
=========================================
- Coverage    87.3%   76.8%   -10.5%     
=========================================
  Files        1316     542     -774     
  Lines       53813   27142   -26671     
  Branches     1170     202     -968     
=========================================
- Hits        46988   20857   -26131     
+ Misses       6576    6235     -341     
+ Partials      249      50     -199     

Flag	Coverage Δ
integrationtests	60.8% <ø> (-4.4%)	⬇️
unittests	86.6% <ø> (+1.4%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

see 912 files with indirect coverage changes

[sonarqubecloud]

Quality Gate passed

Kudos, no new issues were introduced!

0 New issues
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud