Global SDK state (mainly: strict mode) is not correctly handled in our Python test suite

[abey79]

Our Python test suite handles extremely bad the global state of the SDK, which leaks from tests to tests.

Symptom

For example, the suite currently passes if run in its entirety:

$  pixi run -e py pytest -c rerun_py/pyproject.toml rerun_py/tests
(passes)

If you run only the quaternion suite, it fails:

$ pixi run -e py pytest -c rerun_py/pyproject.toml rerun_py/tests/unit/test_quaternion.py
(fails test_quaternion_array_invalid)

Why? Because that test expect strict mode but doesn't set it. It just happens to be set by the previous test(s) in the suite.

Proof:

$ RERUN_STRICT=1 pixi run -e py pytest -c rerun_py/pyproject.toml rerun_py/tests/unit/test_quaternion.py
(passes)

Solution

In theory, tests should each receive a rr fixture that corresponds to an isolated instance of the rerun-sdk module. Somehow. In practice, maybe some Python module shenanigans might achieve that (needs to be researched and experimented with).