Importlib-resources 6.2.0 and 6.3.0 break pytest_rewrite with jsonschema and likely other packages

[potiuk]

Not sure exactly how to reproduce it, but I thought I should report it.

Importlb-resources 6.2.0 and 6.3.0 break how pytest_rewrite works in case of some asserts. I have a fully reproducible way of triggering it, though it's not a "minmum reproducible" case. The issue started to happen today after 6.2.0 release in Airflow CI and I narrowed it down and reproduced it, but I am not sure how to reproduce it "easily".

The steps I have now:

1. Clone airflow repo:

> git clone git@github.com:apache/airflow.git

2. Checkout todays main (I am planning to limit importlib_resources so main can stop showing the error):

> git checkout 72d19565d84744b59f3d22b32c09ab5fc553b779

3. create venv and install breeze sub-project (develipment environment of Airflow). Make sure to install latest versions of packages

> pip install -e ./dev/breeze --uppgrade --upgrade-strategy eager

Check importlib_resources version:

> pip freeze | grep importlib
importlib_metadata==7.0.2
importlib_resources==6.3.0

4. go to ./dev/breeze and run tests

> cd ./dev/breeze
> pytest tests

Observe many tests failing (stack trace below)

5. Downgrade to importlib_resources 6.2.0:

> pip install importlib_resources==6.2.0
Collecting importlib_resources==6.2.0
  Using cached importlib_resources-6.2.0-py3-none-any.whl.metadata (3.9 kB)
Requirement already satisfied: zipp>=3.1.0 in /home/jarek/.pyenv/versions/3.8.17/envs/test-pytest-rewrite/lib/python3.8/site-packages (from importlib_resources==6.2.0) (3.17.0)
Using cached importlib_resources-6.2.0-py3-none-any.whl (35 kB)
Installing collected packages: importlib_resources
  Attempting uninstall: importlib_resources
    Found existing installation: importlib_resources 6.3.0
    Uninstalling importlib_resources-6.3.0:
      Successfully uninstalled importlib_resources-6.3.0
Successfully installed importlib_resources-6.2.0

Running pytest test produces the same error

6. Downgrade to importlib_resources 6.1.3:

> pip install importlib_resources==6.1.3
Collecting importlib_resources==6.1.3
  Using cached importlib_resources-6.1.3-py3-none-any.whl.metadata (3.9 kB)
Requirement already satisfied: zipp>=3.1.0 in /home/jarek/.pyenv/versions/3.8.17/envs/test-pytest-rewrite/lib/python3.8/site-packages (from importlib_resources==6.1.3) (3.17.0)
Using cached importlib_resources-6.1.3-py3-none-any.whl (34 kB)
Installing collected packages: importlib_resources
  Attempting uninstall: importlib_resources
    Found existing installation: importlib_resources 6.2.0
    Uninstalling importlib_resources-6.2.0:
      Successfully uninstalled importlib_resources-6.2.0
Successfully installed importlib_resources-6.1.3

Running pytest test succeeds.

It looks like some changes in importlib_resources interfere with some of the assert-rewrite mechanims that pytest uses.

---

Example stack trace:

home/jarek/code/airflow/dev/breeze/src/airflow_breeze/prepare_providers/provider_documentation.py:488: in _verify_changelog_exists
    provider_details = get_provider_details(package)
/home/jarek/code/airflow/dev/breeze/src/airflow_breeze/utils/packages.py:484: in get_provider_details
    provider_info = get_provider_packages_metadata().get(provider_id)
/home/jarek/code/airflow/dev/breeze/src/airflow_breeze/utils/packages.py:161: in get_provider_packages_metadata
    refresh_provider_metadata_from_yaml_file(provider_yaml_path)
/home/jarek/code/airflow/dev/breeze/src/airflow_breeze/utils/packages.py:131: in refresh_provider_metadata_from_yaml_file
    import jsonschema
<frozen importlib._bootstrap>:991: in _find_and_load
    ???
<frozen importlib._bootstrap>:975: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:671: in _load_unlocked
    ???
/home/jarek/.pyenv/versions/3.8.17/envs/test-pytest-rewrite/lib/python3.8/site-packages/_pytest/assertion/rewrite.py:186: in exec_module
    exec(co, module.__dict__)
/home/jarek/.pyenv/versions/3.8.17/envs/test-pytest-rewrite/lib/python3.8/site-packages/jsonschema/__init__.py:16: in <module>
    from jsonschema.validators import (
<frozen importlib._bootstrap>:991: in _find_and_load
    ???
<frozen importlib._bootstrap>:975: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:671: in _load_unlocked
    ???
/home/jarek/.pyenv/versions/3.8.17/envs/test-pytest-rewrite/lib/python3.8/site-packages/_pytest/assertion/rewrite.py:186: in exec_module
    exec(co, module.__dict__)
/home/jarek/.pyenv/versions/3.8.17/envs/test-pytest-rewrite/lib/python3.8/site-packages/jsonschema/validators.py:589: in <module>
    meta_schema=SPECIFICATIONS.contents(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <Registry (0 resources)>, uri = 'http://json-schema.org/draft-03/schema#'

    def contents(self, uri: URI) -> D:
        """
        Retrieve the (already crawled) contents identified by the given URI.
        """
        # Empty fragment URIs are equivalent to URIs without the fragment.
        # TODO: Is this true for non JSON Schema resources? Probably not.
>       return self._resources[uri.rstrip("#")].contents
E       KeyError: 'http://json-schema.org/draft-03/schema'

[rouault]

We were just hit by that on the GDAL project: OSGeo/gdal@6703d30

[bunchesofdonald]

Same with the Prefect project: https://github.com/PrefectHQ/prefect/actions/runs/8257621747/job/22588504279

[jaraco]

This issue is likely a duplicate of #298.