-
-
Notifications
You must be signed in to change notification settings - Fork 46
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Loading file in library fails in downstream packages #298
Comments
Seems duplicate of #299 |
(I've seen same error message when pytest_rewrite does it's job) |
The only change introduced with 6.2.0 was #295. |
#257 is back since importlib_resources 6.2.0 |
The issue with jsonschema seems to be masking the underlying error, so it's hard to tell if it's the same FileNotFoundError. That FileNotFoundError comes from the CompatibilityFiles adapter, which shouldn't be reached in the case of a legitimate resource provider: importlib_resources/importlib_resources/_adapters.py Lines 138 to 139 in 9f5b437
Unfortunately, I've not yet been able to replicate the issue, even when using pytest:
Can anyone help devise a reproducer that I can run, preferably something minimal and including importlib_resources in the traceback? I need help reproducing what it is about the failing environments that leads to the CompatibiltyFiles / OrphanPath objects. Maybe something that minimally uses jsonschema might be helpful. |
I thought I'd try to ascertain where jsonschema is going wrong in the handling of resources, but the only mention of importlib_resources in the project is in the pyproject.toml. |
I could revert the change introduced in 6.2.0, but at this stage, it looks like a lot of the issues have already been worked around by pinning importlib_resources, so I'd like to focus on fixing forward, but do let me know if reverting the change would provide benefit to your project (and how) and I'll consider reverting to limit the damage while investigating a long-term solution. |
I've given it a try but I couldn't produce a MRE for this. I've updated our project to restrict Also fwiw, at least for the way we use |
Avoid downstream CI errors caused by python/importlib_resources#298
Avoid downstream CI errors caused by python/importlib_resources#298
We also ran into this with CI error. However, pinning is fine for us, and we can probably migrate things over to the standard library equivalent now anyhow. |
Avoid downstream CI errors caused by python/importlib_resources#298
Building on the repro in #299, I'm able to replicate the reported KeyError with this Dockerfile: FROM jaraco/multipy-tox
RUN git clone https://github.com/apache/airflow
WORKDIR airflow
RUN git checkout 72d19565d84744b59f3d22b32c09ab5fc553b779
RUN py -3.8 -m venv .venv
RUN py -m pip install -e ./dev/breeze
CMD py -m pytest dev/breeze/tests -x |
After some troubleshooting, I think I understand the problem better now. In When running under pytest with the assert rewrite hook enabled (and presumably some other settings, because I couldn't replicate the issue by simply importing jsonschema in a pytest), the loader for While it was intentional (in #297) to expose third-party traversable readers when present, it was not the intention to give the degenerate CompatibilityFiles wrapper precedence for loaders without any implementation. |
What I don't yet understand is why doesn't this issue manifest on newer Pythons. Presumably the same AssertionRewritingHook loader exists and would similarly get wrapped in a degenerate I created this tox recipe to run in [tox]
package_root = dev/breeze
[testenv]
commands =
pytest dev/breeze/tests {posargs}
package = editable |
Aha - It's because pytest's hook implements a resource provider on Python 3.10+, so even though the hook is present, it's deferring to the stdlib readers. |
Yep. importlib_resources is not even installed in Airflow for 3.10 + |
…aders and instead prefer the standard readers before once again falling back to any CompatibilityFiles reader. Closes #298.
After pushing a proposed fix, I've confirmed the fix has the intended effect with this tox.ini file:
Running it in apache/airflow@72d19565d8 with |
Releasing as v6.3.2; please report back if it has any unintended consequences or doesn't solve the issue for another use case. |
Reporting back that 6.3.2 fixed the issue for us. Thanks! |
The problem with importlib_resources interacting badly with pytest_rewrite has been solved in 6.3.2 version today and we can simply skip over the bad versions. See python/importlib_resources#298
Running the PR on Airflow apache/airflow#38294 |
The problem with importlib_resources interacting badly with pytest_rewrite has been solved in 6.3.2 version today and we can simply skip over the bad versions. See python/importlib_resources#298
Confirmed! Works! |
The problem with importlib_resources interacting badly with pytest_rewrite has been solved in 6.3.2 version today and we can simply skip over the bad versions. See python/importlib_resources#298 (cherry picked from commit 2cb4209)
Thanks for quick handling It @jaraco ! |
…sue with pytest with jsonschema" This reverts commit 6703d30. According to python/importlib_resources#298, the issue has been fixed in importlib-resources 6.3.2
…sue with pytest with jsonschema" This reverts commit 6703d30. According to python/importlib_resources#298, the issue has been fixed in importlib-resources 6.3.2
The problem with importlib_resources interacting badly with pytest_rewrite has been solved in 6.3.2 version today and we can simply skip over the bad versions. See python/importlib_resources#298 GitOrigin-RevId: 2cb4209f59dd285eface337a28a173be6e194057
The problem with importlib_resources interacting badly with pytest_rewrite has been solved in 6.3.2 version today and we can simply skip over the bad versions. See python/importlib_resources#298 GitOrigin-RevId: 2cb4209f59dd285eface337a28a173be6e194057
The problem with importlib_resources interacting badly with pytest_rewrite has been solved in 6.3.2 version today and we can simply skip over the bad versions. See python/importlib_resources#298 GitOrigin-RevId: 2cb4209f59dd285eface337a28a173be6e194057
Hey 👋
Starting with release
6.2.0
, downstream packages dependent on a library that uses the equivalent offail with
FileNotFoundError: Can't open orphan path
.This seems to only happen in Python 3.8 and 3.9.
For reference, meltano/sdk#2310 is a PR where I'm trying to fix things on my end in case I was using your APIs incorrectly, but still a breaking change might have been inadvertently introduced.
The text was updated successfully, but these errors were encountered: