Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

avoid using __file__ in pytest_plugin_registered as can be wrong on Windows #11825

Merged
merged 4 commits into from
Jan 17, 2024

Conversation

woutdenolf
Copy link
Contributor

@woutdenolf woutdenolf commented Jan 16, 2024

closes #11816

Alternative fix to #11821.

The plugin name is equal to the path for conftest plugins (suggested in #11816 (comment)). Use that instead of plugin.__file__.

Test with the h5py CI that was broken by the issue: h5py/h5py#2370

Another way of testing this MR:

python -m pip install "ewokscore==0.7.1" matplotlib ipykernel^
    git+https://github.com/woutdenolf/pytest.git@fix_missing_fixture_issue --no-cache
python -m pytest -v --pyargs ewokscore.tests

nodeid = ""
if os.sep != nodes.SEP:
nodeid = nodeid.replace(os.sep, nodes.SEP)
plugin_name = self.config.pluginmanager.get_name(plugin)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bluetech Is that what you meant in #11816 (comment)?

@woutdenolf woutdenolf marked this pull request as ready for review January 16, 2024 15:13
Copy link
Member

@bluetech bluetech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, exactly, this LGTM.

Test failures seem unrelated. "hypothesis.errors.HypothesisSideeffectWarning: Slow code in plugin: avoid lazy evaluation of text() at import time!" never seen this before. Unless the get_name slowness is actually the culprit... I will re-run the jobs later.

As for optimizing the get_name call, I can do it in a separate PR, unless you want to do it here already. The best approach I think is to add a plugin_name parameter to the pytest_plugin_registered hook.

@woutdenolf
Copy link
Contributor Author

woutdenolf commented Jan 16, 2024

The best approach I think is to add a plugin_name parameter to the pytest_plugin_registered hook.

No idea what this means so if you could handle it in another MR, that would be great.

@bluetech
Copy link
Member

The Hypothesis issue seems to reproduce when re-running the jobs, though some jobs pass. The (existing, seems to be unrelated to this PR) test is:

@hypothesis.given(strategies.text() | strategies.binary())
@hypothesis.settings(
    deadline=400.0
)  # very close to std deadline and CI boxes are not reliable in CPU power
def test_idval_hypothesis(self, value) -> None:
    escaped = IdMaker([], [], None, None, None, None, None)._idval(value, "a", 6)
    assert isinstance(escaped, str)
    escaped.encode("ascii")

The error is:

______________________ TestMetafunc.test_idval_hypothesis ______________________

self = <metafunc.TestMetafunc object at 0x7f5c52dfd810>

    @hypothesis.given(strategies.text() | strategies.binary())
>   @hypothesis.settings(
        deadline=400.0
    )  # very close to std deadline and CI boxes are not reliable in CPU power

/home/runner/work/pytest/pytest/testing/python/metafunc.py:296: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/home/runner/work/pytest/pytest/.tox/py311-coverage/lib/python3.11/site-packages/hypothesis/core.py:669: in process_arguments_to_given
    s.validate()
/home/runner/work/pytest/pytest/.tox/py311-coverage/lib/python3.11/site-packages/hypothesis/strategies/_internal/strategies.py:417: in validate
    self.do_validate()
/home/runner/work/pytest/pytest/.tox/py311-coverage/lib/python3.11/site-packages/hypothesis/strategies/_internal/strategies.py:684: in do_validate
    for e in self.element_strategies:
/home/runner/work/pytest/pytest/.tox/py311-coverage/lib/python3.11/site-packages/hypothesis/strategies/_internal/strategies.py:659: in element_strategies
    if not arg.is_empty:
/home/runner/work/pytest/pytest/.tox/py311-coverage/lib/python3.11/site-packages/hypothesis/strategies/_internal/strategies.py:136: in accept
    recur(self)
/home/runner/work/pytest/pytest/.tox/py311-coverage/lib/python3.11/site-packages/hypothesis/strategies/_internal/strategies.py:132: in recur
    mapping[strat] = getattr(strat, calculation)(recur)
/home/runner/work/pytest/pytest/.tox/py311-coverage/lib/python3.11/site-packages/hypothesis/strategies/_internal/lazy.py:89: in calc_is_empty
    return recur(self.wrapped_strategy)
/home/runner/work/pytest/pytest/.tox/py311-coverage/lib/python3.11/site-packages/hypothesis/strategies/_internal/lazy.py:104: in wrapped_strategy
    check_sideeffect_during_initialization("lazy evaluation of {!r}", self)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

what = 'lazy evaluation of {!r}', extra = '', fmt_args = (text(),)
msg = 'lazy evaluation of text()'

    def check_sideeffect_during_initialization(
        what: str, *fmt_args: object, extra: str = ""
    ) -> None:
        """Called from locations that should not be executed during initialization, for example
        touching disk or materializing lazy/deferred strategies from plugins. If initialization
        is in progress, a warning is emitted.
    
        Note that computing the repr can take nontrivial time or memory, so we avoid doing so
        unless (and until) we're actually emitting the warning.
        """
        global _first_postinit_what
        # This is not a particularly hot path, but neither is it doing productive work, so we want to
        # minimize the cost by returning immediately. The drawback is that we require
        # notice_initialization_restarted() to be called if in_initialization changes away from zero.
        if _first_postinit_what is not None:
            return
        elif _hypothesis_globals.in_initialization:
            # Note: -Werror is insufficient under pytest, as doesn't take effect until
            # test session start.
            msg = what.format(*fmt_args)
>           warnings.warn(
                f"Slow code in plugin: avoid {msg} at import time!  Set PYTHONWARNINGS=error "
                "to get a traceback and show which plugin is responsible." + extra,
                HypothesisSideeffectWarning,
                stacklevel=3,
            )
E           hypothesis.errors.HypothesisSideeffectWarning: Slow code in plugin: avoid lazy evaluation of text() at import time!  Set PYTHONWARNINGS=error to get a traceback and show which plugin is responsible.

/home/runner/work/pytest/pytest/.tox/py311-coverage/lib/python3.11/site-packages/hypothesis/configuration.py:67: HypothesisSideeffectWarning

@Zac-HD Maybe you have an idea? The test seems run of the mill to me and I don't see any import-time evaluation going on but I don't have much experience with Hypothesis.

@Zac-HD
Copy link
Member

Zac-HD commented Jan 17, 2024

We just added that warning last night, and I suspect a pytest-specific interaction... I suggest globally ignoring HypothesisSideeffectWarning for now to unblock this PR and we'll look into a better fix upstream.

woutdenolf and others added 2 commits January 17, 2024 15:01
Otherwise mypy doesn't fully recognize pluggy's typing for some reason
or another.
@bluetech
Copy link
Member

Added the change to the hook. Did it in this PR to make backporting easier.

…ed` hook

We have a use case for this in the next commit.

The name can be obtained by using `manager.get_name(plugin)`, however
this is currently O(num plugins) in pluggy, which would be good to
avoid. Besides, it seems generally useful.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Main and 8.0.x (not released yet): cannot find fixture when using --pyargs
3 participants