Banning non runtime type in from_type

[Cheukting]

Banning the non-runtime types in from_type. See #2978 and #3280 for more detail.

[Cheukting]

@Zac-HD I think I would like your review now, the rest of the failed tests are parallelization issues (ERROR collecting gw1) Please kindly guide me to solve the issue and any refactoring if needed.

[Zac-HD]

@Zac-HD I think I would like your review now, the rest of the failed tests are parallelization issues (ERROR collecting gw1) Please kindly guide me to solve the issue and any refactoring if needed.

Ah-ha, got it: the problem is that NON_RUNTIME_TYPES is a frozenset, and on Python 3.9+ the iteration order can (and does) vary between the different pytest processes: if you look closely at the diff here, e.g. ctrl-f Never, you'll see that it's a comparison between shuffled lines. I think the best solution is to give them a stable ordering, such as:

@pytest.mark.parametrize("non_runtime_type", sorted(NON_RUNTIME_TYPES, key=lambda t: t.__name__))

[Zac-HD]

This looks really nice Cheuk - thanks again!

[Cheukting]

@Zac-HD I think I would like your review now, the rest of the failed tests are parallelization issues (ERROR collecting gw1) Please kindly guide me to solve the issue and any refactoring if needed.

Ah-ha, got it: the problem is that NON_RUNTIME_TYPES is a frozenset, and on Python 3.9+ the iteration order can (and does) vary between the different pytest processes: if you look closely at the diff here, e.g. ctrl-f Never, you'll see that it's a comparison between shuffled lines. I think the best solution is to give them a stable ordering, such as:

@pytest.mark.parametrize("non_runtime_type", sorted(NON_RUNTIME_TYPES, key=lambda t: t.__name__))

The only problem is that they don't have __name__ but if we only need a consistent order I just use str

[Cheukting]

@Zac-HD pytest in 3.11 is doing something "interesting" here, the raises gives something internal again: https://github.com/HypothesisWorks/hypothesis/runs/6325869719?check_suite_focus=true#step:6:880

[Zac-HD]

Hmm, I'd guess that this is a real behavior difference under Python 3.11 due to some internal change to Concatenate (this was really common up until 3.9 or so, and we are looking at internal APIs). No particular advice beyond "open a repl or debugger each in 3.10 and 3.11, and step through the logic until you see what's different".

But otherwise we're looking ready to merge!

[Cheukting]

I love investigations and will give you an update. Then we will see what is the next step (fixing this behavior or merge and open new issue for it) will let you know what I find.

…

On Fri, 6 May 2022 at 18:59, Zac Hatfield-Dodds ***@***.***> wrote: Hmm, I'd guess that this is a real behavior difference under Python 3.11 due to some internal change to Concatenate (this was *really* common up until 3.9 or so, and we *are* looking at internal APIs). No particular advice beyond "open a repl or debugger each in 3.10 and 3.11, and step through the logic until you see what's different". But otherwise we're looking ready to merge! — Reply to this email directly, view it on GitHub <#3331 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AG3N26JSNHSUVEINEIXX4J3VIVMZZANCNFSM5U4U7CVQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[Cheukting]

@Zac-HD On a 2nd thought, this discrepancy is so similar to the one we found in python/cpython#92118 maybe it will get fixed in the next release

[Zac-HD]

Makes sense - I'm planning to update our CI for beta1 tomorrow, and will rebase your PR on top of those changes then 👍

[Cheukting]

Makes sense - I'm planning to update our CI for beta1 tomorrow, and will rebase your PR on top of those changes then 👍

Just so you know 3.11.0-beta 1 is out :-)

[Zac-HD]

Yep, and pyenv updated overnight so #3337 is unblocked 😁

[Zac-HD]

Woohoo!

I ended up breaking a few things myself while refactoring, but this all looks good and ready to go 🚀