Different behaviour of 1.0.0 wheels from pypi

[klausweiss]

Hello,

I noticed that the manylinux_2_17_x86_64.manylinux2014_x86_64 and py3-none-any pypi wheels behave differently.

Bug Report

...but I don't know specifically what the issue is 😬
I wrote a mypy plugin. It used to work correctly with mypy 0.920-0.991. Then the tests started failing in CI when 1.0.0 was released. I've easily reproduced that locally. Wanted to get to the bottom of it, so downloaded mypy source, checked out v1.0.0, installed the local package and... the tests passed.
I've only later discovered that when I use the py3-none-any wheel from pypi it also works as expected.

The plugin hooks into get_type_analyze_hook, get_method_signature_hook and get_function_signature_hook modifying instances of a specific class (joining Protocols together). I understand this is no place to seek help in my project, but I believe the issue is with these particular distributions and not on my end, since everything works correctly when using the source or the generic wheel.

Locally I'm testing everything with python 3.10.9, but I've also confirmed this is the case with 3.11.1 and it failed with 3.7.15 in CI (Github Actions).

To Reproduce

I wish I could provide a simpler example, but I'm afraid I can't. I've tried narrowing the issue down, but couldn't do it with a compiled version of mypy and the source version works beautifully 😬 .

git clone https://github.com/klausweiss/typing-protocol-intersection.git --branch mypy-1.0.0
cd typing-protocol-intersection
python -m venv venv-manylinux
source venv-manylinux/bin/activate
pip download mypy==1.0.0 --platform=manylinux_2_17_x86_64 --only-binary=:all: 
pip install *.whl
mypy --config-file tests/test-mypy.ini tests/testcases/in_generic_param_unhappy_path.py --no-incremental

Outcome is

tests/testcases/in_generic_param_unhappy_path.py:49:15: error: Argument 1 to "get_x_y_1" has incompatible type "ProtocolIntersection[HasX]"; expected "DesiredObject"  [arg-type]
tests/testcases/in_generic_param_unhappy_path.py:49:15: note: "ProtocolIntersection" is missing following "DesiredObject" protocol member:
tests/testcases/in_generic_param_unhappy_path.py:49:15: note:     y
Found 1 error in 1 file (checked 1 source file)

Expected Behavior

Outcome:

tests/testcases/in_generic_param_unhappy_path.py:49:15: error: Argument 1 to "get_x_y_1" has incompatible type "ProtocolIntersection[HasX]"; expected "DesiredObject"  [arg-type]
tests/testcases/in_generic_param_unhappy_path.py:49:15: note: "ProtocolIntersection" is missing following "DesiredObject" protocol member:
tests/testcases/in_generic_param_unhappy_path.py:49:15: note:     y
tests/testcases/in_generic_param_unhappy_path.py:50:15: error: Argument 1 to "get_x_y_2" has incompatible type "typing_protocol_intersection.types.ProtocolIntersection[HasX]"; expected "typing_protocol_intersection.types.ProtocolIntersection[HasY, HasX]"  [arg-type]
tests/testcases/in_generic_param_unhappy_path.py:50:15: note: "ProtocolIntersection" is missing following "ProtocolIntersection" protocol member:
tests/testcases/in_generic_param_unhappy_path.py:50:15: note:     y
Found 2 errors in 1 file (checked 1 source file)

You get the expected outcome when you follow the same steps as above, but downloading a py3-any-none wheel instead:

git clone https://github.com/klausweiss/typing-protocol-intersection.git --branch mypy-1.0.0
cd typing-protocol-intersection
python -m venv venv-manylinux
source venv-manylinux/bin/activate
pip download mypy==1.0.0 --platform=none --only-binary=:all: 
pip install *.whl
mypy --config-file tests/test-mypy.ini tests/testcases/in_generic_param_unhappy_path.py --no-incremental

Again, that's also the behavior I observed when cloning the mypy repo, checking out v1.0.0, and installing from a local directory.

Actual Behavior

See the To Reproduce section.

Your Environment

• Mypy version used: v1.0.0
• Mypy command-line flags: --no-incremental --config-file tests/test-mypy.ini
• Mypy configuration options from mypy.ini (and other config files):
  • contents of tests/test-mypy.ini:

    [mypy]
    plugins = ../typing_protocol_intersection/mypy_plugin.py
    check_untyped_defs = True
    disallow_any_generics = True
    disallow_any_unimported = True
    disallow_incomplete_defs = True
    disallow_subclassing_any = True
    disallow_untyped_calls = True
    disallow_untyped_decorators = True
    disallow_untyped_defs = True
    implicit_reexport = False
    no_implicit_optional = True
    show_column_numbers = True
    show_error_codes = True
    strict_equality = True
    strict_optional = True
    warn_no_return = True
    warn_redundant_casts = True
    warn_return_any = True
    warn_unused_configs = True
    

• Python version used: 3.10.9 but also with 3.11.1 and 3.7.15, haven't checked any others
• OS: Manjaro Linux 5.15.91, glibc version 2.36

[klausweiss]

I've identified that v1.0.0+dev.f37a0aeeb89b87de2d2bcdb7a749e4e6f63a3185 was the first build that introduced the issue.

Was able to reproduce it with locally-built wheels - cloned mypy locally, checkedout the commit that's of my interest and installed mypy from the local directory in the venv from above, having set the MYPY_USE_MYPYC="1" environment variable.

This allowed me to identify the commit that introduced the regression: 28e5436

This might break some plugins, so we should document this in release notes and the relevant issue that tracks plugin incompatibilities.

👋

I can see it was announced (#6617 (comment)), but never would have thought it could affect my plugin. Subscribed to the Announcement issue just now. Time to dive into the mypy internals to understand why it was affected, I guess.

[hauntsaninja]

Thanks for narrowing that down 🤔 mypyc must be miscompiling something to do with IncomparableTypeName

@JukkaL do you have any suspicious for why mypyc-compiled mypy is different from interpreted mypy with this plugin?

[JukkaL]

These are some things that could cause issues in plugins:

• inheritance from certain mypy implementation classes (and overriding methods/properties)
• defining a subclass of str (or a few other builtin types)
• catching built-in exceptions, such as TypeError or ValueError

Each of these could plausibly generate issues when running compiled mypy. I notice that the plugin actually does inherit from str (IncomparableTypeName). This is the likely reason why things go wrong. Previously the type of fullname was Any when compiling so no specialized str primitives could be used by mypyc. Now str primitives are used, and these apparently don't support str subclassing properly. In particular, overriding __eq__ does not seem to be supported.

We should either document the incompatibility (if it's undocumented) or fix it. Fixing it will probably have a slight performance penalty, but it may well be worth it, as deviations such as these are surprising and hard to debug.