Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Type is incompatible with Callable when using Concatenate #14321

Closed
ringohoffman opened this issue Dec 20, 2022 · 1 comment · Fixed by #15913
Closed

Type is incompatible with Callable when using Concatenate #14321

ringohoffman opened this issue Dec 20, 2022 · 1 comment · Fixed by #15913
Labels
bug mypy got something wrong topic-paramspec PEP 612, ParamSpec, Concatenate

Comments

@ringohoffman
Copy link

Bug Report

As expected, a class A can be typed as Type[A] or Callable[P, A], where P is a ParamSpec.

Using Concatenate then, you should also be able to bind A to Callable[Concatenate[<type>, P], A], where <type> is the type of A's first constructor parameter, but mypy currently errors when you attempt this.

To Reproduce

gist: https://gist.github.com/mypy-play/1cab71a52f3d7f3a4e9045e6e68eabe0
mypy-playground: https://mypy-play.net/?mypy=latest&python=3.10&gist=1cab71a52f3d7f3a4e9045e6e68eabe0

from __future__ import annotations

from typing import Callable, Concatenate, ParamSpec

P = ParamSpec("P")


class A:
    def __init__(self, a_param_1: str) -> None:
        ...

    @classmethod
    def add_params(cls: Callable[P, A]) -> Callable[Concatenate[float, P], A]:
        def new_constructor(i: float, *args: P.args, **kwargs: P.kwargs) -> A:
            return cls(*args, **kwargs)
        return new_constructor
    
    @classmethod
    def remove_params(cls: Callable[Concatenate[str, P], A]) -> Callable[P, A]:
        def new_constructor(*args: P.args, **kwargs: P.kwargs) -> A:
            return cls("my_special_str", *args, **kwargs)
        return new_constructor

A.add_params()  # OK
A.remove_params()  # [misc] mypy(error): Invalid self argument "Type[A]" to attribute function "remove_params" with type "Callable[[Callable[[str, **P], A]], Callable[P, A]]"

Expected Behavior

No error should be raised.

Actual Behavior

error: Invalid self argument "Type[A]" to attribute function "remove_params" with type "Callable[[Callable[[str, **P], A]], Callable[P, A]]"  [misc]

Your Environment

  • Mypy version used: mypy 0.991 (compiled: yes)
  • Python version used: 3.10.6
@ringohoffman ringohoffman added the bug mypy got something wrong label Dec 20, 2022
@JelleZijlstra JelleZijlstra added the topic-paramspec PEP 612, ParamSpec, Concatenate label Dec 20, 2022
@ringohoffman
Copy link
Author

The difference in behavior between add_params and remove_params seems to come from this line in check_self_arg :

if subtypes.is_subtype(dispatched_arg_type, erase_typevars(erase_to_bound(selfarg))):
name selfarg dispatched_arg_type subtypes.is_subtype(...)
add_params def (*P.args, **P.kwargs) -> A def (a_param_1: builtins.str) -> A True from are_trivial_parameters(...) in are_parameters_compatible
remove_params def (builtins.str, *P.args, **P.kwargs) -> A def (a_param_1: builtins.str) -> A False from not allow_partial_overlap in are_parameters_compatible

In the case of remove_params, by setting is_callable_compatible(allow_partial_overlap=right.from_concatenate or left.from_concatenate) in visit_callable_type, we can progress a little further into are_parameters_compatible... however we then return False from the condition in "Phase 2":

if (
    right_by_name is not None
    and right_by_pos is not None
    and right_by_name != right_by_pos
    and (right_by_pos.required or right_by_name.required)
    and strict_concatenate_check
):
    return False
right_by_name right_by_pos
FormalArgument(name='a_param_1', pos=None, typ=Any, required=False) FormalArgument(name=None, pos=0, typ=builtins.str, required=True)

I feel like if def (a_param_1: builtins.str) -> A is considered a subtype of def (*P.args, **P.kwargs) -> A, then it should also be a subtype of def (builtins.str, *P.args, **P.kwargs) -> A, since this seems to be how Concatenate works in similar constructions:

from typing import ParamSpec, TypeVar

P = ParamSpec("P")
T = TypeVar("T")


def decorator(f: Callable[Concatenate[str, P], T]) -> Callable[P, T]:
    def new_func(*args: P.args, **kwargs: P.kwargs) -> T:
        return f("my_special_str", *args, **kwargs)
    return new_func


@decorator  # OK
def foo(foo_param_1: str) -> None:
    ...

@A5rocks @cdce8p @JukkaL Does my reasoning and proposed change seem sound? What change related to the "Phase 2" condition should be made s.t. the subtype check passes?

hauntsaninja pushed a commit that referenced this issue Sep 14, 2023
Fixes #15734
Fixes #15188
Fixes #14321
Fixes #13107 (plain Callable was
already working, this fixes the protocol example)
Fixes #16058

It looks like treating trivial suffixes (especially for erased
callables) as "whatever works" is a right thing, because it reflects the
whole idea of why we normally check subtyping with respect to an e.g.
erased type. As you can see this fixes a bunch of issues. Note it was
necessary to make couple more tweaks to make everything work smoothly:
* Adjust self-type erasure level in `checker.py` to match other places.
* Explicitly allow `Callable` as a `self`/`cls` annotation (actually I
am not sure we need to keep this check at all, since we now have good
inference for self-types, and we check they are safe either at
definition site or at call site).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug mypy got something wrong topic-paramspec PEP 612, ParamSpec, Concatenate
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants