Crash on definition of TypedDict method

[dfee]

I'm not sure if this is a bug or a feature, but I was considering migrating from NamedTuple to TypedDict, and ran into a problem where methods don't seem to be supported on TypedDicts.

class User(TypedDict):
    id: str
    firstName: str
    lastName: str
    password: str

    # error:Invalid statement in TypedDict definition; expected "field_name: field_type"
    @classmethod
    def from_row(cls, row: RowProxy) -> typing.Optional["User"]:
        if not row:
            return None
        return User(id=row.id, firstName=row.first_name, lastName=row.last_name)

Despite the error, the code runs fine. So, I'm looking for clarification that this is / isn't a bug, and maybe some reasoning about

[dfee]

Also, when used as a var-keyword param, TypedDict doesn't work as I'd expect it to:

class LoginParams(TypedDict):
    emailAddress: str
    password: str

async def resolve_login(
    root: None, info: GraphQLResolveInfo, ctx: Context, **params: LoginParams
) -> typing.Optional[UserType]:
    user = get_user_by_email_address(params['not_emailAddress']) # does not warn

I noticed there is a dedicated type KwArg but that's unfortunate as I can't rely on my var-keyword params to always share the same type.

[erickpeirson]

@dfee Looks like it's explicitly proscribed in PEP 589:

Methods are not allowed, since the runtime type of a TypedDict object will always be just dict (it is never a subclass of dict).

Doesn't really address the rationale. I couldn't find any discussion about this (or much else, for that matter) in the PR for PEP589 or the typing-sig mailing list. Maybe someone else knows the story?

[ilevkivskyi]

@dfee @erickpeirson Only class methods accessed on class object work, normal methods will not work at runtime. We can of course allow specifically this case, but this is quite niche and therefore low-priority.

(The ** problem is a separate one, there is an existing issue about this.)

[AlexWaygood]

The crash reproduces on mypy 0.950 with this code snippet:

from typing import TypedDict

class RowProxy:
    first_name: str
    last_name: str

class User(TypedDict, total=False):
    firstName: str
    lastName: str

    @classmethod
    def from_row(cls, row: RowProxy) -> "User":
        return User(firstName=row.first_name, lastName=row.last_name)

mypy traceback:

main.py:11: error: Invalid statement in TypedDict definition; expected "field_name: field_type"
Traceback (most recent call last):
  File "/opt/python3.9/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/python3.9/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/mypy/__main__.py", line 34, in <module>
    console_entry()
  File "mypy/build.py", line 1976, in wrap_context
  File "mypy/build.py", line 2226, in finish_passes
  File "mypy/build.py", line 2249, in _patch_indirect_dependencies
  File "mypy/types.py", line 1722, in __hash__
  File "mypy/types.py", line 1563, in is_type_obj
  File "mypy/nodes.py", line 2782, in is_metaclass
  File "mypy/nodes.py", line 2790, in has_base
AttributeError: attribute 'TypeInfo' of 'mro' undefined
main.py: error: INTERNAL ERROR -- Please try using mypy master on Github:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 0.950
main.py: : note: use --pdb to drop into pdb

[JelleZijlstra]

There are really two issues here:

• mypy crashes when it sees a TypedDict method. This is bad and should be fixed.
• mypy doesn't support TypedDict classmethods, even though it could. This is a low-priority feature request and probably should be made on https://github.com/python/typing instead.

[AlexWaygood]

The crash has existed since mypy 0.710, which is the earliest version of mypy that recognises typing_extensions.TypedDict: https://mypy-play.net/?mypy=0.710&python=3.10&gist=5ccd458a05e7de64b25e37f04b276d03

[AlexWaygood]

Interesting, it looks like the traceback on master is slightly different:

(venv) C:\Users\alexw\coding\mypy>mypy crash.py --show-traceback
crash.py:11: error: Invalid statement in TypedDict definition; expected "field_name: field_type"
crash.py: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 0.960+dev.4f07c79aea0fef61ab649d6acedf01186f1054eb
Traceback (most recent call last):
  File "C:\Users\alexw\coding\mypy\venv\Scripts\mypy-script.py", line 33, in <module>
    sys.exit(load_entry_point('mypy', 'console_scripts', 'mypy')())
  File "c:\users\alexw\coding\mypy\mypy\__main__.py", line 12, in console_entry
    main(None, sys.stdout, sys.stderr)
  File "c:\users\alexw\coding\mypy\mypy\main.py", line 96, in main
    res, messages, blockers = run_build(sources, options, fscache, t0, stdout, stderr)
  File "c:\users\alexw\coding\mypy\mypy\main.py", line 173, in run_build
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "c:\users\alexw\coding\mypy\mypy\build.py", line 181, in build
    result = _build(
  File "c:\users\alexw\coding\mypy\mypy\build.py", line 257, in _build
    graph = dispatch(sources, manager, stdout)
  File "c:\users\alexw\coding\mypy\mypy\build.py", line 2752, in dispatch
    process_graph(graph, manager)
  File "c:\users\alexw\coding\mypy\mypy\build.py", line 3110, in process_graph
    process_stale_scc(graph, scc, manager)
  File "c:\users\alexw\coding\mypy\mypy\build.py", line 3211, in process_stale_scc
    graph[id].finish_passes()
  File "c:\users\alexw\coding\mypy\mypy\build.py", line 2236, in finish_passes
    with self.wrap_context():
  File "C:\Users\alexw\AppData\Local\Programs\Python\Python310\lib\contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "c:\users\alexw\coding\mypy\mypy\build.py", line 1981, in wrap_context
    yield
  File "c:\users\alexw\coding\mypy\mypy\build.py", line 2244, in finish_passes
    self._patch_indirect_dependencies(self.type_checker().module_refs, self.type_map())
  File "c:\users\alexw\coding\mypy\mypy\build.py", line 2268, in _patch_indirect_dependencies
    types = set(type_map.values())
  File "c:\users\alexw\coding\mypy\mypy\types.py", line 1769, in __hash__
    return hash((self.ret_type, self.is_type_obj(),
  File "c:\users\alexw\coding\mypy\mypy\types.py", line 1610, in is_type_obj    return self.fallback.type.is_metaclass()
  File "c:\users\alexw\coding\mypy\mypy\nodes.py", line 2985, in __getattribute__
    raise AssertionError(object.__getattribute__(self, 'msg'))
AssertionError: fallback can't be filled out until semanal
crash.py: : note: use --pdb to drop into pdb

[AlexWaygood]

The classmethod is represented internally in mypy as a mypy.types.CallableType instance.

In mypy.build.State._patch_indirect_dependencies(), the function call set(type_map.values()) fails due to this particular instance of CallableType being one of the .values(), but also being unhashable. The reason why this instance of CallableType is unhashable is because CallableType.__hash__ calls self.is_type_obj(), and CallableType.is_type_obj() assumes that self.fallback.type is not a FakeInfo. But this particular instance of CallableType has a FakeInfo as self.fallback.type.

I can see three options here:

1. Change CallableType.__hash__ so that it doesn't call self.is_type_obj() if self.fallback.type is an instance of FakeInfo.
2. Change CallableType.is_type_obj so that it doesn't crash if self.fallback.type is an instance of FakeInfo.
3. Something else that I've missed.

I'm inclined towards Option (1), because I feel like something unusual is almost certainly going on if self.is_type_obj() is being called, despite self.fallback.type being a FakeInfo. We should probably know if that's happening, rather than letting it pass silently; CallableType.__hash__, on the other hand, should probably never fail, even if something weird's going on. Here's a proof of concept: AlexWaygood@68bf1c6. Happy to turn it into a PR if it looks viable, but there might be something I'm missing.

[AlexWaygood]

We've now fixed the crash. TypedDict classmethods are still unsupported by mypy, but as @JelleZijlstra says, that kind of feature request should probably be made at the https://github.com/python/typing/issues issue tracker rather than here, as it goes against the current specs and would affect the typing system as a whole rather than just mypy.