Combining UserDict and TypedDict

[lubo]

Hi. I'm adding strict typing to my project. I've been using this:

class DefaultsDict(collections.UserDict):
    def __init__(self, *args, defaults, **kwargs):
        super().__init__(*args, **kwargs)

        self.defaults = defaults

    def __missing__(self, key):
        return self.defaults[key]

I use DefaultsDict(defaults=foo) and I pass it instead of foo to places where I don't want foo to change and I want to store the changes in a new dictionary. Plus, it allows me to access the original dict via defaults attribute. Now I want to use TypedDict as the defaults and I want DefaultsDict to inherit the types from my TypedDict, so that I can keep using them interchangeably.

Any idea how to do this?

[Daverball]

This is not possible, the only type you can use with a TypedDict is dict or more specifically a dict literal, since otherwise you lose the types of the individual key value pairs, you cannot even use subclasses of dict currently, although PEP 728 may carve out some exceptions for a uniform closed TypedDict. This wouldn't help with your use-case though.

[Daverball]

There is an Intersection PEP that's currently being drafted, that may allow something like this using __new__ though and returning an intersection type, rather than just Self. Intersections with TypedDict are supposed to help with properly typing things like pandas dataclasses.

[Daverball]

Another thing that may make this possible are the proposed KeyType and ValueType modifiers, which would allow you to define a generic class that reuses the KeyType and/or ValueType of a TypeVar bound to a TypedDict.

[Daverball]

But only the intersection case has any chance of being compatible in functions that expect a certain TypedDict to be passed in and even there I wouldn't count on it.

[Daverball]

I thought of one way you could make this work for a specific TypedDict with a little trickery:

if TYPE_CHECKING:
    class Foo(TypedDict):
        a: int
        b: str
else:
   Foo = DefaultsDict

def takes_foo(x: Foo): ...

foo = Foo({'a': 5, 'b': ''})
foo['a'] = 6  # ok
takes_foo(foo)  # also ok

but then the type checker would not be aware of the defaults attribute, so you would not be able to access it without ignoring a type error.

[lubo]

So, until intersection types are available, I guess I'm gonna stick to something like this:

Click to expand

from collections import UserDict
from collections.abc import Mapping
from typing import Any, cast, Generic, TypedDict, TypeVar

_KT = TypeVar("_KT")
_VT = TypeVar("_VT")


class DefaultsDict(UserDict[_KT, _VT]):
    def __init__(
        self,
        *args: Any,
        defaults: Mapping[_KT, _VT],
        **kwargs: Any,
    ) -> None:
        super().__init__(*args, **kwargs)

        self.defaults = defaults

    def __missing__(self, key: _KT) -> _VT:
        return self.defaults[key]


_TDKT = str
_TDVT = object
_TDT = TypeVar("_TDT", bound=Mapping[_TDKT, _TDVT])


class TypedDefaultsDict(DefaultsDict[_TDKT, _TDVT], Generic[_TDT]):
    data: _TDT  # type: ignore[assignment]
    defaults: _TDT

    def __init__(
        self,
        *args: Any,
        defaults: _TDT,
        **kwargs: Any,
    ) -> None:
        super().__init__(*args, defaults=defaults, **kwargs)

    def to_typeddict(self) -> _TDT:
        return cast(_TDT, self)


class Foo(TypedDict):
    ...


def use_foo(foo: Foo):
    ...


foo: Foo = {...}
foo_with_defaults = TypedDefaultsDict[Foo](defaults=foo)

use_foo(foo)
use_foo(foo_with_defaults.to_typeddict())

Foo and TypedDefaultsDict[Foo] are still not fully interchangeable, but at least TypedDefaultsDict now carries the info about the underlying TypedDict type and data and defaults attributes are typed correctly.