Support generic NamedTuples

[ghost]

I tried this for a generic NamedTuple:

S = TypeVar("S")
T = TypeVar("T")

class Pair(Generic[S, T],
    NamedTuple("Pair",[
        ('fst', S),
        ('second', T)])):
    def show(self):
        print(self.fst, self.second)

p = Pair("ab", 22)  # type: Pair[str, int]

Mypy raised an exception: RuntimeError: TypeVarType is already analysed.

Related question: is this the right way to define a generic NamedTuple?

Thanks!

[spkersten]

I think you're seeing the same error as Jukka in python/typing#279.

On 18 mei 2015, at 04:42, Dedoig notifications@github.com wrote:

I tried this for a generic NamedTuple:

S = TypeVar("S")
T = TypeVar("T")

class Pair(Generic[S, T],
NamedTuple("Pair",[
('fst', S),
('second', T)])):
def show(self):
print(self.fst, self.second)

p = Pair("ab", 22) # type: Pair[str, int]
Mypy raised an exception: RuntimeError: TypeVarType is already analysed.

Related question: is this the right way to define a generic NamedTuple?

Thanks!

—
Reply to this email directly or view it on GitHub.

[JukkaL]

The code should work, but I think that the NamedTuple implementation in mypy currently doesn't support generic named tuples (there are also a few other things I left out when implementing named tuples).

I don't think it would be a lot of work to implement, but anything involving generics is at least a little tricky.

[gvanrossum]

Yeah, still reproducible. Here's a simpler repro:

from typing import *
T = TypeVar('T')
class P(Generic[T],
        NamedTuple('P', [('field', T)])):
    pass
p = P("")  # type: P[str]

Output:

x.py:4: error: 'T' is a type variable and only valid in type context
x.py:6: error: Generic tuple types not supported

[gvanrossum]

But it's no longer an exception in mypy, it just prints an error.

[JukkaL]

Here is a rough idea how it might be possible to implement generic named tuples (note that I haven't spend a lot of time on this so this might all make no sense):

• Start with a generic class that inherits from Tuple[...]. Named tuple should just be a generalization of that.
• The representation of the type would still be TupleType, similar to current named tuples. It would be best to not make TupleTupe itself generic, since it would likely complicate the implementation a lot. Instead, maybe just have the TupleType keep the substituted tuple item types. The fallback would be a generic Instance.
• The generic bit would be handled by a mostly ordinary generic TypeInfo that would also have the tuple_type attribute set. tuple_type could contain class type variables and these would need to be substituted when creating a corresponding TupleType.
• Maybe require all type variable to be covariant in generic named tuples so that covariant tuple subtyping is preserved. Not sure if this is necessary.

[JelleZijlstra]

I have a partially working implementation of generic namedtuples on top of #3081; I can work more on it once that one is merged. My general approach is just to make namedtuples more and more like normal classes.

[elazarg]

#3134 too has some support for generic namedtuple falling out of the implementation.

@JelleZijlstra #3081 is indeed conflicting. The general idea is similar, except I do not make the namedtuple look like a class, but rather change it to manipulate the TypeInfo of the previously-constructed class (the way metaclasses work). It also reduces some of the existing code duplication, and avoids messing with AST nodes in the analysis.

(I did not mean to overlap with your work; it happened through the course of refactoring. I think I should try to find a way for these PRs to live together. But I guess #3134 will have hard time to merge anyway due to code churn).

[ilevkivskyi]

#1682 provides an additional repro that should be added to tests when this is fixed.

[mitar]

Any progress on this?

[ilevkivskyi]

Raising priority to normal, since this appeared several times (still not high priority, since the fix is non-trivial).

[wkschwartz]

Another spelling that would be nice if it worked is

from typing import Generic, NamedTuple, TypeVar
T = TypeVar('T')
class Result(Generic[T], NamedTuple):
	result: T
	other_data: str
result: Result[int] = Result(result=1, other_data='a')

As of Mypy version 0.711, this gives the following error:

mypy-test.py:6: error: Generic tuple types not supported
mypy-test.py:6: error: Argument "result" to "Result" has incompatible type "int"; expected "T"

[Phlogistique]

As a result of this bug, the type annotation for concurrent.futures.wait in the Typeshed is broken; it should be a DoneAndNotDoneFutures[_T] named tuple, but this type can not currently be used with mypy. See python/typeshed#3380.

[mortoray]

I'd like to see this working. Presently I want this code:

RefT = TypeVar('RefT')
class Reference(NamedTuple, Generic[RefT]):
	ref: RefT
	where: List[str]

But need to use a dataclass instead, which unfortunately is mutable and can't be deconstructed like a tuple.

@dataclass
class Reference(Generic[RefT]):
	ref: RefT
	where: List[str]

[DetachHead]

this seems to also be an issue when attempting to extend tuple.

from typing import TypeVar
T = TypeVar("T")


class Foo(tuple[T, T]): pass

a: Foo[int] # error: Generic tuple types not supported  [misc]

is this intended behavior? i don't see why this shouldn't be allowed

[CarliJoy]

this seems to also be an issue when attempting to extend tuple.

from typing import TypeVar
T = TypeVar("T")


class Foo(tuple[T, T]): pass

a: Foo[int] # error: Generic tuple types not supported  [misc]

is this intended behavior? i don't see why this shouldn't be allowed

Have the same Problem like @DetachHead is it as compilcated to fix as for NamedTuples? Is it the same error?

[AlexWaygood]

To update on this issue: generic NamedTuples are now supported by the runtime in 3.11, and python/typing_extensions#7 tracks the progress of a backport via typing_extensions. Mypy support for the feature is still missing, however.