bpo-42345: Fix three issues with typing.Literal parameters #23294

uriyyo · 2020-11-15T13:25:30Z

Literal equality no longer depends on the order of arguments.

Fix issue related to typing.Literal caching by adding typed parameter to typing._tp_cache function.

Add deduplication of typing.Literal arguments.

https://bugs.python.org/issue42345

Automerge-Triggered-By: GH:gvanrossum

Fidget-Spinner

Hello, thank you for your PR to cpython! This looks very good to me, just one thing about de-duplicating literals pointed out in the review below.

Lib/typing.py

Fidget-Spinner

I still think that Literal should de-duplicate its __args__ similar to Union, although I now see the set approach won't work. Maybe you can call set on the _value_and_type_iter function, then extract out the parameters again, something like:

def Literal:
    ...
    parameters = [p for p, _ in set(_value_and_type_iter(parameters))]
    return _LiteralGenericAlias(self, parameters)

This probably needs more discussion, and Guido/Ivan's decision on what the expected behavior should be. Maybe we can ask on the bpo issue.

Lib/test/test_typing.py

uriyyo · 2020-11-15T16:01:27Z

@Fidget-Spinner There is a problem with Literal caching.

Code like this will fail because of type caching:

>>> Literal[0] == Literal[False]
False

Code below will show the root cause of this problem:

>>> (Literal[0], Literal[False])
(Litera[0], Literal[0])

As a solution, we can disable caching for a Literal.

What do you think about that?

Fidget-Spinner · 2020-11-15T16:21:04Z

Code like this will fail because of type caching:

One possible workaround for that is to pass typed=True for lru_cache in tp_cache, then it will differentiate 0 and False.

The only problem is I realized that set will still fail for passing in dict Literal.

A possible workaround is to just loop through them and check the other args one by one (not particularly efficient but I don't expect Literals to be too long, and with cache it should offset some of it), something like what _remove_dups_flatten for Union does, but instead you can't use set due to non-hashable args. Something like:

# more code here to keep track and store the unique elements
...
for index, i_elem in enumerate(parameters):
    is_duplicate = False
    for j_elem in parameters[index:]:
        if i_elem == j_elem:
            is_duplicate = True
            break
    ...
    # more code for logic to handle the unique elements

I think before we progress further, we should wait for a core dev to respond on the bpo. They might think just having comparisons work is sufficient, and that de-duplicating args isn't needed. So let's save our effort for now 😉 .

Edit:
I just remembered that _tp_cache will just default to non-cache behavior for dict literals, so it's fine to just leave it mostly untouched.

uriyyo · 2020-11-15T16:56:55Z

@Fidget-Spinner I forget about lru_cache typed parameter, thanks for the reminder

I totally agree with you, let wait for a decision from Guido and Ivan regarding Literal args de-duplicating.

As for me, we can skip args de-duplication because Literal equals method will convert everything to set so duplicates won't be a problem.

gvanrossum · 2020-11-15T17:07:07Z

Please change the title and news text. We don’t use the style that describes the new state, we describe the action of the PR as a change to the behavior. For examples, just look at the commit log.

uriyyo · 2020-11-15T18:00:30Z

I have updated the title and news.

Sorry for that, I didn't notice this convention. My bad.

gvanrossum

LG, except for one nit.

You might mention in the description (not the title) and the news file that you also fixed an issue where incorrectly Literal[0] == Literal[False] -- good catch, that!

Lib/typing.py

uriyyo · 2020-11-16T09:44:42Z

@gvanrossum I have seen your message regarding Literal args deduplication, so I have implemented it in the scope of this PR.

Fidget-Spinner

Getting very close :). Thank you for your patience.

Lib/typing.py

Fidget-Spinner

LGTM, and sorry for the weird tangents I went off on in some places!

uriyyo · 2020-11-16T16:28:08Z

@Fidget-Spinner thanks for the review and all comments 🙂

What will be the next step regarding this PR?

Fidget-Spinner · 2020-11-16T16:31:59Z

@Fidget-Spinner thanks for the review and all comments 🙂

What will be the next step regarding this PR?

@uriyyo A core dev like Guido/Ivan will take a look, once they're satisfied and approve it, it will be merged into 3.10. And probably backported to 3.9 and 3.8 too (done automatically via bots). This may take some time depending on how busy the core devs are.

I'm looking forward to see your first commit to cpython :).

gvanrossum · 2020-11-16T18:42:12Z

I will look at it some time this week.

Lib/typing.py

Dominik1123 · 2020-11-16T23:04:53Z

Should the docs of typing.get_args be updated to mention Literal as well (similar to #23254)?

uriyyo · 2020-11-17T07:54:14Z

@gvanrossum Looks like I did it🙂

…23294) (GH-23335) Literal equality no longer depends on the order of arguments. Fix issue related to `typing.Literal` caching by adding `typed` parameter to `typing._tp_cache` function. Add deduplication of `typing.Literal` arguments. (cherry picked from commit f03d318) Co-authored-by: Yurii Karabas <1998uriyyo@gmail.com>

Dominik1123 · 2020-11-17T17:17:43Z

Should the docs of typing.get_args be updated to mention Literal as well (similar to #23254)?

Sure, but I would make a minimal change, e.g. changing "Union" in that paragraph to "Union or Literal".

@uriyyo What about updating the docs of typing.get_args?

Fidget-Spinner · 2020-11-18T15:38:28Z

Should the docs of typing.get_args be updated to mention Literal as well (similar to #23254)?

Sure, but I would make a minimal change, e.g. changing "Union" in that paragraph to "Union or Literal".

@uriyyo What about updating the docs of typing.get_args?

@Dominik1123, not sure if this is a bug, but hash() of Literal is dependent on args order in the current code, so the type caching problem won't happen. Currently, it's still only Union affected.

>>> hash(Literal[1, 2, 3])
3136223724409791019
>>> hash(Literal[3, 2, 1])
4438910888190981000

uriyyo · 2020-11-18T15:44:16Z

@Fidget-Spinner Looks like we should not update docs because Literal can't be parametrized with generics so this problem will not exist for Literal. Please correct me if I miss smth.

gvanrossum · 2020-11-18T15:47:53Z

But Literal can occur inside other types, e.g. Tuple[str, str, Literal[0, 1, 2]].

Fidget-Spinner · 2020-11-18T15:48:40Z

@uriyyo yeah, sorry if I wasn't clear in my original statement - I was in support of not updating the docs. Although my reasoning is slightly different. It's not about generics, the behavior Dominik mentioned happens when Union's nested in anything. The real reason why we don't need a docs update is because the __hash__ of Literal in your patch is order-dependent.

Edit:
Eg:

# Union with __args__ not preserving order when nested due to type caching.
>>> Tuple[Union[int, str]].__args__
(typing.Union[int, str],)
>>> Tuple[Union[str, int]].__args__
(typing.Union[int, str],)

# Literal doesn't have that issue
>>> Tuple[Literal[1, 2]].__args__
(typing.Literal[1, 2],)
>>> Tuple[Literal[2, 1]].__args__
(typing.Literal[2, 1],)

uriyyo · 2020-11-18T15:55:50Z

@Fidget-Spinner Thanks for the explanation it's clear to me now.

Fidget-Spinner · 2020-11-18T16:05:05Z

@uriyyo Awesome! I'm glad I was able to help :).

Oops, forgot to say this: Congrats to your first commit to CPython!

Dominik1123 · 2020-11-18T18:51:07Z

Should the docs of typing.get_args be updated to mention Literal as well (similar to #23254)?

Sure, but I would make a minimal change, e.g. changing "Union" in that paragraph to "Union or Literal".

@uriyyo What about updating the docs of typing.get_args?

@Dominik1123, not sure if this is a bug, but hash() of Literal is dependent on args order in the current code, so the type caching problem won't happen. Currently, it's still only Union affected.
>>> hash(Literal[1, 2, 3])
3136223724409791019
>>> hash(Literal[3, 2, 1])
4438910888190981000

@Fidget-Spinner @uriyyo I see, I didn't notice before. But isn't this very unusual in terms of behavior? So we can have two objects that compare equal but their hashes are different. AFAIK nowhere else in Python this is the case. The closest that comes to it in terms "being equal, but not having the same hash" is set and frozenset but in that case at least set will raise. Now for Literal it's yet another level, since the hashes are actually computed, and it's even the same type.

To quote the Python data model on __hash__:

The only required property is that objects which compare equal have the same hash value;

So according to that it's a bug and the hash should not be order dependent.

uriyyo · 2020-11-18T19:04:55Z

@Dominik1123 Agree with you, looks like it's a bug related to Literal hash implementation, basically, we should change implementation from:

    def __hash__(self):
        return hash(tuple(_value_and_type_iter(self.__args__)))

to:

    def __hash__(self):
        return hash(frozentset(_value_and_type_iter(self.__args__)))

@Fidget-Spinner should we create issue regarding this bug?

Dominik1123 · 2020-11-18T19:26:21Z

@Dominik1123 Agree with you, looks like it's a bug related to Literal hash implementation, basically, we should change implementation from:
    def __hash__(self):
        return hash(tuple(_value_and_type_iter(self.__args__)))
to:
    def __hash__(self):
        return hash(frozentset(_value_and_type_iter(self.__args__)))

I'm not sure if frozenset is too restrictive in terms of the values it can work with. Right now PEP 586 disallows literals of mutable types, but only at type check time. The PEP also contains the following comment:

[...] the actual implementation of typing.Literal will not perform any checks at runtime. [...] This is partly to help us preserve flexibility in case we want to expand the scope of what Literal can be used for in the future [...]

Using frozenset would take away on that flexibility regarding literals of mutable types. Not sure if that's relevant though. I think we should wait for @gvanrossum to comment on this.

gvanrossum · 2020-11-18T23:23:09Z

Whoa, this is a bug. It not allowed to have two objects that compare equal but have a different hash. So since Literal[1, 2] == Literal[2, 1], those two should have the same hash. I'm sorry I didn't catch this!

Using set() or frozenset() here is totally fine, we already use those in __eq__. I'd use set().

uriyyo · 2020-11-18T23:27:06Z

@gvanrossum Should we create a separate issue or just open a new PR that will point to the same issue as this PR?

gvanrossum · 2020-11-18T23:35:31Z

Create a new PR pointing to the same issue -- I already reopened it.

Fix hash implementation of `typing.Literal`. Update docs regarding `typing.Litaral` caching. Base implementation was done in PR #23294.

Fix hash implementation of `typing.Literal`. Update docs regarding `typing.Litaral` caching. Base implementation was done in PR pythonGH-23294. (cherry picked from commit 1b54077) Co-authored-by: Yurii Karabas <1998uriyyo@gmail.com>

Fix hash implementation of `typing.Literal`. Update docs regarding `typing.Litaral` caching. Base implementation was done in PR GH-23294. (cherry picked from commit 1b54077) Co-authored-by: Yurii Karabas <1998uriyyo@gmail.com>

…23294) Literal equality no longer depends on the order of arguments. Fix issue related to `typing.Literal` caching by adding `typed` parameter to `typing._tp_cache` function. Add deduplication of `typing.Literal` arguments.

Fix hash implementation of `typing.Literal`. Update docs regarding `typing.Litaral` caching. Base implementation was done in PR python#23294.

@deprecated

Changelog: ============ - Fix use of @deprecated on classes with __new__ but no __init__. - Fix regression in version 4.6.1 where comparing a generic class against a runtime-checkable protocol using isinstance() would cause AttributeError to be raised if using Python 3.7. - Change deprecated @runtime to formal API @runtime_checkable in the error message. - Fix regression in 4.6.0 where attempting to define a Protocol that was generic over a ParamSpec or a TypeVarTuple would cause TypeError to be raised. - typing_extensions is now documented at https://typing-extensions.readthedocs.io/en/latest/. - Add typing_extensions.Buffer, a marker class for buffer types, as proposed by PEP 688. Equivalent to collections.abc.Buffer in Python 3.12. - Backport two CPython PRs fixing various issues with typing.Literal: python/cpython#23294 and python/cpython#23383. Both CPython PRs were originally, and both were backported to Python >=3.9.1, but no earlier. - A side effect of one of the changes is that equality comparisons of Literal objects will now raise a TypeError if one of the Literal objects being compared has a mutable parameter. (Using mutable parameters with Literal is not supported by PEP 586 or by any major static type checkers.) - Literal is now reimplemented on all Python versions <= 3.10.0. - Backport CPython PR 26067, ensuring that isinstance() calls on protocols raise TypeError when the protocol is not decorated with @runtime_checkable. - Backport several significant performance improvements to runtime-checkable protocols that have been made in Python 3.12 (see python/cpython#74690 for details). - A side effect of one of the performance improvements is that the members of a runtime-checkable protocol are now considered "frozen" at runtime as soon as the class has been created. Monkey-patching attributes onto a runtime-checkable protocol will still work, but will have no impact on isinstance() checks comparing objects to the protocol. See "What's New in Python 3.12" for more details. - isinstance() checks against runtime-checkable protocols now use inspect.getattr_static() rather than hasattr() to lookup whether attributes exist (backporting python/cpython#103034). - Backport the ability to define __init__ methods on Protocol classes, a change made in Python 3.11 (originally implemented in python/cpython#31628 - Speedup isinstance(3, typing_extensions.SupportsIndex) by >10x on Python <3.12. - Add typing_extensions versions of SupportsInt, SupportsFloat, SupportsComplex, SupportsBytes, SupportsAbs and SupportsRound. These have the same semantics as the versions from the typing module, but isinstance() checks against the typing_extensions versions are >10x faster at runtime on Python <3.12. - Add __orig_bases__ to non-generic TypedDicts, call-based TypedDicts, and call-based NamedTuples. - Add typing_extensions.get_original_bases, a backport of types.get_original_bases, introduced in Python 3.12 (CPython PR python/cpython#101827, originally - This function should always produce correct results when called on classes constructed using features from typing_extensions. - Constructing a call-based TypedDict using keyword arguments for the fields now causes a DeprecationWarning to be emitted. This matches the behaviour of typing.TypedDict on 3.11 and 3.12. - Backport the implementation of NewType from 3.10 (where it is implemented as a class rather than a function). This allows user-defined NewTypes to be pickled. - Fix tests and import on Python 3.12, where typing.TypeVar can no longer be subclassed. - Add typing_extensions.TypeAliasType, a backport of typing.TypeAliasType from PEP 695. - Backport changes to the repr of typing.Unpack that were made in order to implement PEP 692 (backport of python/cpython#104048). (From OE-Core rev: 2b1d07c7deb4f0247765bc737fb11a1747143edf) Signed-off-by: Wang Mingyu <wangmy@fujitsu.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>

@deprecated

Changelog: ============ - Fix use of @deprecated on classes with __new__ but no __init__. - Fix regression in version 4.6.1 where comparing a generic class against a runtime-checkable protocol using isinstance() would cause AttributeError to be raised if using Python 3.7. - Change deprecated @runtime to formal API @runtime_checkable in the error message. - Fix regression in 4.6.0 where attempting to define a Protocol that was generic over a ParamSpec or a TypeVarTuple would cause TypeError to be raised. - typing_extensions is now documented at https://typing-extensions.readthedocs.io/en/latest/. - Add typing_extensions.Buffer, a marker class for buffer types, as proposed by PEP 688. Equivalent to collections.abc.Buffer in Python 3.12. - Backport two CPython PRs fixing various issues with typing.Literal: python/cpython#23294 and python/cpython#23383. Both CPython PRs were originally, and both were backported to Python >=3.9.1, but no earlier. - A side effect of one of the changes is that equality comparisons of Literal objects will now raise a TypeError if one of the Literal objects being compared has a mutable parameter. (Using mutable parameters with Literal is not supported by PEP 586 or by any major static type checkers.) - Literal is now reimplemented on all Python versions <= 3.10.0. - Backport CPython PR 26067, ensuring that isinstance() calls on protocols raise TypeError when the protocol is not decorated with @runtime_checkable. - Backport several significant performance improvements to runtime-checkable protocols that have been made in Python 3.12 (see python/cpython#74690 for details). - A side effect of one of the performance improvements is that the members of a runtime-checkable protocol are now considered "frozen" at runtime as soon as the class has been created. Monkey-patching attributes onto a runtime-checkable protocol will still work, but will have no impact on isinstance() checks comparing objects to the protocol. See "What's New in Python 3.12" for more details. - isinstance() checks against runtime-checkable protocols now use inspect.getattr_static() rather than hasattr() to lookup whether attributes exist (backporting python/cpython#103034). - Backport the ability to define __init__ methods on Protocol classes, a change made in Python 3.11 (originally implemented in python/cpython#31628 - Speedup isinstance(3, typing_extensions.SupportsIndex) by >10x on Python <3.12. - Add typing_extensions versions of SupportsInt, SupportsFloat, SupportsComplex, SupportsBytes, SupportsAbs and SupportsRound. These have the same semantics as the versions from the typing module, but isinstance() checks against the typing_extensions versions are >10x faster at runtime on Python <3.12. - Add __orig_bases__ to non-generic TypedDicts, call-based TypedDicts, and call-based NamedTuples. - Add typing_extensions.get_original_bases, a backport of types.get_original_bases, introduced in Python 3.12 (CPython PR python/cpython#101827, originally - This function should always produce correct results when called on classes constructed using features from typing_extensions. - Constructing a call-based TypedDict using keyword arguments for the fields now causes a DeprecationWarning to be emitted. This matches the behaviour of typing.TypedDict on 3.11 and 3.12. - Backport the implementation of NewType from 3.10 (where it is implemented as a class rather than a function). This allows user-defined NewTypes to be pickled. - Fix tests and import on Python 3.12, where typing.TypeVar can no longer be subclassed. - Add typing_extensions.TypeAliasType, a backport of typing.TypeAliasType from PEP 695. - Backport changes to the repr of typing.Unpack that were made in order to implement PEP 692 (backport of python/cpython#104048). (From OE-Core rev: a37154b9166323d05cca970ebb37bee0d5250893) Signed-off-by: Wang Mingyu <wangmy@fujitsu.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

@deprecated

Changelog: ============ - Fix use of @deprecated on classes with __new__ but no __init__. - Fix regression in version 4.6.1 where comparing a generic class against a runtime-checkable protocol using isinstance() would cause AttributeError to be raised if using Python 3.7. - Change deprecated @runtime to formal API @runtime_checkable in the error message. - Fix regression in 4.6.0 where attempting to define a Protocol that was generic over a ParamSpec or a TypeVarTuple would cause TypeError to be raised. - typing_extensions is now documented at https://typing-extensions.readthedocs.io/en/latest/. - Add typing_extensions.Buffer, a marker class for buffer types, as proposed by PEP 688. Equivalent to collections.abc.Buffer in Python 3.12. - Backport two CPython PRs fixing various issues with typing.Literal: python/cpython#23294 and python/cpython#23383. Both CPython PRs were originally, and both were backported to Python >=3.9.1, but no earlier. - A side effect of one of the changes is that equality comparisons of Literal objects will now raise a TypeError if one of the Literal objects being compared has a mutable parameter. (Using mutable parameters with Literal is not supported by PEP 586 or by any major static type checkers.) - Literal is now reimplemented on all Python versions <= 3.10.0. - Backport CPython PR 26067, ensuring that isinstance() calls on protocols raise TypeError when the protocol is not decorated with @runtime_checkable. - Backport several significant performance improvements to runtime-checkable protocols that have been made in Python 3.12 (see python/cpython#74690 for details). - A side effect of one of the performance improvements is that the members of a runtime-checkable protocol are now considered "frozen" at runtime as soon as the class has been created. Monkey-patching attributes onto a runtime-checkable protocol will still work, but will have no impact on isinstance() checks comparing objects to the protocol. See "What's New in Python 3.12" for more details. - isinstance() checks against runtime-checkable protocols now use inspect.getattr_static() rather than hasattr() to lookup whether attributes exist (backporting python/cpython#103034). - Backport the ability to define __init__ methods on Protocol classes, a change made in Python 3.11 (originally implemented in python/cpython#31628 - Speedup isinstance(3, typing_extensions.SupportsIndex) by >10x on Python <3.12. - Add typing_extensions versions of SupportsInt, SupportsFloat, SupportsComplex, SupportsBytes, SupportsAbs and SupportsRound. These have the same semantics as the versions from the typing module, but isinstance() checks against the typing_extensions versions are >10x faster at runtime on Python <3.12. - Add __orig_bases__ to non-generic TypedDicts, call-based TypedDicts, and call-based NamedTuples. - Add typing_extensions.get_original_bases, a backport of types.get_original_bases, introduced in Python 3.12 (CPython PR python/cpython#101827, originally - This function should always produce correct results when called on classes constructed using features from typing_extensions. - Constructing a call-based TypedDict using keyword arguments for the fields now causes a DeprecationWarning to be emitted. This matches the behaviour of typing.TypedDict on 3.11 and 3.12. - Backport the implementation of NewType from 3.10 (where it is implemented as a class rather than a function). This allows user-defined NewTypes to be pickled. - Fix tests and import on Python 3.12, where typing.TypeVar can no longer be subclassed. - Add typing_extensions.TypeAliasType, a backport of typing.TypeAliasType from PEP 695. - Backport changes to the repr of typing.Unpack that were made in order to implement PEP 692 (backport of python/cpython#104048). Signed-off-by: Wang Mingyu <wangmy@fujitsu.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [typing-extensions](https://github.com/python/typing_extensions) ([changelog](https://github.com/python/typing_extensions/blob/main/CHANGELOG.md)) | `==4.5.0` -> `==4.6.3` | [![age](https://badges.renovateapi.com/packages/pypi/typing-extensions/4.6.3/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/pypi/typing-extensions/4.6.3/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/pypi/typing-extensions/4.6.3/compatibility-slim/4.5.0)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/pypi/typing-extensions/4.6.3/confidence-slim/4.5.0)](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>python/typing_extensions</summary> ### [`v4.6.3`](https://github.com/python/typing_extensions/blob/HEAD/CHANGELOG.md#Release-463-June-1-2023) [Compare Source](https://github.com/python/typing_extensions/compare/4.6.2...4.6.3) - Fix a regression introduced in v4.6.0 in the implementation of runtime-checkable protocols. The regression meant that doing `class Foo(X, typing_extensions.Protocol)`, where `X` was a class that had `abc.ABCMeta` as its metaclass, would then cause subsequent `isinstance(1, X)` calls to erroneously raise `TypeError`. Patch by Alex Waygood (backporting the CPython[https://github.com/python/cpython/pull/105152](https://github.com/python/cpython/pull/105152)l/105152). - Sync the repository's LICENSE file with that of CPython. `typing_extensions` is distributed under the same license as CPython itself. - Skip a problematic test on Python 3.12.0b1. The test fails on 3.12.0b1 due to a bug in CPython, which will be fixed in 3.12.0b2. The `typing_extensions` test suite now passes on 3.12.0b1. ### [`v4.6.2`](https://github.com/python/typing_extensions/blob/HEAD/CHANGELOG.md#Release-462-May-25-2023) [Compare Source](https://github.com/python/typing_extensions/compare/4.6.1...4.6.2) - Fix use of `@deprecated` on classes with `__new__` but no `__init__`. Patch by Jelle Zijlstra. - Fix regression in version 4.6.1 where comparing a generic class against a runtime-checkable protocol using `isinstance()` would cause `AttributeError` to be raised if using Python 3.7. ### [`v4.6.1`](https://github.com/python/typing_extensions/blob/HEAD/CHANGELOG.md#Release-461-May-23-2023) [Compare Source](https://github.com/python/typing_extensions/compare/4.6.0...4.6.1) - Change deprecated `@runtime` to formal API `@runtime_checkable` in the error message. Patch by Xuehai Pan. - Fix regression in 4.6.0 where attempting to define a `Protocol` that was generic over a `ParamSpec` or a `TypeVarTuple` would cause `TypeError` to be raised. Patch by Alex Waygood. ### [`v4.6.0`](https://github.com/python/typing_extensions/blob/HEAD/CHANGELOG.md#Release-460-May-22-2023) [Compare Source](https://github.com/python/typing_extensions/compare/4.5.0...4.6.0) - `typing_extensions` is now documented at https://typing-extensions.readthedocs.io/en/latest/. Patch by Jelle Zijlstra. - Add `typing_extensions.Buffer`, a marker class for buffer types, as proposed by PEP 688. Equivalent to `collections.abc.Buffer` in Python 3.12. Patch by Jelle Zijlstra. - Backport two CPython PRs fixing various issues with `typing.Literal`: [https://github.com/python/cpython/pull/23294](https://github.com/python/cpython/pull/23294)3294 [https://github.com/python/cpython/pull/23383](https://github.com/python/cpython/pull/23383)ll/23383. Both CPython PRs were originally by Yurii Karabas, and both were backported to Python >=3.9.1, but no earlier. Patch by Alex Waygood. A side effect of one of the changes is that equality comparisons of `Literal` objects will now raise a `TypeError` if one of the `Literal` objects being compared has a mutable parameter. (Using mutable parameters with `Literal` is not supported by PEP 586 or by any major static type checkers.) - `Literal` is now reimplemented on all Python versions <= 3.10.0. The `typing_extensions` version does not suffer from the bug that was fixed in [https://github.com/python/cpython/pull/29334](https://github.com/python/cpython/pull/29334)9334. (The CPython bugfix was backported to CPython 3.10.1 and 3.9.8, but no earlier.) - Backport [CPython PR 26067](https://github.com/python/cpython/pull/26067) (originally by Yurii Karabas), ensuring that `isinstance()` calls on protocols raise `TypeError` when the protocol is not decorated with `@runtime_checkable`. Patch by Alex Waygood. - Backport several significant performance improvements to runtime-checkable protocols that have been made in Python 3.12 ([https://github.com/python/cpython/issues/74690](https://github.com/python/cpython/issues/74690)es/74690 for details). Patch by Alex Waygood. A side effect of one of the performance improvements is that the members of a runtime-checkable protocol are now considered “frozen” at runtime as soon as the class has been created. Monkey-patching attributes onto a runtime-checkable protocol will still work, but will have no impact on `isinstance()` checks comparing objects to the protocol. See ["What's New in Python 3.12"](https://docs.python.org/3.12/whatsnew/3.12.html#typing) for more details. - `isinstance()` checks against runtime-checkable protocols now use `inspect.getattr_static()` rather than `hasattr()` to lookup whether attributes exist (backport[https://github.com/python/cpython/pull/103034](https://github.com/python/cpython/pull/103034)3034). This means that descriptors and `__getattr__` methods are no longer unexpectedly evaluated during `isinstance()` checks against runtime-checkable protocols. However, it may also mean that some objects which used to be considered instances of a runtime-checkable protocol on older versions of `typing_extensions` may no longer be considered instances of that protocol using the new release, and vice versa. Most users are unlikely to be affected by this change. Patch by Alex Waygood. - Backport the ability to define `__init__` methods on Protocol classes, a change made in Python 3.11 (originally implemented[https://github.com/python/cpython/pull/31628](https://github.com/python/cpython/pull/31628)ll/31628 by Adrian Garcia Badaracco). Patch by Alex Waygood. - Speedup `isinstance(3, typing_extensions.SupportsIndex)` by >10x on Python <3.12. Patch by Alex Waygood. - Add `typing_extensions` versions of `SupportsInt`, `SupportsFloat`, `SupportsComplex`, `SupportsBytes`, `SupportsAbs` and `SupportsRound`. These have the same semantics as the versions from the `typing` module, but `isinstance()` checks against the `typing_extensions` versions are >10x faster at runtime on Python <3.12. Patch by Alex Waygood. - Add `__orig_bases__` to non-generic TypedDicts, call-based TypedDicts, and call-based NamedTuples. Other TypedDicts and NamedTuples already had the attribute. Patch by Adrian Garcia Badaracco. - Add `typing_extensions.get_original_bases`, a backport of [`types.get_original_bases`](https://docs.python.org/3.12/library/types.html#types.get_original_bases), introduced in Python 3.12 (CPython[https://github.com/python/cpython/pull/101827](https://github.com/python/cpython/pull/101827)l/101827, originally by James Hilton-Balfe). Patch by Alex Waygood. This function should always produce correct results when called on classes constructed using features from `typing_extensions`. However, it may produce incorrect results when called on some `NamedTuple` or `TypedDict` classes that use `typing.{NamedTuple,TypedDict}` on Python <=3.11. - Constructing a call-based `TypedDict` using keyword arguments for the fields now causes a `DeprecationWarning` to be emitted. This matches the behaviour of `typing.TypedDict` on 3.11 and 3.12. - Backport the implementation of `NewType` from 3.10 (where it is implemented as a class rather than a function). This allows user-defined `NewType`s to be pickled. Patch by Alex Waygood. - Fix tests and import on Python 3.12, where `typing.TypeVar` can no longer be subclassed. Patch by Jelle Zijlstra. - Add `typing_extensions.TypeAliasType`, a backport of `typing.TypeAliasType` from PEP 695. Patch by Jelle Zijlstra. - Backport changes to the repr of `typing.Unpack` that were made in order to implement [PEP 692](https://peps.python.org/pep-0692/) (backport of [https://github.com/python/cpython/pull/104048](https://github.com/python/cpython/pull/104048)4048). Patch by Alex Waygood. </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://app.renovatebot.com/dashboard#github/allenporter/flux-local).  Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

@deprecated

Changelog: ============ - Fix use of @deprecated on classes with __new__ but no __init__. - Fix regression in version 4.6.1 where comparing a generic class against a runtime-checkable protocol using isinstance() would cause AttributeError to be raised if using Python 3.7. - Change deprecated @runtime to formal API @runtime_checkable in the error message. - Fix regression in 4.6.0 where attempting to define a Protocol that was generic over a ParamSpec or a TypeVarTuple would cause TypeError to be raised. - typing_extensions is now documented at https://typing-extensions.readthedocs.io/en/latest/. - Add typing_extensions.Buffer, a marker class for buffer types, as proposed by PEP 688. Equivalent to collections.abc.Buffer in Python 3.12. - Backport two CPython PRs fixing various issues with typing.Literal: python/cpython#23294 and python/cpython#23383. Both CPython PRs were originally, and both were backported to Python >=3.9.1, but no earlier. - A side effect of one of the changes is that equality comparisons of Literal objects will now raise a TypeError if one of the Literal objects being compared has a mutable parameter. (Using mutable parameters with Literal is not supported by PEP 586 or by any major static type checkers.) - Literal is now reimplemented on all Python versions <= 3.10.0. - Backport CPython PR 26067, ensuring that isinstance() calls on protocols raise TypeError when the protocol is not decorated with @runtime_checkable. - Backport several significant performance improvements to runtime-checkable protocols that have been made in Python 3.12 (see python/cpython#74690 for details). - A side effect of one of the performance improvements is that the members of a runtime-checkable protocol are now considered "frozen" at runtime as soon as the class has been created. Monkey-patching attributes onto a runtime-checkable protocol will still work, but will have no impact on isinstance() checks comparing objects to the protocol. See "What's New in Python 3.12" for more details. - isinstance() checks against runtime-checkable protocols now use inspect.getattr_static() rather than hasattr() to lookup whether attributes exist (backporting python/cpython#103034). - Backport the ability to define __init__ methods on Protocol classes, a change made in Python 3.11 (originally implemented in python/cpython#31628 - Speedup isinstance(3, typing_extensions.SupportsIndex) by >10x on Python <3.12. - Add typing_extensions versions of SupportsInt, SupportsFloat, SupportsComplex, SupportsBytes, SupportsAbs and SupportsRound. These have the same semantics as the versions from the typing module, but isinstance() checks against the typing_extensions versions are >10x faster at runtime on Python <3.12. - Add __orig_bases__ to non-generic TypedDicts, call-based TypedDicts, and call-based NamedTuples. - Add typing_extensions.get_original_bases, a backport of types.get_original_bases, introduced in Python 3.12 (CPython PR python/cpython#101827, originally - This function should always produce correct results when called on classes constructed using features from typing_extensions. - Constructing a call-based TypedDict using keyword arguments for the fields now causes a DeprecationWarning to be emitted. This matches the behaviour of typing.TypedDict on 3.11 and 3.12. - Backport the implementation of NewType from 3.10 (where it is implemented as a class rather than a function). This allows user-defined NewTypes to be pickled. - Fix tests and import on Python 3.12, where typing.TypeVar can no longer be subclassed. - Add typing_extensions.TypeAliasType, a backport of typing.TypeAliasType from PEP 695. - Backport changes to the repr of typing.Unpack that were made in order to implement PEP 692 (backport of python/cpython#104048). (From OE-Core rev: a37154b9166323d05cca970ebb37bee0d5250893) Signed-off-by: Wang Mingyu <wangmy@fujitsu.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bpo-42345: Literal equality no longer depends on order of arguments

03637e2

uriyyo requested review from gvanrossum and ilevkivskyi as code owners November 15, 2020 13:25

the-knights-who-say-ni added the CLA signed label Nov 15, 2020

bedevere-bot added the awaiting review label Nov 15, 2020

Fidget-Spinner reviewed Nov 15, 2020

View reviewed changes

Lib/typing.py Outdated Show resolved Hide resolved

Use value and type for Literal equality check

4fbf590

Fidget-Spinner reviewed Nov 15, 2020

View reviewed changes

Lib/test/test_typing.py Outdated Show resolved Hide resolved

Return Literal equality test

ea06129

Use typed lru_cache for Literal type

b8a597a

Update Misc/NEWS

19b0952

uriyyo changed the title ~~bpo-42345: Literal equality no longer depends on order of arguments~~ bpo-42345: Fix typing.Literal equals method to ignore the order of arguments Nov 15, 2020

gvanrossum reviewed Nov 16, 2020

View reviewed changes

Lib/typing.py Show resolved Hide resolved

uriyyo added 2 commits November 16, 2020 10:53

Update signature of _tp_cache function

9d2187a

Add Literal arguments deduplication

942dece

Fidget-Spinner reviewed Nov 16, 2020

View reviewed changes

Lib/typing.py Show resolved Hide resolved

Lib/typing.py Show resolved Hide resolved

Lib/typing.py Show resolved Hide resolved

Fidget-Spinner approved these changes Nov 16, 2020

View reviewed changes

bedevere-bot added awaiting core review and removed awaiting review labels Nov 16, 2020

Dominik1123 reviewed Nov 16, 2020

View reviewed changes

Lib/typing.py Show resolved Hide resolved

bedevere-bot removed the needs backport to 3.9 only security fixes label Nov 17, 2020

uriyyo mentioned this pull request Nov 18, 2020

bpo-42345: Fix hash implementation of typing.Literal #23383

Merged

miss-islington pushed a commit that referenced this pull request Nov 19, 2020

bpo-42345: Fix hash implementation of typing.Literal (GH-23383)

1b54077

Fix hash implementation of `typing.Literal`. Update docs regarding `typing.Litaral` caching. Base implementation was done in PR #23294.

miss-islington mentioned this pull request Nov 19, 2020

[3.9] bpo-42345: Fix hash implementation of typing.Literal (GH-23383) #23408

Merged

adriangb mentioned this pull request Apr 17, 2023

Incorrect deduplication of Enum values in Union[Literal[1], Literal[SomeEnum.one]] #103592

Closed

AlexWaygood mentioned this pull request Apr 17, 2023

Fix various things with Literal python/typing_extensions#145

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bpo-42345: Fix three issues with typing.Literal parameters #23294

bpo-42345: Fix three issues with typing.Literal parameters #23294

uriyyo commented Nov 15, 2020 •

edited by miss-islington

Loading

Fidget-Spinner left a comment

Fidget-Spinner left a comment •

edited

Loading

uriyyo commented Nov 15, 2020

Fidget-Spinner commented Nov 15, 2020 •

edited

Loading

uriyyo commented Nov 15, 2020

gvanrossum commented Nov 15, 2020

uriyyo commented Nov 15, 2020

gvanrossum left a comment

uriyyo commented Nov 16, 2020

Fidget-Spinner left a comment

Fidget-Spinner left a comment

uriyyo commented Nov 16, 2020

Fidget-Spinner commented Nov 16, 2020 •

edited

Loading

gvanrossum commented Nov 16, 2020

Dominik1123 commented Nov 16, 2020

uriyyo commented Nov 17, 2020

Dominik1123 commented Nov 17, 2020

Fidget-Spinner commented Nov 18, 2020 •

edited

Loading

uriyyo commented Nov 18, 2020

gvanrossum commented Nov 18, 2020

Fidget-Spinner commented Nov 18, 2020 •

edited

Loading

uriyyo commented Nov 18, 2020

Fidget-Spinner commented Nov 18, 2020 •

edited

Loading

Dominik1123 commented Nov 18, 2020

uriyyo commented Nov 18, 2020

Dominik1123 commented Nov 18, 2020

gvanrossum commented Nov 18, 2020

uriyyo commented Nov 18, 2020

gvanrossum commented Nov 18, 2020 via email

bpo-42345: Fix three issues with typing.Literal parameters #23294

bpo-42345: Fix three issues with typing.Literal parameters #23294

Conversation

uriyyo commented Nov 15, 2020 • edited by miss-islington Loading

Fidget-Spinner left a comment

Choose a reason for hiding this comment

Fidget-Spinner left a comment • edited Loading

Choose a reason for hiding this comment

uriyyo commented Nov 15, 2020

Fidget-Spinner commented Nov 15, 2020 • edited Loading

uriyyo commented Nov 15, 2020

gvanrossum commented Nov 15, 2020

uriyyo commented Nov 15, 2020

gvanrossum left a comment

Choose a reason for hiding this comment

uriyyo commented Nov 16, 2020

Fidget-Spinner left a comment

Choose a reason for hiding this comment

Fidget-Spinner left a comment

Choose a reason for hiding this comment

uriyyo commented Nov 16, 2020

Fidget-Spinner commented Nov 16, 2020 • edited Loading

gvanrossum commented Nov 16, 2020

Dominik1123 commented Nov 16, 2020

uriyyo commented Nov 17, 2020

Dominik1123 commented Nov 17, 2020

Fidget-Spinner commented Nov 18, 2020 • edited Loading

uriyyo commented Nov 18, 2020

gvanrossum commented Nov 18, 2020

Fidget-Spinner commented Nov 18, 2020 • edited Loading

uriyyo commented Nov 18, 2020

Fidget-Spinner commented Nov 18, 2020 • edited Loading

Dominik1123 commented Nov 18, 2020

uriyyo commented Nov 18, 2020

Dominik1123 commented Nov 18, 2020

gvanrossum commented Nov 18, 2020

uriyyo commented Nov 18, 2020

gvanrossum commented Nov 18, 2020 via email

uriyyo commented Nov 15, 2020 •

edited by miss-islington

Loading

Fidget-Spinner left a comment •

edited

Loading

Fidget-Spinner commented Nov 15, 2020 •

edited

Loading

Fidget-Spinner commented Nov 16, 2020 •

edited

Loading

Fidget-Spinner commented Nov 18, 2020 •

edited

Loading

Fidget-Spinner commented Nov 18, 2020 •

edited

Loading

Fidget-Spinner commented Nov 18, 2020 •

edited

Loading