Fix the signatures of @(async)contextmanager

[Sachaa-Thanasius]

A continuation of the work and discussion in #2772, #2773, #7430, and #8698. Seemingly unblocked by #11867.

This attempts to deprecate the current less restrictive and less accurate signatures of contextlib.(async)contextmanager using the deprecated decorator to provide users with a transition period (hopefully with less type checker screaming than in previous attempts). The text currently in deprecated() could be modified to convey a time period as well if that could be agreed upon, e.g. "will be disallowed in (3.14 | 6 months | 1 year)".

Additional Rationale
I'm not aware if there're any problems with using deprecated in this manner, but if so, this might be as good a place as any to inspire a discussion about them. I'd argue it's not the worst idea, since technically typeshed's signatures for these functions have been inconsistent with the runtime for ages. Thus, even if this usage isn't based on a runtime change as other uses of deprecated seem to be, it can still do what it's meant to: provide some sort of user feedback on a use case being explicitly less accurate or less valid, and potentially mark a predefined time frame for transitioning from one to the other.

Considering a major blocker in previous attempts to fix the signatures was the amount of "breakage" it would cause, this feels softer, arguably too soft if maintainers don't have the right flag set in their type checkers to generate a "deprecated" warning error. Still, it's better than before; at least with the right type checker setting on, there's some signal about the more correct signature.

Potential Improvements and/or Alternatives

• In the undeprecated overload, we could use custom protocols that aren't as narrow as (Async)Generator in place of the current parameter annotations, as outlined here. They would need to support __(a)next__(), (a)throw(), and probably now (a)close(), based on what's used at runtime. Something like this, perhaps, going off of the linked example:

  from typing import Protocol, TypeVar
  from types import TracebackType
  
  _T = TypeVar("_T")
  
  # The protocol names are wide open for bikeshedding.
  class ThrowableIterator(Protocol[_T]):
      def __next__(self) -> _T: ...
      
      # Note: The most accurate annotated signature for 'throw()' seem to require overloads,
      # if we're going off of 'Generator.throw()', but for the sake of being more succinct,
      # they aren't included here.
      def throw(
          self,
          exc_type: type[BaseException] | None,
          exc_inst: BaseException | None,
          exc_tb: TracebackType | None,
      ) -> Any: ...
      
      # The use of '(a)close()' in '(async)contextmanager' was backported all the way to 3.11.
      # Ref: https://github.com/python/cpython/issues/110378.
      if sys.version_info >= (3, 11):
          def close(self) -> None: ...
          
  class AsyncThrowableIterator(Protocol[_T]): ...  # The async counterpart methods would be in here.

• We can use sys.version_info to mark a clear cutoff point where the signature would no longer be "acceptable". From what I can tell, typeshed doesn't have an explicit deprecation policy for this kind of case, though this might be one way to implement it were it to exist. For example:

  if sys.version_info >= (3, 14):
      # Keep just the accurate signature here.
  else:
      # Keep the new overload and 'deprecated' overload here, or just the original signature
      # if the former isn't accepted.

  I feel like this case might be exceptional enough to warrant explicit deprecation cutoff points like this, given the concern about breakage before and how long the incorrect signature has existed without anything like a fix merged (the first issue linked above is over 5 years old now, for reference).

• We can perform other actions outside of this PR to signal and help with the transition, e.g. pin an informational issue in the issue tracker commenting on the change, the reasons why, and how to adjust.

---

Deferred until mypy supports @deprecated (with release 1.14?)

[Sachaa-Thanasius]

Whoops. Learned something new about renaming branches today. Sorry about that.

[TeamSpen210]

Mypy doesn't yet support @deprecated, so mypy_primer is just going to output nothing regardless of the actual impact. That probably needs to be a prerequisite.

[Sachaa-Thanasius]

Huh, didn't know that. I thought typeshed had an unofficial policy to not use typing-related symbols until a majority of the supported type checkers supported them, so I assumed mypy understood it

EDIT: I knowdeprecated is technically not just typing related, but type checkers still need to implement support for it, so it counts here.

[Sachaa-Thanasius]

Is there any chance mypy_primer could be run here with a different type checker, e.g. pyright, to gauge the impact (EDIT: maybe once with the type checker-specific deprecation flag on vs. off)? AFAIK, the type checker it runs with can be switched, though I don’t know how much trouble this type of adjustment & run would be.

[JelleZijlstra]

I thought typeshed had an unofficial policy to not use typing-related symbols until a majority of the supported type checkers supported them

We don't use features that major type checkers will error on, but mypy just ignores the @deprecated decorator, so it's fine to use it even if mypy doesn't support it.

[Sachaa-Thanasius]

Ah, gotcha. I guess that last primer run was valid, then.

[AlexWaygood]

hopefully with less type checker screaming than in previous attempts

I think you'll get exactly the same amount of type-checker screaming. It's just that they'll be screaming about the signature being "deprecated" rather than it being "incorrect". But that still manifests itself as "the type checker is suddenly and unexpectedly failing in CI" in exactly the same way as an error telling you that the signature was incorrect. So I think the amount of disruption that this would cause is exactly the same.

For mypy, it would of course cause 0 disruption right now, since mypy doesn't yet support @deprecated, but I'd imagine this would cause a huge amount of disruption for pyright users and users of other type checkers that support @deprecated.

[Sachaa-Thanasius]

(Note: Most of my understanding of how type-checkers would visibly react to the presence of @deprecated is based on pyright's current behavior as part of Pylance.)

Cheers for the feedback! It made me realize that I made an incorrect assumption and didn't test it; for some reason, I thought the original suggestion this PR stems from — the direct swap to Generator — would propagate through all the call sites of the decorated function. Either way, it would just be the @(async)contextmanager decorator that's marked as "used in error", so to speak, so the same amount of surface area would be marked by a type checker. Got it.

Still, I think there would be a major difference in the potential disruption compared to before: The "deprecated" error would have its own dedicated configurable flag in one's type checker of choice, and that flag isn't fundamental enough to be turned on by default in people's CI systems across all uses of those type checkers. Pyright, for instance, currently has reportDeprecated on only in strict mode, assuming one isn't more granular in their configuration. Still, that setting is relatively simple to adjust according to one's needs. As a result, the level of effort required to consciously ignore the "deprecation" is much lower as a result (which is arguably a detriment, but for the sake of not causing as much disruption, it's technically a plus).

I will admit, I haven't checked what the mypy team's plans are with their own deprecation-related setting, e.g. whether it will be disabled by default; however, if it is off by default, that would also give projects more freedom to make changes according to deprecations like this at their own pace while not breaking CI as much. Basically, I'm trying to posit that if the relevant flag isn't enabled by default across type checkers, then this idea becomes a lot less likely to trigger type checker CI warnings on a large scale than the original annotation swap does, making it better than nothing.

[srittau]

While I'm in favor of this PR – even if it would cause a huge amount of (easy to fix) warnings – I suggest we wait for the release of a mypy that supports @deprecated. (mypy 1.13 I suppose? python/mypy#17476 has support and was already merged.)

[AlexWaygood]

mypy 1.13 I suppose? python/mypy#17476 has support and was already merged.

I believe @hauntsaninja is planning on releasing 1.13 in the next few days, and it will contain only performance improvements over 1.12, so I expect @deprecated support to land in mypy 1.14.

[github-actions]

According to mypy_primer, this change has no effect on the checked open source code. 🤖🎉

[SpecLad]

Perhaps the type here should be Generator[_T_co, None, object] instead? contextmanager does not actually care what the return type of the generator is. E.g. this:

@contextmanager
def f():
    yield
    return 1

is fine (though pointless).