Enforce the use of deprecated-removed in docs

[ezio-melotti]

In the docs we have two directives that can be used to document deprecations: deprecated and deprecated-removed.

I think we should always prefer the latter:

• it will make it easier to track and document removals
• it will give people a target, so they can plan around it

Even if the removal version gets postponed, it's better to postpone than to say that something is deprecated and then just remove it at an unspecified time in the future.

Currently deprecated is more commonly used:

$ grep -r 'deprecated::' --include=*.rst | wc -l
226
$ grep -r 'deprecated-removed::' --include=*.rst | wc -l
30

• set removal version for deprecated features using deprecated-removed
• automate the documentation of deprecations (see also Better emphasise pending removals in What's New #92308)
• possibly deprecate the deprecated directive and replace it with deprecated-removed

[erlend-aasland]

possibly deprecate the deprecated directive and replace it with deprecated-removed

Note, some C APIs are deprecated but still kept, in order to maintain ABI compatibility, so I think there's still need for the deprecated directive.

[hugovk]

deprecated is also needed for cases such as #31891 (comment), where the SC wanted to leave out the removal version from docs (but not code warnings).

[ezio-melotti]

For those cases, we could either keep using deprecated or deprecated-removed with versions like 4.0, TBD, or never. This will both serve to communicate clearly what is the intention (e.g. it will be available throughout the life of 3.x, we are still deciding/discussing, etc.) and will prevent more uses of deprecated without a target to creep in.

[erlend-aasland]

That's a very good point, Ezio; providing more context is always a good thing.

[rhettinger]

This conflicts with our practice where we sometimes wait longer than required to actually remove a feature.

Also, you've marked this as applying to previous versions of the docs. We don't normally backport doc changes beyond the current version unless they are actual bug fixes (incorrect docs).

[ezio-melotti]

This conflicts with our practice where we sometimes wait longer than required to actually remove a feature.

If this happens then no harm is done -- we can just updated the target and people that already updated their code won't have to worry about it anymore, people that didn't will have some extra time.

We don't normally backport doc changes beyond the current version unless they are actual bug fixes (incorrect docs).

There are some conflicting opinions (we were just discussing this on Discord). Backporting is useful for people reading docs for older versions and prevents merge conflicts while backporting actual bugs. Are there any disadvantages in backporting them (other than the extra time required to check for @miss-islington)?

[erlend-aasland]

We don't normally backport doc changes beyond the current version unless they are actual bug fixes (incorrect docs).

I think you need to adjust your "we"; there are different practices and opinions amongst the core devs.

[CAM-Gerlach]

As a "regular" Python user (er, well, docs team member, PEP editor and triager, but not a core dev), I'd strongly err on the side of being explicit over implicit and specifying something with regard to the removal version, even if that's 4.0, TBD or never.

If a specific 3.x version is known (or can be decided on), it is very helpful to state that up front, even if is later pushed back, in order to be able to plan around it as a library maintainer who wants to support a reasonably wide range of Python versions, and a FOSS application developer with limited bandwidth who wants to prioritize addressing things that will be actively removed soon; not only is this helpful for users like myself who actively watch, test for and fix deprecations, but also in convincing others convincing others of the necessity of doing so.

And if one is not known (yet), in a future major release or the deprecation is forseeably indefinite, this is likewise very helpful, as it both helps explicitly document (for both users and contributors) that an exact removal version is not yet planned, without having to dig through old threads, ask people or get blindsided by an unexpected removal, and these individual distinctions are still important and can result in different appropriate choices as both a Python user and a contributors.

deprecated is also needed for cases such as #31891 (comment), where the SC wanted to leave out the removal version from docs (but not code warnings).

@hugovk @brettcannon I really don't understand the reason for providing the target removal version up front in the DeprecationWarning (which unfortunately many devs will never see, at least not in a timely manner), but hiding it in the actual documentation, unless users click the link and dig through the PEP?

As discussed above, many deprecated items have been deprecated for years or even a decade or more, and some even potentially indefinitely, the lack of a removal version does not convey any of the actual urgency of the imminent complete removal. Furthermore, I'd think it is much better to err on the side of caution, make the current planned removal date clear so users can respond to it with appropriate urgency, rather than encouraging complacency until it is too late to either address or revert it.

If the concern is the possibility it may be pushed back, the docs can be updated instantly, whereas it is much harder and slower to update released Python versions, if that is possible at all, and @brettcannon you yourself mentioned on the PEP 594 Discourse thread that the PEP is now considered a historical document. And as we saw with PEP 563 and others, clearly establishing an initial set date is a great catalyst for users and ecosystems testing the change and sharing any unanticipated impacts that might justify such a delay, before it is too late.

[brettcannon]

@hugovk @brettcannon I really don't understand the reason for providing the target removal version up front in the DeprecationWarning (which unfortunately many devs will never see, at least not in a timely manner), but hiding it in the actual documentation, unless users click the link and dig through the PEP?

In this specific case, it's because we backported the deprecation warning which we rarely do. But if the target removal version slips and a Python version is in maintenance mode, we don't want to go back and edit the docs just to fix that sort of discrepancy. Backporting the deprecation notice while leaving out the removal version also came from the SC as part of the PEP's acceptance.

[CAM-Gerlach]

I can see the argument for not backporting deprecation messages to begin with, so that past Python versions don't show a future Python version as the deprecation version. But particularly given the value of an explicit removal version, I don't understand how backporting categorically changes the calculus on removal versions from any other deprecation, since even with no backporting and a deprecation for the minimum two releases, the version the deprecation message was added in will still no longer be in bugfix mode a minimum of six months prior to the release of the version with the functionality removed, and in many cases much longer.

I can bring the broader docs backport policy issue up with the Docs Team and WG Editorial Board, but in any case, this critical information should at the very least be added to the docs for the deprecation version forward, so that Python's global user community can be properly made aware of the imminent removal. I had done so in #92612 , though for now I can drop that change and move it to a followup PR so that at least the primary fix to the existing message and PEP link can be backported.

[brettcannon]

this critical information should at the very least be added to the docs for the deprecation version forward, so that Python's global user community can be properly made aware of the imminent removal.

I'm fine with what you're proposing for 3.11 onward, but not backporting to earlier versions.

[CAM-Gerlach]

I'm fine with what you're proposing for 3.11 onward, but not backporting to earlier versions.

Sorry for any confusion—that is what I'm proposing for now ("the deprecation version" == "3.11") 😉