gh-101100: Test docs in nit-picky mode

[hugovk]

This builds upon @encukou's encukou#21, see also https://discuss.python.org/t/broken-references-in-sphinx-docs/19463.

This adds two new parts to the docs CI.

---

1. Check files modified in PR

The files are touched and the Sphinx build re-ran. Sphinx only rebuilds the modified files so is quick. But this time we run Sphinx with the -n nit-picky mode enabled to be warned about extra documentation problems.

However, we don't let it fail the build: you may have modified a file that already has lots of issues, but didn't introduce any (or many) new ones.

But we do pipe the warnings through Doc/tools/warnings-to-gh-actions.py which uses GitHub's annotations feature to highlight where exactly the warning occurs. For example, see the demo at https://github.com/encukou/cpython/pull/21/files:

Screenshot

---

2. Check hardcoded list of "known good files"

We want to make sure certain files have no warnings, and remain with no warnings. To begin with, only Doc/whatsnew/3.12.rst is in this list, but we will iteratively add more as we fix them, especially "important" files.

This time, we turn the warnings into errors with -W, because we don't want these files to regress.

For example, here's 3.12.rst failing: https://github.com/hugovk/cpython/actions/runs/4347385122/jobs/7595550243

Screenshot

---

This PR then also fixes 3.12.rst so the CI passes.

TODO:

• Once merged, backport exceptions (new in 3.12), weakref, asyncio-tasks and gzip docs to 3.11/3.10.

• Issue: Fix all Sphinx reference warnings in the documentation #101100

Automerge-Triggered-By: GH:hugovk

Automerge-Triggered-By: GH:hugovk

[CAM-Gerlach]

This is great; thanks @hugovk and @encukou ! I did have some comments; see below.

If adding back legitimate warnings for missing documentation results in the 3.12 What's New no longer being fully clean, you could add library/sqlite3.rst in its place, which Erlend and I have made sure should be clean (documenting what was missing, etc).

[merwok]

Python repos don’t use force pushes

[CAM-Gerlach]

In general, PR authors are discouraged from force-pushing once the PR is submitted to avoid making things more difficult for reviewers, but it's not an absolute blind ban—there's still some room for careful judgement, in specific situations where doing so does not conflict with, or in fact supports, the actual reasons for which we have that guideline, i.e. maximizing reviewability.

In particular, when we originally discussed this, it was agreed that it's fine to force-push before a draft PR is marked for review, as it can be a very helpful tool to organize a potentially very noisy initial commit history into an atomic, clearly-explained series of changes, which can greatly aid reviewability.

Likewise, it can potentially make sense and actually aid rather than hurt reviewability to amend a single commit immediately after pushing it to fix a small obvious typo introduced in that commit—which was the case here, with the fixup pushed less than 60 seconds after the original. There's little to no chance any reviewer would have seen, much less review/comment on the commit that quickly, thus there's minimal practical negative impact there. By contrast, it reduces unnecessary noise in the commit history and preserves a single atomic commit for reviewers to look at, without back-and-forth trivial fixups complicating things (which certainly make my life harder as a reviewer). And of course, you can still easily see both commits and compare them with one click if desired in the PR's full conversation history.

[hugovk]

Is there anything else needed here?

[ezio-melotti]

Looks great!

[CAM-Gerlach]

LGTM from me (minus a nit); thanks @hugovk

[hugovk]

Fixed and all green now!

• PyErr_DisplayException was documented, but the markup was broken
• PyErr_Display isn't documented, but it's deprecated, so let's skip the reference via !
• asyncio.iscoroutine wasn't yet documented, but is now

[miss-islington]

Status check is done, and it's a success ✅.

[hugovk]

Thanks for checking Miss Islington! Shouldn't you automerge too?

[CAM-Gerlach]

Maybe its time to just switch to regular GitHub automerge? Is there a reason to still use the old system?

[hugovk]

Yeah, it's time! Please could you create a https://github.com/python/core-workflow/ issue? We'll need to coordinate deleting some https://github.com/python/miss-islington code.

---

TODO:

• Once merged, backport exceptions (new in 3.12), weakref, asyncio-tasks and gzip docs to 3.11/3.10.

Backport PRs:

• [3.11] gh-101100: Document PyObject_ClearWeakRefs, gzip's name, and asyncio.iscoroutine #103001
• [3.10] gh-101100: Document PyObject_ClearWeakRefs and gzip's name #103002

[CAM-Gerlach]

Sure, I opened python/core-workflow#498 .

We'll need to coordinate deleting some python/miss-islington code.

The code might still be needed to support automerge-by-default for miss-islington PRs, if GItHub's API doesn't have a way for her to enable it when she makes them. But I discussed that further on the issue.

[brandtbucher]

I think this check now breaks when encountering NEWS entries in the Core and Builtins directory:

Error: One of your files includes a space. Consider using a different output format or removing spaces from your filenames. Please submit an issue on this action's GitHub repo.
Error: One of your files includes a space. Consider using a different output format or removing spaces from your filenames.
All: Doc/includes/typestruct.h Include/cpython/object.h Include/internal/pycore_code.h Include/internal/pycore_opcode.h Lib/importlib/_bootstrap_external.py Lib/opcode.py Lib/test/test_dis.py Lib/test/test_sys.py Misc/NEWS.d/next/Core and Builtins/2023-03-23-22-31-15.gh-issue-93533.RyrB_g.rst Objects/typeobject.c Programs/test_frozenmain.h Python/bytecodes.c Python/generated_cases.c.h Python/opcode_metadata.h Python/specialize.c
Added: Misc/NEWS.d/next/Core and Builtins/2023-03-23-22-31-15.gh-issue-93533.RyrB_g.rst
Modified: Doc/includes/typestruct.h Include/cpython/object.h Include/internal/pycore_code.h Include/internal/pycore_opcode.h Lib/importlib/_bootstrap_external.py Lib/opcode.py Lib/test/test_dis.py Lib/test/test_sys.py Objects/typeobject.c Programs/test_frozenmain.h Python/bytecodes.c Python/generated_cases.c.h Python/opcode_metadata.h Python/specialize.c
Removed: 
Renamed: 
Added or modified: Doc/includes/typestruct.h Include/cpython/object.h Include/internal/pycore_code.h Include/internal/pycore_opcode.h Lib/importlib/_bootstrap_external.py Lib/opcode.py Lib/test/test_dis.py Lib/test/test_sys.py Misc/NEWS.d/next/Core and Builtins/2023-03-23-22-31-15.gh-issue-93533.RyrB_g.rst Objects/typeobject.c Programs/test_frozenmain.h Python/bytecodes.c Python/generated_cases.c.h Python/opcode_metadata.h Python/specialize.c
Added, modified or renamed: Doc/includes/typestruct.h Include/cpython/object.h Include/internal/pycore_code.h Include/internal/pycore_opcode.h Lib/importlib/_bootstrap_external.py Lib/opcode.py Lib/test/test_dis.py Lib/test/test_sys.py Misc/NEWS.d/next/Core and Builtins/2023-03-23-22-31-15.gh-issue-93533.RyrB_g.rst Objects/typeobject.c Programs/test_frozenmain.h Python/bytecodes.c Python/generated_cases.c.h Python/opcode_metadata.h Python/specialize.c

[hugovk]

Please see PR #103019.

[encukou]

PyErr_Display isn't documented, but it's deprecated, so let's skip the reference via !

I don't think this is good general policy. Someone encountering a deprecated function in code they inherited should be able to find out what it does and what to use instead.