Upgrade to Python 3.12

[jwhitlock]

Python 3.12.0 was released October 2, 2023. We'll want to upgrade to Python 3.11 first (issue #2731) (done), then ensure our dependencies work with 3.12.

New features

• More flexible f-string parsing, allowing many things previously disallowed (PEP 701).
• Support for the buffer protocol in Python code (PEP 688).
• A new debugging/profiling API (PEP 669).
• Support for isolated subinterpreters with separate Global Interpreter Locks (PEP 684).
• Even more improved error messages. More exceptions potentially caused by typos now make suggestions to the user.
• Support for the Linux perf profiler to report Python function names in traces.
• Many large and small performance improvements (like PEP 709 and support for the BOLT binary optimizer), delivering an estimated 5% overall performance improvement.

Type annotations

• New type annotation syntax for generic classes (PEP 695).
• New override decorator for methods (PEP 698).

Deprecations

• The deprecated wstr and wstr_length members of the C implementation of unicode objects were removed, per PEP 623.
• In the unittest module, a number of long deprecated methods and classes were removed. (They had been deprecated since Python 3.1 or 3.2).
• The deprecated smtpd and distutils modules have been removed (see PEP 594 and PEP 632. The setuptools package continues to provide the distutils module.
• A number of other old, broken and deprecated functions, classes and methods have been removed.
• Invalid backslash escape sequences in strings now warn with SyntaxWarning instead of DeprecationWarning, making them more visible. (They will become syntax errors in the future.)
• The internal representation of integers has changed in preparation for performance enhancements. (This should not affect most users as it is an internal detail, but it may cause problems for Cython-generated code.)

For more details on the changes to Python 3.12, see What's new in Python 3.12.

[jwhitlock]

When 3.12 was deployed to production on December 4th, 2024 with 2024.11.21, we saw the time to process an email increase from about 1-2 seconds to 3 - 3.5 seconds. This resulted in a backlog of emails during peak times (U.S.A business hours), and a delay of up to an hour to forward an email. We reverted to Python 3.11 with 2024.11.21.1. We'll add more metrics so that we can test email processing time on the staging instance and determine why 3.12 was such as significant slowdown.