Python language version support status

[Technologicat]

Long-term support roadmap

Python support schedule (in chart format; original source). Assuming that unpythonic stays in active development, unpythonic will try to support any Python 3.x that is still in official support by the Python devs.

For particular Python versions, this means that, approximately (updated January 2022):

• 3.6 supported until January 2022 (may be dropped soon)
• 3.7 supported until July 2023
• 3.8 supported until October 2024
• 3.9 supported until October 2025
• 3.10 supported until October 2026
• 3.11 not officially supported; not released yet; no EOL date announced yet

Since it's basically just me doing this, and I only have resources to barely support one unified codebase, this implies that at any moment in time, the unpythonic codebase will be based on the oldest currently supported Python version - with possibly the "new way" included as commented-out code (for easy future porting) if it is much more elegant.

I will not backport bug fixes. Any bug fixes will be released in a new version of unpythonic, with Python version support appropriate for its release date.

Current status of Python language version support in unpythonic is tracked by posting new comments to this issue whenever appropriate.

Original posting below.

---

Support for Python 3.4 ended in spring 2019, and for 3.5 in September 2020.

Starting in unpythonic 0.15, drop any code specific to 3.4 or 3.5, and bump the Python version requirement to 3.6. That version still has some years of support remaining, and is the default python3 e.g. on recent Ubuntu versions.

This should simplify especially any macro code that needs to handle the AST for function calls.

[Technologicat]

Ah, maybe it should be documented here that the code should have 3.4 or 3.5 mentioned in the comments anywhere it was foreseen to need an update once we bump the requirement to 3.6.

It may, however, need some refactoring beyond just deleting old code. For example, we should clean up the logic that supports two very different AST representations (3.4, and 3.5+) for *args and **kwargs in a function call. (Grep keywords right there.)

The astcompat module probably shouldn't be dropped yet, if ever. Though bumping the minimum Python version requirement to 3.6, we can ax the 3.5+ and 3.6+ stuff currently in there. We can refactor any use sites to just import these names from ast directly, since they're guaranteed to be there in at least 3.6 (our new minimum) through 3.8 (current Python as of this writing).

Python 3.8 has already plown ahead, changing the AST representation of constants again. As of this writing, the GTS docs haven't updated yet.

So in the near future, we will need to handle Constant. Probably the old constant types are going to be removed at some point in the future - so they should perhaps be added to astcompat already now. This also requires updating all call sites that need to handle constants in the AST to know how to deal with both the old format and the new Python 3.8 Constant. There shouldn't be that many, so it's probably not worth the complexity cost (especially, with regard to readability) to add a layer of abstraction for this. It's simpler if we just handle both representations at each use site.

The new representation of all constants with a single Constant node type - in a unityped language, especially - does seem The Right Thing, so here's hoping it will remain like that at least for a while. The previous major change to the AST (as far as unpythonic is concerned, at least) were the function call changes in 3.5, which were rather major, but it had to change in some way to allow for multiple unpackings in the same call, which is a clear win for writing brief, yet readable code.

So, business as usual when you depend on the specifics of the AST representation in Python (which you must if you do macros), which sits squarely in the no guarantees part of the API...

[Technologicat]

Exciting new things in 3.8 to keep track of:

• PEP 572, Assignment expressions. This could give us a cleaner way to implement the let construct, since one of the major motivations of having a do[] environment (and the whole raison d'être of with envify) is that prior to 3.8, assignment is a statement. The move to use native assignment expressions may need major changes, though. And on the other hand, scoping of let-over-def?
• PEP 570, Positional-only arguments. Finally, Python-level syntax for a feature that was for long only accessible from the C API. This is nice. We may want to make some unpythonic functions accept arguments only positionally.

(From the beta 3 release notes.)

[Technologicat]

"The old format" is Num, Str and NameConstant. Grep for those, they turn up at surprising places in our macro code.

[Technologicat]

The GTS docs have partially updated to 3.8, but as of this writing don't yet have a mention of the new assignment-expression operator :=.

EDIT: the AST node is called NamedExpr.

[Technologicat]

In the semi-distant future, once Python 3.8 becomes the new minimum for unpythonic - on which I'm tempted to wait until PyPy3 implements 3.8 - we could shift our convention for the assignment-expression syntax from name << value to name := value, since Python 3.8 added the := operator specifically for the purpose of denoting assignment in an expression position.

Having read PEP 572, I'm still puzzled as to why the syntactically and semantically uniform as well as typographically more pleasing option didn't win, to just upgrade the existing = into an expression that returns its LHS (after any sequence unpacking/re-packing), but that's the Python community for you.

Note that PEP 572 doesn't make let obsolete, because the semantics of the native assignment expression are different from let and its sisters letseq, letrec. So unpythonic will keep its let-related features at least for the foreseeable future.

What PEP 572 allows is that in our macros, we could repurpose a semantically more appropriate part of the syntax to denote any unpythonic flavors of assignment in an expression position, instead of trampling over the left-shift operator like we currently do. (Grep for << and LShift to find the relevant places.)

[Technologicat]

Ubuntu and Mint have since bumped to 3.8 as the default Python.

As of April 2021, PyPy3 supports 3.7. According to the devblog, the focus is now on 3.8, so it might be Coming Soon™.

[Technologicat]

As of 49d40ff, Python 3.4 and 3.5 support dropped in the 015wip branch.

We now require Python 3.6 or later. The main target for unpythonic 0.15 is Python 3.8, which the development runs on.

Status (April 2021)

As of the mentioned commit, the 015wip branch is still a couple of evenings of work from even getting it to boot up; it's being slowly migrated to mcpyrate and Python 3.8, with support also for 3.9.

So far the Turing tarpit unpythonic/syntax/letdoutil.py has been fully converted, and some initial work has been done on porting the macro interface (unpythonic/syntax/__init__.py) to mcpyrate. Much more conversion work is still needed, but looks doable.

Python versions

Python 3.7 support: no changes to unpythonic needed; the 0.14.3 codebase is already compatible as-is (as indeed reported by the CI process). So this will be fine as long as I don't break anything while porting.

Python 3.8 support: we still need some many changes to the codebase, particularly any places that handle ast.Constant or its predecessors. To find these, grep for "3.8" in the unpythonic codebase. To investigate: do we need to change our handling of arguments nodes, after introduction of positional-only arguments in 3.8?

Python 3.9 support: we need to handle the removal of theIndex wrapper from the slice of Subscript. Also ExtSlice is gone, good riddance. To investigate: dict (or was it set?) supports some new operator in 3.9, do we need to do something regarding this?

Python 3.10 support: a quick scan of the Python 3.10 release notes (as of April 2021, at 3.10.0a7) seems to suggest there are no AST changes this time. This may still change, as the final 3.10 is scheduled for October 2021. It would be nice to support 3.10, too, but no promises there - we can't claim compatibility with "3.10" until there is an official release of 3.10. So, I might make a separate issue for it.

Until GTS officially updates, its issue tracker may be useful to look at. As of this writing, nothing about 3.10 there either, at least yet.

As of Python 3.9, the Python ast docs now include much of the material originally published in GTS (along with a note stating so), but it seems some details are missing. E.g. the documentation for ast.Call says starargs and kwargs are "optional", but does not mention that these fields became obsolete in Python 3.5. Maybe it will improve in the future; this is in any case a good start.

Next steps

The conversion work will focus on getting unpythonic.syntax to boot up on Python 3.8 and 3.9, and then porting unpythonic.test.fixtures to 3.9, so that we get the automated tests running. From there the rest should be relatively smooth sailing.

[Technologicat]

As of 73cf03e, AST changes for supporting 3.6, 3.7, 3.8 and 3.9 are done.

Also, string formatting is now in 3.6+ f-string style, and the detection of Python 3.8 in namelambda has been changed to detect the 3.8.0 release and later.

There's still a lot of work to get all of the macro code fully ported to mcpyrate; this will be tracked in #72.

[Technologicat]

As the changes needed for supporting 3.8 and 3.9 in unpythonic 0.15 are now done (except the macro expander porting, but that's orthogonal), I'm moving this issue to the ∞ milestone, but keeping it open, to track future status of Python version language support in unpythonic.

EDIT: As of end of May 2021, the macro expander porting is also done.

[Technologicat]

Version 0.15.0 supports Python language versions 3.6, 3.7, 3.8 and 3.9.

[Technologicat]

Version 0.15.1 supports Python language versions 3.6, 3.7, 3.8, 3.9 and 3.10.

We should drop 3.6 soon-ish, but there's no practical hurry to do so as long as it's available in GitHub's CI. Once GitHub drops it, it's time to remove 3.6-isms from the codebase.

[Technologicat]

Recent AST changes in the language:

• In Python 3.10, alias nodes must have lineno and col_offset. Mentioned in the full changelog, but not in what's new.
• Python 3.8 and later added end_lineno and end_col_offset. Should look into this wherever we handle source location info. Tracked in Support new source location fields in Python 3.8+ #83 .