Implement support for PEP-695

[zsol]

There will be:

• grammar changes
• scoping changes
• CST changes

These changes are scheduled to be included in 3.12

[charliermarsh]

@zsol - Just for our own internal planning, would you expect / is it a goal for LibCST to support PEP 695 in time for the Python 3.12 release (2023-10-02, at time of writing)? Or will it follow after?

[zsol]

Yeah I want LibCST to support the latest available CPython release. We'll make it happen.

[gabriel4k2]

Really interested in contributing, even on small parts, on this. Is this mainly for maintainers (ie, internally) or are you guys open for contributions?

[zsol]

More than happy to accept contributions! Which parts are interesting? Grammar changes are pure Rust, CST are a mix of Python & Rust, scoping changes are pure Python

[zanieb]

If helpful, here are the changes in Ruff which includes references to the pyflakes implementation and RustPython parser changes :) astral-sh/ruff#6109

[zsol]

#1004 has the grammar and CST changes. Scoping changes seem way more complicated than I anticipated so it won't fit into this weekend :)

[JelleZijlstra]

https://jellezijlstra.github.io/pep695 has a lot of discussion of scoping, in case it's helpful.

[zsol]

I've got the scoping rules more-or-less implemented in #1014, but I have two questions:

1. (this is about the PEP) How do type parameters on function and class definitions interact with decorators? Intuitively it feels like decorators should wrap the whole class so they should not have visibility into type variables declared on the thing they're decorating - but I haven't found anything specific about this in the PEP or the discussion you mentioned @JelleZijlstra
2. (this is about my implementation) The way Scope provider changes for type annotations #1014 currently implements the scoping rules, a function's body will have access to the immediately enclosing class's declarations, but only if the function has type parameters. This is obviously incorrect, but I'm not sure how to address this.

[JelleZijlstra]

1. Decorators are evaluated outside the annotation scope used for type params, meaning they don't have access to the type params. This is mentioned in the PEP if you search for "decorator". It's also discussed in the docs at https://docs.python.org/3.12/reference/compound_stmts.html#generic-classes

2. Haven't looked at the code, but possibly looking at symtable.c in CPython can inspire a solution.

[charliermarsh]

This is awesome -- thank you @zsol! Are you able to publish a new version to crates.io with Python 3.12 support?