Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add test for A᧚ = 0 #1557

Closed
Zac-HD opened this issue Jul 21, 2020 · 3 comments · Fixed by #2816
Closed

Add test for A᧚ = 0 #1557

Zac-HD opened this issue Jul 21, 2020 · 3 comments · Fixed by #2816
Labels
C: maintenance Related to project maintenance, e.g. CI, testing, policy changes, releases good first issue Good for newcomers

Comments

@Zac-HD
Copy link
Contributor

Zac-HD commented Jul 21, 2020 •
edited
Loading

Black v19.10b0 fails to parse certain assignments involving unicode identifiers - playground link here.

A᧚
A፩

This code is in fact valid Python, as you can confirm by pasting it into a repl or with compile("A\u19da = 0").

Found, as with most of my bugs, via Hypothesmith. Given that this applies to multiple unicode digit characters, it might be due to unicode digits in identifiers?
@ichard26
Copy link
Collaborator

Yapf has the same problem, google/yapf#403, lib2to3 (of which blib2to3 is built on) doesn't really have full unicode support.

@Zac-HD Zac-HD mentioned this issue Jul 30, 2020
Merged
@ichard26
Copy link
Collaborator

Resolved by 5e2bb52 first available in 21.12b0 which broadened the acceptable character set for NAMES significantly. Thanks for reporting!

@JelleZijlstra
Copy link
Collaborator

We should add a regression test to make sure this doesn't regress if we change our tokenizer.

@JelleZijlstra JelleZijlstra reopened this Dec 14, 2021
@JelleZijlstra JelleZijlstra added the good first issue Good for newcomers label Dec 14, 2021
@JelleZijlstra JelleZijlstra changed the title Cannot parse A᧚ = 0 Add test for A᧚ = 0 Dec 14, 2021
@ichard26 ichard26 added the C: maintenance Related to project maintenance, e.g. CI, testing, policy changes, releases label Dec 14, 2021
hauntsaninja pushed a commit to hauntsaninja/black that referenced this issue Jan 28, 2022
Tests for unicode identifiers
141e86f 
Resolves psf#1557
hauntsaninja pushed a commit to hauntsaninja/black that referenced this issue Jan 28, 2022
Tests for unicode identifiers
0378f9a 
Resolves psf#1557
@hauntsaninja hauntsaninja mentioned this issue Jan 28, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
C: maintenance Related to project maintenance, e.g. CI, testing, policy changes, releases good first issue Good for newcomers
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Tests for unicode identifiers hauntsaninja/black
3 participants
@JelleZijlstra @Zac-HD @ichard26