Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Infinite recursion in the tokeniser when showing warnings #99891

Closed
pablogsal opened this issue Nov 30, 2022 · 0 comments
Closed

Infinite recursion in the tokeniser when showing warnings #99891

pablogsal opened this issue Nov 30, 2022 · 0 comments

Comments

@pablogsal
Copy link
Member

pablogsal commented Nov 30, 2022

Turns out that showing warnings in the tokenizer is quite tricky because in the process of showing the warning we need to fetch the encoding which needs to tokenize the first two lines and if the warning is there that leads to an infinite loop. Check for instance with a file contaning:

0b1and x

Notice that this only happens when tokenizing files.

Linked PRs

pablogsal added a commit to pablogsal/cpython that referenced this issue Nov 30, 2022
pablogsal added a commit to pablogsal/cpython that referenced this issue Nov 30, 2022
…warnings

Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
miss-islington pushed a commit that referenced this issue Nov 30, 2022
pablogsal added a commit to pablogsal/cpython that referenced this issue Nov 30, 2022
…howing warnings (pythonGH-99893)

Automerge-Triggered-By: GH:pablogsal.
(cherry picked from commit 417206a)

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
miss-islington pushed a commit that referenced this issue Dec 1, 2022
… warnings (GH-99893) (GH-99896)

Automerge-Triggered-By: GH:pablogsal.
(cherry picked from commit 417206a)

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
@lysnikolaou lysnikolaou changed the title Unfinite recursion in the tokeniser when showing warnings Infinite recursion in the tokeniser when showing warnings Dec 1, 2022
carljm added a commit to carljm/cpython that referenced this issue Dec 1, 2022
* main: (112 commits)
  pythongh-99894: Ensure the local names don't collide with the test file in traceback suggestion error checking (python#99895)
  pythongh-99612: Fix PyUnicode_DecodeUTF8Stateful() for ASCII-only data (pythonGH-99613)
  Doc: Add summary line to isolation_level & autocommit sqlite3.connect params (python#99917)
  pythonGH-98906 ```re``` module: ```search() vs. match()``` section should mention ```fullmatch()``` (pythonGH-98916)
  pythongh-89189: More compact range iterator (pythonGH-27986)
  bpo-47220: Document the optional callback parameter of weakref.WeakMethod (pythonGH-25491)
  pythonGH-99905: Fix output of misses in summarize_stats.py execution counts (pythonGH-99906)
  pythongh-99845: PEP 670: Convert PyObject macros to functions (python#99850)
  pythongh-99845: Use size_t type in __sizeof__() methods (python#99846)
  pythonGH-99877)
  Fix typo in exception message in `multiprocessing.pool` (python#99900)
  pythongh-87092: move all localsplus preparation into separate function called from assembler stage (pythonGH-99869)
  pythongh-99891: Fix infinite recursion in the tokenizer when showing warnings (pythonGH-99893)
  pythongh-99824: Document that sqlite3.connect implicitly open a transaction if autocommit=False (python#99825)
  pythonGH-81057: remove static state from suggestions.c (python#99411)
  Improve zip64 limit error message (python#95892)
  pythongh-98253: Break potential reference cycles in external code worsened by typing.py lru_cache (python#98591)
  pythongh-99127: Allow some features of syslog to the main interpreter only (pythongh-99128)
  pythongh-82836: fix private network check (python#97733)
  Docs: improve accuracy of socketserver reference (python#24767)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants