Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TSan reported race condition in _PyPegen_is_memoized in test_importlib #122581

Closed
colesbury opened this issue Aug 1, 2024 · 1 comment
Closed

TSan reported race condition in _PyPegen_is_memoized in test_importlib #122581

colesbury opened this issue Aug 1, 2024 · 1 comment
Labels
3.13 bugs and security fixes 3.14 new features, bugs and security fixes topic-free-threading type-bug An unexpected behavior, bug, or error

Comments

@colesbury
Copy link
Contributor

colesbury commented Aug 1, 2024 •
edited by bedevere-app bot
Loading

Bug report

I think this is a race on the global _PyRuntime.parser.memo_statistics. That's only used in the debug build for stats, so the race isn't particularly dangerous, but we should fix it.

WARNING: ThreadSanitizer: data race (pid=8921)
  Write of size 8 at 0x55b27b2a92f0 by thread T84:
    #0 _PyPegen_is_memoized /home/runner/work/cpython/cpython/Parser/pegen.c:351:39 (python+0x166292) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #1 bitwise_or_rule /home/runner/work/cpython/cpython/Parser/parser.c:[12](https://github.com/python/cpython/actions/runs/10204655178/job/28233674832?pr=122577#step:13:13)893:9 (python+0x1842c2) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #2 bitwise_or_raw /home/runner/work/cpython/cpython/Parser/parser.c:12951:18 (python+0x184511) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #3 bitwise_or_rule /home/runner/work/cpython/cpython/Parser/parser.c:12906:22 (python+0x184511)
    #4 comparison_rule /home/runner/work/cpython/cpython/Parser/parser.c:12146:18 (python+0x19bc7f) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #5 inversion_rule /home/runner/work/cpython/cpython/Parser/parser.c:12097:31 (python+0x19b869) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #6 conjunction_rule /home/runner/work/cpython/cpython/Parser/parser.c:11974:18 (python+0x19a9b8) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #7 disjunction_rule /home/runner/work/cpython/cpython/Parser/parser.c:11886:18 (python+0x1938f8) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #8 expression_rule /home/runner/work/cpython/cpython/Parser/parser.c:11174:18 (python+0x182e81) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #9 named_expression_rule /home/runner/work/cpython/cpython/Parser/parser.c:11832:31 (python+0x181b40) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #10 if_stmt_rule /home/runner/work/cpython/cpython/Parser/parser.c:5927:18 (python+0x1769b7) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #11 compound_stmt_rule /home/runner/work/cpython/cpython/Parser/parser.c:2114:28 (python+0x1769b7)
    #12 statement_rule /home/runner/work/cpython/cpython/Parser/parser.c:1419:18 (python+0x17582f) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #[13](https://github.com/python/cpython/actions/runs/10204655178/job/28233674832?pr=122577#step:13:14) _loop1_3_rule /home/runner/work/cpython/cpython/Parser/parser.c:25738:30 (python+0x17582f)
    #14 statements_rule /home/runner/work/cpython/cpython/Parser/parser.c:1376:18 (python+0x17582f)
    #15 file_rule /home/runner/work/cpython/cpython/Parser/parser.c:1178:18 (python+0x170f5a) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #16 _PyPegen_parse /home/runner/work/cpython/cpython/Parser/parser.c:42410:18 (python+0x170f5a)
    #17 _PyPegen_run_parser /home/runner/work/cpython/cpython/Parser/pegen.c:869:17 (python+0x167e01) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #18 _PyPegen_run_parser_from_string /home/runner/work/cpython/cpython/Parser/pegen.c:992:[14](https://github.com/python/cpython/actions/runs/10204655178/job/28233674832?pr=122577#step:13:15) (python+0x1684e7) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #19 _PyParser_ASTFromString /home/runner/work/cpython/cpython/Parser/peg_api.c:13:21 (python+0x2175dd) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #20 Py_CompileStringObject /home/runner/work/cpython/cpython/Python/pythonrun.c:1435:11 (python+0x5fa5ab) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
...

Previous write of size 8 at 0x55b27b2a92f0 by thread T83:
    #0 _PyPegen_is_memoized /home/runner/work/cpython/cpython/Parser/pegen.c:351:39 (python+0x166292) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #1 bitwise_or_rule /home/runner/work/cpython/cpython/Parser/parser.c:12893:9 (python+0x1842c2) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #2 bitwise_or_raw /home/runner/work/cpython/cpython/Parser/parser.c:12951:18 (python+0x184511) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #3 bitwise_or_rule /home/runner/work/cpython/cpython/Parser/parser.c:12906:22 (python+0x184511)
    #4 comparison_rule /home/runner/work/cpython/cpython/Parser/parser.c:12146:18 (python+0x19bc7f) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #5 inversion_rule /home/runner/work/cpython/cpython/Parser/parser.c:12097:31 (python+0x19b869) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #6 conjunction_rule /home/runner/work/cpython/cpython/Parser/parser.c:11974:18 (python+0x19a9b8) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #7 disjunction_rule /home/runner/work/cpython/cpython/Parser/parser.c:11886:18 (python+0x1938f8) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #8 expression_rule /home/runner/work/cpython/cpython/Parser/parser.c:11174:18 (python+0x182e81) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #9 star_expression_rule /home/runner/work/cpython/cpython/Parser/parser.c:11551:31 (python+0x1a0839) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #10 star_expressions_rule /home/runner/work/cpython/cpython/Parser/parser.c:11391:18 (python+0x19f602) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #11 simple_stmt_rule /home/runner/work/cpython/cpython/Parser/parser.c:17[63](https://github.com/python/cpython/actions/runs/10204655178/job/28233674832?pr=122577#step:13:64):18 (python+0x20241e) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #12 simple_stmts_rule /home/runner/work/cpython/cpython/Parser/parser.c:1618:18 (python+0x177d4a) (BuildId: 5a0193[64](https://github.com/python/cpython/actions/runs/10204655178/job/28233674832?pr=122577#step:13:65)4aa87641efb42b5a646a83190f2fd33b)
    #13 statement_rule /home/runner/work/cpython/cpython/Parser/parser.c:1443:34 (python+0x175911) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #14 _loop1_3_rule /home/runner/work/cpython/cpython/Parser/parser.c:25738:30 (python+0x175911)
    #15 statements_rule /home/runner/work/cpython/cpython/Parser/parser.c:1376:18 (python+0x175911)
    #16 file_rule /home/runner/work/cpython/cpython/Parser/parser.c:1178:18 (python+0x170f5a) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #17 _PyPegen_parse /home/runner/work/cpython/cpython/Parser/parser.c:42410:18 (python+0x170f5a)
    #18 _PyPegen_run_parser /home/runner/work/cpython/cpython/Parser/pegen.c:869:17 (python+0x1[67](https://github.com/python/cpython/actions/runs/10204655178/job/28233674832?pr=122577#step:13:68)e01) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #19 _PyPegen_run_parser_from_string /home/runner/work/cpython/cpython/Parser/pegen.c:992:14 (python+0x1[68](https://github.com/python/cpython/actions/runs/10204655178/job/28233674832?pr=122577#step:13:69)4e7) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #20 _PyParser_ASTFromString /home/runner/work/cpython/cpython/Parser/peg_api.c:13:21 (python+0x2175dd) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #21 Py_CompileStringObject /home/runner/work/cpython/cpython/Python/pythonrun.c:1435:11 (python+0x5fa5ab) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
...

Job: https://github.com/python/cpython/actions/runs/10204655178/job/28233674832?pr=122577
Full log archive: https://gist.github.com/colesbury/bfd65049b17505fcae64f5afd3ed388e

Linked PRs
@colesbury colesbury added type-bug An unexpected behavior, bug, or error 3.13 bugs and security fixes topic-free-threading 3.14 new features, bugs and security fixes labels Aug 1, 2024
@colesbury
Copy link
Contributor Author

cc @lysnikolaou, since you were looking at parser thread-safety recently

lysnikolaou added a commit to lysnikolaou/cpython that referenced this issue Aug 5, 2024
@lysnikolaou
pythongh-122581: Avoid data races when collecting parser statistics
4379bd1
@bedevere-app bedevere-app bot mentioned this issue Aug 5, 2024
Merged
lysnikolaou added a commit that referenced this issue Aug 6, 2024
@lysnikolaou
gh-122581: Avoid data races when collecting parser statistics (#122694)
ce0d66c
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Aug 6, 2024
@lysnikolaou @miss-islington
pythongh-122581: Avoid data races when collecting parser statistics (p…
75ccc7c 
…ythonGH-122694)

(cherry picked from commit ce0d66c)

Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
@bedevere-app bedevere-app bot mentioned this issue Aug 6, 2024
Merged
kumaraditya303 pushed a commit that referenced this issue Aug 7, 2024
@miss-islington @lysnikolaou
[3.13] gh-122581: Avoid data races when collecting parser statistics (G…
9fb6bfb 
…H-122694) (#122733)

gh-122581: Avoid data races when collecting parser statistics (GH-122694)
(cherry picked from commit ce0d66c)

Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
brandtbucher pushed a commit to brandtbucher/cpython that referenced this issue Aug 7, 2024
@lysnikolaou @brandtbucher
pythongh-122581: Avoid data races when collecting parser statistics (p…
1fb77c9 
…ython#122694)
blhsing pushed a commit to blhsing/cpython that referenced this issue Aug 22, 2024
@lysnikolaou @blhsing
pythongh-122581: Avoid data races when collecting parser statistics (p…
9924072 
…ython#122694)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
3.13 bugs and security fixes 3.14 new features, bugs and security fixes topic-free-threading type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@colesbury @kumaraditya303