PEP 709 segfault with nested comprehensions plus lambdas

[JelleZijlstra]

This is another crasher related to PEP 709.

def f():
    [([lambda: x for x in range(4)], lambda: x) for x in range(3)]

Stack trace:

(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x8)
  * frame #0: 0x00000001001662f0 python.exe`Py_TYPE(ob=0x0000000000000000) at object.h:204:16 [opt]
    frame #1: 0x000000010015d4e0 python.exe`_PyEval_EvalFrameDefault(tstate=<unavailable>, frame=0x0000000100a60020, throwflag=<unavailable>) at bytecodes.c:1398:31 [opt]
    frame #2: 0x0000000100157c00 python.exe`_PyEval_EvalFrame(tstate=<unavailable>, frame=<unavailable>, throwflag=<unavailable>) at pycore_ceval.h:87:16 [opt]
    frame #3: 0x0000000100157b20 python.exe`_PyEval_Vector(tstate=0x0000000100567f50, func=0x0000000100e19910, locals=0x0000000100e2cd70, args=0x0000000000000000, argcount=0, kwnames=0x0000000000000000) at ceval.c:1576:12 [opt]
    frame #4: 0x0000000100157a0c python.exe`PyEval_EvalCode(co=0x0000000100e7a110, globals=0x0000000100e2cd70, locals=0x0000000100e2cd70) at ceval.c:567:21 [opt]
    frame #5: 0x00000001001bcce0 python.exe`run_eval_code_obj(tstate=0x0000000100567f50, co=0x0000000100e7a110, globals=0x0000000100e2cd70, locals=0x0000000100e2cd70) at pythonrun.c:1695:9 [opt]
    frame #6: 0x00000001001bae50 python.exe`run_mod(mod=<unavailable>, filename=<unavailable>, globals=0x0000000100e2cd70, locals=0x0000000100e2cd70, flags=<unavailable>, arena=<unavailable>) at pythonrun.c:1716:19 [opt]
    frame #7: 0x00000001001ba42c python.exe`pyrun_file(fp=0x0000000209e4cd10, filename=0x0000000100e89b60, start=<unavailable>, globals=0x0000000100e2cd70, locals=0x0000000100e2cd70, closeit=1, flags=0x000000016fdff478) at pythonrun.c:1616:15 [opt]
    frame #8: 0x00000001001b99dc python.exe`_PyRun_SimpleFileObject(fp=0x0000000209e4cd10, filename=0x0000000100e89b60, closeit=1, flags=0x000000016fdff478) at pythonrun.c:433:13 [opt]
    frame #9: 0x00000001001b96cc python.exe`_PyRun_AnyFileObject(fp=0x0000000209e4cd10, filename=0x0000000100e89b60, closeit=1, flags=0x000000016fdff478) at pythonrun.c:78:15 [opt]
    frame #10: 0x00000001001d9094 python.exe`pymain_run_file_obj(program_name=0x0000000100e89bd0, filename=0x0000000100e89b60, skip_source_first_line=0) at main.c:360:15 [opt]
    frame #11: 0x00000001001d8c64 python.exe`pymain_run_file(config=0x000000010054a2c0) at main.c:379:15 [opt]
    frame #12: 0x00000001001d8418 python.exe`pymain_run_python(exitcode=0x000000016fdff5dc) at main.c:610:21 [opt]
    frame #13: 0x00000001001d82a0 python.exe`Py_RunMain at main.c:689:5 [opt]
    frame #14: 0x00000001001d8518 python.exe`pymain_main(args=0x000000016fdff640) at main.c:719:12 [opt]
    frame #15: 0x00000001001d855c python.exe`Py_BytesMain(argc=<unavailable>, argv=<unavailable>) at main.c:743:12 [opt]
    frame #16: 0x0000000100003de4 python.exe`main(argc=<unavailable>, argv=<unavailable>) at python.c:15:12 [opt]
    frame #17: 0x000000010086108c dyld`start + 520

Interestingly, this is an interpreter crash, not a compiler crash, even though we never call the function. The crash happens inside COPY_FREE_VARS, because the function's func_closure is NULL.

When we disassemble it we can see why:

>>> dis.dis("""
... def f():
...     [([lambda: x for x in range(4)], lambda: x) for x in range(3)]
... """)
              0 COPY_FREE_VARS           1

  0           2 RESUME                   0

  2           4 LOAD_CONST               0 (<code object f at 0x104d2b740, file "<dis>", line 2>)
              6 MAKE_FUNCTION            0
              8 STORE_NAME               0 (f)
             10 RETURN_CONST             1 (None)

Disassembly of <code object f at 0x104d2b740, file "<dis>", line 2>:
<snip>

There is a COPY_FREE_VARS at the module scope, but there is no closure to copy at that level.

Still reproduces with the fix from #104394 applied. cc @carljm

Linked PRs

• gh-104404: fix crasher with nested comprehensions plus lambdas #104442

[brandtbucher]

You two make a good team.