Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for JUMP_BACKWARD #472

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

Support for JUMP_BACKWARD #472

wants to merge 5 commits into from

Conversation

greenozon
Copy link
Contributor

According to this article the JUMP_BACKWARD was a replacement for the same opcode JUMP_ABSOLUTE
https://bugs.python.org/issue47120

Test Case

def JUMP_BACKWARD():
    iterable = [1, 2, 3]
    for item in iterable:
        pass

    while True:
        break

pycdas for TC

[Code]
    File Name: jmp_back.py
    Object Name: <module>
    Qualified Name: <module>
    Arg Count: 0
    Pos Only Arg Count: 0
    KW Only Arg Count: 0
    Stack Size: 1
    Flags: 0x00000000
    [Names]
        'JUMP_BACKWARD'
    [Locals+Names]
    [Constants]
        [Code]
            File Name: jmp_back.py
            Object Name: JUMP_BACKWARD
            Qualified Name: JUMP_BACKWARD
            Arg Count: 0
            Pos Only Arg Count: 0
            KW Only Arg Count: 0
            Stack Size: 2
            Flags: 0x00000003 (CO_OPTIMIZED | CO_NEWLOCALS)
            [Names]
            [Locals+Names]
                'iterable'
                'item'
            [Constants]
                None
                (
                    1
                    2
                    3
                )
            [Disassembly]
                0       RESUME                          0
                2       BUILD_LIST                      0
                4       LOAD_CONST                      1: (1, 2, 3)
                6       LIST_EXTEND                     1
                8       STORE_FAST                      0: iterable
                10      LOAD_FAST                       0: iterable
                12      GET_ITER
                14      FOR_ITER                        2 (to 20)
                18      STORE_FAST                      1: item
                20      JUMP_BACKWARD                   4 (to 14)
                22      END_FOR
                24      NOP
                26      RETURN_CONST                    0: None
        None
    [Disassembly]
        0       RESUME                          0
        2       LOAD_CONST                      0: <CODE> JUMP_BACKWARD
        4       MAKE_FUNCTION                   0
        6       STORE_NAME                      0: JUMP_BACKWARD
        8       RETURN_CONST                    1: None

before

def JUMP_BACKWARD():
Unsupported opcode: JUMP_BACKWARD
    iterable = [
        1,
        2,
        3]
# WARNING: Decompyle incomplete

after

def JUMP_BACKWARD():
Unsupported opcode: END_FOR
    iterable = [
        1,
        2,
        3]
# WARNING: Decompyle incomplete

@greenozon
Copy link
Contributor Author

@zrax Do you think it makes sense to add support into this PR for the next missed operand?

PS the test was done on Python 3.12

@greenozon greenozon mentioned this pull request Mar 16, 2024
33 tasks
@TiZCrocodile
Copy link
Contributor

I compiled with pycdc, and it doesn't work if you just add the line case Pyc::JUMP_BACKWARD_A:
thats what I got:
the source:

l = [ch for ch in 'hello']

python 3.11: (didn't work, because JUMP_BACKWARD is used, but implementation isn't exactly like JUMP_ABSOLUTE)

# Source Generated with Decompyle++
# File: test_311.pyc (Python 3.11)

l = 'hello'()

python 3.8: (still didn't work, but thats because of list comprehension and lambda together is not supported yet as you can see)

# Source Generated with Decompyle++
# File: test_38.pyc (Python 3.8)

l = (lambda .0: [ ch for ch in .0 ])('hello')

the bytecode is almost exactly the same, only difference is JUMP_BACKWARD instead of JUMP_ABSOLUTE

@greenozon
Copy link
Contributor Author

do you mean this opcode has diff meaning in diff Python versions?

@TiZCrocodile
Copy link
Contributor

TiZCrocodile commented Mar 16, 2024

I mean that the bytecode of this source code

l = [ch for ch in 'hello']

between version 3.10 and 3.11 is this:
3.10:



  1           0 LOAD_CONST               0 (<code object <listcomp> at 0x000001AEABDC8F50, file "test.py", line 1>)
              2 LOAD_CONST               1 ('<listcomp>')
              4 MAKE_FUNCTION            0
              6 LOAD_CONST               2 ('hello')
              8 GET_ITER
             10 CALL_FUNCTION            1
             12 STORE_NAME               0 (l)
             14 LOAD_CONST               3 (None)
             16 RETURN_VALUE

Disassembly of <code object <listcomp> at 0x000001AEABDC8F50, file "test.py", line 1>:
  1           0 BUILD_LIST               0
              2 LOAD_FAST                0 (.0)
        >>    4 FOR_ITER                 4 (to 14)
              6 STORE_FAST               1 (ch)
              8 LOAD_FAST                1 (ch)
             10 LIST_APPEND              2
             12 JUMP_ABSOLUTE            2 (to 4)
        >>   14 RETURN_VALUE

3.11:

  0           0 RESUME                   0

  1           2 LOAD_CONST               0 (<code object <listcomp> at 0x0000028633FF92E0, file "test.py", line 1>)
              4 MAKE_FUNCTION            0
              6 LOAD_CONST               1 ('hello')
              8 GET_ITER
             10 PRECALL                  0
             14 CALL                     0
             24 STORE_NAME               0 (l)
             26 LOAD_CONST               2 (None)
             28 RETURN_VALUE

Disassembly of <code object <listcomp> at 0x0000028633FF92E0, file "test.py", line 1>:
  1           0 RESUME                   0
              2 BUILD_LIST               0
              4 LOAD_FAST                0 (.0)
        >>    6 FOR_ITER                 4 (to 16)
              8 STORE_FAST               1 (ch)
             10 LOAD_FAST                1 (ch)
             12 LIST_APPEND              2
             14 JUMP_BACKWARD            5 (to 6)
        >>   16 RETURN_VALUE

if you look, there is not very much difference in the bytecode,
but the decompiled source code for these pyc's are different, which means that another thing has to be done, because even after adding the line you added, its still not supported.

@TiZCrocodile
Copy link
Contributor

i would suggest to add, after the lines:

if (mod->verCompare(3, 10) >= 0)
    offs *= sizeof(uint16_t); // // BPO-27129

to add this lines, for it to work:

if (opcode == Pyc::JUMP_BACKWARD_A)
    offs = pos - offs;

then this way it is exactly like the jump absolute, and it will work, then you can compile the test file "test_for_loop_py3.8.py" for version 3.11 and add it to the compiled folder, i saw there is return None in the end, so maybe also get the whole code inside a function and remove the return statement,
after that compile it for 3.10,3.11,
tokenize by the token_dump script in the scripts folder
and thats all i think.

@Vendettail
Copy link

Proposal:

        case Pyc::JUMP_BACKWARD_A:
        case Pyc::JUMP_FORWARD_A:
        case Pyc::INSTRUMENTED_JUMP_FORWARD_A:
            {
                int offs = operand;
                if (mod->verCompare(3, 10) >= 0)
                    offs *= sizeof(uint16_t); // // BPO-27129

                if (opcode == Pyc::JUMP_BACKWARD_A)
                    offs *= -1;

@greenozon greenozon mentioned this pull request Jul 22, 2024
41 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants