Python Binding: Not yet support UC_HOOK_TLB_FILL?

[DynamicLoader]

I'd installed version 2.1.1

When using hook_add, it just throw a exception. I digged into it and there is no impl of UC_HOOK_TLB_FILL.

        handlers: Mapping[int, Callable[[], Tuple]] = {
            uc.UC_HOOK_INTR               : __hook_intr,
            uc.UC_HOOK_INSN               : __hook_insn,
            uc.UC_HOOK_CODE               : __hook_code,
            uc.UC_HOOK_BLOCK              : __hook_code,
            uc.UC_HOOK_MEM_READ_UNMAPPED  : __hook_invalid_mem,
            uc.UC_HOOK_MEM_WRITE_UNMAPPED : __hook_invalid_mem,
            uc.UC_HOOK_MEM_FETCH_UNMAPPED : __hook_invalid_mem,
            uc.UC_HOOK_MEM_READ_PROT      : __hook_invalid_mem,
            uc.UC_HOOK_MEM_WRITE_PROT     : __hook_invalid_mem,
            uc.UC_HOOK_MEM_FETCH_PROT     : __hook_invalid_mem,
            uc.UC_HOOK_MEM_READ           : __hook_mem,
            uc.UC_HOOK_MEM_WRITE          : __hook_mem,
            uc.UC_HOOK_MEM_FETCH          : __hook_mem,
            # uc.UC_HOOK_MEM_READ_AFTER
            uc.UC_HOOK_INSN_INVALID       : __hook_invalid_insn,
            uc.UC_HOOK_EDGE_GENERATED     : __hook_edge_gen,
            uc.UC_HOOK_TCG_OPCODE         : __hook_tcg_opcode
        }

But the defination is in const...

[wtdcode]

@elicn Would you like to add this? Or I can also help.

[elicn]

Sure, let me look into this.

[DynamicLoader]

Great to hear that! I'd compile it and try, but something just wrong....
I had set tlb mode to virtual and register the callback, like below:

def hook_tlb_fill(uc : Uc, access, address, size, value, user_data):
    print(">>> TLB Fill at 0x%x, data size = %u" % (address, size))
    uc.ctl_flush_tb()

# <Some other init>

mu = Uc(UC_ARCH_RISCV, UC_MODE_64)
mu.ctl_set_tlb_mode(UC_TLB_VIRTUAL)
mu.hook_add(UC_HOOK_TLB_FILL, hook_tlb_fill)

# <some init of memory  and registers>

mu.emu_start(entry_point, code_end)

And it just throw:

Traceback (most recent call last):
  File "main.py", line 172, in <module>
    mu.emu_start(entry_point, code_end)
  File ".conda\Lib\site-packages\unicorn\unicorn_py3\unicorn.py", line 768, in emu_start
    raise UcError(status)
unicorn.unicorn_py3.unicorn.UcError: Unhandled CPU exception (UC_ERR_EXCEPTION)

What should I do, please?

[elicn]

What if you remove the hook? Is it still happening?

[DynamicLoader]

No, it only happens when VIRTUAL and hook set.

[PhilippTakacs]

Your hook doesn't return a mapping, so the emulation will cause a pagefault.

Also why do want to clean the translation buffer?

[DynamicLoader]

I just test to see if it's needed to clear. (Learned the function from the wiki). I dont get any doc about what should the function return....

[wtdcode]

You should have a check for this file: https://github.com/unicorn-engine/unicorn/blob/master/samples/sample_mmu.c

[DynamicLoader]

OK. But the problem seems to happen before the callback. It should call the callback at least one time, but i didnt see any output from the callback. I'd give a min PoC if needed.

[DynamicLoader]

I'd found the problem. Please see the comment on the commit here.

Also I used a wrong callback proto so it didn't get called. :-)

[elicn]

Ah, snap.. I missed this.
Let me push a fix later on today.

[DynamicLoader]

But there is still something strange. When I just set the paddr = vaddr and always return true in tlb callback, the first time it success and executed, but the second time it will throw UC_ERR_EXCEPTION. If not register the callback, all things go normally.

I wonder what does the default callback (lets said it, i didn't dig into the source code) do when set VIRTUAL but not registering an callback.

Edit: I had read the source code and it just set paddr to the page of vaddr directly, the same as what I do, if not registering the callback...

[PhilippTakacs]

But there is still something strange. When I just set the paddr = vaddr and always return true in tlb callback, the first time it success and executed, but the second time it will throw UC_ERR_EXCEPTION. If not register the callback, all things go normally.

Can you provide a full example for this behavior?

[DynamicLoader]

Here is a min example:

from unicorn import *
from unicorn.riscv_const import *

# The program in C; no read from memory after optimization
"""
int a[4096];
int b[4096];
int c[4096];

int start(){
    for(int i =0;i<4096;i++){
        a[i] = i;
        b[i] = i*2;
        c[i] = a[i] + b[i];
    }
    return 0;
}
"""

# Compiled with 'riscv64-unknown-elf-gcc -nostdlib -static -nostartfiles -O1'
# LD: .text = 0x2000, .stack (end) = 0x13000 (0x4000)
inst = b"-f\x13\x06\x06\x00\x8df\x93\x86\x06\x00\x1dg\x13\x07\x07\x00\x81E\x81G\x05h\x1c\xc2\x1b\x95\x17\x00\x88\xc2\x0c\xc3\x85'\x11\x06\x91\x06\x8d%\x11\x07\xe3\x96\x07\xff\x01E\x82\x80"

mu = Uc(UC_ARCH_RISCV, UC_MODE_RISCV64)
mu.mem_map(0x2000, 0x11000, UC_PROT_ALL) # Code + Data
mu.mem_write(0x2000, inst)

mu.reg_write(UC_RISCV_REG_RA, 0x2000 + len(inst)) # End of code
mu.reg_write(UC_RISCV_REG_SP, 0x13000) # Stack base

mu.hook_add(UC_HOOK_CODE, lambda uc, address, size, user_data: print("[C] 0x%x (0x%x)" %(address, size)))

def hook_tlb_fill(uc : Uc, vaddr, memtype, value, user_data) -> bool:
    memtype_name_map = {
        UC_MEM_FETCH: "FETCH",
        UC_MEM_READ: "READ",
        UC_MEM_WRITE: "WRITE",
        UC_MEM_READ_UNMAPPED: "READ_UNMAPPED",
        UC_MEM_WRITE_UNMAPPED: "WRITE_UNMAPPED",
        UC_MEM_FETCH_UNMAPPED: "FETCH_UNMAPPED",
        UC_MEM_READ_PROT: "READ_PROT",
        UC_MEM_WRITE_PROT: "WRITE_PROT",
        UC_MEM_FETCH_PROT: "FETCH_PROT",
        UC_MEM_READ_AFTER: "READ_AFTER",
    }
    print(">>> TLB Fill at 0x%x with memtype %s" % (vaddr, memtype_name_map[memtype]))
    value.paddr = vaddr & ~0xFFF
    value.perm = UC_PROT_ALL
    print(">>> TLB Fill success: paddr = 0x%x, perm = %d" % (value.paddr, value.perm))
    return True

mu.ctl_set_tlb_mode(UC_TLB_VIRTUAL)
mu.hook_add(UC_HOOK_TLB_FILL, hook_tlb_fill)

mu.emu_start(0x2000, 0x2000 + len(inst))

Sample output:

>>> TLB Fill at 0x2000 with memtype FETCH
>>> TLB Fill success: paddr = 0x2000, perm = 7
[C] 0x2000 (0x2)
[C] 0x2002 (0x4)
[C] 0x2006 (0x2)
[C] 0x2008 (0x4)
[C] 0x200c (0x2)
[C] 0x200e (0x4)
[C] 0x2012 (0x2)
[C] 0x2014 (0x2)
[C] 0x2016 (0x2)
[C] 0x2018 (0x2)
>>> TLB Fill at 0xb000 with memtype WRITE
>>> TLB Fill success: paddr = 0xb000, perm = 7
Traceback (most recent call last):
  File "min.py", line 55, in <module>
    mu.emu_start(0x2000, 0x2000 + len(inst))
  File ".conda\Lib\site-packages\unicorn\unicorn_py3\unicorn.py", line 768, in emu_start
    raise UcError(status)
unicorn.unicorn_py3.unicorn.UcError: Unhandled CPU exception (UC_ERR_EXCEPTION)

Running on Windows 11 23H2 with 3.11.10, unicorn build dev.

[DynamicLoader]

Any Idea?

[DynamicLoader]

Solved. Mistake in 'perm'. should be 'perms'.