The problem with static analysis of VMP-like protectors. CODE && DATA detect. ROP problems

[Nitr0-G]

The problem lies in the obvious disadvantage of static analysis alone. I'll tell you everything now:
We take an example(MsgB) from VMP 2.13.2
Clear example: prog1.zip
VMP 2.13.2 example: prog1.vmp.exe.zip

If you go into the example with VMP, you will see that the sections with variables are empty, and API calls are encrypted

VMP 2.13.2 example:

VMP 2.13.2 RDATA

VMP 2.13.2 DATA

VMP 2.13.2 RDATA API

Clear example:

Clear RDATA

Clear DATA

Clear RDATA API

;---END

It's contains both example in RDATA API

On the first line - MessageBoxA

On the second line - ExitProcess

Also, do not pay attention to the inscription in MsgB, first I packed VMP 3.8.1, but I thought it was better to start with something old

As you can see in the case of VMP 2.13.2, the VMP loader deletes everything, and it is impossible to perform static analysis of the virtualized or mutated EntryPoint function, since our OEP is essentially virtualized/mutated. We can't even dump and start the analysis in static, since the OEP is virtualized/mutated. I'm still thinking about solving this problem. If anyone has any ideas, I will be glad if you can share it

p.s.
When trying to deobfuscate the example with VMP, LLVM deletes the entire code (it does the right thing). LLVM, or rather its optimization phases, delete all the code because they do not see the point in it due to the fact that there are no lines and WinAPI doesn't remain

[Nitr0-G]

So far I have two kinds of ideas:

Static analysis:
Write import unpacker && string decrypter for VMP 2.13.2 .

Dynamic analysis:
Trace all the code through Unicorn to the desired address. Create a virtual PE file where I will write the code trace in the .code section of the data that I touched .code section to section .data and .rdata (if they are constants) respectively. I will also copy to the section .code all graphs and functions that I don't have in a virtual PE file via hash analysis and give virtual PE file to Retdec function.

[Nitr0-G]

Also, retdec does not separate data from code if they are in the same section.

[Nitr0-G]

CODE vs DATA

I have an idea how to do it in semi-static and dynamic!

It is necessary to find the EP and the section where this whole thing is executed, just change the rights in the section from RWE to WE, so you can find the entire readable date to separate from the functions, you will just have to look at the function itself and decide whether it makes sense or not, through the analysis of the instructions of the function itself. The writable date can be found by leaving only RE and looking for what is being recorded and where, so it will be possible to find even those places where the results from the function are recorded! i.e. bool, etc. values for lifting

[Nitr0-G]

In general, after a day of reflection, I came to the conclusion that it is impossible to do this task with static analysis, and it is quite limiting. Under the retdec refinement, I chose the unicorn emulator as a universal solution, initially I thought to take pintool, but this is not a very good choice for virus processing. I am running the task under Windows, so compatibility at first will be only with PE files and x86/x64 architecture. The loader emulator for PE files is already ready.

[Nitr0-G]

To begin with, I need to go through all possible branches and assemble a full-fledged route to a certain VA, where the segment I need ends, after which I have to upload all the information to a bin file or create a virtual PE file manually, where the entire program route will be located with all the disassembled variables that were used on this route.

[Nitr0-G]

So, I also discovered a problem with ROP and incorrect lifting of the program when using ROP obfuscation
prog1MsgRet.zip

Above you can see a simple example where ROP is used when accessing WinAPI, it can also be done with any other function

[Nitr0-G]

https://github.com/Nitr0-G/DynamicRetDec
There is a small part of the tool here.. The collector will go through 5 phases, and then reset all the code to lifting via retdec