Improve aleth-vm tool interface

[chfast]

From @holiman:

It's here: ethereum/tests#249 (comment) . But I'll add my tldr; (which also became quite long):

Input

The evm should take the following inputs:

• --code <code> - code to be executed.
• --codeFile <file> - file containing code to be executed. Sometimes really large chunks of input cannot be passed through bash.
• --gas <int>
• --price <int>
• --sender <address> - address of ORIGIN
• --receiver <address - address of ADDRESS
• --input <code> : CALLDATA
• --value <int>
• --json - boolean flag, output json output for each opcode or not (it's useful to disable json when benchmarking)
• --nomemory - disable showing the full memory output for each op
• --create - if specified, it's executed as initcode
• --prestate - a chain specification, the same one that the client normally would use.

Basically, the evm should be able to run things very simply, like so:

#evm --code 6040 --json run
{"pc":0,"op":96,"gas":"0x2540be400","gasCost":"0x3","memory":"0x","memSize":0,"stack":[],"depth":1,"error":null,"opName":"PUSH1"}
{"pc":2,"op":0,"gas":"0x2540be3fd","gasCost":"0x0","memory":"0x","memSize":0,"stack":["0x40"],"depth":1,"error":null,"opName":"STOP"}
{"output":"","gasUsed":"0x3","time":141485}

But it should also be able to reconstruct an actual on-chain transaction, with complex options including prestate, where no code is passed, since it's already been showed into the prestate:

./evm --prestate /home/martin/workspace/evmlab/output//0xd6d519-genesis-geth_wq38zsy5.json --gas 150000 --sender 0x69ea6b31ef305d6b99bb2d4c9d99456fa108b02a --receiver 0xb97048628db6b661d4c2aa833e95dbe1a905b280 --input a9059cbb0000000000000000000000008eef795fd9150f118bddeca556a5a2a2438ab865000000000000000000000000000000000000000000000081ebd8ffd6b2a58000 --json run

Output

The evm should output a json object for each operation. Example:

{"pc":0,"op":96,"gas":"0x2540be400","gasCost":"0x3","memory":"0x","memSize":0,"stack":[],"depth":1,"error":null,"opName":"PUSH1"}

Required: pc, op, gas, stack, depth
Optional: opName, gasCost, error

The stack, memory and memSize are the values before execution of the op.

At the end of execution, some summarical info is good, e.g.

{"output":"","gasUsed":"0x3","time":141485}

When errors occur, geth and parity handles them differently.

Minor changes to how things work is ok, we can handle discrepancies in format and minor quirks.

[turing228]

I'll do it!

[AnthonyBroadCrawford]

@chfast Mind if I run with this issue?

[chfast]

@AnthonyBroadCrawford if you still interested, yes.

[AnthonyBroadCrawford]

@chfast Actually, I am. I've been super busy lately but I have a nice month where I have the capacity to complete this. I'll provide another update by the end of the week. Thank you so much for the nudge.

[josephnicholas]

@chfast quite confused, should this be an aleth change or the ethereum/test change? Or both Add interface and add test?

[chfast]

This is about aleth-vm tool here. And it's compared to evm tool from go-ethereum.

[josephnicholas]

@chfast okay, I would love to work on this, but where could I ask some questions, if I'm stuck on something. The description above is basically the feature to add in aleth-vm am I correct?

[gumb0]

@josephnicholas Feel free to reach us at https://gitter.im/ethereum/aleth (better tag either me or @chfast, otherwise we might miss it)

These features I think are already present in some form in aleth-vm (maybe except --codeFile), the task is mostly about reworking the CLI interface and output format.

[josephnicholas]

@gumb0 I started on this yesterday, I tried to compare aleth-vm options with evm(from geth). There were some options that were in evm and not in aleth-vm.

So, to ask once again on the coverage of this task, should I just implement --codeFile or some others? Like receiver, nomemory?

Maybe I can assume codeFile is the priority? then, another thing is,(since I am quite new to Ethereum), is it a direct migration from geth EVM code?

[holiman]

Update from my side:
What would be really useful, more than specifying code and codefie, would be to have statetest. Geth evm and parity evmbin both have this. A generalized state test incorporates

• prestate (code, balance, nonce etc)
• chain configuration (fork blocks)
• environment info (block number, timestamp etc)

This is what we use to do fuzzing, and is easier to use as a canonical representation of a prestate than trying to provide each client with a custom genesis, code etc.

[holiman]

the option nomemory is useful, since memory can be a megabyte, and outputting a megabyte at every op becomes very unwieldy. I'd even go so far as to say that the default might be without memory (having memorySize is important, though).

One important aspect is that the execution should not hold on to the data during the entire execution, and spit it out as a json array. Instead, it should output jsonl, one line per op.

[gumb0]

So, to ask once again on the coverage of this task, should I just implement --codeFile or some others? Like receiver, nomemory?

@josephnicholas Yes, please start with anything. If it gets too big of a change, it's better to finish first some subset of requirements in the description.

Maybe I can assume codeFile is the priority? then, another thing is,(since I am quite new to Ethereum), is it a direct migration from geth EVM code?

You don't need to consult with geth code for that. --codeFile just reads the file contents (there's a helper for this somewhere) and passes it to the same place as --code parameter.

[gumb0]

@holiman Thanks for the input, I'd say running a state test would be another, somewhat more complicated task. It's the same format as the tests in https://github.com/ethereum/tests/tree/develop/GeneralStateTests, right?

This issue is more about normalizing the interface of existing functionality of aleth-vm.

[holiman]

Yes. An example testcase and the output from geth and parity respectively can be found here: https://github.com/holiman/goevmlab/tree/master/evms/testdata

[gumb0]

You don't need to consult with geth code for that. --codeFile just reads the file contents (there's a helper for this somewhere) and passes it to the same place as --code parameter.

@josephnicholas Actually aleth-vm already does that, just not with --codeFile option, but with aleth-vm <filename>. This needs to be changed.

[josephnicholas]

@gumb0 Thanks, I will prioritize --codeFile first then work my way up.

[gumb0]

Hey @josephnicholas do you still plan to work on this?

[josephnicholas]

@gumb0 yes.. went on vacation will continue working on this when I get back

[gumb0]

After the changes in #5839 the output from aleth-vm --json should be already similar to the one suggested here.

[josephnicholas]

@gumb0 my apologies on not giving anything on what I promised, there is no excuse but I have been really busy with my day job. :(

[holiman]

@gumb0 awesome! I don't suppose there are releases for download to try out somewhere? Or maybe docker images? EDIT docker image location in readme, doh

[chfast]

@holiman Looks like aleth-vm is not part of the docker image. I can fix this soon.

We can also make a snapshot release so binaries will be available on GitHub.

[holiman]

I actually did manage to build it. However:

[user@work build]$ ./aleth-vm/aleth-vm --code 6040
Gas used: 3 (+21000 for transaction, -0 refunded)
Output: 
0 logs.
2 operations in 0.000113 seconds.
Maximum memory usage: 0 bytes
Expensive operations:
[user@work build]$ ./aleth-vm/aleth-vm --code 6040 --json
--code argument overwritten by input file --json
Gas used: 0 (+21000 for transaction, -0 refunded)
Output: 
0 logs.
0 operations in 0 seconds.
Maximum memory usage: 0 bytes
Expensive operations:

git version 43780e1cfaad9d887b5cd730a5df6abcea81f302

[holiman]

The stdout contains non-jsonl Test Case "customTestSuite":

Test Case "customTestSuite": 
{"depth":"1","gas":"2956484","gasCost":"1","memSize":0,"memory":[],"op":91,"opName":"JUMPDEST","pc":"0","stack":[],"storage":{}}
{"depth":"1","gas":"2956483","gasCost":"2","memSize":0,"memory":[],"op":61,"opName":"RETURNDATASIZE","pc":"1","stack":[]}
{"depth":"1","gas":"2956481","gasCost":"2","memSize":0,"memory":[],"op":48,"opName":"ADDRESS","pc":"2","stack":["0x00"]}
{"depth":"1","gas":"2956479","gasCost":"3","memSize":0,"memory":[],"op":172,"opName":"PUSHC","pc":"3","stack":["0x00","0xc94f5374fce5edbc8e2a8697c15331677e6ebf0b"]}
{"depth":"1","gas":"2956476","gasCost":"3","memSize":0,"memory":[],"op":172,"opName":"PUSHC","pc":"30","stack":["0x00","0xc94f5374fce5edbc8e2a8697c15331677e6ebf0b","0xc940b5f2046740058558468f238b85db7f6bbe3f3d51e92a3e32"]}
{"depth":"1","gas":"2956473","gasCost":"3","memSize":0,"memory":[],"op":83,"opName":"MSTORE8","pc":"40","stack":["0x00","0xc94f5374fce5edbc8e2a8697c15331677e6ebf0b","0xc940b5f2046740058558468f238b85db7f6bbe3f3d51e92a3e32","0xb7f7c4147541c695f3"]}
{"output": "", "gasUsed": "3000000", "error": "OutOfGas"}
{"stateRoot": "a2b3391f7a85bf1ad08dc541a1b99da3c591c156351391f26ec88c557ff12134"}

It's not a biggie, I can easily ignore that, just FYI

[chfast]

The stdout contains non-jsonl Test Case "customTestSuite":

@gumb0 @winsvega is there a way to remove this line from output? I couldn't find what controls the output of this.

[gumb0]

@holiman sorry about confusion, for aleth-vm the command line is actually like this:

> aleth-vm/aleth-vm --code 6040 --flat --mnemonics trace
{"depth":"1","gas":"9223372036854775807","gasCost":"3","memSize":0,"memory":[],"op":96,"opName":"PUSH1","pc":"0","stack":[],"storage":{}}
{"depth":"1","gas":"9223372036854775804","gasCost":"0","memSize":0,"memory":[],"op":0,"opName":"STOP","pc":"2","stack":["0x40"]}

It doesn't have summary lines in the end, and it still accumulates all lines before printing, unlike in testeth.
I can add it for aleth-vm, too, if it's needed.

[holiman]

I can add it for aleth-vm, too, if it's needed.

Thanks, but that's fine, testeth works for me

[josephnicholas]

Good day @gumb0 and @chfast so based on the original list how many are done and are open for development?

• code
• codefile
• gas
• price same as gas-price
• sender
• receiver same as origin
• input
• value
• json
• nomemory
• create
• prestate

Pleas correct me if I am wrong on the list, but the remaining stuff to worked on are the remaining unchecked ones correct?

Thanks.

[holiman]

Just fyi, that from my end, I'm happy about the current implementation via testeth, and don't need any further modifications at this point. I package everything into a statetest and exeute it, and the output format is now fine

[gumb0]

@josephnicholas We think it's not worth to spend more time on this, as further changes are not really needed by anyone. So we'll close this.