Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/evm: implement input txs via rlp in t8n tool #23138

Merged
merged 3 commits into from
Aug 7, 2021
Merged

cmd/evm: implement input txs via rlp in t8n tool #23138

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b2358a3
cmd/evm: implement input txs via rlp in t8n tool
holiman May 28, 2021
396a768
cmd/evm: fix typo
holiman Jul 27, 2021
071174c
internal/t8ntool: handle rlp txs via stdin
holiman Jul 28, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
117 changes: 93 additions & 24 deletions cmd/evm/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,15 @@ The `evm t8n` tool is a stateless state transition utility. It is a utility
which can

1. Take a prestate, including
- Accounts,
- Block context information,
- Previous blockshashes (*optional)
- Accounts,
- Block context information,
- Previous blockshashes (*optional)
2. Apply a set of transactions,
3. Apply a mining-reward (*optional),
4. And generate a post-state, including
- State root, transaction root, receipt root,
- Information about rejected transactions,
- Optionally: a full or partial post-state dump
- State root, transaction root, receipt root,
- Information about rejected transactions,
- Optionally: a full or partial post-state dump

## Specification

Expand All @@ -37,6 +37,8 @@ Command line params that has to be supported are
--output.result result Determines where to put the result (stateroot, txroot etc) of the post-state.
`stdout` - into the stdout output
`stderr` - into the stderr output
--output.body value If set, the RLP of the transactions (block body) will be written to this file.
--input.txs stdin stdin or file name of where to find the transactions to apply. If the file prefix is '.rlp', then the data is interpreted as an RLP list of signed transactions.The '.rlp' format is identical to the output.body format. (default: "txs.json")
--state.fork value Name of ruleset to use.
--state.chainid value ChainID to use (default: 1)
--state.reward value Mining reward. Set to -1 to disable (default: 0)
Expand Down Expand Up @@ -110,7 +112,10 @@ Two resulting files:
}
],
"rejected": [
1
{
"index": 1,
"error": "nonce too low: address 0x8A8eAFb1cf62BfBeb1741769DAE1a9dd47996192, tx: 0 state: 1"
}
]
}
```
Expand Down Expand Up @@ -156,7 +161,10 @@ Output:
}
],
"rejected": [
1
{
"index": 1,
"error": "nonce too low: address 0x8A8eAFb1cf62BfBeb1741769DAE1a9dd47996192, tx: 0 state: 1"
}
]
}
}
Expand All @@ -168,9 +176,9 @@ Mining rewards and ommer rewards might need to be added. This is how those are a

- `block_reward` is the block mining reward for the miner (`0xaa`), of a block at height `N`.
- For each ommer (mined by `0xbb`), with blocknumber `N-delta`
- (where `delta` is the difference between the current block and the ommer)
- The account `0xbb` (ommer miner) is awarded `(8-delta)/ 8 * block_reward`
- The account `0xaa` (block miner) is awarded `block_reward / 32`
- (where `delta` is the difference between the current block and the ommer)
- The account `0xbb` (ommer miner) is awarded `(8-delta)/ 8 * block_reward`
- The account `0xaa` (block miner) is awarded `block_reward / 32`

To make `state_t8n` apply these, the following inputs are required:

Expand Down Expand Up @@ -220,7 +228,7 @@ Output:
### Future EIPS

It is also possible to experiment with future eips that are not yet defined in a hard fork.
Example, putting EIP-1344 into Frontier:
Example, putting EIP-1344 into Frontier:
```
./evm t8n --state.fork=Frontier+1344 --input.pre=./testdata/1/pre.json --input.txs=./testdata/1/txs.json --input.env=/testdata/1/env.json
```
Expand All @@ -229,41 +237,102 @@ Example, putting EIP-1344 into Frontier:

The `BLOCKHASH` opcode requires blockhashes to be provided by the caller, inside the `env`.
If a required blockhash is not provided, the exit code should be `4`:
Example where blockhashes are provided:
Example where blockhashes are provided:
```
./evm t8n --input.alloc=./testdata/3/alloc.json --input.txs=./testdata/3/txs.json --input.env=./testdata/3/env.json --trace
./evm --verbosity=1 t8n --input.alloc=./testdata/3/alloc.json --input.txs=./testdata/3/txs.json --input.env=./testdata/3/env.json --trace
INFO [07-27|11:53:40.960] Trie dumping started root=b7341d..857ea1
INFO [07-27|11:53:40.960] Trie dumping complete accounts=3 elapsed="103.298µs"
INFO [07-27|11:53:40.960] Wrote file file=alloc.json
INFO [07-27|11:53:40.960] Wrote file file=result.json

```

```
cat trace-0-0x72fadbef39cd251a437eea619cfeda752271a5faaaa2147df012e112159ffb81.jsonl | grep BLOCKHASH -C2
```
```
{"pc":0,"op":96,"gas":"0x5f58ef8","gasCost":"0x3","memory":"0x","memSize":0,"stack":[],"returnStack":[],"returnData":"0x","depth":1,"refund":0,"opName":"PUSH1","error":""}
{"pc":2,"op":64,"gas":"0x5f58ef5","gasCost":"0x14","memory":"0x","memSize":0,"stack":["0x1"],"returnStack":[],"returnData":"0x","depth":1,"refund":0,"opName":"BLOCKHASH","error":""}
{"pc":3,"op":0,"gas":"0x5f58ee1","gasCost":"0x0","memory":"0x","memSize":0,"stack":["0xdac58aa524e50956d0c0bae7f3f8bb9d35381365d07804dd5b48a5a297c06af4"],"returnStack":[],"returnData":"0x","depth":1,"refund":0,"opName":"STOP","error":""}
{"output":"","gasUsed":"0x17","time":142709}
{"pc":0,"op":96,"gas":"0x5f58ef8","gasCost":"0x3","memory":"0x","memSize":0,"stack":[],"returnData":"0x","depth":1,"refund":0,"opName":"PUSH1","error":""}
{"pc":2,"op":64,"gas":"0x5f58ef5","gasCost":"0x14","memory":"0x","memSize":0,"stack":["0x1"],"returnData":"0x","depth":1,"refund":0,"opName":"BLOCKHASH","error":""}
{"pc":3,"op":0,"gas":"0x5f58ee1","gasCost":"0x0","memory":"0x","memSize":0,"stack":["0xdac58aa524e50956d0c0bae7f3f8bb9d35381365d07804dd5b48a5a297c06af4"],"returnData":"0x","depth":1,"refund":0,"opName":"STOP","error":""}
{"output":"","gasUsed":"0x17","time":156276}
```

In this example, the caller has not provided the required blockhash:
```
./evm t8n --input.alloc=./testdata/4/alloc.json --input.txs=./testdata/4/txs.json --input.env=./testdata/4/env.json --trace
```
```
ERROR(4): getHash(3) invoked, blockhash for that block not provided
```
Error code: 4

### Chaining

Another thing that can be done, is to chain invocations:
```
./evm t8n --input.alloc=./testdata/1/alloc.json --input.txs=./testdata/1/txs.json --input.env=./testdata/1/env.json --output.alloc=stdout | ./evm t8n --input.alloc=stdin --input.env=./testdata/1/env.json --input.txs=./testdata/1/txs.json
INFO [01-21|22:41:22.963] rejected tx index=1 hash=0557ba..18d673 from=0x8A8eAFb1cf62BfBeb1741769DAE1a9dd47996192 error="nonce too low: address 0x8A8eAFb1cf62BfBeb1741769DAE1a9dd47996192, tx: 0 state: 1"
INFO [01-21|22:41:22.966] rejected tx index=0 hash=0557ba..18d673 from=0x8A8eAFb1cf62BfBeb1741769DAE1a9dd47996192 error="nonce too low: address 0x8A8eAFb1cf62BfBeb1741769DAE1a9dd47996192, tx: 0 state: 1"
INFO [01-21|22:41:22.967] rejected tx index=1 hash=0557ba..18d673 from=0x8A8eAFb1cf62BfBeb1741769DAE1a9dd47996192 error="nonce too low: address 0x8A8eAFb1cf62BfBeb1741769DAE1a9dd47996192, tx: 0 state: 1"
INFO [07-27|11:53:41.049] rejected tx index=1 hash=0557ba..18d673 from=0x8A8eAFb1cf62BfBeb1741769DAE1a9dd47996192 error="nonce too low: address 0x8A8eAFb1cf62BfBeb1741769DAE1a9dd47996192, tx: 0 state: 1"
INFO [07-27|11:53:41.050] Trie dumping started root=84208a..ae4e13
INFO [07-27|11:53:41.050] Trie dumping complete accounts=3 elapsed="59.412µs"
INFO [07-27|11:53:41.050] Wrote file file=result.json
INFO [07-27|11:53:41.051] rejected tx index=0 hash=0557ba..18d673 from=0x8A8eAFb1cf62BfBeb1741769DAE1a9dd47996192 error="nonce too low: address 0x8A8eAFb1cf62BfBeb1741769DAE1a9dd47996192, tx: 0 state: 1"
INFO [07-27|11:53:41.051] rejected tx index=1 hash=0557ba..18d673 from=0x8A8eAFb1cf62BfBeb1741769DAE1a9dd47996192 error="nonce too low: address 0x8A8eAFb1cf62BfBeb1741769DAE1a9dd47996192, tx: 0 state: 1"
INFO [07-27|11:53:41.052] Trie dumping started root=84208a..ae4e13
INFO [07-27|11:53:41.052] Trie dumping complete accounts=3 elapsed="45.734µs"
INFO [07-27|11:53:41.052] Wrote file file=alloc.json
INFO [07-27|11:53:41.052] Wrote file file=result.json

```
What happened here, is that we first applied two identical transactions, so the second one was rejected.
What happened here, is that we first applied two identical transactions, so the second one was rejected.
Then, taking the poststate alloc as the input for the next state, we tried again to include
the same two transactions: this time, both failed due to too low nonce.

In order to meaningfully chain invocations, one would need to provide meaningful new `env`, otherwise the
actual blocknumber (exposed to the EVM) would not increase.

### Transactions in RLP form

It is possible to provide already-signed transactions as input to, using an `input.txs` which ends with the `rlp` suffix.
The input format for RLP-form transactions is _identical_ to the _output_ format for block bodies. Therefore, it's fully possible
to use the evm to go from `json` input to `rlp` input.

The following command takes **json** the transactions in `./testdata/13/txs.json` and signs them. After execution, they are output to `signed_txs.rlp`.:
```
./evm t8n --state.fork=London --input.alloc=./testdata/13/alloc.json --input.txs=./testdata/13/txs.json --input.env=./testdata/13/env.json --output.result=alloc_jsontx.json --output.body=signed_txs.rlp
INFO [07-27|11:53:41.124] Trie dumping started root=e4b924..6aef61
INFO [07-27|11:53:41.124] Trie dumping complete accounts=3 elapsed="94.284µs"
INFO [07-27|11:53:41.125] Wrote file file=alloc.json
INFO [07-27|11:53:41.125] Wrote file file=alloc_jsontx.json
INFO [07-27|11:53:41.125] Wrote file file=signed_txs.rlp

```

The `output.body` is the rlp-list of transactions, encoded in hex and placed in a string a'la `json` encoding rules:
```
cat signed_txs.rlp
"0xf8d2b86702f864010180820fa08284d09411111111111111111111111111111111111111118080c001a0b7dfab36232379bb3d1497a4f91c1966b1f932eae3ade107bf5d723b9cb474e0a06261c359a10f2132f126d250485b90cf20f30340801244a08ef6142ab33d1904b86702f864010280820fa08284d09411111111111111111111111111111111111111118080c080a0d4ec563b6568cd42d998fc4134b36933c6568d01533b5adf08769270243c6c7fa072bf7c21eac6bbeae5143371eef26d5e279637f3bd73482b55979d76d935b1e9"
```

We can use `rlpdump` to check what the contents are:
```
rlpdump -hex $(cat signed_txs.rlp | jq -r )
[
02f864010180820fa08284d09411111111111111111111111111111111111111118080c001a0b7dfab36232379bb3d1497a4f91c1966b1f932eae3ade107bf5d723b9cb474e0a06261c359a10f2132f126d250485b90cf20f30340801244a08ef6142ab33d1904,
02f864010280820fa08284d09411111111111111111111111111111111111111118080c080a0d4ec563b6568cd42d998fc4134b36933c6568d01533b5adf08769270243c6c7fa072bf7c21eac6bbeae5143371eef26d5e279637f3bd73482b55979d76d935b1e9,
]
```
Now, we can now use those (or any other already signed transactions), as input, like so:
```
./evm t8n --state.fork=London --input.alloc=./testdata/13/alloc.json --input.txs=./signed_txs.rlp --input.env=./testdata/13/env.json --output.result=alloc_rlptx.json
INFO [07-27|11:53:41.253] Trie dumping started root=e4b924..6aef61
INFO [07-27|11:53:41.253] Trie dumping complete accounts=3 elapsed="128.445µs"
INFO [07-27|11:53:41.253] Wrote file file=alloc.json
INFO [07-27|11:53:41.255] Wrote file file=alloc_rlptx.json

```

You might have noticed that the results from these two invocations were stored in two separate files.
And we can now finally check that they match.
```
cat alloc_jsontx.json | jq .stateRoot && cat alloc_rlptx.json | jq .stateRoot
"0xe4b924a6adb5959fccf769d5b7bb2f6359e26d1e76a2443c5a91a36d826aef61"
"0xe4b924a6adb5959fccf769d5b7bb2f6359e26d1e76a2443c5a91a36d826aef61"
```
6 changes: 4 additions & 2 deletions cmd/evm/internal/t8ntool/flags.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,8 +79,10 @@ var (
Value: "env.json",
}
InputTxsFlag = cli.StringFlag{
Name: "input.txs",
Usage: "`stdin` or file name of where to find the transactions to apply.",
Name: "input.txs",
Usage: "`stdin` or file name of where to find the transactions to apply. " +
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can add a comment that RLP list over stdin is not supported

echo signed_txs.rlp | ./evm t8n --state.fork=London --input.alloc=./testdata/13/alloc.json --input.txs=stdin --input.env=./testdata/13/env.json --output.result=alloc_rlptx.json
ERROR(10): failed unmarshaling stdin: invalid character 's' looking for beginning of value

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heh. Try cat instead of echo :)

Copy link
Member

@MariusVanDerWijden MariusVanDerWijden Jul 27, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm doofus, but the general point still stands. We can't read RLP from stdin, because L147 tries to decode into an input object

cat signed_txs.rlp | ./evm t8n --state.fork=London --input.alloc=./testdata/13/alloc.json --input.txs=stdin --input.env=./testdata/13/env.json --output.result=alloc_rlptx.json
ERROR(10): failed unmarshaling stdin: json: cannot unmarshal string into Go value of type t8ntool.input

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, good catch! Fixing...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed this in my last commit. One can't just take the output and feed to the input, since the input can contain many things. Therefore, you need to envelop the leaf json into an object, and call it txsRlp:

[user@work evm]$ cat signed_txs.rlp.json| jq . 
{
  "txsRlp": "0xf8d2b86702f864010180820fa08284d09411111111111111111111111111111111111111118080c001a0b7dfab36232379bb3d1497a4f91c1966b1f932eae3ade107bf5d723b9cb474e0a06261c359a10f2132f126d250485b90cf20f30340801244a08ef6142ab33d1904b86702f864010280820fa08284d09411111111111111111111111111111111111111118080c080a0d4ec563b6568cd42d998fc4134b36933c6568d01533b5adf08769270243c6c7fa072bf7c21eac6bbeae5143371eef26d5e279637f3bd73482b55979d76d935b1e9"
}
[user@work evm]$ cat signed_txs.rlp.json| ./evm t8n --state.fork=London --input.alloc=./testdata/13/alloc.json --input.txs=stdin --input.env=./testdata/13/env.json --output.result=alloc_rlptx.json
INFO [07-28|09:56:08.806] Trie dumping started                     root=672261..31e1e5
INFO [07-28|09:56:08.806] Trie dumping complete                    accounts=3 elapsed="211.905µs"
INFO [07-28|09:56:08.806] Wrote file                               file=alloc.json
INFO [07-28|09:56:08.810] Wrote file                               file=alloc_rlptx.json

"If the file prefix is '.rlp', then the data is interpreted as an RLP list of signed transactions." +
"The '.rlp' format is identical to the output.body format.",
Value: "txs.json",
}
RewardFlag = cli.Int64Flag{
Expand Down
43 changes: 40 additions & 3 deletions cmd/evm/internal/t8ntool/transition.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ import (
"math/big"
"os"
"path"
"strings"

"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/common/hexutil"
Expand Down Expand Up @@ -72,6 +73,7 @@ type input struct {
Alloc core.GenesisAlloc `json:"alloc,omitempty"`
Env *stEnv `json:"env,omitempty"`
Txs []*txWithKey `json:"txs,omitempty"`
TxRlp string `json:"txsRlp,omitempty"`
}

func Main(ctx *cli.Context) error {
Expand Down Expand Up @@ -199,11 +201,44 @@ func Main(ctx *cli.Context) error {
}
defer inFile.Close()
decoder := json.NewDecoder(inFile)
if err := decoder.Decode(&txsWithKeys); err != nil {
return NewError(ErrorJson, fmt.Errorf("failed unmarshaling txs-file: %v", err))
if strings.HasSuffix(txStr, ".rlp") {
var body hexutil.Bytes
if err := decoder.Decode(&body); err != nil {
return err
}
var txs types.Transactions
if err := rlp.DecodeBytes(body, &txs); err != nil {
return err
}
for _, tx := range txs {
txsWithKeys = append(txsWithKeys, &txWithKey{
key: nil,
tx: tx,
})
}
} else {
if err := decoder.Decode(&txsWithKeys); err != nil {
return NewError(ErrorJson, fmt.Errorf("failed unmarshaling txs-file: %v", err))
}
}
} else {
txsWithKeys = inputData.Txs
if len(inputData.TxRlp) > 0 {
// Decode the body of already signed transactions
body := common.FromHex(inputData.TxRlp)
var txs types.Transactions
if err := rlp.DecodeBytes(body, &txs); err != nil {
return err
}
for _, tx := range txs {
txsWithKeys = append(txsWithKeys, &txWithKey{
key: nil,
tx: tx,
})
}
} else {
// JSON encoded transactions
txsWithKeys = inputData.Txs
}
}
// We may have to sign the transactions.
signer := types.MakeSigner(chainConfig, big.NewInt(int64(prestate.Env.Number)))
Expand Down Expand Up @@ -365,13 +400,15 @@ func dispatchOutput(ctx *cli.Context, baseDir string, result *ExecutionResult, a
return NewError(ErrorJson, fmt.Errorf("failed marshalling output: %v", err))
}
os.Stdout.Write(b)
os.Stdout.Write([]byte("\n"))
}
if len(stdErrObject) > 0 {
b, err := json.MarshalIndent(stdErrObject, "", " ")
if err != nil {
return NewError(ErrorJson, fmt.Errorf("failed marshalling output: %v", err))
}
os.Stderr.Write(b)
os.Stderr.Write([]byte("\n"))
}
return nil
}
23 changes: 23 additions & 0 deletions cmd/evm/testdata/13/alloc.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
{
"0x1111111111111111111111111111111111111111" : {
"balance" : "0x010000000000",
"code" : "0xfe",
"nonce" : "0x01",
"storage" : {
}
},
"0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b" : {
"balance" : "0x010000000000",
"code" : "0x",
"nonce" : "0x01",
"storage" : {
}
},
"0xd02d72e067e77158444ef2020ff2d325f929b363" : {
"balance" : "0x01000000000000",
"code" : "0x",
"nonce" : "0x01",
"storage" : {
}
}
}
12 changes: 12 additions & 0 deletions cmd/evm/testdata/13/env.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{
"currentCoinbase" : "0x2adc25665018aa1fe0e6bc666dac8fc2697ff9ba",
"currentDifficulty" : "0x020000",
"currentNumber" : "0x01",
"currentTimestamp" : "0x079e",
"previousHash" : "0xcb23ee65a163121f640673b41788ee94633941405f95009999b502eedfbbfd4f",
"currentGasLimit" : "0x40000000",
"currentBaseFee" : "0x036b",
"blockHashes" : {
"0" : "0xcb23ee65a163121f640673b41788ee94633941405f95009999b502eedfbbfd4f"
}
}
4 changes: 4 additions & 0 deletions cmd/evm/testdata/13/readme.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
## Input transactions in RLP form

This testdata folder is used to examplify how transaction input can be provided in rlp form.
Please see the README in `evm` folder for how this is performed.
34 changes: 34 additions & 0 deletions cmd/evm/testdata/13/txs.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
[
{
"input" : "0x",
"gas" : "0x84d0",
"nonce" : "0x1",
"to" : "0x1111111111111111111111111111111111111111",
"value" : "0x0",
"v" : "0x0",
"r" : "0x0",
"s" : "0x0",
"secretKey" : "0x41f6e321b31e72173f8ff2e292359e1862f24fba42fe6f97efaf641980eff298",
"chainId" : "0x1",
"type" : "0x2",
"maxFeePerGas" : "0xfa0",
"maxPriorityFeePerGas" : "0x0",
"accessList" : []
},
{
"input" : "0x",
"gas" : "0x84d0",
"nonce" : "0x2",
"to" : "0x1111111111111111111111111111111111111111",
"value" : "0x0",
"v" : "0x0",
"r" : "0x0",
"s" : "0x0",
"secretKey" : "0x41f6e321b31e72173f8ff2e292359e1862f24fba42fe6f97efaf641980eff298",
"chainId" : "0x1",
"type" : "0x2",
"maxFeePerGas" : "0xfa0",
"maxPriorityFeePerGas" : "0x0",
"accessList" : []
}
]
Loading