Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Partial EIP1186 eth_getProof implementation #6560

Merged
merged 21 commits into from
Mar 9, 2023
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
f891802
Define the eth_getProof structures in a common location which
mmontour1306 Jan 9, 2023
1f5e62f
Stub to return an account proof for latest block, or an error
mmontour1306 Jan 9, 2023
8131190
Add functions to request Proof generation, providing an AccountResult
mmontour1306 Jan 10, 2023
c9e3023
Stub to tell GenStructStep() that proof generation is needed
mmontour1306 Jan 10, 2023
4f25c28
Intercept nodes before they're hashed.
mmontour1306 Jan 10, 2023
a4f51ab
Implement the eth_getProof calculation.
mmontour1306 Jan 11, 2023
a706f7e
Merge remote-tracking branch 'upstream/devel' into mm/AccountProof
mmontour1306 Jan 13, 2023
a7f5e16
Rename "common" to "libcommon" to match upstream change.
mmontour1306 Jan 13, 2023
03256a2
Merge remote-tracking branch 'upstream/devel' into mm/AccountProof
mmontour1306 Jan 17, 2023
4bfe792
Merge remote-tracking branch 'upstream/devel' into mm/AccountProof
mmontour1306 Jan 19, 2023
4a82425
Merge remote-tracking branch 'upstream/devel' into mm/AccountProof
mmontour1306 Jan 20, 2023
f723867
Merge remote-tracking branch 'upstream/devel' into mm/AccountProof
mmontour1306 Jan 23, 2023
7291c46
Merge remote-tracking branch 'upstream/devel' into mm/AccountProof
mmontour1306 Jan 23, 2023
646a9a9
Merge remote-tracking branch 'upstream/devel' into mm/AccountProof
mmontour1306 Jan 25, 2023
87df787
Move the code to populate the Account fields of the proof response
mmontour1306 Jan 25, 2023
06ad495
Reverts the changes to the function signatures in hashbuilder.go,
mmontour1306 Jan 26, 2023
009a85e
Merge remote-tracking branch 'upstream/devel' into mm/AccountProof
mmontour1306 Jan 26, 2023
9f2e005
Merge remote-tracking branch 'upstream/devel' into mm/AccountProof
mmontour1306 Feb 2, 2023
10e2ef3
Merge remote-tracking branch 'upstream/devel' into mm/AccountProof
mmontour1306 Feb 17, 2023
ee505e8
Import "common".
mmontour1306 Feb 17, 2023
4f2ef0c
Clean up a couple of debug messages.
mmontour1306 Feb 17, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion cmd/rpcdaemon/commands/eth_api.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ import (
"github.com/ledgerwatch/erigon/consensus/misc"
"github.com/ledgerwatch/erigon/core/rawdb"
"github.com/ledgerwatch/erigon/core/types"
"github.com/ledgerwatch/erigon/core/types/accounts"
ethFilters "github.com/ledgerwatch/erigon/eth/filters"
"github.com/ledgerwatch/erigon/rpc"
ethapi2 "github.com/ledgerwatch/erigon/turbo/adapter/ethapi"
Expand Down Expand Up @@ -87,7 +88,7 @@ type EthAPI interface {
SendTransaction(_ context.Context, txObject interface{}) (libcommon.Hash, error)
Sign(ctx context.Context, _ libcommon.Address, _ hexutil.Bytes) (hexutil.Bytes, error)
SignTransaction(_ context.Context, txObject interface{}) (libcommon.Hash, error)
GetProof(ctx context.Context, address libcommon.Address, storageKeys []string, blockNr rpc.BlockNumber) (*interface{}, error)
GetProof(ctx context.Context, address libcommon.Address, storageKeys []string, blockNr rpc.BlockNumberOrHash) (*accounts.AccProofResult, error)
CreateAccessList(ctx context.Context, args ethapi2.CallArgs, blockNrOrHash *rpc.BlockNumberOrHash, optimizeGas *bool) (*accessListResult, error)

// Mining related (see ./eth_mining.go)
Expand Down
52 changes: 48 additions & 4 deletions cmd/rpcdaemon/commands/eth_call.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,13 +21,15 @@ import (
"github.com/ledgerwatch/erigon/core"
"github.com/ledgerwatch/erigon/core/state"
"github.com/ledgerwatch/erigon/core/types"
"github.com/ledgerwatch/erigon/core/types/accounts"
"github.com/ledgerwatch/erigon/core/vm"
"github.com/ledgerwatch/erigon/crypto"
"github.com/ledgerwatch/erigon/eth/tracers/logger"
"github.com/ledgerwatch/erigon/params"
"github.com/ledgerwatch/erigon/rpc"
"github.com/ledgerwatch/erigon/turbo/rpchelper"
"github.com/ledgerwatch/erigon/turbo/transactions"
"github.com/ledgerwatch/erigon/turbo/trie"
)

var latestNumOrHash = rpc.BlockNumberOrHashWithNumber(rpc.LatestBlockNumber)
Expand Down Expand Up @@ -301,10 +303,52 @@ func (api *APIImpl) EstimateGas(ctx context.Context, argsOrNil *ethapi2.CallArgs
return hexutil.Uint64(hi), nil
}

// GetProof not implemented
func (api *APIImpl) GetProof(ctx context.Context, address libcommon.Address, storageKeys []string, blockNr rpc.BlockNumber) (*interface{}, error) {
var stub interface{}
return &stub, fmt.Errorf(NotImplemented, "eth_getProof")
// GetProof is partially implemented; no Storage proofs; only for the latest block
func (api *APIImpl) GetProof(ctx context.Context, address libcommon.Address, storageKeys []string, blockNrOrHash rpc.BlockNumberOrHash) (*accounts.AccProofResult, error) {

tx, err := api.db.BeginRo(ctx)
if err != nil {
return nil, err
}
defer tx.Rollback()

blockNr, _, _, err := rpchelper.GetBlockNumber(blockNrOrHash, tx, api.filters)
if err != nil {
return nil, err
}

latestBlock, err := rpchelper.GetLatestBlockNumber(tx)
if err != nil {
return nil, err
} else if blockNr != latestBlock {
return nil, fmt.Errorf(NotImplemented, "eth_getProof for block != latest")
} else if len(storageKeys) != 0 {
return nil, fmt.Errorf(NotImplemented, "eth_getProof with storageKeys")
} else {
addrHash, err := common.HashData(address[:])
if err != nil {
return nil, err
}

rl := trie.NewRetainList(0)
rl.AddKey(addrHash[:])

loader := trie.NewFlatDBTrieLoader("getProof")
trace := true
if err := loader.Reset(rl, nil, nil, trace); err != nil {
return nil, err
}

var accProof accounts.AccProofResult
accProof.Address = address
loader.SetProofReturn(&accProof)

_, err = loader.CalcTrieRoot(tx, nil, nil)
if err != nil {
return nil, err
}
return &accProof, nil
}
}

func (api *APIImpl) tryBlockFromLru(hash libcommon.Hash) *types.Block {
Expand Down
22 changes: 22 additions & 0 deletions core/types/accounts/account_proof.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
package accounts

import (
libcommon "github.com/ledgerwatch/erigon-lib/common"
"github.com/ledgerwatch/erigon/common/hexutil"
)

// Result structs for GetProof
type AccProofResult struct {
Address libcommon.Address `json:"address"`
AccountProof []string `json:"accountProof"`
Balance *hexutil.Big `json:"balance"`
CodeHash libcommon.Hash `json:"codeHash"`
Nonce hexutil.Uint64 `json:"nonce"`
StorageHash libcommon.Hash `json:"storageHash"`
StorageProof []StorProofResult `json:"storageProof"`
}
type StorProofResult struct {
Key string `json:"key"`
Value *hexutil.Big `json:"value"`
Proof []string `json:"proof"`
}
40 changes: 36 additions & 4 deletions turbo/trie/gen_struct_step.go
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,10 @@ type structInfoReceiver interface {
topHash() []byte
topHashes(prefix []byte, branches, children uint16) []byte
printTopHashes(prefix []byte, branches, children uint16)

// eth_getProof extensions
branchEx(set uint16, doProof bool) error
branchHashEx(set uint16, doProof bool) error
}

// hashCollector gets called whenever there might be a need to create intermediate hash record
Expand Down Expand Up @@ -99,7 +103,10 @@ func (GenStructStepHashData) GenStructStepData() {}
// Whenever a `BRANCH` or `BRANCHHASH` opcode is emitted, the set of digits is taken from the corresponding `groups` item, which is
// then removed from the slice. This signifies the usage of the number of the stack items by the `BRANCH` or `BRANCHHASH` opcode.
// DESCRIBED: docs/programmers_guide/guide.md#separation-of-keys-and-the-structure
func GenStructStep(

// GenStructStepEx is extended to support optional generation of an Account Proof during trie_root.go CalcTrieRoot().
// The wrapper below calls it with nil/false defaults so that other callers do not need to be modified.
func GenStructStepEx(
retain func(prefix []byte) bool,
curr, succ []byte,
e structInfoReceiver,
Expand All @@ -109,6 +116,8 @@ func GenStructStep(
hasTree []uint16,
hasHash []uint16,
trace bool,
wantProof func(prefix []byte) bool,
cutoff bool,
) ([]uint16, []uint16, []uint16, error) {
for precLen, buildExtensions := calcPrecLen(groups), false; precLen >= 0; precLen, buildExtensions = calcPrecLen(groups), true {
var precExists = len(groups) > 0
Expand Down Expand Up @@ -164,7 +173,7 @@ func GenStructStep(
}
buildExtensions = true
case *GenStructStepAccountData:
if retain(curr[:maxLen]) {
if retain(curr[:maxLen]) || (wantProof != nil && wantProof(curr[:len(curr)-1])) {
if err := e.accountLeaf(remainderLen, curr, &v.Balance, v.Nonce, v.Incarnation, v.FieldSet, codeSizeUncached); err != nil {
return nil, nil, nil, err
}
Expand Down Expand Up @@ -267,11 +276,21 @@ func GenStructStep(
}
}

var doProof bool
if wantProof != nil {
if maxLen > 0 && wantProof(curr[:maxLen]) {
doProof = true
}
if len(succ) == 0 && maxLen == 0 && cutoff {
doProof = true
}
}

if trace {
e.printTopHashes(curr[:maxLen], 0, groups[maxLen])
}
if retain(curr[:maxLen]) {
if err := e.branch(groups[maxLen]); err != nil {
if retain(curr[:maxLen]) || doProof {
if err := e.branchEx(groups[maxLen], doProof); err != nil {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Is it possible to call e.TopHashes() before/after e.branch() to get info which you need? (instead of introducing e.branchEx)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll look at it. I need the trie nodes before they are hashed and it looked like topHashes() was only giving me the output of that operation.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mmontour1306 , yes, but it has stack of nodes HashBuilder.nodeStack just no method like TopNodes() to return top of stack. Also maybe need read it before hashing, not after.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Anyway, not very important. I just trying to keep GenStruct step simpler. It's already complex. So, if you have any idea about it - you are welcome.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my latest version I have removed the "e.branchEx" stuff, restoring the original function signatures. Instead of passing a "doProof" flag into those functions, I have instead introduced a new function which is called ahead of time to set a flag telling hashbuilder that it should collect the proof data the next time it generates a hash (for a branch node or for the account leaf node). This is the least invasive method I can think of.

The various internal stacks do not appear to ever contain the specific data structures I need for the proof. The raw data is there and it would be possible to construct another copy of the node structure elsewhere, but this would involve duplicating a lot of the hashbuilder code in that other location. The data I need is what's fed into the hashing function inside hashbuilder as it constructs the RLP-encoded data structures.

return nil, nil, nil, err
}
} else {
Expand Down Expand Up @@ -305,6 +324,19 @@ func GenStructStep(
}
return nil, nil, nil, nil
}
func GenStructStep(
retain func(prefix []byte) bool,
curr, succ []byte,
e structInfoReceiver,
h HashCollector2,
data GenStructStepData,
groups []uint16,
hasTree []uint16,
hasHash []uint16,
trace bool,
) ([]uint16, []uint16, []uint16, error) {
return GenStructStepEx(retain, curr, succ, e, h, data, groups, hasTree, hasHash, trace, nil, false)
}

func GenStructStepOld(
retain func(prefix []byte) bool,
Expand Down
Loading