Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(trie): fix ZkMerkleStateTrie returns invalid proofs #63

Merged
merged 5 commits into from
Jan 30, 2024

Conversation

jyc228
Copy link
Collaborator

@jyc228 jyc228 commented Jan 29, 2024

Description

Get proofapi is responding to the wrong proof because it did not reverse the key when zktrie probe.

Summary by CodeRabbit

  • New Features

    • Introduced a method for reversing byte slices to enhance data manipulation capabilities.
  • Enhancements

    • Improved state trie creation logic to support different types based on database properties.
    • Enhanced Merkle tree iteration and update mechanisms for better performance and security.
    • Optimized Zero-Knowledge (ZK) Merkle trie operations with key transformation improvements and more efficient hash computations.
  • Refactor

    • Removed redundant fields and updated methods for clearer, more efficient code in ZK Merkle trie structures.

Copy link

coderabbitai bot commented Jan 29, 2024

Warning

Rate Limit Exceeded

@jyc228 has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 24 minutes and 26 seconds before requesting another review.

How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.
Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.
Please see our FAQ for further information.

Commits Files that changed from the base of the PR and between beb6458 and f648812.

Walkthrough

The changes involve enhancing the handling and processing of data within a blockchain context, specifically focusing on the optimization and security of Merkle Trie operations. A new function for reversing byte slices has been introduced, and various components have been adjusted to improve the creation and verification of zero-knowledge (ZK) state tries. This includes modifications to key handling, state trie creation, and proof generation, emphasizing efficiency and privacy enhancements in blockchain data structures.

Changes

Files Change Summaries
common/bytes.go Added ReverseBytes function to reverse byte slices.
internal/ethapi/api.go Modified newStateTrie to conditionally create tries based on DB properties.
trie/iterator_test.go Updated tree.Update method to use secure hash bytes.
trie/zk/merkle_tree_node.go, trie/zk/util.go Removed ValueHash field from LeafNode, affecting hash caching and computation.
trie/zk_merkle_state_trie.go Modified Prove method to use common.ReverseBytes and adjusted value setting logic.
trie/zk_merkle_trie.go Included mutateKey function and updated several methods for key transformation.
trie/zk_merkle_stack_trie.go Update method now reverses hash parameter before updating.

🐇✨🌿📜
In the garden of code, where the data bytes grow,
Changes bloom, and the Merkle Trees know.
With a hop and a skip, the bytes reverse,
A blockchain's whisper, a cryptographic verse.
🌿✨🐇📜

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit-tests for this file.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit tests for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository from git and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit tests.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • The JSON schema for the configuration file is available here.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/coderabbit-overrides.v2.json

CodeRabbit Discord Community

Join our Discord Community to get help, request features, and share feedback.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 1

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between beb6458 and 4ac11a6.
Files selected for processing (8)
  • common/bytes.go (1 hunks)
  • internal/ethapi/api.go (1 hunks)
  • trie/iterator_test.go (1 hunks)
  • trie/zk/merkle_tree_node.go (1 hunks)
  • trie/zk/util.go (2 hunks)
  • trie/zk_merkle_stack_trie.go (1 hunks)
  • trie/zk_merkle_state_trie.go (1 hunks)
  • trie/zk_merkle_trie.go (2 hunks)
Additional comments: 10
trie/zk_merkle_stack_trie.go (1)
  • 19-19: The modification to reverse the hash parameter in the Update method before passing it to the MerkleTree.Update method aligns with the PR's objective to ensure keys are correctly reversed during probe operations. This change is crucial for addressing the issue with invalid proofs in zkStateTries.
trie/zk/util.go (1)
  • 74-94: The changes in the computeNodeHash function, including the removal of the ValueHash field logic from the LeafNode struct and adjustments in the node hash computation logic, are consistent with the PR's objective to enhance the integrity and reliability of the proof generation process. The reorganization and error handling modifications improve the clarity and maintainability of the code.
trie/zk_merkle_trie.go (1)
  • 15-44: The introduction of the mutateKey function and its usage in methods such as MustGet, Get, GetLeafNode, MustUpdate, Update, MustDelete, and Delete for key transformation is a significant improvement. This change ensures that keys are correctly reversed during probe operations, aligning with the PR's objective to fix invalid proofs in zkStateTries. The implementation is consistent, enhancing the code's maintainability and readability.
common/bytes.go (1)
  • 153-159: The addition of the ReverseBytes function is a valuable utility that supports the PR's objective by providing a standardized way to reverse byte slices. This functionality is essential for ensuring that keys are correctly reversed during probe operations in zkStateTries, contributing to the integrity and reliability of the proof generation process.
trie/zk_merkle_state_trie.go (1)
  • 119-122: The modification to the Prove method to use the common.ReverseBytes function for reversing the key aligns with the PR's objective to ensure keys are correctly reversed during probe operations. This change is crucial for addressing the issue with invalid proofs in zkStateTries, enhancing the integrity and reliability of the proof generation process.
trie/iterator_test.go (1)
  • 645-645: The modification in the TestMerkleTreeIterator function to use zk.MustNewSecureHash(common.LeftPadBytes([]byte(val.k), 32)).Bytes() for the tree.Update method call is a minor but important change. It ensures consistency in how byte slices are handled and passed to the Update method, potentially affecting the test's behavior and its interaction with the zk package. This change aligns with the overall codebase's approach to handling byte slices and hash computations.
internal/ethapi/api.go (4)
  • 759-762: The logic for determining the trie type based on the database's properties (IsZkStateTrie and IsZk) is clear and straightforward. However, ensure that the trie.NewZkMerkleStateTrie, trie.NewZkTrie, and trie.NewStateTrie functions handle errors appropriately and do not return a valid trie object in case of initialization failures.
  • 756-765: > 📝 NOTE

This review was outside the diff hunks, and no overlapping diff hunk was found. Original lines [2592-2595]

The SendTransaction method correctly assembles and signs a transaction based on the provided arguments. However, it's crucial to ensure that the setDefaults method, which fills in missing transaction fields, properly handles all edge cases and inputs. This includes setting reasonable defaults for gas price in different network conditions (e.g., EIP-1559 networks) and ensuring nonce management is accurate to prevent transaction replacement issues.

  • 756-765: > 📝 NOTE

This review was outside the diff hunks, and no overlapping diff hunk was found. Original lines [3285-3290]

The GetRawHeader method in DebugAPI provides a way to retrieve the RLP encoding of a block header. It's important to verify that the RLP encoding and decoding operations are symmetric and that no data is lost or incorrectly encoded during the process. This ensures that the raw header data can be reliably used for debugging or other purposes.

  • 756-765: > 📝 NOTE

This review was outside the diff hunks, and no overlapping diff hunk was found. Original lines [3364-3369]

In the NetAPI struct, methods like Listening, PeerCount, and Version provide network-related information. Ensure that the PeerCount method accurately reflects the number of connected peers, including any potential edge cases where peers are momentarily disconnected or in the process of connecting/disconnecting. This accuracy is crucial for network monitoring and diagnostics.

internal/ethapi/api.go Show resolved Hide resolved
@jyc228 jyc228 requested a review from a team January 29, 2024 08:58
@jyc228 jyc228 self-assigned this Jan 29, 2024
@Pangssu Pangssu changed the title Fix/zkstatetrie invalid proof fix(trie): fix ZkMerkleStateTrie returns invalid proofs Jan 29, 2024
@@ -12,11 +12,36 @@ import (

type ZkMerkleTrie struct {
*zk.MerkleTree
db *Database
db *Database
mutateKey func(key []byte) []byte
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't we rename this? Such as transformKey, preProcessKey, or reverseKey.

Copy link
Collaborator Author

@jyc228 jyc228 Jan 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'll change it to a transformKey.

trie/zk_merkle_trie.go Outdated Show resolved Hide resolved
trie/zk_merkle_stack_trie.go Show resolved Hide resolved
@@ -116,10 +116,10 @@ func (z *ZkMerkleStateTrie) delete(key []byte) error {
}

func (z *ZkMerkleStateTrie) Prove(key []byte, proofDb ethdb.KeyValueWriter) error {
return z.MerkleTree.Prove(key, func(node zk.TreeNode) error {
return z.prove(common.ReverseBytes(key), proofDb, func(node zk.TreeNode) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return z.prove(common.ReverseBytes(key), proofDb, func(node zk.TreeNode) error {
return z.prove(z.mutateKey(key), proofDb, func(node zk.TreeNode) error {

value := node.CanonicalValue()
if leaf, ok := node.(*zk.LeafNode); ok {
if preImage := z.GetKey(zkt.ReverseByteOrder(leaf.Key)); len(preImage) > 0 {
if preImage := z.GetKey(common.ReverseBytes(leaf.Key)); len(preImage) > 0 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if preImage := z.GetKey(common.ReverseBytes(leaf.Key)); len(preImage) > 0 {
if preImage := z.GetKey(z.mutateKey(leaf.Key)); len(preImage) > 0 {

trie/zk/util.go Show resolved Hide resolved
trie/zk/util.go Outdated Show resolved Hide resolved
Copy link
Contributor

@kangsorang kangsorang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jyc228 jyc228 force-pushed the fix/zkstatetrie-invalid-proof branch from 4ac11a6 to 539dff7 Compare January 30, 2024 05:58
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 0

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between beb6458 and 539dff7.
Files selected for processing (8)
  • common/bytes.go (1 hunks)
  • internal/ethapi/api.go (1 hunks)
  • trie/iterator_test.go (1 hunks)
  • trie/zk/merkle_tree_node.go (1 hunks)
  • trie/zk/util.go (2 hunks)
  • trie/zk_merkle_stack_trie.go (1 hunks)
  • trie/zk_merkle_state_trie.go (3 hunks)
  • trie/zk_merkle_trie.go (4 hunks)
Files skipped from review as they are similar to previous changes (8)
  • common/bytes.go
  • internal/ethapi/api.go
  • trie/iterator_test.go
  • trie/zk/merkle_tree_node.go
  • trie/zk/util.go
  • trie/zk_merkle_stack_trie.go
  • trie/zk_merkle_state_trie.go
  • trie/zk_merkle_trie.go

@jyc228 jyc228 force-pushed the fix/zkstatetrie-invalid-proof branch 2 times, most recently from 49f752c to 940f093 Compare January 30, 2024 06:29
@@ -44,7 +104,8 @@ func (z *ZkMerkleTrie) Commit(_ bool) (common.Hash, *trienode.NodeSet, error) {
}
err := z.ComputeAllNodeHash(func(node zk.TreeNode) error { return z.db.Put(node.Hash()[:], node.CanonicalValue()) })
if err != nil {
log.Error("Failed to commit zk merkle trie", "err", err)
z.logger.Error("failed to Commit", "error", err)
return common.Hash{}, nil, nil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return common.Hash{}, nil, nil
return common.Hash{}, nil, err

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 0

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between beb6458 and 940f093.
Files selected for processing (8)
  • common/bytes.go (1 hunks)
  • internal/ethapi/api.go (1 hunks)
  • trie/iterator_test.go (1 hunks)
  • trie/zk/merkle_tree_node.go (1 hunks)
  • trie/zk/util.go (2 hunks)
  • trie/zk_merkle_stack_trie.go (1 hunks)
  • trie/zk_merkle_state_trie.go (3 hunks)
  • trie/zk_merkle_trie.go (4 hunks)
Files skipped from review as they are similar to previous changes (8)
  • common/bytes.go
  • internal/ethapi/api.go
  • trie/iterator_test.go
  • trie/zk/merkle_tree_node.go
  • trie/zk/util.go
  • trie/zk_merkle_stack_trie.go
  • trie/zk_merkle_state_trie.go
  • trie/zk_merkle_trie.go

@jyc228 jyc228 force-pushed the fix/zkstatetrie-invalid-proof branch from 940f093 to f648812 Compare January 30, 2024 06:35
Copy link
Contributor

@seolaoh seolaoh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jyc228 jyc228 merged commit 442e9a1 into dev Jan 30, 2024
2 checks passed
@jyc228 jyc228 deleted the fix/zkstatetrie-invalid-proof branch January 30, 2024 06:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants