Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add native prestate tracer #6907

Merged
merged 19 commits into from
Apr 16, 2024
Merged

Add native prestate tracer #6907

merged 19 commits into from
Apr 16, 2024

Conversation

natebeauregard
Copy link
Contributor

@natebeauregard natebeauregard commented Apr 8, 2024

Changes

  • Adds a native prestateTracer implementation
  • Renames the prestate tracer JS implementation to be prestateTracer_legacy
  • Adds support for serializing/deserializing to and from JSON for UInt256 dictionary keys

Types of changes

What types of changes does your code introduce?

  • Bugfix (a non-breaking change that fixes an issue)
  • New feature (a non-breaking change that adds functionality)
  • Breaking change (a change that causes existing functionality not to work as expected)
  • Optimization
  • Refactoring
  • Documentation update
  • Build-related changes
  • Other: Description

Testing

Requires testing

  • Yes
  • No

If yes, did you write tests?

  • Yes
  • No

Documentation

Requires explanation in Release Notes

  • Yes
  • No

The ITxTracer.StartOperation method signature was updated to include the Execution Environment as a parameter and to remove the depth and isPostMerge parameters since those can now be derived from the execution environment. We should relay to users that this method signature was changed in case anyone is relying on it for their own tracer implementations.

/// <param name="isPostMerge"></param>
/// <remarks>Depends on <see cref="IsTracingInstructions"/></remarks>
void StartOperation(int depth, long gas, Instruction opcode, int pc, bool isPostMerge = false);
void StartOperation(int depth, long gas, Instruction opcode, int pc, Address executingAccount, bool isPostMerge = false);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I spent the past day and a half investigating deriving the executingAccount in the tracer's ReportAction/SetOperationStorage methods without needing to modify the ITxTracer API, however the approach started really blowing up in scope and had a bunch of edge cases that I was having difficulty debugging. Since this PR is meant to optimize performance for the tracers, I felt that having the executing account passed direclty from the virtual machine made the most sense after struggling to derive the executing account in the tracer itself.

If any reviewer feels strongly about not passing through the executing account and instead deriving it in the tracer, would you be able to pair with me on debugging? I feel pretty blocked moving forward with this update.

For some additional information about my investigation, I calculated the initial executing account with

private Address GetInitialExecutingAccount(Address from, Address? to, bool isContractCreation, bool isSystem)
{
    UInt256 nonce = isContractCreation ? _worldState!.GetNonce(from) : 0;
    return to ?? (isSystem
        ? from
        : ContractAddress.From(from, nonce > 0 ? nonce - 1 : nonce));
}

Then, for each modifying operation the executing account was recalculated. Here's a gist link with my attempted changes as well.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe let's do it differently, pass ExecutionEnvironment into ReportAction?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just added a commit to pass through the execution environment as a param in StartOperation as discussed in our daily meeting

@@ -85,7 +85,7 @@ public class UserOperationTxTracer : TxTracer

public override void StartOperation(in ExecutionEnvironment env, long gas, Instruction opcode, int pc)
{
int depth = env.CallDepth;
int depth = env.CallDepth + 1;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question for reviewers - I kept going back and forth on leaving the depth and isPostMerge params in StartOperation, even though they can now be derived from env. Please let me know if you think I should add them back to the method signature instead of deriving them when needed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets keep it simple, if they are not needed, they are not needed.
Why do we need to do +1 here though?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good, I'll derive them when needed then. Prior to my updates for the StartOperation params in this PR, the call depth was increased by one before being passed through to the tracers and I initially forgot to add that logic in with my updates.

_txTracer.StartOperation(vmState.Env.CallDepth + 1, gasAvailable, instruction, programCounter, vmState.Env.TxExecutionContext.BlockExecutionContext.Header.IsPostMerge);

natebeauregard and others added 13 commits April 9, 2024 11:46
Updates NativePrestateTracer to serialize uint256 dictionary keys/vals directly

Co-authored-by: LukaszRozmej <LukaszRozmej@users.noreply.github.com>
Co-authored-by: OlegJakushkin <OlegJakushkin@users.noreply.github.com>
Co-authored-by: LukaszRozmej <LukaszRozmej@users.noreply.github.com>
Co-authored-by: OlegJakushkin <OlegJakushkin@users.noreply.github.com>
* Updates ITxTracer.StartOperation to have the entire execution environment as a param
* Derive the pre-existing depth and isPostMerge params from the execution env
@natebeauregard natebeauregard changed the title Feat/native prestate tracer Add native prestate tracer Apr 9, 2024
@@ -85,7 +85,7 @@ public class UserOperationTxTracer : TxTracer

public override void StartOperation(in ExecutionEnvironment env, long gas, Instruction opcode, int pc)
{
int depth = env.CallDepth;
int depth = env.CallDepth + 1;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets keep it simple, if they are not needed, they are not needed.
Why do we need to do +1 here though?

namespace Nethermind.Evm.Test.Tracing;

[TestFixture]
public class GethLike4byteTracerTests : GethLikeNativeTracerTestsBase
{
[TestCaseSource(nameof(FourByteTracerTests))]
public Dictionary<string, int>? four_byte_tracer_executes_correctly(byte[] code, byte[]? input) =>
(Dictionary<string, int>)ExecuteAndTrace(Native4ByteTracer.FourByteTracer, code, input).CustomTracerResult?.Value;
(Dictionary<string, int>)ExecuteAndTrace(Native4ByteTracer.FourByteTracer, code, null, null, input).CustomTracerResult?.Value;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

those nulls don't look great

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, I'll update this!

Comment on lines 9 to 11
public Address From;
public Address? To;
public readonly Address? Beneficiary = beneficiary;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't those be inferred from TxExecutionContext and BlockExecutionContext accesible from ExecutionEnvironment? If not maybe add them there?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, they can be inferred through TxExecutionContext if I add them as properties in that struct and add them while building the execution environment. I'll then need to add the execution environment as a param for the ReportAction method so that I can pull off the transaction from address, to address, and beneficiary address on the first call. I think it would be worthwhile exposing the execution environment for ReportAction as well in case we want to add any new native tracers in the future that could take advantage of that so I'll go ahead and make this change.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added the spike for this approach on a commit in a separate branch since I realized that having this logic in the ReportAction method would be kind of hacky since the nonce for the transaction sender is incremented prior to entering ReportAction and would need special handling for the initial case to decrement it by one. I thought about exposing another trace method to the ITxTracer API to mirror the geth OnTxStart hook, but felt like that might be overkill for this use case since the information can just be passed from the constructor.

Copy link
Member

@LukaszRozmej LukaszRozmej left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

few minor touches

@LukaszRozmej LukaszRozmej mentioned this pull request Apr 10, 2024
16 tasks
@natebeauregard natebeauregard merged commit 2508015 into master Apr 16, 2024
67 checks passed
@natebeauregard natebeauregard deleted the feat/native-prestate-tracer branch April 16, 2024 17:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants