eth/tracers: package restructuring #23857

holiman · 2021-11-04T17:44:48Z

This is a follow-up to #23708. This PR restructures the tracers, so the hierarchy is

eth/tracers
- js - here resides the js tracers.
- native here resides tracers written in go. The package-doc here explains how to create a new tracer and build it for inclusion in the binary.
- internal/tracetest/ this is where tests that require both/all engines live: tests for callTracer, callTracerLegacy and callTracerNative.

The child packages register themselves via submitting lookup-functions. The eth/tracers package then invokes the lookups, and hopefully a tracer is returned.
This means that in order to use tracers, one has to make sure to load the packages, since this triggers the package init function. This is done in eth/tracers/testing, and also in cmd/geth/.

An upside of this is that we can add experimental engines, e.g. merge in a goja engine -- but as long as we don't import the package from cmd/geth, we won't actually include it in the binary release. This also means that people using geth as a library will not include e.g. duktape.

The api-tests were carefully rewritten to not require the tracers, but instead rely only on the results obtained via 'vanilla' tracing -- i.e with structlog output.
cc @s1na

fjl · 2021-11-04T18:31:22Z

Please move the shared testing package to "eth/tracers/internal/tracetest".

Two reasons: there is no need for packages outside of eth/tracers to import this functionality, at least for now. And calling it "testing" is not great because it conflicts with the stdlib package testing.

holiman · 2021-11-04T22:44:16Z

Good points @fjl, done!

s1na · 2021-11-08T18:12:27Z

eth/tracers/tracers.go

+// then the lookup will be placed last. This is typically meant for interpreted
+// engines (js) which can evaluate dynamic user-supplied code.
+func RegisterLookup(wildcard bool, lookup lookupFunc) {
+	if wildcard {


Just an observation that this won't work if there are e.g. 2 JS engines (one will starve). Not that we'll ever have 2. But I guess generally I never liked that we're using one parameter to either select from given tracers or to evaluate a JS script. I'd have preferred sth like {jscode: "{...}"} then we could have a separate function for initializing a tracer with dynamic code.

Yes, we can only have one 'wildcard-matcher'. In hindsight, yes, it would have been nicer with a marker for it, e.g //javascript\n

holiman · 2021-11-08T18:33:11Z

eth/tracers/native/call.go

@@ -145,6 +154,7 @@ func (t *callTracer) GetResult() (json.RawMessage, error) {
 	return json.RawMessage(res), t.reason
 }

+// Stop terminates execution of the tracer at the first opportune moment.


Oh right, might want to put back that stop via interrupting the vm that I prevented you from adding previously :)

* eth/tracers: restructure tracer package * core/vm/runtime: load js tracers * eth/tracers: mv bigint js code to own file * eth/tracers: add method docs for native tracers * eth/tracers: minor doc fix * core,eth: cancel evm on nativecalltracer stop * core/vm: fix failing test Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>

ghost · 2021-12-01T23:15:24Z

@fkimono185

ghost · 2021-12-01T23:15:38Z

I like

* eth/tracers: implement debug.intermediateRoots (ethereum#23594) This PR implements a new debug method, which I've talked briefly about to some other client developers. It allows the caller to obtain the intermediate state roots for a block (which might be either a canon block or a 'bad' block). Signed-off-by: wenbiao <delweng@gmail.com> * core, rpc: disable memory output by default in traces (ethereum#23558) * core: cmd: invert disableMemory * core: fix missed inversion * cmd/evm: preserve Flags but change default value * Apply suggestions from code review Co-authored-by: Martin Holst Swende <martin@swende.se> Co-authored-by: Martin Holst Swende <martin@swende.se> Signed-off-by: wenbiao <delweng@gmail.com> * eth/tracers: abort evm execution when trace is aborted (ethereum#23580) Signed-off-by: wenbiao <delweng@gmail.com> * eth/tracers: avoid unsyncronized mutations on trie database (ethereum#23632) This PR fixes an issue in traceChain, where the statedb Commit operation was performed asynchronously with dereference-operations agains the underlying trie.Database instance. Due to how the reference counting works within the trie database (where parent count is recursively updated when new parents are added), doing dereferencing in the middle of Commit can cause the refcount to become wrong, leading to an inconsistent state. This was fixed by doing Commit/Deref from the same routine. Signed-off-by: wenbiao <delweng@gmail.com> * core,eth: call frame tracing (ethereum#23087) This change introduces 2 new optional methods; `enter()` and `exit()` for js tracers, and makes `step()` optiona. The two new methods are invoked when entering and exiting a call frame (but not invoked for the outermost scope, which has it's own methods). Currently these are the data fields passed to each of them: enter: type (opcode), from, to, input, gas, value exit: output, gasUsed, error The PR also comes with a re-write of the callTracer. As a backup we keep the previous tracing script under the name `callTracerLegacy`. Behaviour of both tracers are equivalent for the most part, although there are some small differences (improvements), where the new tracer is more correct / has more information. Signed-off-by: wenbiao <delweng@gmail.com> * eth/tracers: re-write of 4byte tracer using enter/exit (ethereum#23622) * eth/tracers: add re-write of 4byte tracer using enter/exit * eth/tracers: fix 4byte indent Signed-off-by: wenbiao <delweng@gmail.com> * eth/tracers: tx.BaseFee not implemented Signed-off-by: wenbiao <delweng@gmail.com> * eth/tracers: do the JSON serialization via .js to capture C faults Signed-off-by: wenbiao <delweng@gmail.com> * eth/tracers: fix callTracer fault handling (ethereum#23667) * eth/tracers: fix calltracer fault handling * eth/tracers: fix calltracer indentation Signed-off-by: wenbiao <delweng@gmail.com> * eth/tracers: invoke enter/exit on 0-value calls to inex accounts (ethereum#23828) Signed-off-by: wenbiao <delweng@gmail.com> * eth: make traceChain avoid OOM on long-running tracing (ethereum#23736) This PR changes long-running chain tracing, so that it at some points releases the memory trie db, and switch over to a fresh disk-backed trie. Signed-off-by: wenbiao <delweng@gmail.com> * eth/tracers: expose contextual infos (block hash, tx hash, tx index) Signed-off-by: wenbiao <wenbiao.zheng@ambergroup.io> * eth/tracers: redefine Context Signed-off-by: wenbiao <wenbiao.zheng@ambergroup.io> * eth/tracers: support for golang tracers + add golang callTracer (ethereum#23708) * eth/tracers: add basic native loader * eth/tracers: add GetResult to tracer interface * eth/tracers: add native call tracer * eth/tracers: fix call tracer json result * eth/tracers: minor fix * eth/tracers: fix * eth/tracers: fix benchTracer * eth/tracers: test native call tracer * eth/tracers: fix * eth/tracers: rm extra make Co-authored-by: Martin Holst Swende <martin@swende.se> * eth/tracers: rm extra make * eth/tracers: make callFrame private * eth/tracers: clean-up and comments * eth/tracers: add license * eth/tracers: rework the model a bit * eth/tracers: move tracecall tests to subpackage * cmd/geth: load native tracers * eth/tracers: minor fix * eth/tracers: impl stop * eth/tracers: add native noop tracer * renamings Co-authored-by: Martin Holst Swende <martin@swende.se> * eth/tracers: more renamings * eth/tracers: make jstracer non-exported, avoid cast * eth/tracers, core/vm: rename vm.Tracer to vm.EVMLogger for clarity * eth/tracers: minor comment fix * eth/tracers/testing: lint nitpicks * core,eth: cancel evm on nativecalltracer stop * Revert "core,eth: cancel evm on nativecalltracer stop" This reverts commit 01bb908. * eth/tracers: linter nits * eth/tracers: fix output on err Co-authored-by: Martin Holst Swende <martin@swende.se> Signed-off-by: wenbiao <wenbiao.zheng@ambergroup.io> * eth/tracers: make native calltracer default (ethereum#23867) Signed-off-by: wenbiao <wenbiao.zheng@ambergroup.io> * eth/tracers: package restructuring (ethereum#23857) * eth/tracers: restructure tracer package * core/vm/runtime: load js tracers * eth/tracers: mv bigint js code to own file * eth/tracers: add method docs for native tracers * eth/tracers: minor doc fix * core,eth: cancel evm on nativecalltracer stop * core/vm: fix failing test Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com> Signed-off-by: wenbiao <wenbiao.zheng@ambergroup.io> * eth/tracers: ethapi.TransactionArgs was not merged Signed-off-by: wenbiao <wenbiao.zheng@ambergroup.io> * eth/tracers: fix the api_test with ErrInsufficientFunds to ErrInsufficientFundsForTransfer Signed-off-by: wenbiao <wenbiao.zheng@ambergroup.io> * eth/tracers: check posa before statedb.Prepare in IntermiateRoots api Signed-off-by: wenbiao <wenbiao.zheng@ambergroup.io> * eth/tracers: make js calltracer default, compatible with old version Signed-off-by: wenbiao <wenbiao.zheng@ambergroup.io> * eth/tracers: fix the default callTrace name of callTracerJs Signed-off-by: wenbiao <wenbiao.zheng@ambergroup.io> * Revert "eth/tracers: fix the default callTrace name of callTracerJs" This reverts commit 62a3bc215d9f07e422a4c659289bb3ba4f9ed2fa. Signed-off-by: wenbiao <wenbiao.zheng@ambergroup.io> * Revert "eth/tracers: make js calltracer default, compatible with old version" This reverts commit 85ef42c0ea651f0b228d4209b1b2598b24e12f1f. Signed-off-by: wenbiao <wenbiao.zheng@ambergroup.io> * eth/tracers: fix the variable race condition Signed-off-by: wenbiao <wenbiao.zheng@ambergroup.io> Co-authored-by: Martin Holst Swende <martin@swende.se> Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de> Co-authored-by: Sina Mahmoodi <1591639+s1na@users.noreply.github.com> Co-authored-by: Péter Szilágyi <peterke@gmail.com> Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>

* eth/tracers: restructure tracer package * core/vm/runtime: load js tracers * eth/tracers: mv bigint js code to own file * eth/tracers: add method docs for native tracers * eth/tracers: minor doc fix * core,eth: cancel evm on nativecalltracer stop * core/vm: fix failing test Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>

holiman force-pushed the restructure_tracers branch from ae96c74 to 50ba294 Compare November 4, 2021 18:21

holiman force-pushed the restructure_tracers branch 2 times, most recently from c5ac572 to d8b80f3 Compare November 4, 2021 22:40

holiman force-pushed the restructure_tracers branch from d8b80f3 to 4a4ab56 Compare November 5, 2021 10:52

holiman marked this pull request as ready for review November 5, 2021 10:53

holiman requested review from karalabe and rjl493456442 as code owners November 5, 2021 10:53

holiman force-pushed the restructure_tracers branch 2 times, most recently from d97596d to 20674c8 Compare November 5, 2021 12:31

holiman requested a review from s1na November 5, 2021 16:08

holiman added 2 commits November 8, 2021 14:20

eth/tracers: restructure tracer package

0de57c2

core/vm/runtime: load js tracers

5377372

holiman force-pushed the restructure_tracers branch from f790a68 to 5377372 Compare November 8, 2021 13:21

s1na added 3 commits November 8, 2021 17:34

eth/tracers: mv bigint js code to own file

390665b

eth/tracers: add method docs for native tracers

1588828

eth/tracers: minor doc fix

c6132f6

s1na reviewed Nov 8, 2021

View reviewed changes

holiman commented Nov 8, 2021

View reviewed changes

s1na and others added 2 commits November 8, 2021 20:37

core,eth: cancel evm on nativecalltracer stop

2b60c0e

core/vm: fix failing test

86ecffe

holiman added the status:triage label Nov 9, 2021

fjl approved these changes Nov 9, 2021

View reviewed changes

holiman merged commit 6b9c77f into ethereum:master Nov 9, 2021

holiman added this to the 1.10.13 milestone Nov 9, 2021

holiman deleted the restructure_tracers branch November 10, 2021 18:32

jsvisa mentioned this pull request Nov 24, 2021

[R4R]Feature/backport geth native trace bnb-chain/bsc#581

Merged

yasyzb mentioned this pull request Apr 18, 2022

feature: native trace BytomDAO/bmc#19

Merged

trinhdn2 mentioned this pull request Jun 15, 2023

Update EVM trace log capturing BuildOnViction/victionchain#356

Closed

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

eth/tracers: package restructuring #23857

eth/tracers: package restructuring #23857

holiman commented Nov 4, 2021 •

edited

Loading

fjl commented Nov 4, 2021 •

edited

Loading

holiman commented Nov 4, 2021

s1na Nov 8, 2021

holiman Nov 8, 2021

holiman Nov 8, 2021

ghost commented Dec 1, 2021

ghost commented Dec 1, 2021

eth/tracers: package restructuring #23857

eth/tracers: package restructuring #23857

Conversation

holiman commented Nov 4, 2021 • edited Loading

fjl commented Nov 4, 2021 • edited Loading

holiman commented Nov 4, 2021

s1na Nov 8, 2021

Choose a reason for hiding this comment

holiman Nov 8, 2021

Choose a reason for hiding this comment

holiman Nov 8, 2021

Choose a reason for hiding this comment

ghost commented Dec 1, 2021

ghost commented Dec 1, 2021

holiman commented Nov 4, 2021 •

edited

Loading

fjl commented Nov 4, 2021 •

edited

Loading