Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

core/txpool: don't inject lazy resolved transactions into the container #28917

Merged
merged 2 commits into from
Feb 2, 2024
Merged

core/txpool: don't inject lazy resolved transactions into the container #28917

merged 2 commits into from
Feb 2, 2024

Conversation

karalabe
Copy link
Member

@karalabe karalabe commented Feb 2, 2024 •
edited
Loading

When retrieving transactions for mining, we return lazy wrappers. The purpose is to delay loading blobs from disk until they are absolutely necessary and seem to really really really go into a next block.

The original wrapper however when "Resolved", pulled in the full transaction and stored it in the lazy wrapper. This seems like a good idea as it avoids loading the transaction a second time, but:

  • We never ever want to load a transaction a second time. We load it when we want to insert into a block, and then we either push it into the block or discard it. We never need it again (until the next block when we remake the lazy list). Storing it is thus pointless.
  • However, storing it means that our original light list starts to get heavy. And for every transaction that the miner resolves, the full blobs remain in memory until the entire list gets dereferenced. Depending on how the miner's implemented, this might be 12s until the block is retrieved.

With recommits, this work gets repeated multiple times, worsening the effect.

@karalabe karalabe added this to the 1.13.12 milestone Feb 2, 2024
holiman
holiman approved these changes Feb 2, 2024
View reviewed changes
Copy link
Contributor

@holiman holiman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@karalabe karalabe merged commit 47d76c5 into ethereum:master Feb 2, 2024
2 of 3 checks passed
@dnhn dnhn mentioned this pull request Feb 9, 2024
Merged
lukanus pushed a commit to blocknative/go-ethereum that referenced this pull request Feb 19, 2024
@AusIV @holiman
@bodhi-crypo @protolambda @keienWang @lightclient @rjl493456442 @karalabe @zoereco @ziogaschr @rex4539 @Halimao @fjl @lmittmann @s1na
Merge/v1.13.12 (#159)
786460e 
* params: begin v.1.13.12 release cycle

* internal/flags: fix typo (ethereum#28876)

* core/types: fix and test handling of faulty nil-returning signer (ethereum#28879)

This adds an error if the signer returns a nil value for one of the signature value fields.

* README.md: fix travis badge (ethereum#28889)

The hyperlink in the README file that directs to the Travis CI build was broken.
This commit updates the link to point to the corrent build page.

* eth/catalyst: allow payload attributes v1 in fcu v2 (ethereum#28882)

At some point, `ForkchoiceUpdatedV2` stopped working for `PayloadAttributesV1` while `paris` was active. This was causing a few failures in hive. This PR fixes that, and also adds a gate in `ForkchoiceUpdatedV1` to disallow `PayloadAttributesV3`.

* docs/postmortems: fix outdated link (ethereum#28893)

* core: reset tx lookup cache if necessary (ethereum#28865)

This pull request resets the txlookup cache if chain reorg happens, 
preventing them from remaining reachable. It addresses failures in
the hive tests.

* build: fix problem with windows line-endings in CI download (ethereum#28900)

fixes ethereum#28890

* eth/downloader: fix skeleton cleanup (ethereum#28581)

* eth/downloader: fix skeleton cleanup

* eth/downloader: short circuit if nothing to delete

* eth/downloader: polish the logic in cleanup

* eth/downloader: address comments

* deps: update memsize (ethereum#28916)

* core/txpool/blobpool: post-crash cleanup and addition/removal metrics (ethereum#28914)

* core/txpool/blobpool: clean up resurrected junk after a crash

* core/txpool/blobpool: track transaction insertions and rejections

* core/txpool/blobpool: linnnnnnnt

* core/txpool: don't inject lazy resolved transactions into the container (ethereum#28917)

* core/txpool: don't inject lazy resolved transactions into the container

* core/txpool: minor typo fixes

* core/types: fix typo (ethereum#28922)

* p2p: fix accidental termination of portMappingLoop (ethereum#28911)

* internal/flags: fix --miner.gasprice default listing (ethereum#28932)

* all: fix typos in comments (ethereum#28881)

* Makefile: add help target to display available targets (ethereum#28845)


Co-authored-by: Martin HS <martin@swende.se>
Co-authored-by: Felix Lange <fjl@twurst.com>

* core: cache transaction indexing tail in memory (ethereum#28908)

* eth, miner: fix enforcing the minimum miner tip (ethereum#28933)

* eth, miner: fix enforcing the minimum miner tip

* ethclient/simulated: fix failing test due the min tip change

* accounts/abi/bind: fix simulater gas tip issue

* core/state, core/vm: minor uint256 related perf improvements (ethereum#28944)

* cmd,internal/era: implement `export-history` subcommand (ethereum#26621)

* all: implement era format, add history importer/export

* internal/era/e2store: refactor e2store to provide ReadAt interface

* internal/era/e2store: export HeaderSize

* internal/era: refactor era to use ReadAt interface

* internal/era: elevate anonymous func to named

* cmd/utils: don't store entire era file in-memory during import / export

* internal/era: better abstraction between era and e2store

* cmd/era: properly close era files

* cmd/era: don't let defers stack

* cmd/geth: add description for import-history

* cmd/utils: better bytes buffer

* internal/era: error if accumulator has more records than max allowed

* internal/era: better doc comment

* internal/era/e2store: rm superfluous reader, rm superfluous testcases, add fuzzer

* internal/era: avoid some repetition

* internal/era: simplify clauses

* internal/era: unexport things

* internal/era,cmd/utils,cmd/era: change to iterator interface for reading era entries

* cmd/utils: better defer handling in history test

* internal/era,cmd: add number method to era iterator to get the current block number

* internal/era/e2store: avoid double allocation during write

* internal/era,cmd/utils: fix lint issues

* internal/era: add ReaderAt func so entry value can be read lazily

Co-authored-by: lightclient <lightclient@protonmail.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>

* internal/era: improve iterator interface

* internal/era: fix rlp decode of header and correctly read total difficulty

* cmd/era: fix rebase errors

* cmd/era: clearer comments

* cmd,internal: fix comment typos

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>

* core,params: add holesky to default genesis function (ethereum#28903)

* node, rpc: add configurable HTTP request limit (ethereum#28948)

Adds a configurable HTTP request limit, and bumps the engine default

* all: fix docstring names (ethereum#28923)

* fix wrong comment

* reviewers input

* Update log/handler_glog.go

---------

Co-authored-by: Martin HS <martin@swende.se>

* ethclient/simulated: fix typo (ethereum#28952)

(ethclient/simulated):fix typo

* eth/gasprice: fix percentile validation in eth_feeHistory (ethereum#28954)

* cmd/devp2p, eth: drop support for eth/67 (ethereum#28956)

* params, core/forkid: add mainnet timestamp for Cancun (ethereum#28958)

* params: add cancun timestamp for mainnet

* core/forkid: add test for mainnet cancun forkid

* core/forkid: update todo tests for cancun

* internal/ethapi: add support for blobs in eth_fillTransaction (ethereum#28839)

This change adds support for blob-transaction in certain API-endpoints, e.g. eth_fillTransaction. A follow-up PR will add support for signing such transactions.

* internal/era: update block index format to be based on record offset (ethereum#28959)

As mentioned in ethereum#26621, the block index format for era1 is not in line with the regular era block index. This change modifies the index so all relative offsets are based against the beginning of the block index record.

* params: go-ethereum v1.13.12 stable

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: alex <152680487+bodhi-crypo@users.noreply.github.com>
Co-authored-by: protolambda <proto@protolambda.com>
Co-authored-by: KeienWang <42377006+keienWang@users.noreply.github.com>
Co-authored-by: lightclient <14004106+lightclient@users.noreply.github.com>
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
Co-authored-by: zoereco <158379334+zoereco@users.noreply.github.com>
Co-authored-by: Chris Ziogas <ziogaschr@gmail.com>
Co-authored-by: Dimitris Apostolou <dimitris.apostolou@icloud.com>
Co-authored-by: Halimao <1065621723@qq.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: lmittmann <3458786+lmittmann@users.noreply.github.com>
Co-authored-by: Sina Mahmoodi <1591639+s1na@users.noreply.github.com>
Co-authored-by: Austin Roberts <austin.roberts@rivet.cloud>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@MariusVanDerWijden MariusVanDerWijden MariusVanDerWijden approved these changes

@holiman holiman holiman approved these changes

@rjl493456442 rjl493456442 Awaiting requested review from rjl493456442 rjl493456442 is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
1.13.12
Development

Successfully merging this pull request may close these issues.
3 participants
@karalabe @holiman @MariusVanDerWijden