Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: update miner mempool iterator query to consider both nonces and fee rates #5541

Open
wants to merge 9 commits into
base: develop
Choose a base branch
from

Conversation

rafaelcr
Copy link

@rafaelcr rafaelcr commented Dec 7, 2024

Problem

The current query used by Stacks miners to iterate over pending mempool transactions sorts them only by fee_rate in descending order. This approach creates the following problems:

  1. Since it does not consider transaction nonces, non-mineable transactions with high fees but old (or very future) nonces go to the top of the list and get visited every time, thus wasting block construction time budget.
  2. Transactions with null fee rates are sent to the bottom of the list even if their nonces should be evaluated next, forcing the algorithm to query them separately with some probability of inclusion.

Solution

This PR changes the transaction selection query to a new one that prioritizes those that can be confirmed as fast as possible (lowest valid nonces) with the highest fees as possible. This means that even if it doesn't select the transactions with the highest global fees first, it will select those that can be mined faster therefore optimizing block building time and allowing the miner to fit in more transactions in total within the allotted time.

The new query also mixes in transactions with null fee rates in results by simulating a fee rate for the purposes of ordering only.

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

1 similar comment
@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Comment on lines +1699 to +1702
CASE
WHEN fee_rate IS NULL THEN (ABS(RANDOM()) % 10000 / 10000.0) * (SELECT MAX(fee_rate) FROM mempool)
ELSE fee_rate
END AS sort_fee_rate
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@obycode I added a simulated fee rate (for sorting only) when it's NULL. It successfully mixes in null txs with estimated txs in a randomized order. What do you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a great idea! I like it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you also take into account the settings.consider_no_estimate_tx_prob here? I'm not sure how important it is to keep that config parameter.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe we don't need it anymore, however, if necessary I could use it as a probability to skip a null transaction altogether. Thoughts?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think probably this random mix in is good on its own and we can deprecate that option.

@rafaelcr rafaelcr changed the title (draft) fix: update miner mempool transaction iterator query to consider fee rate and nonce fix: update miner mempool transaction iterator query to consider fee rate and nonce Dec 15, 2024
@rafaelcr rafaelcr marked this pull request as ready for review December 15, 2024 00:09
@rafaelcr rafaelcr requested a review from a team as a code owner December 15, 2024 00:09
@rafaelcr rafaelcr changed the title fix: update miner mempool transaction iterator query to consider fee rate and nonce fix: update miner mempool iterator query to consider both nonces and fee rates Dec 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants