[NET] Inv, mempool and validation improvements #2118

Coming from btc@7659438a63ef162b4a4f942f86683ae6785f8162

…ed non-recursively, so simplify the logic and eliminate an unnecessary parameter Coming from btc@5de2baa138cda501038a4558bc169b2cfe5b7d6b

…prevent the descendant walking algorithm from doing too much work by marking the parent transaction as dirty. However to implement ancestor tracking, it's not possible to similarly mark those descendant transactions as dirty without having to calculate them to begin with. This commit removes the work limit altogether. With appropriate chain limits (-limitdescendantcount) the concern about doing too much work inside this function should be mitigated. Coming from btc@76a76321d2f36992178ddaaf4d023c5e33c14fbf

Redo the feerate index to be based on mining score, rather than fee. Update mempool_packages.py to test prioritisetransaction's effect on package scores. Coming from btc@eb306664e786ae43d539fde66f0fbe2a3e89d910

…state to each mempool entry, similar to descendant tracking, but also including caching sigops-with-ancestors (as that metric will be helpful to future code that implements better transaction selection in CreatenewBlock). Coming from btc@72abd2ce3c5ad8157d3a993693df1919a6ad79c3

Coming from btc@ce019bf90fe89c1256a89c489795987ef0b8a18f

Previously Bitcoin would send 1/4 of transactions out to all peers instantly. This causes high overhead because it makes >80% of INVs size 1. Doing so harms privacy, because it limits the amount of source obscurity a transaction can receive. These randomized broadcasts also disobeyed transaction dependencies and required use of the orphan pool. Because the orphan pool is so small this leads to poor propagation for dependent transactions. When the bypass wasn't in effect, transactions were sent in the order they were received. This avoided creating orphans but undermines privacy fairly significantly. This commit: Eliminates the bypass. The bypass is replaced by halving the average delay for outbound peers. Sorts candidate transactions for INV by their topological depth then by their feerate (then hash); removing the information leakage and providing priority service to higher fee transactions. Limits the amount of transactions sent in a single INV to 7tx/sec (and twice that for outbound); this limits the harm of low fee transaction floods, gives faster relay service to higher fee transactions. The 7 sounds lower than it really is because received advertisements need not be sent, and because the aggregate rate is multipled by the number of peers. Coming from btc@f2d3ba73860e875972738d1da1507124d0971ae5

Saves about 10% of application memory usage once the mempool warms up. Since the mempool is DynamicUsage-regulated, this will translate to a larger mempool in the same amount of space. Map value type: eliminate the vin index; no users of the map need to know which input of the transaction is spending the prevout. Map key type: replace the COutPoint with a pointer to a COutPoint. A COutPoint is 36 bytes, but each COutPoint is accessible from the same map entry's value. A trivial DereferencingComparator functor allows indirect map keys, but the resulting syntax is misleading: `map.find(&outpoint)`. Implement an indirectmap that acts as a wrapper to a map that uses a DereferencingComparator, supporting a syntax that accurately reflect the container's semantics: inserts and iterators use pointers since they store pointers and need them to remain constant and dereferenceable, but lookup functions take const references. Coming from btc@9805f4af7ecb6becf8a146bd845fb131ffa625c9

Coming from btc@96918a2f0990a8207d7631b8de73af8ae5d24aeb

An adaptation of btc@dc13dcd2bec2613a1cd5e0395b09b449d176146f with a tier two INV vector.

Handle mempool requests in send loop, subject to trickle By eliminating queued entries from the mempool response and responding only at trickle time, this makes the mempool no longer leak transaction arrival order information (as the mempool itself is also sorted)-- at least no more than relay itself leaks it.

Move bloom and feerate filtering to just prior to tx sending. This will avoid sending more pointless INVs around updates, and prevents using filter updates to timetag transactions. Also adds locking for fRelayTxes.

…f and introduced the timeLastMempoolReq coming from btc#8080.

Coming from btc@288d85ddf2e0a0c9d25a23db56052883170466d0

…t runner.

zapwallettxes previously did not interact well with persistent mempool. zapwallettxes would cause wallet transactions to be zapped, but they would then be reloaded from the mempool on startup. This commit softsets persistmempool to false if zapwallettxes is enabled so transactions are actually zapped.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[NET] Inv, mempool and validation improvements #2118

[NET] Inv, mempool and validation improvements #2118

Commits on Jan 18, 2021