Receipt key with block number

[asdacap]

• Prefix receipt's data key with block number like erigon.
• Reduced write amplification during sync. Total write goes down from 5TB to 2.65TB.
• Does not noticably increase disk space use (in my run the one without blocknumber is actually slighly higher). This is likely because rocksdb have delta compression turned on by default for keys, so multiple keys with same prefix got compressed really well.
• Unexpectedly did not improve RPC performance, nor does it reduce iops. I guess on SSDs at least, it just reduce write amplification. Maybe with readahead flag it'll do something. not even with readahead. I guess 50iops per receipts means something else is using it...
• Tested on non-compact receipts as compact receipts still need to load blocks.

Changes

• Address receipts by blocknum+hash.

Types of changes

What types of changes does your code introduce?

• Bugfix (a non-breaking change that fixes an issue)
• New feature (a non-breaking change that adds functionality)
• Breaking change (a change that causes existing functionality not to work as expected)
• Optimization
• Refactoring
• Documentation update
• Build-related changes
• Other: Description

Testing

Requires testing

• Yes
• No

If yes, did you write tests?

• Yes
• No

Notes on testing

• Scanned log with CL filter with archive node. Log matches.

[LukaszRozmej]

I think to have performance improvement we should enable sorting in database

[asdacap]

Sorting in database?

[LukaszRozmej]

Was thinking about SetAdviseRandomOnOpen but it is probably wrong suggestion

[LukaszRozmej]

I see one problem - this won't work without resync right? If someone just upgrades DB it will break?

[asdacap]

No, it works fine. I can sync without it, then read with new change.

[LukaszRozmej]

Overall cool.

Maybe we should decide the key scheme at the begining - read 1 key from blocks/receipts DB:

• if it has prefix - use prefix scheme
• if it doesn't have prefix or no DB - use old scheme

This way we would keep the existing databases consistent and avoid double read from existing databases.

[asdacap]

Ok, but which key to read?

[LukaszRozmej]

Ok, but which key to read?

whichever from block db? like open iterator get first key or something similar?

[asdacap]

Done