Bad Block Comparator - Plugin

[jflo]

In order to quickly diagnose bad blocks, it is necessary to compare block traces between two clients at the same state. Usually, this requires either quick action (most clients have limits on how much historical state they can maintain, since it is only needed for reorgs) or access to an archive node.

To solve the recency issue, it would be helpful to have a plugin in Besu that could be configured with a peer execution client, which could be asked for a trace of the same block as soon as a bad one is detected and added to the bad block manager. It could work something like this:

• when a bad block is detected, trace it and store the trace
• contact the configured peer, and immediately trace the same block
• normalize the output, trace formats are not quite the same
• provide both versions of the raw trace, and a delta between their normal forms via debug apis.

This would automate a very large portion of the legwork needed to diagnose issues.

[mbaxter]

Plugin development has been started in a new repo here: https://github.com/Consensys/besu-bad-block-plugin

[macfarla]

Removing the help wanted label since @mbaxter is working on this - thanks!

[mbaxter]

I've stopped work on this and unassigned it. If anyone wants to pick this up again, bad block events have been exposed through the plugins api here and a plugin project has been started here. The new plugin basically just contains scaffolding currently. Next steps would be to consume the bad block events from besu and implement the core logic.

[non-fungible-nelson]

@jflo what shall we do with this one?

[jflo]

Icebox seems like the right place