[EVM-703]: Saving events in boltdb should have retry mechanism

[goran-ethernal]

Description

Through this PR we implemented retry of inserting bridge events (stateSync, exit, and transfer events). Since errors on inserting events in boltdb can happen (db errors, disk errors, etc.), we needed to provide a way to get missed events (events that failed on boltdb insert). This is crucial, since if a relayer node fails inserting some exit or stateSync events, those deposits and withdrawals will never be executed.

This PR implements eventsGetter[T contractsapi.EventAbi] struct, which gets desired events from transaction receipts by providing from and to block.

// eventsGetter is a struct for getting missed and current events
// of specified type from specified blocks
type eventsGetter[T contractsapi.EventAbi] struct {
	// blockchain is an abstraction of blockchain that provides necessary functions
	// for querying blockchain data (blocks, receipts, etc.)
	blockchain blockchainBackend
	// saveEventsFn is a plugin function used to return gotten events
	// and/or to save them to some db
	saveEventsFn func([]T) error
	// parseEventFn is a plugin function used to parse the event from transaction log
	parseEventFn func(*types.Header, *ethgo.Log) (T, bool, error)
	// isValidLogFn is a plugin function that validates the log
	// for example: if it was sent from the desired address
	isValidLogFn func(*types.Log) bool
}

// getFromBlocks gets events of specified type from specified blocks
// and saves them using the provided saveEventsFn
func (e *eventsGetter[T]) getFromBlocks(fromBlock, toBlock uint64) error 

eventGetter uses Go generics so it can be used on any event from edge core-contracts. It uses the plugin functions to have a more generic way of setting up events validation, events parsing and saving.

Exit events retry

checkpointManager now uses eventGetter[ExitEvent] to get missed events. To know which block was last processed for exit events, a new bucket is added to CheckpointStore, called lastProcessedBlock. It is a bucket with single entry in it which indicates the last block that was processed for exit events. It is saved under a fixed key lastProcessedBlockKey.
On each block finalization by consensus or getting the block from syncer, checkpointManager will read the lastSaved bucket to know which block was last processed, and if it missed some blocks, it will use the eventGetter to gather all missed exit events and save them into boltdb through eventGetter plugin functions.

Transfer events retry (for voting power changes)

stakeManager (which updates the voting power of validators on childchain, based on Transfer events from ValidatorSet contract, when their stake from root gets bridged to child), now uses eventGetter[contractsapi.TransferEvent] to get missed events. StakeManagerStore already had a mechanism in boltdb to know which block was last processed for TransferEvents so we just plugged in that logic to eventGetter. Same as in checkpointManager, on each block finalization by consensuis or getting the block from syncer, it will read the last processed block from its boltdb store, and will use eventGetter to collect any missed blocks so that it can update the voting power of validators correctly.

State sync retry

Since stateSync events are events gotten from transactions on rootchain, we can not use the eventGetter for this, since it's limited to childchain use only. We get stateSync events from evenTracker, a third party library that syncs with rootchain and gets the desired logs. This is used on edge from the beginning of polybft consensus protocol, but it didn't propagate errors that might happen when client saves the gotten stateSync events to db. AddLog function now returns an error, and eventTrackerStore (used for storing stateSyncEvents), will propagate the error, and syncing with rootchain will retry, meaning that it will try to save the last log to client again.

Changes include

• Bugfix (non-breaking change that solves an issue)
• Hotfix (change that solves an urgent issue, and requires immediate attention)
• New feature (non-breaking change that adds functionality)
• Breaking change (change that is not backwards-compatible and/or changes current functionality)

Checklist

• I have assigned this PR to myself
• I have added at least 1 reviewer
• I have added the relevant labels
• I have updated the official documentation
• I have added sufficient documentation in code

Testing

• I have tested this code with the official test suite
• I have tested this code manually

[vcastellm]

Aside from the small comments LGTM