wip: remove send + sync from db tx #4215
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
An attempt to remove the
Send + Sync
bounds from database transactions.Removing these opens up the possibility for us to use e.g. redb, or other databases where write transactions are not necessarily at least
Send
.In order to facilitate this, the PR also removes the async portions of the stage trait and the pipeline, instead committing to becoming near-fully synchronous. A few reasons:
Send + Sync
On point 2, @mattsse suggested that we should add a new function to the
Stage
trait that can execute async code and determine whether the stage is ready for execution, akin to theService::poll_ready
function in tower. For headers and bodies, we would poll the downloaders, put the results into a buffer in the stage itself, and determine whether we are ready to write the buffer to disk (in which case the stage is ready and it is executed sync)Some pain points:
Send + Sync
as a boundSend + Sync
, things likeLatestStateProvider
,DatabaseProvider
etc. cannot satisfy this boundRemoving the
Send + Sync
requirement from the provider traits makes everything almost compile, except for RPC.RPC makes heavy use of the provider traits. Adding
Send + Sync
where Rustc complains in RPC works, until you encounter code that use the state provider factory.The state provider factory has functions to get e.g. the latest state, which yields a
Box<dyn StateProvider + 'a>
. AddingSend
to this (as Rustc requires) gives you a new error - the state provider factory cannot fulfull theSend
bound when it wants to return aLatestStateProvider
(or any other provider that use the db tx trait).Not entirely sure what direction to go in here yet, and I already nuked all of my
Send + Sync
bound removal from the providers crate to start fresh