Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: swap Dashmap for StaticFileWriters on StaticFileProvider #10089

Merged
merged 7 commits into from
Aug 7, 2024

Conversation

joshieDo
Copy link
Collaborator

@joshieDo joshieDo commented Aug 5, 2024

Introduces:

pub(crate) struct StaticFileWriters {
    headers: RwLock<Option<StaticFileProviderRW>>,
    transactions: RwLock<Option<StaticFileProviderRW>>,
    receipts: RwLock<Option<StaticFileProviderRW>>,
}

This allows holding mutable references to two different static file segment writers without risk of deadlocking. Which is not possible with dashmap

This allows reverting #10069 .

Run the script with 0/100 failures

@joshieDo joshieDo added the A-static-files Related to static files label Aug 5, 2024
@joshieDo joshieDo requested a review from mattsse August 5, 2024 14:58
@joshieDo joshieDo requested a review from fgimenez August 5, 2024 15:00
Comment on lines 23 to 29
/// Static file writers for every known [`StaticFileSegment`].
#[derive(Debug, Default)]
pub(crate) struct StaticFileWriters {
headers: RwLock<Option<StaticFileProviderRW>>,
transactions: RwLock<Option<StaticFileProviderRW>>,
receipts: RwLock<Option<StaticFileProviderRW>>,
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah I see this is now flattened.

imo this is a lot more readible.

@joshieDo joshieDo added the S-blocked This cannot more forward until something else changes label Aug 5, 2024
@joshieDo
Copy link
Collaborator Author

joshieDo commented Aug 5, 2024

blocking until the next release just in case


fn deref(&self) -> &Self::Target {
// This is always created by [`StaticFileWriters::get_or_create`]
self.0.as_ref().expect("should exist")
Copy link
Member

@emhane emhane Aug 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
self.0.as_ref().expect("should exist")
self.0.as_ref().expect("static file writer provider should be init")

very hacky, but ok, since we don't have a trait for the static file writer providers here right, so can't do noop impl for () instead of Option<StaticFileProviderRW>.

more verbose error message helps find it in the code base though, cause this is bound to cause a bug in future when we rebuild smthg in the bootstrap process.

@emhane
Copy link
Member

emhane commented Aug 5, 2024

in general - good call. was thinking after #10034, that it's better not to abstract away locks anywhere in the codebase with dashmap, harder to debug than when locking is explicit in reth code.

@joshieDo joshieDo removed the S-blocked This cannot more forward until something else changes label Aug 7, 2024
@joshieDo joshieDo enabled auto-merge August 7, 2024 10:32
@joshieDo joshieDo added this pull request to the merge queue Aug 7, 2024
Merged via the queue into main with commit 7486d5b Aug 7, 2024
33 checks passed
@joshieDo joshieDo deleted the joshie/sf-writers branch August 7, 2024 10:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-static-files Related to static files
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants