Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core/rawdb: fsync head data file before closing it #26490

Merged
merged 1 commit into from
Jan 13, 2023

Conversation

rjl493456442
Copy link
Member

This PR fixes an issue which might result in data lost in freezer.

Whenever mutation happens in freezer, all data will be written into head data file
and it will be rotated with a new one in case the size of file reaches the threshold.

Theoretically, the rotated old data file should be fsync'd to prevent data loss.
In freezer.Sync function, we only fsync: (1) index file (2) meta file and (3) head
data file. So this PR forcibly fsync the head data file if mutation happens in the
boundary of data file.

Attention, we need to measure to performance impact brought by this change.

@fjl fjl changed the title core/rawdb: fsync head data file once it's full core/rawdb: fsync head data file before closing it Jan 13, 2023
Copy link
Contributor

@holiman holiman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@holiman
Copy link
Contributor

holiman commented Jan 13, 2023

Attention, we need to measure to performance impact brought by this change.

The biggest 'churn' is bodies, which are currently at 148 cdat files. This means that during a sync, we will do an extra fsync 148 times. The total number of data-files are ~225, so basically, we will do fsync an extra two hundred times, during a ~12 hour sync. The performance impact can't be anything to worry about IMO

@holiman holiman added this to the 1.11.0 milestone Jan 13, 2023
@holiman holiman merged commit e04d63e into ethereum:master Jan 13, 2023
shekhirin pushed a commit to shekhirin/go-ethereum that referenced this pull request Jun 6, 2023
This PR fixes an issue which might result in data lost in freezer.

Whenever mutation happens in freezer, all data will be written into head data file
and it will be rotated with a new one in case the size of file reaches the threshold.

Theoretically, the rotated old data file should be fsync'd to prevent data loss.
In freezer.Sync function, we only fsync: (1) index file (2) meta file and (3) head
data file. So this PR forcibly fsync the head data file if mutation happens in the
boundary of data file.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants