Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fat] align start of data region to MiB #2387

Closed
wants to merge 1 commit into from

Conversation

slackingfred
Copy link
Contributor

@slackingfred slackingfred commented Dec 18, 2023

Rufus uses fat32format to create FAT32 volumes larger than 32GB. That piece of code is from 10+ years ago and is not optimized for Advanced Format disks. To be specific, when using fat32format, even if the partition itself is 4K (or MiB)-aligned, you may still end up with non-aligned I/O operations, causing performance degradation.

Most modern filesystems, including the younger sibling in the FAT family, uniformly uses "clusters" or "blocks" as (logical) I/O units. For example, NTFS boot code occupies clusters 0 and 1 (typically 8KiB in total); ext4 superblock occupies block 0, even though the superblock itself fits in 1KiB of space, leaving 3KiB unused.

However, FAT or FAT32 only divides the data zone into clusters. In metadata zone (boot code, FATs, etc.) there are only sectors. In case of metadata zone size not being a multiple of cluster size, FAT32 cluster boundaries will not align with physical block boundaries on the underlying device.

An example of misaligned FAT32 internal structure:

Demo of FAT(32) internal misalignment

(Green - logical clusters; yellow - physical blocks; the example uses 1 cluster = 4 logical sectors and 3 "reserved sectors" in total)

Fortunately, the FAT32 number of reserved sectors is pretty arbitrary. By adjusting that number, we'll be able to ensure the data zone begins on physical block boundary.

Since Vista, Windows has been aligning FAT32 data zones aggressively, to multiples of 4MiB; I don't know the notion behind that behavior. Because Rufus (and Windows diskpart) align partitions to 1MiB, I decide to also align the data zone to 1MiB, which is multiple of all common cluster sizes. The resulting "reserved sector count" will be in the range of [32, 2080).

Misaligned data zone under the old behavior (60308 is not a multiple of 64 [sectors per cluster]):

fat32format

Log showing the new behavior (310 reserved sectors):

With_log

Aligned data zone under the new behavior (120832 is a multiple of 32):

Aligned_to_59MB

Chkdsk showing the newly formatted volume is ok:

No_problem

I tested this on two moderately large devices (80G [ancient] HDD and 128G TF card) and was successful booting from the resulting media.

@pbatard
Copy link
Owner

pbatard commented Dec 18, 2023

I don't mind adding this at first glance, but can you explain what problem you are trying to solve?
What is the issue/error that requires this change?

@slackingfred slackingfred marked this pull request as ready for review January 4, 2024 08:30
@slackingfred
Copy link
Contributor Author

I don't mind adding this at first glance, but can you explain what problem you are trying to solve? What is the issue/error that requires this change?

Happy new year! I finally found some time to do tests and write a description for this.

The benchmark data from my 128G TF card justifies the change very well. While sequential performance mostly stayed the same, random write speed increased by a whopping 70% after the change. (Random read improved by a modest 6% to 7%.)

Before:

128 GB (1)

After:

128 GB (2)

I don't have SMR HDDs at hand, but I assume they will exhibit a similar performance improvement. As for SSDs, their advanced firmware and caches would mitigate the performance hit of unaligned I/Os, thus the difference might not be as significant.

@pbatard
Copy link
Owner

pbatard commented Jan 10, 2024

Applied. Thanks!

@pbatard pbatard closed this in 2cebf91 Jan 10, 2024
@Saibamen
Copy link

Why this PR has wrong status? Closed instead of merged?

@pbatard
Copy link
Owner

pbatard commented May 10, 2024

Because not all PRs get applied through GitHub's specific merge actions. On account that the .rc needs to be updated through scripts on every commit, that won't apply through a GitHub merge, I perform an actual merge of a PRs in a single commit (which I see as better than have 2 commits listed, one for the actual change, and one to tell that the PR branch was merged).

Please bear in mind that GitHub != git, so don't expect all repos to behave the same as GitHub repos where people use the default GitHub mechanisms for merging commits, as this'll only give you a very partial and incomplete view of how git can and is actually being used.

Also, "closed" does not mean "rejected". Instead "closed" here very much means that the proposed PR has been applied (in the commit that is referenced on the line that mentions how the PR was closed).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants