Improve performance for zpool trim on Linux #15843
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
usaleem-ix
force-pushed
the
NAS-125882
branch
from
a839588
to
96c10a5
Compare
On Linux, ZFS uses blkdev_issue_discard in vdev_disk_io_trim to issue trim command which is synchronus. This commit updates vdev_disk_io_trim to use __blkdev_issue_discard, which is asynchronus. Unfortunately there isn't any asynchronus version for blkdev_issue_secure_erase, so performance of secure trim will still suffer. Signed-off-by: Umer Saleem <usaleem@ixsystems.com>
usaleem-ix
force-pushed
the
NAS-125882
branch
from
96c10a5
to
6beb1c1
Compare
amotin
approved these changes
Feb 1, 2024
behlendorf
requested changes
Feb 1, 2024
behlendorf
approved these changes
Feb 2, 2024
behlendorf
added
Status: Accepted
tonyhutter
pushed a commit
that referenced
this pull request
Feb 22, 2024
On Linux, ZFS uses blkdev_issue_discard in vdev_disk_io_trim to issue trim command which is synchronous. This commit updates vdev_disk_io_trim to use __blkdev_issue_discard, which is asynchronous. Unfortunately there isn't any asynchronous version for blkdev_issue_secure_erase, so performance of secure trim will still suffer. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Alexander Motin <mav@FreeBSD.org> Signed-off-by: Umer Saleem <usaleem@ixsystems.com> Closes #15843
lundman
pushed a commit
to openzfsonwindows/openzfs
that referenced
this pull request
Mar 13, 2024
On Linux, ZFS uses blkdev_issue_discard in vdev_disk_io_trim to issue trim command which is synchronous. This commit updates vdev_disk_io_trim to use __blkdev_issue_discard, which is asynchronous. Unfortunately there isn't any asynchronous version for blkdev_issue_secure_erase, so performance of secure trim will still suffer. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Alexander Motin <mav@FreeBSD.org> Signed-off-by: Umer Saleem <usaleem@ixsystems.com> Closes openzfs#15843
lundman
pushed a commit
to openzfsonwindows/openzfs
that referenced
this pull request
Mar 13, 2024
On Linux, ZFS uses blkdev_issue_discard in vdev_disk_io_trim to issue trim command which is synchronous. This commit updates vdev_disk_io_trim to use __blkdev_issue_discard, which is asynchronous. Unfortunately there isn't any asynchronous version for blkdev_issue_secure_erase, so performance of secure trim will still suffer. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Alexander Motin <mav@FreeBSD.org> Signed-off-by: Umer Saleem <usaleem@ixsystems.com> Closes openzfs#15843
lundman
pushed a commit
to openzfsonwindows/openzfs
that referenced
this pull request
Mar 13, 2024
On Linux, ZFS uses blkdev_issue_discard in vdev_disk_io_trim to issue trim command which is synchronous. This commit updates vdev_disk_io_trim to use __blkdev_issue_discard, which is asynchronous. Unfortunately there isn't any asynchronous version for blkdev_issue_secure_erase, so performance of secure trim will still suffer. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Alexander Motin <mav@FreeBSD.org> Signed-off-by: Umer Saleem <usaleem@ixsystems.com> Closes openzfs#15843
|
@usaleem-ix do you have any info on hand about the performance improvements this change gives? I'm not doubting that there's something there, I just want to understand the before & after a little better before I make any changes in this area. Thanks! |
|
@robn there was an issue where users with typically large pools were experiencing very slow trim operation. Before this change, we saw that ZFS uses synchronous TRIM KPIs on Linux ( The problem with synchronous KPI is that ZFS currently has only 4 TRIM issue taskq threads, that means on Linux it can have only 4 TRIM requests per pool running same time. That may be OK for small non-fragmented pool, but for some cases it becomes a problem. So we introduced this patch to use asynchronous KPI ( Unfortunately, I don't have performance improvement numbers at hand for this. |
|
No problem, I appreciate the info! That's about what my guess was too. Cheers! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation and Context
On Linux, ZFS uses
blkdev_issue_discardinvdev_disk_io_trimto issue trim command which is synchronus. Inblkdev_issue_discard,submit_bio_waitis used to after calling__blkdev_issue_discard.Description
This commit updates
vdev_disk_io_trimto use__blkdev_issue_discard, which is asynchronous, and submit bio viavdev_submit_bio.Unfortunately, there isn't any asynchronus version for
blkdev_issue_secure_erase, so performance of secure trim will still suffer.How Has This Been Tested?
CI test runs and manual runs of
zpool trim <pool>Types of changes
Checklist:
Signed-off-by.