Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend import_progress kstat with a notes field #15539

Merged
merged 1 commit into from
Dec 5, 2023

Conversation

don-brady
Copy link
Contributor

Motivation and Context

As reported and observed in other issues, the time for an unclean pool import can take a long time. It would be nice to be able to observe where the time is being spent for imports that are taking a long time to complete.

Description

This change adds more details to the existing import_progress kstat in the form of import progress notes.

This information is also logged to the zfs debug messages.

Sponsored-By: OpenDrives Inc.
Sponsored-By: Klara Inc.

Here's an example of output from the zfs debug messages:

1700151770   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' Starting import
1700151770   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' using uberblock with txg=33
1700151770   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' Loading checkpoint txg
1700151770   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' Loading indirect vdev metadata
1700151770   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' Checking feature flags
1700151770   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' Loading special MOS directories
1700151770   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' Loading properties
1700151770   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' Loading AUX vdevs
1700151770   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' Loading vdev metadata
1700151770   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' read 0 of 18 log space maps in 0 ms
1700151770   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' read 1 of 18 log space maps in 0 ms
1700151770   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' read 2 of 18 log space maps in 0 ms
1700151770   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' read 3 of 18 log space maps in 0 ms
1700151770   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' read 4 of 18 log space maps in 0 ms
1700151770   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' read 5 of 18 log space maps in 0 ms
1700151770   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' read 6 of 18 log space maps in 0 ms
1700151770   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' read 7 of 18 log space maps in 1 ms
1700151770   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' read 8 of 18 log space maps in 1 ms
1700151770   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' read 9 of 18 log space maps in 1 ms
1700151770   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' read 10 of 18 log space maps in 1 ms
1700151770   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' read 11 of 18 log space maps in 1 ms
1700151770   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' read 12 of 18 log space maps in 1 ms
1700151770   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' read 13 of 18 log space maps in 1 ms
1700151770   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' read 14 of 18 log space maps in 1 ms
1700151770   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' read 15 of 18 log space maps in 1 ms
1700151770   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' read 16 of 18 log space maps in 1 ms
1700151770   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' read 17 of 18 log space maps in 1 ms
1700151770   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' read 18 log space maps (18 total blocks - blksz = 131072 bytes) in 2 ms
1700151770   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' Loading dedup tables
1700151770   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' Loading BRT
1700151770   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' Verifying Log Devices
1700151771   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' Verifying pool data
1700151771   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' Calculating deflated space
1700151771   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' Starting import
1700151771   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' Claiming ZIL blocks
1700151771   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' Syncing ZIL claims
1700151771   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' Updating configs
1700151771   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' Starting resilvers
1700151771   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' Restarting device removals
1700151771   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' Cleaning up inconsistent objsets
1700151771   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' Cleaning up temporary uerrefs
1700151771   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' Restarting Initialize
1700151771   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' Restarting TRIM
1700151771   spa_misc.c:2306:spa_import_progress_set_notes(): 'testpool1' Finished importing

How Has This Been Tested?

Added a new ZTS test, zpool_import_status, to verify the kstat and zfs dbgmsg outputs.
Also ran ztest and performed some manual pool import testing

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Performance enhancement (non-breaking change which improves efficiency)
  • Code cleanup (non-breaking change which makes code smaller or more readable)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
  • Documentation (a change to man pages or other documentation)

Checklist:

module/zfs/spa_log_spacemap.c Outdated Show resolved Hide resolved
module/zfs/spa_misc.c Outdated Show resolved Hide resolved
include/sys/spa.h Outdated Show resolved Hide resolved
Copy link
Contributor

@behlendorf behlendorf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should also add some progress notes to spa_activity_check() since with MMP enabled this can take a significant amount of time when the pool hasn't been cleanly exported.

module/zfs/spa.c Outdated Show resolved Hide resolved
@behlendorf behlendorf added the Status: Code Review Needed Ready for review and testing label Nov 17, 2023
@don-brady
Copy link
Contributor Author

Updated to address review feedback

module/zfs/spa.c Outdated Show resolved Hide resolved
module/zfs/spa.c Outdated Show resolved Hide resolved
module/zfs/spa.c Outdated Show resolved Hide resolved
module/zfs/spa.c Outdated Show resolved Hide resolved
module/zfs/spa.c Outdated Show resolved Hide resolved
module/zfs/spa_log_spacemap.c Outdated Show resolved Hide resolved
module/zfs/spa.c Outdated Show resolved Hide resolved
module/zfs/spa_misc.c Outdated Show resolved Hide resolved
module/zfs/spa_misc.c Outdated Show resolved Hide resolved
module/zfs/spa.c Outdated Show resolved Hide resolved
module/zfs/spa.c Outdated Show resolved Hide resolved
@don-brady
Copy link
Contributor Author

Another pass to address Alexander's feedback.

Detail the import progress of log spacemaps as they can take a very
long time.  Also grab the spa_note() messages to, as they provide
insight into what is happening

Sponsored-By: OpenDrives Inc.
Sponsored-By: Klara Inc.
Co-authored-by: Don Brady <don.brady@klarasystems.com>
Signed-off-by: Don Brady <don.brady@klarasystems.com>
@don-brady
Copy link
Contributor Author

Fixed Linux-specific date format

@behlendorf behlendorf added Status: Accepted Ready to integrate (reviewed, tested) and removed Status: Code Review Needed Ready for review and testing labels Dec 5, 2023
@behlendorf behlendorf merged commit 687e4d7 into openzfs:master Dec 5, 2023
22 of 26 checks passed
lundman pushed a commit to openzfsonwindows/openzfs that referenced this pull request Dec 12, 2023
Detail the import progress of log spacemaps as they can take a very
long time.  Also grab the spa_note() messages to, as they provide
insight into what is happening

Sponsored-By: OpenDrives Inc.
Sponsored-By: Klara Inc.
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Don Brady <don.brady@klarasystems.com>
Co-authored-by: Allan Jude <allan@klarasystems.com>
Closes openzfs#15539
tonyhutter pushed a commit that referenced this pull request May 2, 2024
Detail the import progress of log spacemaps as they can take a very
long time.  Also grab the spa_note() messages to, as they provide
insight into what is happening

Sponsored-By: OpenDrives Inc.
Sponsored-By: Klara Inc.
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Don Brady <don.brady@klarasystems.com>
Co-authored-by: Allan Jude <allan@klarasystems.com>
Closes #15539
allanjude pushed a commit to KlaraSystems/zfs that referenced this pull request May 21, 2024
Detail the import progress of log spacemaps as they can take a very
long time.  Also grab the spa_note() messages to, as they provide
insight into what is happening

Sponsored-By: OpenDrives Inc.
Sponsored-By: Klara Inc.
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Don Brady <don.brady@klarasystems.com>
Co-authored-by: Allan Jude <allan@klarasystems.com>
Closes openzfs#15539
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Status: Accepted Ready to integrate (reviewed, tested)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants