zstreamdump: add per-record-type counters and an overhead counter

[allanjude]

Signed-off-by: Allan Jude allanjude@freebsd.org

Motivation and Context

zstreamdump is a tool used to analyze the contents of ZFS replication streams.
I am using zstreamdump to debug zfs send size estimation, and found it
useful to get a breakdown of the actual size of the replication stream, and
what that size was made up of.

Description

Count the bytes of payload for each replication record type
Count the bytes of overhead (replication records themselves)
Include these counters in the output summary at the end of the run.

Example output:

SUMMARY:
SUMMARY:
        Total DRR_BEGIN records = 5 (752 bytes)
        Total DRR_END records = 6 (0 bytes)
        Total DRR_OBJECT records = 764290 (130437264 bytes)
        Total DRR_FREEOBJECTS records = 39 (0 bytes)
        Total DRR_WRITE records = 504587 (3902546432 bytes)
        Total DRR_WRITE_BYREF records = 0 (0 bytes)
        Total DRR_WRITE_EMBEDDED records = 267961 (17836736 bytes)
        Total DRR_FREE records = 768868 (0 bytes)
        Total DRR_SPILL records = 0 (0 bytes)
        Total records = 2305756
        Total payload size = 4050821184 (0xf172a040)
        Total header overhead = 719395872 (0x2ae11c20)
        Total stream length = 4770217056 (0x11c53bc60)

How Has This Been Tested?

The counters correctly sum up to the total stream length, which matches the actual stream length.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Performance enhancement (non-breaking change which improves efficiency)
• Code cleanup (non-breaking change which makes code smaller or more readable)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation (a change to man pages or other documentation)

Checklist:

• My code follows the ZFS on Linux code style requirements.
• I have updated the documentation accordingly.
• I have read the contributing document.
• I have added tests to cover my changes.
• All new and existing tests passed.
• All commit messages are properly formatted and contain Signed-off-by.

[behlendorf]

Thanks, this'll be really handy. Just a few suggestions.

[ahrens]

I think the summary should have a "total payload size" and "total header overhead", the sum of which would be the "total stream length". The "total payload" seems more relevant than "total WRITE record payload" which is what we have today (I think).

[behlendorf]

Thanks for refreshing this. It looks like several of the rsend tests failed because they parse the output of zstreamdump. I'd expect that updating the verify_stream_size function in zfs-tests/tests/functional/rsend/rsend.kshlib would should resolve the issue.

[codecov]

Codecov Report

Merging #8432 into master will increase coverage by 0.06%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #8432      +/-   ##
==========================================
+ Coverage   78.51%   78.58%   +0.06%     
==========================================
  Files         382      382              
  Lines      117840   117858      +18     
==========================================
+ Hits        92526    92613      +87     
+ Misses      25314    25245      -69

Flag	Coverage Δ
#kernel	79.27% <ø> (+0.04%)	⬆️
#user	66.84% <100%> (-0.19%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b1b4ac2...800ada4. Read the comment docs.

[ahrens]

If you feel like cleaning this up a bit, we could avoid repeating the P2ROUNDUP 3x:

payload_size = ...;
ssread(... payload_size ...);
if (dump)
  print_block (... payload_size)