Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARC refcount fixes #8000

Closed
wants to merge 2 commits into from
Closed

ARC refcount fixes #8000

wants to merge 2 commits into from

Conversation

behlendorf
Copy link
Contributor

Motivation and Context

Resolve #7219 which is occasionally encountered by ztest and was introduced by the encryption changes.

Additionally, resolve an unrelated reference count discrepancy accidentally introduced when porting compressed send/recv. This may potentially explain #7820 though I was unable to reproduce the symptoms locally.

Description

  • Fix arc_relase() refcount

    Update arc_release to use arc_buf_size(). This hunk was accidentally
    dropped when porting compressed send/recv, 2aa3438.

  • Add zfs_refcount_transfer_ownership_many()

    When debugging is enabled and a zfs_refcount_t contains multiple holders
    using the same key, but different ref_counts, the wrong reference_t may
    be transferred. Add a zfs_refcount_transfer_ownership_many() function,
    like the existing zfs_refcount_*_many() functions, to match and transfer
    the correct refcount_t;

    This issue may occur when using encryption with refcount debugging
    enabled. An arc_buf_hdr_t can have references for both the
    hdr->b_l1hdr.b_pabd and hdr->b_crypt_hdr.b_rabd which both use the
    hdr as the reference holder. When unsharing the buffer the p_abd
    should be transferred.

    This issue does not impact production builds because refcount holders
    are not tracked.

How Has This Been Tested?

Local ztest runs. Prior to this change I was able to reproduce the #7219 approximately once an hour using ztest. With the fix applied I've thus far been unable to reproduce the issue. I'm pushing this change to the CI for additional ztest coverage and will let my local ztest run overnight.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Performance enhancement (non-breaking change which improves efficiency)
  • Code cleanup (non-breaking change which makes code smaller or more readable)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation (a change to man pages or other documentation)

Checklist:

  • My code follows the ZFS on Linux code style requirements.
  • I have updated the documentation accordingly.
  • I have read the contributing document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.
  • All commit messages are properly formatted and contain Signed-off-by.
  • Change has been approved by a ZFS on Linux member.

When debugging is enabled and a zfs_refcount_t contains multiple holders
using the same key, but different ref_counts, the wrong reference_t may
be transferred.  Add a zfs_refcount_transfer_ownership_many() function,
like the existing zfs_refcount_*_many() functions, to match and transfer
the correct refcount_t;

This issue may occur when using encryption with refcount debugging
enabled.  An arc_buf_hdr_t can have references for both the
hdr->b_l1hdr.b_pabd and hdr->b_crypt_hdr.b_rabd which both use the
hdr as the reference holder.  When unsharing the buffer the p_abd
should be transferred.

This issue does not impact production builds because refcount holders
are not tracked.

Signed-off-by: Tom Caputi <tcaputi@datto.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#7219
Update arc_release to use arc_buf_size().  This hunk was accidentally
dropped when porting compressed send/recv, 2aa3438.

Signed-off-by: Tom Caputi <tcaputi@datto.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
TEST_ZTEST_TIMEOUT=3600
Issue openzfs#7219
@behlendorf behlendorf added the Status: Code Review Needed Ready for review and testing label Oct 8, 2018
@behlendorf behlendorf requested a review from tcaputi October 8, 2018 22:36
Copy link
Contributor

@tcaputi tcaputi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Maybe add a comment somewhere that notes that refcounts adds / removes must be matched.

@behlendorf behlendorf added Status: Accepted Ready to integrate (reviewed, tested) and removed Status: Code Review Needed Ready for review and testing labels Oct 9, 2018
@codecov
Copy link

codecov bot commented Oct 9, 2018

Codecov Report

Merging #8000 into master will decrease coverage by 0.26%.
The diff coverage is 71.42%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #8000      +/-   ##
==========================================
- Coverage   78.68%   78.41%   -0.27%     
==========================================
  Files         377      377              
  Lines      114213   114215       +2     
==========================================
- Hits        89871    89566     -305     
- Misses      24342    24649     +307
Flag Coverage Δ
#kernel 78.7% <42.85%> (-0.17%) ⬇️
#user 67.56% <71.42%> (-0.46%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dfbe267...4c4fb62. Read the comment docs.

@behlendorf behlendorf closed this in d7e4b30 Oct 9, 2018
behlendorf added a commit that referenced this pull request Oct 9, 2018
Update arc_release to use arc_buf_size().  This hunk was accidentally
dropped when porting compressed send/recv, 2aa3438.

Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Signed-off-by: Tom Caputi <tcaputi@datto.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #8000
@behlendorf
Copy link
Contributor Author

We should cherry-pick 5e8ff25 for 0.7.12.

tonyhutter pushed a commit to LLNL/zfs that referenced this pull request Oct 10, 2018
Update arc_release to use arc_buf_size().  This hunk was accidentally
dropped when porting compressed send/recv, 2aa3438.

Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Signed-off-by: Tom Caputi <tcaputi@datto.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes openzfs#8000
tonyhutter pushed a commit to tonyhutter/zfs that referenced this pull request Oct 31, 2018
Update arc_release to use arc_buf_size().  This hunk was accidentally
dropped when porting compressed send/recv, 2aa3438.

Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Signed-off-by: Tom Caputi <tcaputi@datto.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes openzfs#8000
tonyhutter pushed a commit to tonyhutter/zfs that referenced this pull request Nov 5, 2018
Update arc_release to use arc_buf_size().  This hunk was accidentally
dropped when porting compressed send/recv, 2aa3438.

Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Signed-off-by: Tom Caputi <tcaputi@datto.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes openzfs#8000
tonyhutter pushed a commit that referenced this pull request Nov 13, 2018
Update arc_release to use arc_buf_size().  This hunk was accidentally
dropped when porting compressed send/recv, 2aa3438.

Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Signed-off-by: Tom Caputi <tcaputi@datto.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #8000
GregorKopka pushed a commit to GregorKopka/zfs that referenced this pull request Jan 7, 2019
When debugging is enabled and a zfs_refcount_t contains multiple holders
using the same key, but different ref_counts, the wrong reference_t may
be transferred.  Add a zfs_refcount_transfer_ownership_many() function,
like the existing zfs_refcount_*_many() functions, to match and transfer
the correct refcount_t;

This issue may occur when using encryption with refcount debugging
enabled.  An arc_buf_hdr_t can have references for both the
hdr->b_l1hdr.b_pabd and hdr->b_crypt_hdr.b_rabd both of which use
the hdr as the reference holder.  When unsharing the buffer the
p_abd should be transferred.

This issue does not impact production builds because refcount holders
are not tracked.

Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Signed-off-by: Tom Caputi <tcaputi@datto.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes openzfs#7219
Closes openzfs#8000
GregorKopka pushed a commit to GregorKopka/zfs that referenced this pull request Jan 7, 2019
Update arc_release to use arc_buf_size().  This hunk was accidentally
dropped when porting compressed send/recv, 2aa3438.

Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Signed-off-by: Tom Caputi <tcaputi@datto.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes openzfs#8000
@behlendorf behlendorf deleted the issue-7219 branch April 19, 2021 21:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Status: Accepted Ready to integrate (reviewed, tested)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ztest fails with "No such hold ... on refcount ..." in arc_free_data_impl->refcount_remove_many
3 participants