Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

zfs-2.2.5 patchset #16359

Merged
merged 48 commits into from
Aug 6, 2024
Merged

Commits on May 13, 2024

  1. Replace P2ALIGN with P2ALIGN_TYPED and delete P2ALIGN.

    In P2ALIGN, the result would be incorrect when align is unsigned
    integer and x is larger than max value of the type of align.
    In that case, -(align) would be a positive integer, which means
    high bits would be zero and finally stay zero after '&' when
    align is converted to a larger integer type.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed-by: Youzhong Yang <yyang@mathworks.com>
    Signed-off-by: Qiuhao Chen <chenqiuhao1997@gmail.com>
    Closes openzfs#15940
    chenqiuhao1997 authored and behlendorf committed May 13, 2024
    Configuration menu
    Copy the full SHA
    9edf6af View commit details
    Browse the repository at this point in the history

Commits on May 29, 2024

  1. Refactor dbuf_read() for safer decryption

    In dbuf_read_verify_dnode_crypt():
     - We don't need original dbuf locked there. Instead take a lock
    on a dnode dbuf, that is actually manipulated.
     - Block decryption for a dnode dbuf if it is currently being
    written.  ARC hash lock does not protect anonymous buffers, so
    arc_untransform() is unsafe when used on buffers being written,
    that may happen in case of encrypted dnode buffers, since they
    are not copied by dbuf_dirty()/dbuf_hold_copy().
    
    In dbuf_read():
     - If the buffer is in flight, recheck its compression/encryption
    status after it is cached, since it may need arc_untransform().
    
    Tested-by: Rich Ercolani <rincebrain@gmail.com>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by:	Alexander Motin <mav@FreeBSD.org>
    Sponsored by:	iXsystems, Inc.
    Closes openzfs#16104
    amotin authored and behlendorf committed May 29, 2024
    Configuration menu
    Copy the full SHA
    b474dfa View commit details
    Browse the repository at this point in the history
  2. L2ARC: Cleanup buffer re-compression

    When compressed ARC is disabled, we may have to re-compress when
    writing into L2ARC.  If doing so we can't fit it into the original
    physical size, we should just fail immediately, since even if it
    may still fit into allocation size, its checksum will never match.
    
    While there, refactor the code similar to other compression places
    without using abd_return_buf_copy().
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by:	Alexander Motin <mav@FreeBSD.org>
    Sponsored by:	iXsystems, Inc.
    Closes openzfs#16038
    amotin authored and behlendorf committed May 29, 2024
    Configuration menu
    Copy the full SHA
    0f1e8ba View commit details
    Browse the repository at this point in the history
  3. Make more taskq parameters writable

    There is no reason for these module parameters to be read-only.
    Being modified they just apply on next pool import/creation, that
    is useful for testing different values.
    
    Reviewed-by: Rich Ercolani <rincebrain@gmail.com>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by:	Alexander Motin <mav@FreeBSD.org>
    Sponsored by:	iXsystems, Inc.
    Closes openzfs#16118
    amotin authored and behlendorf committed May 29, 2024
    Configuration menu
    Copy the full SHA
    938d158 View commit details
    Browse the repository at this point in the history
  4. Slightly improve dnode hash

    As I understand just for being less predictable dnode hash includes
    8 bits of objset pointer, starting at 6.  But since objset_t is
    more than 1KB in size, its allocations are likely aligned to 2KB,
    that means 11 lower bits provide no entropy. Just take the 8 bits
    starting from 11.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by:	Alexander Motin <mav@FreeBSD.org>
    Sponsored by:	iXsystems, Inc.
    Closes openzfs#16131
    amotin authored and behlendorf committed May 29, 2024
    Configuration menu
    Copy the full SHA
    6724746 View commit details
    Browse the repository at this point in the history
  5. Fix scn_queue races on very old pools

    Code for pools before version 11 uses dmu_objset_find_dp() to scan
    for children datasets/clones.  It calls enqueue_clones_cb() and
    enqueue_cb() callbacks in parallel from multiple taskq threads.
    It ends up bad for scan_ds_queue_insert(), corrupting scn_queue
    AVL-tree.  Fix it by introducing a mutex to protect those two
    scan_ds_queue_insert() calls.  All other calls are done from the
    sync thread and so serialized.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed-by: Brian Atkinson <batkinson@lanl.gov>
    Signed-off-by: Alexander Motin <mav@FreeBSD.org>
    Sponsored by:	iXsystems, Inc.
    Closes openzfs#16162
    amotin authored and behlendorf committed May 29, 2024
    Configuration menu
    Copy the full SHA
    41f2a9c View commit details
    Browse the repository at this point in the history
  6. Fix ZIL clone records for legacy holes

    Previous code overengineered cloned range calculation by using
    BP_GET_LSIZE(). The problem is that legacy holes don't have the
    logical size, so result will be wrong.  But we also don't need
    to look on every block size, since they all must be identical.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed-by: Brian Atkinson <batkinson@lanl.gov>
    Signed-off-by: Alexander Motin <mav@FreeBSD.org>
    Sponsored by:	iXsystems, Inc.
    Closes openzfs#16165
    amotin authored and behlendorf committed May 29, 2024
    Configuration menu
    Copy the full SHA
    4c484d6 View commit details
    Browse the repository at this point in the history
  7. ZAP: Fix leaf references on zap_expand_leaf() errors

    Depending on kind of error zap_expand_leaf() may return with or
    without valid leaf reference held.  Make sure it returns NULL if
    due to error it has no leaf to return.  Make its callers to check
    the returned leaf pointer, and release the leaf if it is not NULL.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Alexander Motin <mav@FreeBSD.org>
    Sponsored by:	iXsystems, Inc.
    Closes openzfs#12366 
    Closes openzfs#16159
    amotin authored and behlendorf committed May 29, 2024
    Configuration menu
    Copy the full SHA
    fa4b1a4 View commit details
    Browse the repository at this point in the history
  8. FreeBSD: Add zfs_link_create() error handling

    Originally Solaris didn't expect errors there, but they may happen
    if we fail to add entry into ZAP.  Linux fixed it in openzfs#7421, but it
    was never fully ported to FreeBSD.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Alexander Motin <mav@FreeBSD.org>
    Sponsored-By: iXsystems, Inc.
    Closes openzfs#13215
    Closes openzfs#16138
    amotin authored and behlendorf committed May 29, 2024
    Configuration menu
    Copy the full SHA
    4c0fbd8 View commit details
    Browse the repository at this point in the history
  9. Fix assertion in Persistent L2ARC

    At the end of l2arc_evict() fix an assertion in the case that l2ad_hand
    + distance == l2ad_end.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: George Amanakis <gamanakis@gmail.com>
    Closes openzfs#16202
    Closes openzfs#16207
    gamanakis authored and behlendorf committed May 29, 2024
    Configuration menu
    Copy the full SHA
    2eab4f7 View commit details
    Browse the repository at this point in the history

Commits on Jul 15, 2024

  1. head_errlog: fix use-after-free

    In the commit of the head_errlog feature we introduced a bug in
    dsl_dataset_promote_sync(): we may dereference origin_head and hds, both
    dereferencing ddpa after calling promote_sync() on ddpa.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed-by: Chunwei Chen <david.chen@nutanix.com>
    Reviewed-by: Rob Norris <robn@despairlabs.com>
    Reviewed-by: Tony Hutter <hutter2@llnl.gov>
    Signed-off-by: George Amanakis <gamanakis@gmail.com>
    Closes openzfs#16272
    Closes openzfs#16273
    gamanakis authored Jul 15, 2024
    Configuration menu
    Copy the full SHA
    54ef0fd View commit details
    Browse the repository at this point in the history

Commits on Jul 16, 2024

  1. Linux 6.7 compat: detect if kernel defines intptr_t

    Since Linux 6.7 the kernel has defined intptr_t. Clang has
    -Wtypedef-redefinition by default, which causes the build to fail
    because we also have a typedef for intptr_t.
    
    Since its better to use the kernel's if it exists, detect it and skip
    our own.
    
    Sponsored-by: https://despairlabs.com/sponsor/
    Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Rob Norris <robn@despairlabs.com>
    Closes openzfs#16201
    robn authored and tonyhutter committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    f4e2aed View commit details
    Browse the repository at this point in the history
  2. Linux 6.9: Call add_disk() from workqueue to fix zfs_allow_010_pos (o…

    …penzfs#16282)
    
    The 6.9 kernel behaves differently in how it releases block devices.  In
    the common case it will async release the device only after the return
    to userspace.  This is different from the 6.8 and older kernels which
    release the block devices synchronously.  To get around this, call
    add_disk() from a workqueue so that the kernel uses a different
    codepath to release our zvols in the way we expect.  This stops
    zfs_allow_010_pos from hanging.
    
    Fixes: openzfs#16089
    
    Signed-off-by: Tony Hutter <hutter2@llnl.gov>
    Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de>
    Reviewed-by: Rob Norris <rob.norris@klarasystems.com>
    tonyhutter committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    c24a039 View commit details
    Browse the repository at this point in the history
  3. Linux 6.9: Fix UBSAN errors in zap_micro.c

    You can use the UBSAN_SANITIZE_* Kbuild options to exclude certain
    kernel objects from the UBSAN checks.  We previously excluded
    zap_micro.o with:
    
    UBSAN_SANITIZE_zap_micro.o := n
    
    For some reason that didn't work for the 6.9 kernel, which wants us
    to use:
    
    UBSAN_SANITIZE_zfs/zap_micro.o := n
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Tony Hutter <hutter2@llnl.gov>
    Closes openzfs#16278
    Closes openzfs#16330
    tonyhutter committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    d7bf0e5 View commit details
    Browse the repository at this point in the history
  4. Linux 6.10: rework queue limits setup

    Linux has started moving to a model where instead of applying block
    queue limits through individual modification functions, a complete
    limits structure is built up and applied atomically, either when the
    block device or open, or some time afterwards. As of 6.10 this
    transition appears only partly completed.
    
    This commit matches that model within OpenZFS in a way that should work
    for past and future kernels. We set up a queue limits structure with any
    limits that have had their modification functions removed. For newer
    kernels that can have limits applied at block device open
    (HAVE_BLK_ALLOC_DISK_2ARG), we have a conversion function to turn the
    OpenZFS queue limits structure into Linux's queue_limits structure,
    which can then be passed in. For older kernels, we provide an
    application function that just calls the old functions for each limit in
    the structure.
    
    Signed-off-by: Rob Norris <robn@despairlabs.com>
    Sponsored-by: https://despairlabs.com/sponsor/
    Reviewed-by: Tony Hutter <hutter2@llnl.gov>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    robn authored and tonyhutter committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    0342c4a View commit details
    Browse the repository at this point in the history
  5. Linux 6.10: work harder to avoid kmem_cache_alloc reuse

    Linux 6.10 change kmem_cache_alloc to be a macro, rather than a
    function, such that the old #undef for it in spl-kmem-cache.c would
    remove its definition completely, breaking the build.
    
    This inverts the model used before. Rather than always defining the
    kmem_cache_* macro, then undefining then inside spl-kmem-cache.c,
    instead we make a special tag to indicate we're currently inside
    spl-kmem-cache.c, and not defining those in macros in the first place,
    so we can use the kernel-supplied kmem_cache_* functions to implement
    spl_kmem_cache_*, as we expect.
    
    For all other callers, we create the macros as normal and remove access
    to the kernel's own conflicting names.
    
    Signed-off-by: Rob Norris <robn@despairlabs.com>
    Sponsored-by: https://despairlabs.com/sponsor/
    Reviewed-by: Tony Hutter <hutter2@llnl.gov>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    robn authored and tonyhutter committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    3ea3649 View commit details
    Browse the repository at this point in the history
  6. Linux 5.16: use bdev_nr_bytes() to get device capacity

    This helper was introduced long ago, in 5.16. Since 6.10, bd_inode no
    longer exists, but the helper has been updated, so detect it and use it
    in all versions where it is available.
    
    Signed-off-by: Rob Norris <robn@despairlabs.com>
    Sponsored-by: https://despairlabs.com/sponsor/
    Reviewed-by: Tony Hutter <hutter2@llnl.gov>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    robn authored and tonyhutter committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    7d8e2a7 View commit details
    Browse the repository at this point in the history
  7. ZTS: Fix redacted_send failures on FreeBSD

    We're seeing failures for redacted_deleted and redacted_mount
    on FreeBSD 13-15:
    
        09:58:34.74 diff: /dev/fd/3: No such file or directory
        09:58:34.74 ERROR: diff /dev/fd/3 /dev/fd/4 exited 2
    
    The test was trying to diff the file listings between two directories to
    see if they are the same.  The workaround is to do a string comparison
    of the directory listings instead of using `diff`.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Tony Hutter <hutter2@llnl.gov>
    Closes openzfs#16224
    tonyhutter committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    97f1eb8 View commit details
    Browse the repository at this point in the history
  8. ZTS: handle FreeBSD version numbers correctly (openzfs#16340)

    FreeBSD patchlevel versions are optional and, if present, in a different
    location in the version string.
    
    Sponsored-by: https://despairlabs.com/sponsor/
    
    Signed-off-by: Rob Norris <robn@despairlabs.com>
    Reviewed-by: Alexander Motin <mav@FreeBSD.org>
    Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de>
    Reviewed-by: Tony Hutter <hutter2@llnl.gov>
    robn authored and tonyhutter committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    da9da6a View commit details
    Browse the repository at this point in the history
  9. Linux 6.9 compat: META (openzfs#16358)

    Update the META file to reflect compatibility with the 6.9
    kernel.
    
    Signed-off-by: Tony Hutter <hutter2@llnl.gov>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    tonyhutter committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    bb401c0 View commit details
    Browse the repository at this point in the history
  10. one-word manpage correction: snapshot->rollback (openzfs#16294)

    This commit fixes what is probably a copy-paste mistake. The
    `dracut.zfs` manpage claims that the `bootfs.rollback` option executes
    `zfs snapshot -Rf`. `zfs snapshot` does not have a `-R` option. `zfs
    rollback` does.
    
    Signed-off-by: Alphan Yılmaz <alphanyilmaz@gmail.com>
    Reviewed-by: Rob Norris <rob.norris@klarasystems.com>
    Reviewed-by: George Melikov <mail@gmelikov.ru>
    Reviewed-by: Tony Hutter <hutter2@llnl.gov>
    a1ea321 authored and tonyhutter committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    08da054 View commit details
    Browse the repository at this point in the history
  11. Fix missing semicolon in trace_dbuf.h (openzfs#16281)

    On fedora 40, on the 6.9.4 kernel (in updates-testing), assign_str
    expands to a "do {<stuff> } while(0)" loop.  Without this semicolon,
    the while(0) is unterminated, causing a cascade of useless errors.
    With this semicolon, it compiles fine.  It also compiles fine on 6.8.11
    (the previous kernel).  I have not tested earlier kernels than that, but
    at worst it should add a pointless semicolon.
    
    All other instances in the source tree are already terminated with
    semicolons.
    
    Signed-off-by: Daniel Berlin <dberlin@dberlin.org>
    Reviewed-by: Alexander Motin <mav@FreeBSD.org>
    Reviewed-by: Tony Hutter <hutter2@llnl.gov>
    dberlin authored and tonyhutter committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    dfdac38 View commit details
    Browse the repository at this point in the history

Commits on Jul 17, 2024

  1. zts: allow running a single test by name only

    Specifying a single test is kind of a hassle, because the full relative
    path under the test suite dir has to be included, but it's not always
    clear what that path even is.
    
    This change allows `-t` to take the name of a single test instead of a
    full path. If the value has no `/` characters, we search for a file of
    that name under the test root, and if found, use that as the full test
    path instead.
    
    Sponsored-by: Klara, Inc.
    Sponsored-by: Wasabi Technology, Inc.
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed-by: Akash B <akash-b@hpe.com>
    Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de>
    Signed-off-by: Rob Norris <rob.norris@klarasystems.com>
    Closes openzfs#16088
    robn authored and tonyhutter committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    f14a62e View commit details
    Browse the repository at this point in the history
  2. zts: add a debug option to get full test output

    The test runner accumulates output from individual tests, then writes it
    to the log at the end. If a test hangs or crashes the system half way
    through, we get no insight into how it got to where it did.
    
    This adds a -D option for "debug". When set, all test output is written
    to stdout.
    
    Sponsored-by: Klara, Inc.
    Sponsored-by: Wasabi Technology, Inc.
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed-by: Akash B <akash-b@hpe.com>
    Signed-off-by: Rob Norris <rob.norris@klarasystems.com>
    Closes openzfs#16096
    robn authored and tonyhutter committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    ad8c8c1 View commit details
    Browse the repository at this point in the history
  3. abd_iter_page: rework to handle multipage scatterlists

    Previously, abd_iter_page() would assume that every scatterlist would
    contain a single page (compound or no), because that's all we ever
    create in abd_alloc_chunks(). However, scatterlists can contain multiple
    pages of arbitrary provenance, and if we get one of those, we'd get all
    the math wrong.
    
    This reworks things to handle multiple pages in a scatterlist, by
    properly finding the right page within it for the given offset, and
    understanding better where the end of the page is and not crossing it.
    
    Sponsored-by: Klara, Inc.
    Sponsored-by: Wasabi Technology, Inc.
    Reported-by: Brian Atkinson <batkinson@lanl.gov>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed-by: Brian Atkinson <batkinson@lanl.gov>
    Signed-off-by: Rob Norris <rob.norris@klarasystems.com>
    Closes openzfs#16108
    robn authored and tonyhutter committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    fa2480f View commit details
    Browse the repository at this point in the history
  4. find_system_library: fix var cleanup when library not found

    The "not found" path is attempting to clear SOMELIB_CFLAGS and
    SOMELIB_LIBS by resetting them in AC_SUBST(). However, the second arg to
    AC_SUBST is expanded in autoconf with `m4_ifvaln([$2], [[$1]=$2])`,
    which is defined as "if the first arg is non-empty". The m4 "empty"
    construction is [], therefore, the existing AC_SUBST calls never modify
    the variables at all.
    
    The effect of this is that leftovers from the library test can leak out.
    At least, if a library header is found in the first stage, but the
    library itself is not, -lsomelib is added to SOMELIB_LIBS and further
    tests done. If that library is not found, SOMELIB_LIBS will not be
    cleared.
    
    For most of our library tests this hasn't been a problem, as they're
    either always found properly via pkg-config or set directly, or the
    calling test immediately aborts configure. For an optional dependency
    however, an apparent "partial" result where the header is found but no
    corresponding library causes link errors later.
    
    I think a complete fix should probably not be setting SOMELIB_xxx until
    the final result is known, but for now, adjusting the AC_SUBST calls to
    explictly set the empty shell string (which is not "empty" to m4) at
    least restores the intent.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Rob Norris <robn@despairlabs.com>
    Sponsored-by: https://despairlabs.com/sponsor/
    Closes openzfs#16140
    robn authored and tonyhutter committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    32cd2da View commit details
    Browse the repository at this point in the history
  5. Only provide execvpe(3) when needed

    Check for the existence of execvpe(3) and only provide the FreeBSD
    compat version if required.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by:	Brooks Davis <brooks.davis@sri.com>
    Closes openzfs#15609
    brooksdavis authored and tonyhutter committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    5668411 View commit details
    Browse the repository at this point in the history
  6. libspl/assert: show process/task details in assert output

    Makes it much easier to see what thing complained.
    
    Getting thread id, program name and thread name vary wildly between
    Linux and FreeBSD, so those are set up in macros. pthread_getname_np()
    did not appear in musl until very recently, but the same info has always
    been available via prctl(PR_GET_NAME), so we use that instead.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Rob Norris <robn@despairlabs.com>
    Sponsored-by: https://despairlabs.com/sponsor/
    Closes openzfs#16140
    robn authored and tonyhutter committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    96cad4c View commit details
    Browse the repository at this point in the history
  7. libspl/assert: add lock around assertion output

    If multiple threads trip an assertion at the same moment (quite common),
    they can be printing at the same time, and their output gets messy.
    
    This adds a simple lock around the whole thing, to prevent a second task
    printing assert output before the first has finished.
    
    Additionally, if libspl_assert_ok is not set, abort() is called without
    dropping the lock, so that any other asserting tasks will be killed
    before starting any output, rather than only getting part-way through.
    This is a tradeoff; it's assumed that multiple threads asserting at the
    same moment are likely the same fault in different instances of a
    thread, and so there won't be any more useful information from the other
    tasks anyway.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Rob Norris <robn@despairlabs.com>
    Sponsored-by: https://despairlabs.com/sponsor/
    Closes openzfs#16140
    robn authored and tonyhutter committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    3ca305f View commit details
    Browse the repository at this point in the history
  8. libspl/assert: dump backtrace in assert

    Adds a check for the backtrace() function. If available, uses it to show
    a stack backtrace in the assertion output.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Rob Norris <robn@despairlabs.com>
    Sponsored-by: https://despairlabs.com/sponsor/
    Closes openzfs#16140
    robn authored and tonyhutter committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    21f66db View commit details
    Browse the repository at this point in the history
  9. libspl/assert: use libunwind for backtrace when available

    libunwind seems to do a better job of resolving a symbols than
    backtrace(), and is also useful on platforms that don't have backtrace()
    (eg musl). If it's available, use it.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Rob Norris <robn@despairlabs.com>
    Sponsored-by: https://despairlabs.com/sponsor/
    Closes openzfs#16140
    robn authored and tonyhutter committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    8868621 View commit details
    Browse the repository at this point in the history
  10. Unbreak FreeBSD cross-build on MacOS broken in 051460b

    MacOS used FreeBSD-compatible getprogname() and pthread_getname_np().
    But pthread_getthreadid_np() does not exist on MacOS. This implements
    libspl_gettid() using pthread_threadid_np() to get the thread id
    of the current thread.
    
    Tested with FreeBSD GitHub actions
    freebsd-src/.github/workflows/cross-bootstrap-tools.yml
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed-by: Rob Norris <rob.norris@klarasystems.com>
    Signed-off-by: Martin Matuska <mm@FreeBSD.org>
    Closes openzfs#16167
    mmatuska authored and tonyhutter committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    bc42d96 View commit details
    Browse the repository at this point in the history
  11. libspl_assert: always link -lpthread on FreeBSD

    The pthread_* functions are in -lpthread on FreeBSD. Some of them are
    implicitly linked through libc, but on FreeBSD 13 at least
    pthread_getname_np() is not. Just be explicit, since -lpthread is the
    documented interface anyway.
    
    Sponsored-by: https://despairlabs.com/sponsor/
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Rob Norris <robn@despairlabs.com>
    Closes openzfs#16168
    robn authored and tonyhutter committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    2a2e358 View commit details
    Browse the repository at this point in the history
  12. zdb: bring crash handling over from ztest

    ztest has a very nice ability to show a backtrace when there's an
    unexpected crash. zdb is used often enough on corrupted data and can
    blow up too, so nice output is useful there too.
    
    Sponsored-by: Klara, Inc.
    Sponsored-by: Wasabi Technology, Inc.
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Rob Norris <rob.norris@klarasystems.com>
    Closes openzfs#16181
    robn authored and tonyhutter committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    d06c8de View commit details
    Browse the repository at this point in the history
  13. Use memset to zero stack allocations containing unions

    C99 6.7.8.17 says that when an undesignated initialiser is used, only
    the first element of a union is initialised. If the first element is not
    the largest within the union, how the remaining space is initialised is
    up to the compiler.
    
    GCC extends the initialiser to the entire union, while Clang treats the
    remainder as padding, and so initialises according to whatever
    automatic/implicit initialisation rules are currently active.
    
    When Linux is compiled with CONFIG_INIT_STACK_ALL_PATTERN,
    -ftrivial-auto-var-init=pattern is added to the kernel CFLAGS. This flag
    sets the policy for automatic/implicit initialisation of variables on
    the stack.
    
    Taken together, this means that when compiling under
    CONFIG_INIT_STACK_ALL_PATTERN on Clang, the "zero" initialiser will only
    zero the first element in a union, and the rest will be filled with a
    pattern. This is significant for aes_ctx_t, which in
    aes_encrypt_atomic() and aes_decrypt_atomic() is initialised to zero,
    but then used as a gcm_ctx_t, which is the fifth element in the union,
    and thus gets pattern initialisation. Later, it's assumed to be zero,
    resulting in a hang.
    
    As confusing and undiscoverable as it is, by the spec, we are at fault
    when we initialise a structure containing a union with the zero
    initializer. As such, this commit replaces these uses with an explicit
    memset(0).
    
    Sponsored-by: Klara, Inc.
    Sponsored-by: Wasabi Technology, Inc.
    Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Rob Norris <rob.norris@klarasystems.com>
    Closes openzfs#16135
    Closes openzfs#16206
    robn authored and tonyhutter committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    27cc6df View commit details
    Browse the repository at this point in the history
  14. Destroy ARC buffer in case of fill error

    In case of error dmu_buf_fill_done() returns the buffer back into
    DB_UNCACHED state.  Since during transition from DB_UNCACHED into
    DB_FILL state dbuf_noread() allocates an ARC buffer, we must free
    it here, otherwise it will be leaked.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed-by: Jorgen Lundman <lundman@lundman.net>
    Signed-off-by: Alexander Motin <mav@FreeBSD.org>
    Sponsored by: iXsystems, Inc.
    Closes openzfs#15665
    Closes openzfs#15802
    Closes openzfs#16216
    amotin authored and tonyhutter committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    ba3c769 View commit details
    Browse the repository at this point in the history
  15. Some improvements to metaslabs eviction

    - Add old eviction for special and dedup metaslab classes. Those
    vdevs may be potentially big and fragmented with large metaslabs,
    while their asynchronous write pattern is not really different
    from normal class. It seems an omission to not evict old metaslabs
    from them.
     - If we have metaslab preload enabled, which means we are not too
    low on memory, do not evict active metaslabs even if they are not
    used for some time.  Eviction of active metaslabs means we won't
    be able to write anything until we load them, that may take some
    time, that is straight opposite to metaslab preload goals.  For
    small systems the memory saving should be less important after
    recent reduction in number of allocators and so open metaslabs.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by:	Alexander Motin <mav@FreeBSD.org>
    Sponsored by:	iXsystems, Inc.
    Closes openzfs#16214
    amotin authored and tonyhutter committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    13ccbbb View commit details
    Browse the repository at this point in the history
  16. disable automatic dependency tracking for dkms builds

    Previously the dkms build left some unwanted files
    in `/usr/lib/modules` which could cause package
    managers to not properly clean up old kernels.
    
    Reviewed-by: Tony Hutter <hutter2@llnl.gov>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Martin Wagner <martin.wagner.dev@gmail.com>
    Closes openzfs#16221 
    Closes openzfs#16241
    SoongNoonien authored and tonyhutter committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    c950c5d View commit details
    Browse the repository at this point in the history
  17. zts: test single-disk pool resumes properly after disk pull

    A single disk pool should suspend when its disk fails and hold the IO.
    When the disk is returned, the pool should return and the IO be
    reissued, leaving everything in good shape.
    
    Sponsored-by: Klara, Inc.
    Sponsored-by: Wasabi Technology, Inc.
    Signed-off-by: Rob Norris <rob.norris@klarasystems.com>
    Reviewed-by: Jorgen Lundman <lundman@lundman.net>
    Reviewed-by: Tony Hutter <hutter2@llnl.gov>
    Reviewed-by: Don Brady <don.brady@klarasystems.com>
    robn authored and tonyhutter committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    25c4271 View commit details
    Browse the repository at this point in the history
  18. vdev_open: clear async fault flag after reopen

    After c3f2f1a, vdev_fault_wanted is set on a vdev after a probe fails.
    An end-of-txg async task is charged with actually faulting the vdev.
    
    In a single-disk pool, the probe failure will degrade the last disk, and
    then suspend the pool. However, vdev_fault_wanted is not cleared. After
    the pool returns, the transaction finishes and the async task runs and
    faults the vdev, which suspends the pool again.
    
    The fix is simple: when reopening a vdev, clear the async fault flag. If
    the vdev is still failed, the startup probe will quickly notice and
    degrade/suspend it again. If not, all is well!
    
    Sponsored-by: Klara, Inc.
    Sponsored-by: Wasabi Technology, Inc.
    Co-authored-by: Don Brady <don.brady@klarasystems.com>
    Signed-off-by: Rob Norris <rob.norris@klarasystems.com>
    Reviewed-by: Jorgen Lundman <lundman@lundman.net>
    Reviewed-by: Tony Hutter <hutter2@llnl.gov>
    Reviewed-by: Don Brady <don.brady@klarasystems.com>
    2 people authored and tonyhutter committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    4d2f7f9 View commit details
    Browse the repository at this point in the history
  19. zdb: dump ZAP_FLAG_UINT64_KEY ZAPs properly (openzfs#16334)

    These are used for DDT and BRT stores. There's limited information
    available to produce meaningful output, but at least we can put
    something on screen rather than crashing.
    
    Sponsored-by: Klara, Inc.
    Sponsored-by: Wasabi Technology, Inc.
    
    Signed-off-by: Rob Norris <rob.norris@klarasystems.com>
    Reviewed-by: Alexander Motin <mav@FreeBSD.org>
    Reviewed-by: Tony Hutter <hutter2@llnl.gov>
    robn authored and tonyhutter committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    9835255 View commit details
    Browse the repository at this point in the history

Commits on Jul 23, 2024

  1. FreeBSD: Use a statement expression to implement SET_ERROR() (openzfs…

    …#16284)
    
    This way we can avoid making assumptions about the SDT probe
    implementation.  No functional change intended.
    
    Signed-off-by: Mark Johnston <markj@FreeBSD.org>
    Reviewed-by: Alexander Motin <mav@FreeBSD.org>
    Reviewed-by: Allan Jude <allan@klarasystems.com>
    Reviewed-by: Rob Norris <rob.norris@klarasystems.com>
    Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de>
    Reviewed-by: Tony Hutter <hutter2@llnl.gov>
    markjdb authored and tonyhutter committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    14cce09 View commit details
    Browse the repository at this point in the history
  2. AUTHORS: refresh with recent new contributors (openzfs#16362)

    Sponsored-by: https://despairlabs.com/sponsor/
    
    Signed-off-by: Rob Norris <robn@despairlabs.com>
    Reviewed-by: Tony Hutter <hutter2@llnl.gov>
    Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de>
    Reviewed-by: George Melikov <mail@gmelikov.ru>
    robn authored and tonyhutter committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    9ad205e View commit details
    Browse the repository at this point in the history
  3. Fix long_free_dirty accounting for small files (openzfs#16264)

    For files smaller than recordsize, it's most likely that they don't have
    L1 blocks. However, current calculation will always return at least 1 L1
    block.
    
    In this change, we check dnode level to figure out if it has L1 blocks
    or not, and return 0 if it doesn't. This will reduce the chance of
    unnecessary throttling when deleting a large number of small files.
    
    Signed-off-by: Chunwei Chen <david.chen@nutanix.com>
    Co-authored-by: Chunwei Chen <david.chen@nutanix.com>
    Reviewed-by: Alexander Motin <mav@FreeBSD.org>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    2 people authored and tonyhutter committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    ef08cb2 View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2024

  1. Linux 6.9: Fix UBSAN errors in sa.c (openzfs#16380)

    This is a follow-on to 156a641
    that ignores UBSAN errors in sa.c.
    
    Thank you @thwalker3 for the fix.
    
    Original-patch-by: @thwalker3
    Signed-off-by: Tony Hutter <hutter2@llnl.gov>
    Closes openzfs#16278
    Closes openzfs#16330
    Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    tonyhutter committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    b5835ed View commit details
    Browse the repository at this point in the history

Commits on Jul 30, 2024

  1. ZTS: Make do_vol_test() more deterministic (openzfs#16379)

    - Explicitly disable compression since mkfile uses a zero buffer.
     - Explicitly sync file systems instead of waiting for timeout.
    
    Signed-off-by:	Alexander Motin <mav@FreeBSD.org>
    Sponsored by:	iXsystems, Inc.
    Reviewed-by: George Melikov <mail@gmelikov.ru>
    Reviewed-by: Tony Hutter <hutter2@llnl.gov>
    amotin authored and tonyhutter committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    dd5de55 View commit details
    Browse the repository at this point in the history

Commits on Aug 3, 2024

  1. [2.2.5-only] Make 'rmmod zfs' work after zfs-2.2.4 (openzfs#16406)

    db65272 was added to zfs-2.2.4 to stub in the
    VDEV_PROP_RAIDZ_EXPANDING enum without adding the RAIDz expansion
    feature.  This was needed to provide the right enum count for when the
    VDEV_PROP_SLOW_IO proprieties got added.  This had the unfortunate side
    effect of breaking module removal though.
    
    Specifically, with the VDEV_PROP_RAIDZ_EXPANDING stub added,
    the module would correctly omit making kobjects for the RAIDz expansion
    vdev property, but then would try to blindly remove its non-existent
    kobjects during module unload.
    
    This commit fixes the issue by checking for an uninitialized kobject.
    
    Fixes: openzfs#16249
    
    Signed-off-by: Tony Hutter <hutter2@llnl.gov>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed-by: Ameer Hamza <ahamza@ixsystems.com>
    Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de>
    tonyhutter committed Aug 3, 2024
    Configuration menu
    Copy the full SHA
    6f27c4c View commit details
    Browse the repository at this point in the history
  2. Tag zfs-2.2.5

    META file and changelog updated.
    
    Signed-off-by: Tony Hutter <hutter2@llnl.gov>
    tonyhutter committed Aug 3, 2024
    Configuration menu
    Copy the full SHA
    33174af View commit details
    Browse the repository at this point in the history