Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

zfs-2.1.7 patchset #14162

Merged
merged 83 commits into from
Dec 1, 2022
Merged

Commits on Oct 6, 2022

  1. initramfs: use mount.zfs instead of mount

    A followup to d7a6740
    
    For `mount -t zfs -o opts ds mp` command line
    some implementations of `mount(8)`, e. g. Busybox in Debian
    work as follows:
    
    ```
    newfstatat(AT_FDCWD, "ds", 0x7fff826f4ab0, 0) = -1
    mount("ds", "mp", "zfs", MS_SILENT, NULL) = 0
    ```
    
    The logic above skips completely `mount.zfs` and prevents us
    from reading filesystem properties and applying mount options.
    
    For comparison, the coreutils `mount(8)` implementation does:
    
    ```
    openat(AT_FDCWD, "/proc/filesystems", O_RDONLY|O_CLOEXEC) = 3
    // figure out that zfs is a `nodev` filesystem and look for a helper
    newfstatat(AT_FDCWD, "/sbin/mount.zfs" ...) = 0
    execve("/sbin/mount.zfs" ...) = 0
    ```
    
    Using `mount.zfs` in initramfs would help circumvent deficiencies
    of some of `mount(8)` implementations. `mount -t zfs` translates
    to `mount.zfs` invocation, except for cases when explicitly disabled
    by `-i`.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: szubersk <szuberskidamian@gmail.com>
    Closes openzfs#13305
    (cherry picked from commit 35d81a7)
    szubersk authored and behlendorf committed Oct 6, 2022
    1 Configuration menu
    Copy the full SHA
    4d22bef View commit details
    Browse the repository at this point in the history

Commits on Oct 12, 2022

  1. etc: mask zfs-load-key.service

    Otherwise, systemd-sysv-generator will generate a service equivalent
    that breaks the boot: under systemd this is covered by
    zfs-mount-generator
    
    We already do this for zfs-import.service, and other init scripts are
    suppressed automatically by the "actual" .service files
    
    Fixes: commit f04b976 ("Add init script
     to load keys")
    Reviewed-by: George Melikov <mail@gmelikov.ru>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
    Closes openzfs#14010
    Closes openzfs#14019
    nabijaczleweli authored Oct 12, 2022
    Configuration menu
    Copy the full SHA
    8cf59e9 View commit details
    Browse the repository at this point in the history
  2. kcfpool_alloc() should have its argument list marked void

    This error occurred when building on Gentoo with debugging enabled:
    
    zfs-kmod-2.1.6/work/zfs-2.1.6/module/icp/core/kcf_sched.c:1277:14:
    error: a function declaration without a prototype is deprecated
    in all versions of C [-Werror,-Wstrict-prototypes]
      kcfpool_alloc()
                   ^
                   void
    1 error generated.
    
    This function is not present in master.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed-by: Tony Hutter <hutter2@llnl.gov>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#14023
    ryao authored Oct 12, 2022
    Configuration menu
    Copy the full SHA
    b0bc882 View commit details
    Browse the repository at this point in the history

Commits on Oct 20, 2022

  1. contrib: dracut: zfs-{rollback,snapshot}-bootfs: explicit snapname fix

    Due to a missing semicolon on the ExecStart line, it wasn't possible
    to specify the snapshot name on the bootfs.{rollback,snapshot}
    kernel parameters if the boot dataset name was obtained from the
    root=zfs:... kernel parameter.
    
    Reviewed-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Gregory Bartholomew <gregory.lee.bartholomew@gmail.com>
    Closes openzfs#13585
    gregory-lee-bartholomew authored and behlendorf committed Oct 20, 2022
    Configuration menu
    Copy the full SHA
    3b935cc View commit details
    Browse the repository at this point in the history
  2. contrib: dracut: zfs-snapshot-bootfs: exit status fix

    Correct misplaced `-` is the original backport of openzfs#13769.
    
    Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Issue openzfs#13769
    behlendorf committed Oct 20, 2022
    Configuration menu
    Copy the full SHA
    7795975 View commit details
    Browse the repository at this point in the history

Commits on Oct 21, 2022

  1. Fix sequential resilver drive failure race condition

    This patch handles the race condition on simultaneous failure of
    2 drives, which misses the vdev_rebuild_reset_wanted signal in
    vdev_rebuild_thread. We retry to catch this inside the
    vdev_rebuild_complete_sync function.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Reviewed-by: Dipak Ghosh <dipak.ghosh@hpe.com>
    Reviewed-by: Akash B <akash-b@hpe.com>
    Signed-off-by: Samuel Wycliffe J <samwyc@hpe.com>
    Closes openzfs#14041
    Closes openzfs#14050
    samwyc authored and behlendorf committed Oct 21, 2022
    Configuration menu
    Copy the full SHA
    fc1c005 View commit details
    Browse the repository at this point in the history

Commits on Oct 26, 2022

  1. FreeBSD: Fix a pair of bugs in zfs_fhtovp()

    - Add a zfs_exit() call in an error path, otherwise a lock is leaked.
    - Remove the fid_gen > 1 check.  That appears to be Linux-specific:
      zfsctl_snapdir_fid() sets fid_gen to 0 or 1 depending on whether the
      snapshot directory is mounted.  On FreeBSD it fails, making snapshot
      dirs inaccessible via NFS.
    
    Reviewed-by: Alexander Motin <mav@FreeBSD.org>
    Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
    Reviewed-by: Andriy Gapon <avg@FreeBSD.org>
    Signed-off-by: Mark Johnston <markj@FreeBSD.org>
    Fixes: 43dbf88 ("FreeBSD: vfsops: use setgen for error case")
    Closes openzfs#14001
    Closes openzfs#13974
    (cherry picked from commit ed566bf)
    markjdb authored and behlendorf committed Oct 26, 2022
    Configuration menu
    Copy the full SHA
    4e3fecb View commit details
    Browse the repository at this point in the history

Commits on Oct 27, 2022

  1. FreeBSD: vn_flush_cached_data: observe vnode locking contract

    vm_object_page_clean() expects that the associated vnode is locked
    as VOP_PUTPAGES() may get called on the vnode.
    
    Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
    Signed-off-by: Andriy Gapon <avg@FreeBSD.org>
    Closes openzfs#14079
    (cherry picked from commit 41133c9)
    avg-I authored and behlendorf committed Oct 27, 2022
    Configuration menu
    Copy the full SHA
    04f1983 View commit details
    Browse the repository at this point in the history

Commits on Nov 1, 2022

  1. Add options to zfs redundant_metadata property

    Currently, additional/extra copies are created for metadata in
    addition to the redundancy provided by the pool(mirror/raidz/draid),
    due to this 2 times more space is utilized per inode and this decreases
    the total number of inodes that can be created in the filesystem. By
    setting redundant_metadata to none, no additional copies of metadata
    are created, hence can reduce the space consumed by the additional
    metadata copies and increase the total number of inodes that can be
    created in the filesystem.  Additionally, this can improve file create
    performance due to the reduced amount of metadata which needs
    to be written.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed-by: Dipak Ghosh <dipak.ghosh@hpe.com>
    Signed-off-by: Akash B <akash-b@hpe.com>
    Closes openzfs#13680
    akashb-22 authored and behlendorf committed Nov 1, 2022
    Configuration menu
    Copy the full SHA
    7ac732b View commit details
    Browse the repository at this point in the history
  2. Bring per_txg_dirty_frees_percent back to 30

    The current value causes significant artificial slowdown during mass
    parallel file removal, which can be observed both on FreeBSD and Linux
    when running real workloads.
    
    Sample results from Linux doing make -j 96 clean after an allyesconfig
    modules build:
    
    before: 4.14s user 6.79s system 48% cpu 22.631 total
    after:	4.17s user 6.44s system 153% cpu 6.927 total
    
    FreeBSD results in the ticket.
    
    Reviewed-by: Alexander Motin <mav@FreeBSD.org>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Reviewed-by: George Melikov <mail@gmelikov.ru>
    Signed-off-by:	Mateusz Guzik <mjguzik@gmail.com>
    Closes openzfs#13932
    Closes openzfs#13938
    mjguzik authored and behlendorf committed Nov 1, 2022
    Configuration menu
    Copy the full SHA
    c8d6a91 View commit details
    Browse the repository at this point in the history
  3. Fix panic in dsl_process_sub_livelist for EINTR

    = Issue
    
    Recently we hit an assertion panic in `dsl_process_sub_livelist` while
    exporting the spa and interrupting `bpobj_iterate_nofree`. In that case
    `bpobj_iterate_nofree` stops mid-way returning an EINTR without clearing
    the intermediate AVL tree that keeps track of the livelist entries it
    has encountered so far. At that point the code has a VERIFY for the
    number of elements of the AVL expecting it to be zero (which is not the
    case for EINTR).
    
    = Fix
    
    Cleanup any intermediate state before destroying the AVL when
    encountering EINTR. Also added a comment documenting the scenario where
    the EINTR comes up. There is no need to do anything else for the calles
    of `dsl_process_sub_livelist` as they already handle the EINTR case.
    
    Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
    Reviewed-by: Mark Maybee <mark.maybee@delphix.com>
    Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Signed-off-by: Serapheim Dimitropoulos <serapheim@delphix.com>
    Closes openzfs#13939
    sdimitro authored and behlendorf committed Nov 1, 2022
    Configuration menu
    Copy the full SHA
    37763ea View commit details
    Browse the repository at this point in the history
  4. Remove ambiguity on demand vs prefetch stats reported by arc_summary

    arc_summary currently list prefetch stats as "demand prefetch"
    However, a hit/miss can be due to demand or prefetch, not both.
    To remove any confusion, this patch removes the "Demand" word
    from the affected lines.
    
    Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Reviewed-by: Alexander Motin <mav@FreeBSD.org>
    Reviewed-by: George Melikov <mail@gmelikov.ru>
    Signed-off-by: Gionatan Danti <g.danti@assyoma.it>
    Closes openzfs#13985
    shodanshok authored and behlendorf committed Nov 1, 2022
    Configuration menu
    Copy the full SHA
    820edcb View commit details
    Browse the repository at this point in the history
  5. zvol_wait logic may terminate prematurely

    Setups that have a lot of zvols may see zvol_wait terminate prematurely
    even though the script is still making progress.  For example, we have a
    customer that called zvol_wait for ~7100 zvols and by the last iteration
    of that script it was still waiting on ~2900. Similarly another one
    called zvol_wait for 2200 and by the time the script terminated there
    were only 50 left.
    
    This patch adjusts the logic to stay within the outer loop of the script
    if we are making any progress whatsoever.
    
    Reviewed-by: George Wilson <gwilson@delphix.com>
    Reviewed-by: Pavel Zakharov <pavel.zakharov@delphix.com>
    Reviewed-by: Don Brady <don.brady@delphix.com>
    Signed-off-by: Serapheim Dimitropoulos <serapheim@delphix.com>
    Closes openzfs#13998
    sdimitro authored and behlendorf committed Nov 1, 2022
    Configuration menu
    Copy the full SHA
    25096e1 View commit details
    Browse the repository at this point in the history
  6. Stop ganging due to past vdev write errors

    = Problem
    
    While examining a customer's system we noticed unreasonable space
    usage from a few snapshots due to gang blocks. Under some further
    analysis we discovered that the pool would create gang blocks because
    all its disks had non-zero write error counts and they'd be skipped
    for normal metaslab allocations due to the following if-clause in
    `metaslab_alloc_dva()`:
    ```
    	/*
    	 * Avoid writing single-copy data to a failing,
    	 * non-redundant vdev, unless we've already tried all
    	 * other vdevs.
    	 */
    	if ((vd->vdev_stat.vs_write_errors > 0 ||
    	    vd->vdev_state < VDEV_STATE_HEALTHY) &&
    	    d == 0 && !try_hard && vd->vdev_children == 0) {
    		metaslab_trace_add(zal, mg, NULL, psize, d,
    		    TRACE_VDEV_ERROR, allocator);
    		goto next;
    	}
    ```
    
    = Proposed Solution
    
    Get rid of the predicate in the if-clause that checks the past
    write errors of the selected vdev. We still try to allocate from
    HEALTHY vdevs anyway by checking vdev_state so the past write
    errors doesn't seem to help us (quite the opposite - it can cause
    issues in long-lived pools like the one from our customer).
    
    = Testing
    
    I first created a pool with 3 vdevs:
    ```
    $ zpool list -v volpool
    NAME        SIZE  ALLOC   FREE
    volpool    22.5G   117M  22.4G
      xvdb     7.99G  40.2M  7.46G
      xvdc     7.99G  39.1M  7.46G
      xvdd     7.99G  37.8M  7.46G
    ```
    
    And used `zinject` like so with each one of them:
    ```
    $ sudo zinject -d xvdb -e io -T write -f 0.1 volpool
    ```
    
    And got the vdevs to the following state:
    ```
    $ zpool status volpool
      pool: volpool
     state: ONLINE
    status: One or more devices has experienced an unrecoverable error.
    ...<cropped>..
    action: Determine if the device needs to be replaced, and clear the
    ...<cropped>..
    config:
    
    	NAME        STATE     READ WRITE CKSUM
    	volpool     ONLINE       0     0     0
    	  xvdb      ONLINE       0     1     0
    	  xvdc      ONLINE       0     1     0
    	  xvdd      ONLINE       0     4     0
    
    ```
    
    I also double-checked their write error counters with sdb:
    ```
    sdb> spa volpool | vdev | member vdev_stat.vs_write_errors
    (uint64_t)0  # <---- this is the root vdev
    (uint64_t)2
    (uint64_t)1
    (uint64_t)1
    ```
    
    Then I checked that I the problem was reproduced in my VM as I the
    gang count was growing in zdb as I was writting more data:
    ```
    $ sudo zdb volpool | grep gang
            ganged count:              1384
    
    $ sudo zdb volpool | grep gang
            ganged count:              1393
    
    $ sudo zdb volpool | grep gang
            ganged count:              1402
    
    $ sudo zdb volpool | grep gang
            ganged count:              1414
    ```
    
    Then I updated my bits with this patch and the gang count stayed the
    same.
    
    Reviewed-by: Mark Maybee <mark.maybee@delphix.com>
    Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
    Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Signed-off-by: Serapheim Dimitropoulos <serapheim@delphix.com>
    Closes openzfs#14003
    sdimitro authored and behlendorf committed Nov 1, 2022
    Configuration menu
    Copy the full SHA
    37d5a3e View commit details
    Browse the repository at this point in the history
  7. CI: bump actions/checkout to v3

    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: George Melikov <mail@gmelikov.ru>
    Closes openzfs#14018
    gmelikov authored and behlendorf committed Nov 1, 2022
    Configuration menu
    Copy the full SHA
    1bbc09e View commit details
    Browse the repository at this point in the history
  8. CI: bump actions/upload-artifact to v3

    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: George Melikov <mail@gmelikov.ru>
    Closes openzfs#14018
    gmelikov authored and behlendorf committed Nov 1, 2022
    Configuration menu
    Copy the full SHA
    4dd9c3b View commit details
    Browse the repository at this point in the history
  9. Linux: Remove ZFS_AC_KERNEL_SRC_MODULE_PARAM_CALL_CONST autotools check

    On older kernels, the definition for `module_param_call()` typecasts
    function pointers to `(void *)`, which triggers -Werror, causing the
    check to return false when it should return true.
    
    Fixing this breaks the build process on some older kernels because they
    define a `__check_old_set_param()` function in their headers that checks
    for a non-constified `->set()`. We workaround that through the c
    preprocessor by defining `__check_old_set_param(set)` to `(set)`, which
    prevents the build failures.
    
    However, it is now apparent that all kernels that we support have
    adopted the GRSecurity change, so there is no need to have an explicit
    autotools check for it anymore. We therefore remove the autotools check,
    while adding the workaround to our headers for the build time
    non-constified `->set()` check done by older kernel headers.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed-by: Jorgen Lundman <lundman@lundman.net>
    Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
    Reviewed-by: Alexander Motin <mav@FreeBSD.org>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#13984
    Closes openzfs#14004
    ryao authored and behlendorf committed Nov 1, 2022
    Configuration menu
    Copy the full SHA
    7a1b6c5 View commit details
    Browse the repository at this point in the history
  10. zfs_domount: fix double-disown of dataset / double-free of zfsvfs_t

    Before this patch, in zfs_domount, if zfs_root or d_make_root fails, we
    leave zfsvfs != NULL. This will lead to execution of the error handling
    `if` statement at the `out` label, and hence to a call to
    dmu_objset_disown and zfsvfs_free.
    
    However, zfs_umount, which we call upon failure of zfs_root and
    d_make_root already does dmu_objset_disown and zfsvfs_free.
    
    I suppose this patch rather adds to the brittleness of this part of the
    code base, but I don't want to invest more time in this right now.
    To add a regression test, we'd need some kind of fault injection
    facility for zfs_root or d_make_root, which doesn't exist right now.
    And even then, I think that regression test would be too closely tied
    to the implementation.
    
    To repro the double-disown / double-free, do the following:
    1. patch zfs_root to always return an error
    2. mount a ZFS filesystem
    
    Here's the stack trace you would see then:
    
      VERIFY3(ds->ds_owner == tag) failed (0000000000000000 == ffff9142361e8000)
      PANIC at dsl_dataset.c:1003:dsl_dataset_disown()
      Showing stack for process 28332
      CPU: 2 PID: 28332 Comm: zpool Tainted: G           O      5.10.103-1.nutanix.el7.x86_64 #1
      Call Trace:
       dump_stack+0x74/0x92
       spl_dumpstack+0x29/0x2b [spl]
       spl_panic+0xd4/0xfc [spl]
       dsl_dataset_disown+0xe9/0x150 [zfs]
       dmu_objset_disown+0xd6/0x150 [zfs]
       zfs_domount+0x17b/0x4b0 [zfs]
       zpl_mount+0x174/0x220 [zfs]
       legacy_get_tree+0x2b/0x50
       vfs_get_tree+0x2a/0xc0
       path_mount+0x2fa/0xa70
       do_mount+0x7c/0xa0
       __x64_sys_mount+0x8b/0xe0
       do_syscall_64+0x38/0x50
       entry_SYSCALL_64_after_hwframe+0x44/0xa9
    
    Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
    Co-authored-by: Christian Schwarz <christian.schwarz@nutanix.com>
    Signed-off-by: Christian Schwarz <christian.schwarz@nutanix.com>
    Closes openzfs#14025
    problame authored and behlendorf committed Nov 1, 2022
    Configuration menu
    Copy the full SHA
    df00027 View commit details
    Browse the repository at this point in the history
  11. Linux compat: fix DECLARE_EVENT_CLASS() test when ZFS is built-in

    ZFS_LINUX_TRY_COMPILE_HEADER macro doesn't take CONFIG_ZFS=y into
    account. As a result, on several latest Linux versions, configure
    script marks DECLARE_EVENT_CLASS() available for non-GPL when ZFS
    is being built as a module, but marks it unavailable when ZFS is
    built-in.
    Follow the logic of the neighbor macros and adjust
    ZFS_LINUX_TRY_COMPILE_HEADER accordingly, so that it doesn't try
    to look for a .ko when ZFS is built-in.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Signed-off-by: Alexander Lobakin <alobakin@pm.me>
    Closes openzfs#14006
    solbjorn authored and behlendorf committed Nov 1, 2022
    Configuration menu
    Copy the full SHA
    3e767e3 View commit details
    Browse the repository at this point in the history
  12. Linux 6.0 compat: META

    Update the META file to reflect compatibility with the 6.0 kernel.
    
    Reviewed-by: George Melikov <mail@gmelikov.ru>
    Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Closes openzfs#14091
    behlendorf committed Nov 1, 2022
    Configuration menu
    Copy the full SHA
    7ce097c View commit details
    Browse the repository at this point in the history
  13. Linux 6.1 compat: change order of sys/mutex.h includes

    After Linux 6.1-rc1 came out, the build started failing to build a
    couple of the files in the linux spl code due to the mutex_init
    redefinition. Moving the sys/mutex.h include to a lower position within
    these two files appears to fix the problem.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Coleman Kane <ckane@colemankane.org>
    Closes openzfs#14040
    ckane authored and behlendorf committed Nov 1, 2022
    Configuration menu
    Copy the full SHA
    212ba9b View commit details
    Browse the repository at this point in the history
  14. Propagate extent_bytes change to autotrim thread

    The autotrim thread only reads zfs_trim_extent_bytes_min and
    zfs_trim_extent_bytes_max variable only on thread start.  We
    should check for parameter changes during thread execution to
    allow parameter changes take effect without needing to disable
    then restart the autotrim.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Václav Skála <skala@vshosting.cz>
    Closes openzfs#14077
    vaclavskala authored and behlendorf committed Nov 1, 2022
    Configuration menu
    Copy the full SHA
    8929355 View commit details
    Browse the repository at this point in the history
  15. Fix ARC target collapse when zfs_arc_meta_limit_percent=100

    Reclaim metadata when arc_available_memory < 0 even if
    meta_used is not bigger than arc_meta_limit.
    
    As described in openzfs#14054 if
    zfs_arc_meta_limit_percent=100 then ARC target can collapse to
    arc_min due to arc_purge not freeing any metadata.
    
    This patch lets arc_prune to do its work when arc_available_memory
    is negative even if meta_used is not bigger than arc_meta_limit,
    avoiding ARC target collapse.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Gionatan Danti <g.danti@assyoma.it>
    Closes openzfs#14054 
    Closes openzfs#14093
    shodanshok authored and behlendorf committed Nov 1, 2022
    Configuration menu
    Copy the full SHA
    1d2b056 View commit details
    Browse the repository at this point in the history
  16. quota: extend quota for dataset

    This patch relax the quota limitation for dataset by around 3%.
    What this means is that user can write more data then the quota is
    set to. However thanks to that we can get more stable bandwidth, in
    case when we are overwriting data in-place, and not consuming any
    additional space.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Allan Jude <allan@klarasystems.com>
    Signed-off-by: Mariusz Zaborski <oshogbo@vexillium.org>
    Sponsored-by: Zededa Inc.
    Sponsored-by: Klara Inc.
    Closes openzfs#13839
    oshogbo authored and behlendorf committed Nov 1, 2022
    Configuration menu
    Copy the full SHA
    186e39f View commit details
    Browse the repository at this point in the history
  17. zil: Relax assertion in zil_parse

    Rather than panic debug builds when we fail to parse a whole ZIL, let's
    instead improve the logging of errors and continue like in a release
    build.
    
    Reviewed-by: Alexander Motin <mav@FreeBSD.org>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Ryan Moeller <ryan@iXsystems.com>
    Closes openzfs#14116
    Ryan Moeller authored and behlendorf committed Nov 1, 2022
    Configuration menu
    Copy the full SHA
    b27c7a1 View commit details
    Browse the repository at this point in the history

Commits on Nov 4, 2022

  1. Deny receiving into encrypted datasets if the keys are not loaded (op…

    …enzfs#14139)
    
    Commit 68ddc06 introduced support
    for receiving unencrypted datasets as children of encrypted ones but
    unfortunately got the logic upside down. This resulted in failing to
    deny receives of incremental sends into encrypted datasets without
    their keys loaded. If receiving a filesystem, the receive was done
    into a newly created unencrypted child dataset of the target. In
    case of volumes the receive made the target volume undeletable since
    a dataset was created below it, which we obviously can't handle.
    Incremental streams with embedded blocks are affected as well.
    
    We fix the broken logic to properly deny receives in such cases.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Attila Fülöp <attila@fueloep.org>
    Closes openzfs#13598
    Closes openzfs#14055
    Closes openzfs#14119
    AttilaFueloep authored Nov 4, 2022
    Configuration menu
    Copy the full SHA
    cd1f023 View commit details
    Browse the repository at this point in the history

Commits on Nov 7, 2022

  1. zed: Prevent special vdev to be replaced by hot spare

    Special vdevs should not be replaced by a hot spare.
    Log vdevs already support this, extending the
    functionality for special vdevs.
    
    Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
    Reviewed-by: Tony Hutter <hutter2@llnl.gov>
    Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Reviewed-by: Alexander Motin <mav@FreeBSD.org>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Ameer Hamza <ahamza@ixsystems.com>
    Closes openzfs#14129
    ixhamza authored and behlendorf committed Nov 7, 2022
    Configuration menu
    Copy the full SHA
    ca3a675 View commit details
    Browse the repository at this point in the history

Commits on Nov 8, 2022

  1. zed: Avoid core dump if wholedisk property does not exist

    zed aborts and dumps core in vdev_whole_disk_from_config() if
    wholedisk property does not exist. make_leaf_vdev() adds the
    property but there may be already pools that don't have the
    wholedisk in the label.
    
    Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
    Reviewed-by: Alexander Motin <mav@FreeBSD.org>
    Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Signed-off-by: Ameer Hamza <ahamza@ixsystems.com>
    Closes openzfs#14062
    ixhamza authored and behlendorf committed Nov 8, 2022
    Configuration menu
    Copy the full SHA
    8c0684d View commit details
    Browse the repository at this point in the history
  2. dsl_prop_known_index(): check for invalid prop

    Resolve UBSAN array-index-out-of-bounds error in zprop_desc_t.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: szubersk <szuberskidamian@gmail.com>
    Closes openzfs#14142
    Closes openzfs#14147
    szubersk authored and behlendorf committed Nov 8, 2022
    Configuration menu
    Copy the full SHA
    0f4ee29 View commit details
    Browse the repository at this point in the history

Commits on Nov 30, 2022

  1. libzfs recv: Check if user prop before inheritable

    User props trigger an assert in zfs_prop_inheritable(), we must check
    if the prop is a user prop first.
    
    Signed-off-by: Ryan Moeller <ryan@iXsystems.com>
    
    Backported as snippit from:
    63652e1 Add --enable-asan and --enable-ubsan switches
    Ryan Moeller authored and tonyhutter committed Nov 30, 2022
    Configuration menu
    Copy the full SHA
    1d9aa83 View commit details
    Browse the repository at this point in the history
  2. icp: properly fix all RETs in x86_64 Asm code

    Commit 43569ee ("Fix objtool: missing int3 after ret warning")
    addressed replacing all `ret`s in x86 asm code to a macro in the
    Linux kernel in order to enable SLS. That was done by copying the
    upstream macro definitions and fixed objtool complaints.
    Since then, several more mitigations were introduced, including
    Rethunk. It requires to have a jump to one of the thunks in order
    to work, so the RET macro was changed again. And, as ZFS code
    didn't use the mainline defition, but copied it, this is currently
    missing.
    
    Objtool reminds about it time to time (Clang 16, CONFIG_RETHUNK=y):
    
    fs/zfs/lua/zlua.o: warning: objtool: setjmp+0x25: 'naked' return
     found in RETHUNK build
    fs/zfs/lua/zlua.o: warning: objtool: longjmp+0x27: 'naked' return
     found in RETHUNK build
    
    Do it the following way:
    * if we're building under Linux, unconditionally include
      <linux/linkage.h> in the related files. It is available in x86
      sources since even pre-2.6 times, so doesn't need any conftests;
    * then, if RET macro is available, it will be used directly, so that
      we will always have the version actual to the kernel we build;
    * if there's no such macro, we define it as a simple `ret`, as it
      was on pre-SLS times.
    
    This ensures we always have the up-to-date definition with no need
    to update it manually, and at the same time is safe for the whole
    variety of kernels ZFS module supports.
    Then, there's a couple more "naked" rets left in the code, they're
    just defined as:
    
    	.byte 0xf3,0xc3
    
    In fact, this is just:
    
    	rep ret
    
    `rep ret` instead of just `ret` seems to mitigate performance issues
    on some old AMD processors and most likely makes no sense as of
    today.
    Anyways, address those rets, so that they will be protected with
    Rethunk and SLS. Include <sys/asm_linkage.h> here which now always
    has RET definition and replace those constructs with just RET.
    This wipes the last couple of places with unpatched rets objtool's
    been complaining about.
    
    Reviewed-by: Attila Fülöp <attila@fueloep.org>
    Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de>
    Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Alexander Lobakin <alobakin@pm.me>
    Closes openzfs#14035
    solbjorn authored and tonyhutter committed Nov 30, 2022
    Configuration menu
    Copy the full SHA
    ee93cbc View commit details
    Browse the repository at this point in the history
  3. icp: fix rodata being marked as text in x86 Asm code

    objtool properly complains that it can't decode some of the
    instructions from ICP x86 Asm code. As mentioned in the Makefile,
    where those object files were excluded from objtool check (but they
    can still be visible under IBT and LTO), those are just constants,
    not code.
    In that case, they must be placed in .rodata, so they won't be
    marked as "allocatable, executable" (ax) in EFL headers and this
    effectively prevents objtool from trying to decode this data. That
    reveals a whole bunch of other issues in ICP Asm code, as previously
    objtool was bailing out after that warning message.
    
    Reviewed-by: Attila Fülöp <attila@fueloep.org>
    Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de>
    Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Alexander Lobakin <alobakin@pm.me>
    Closes openzfs#14035
    
    Conflicts:
    	module/Kbuild.in
    solbjorn authored and tonyhutter committed Nov 30, 2022
    Configuration menu
    Copy the full SHA
    33bc03d View commit details
    Browse the repository at this point in the history

Commits on Dec 1, 2022

  1. icp: fix all !ENDBR objtool warnings in x86 Asm code

    Currently, only Blake3 x86 Asm code has signs of being ENDBR-aware.
    At least, under certain conditions it includes some header file and
    uses some custom macro from there.
    Linux has its own NOENDBR since several releases ago. It's defined
    in the same <asm/linkage.h>, so currently <sys/asm_linkage.h>
    already is provided with it.
    
    Let's unify those two into one %ENDBR macro. At first, check if it's
    present already. If so -- use Linux kernel version. Otherwise, try
    to go that second way and use %_CET_ENDBR from <cet.h> if available.
    If no, fall back to just empty definition.
    This fixes a couple more 'relocations to !ENDBR' across the module.
    And now that we always have the latest/actual ENDBR definition, use
    it at the entrance of the few corresponding functions that objtool
    still complains about. This matches the way how it's used in the
    upstream x86 core Asm code.
    
    Reviewed-by: Attila Fülöp <attila@fueloep.org>
    Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de>
    Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Alexander Lobakin <alobakin@pm.me>
    Closes openzfs#14035
    solbjorn authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    ab22031 View commit details
    Browse the repository at this point in the history
  2. zpool_load_compat() should create strings of length ZFS_MAXPROPLEN

    Otherwise, `strlcat()` can overflow them.
    
    Coverity found this.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed-by: Neal Gompa <ngompa@datto.com>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#13866
    ryao authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    7928257 View commit details
    Browse the repository at this point in the history
  3. Cleanup: Make memory barrier definitions consistent across kernels

    We inherited membar_consumer() and membar_producer() from OpenSolaris,
    but we had replaced membar_consumer() with Linux's smp_rmb() in
    zfs_ioctl.c. The FreeBSD SPL consequently implemented a shim for the
    Linux-only smp_rmb().
    
    We reinstate membar_consumer() in platform independent code and fix the
    FreeBSD SPL to implement membar_consumer() in a way analogous to Linux.
    
    Reviewed-by: Konstantin Belousov <kib@FreeBSD.org>
    Reviewed-by: Mateusz Guzik <mjguzik@gmail.com>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed-by: Neal Gompa <ngompa@datto.com>
    Reviewed-by: Alexander Motin <mav@FreeBSD.org>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#13843
    ryao authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    b247d47 View commit details
    Browse the repository at this point in the history
  4. Remove incorrect free() in zfs_get_pci_slots_sys_path()

    Coverity found this. We attempted to free tmp, which is a pointer to a
    string that should be freed by the caller.
    
    Reviewed-by: Neal Gompa <ngompa@datto.com>
    Reviewed-by: Tony Hutter <hutter2@llnl.gov>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#13864
    ryao authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    3f380df View commit details
    Browse the repository at this point in the history
  5. Fix use-after-free bugs in icp code

    These were reported by Coverity as "Read from pointer after free" bugs.
    Presumably, it did not report it as a use-after-free bug because it does
    not understand the inline assembly that implements the atomic
    instruction.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#13881
    ryao authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    1d5e569 View commit details
    Browse the repository at this point in the history
  6. Handle ECKSUM as new EZFS_CKSUM ‒ "insufficient replicas"

    Add a meaningful error message for ECKSUM to common error messages.
    
    Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
    Closes openzfs#6805
    Closes openzfs#13808
    Closes openzfs#13898
    nabijaczleweli authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    39a39b8 View commit details
    Browse the repository at this point in the history
  7. Fix null pointer dereferences in PAM

    Coverity caught these.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#13889
    ryao authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    12b859c View commit details
    Browse the repository at this point in the history
  8. Linux: Fix use-after-free in zfsvfs_create()

    Coverity reported that we pass a pointer to zfsvfs to
    `dmu_objset_disown()` after freeing zfsvfs in zfsvfs_create_impl() after
    a failure in zfsvfs_init().
    
    We have nearly identical duplicate versions of this code for FreeBSD and
    Linux, but interestingly, the FreeBSD version of this code differs in
    such a way that it does not suffer from this bug. We remove the
    difference from the FreeBSD version to fix this bug.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#13883
    ryao authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    9f1691a View commit details
    Browse the repository at this point in the history
  9. FreeBSD: Fix uninitialized pointer read in spa_import_rootpool()

    The FreeBSD project's coverity scans found this.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#13923
    ryao authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    c6d93d0 View commit details
    Browse the repository at this point in the history
  10. Fix potential NULL pointer dereference in zfsdle_vdev_online()

    Coverity complained about this.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed-by: Chunwei Chen <david.chen@nutanix.com>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#13903
    ryao authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    1ff8f41 View commit details
    Browse the repository at this point in the history
  11. Call va_end() before return in zpool_standard_error_fmt()

    Commit ecd6cf800b63704be73fb264c3f5b6e0dafc068d by marks in OpenSolaris
    at Tue Jun 26 07:44:24 2007 -0700 introduced a bug where we fail to call
    `va_end()` before returning.
    
    The man page for va_start() says:
    
    "Each invocation of va_start() must be matched by a corresponding
    invocation of va_end() in the same function."
    
    Coverity complained about this.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed-by: Chunwei Chen <david.chen@nutanix.com>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#13904
    ryao authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    fbe150f View commit details
    Browse the repository at this point in the history
  12. set_global_var_parse_kv() should pass the pointer from strdup()

    A comment says that the caller should free k_out, but the pointer passed
    via k_out is not the same pointer we received from strdup(). Instead,
    it is a pointer into the region we received from strdup(). The free
    function should always be called with the original pointer, so this is
    likely a bug.
    
    We solve this by calling `strdup()` a second time and then freeing the
    original pointer.
    
    Coverity reported this as a memory leak.
    
    Reviewed-by: Neal Gompa <ngompa@datto.com>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#13867
    ryao authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    e11c432 View commit details
    Browse the repository at this point in the history
  13. PAM: Fix uninitialized value read

    Clang's static analyzer found that config.uid is uninitialized when
    zfs_key_config_load() returns an error.
    
    Oddly, this was not included in the unchecked return values that
    Coverity found.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#13957
    ryao authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    5313611 View commit details
    Browse the repository at this point in the history
  14. Fix unreachable code in zstreamdump

    82226e4 was intended to prevent a
    warning from being printed in situations where it was inappropriate, but
    accidentally disabled it entirely by setting featureflags in the wrong
    case statement.
    
    Coverity reported this as dead code.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#13946
    ryao authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    d4df36d View commit details
    Browse the repository at this point in the history
  15. Fix potential NULL pointer dereference in dsl_dataset_promote_check()

    If the `list_head()` returns NULL, we dereference it, right before we
    check to see if it returned NULL.
    
    We have defined two different pointers that both point to the same
    thing, which are `origin_head` and `origin_ds`. Almost everything uses
    `origin_ds`, so we switch them to use `origin_ds`.
    
    We also promote `origin_ds` to a const pointer so that the compiler
    verifies that nothing modifies it.
    
    Coverity complained about this.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed-by: Neal Gompa <ngompa@datto.com>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#13967
    ryao authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    c562bbe View commit details
    Browse the repository at this point in the history
  16. PAM: Fix unchecked return value from zfs_key_config_load()

    9a49c6b was intended to fix this issue,
    but I had missed the case in pam_sm_open_session(). Clang's static
    analyzer had not reported it and I forgot to look for other cases.
    
    Interestingly, GCC gcc-12.1.1_p20220625's static analyzer had caught
    this as multiple double-free bugs, since another failure after the
    failure in zfs_key_config_load() will cause us to attempt to free the
    memory that zfs_key_config_load() was supposed to allocate, but had
    cleaned up upon failure.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#13978
    ryao authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    fa74250 View commit details
    Browse the repository at this point in the history
  17. scripts/enum-extract.pl should not hard code perl path

    This is a portability issue. The issue had already been fixed for
    scripts/cstyle.pl by 2dbf1bf.
    scripts/enum-extract.pl was added to the repository the following year
    without this portability fix.
    
    Michael Bishop informed me that this broke his attempt to build ZFS
    2.1.6 on NixOS, since he was building manually outside of their package
    manager (that usually rewrites the shebangs to NixOS' unusual paths).
    NixOS puts all of the paths into $PATH, so scripts that portably rely
    on env to find the interpreter still work.
    
    Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de>
    Reviewed-by: George Melikov <mail@gmelikov.ru>
    Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#14012
    ryao authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    d05f247 View commit details
    Browse the repository at this point in the history
  18. Fix potential NULL pointer dereference in lzc_ioctl()

    Users are allowed to pass NULL to resultp, but we unconditionally assume
    that they never do. When an external user does pass NULL to resultp, we
    dereference a NULL pointer.
    
    Clang's static analyzer complained about this.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#14008
    ryao authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    d016ca1 View commit details
    Browse the repository at this point in the history
  19. Fix theoretical array overflow in lua_typename()

    Out of the 12 defects in lua that coverity reports, 5 of them involve
    `lua_typename()` and out of the dozens of defects in ZFS that lua
    reports, 3 of them involve `lua_typename()` due to the ZCP code. Given
    all of the uses of `lua_typename()` in the ZCP code, I was surprised
    that there were not more. It appears that only 2 were reported because
    only 3 called `lua_type()`, which does a defective sanity check that
    allows invalid types to be passed.
    
    lua/lua@d4fb848 addressed this in
    upstream lua 5.3. Unfortunately, we did not get that fix since we use
    lua 5.2 and we do not have assertions enabled in lua, so the upstream
    solution would not do anything.
    
    While we could adopt the upstream solution and enable assertions, a
    simpler solution is to fix the issue by making `lua_typename()` return
    `internal_type_error` whenever it is called with an invalid type. This
    avoids the array overflow and if we ever see it appear somewhere, we
    will know there is a problem with the lua interpreter.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#13947
    ryao authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    2453f90 View commit details
    Browse the repository at this point in the history
  20. ZED: Fix uninitialized value reads

    Coverity complained about a couple of uninitialized value reads in ZED.
    
     * zfs_deliver_dle() can pass an uninitialized string to zed_log_msg()
     * An uninitialized sev.sigev_signo is passed to timer_create()
    
    The former would log garbage while the latter is not a real issue, but
    we might as well suppress it by initializing the field to 0 for
    consistency's sake.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#14047
    ryao authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    4351d18 View commit details
    Browse the repository at this point in the history
  21. Fix NULL pointer dereference in zdb

    Clang's static analyzer complained that we dereference a NULL pointer in
    dump_path() if we return 0 when there is an error.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#14044
    ryao authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    5eaad8b View commit details
    Browse the repository at this point in the history
  22. fm_fmri_hc_create() must call va_end() before returning

    clang-tidy caught this.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#14044
    ryao authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    a5f17a9 View commit details
    Browse the repository at this point in the history
  23. Fix NULL pointer passed to strlcpy from zap_lookup_impl()

    Clang's static analyzer pointed out that whenever zap_lookup_by_dnode()
    is called, we have the following stack where strlcpy() is passed a NULL
    pointer for realname from zap_lookup_by_dnode():
    
    strlcpy()
    zap_lookup_impl()
    zap_lookup_norm_by_dnode()
    zap_lookup_by_dnode()
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#14044
    ryao authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    bbec0e6 View commit details
    Browse the repository at this point in the history
  24. Fix NULL pointer dereference in spa_open_common()

    Calling spa_open() will pass a NULL pointer to spa_open_common()'s
    config parameter. Under the right circumstances, we will dereference the
    config parameter without doing a NULL check.
    
    Clang's static analyzer found this.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#14044
    ryao authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    409c99a View commit details
    Browse the repository at this point in the history
  25. set_global_var() should not pass NULL pointers to dlclose()

    Both Coverity and Clang's static analyzer caught this.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#14044
    ryao authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    89c41f3 View commit details
    Browse the repository at this point in the history
  26. Fix possible NULL pointer dereference in sha2_mac_init()

    If mechanism->cm_param is NULL, passing mechanism to
    PROV_SHA2_GET_DIGEST_LEN() will dereference a NULL pointer.
    
    Coverity reported this.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#14044
    ryao authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    af2e53f View commit details
    Browse the repository at this point in the history
  27. Fix memory leaks in dmu_send()/dmu_send_obj()

    If we encounter an EXDEV error when using the redacted snapshots
    feature, the memory used by dspp.fromredactsnaps is leaked.
    
    Clang's static analyzer caught this during an experiment in which I had
    annotated various headers in an attempt to improve the results of static
    analysis.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#13973
    ryao authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    3830858 View commit details
    Browse the repository at this point in the history
  28. Fix theoretical use of uninitialized values

    Clang's static analyzer complains about this.
    
    In get_configs(), if we have an invalid configuration that has no top
    level vdevs, we can read a couple of uninitialized variables. Aborting
    upon seeing this would break the userland tools for healthy pools, so we
    instead initialize the two variables to 0 to allow the userland tools to
    continue functioning for the pools with valid configurations.
    
    In zfs_do_wait(), if no wait activities are enabled, we read an
    uninitialized error variable.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#14043
    ryao authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    4c59fde View commit details
    Browse the repository at this point in the history
  29. Remove zpl_revalidate: fix snapshot rollback

    Open files, which aren't present in the snapshot, which is being
    roll-backed to, need to disappear from the visible VFS image of
    the dataset.
    
    Kernel provides d_drop function to drop invalid entry from
    the dcache, but inode can be referenced by dentry multiple dentries.
    
    The introduced zpl_d_drop_aliases function walks and invalidates
    all aliases of an inode.
    
    Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
    Reviewed-by: Alexander Motin <mav@FreeBSD.org>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Pavel Snajdr <snajpa@snajpa.net>
    Closes openzfs#9600
    Closes openzfs#14070
    snajpa authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    52e658e View commit details
    Browse the repository at this point in the history
  30. Fix too few arguments to formatting function

    CodeQL reported that when the VERIFY3U condition is false, we do not
    pass enough arguments to `spl_panic()`. This is because the format
    string from `snprintf()` was concatenated into the format string for
    `spl_panic()`, which causes us to have an unexpected format specifier.
    
    A CodeQL developer suggested fixing the macro to have a `%s` format
    string that takes a stringified RIGHT argument, which would fix this.
    However, upon inspection, the VERIFY3U check was never necessary in the
    first place, so we remove it in favor of just calling `snprintf()`.
    
    Lastly, it is interesting that every other static analyzer run on the
    codebase did not catch this, including some that made an effort to catch
    such things. Presumably, all of them relied on header annotations, which
    we have not yet done on `spl_panic()`. CodeQL apparently is able to
    track the flow of arguments on their way to annotated functions, which
    llowed it to catch this when others did not. A future patch that I have
    in development should annotate `spl_panic()`, so the others will catch
    this too.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#14098
    ryao authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    e9a8fb1 View commit details
    Browse the repository at this point in the history
  31. Avoid null pointer dereference in dsl_fs_ss_limit_check()

    Check for cr == NULL before dereferencing it in
    dsl_enforce_ds_ss_limits() to lookup the zone/jail ID.
    
    Reported-by: Coverity (CID 1210459)
    Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Allan Jude <allan@klarasystems.com>
    Closes openzfs#14103
    allanjude authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    ac01b87 View commit details
    Browse the repository at this point in the history
  32. Address warnings about possible division by zero from clangsa

    * The complaint in ztest_replay_write() is only possible if something
       went horribly wrong. An assertion will silence this and if it goes
       off, we will know that something is wrong.
     * The complaint in spa_estimate_metaslabs_to_flush() is not impossible,
       but seems very unlikely. We resolve this by passing the value from
       the `MIN()` that does not go to infinity when the variable is zero.
    
    There was a third report from Clang's scan-build, but that was a
    definite false positive and disappeared when checked again through
    Clang's static analyzer with Z3 refution via CodeChecker.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#14124
    ryao authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    256b74d View commit details
    Browse the repository at this point in the history
  33. Make 1-bit bitfields unsigned

    This fixes -Wsingle-bit-bitfield-constant-conversion warning from
    clang-16 like:
    
    lib/libzfs/libzfs_dataset.c:4529:19: error: implicit truncation
      from 'int' to a one-bit wide bit-field changes value from
      1 to -1 [-Werror,-Wsingle-bit-bitfield-constant-conversion]
                    flags.nounmount = B_TRUE;
    				^ ~~~~~~
    
    Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
    Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Signed-off-by: Brooks Davis <brooks.davis@sri.com>
    Closes openzfs#14125
    brooksdavis authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    572bd18 View commit details
    Browse the repository at this point in the history
  34. Remove an unused variable

    Clang-16 detects this set-but-unused variable which is assigned and
    incremented, but never referenced otherwise.
    
    Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
    Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Signed-off-by: Brooks Davis <brooks.davis@sri.com>
    Closes openzfs#14125
    brooksdavis authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    5f53a44 View commit details
    Browse the repository at this point in the history
  35. Expose zfs_vdev_open_timeout_ms as a tunable

    Some of our customers have been occasionally hitting zfs import failures
    in Linux because udevd doesn't create the by-id symbolic links in time
    for zpool import to use them. The main issue is that the
    systemd-udev-settle.service that zfs-import-cache.service and other
    services depend on is racy. There is also an openzfs issue filed (see
    openzfs#10891) outlining the problem and
    potential solutions.
    
    With the proper solutions being significant in terms of complexity and
    the priority of the issue being low for the time being, this patch
    exposes `zfs_vdev_open_timeout_ms` as a tunable so people that are
    experiencing this issue often can increase it as a workaround.
    
    Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
    Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Reviewed-by: Alexander Motin <mav@FreeBSD.org>
    Reviewed-by: Don Brady <don.brady@delphix.com>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Serapheim Dimitropoulos <serapheim@delphix.com>
    Closes openzfs#14133
    sdimitro authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    85537f7 View commit details
    Browse the repository at this point in the history
  36. FreeBSD: Fix out of bounds read in zfs_ioctl_ozfs_to_legacy()

    There is an off by 1 error in the check. Fortunately, this function does
    not appear to be used in kernel space, despite being compiled as part of
    the kernel module. However, it is used in userspace. Callers of
    lzc_ioctl_fd() likely will crash if they attempt to use the
    unimplemented request number.
    
    This was reported by FreeBSD's coverity scan.
    
    Reported-by: Coverity (CID 1432059)
    Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
    Reviewed-by: Damian Szuberski <szuberskidamian@gmail.com>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#14135
    ryao authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    957c377 View commit details
    Browse the repository at this point in the history
  37. Fix arc_p aggressive increase

    The original ARC paper called for an initial 50/50 MRU/MFU split
    and this is accounted in various places where arc_p = arc_c >> 1,
    with further adjustment based on ghost lists size/hit. However, in
    current code both arc_adapt() and arc_get_data_impl() aggressively
    grow arc_p until arc_c is reached, causing unneeded pressure on
    MFU and greatly reducing its scan-resistance until ghost list
    adjustments kick in.
    
    This patch restores the original behavior of initially having arc_p
    as 1/2 of total ARC, without preventing MRU to use up to 100% total
    ARC when MFU is empty.
    
    Reviewed-by: Alexander Motin <mav@FreeBSD.org>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Gionatan Danti <g.danti@assyoma.it>
    Closes openzfs#14137
    Closes openzfs#14120
    shodanshok authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    d9de079 View commit details
    Browse the repository at this point in the history
  38. Handle and detect openzfs#13709's unlock regression (openzfs#14161)

    In openzfs#13709, as in openzfs#11294 before it, it turns out that 63a2645 still had
    the same failure mode as when it was first landed as d1d4769, and
    fails to unlock certain datasets that formerly worked.
    
    Rather than reverting it again, let's add handling to just throw out
    the accounting metadata that failed to unlock when that happens, as
    well as a test with a pre-broken pool image to ensure that we never get
    bitten by this again.
    
    Fixes: openzfs#13709
    
    Signed-off-by: Rich Ercolani <rincebrain@gmail.com>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed-by: Tony Hutter <hutter2@llnl.gov>
    rincebrain authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    fa7d572 View commit details
    Browse the repository at this point in the history
  39. Correct multipathd.target to .service

    openzfs#9863 says it "orders
    zfs-import-cache.service and zfs-import-scan.service after
    multipathd.service" but the commit (79add96) actually
    ordered them after .target.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Laura Hild <lsh@jlab.org>
    Closes openzfs#12709
    Closes openzfs#14171
    quartsize authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    2662b8e View commit details
    Browse the repository at this point in the history
  40. linux: libspl: zone: () -> (void)

    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
    Closes openzfs#12968
    nabijaczleweli authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    ad0379b View commit details
    Browse the repository at this point in the history
  41. tests: cmd: draid: remove unused and undocumented -v

    Found with -Wunused-but-set-variable on Clang trunk
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
    Closes openzfs#13304
    nabijaczleweli authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    670d66e View commit details
    Browse the repository at this point in the history
  42. module: zfs: vdev_removal: remove unused num_indirect

    Found with -Wunused-but-set-variable on Clang trunk
    
    Fixes: a1d477c ("OpenZFS 7614, 9064 - zfs device evacuation/removal")
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
    Closes openzfs#13304
    nabijaczleweli authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    32f7499 View commit details
    Browse the repository at this point in the history
  43. Remove final K&R definitions

    Clang trunk now warns -Wstrict-prototypes on this, and they're removed
    in C2x
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
    Closes openzfs#13447
    nabijaczleweli authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    108c07c View commit details
    Browse the repository at this point in the history
  44. Fix clang 13 compilation errors

    ```
    os/linux/zfs/zvol_os.c:1111:3: error: ignoring return value of function
      declared with 'warn_unused_result' attribute [-Werror,-Wunused-result]
                    add_disk(zv->zv_zso->zvo_disk);
                    ^~~~~~~~ ~~~~~~~~~~~~~~~~~~~~
    
    zpl_xattr.c:1579:1: warning: no previous prototype for function
      'zpl_posix_acl_release_impl' [-Wmissing-prototypes]
    ```
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: szubersk <szuberskidamian@gmail.com>
    Closes openzfs#13551
    (cherry picked from commit 9884319)
    szubersk authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    3d1e808 View commit details
    Browse the repository at this point in the history
  45. Lua: Fix bad bitshift in lua_strx2number()

    The port of lua to OpenZFS modified lua to use int64_t for numbers
    instead of double. As part of this, a function for calculating
    exponentiation was replaced with a bit shift. Unfortunately, it did not
    handle negative values. Also, it only supported exponents numbers with
    7 digits before before overflow. This supports exponents up to 15 digits
    before overflow.
    
    Clang's static analyzer reported this as "Result of operation is garbage
    or undefined" because the exponent was negative.
    
    Reviewed-by: Damian Szuberski <szuberskidamian@gmail.com>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#14204
    ryao authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    0e3abd2 View commit details
    Browse the repository at this point in the history
  46. Fix NULL pointer dereference in dbuf_prefetch_indirect_done()

    When ZFS is built with assertions, a prefetch is done on a redacted
    blkptr and `dpa->dpa_dnode` is NULL, we will have a NULL pointer
    dereference in `dbuf_prefetch_indirect_done()`.
    
    Both Coverity and Clang's Static Analyzer caught this.
    
    Reported-by: Coverity (CID 1524671)
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
    Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Closes openzfs#14210
    ryao authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    e48aaef View commit details
    Browse the repository at this point in the history
  47. Don't leak packed recieved proprties

    When local properties (e.g., from -o and -x) are provided, don't leak
    the packed representation of the received properties due to variable
    reuse.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Brooks Davis <brooks.davis@sri.com>
    Closes openzfs#14197
    brooksdavis authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    c4468a7 View commit details
    Browse the repository at this point in the history
  48. Make autodetection disable pyzfs for kernel/srpm configurations

    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
    Signed-off-by: szubersk <szuberskidamian@gmail.com>
    Closes openzfs#13394
    Closes openzfs#14178
    szubersk authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    2c50512 View commit details
    Browse the repository at this point in the history
  49. Fix setting the large_block feature after receiving a snapshot

    We are not allowed to dirty a filesystem when done receiving
    a snapshot. In this case the flag SPA_FEATURE_LARGE_BLOCKS will
    not be set on that filesystem since the filesystem is not on
    dp_dirty_datasets, and a subsequent encrypted raw send will fail.
    Fix this by checking in dsl_dataset_snapshot_sync_impl() if the feature
    needs to be activated and do so if appropriate.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: George Amanakis <gamanakis@gmail.com>
    Closes openzfs#13699
    Closes openzfs#13782
    gamanakis authored and tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    c8d2ab0 View commit details
    Browse the repository at this point in the history
  50. zfs-2.1.7: Use ubuntu-20.04 for zloop and sanity builders

    The zfs-2.1.7 branch is still using the older 'python-dev'
    package names rather than the newer 'python3-dev' packages that
    are required for 'ubuntu-latest'.  Use 'ubuntu-20.04' instead of
    'ubuntu-latest' to get around this.
    
    Signed-off-by: Tony Hutter <hutter2@llnl.gov>
    tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    7819b12 View commit details
    Browse the repository at this point in the history
  51. Tag zfs-2.1.7

    META file and changelog updated.
    
    Signed-off-by: Tony Hutter <hutter2@llnl.gov>
    tonyhutter committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    21bd766 View commit details
    Browse the repository at this point in the history