Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build(deps): bump certifi from 2023.7.22 to 2024.7.4 in /drivers/gpu/drm/ci/xfails #706

Open
wants to merge 92 commits into
base: master
Choose a base branch
from

Commits on Jun 23, 2024

  1. bcachefs: Fix missing spaces in journal_entry_dev_usage_to_text

    Fixed missing spaces displayed in journal_entry_dev_usage_to_text
    while adjusting the display format to improve readability.
    
    before:
    ```
     # bcachefs list_journal -a -t alloc:1:0 /dev/sdb
     ...
         dev_usage: dev=0free: buckets=233180 sectors=0 fragmented=0sb: buckets=13 sectors=6152 fragmented=504journal: buckets=1847 sectors=945664 fragmented=0btree: buckets=20 sectors=10240 fragmented=0user: buckets=1419 sectors=726513 fragmented=15cached: buckets=0 sectors=0 fragmented=0parity: buckets=0 sectors=0 fragmented=0stripe: buckets=0 sectors=0 fragmented=0need_gc_gens: buckets=0 sectors=0 fragmented=0need_discard: buckets=1 sectors=0 fragmented=0
    ```
    
    after:
    ```
     # bcachefs list_journal -a -t alloc:1:0 /dev/sdb
     ...
         dev_usage: dev=0
           free: buckets=233180 sectors=0 fragmented=0
           sb: buckets=13 sectors=6152 fragmented=504
           journal: buckets=1847 sectors=945664 fragmented=0
           btree: buckets=20 sectors=10240 fragmented=0
           user: buckets=1419 sectors=726513 fragmented=15
           cached: buckets=0 sectors=0 fragmented=0
           parity: buckets=0 sectors=0 fragmented=0
           stripe: buckets=0 sectors=0 fragmented=0
           need_gc_gens: buckets=0 sectors=0 fragmented=0
           need_discard: buckets=1 sectors=0 fragmented=0
    ```
    Signed-off-by: Youling Tang <tangyouling@kylinos.cn>
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Youling Tang authored and Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    d2a1da1 View commit details
    Browse the repository at this point in the history
  2. bcachefs: Align the display format of btrees/inodes/keys

    Before patch:
    ```
     #cat btrees/inodes/keys
     u64s 17 type inode_v3 0:4096:U32_MAX len 0 ver 0:   mode=40755
       flags= (16300000)
       bi_size=0
    ```
    
    After patch:
    ```
     #cat btrees/inodes/keys
     u64s 17 type inode_v3 0:4096:U32_MAX len 0 ver 0:
       mode=40755
       flags=(16300000)
       bi_size=0
    ```
    
    Signed-off-by: Youling Tang <tangyouling@kylinos.cn>
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Youling Tang authored and Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    c28b614 View commit details
    Browse the repository at this point in the history
  3. bcachefs: Use filemap_read() to simplify the execution flow

    Using filemap_read() can reduce unnecessary code execution
    for non IOCB_DIRECT paths.
    
    Signed-off-by: Youling Tang <tangyouling@kylinos.cn>
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Youling Tang authored and Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    2131e93 View commit details
    Browse the repository at this point in the history
  4. bcachefs: fix missing include

    fs-common.h needs dirent.h for enum bch_rename_mode
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    e5ee3dd View commit details
    Browse the repository at this point in the history
  5. bcachefs: add might_sleep() annotations for fsck_err()

    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    7a0aa7d View commit details
    Browse the repository at this point in the history
  6. bcachefs: Use try_cmpxchg() family of functions instead of cmpxchg()

    Use try_cmpxchg() family of functions instead of
    cmpxchg (*ptr, old, new) == old. x86 CMPXCHG instruction returns
    success in ZF flag, so this change saves a compare after cmpxchg
    (and related move instruction in front of cmpxchg).
    
    Also, try_cmpxchg() implicitly assigns old *ptr value to "old" when
    cmpxchg fails. There is no need to re-read the value in the loop.
    
    No functional change intended.
    
    Signed-off-by: Uros Bizjak <ubizjak@gmail.com>
    Cc: Kent Overstreet <kent.overstreet@linux.dev>
    Cc: Brian Foster <bfoster@redhat.com>
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    ubizjak authored and Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    c72488d View commit details
    Browse the repository at this point in the history
  7. bcachefs: Check for bsets past bch_btree_ptr_v2.sectors_written

    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    4ebf23a View commit details
    Browse the repository at this point in the history
  8. bcachefs: btree_ptr_sectors_written() now takes bkey_s_c

    this is for the userspace metadata dump tool
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    15d1ac2 View commit details
    Browse the repository at this point in the history
  9. bcachefs: make offline fsck set read_only fs flag

    A subsequent change will remove "read_only" as a mount option in favor
    of the standard option "ro", meaning the userspace fsck command cannot
    pass it to the fsck ioctl. Instead, in offline fsck, set "read_only"
    kernel-side without trying to parse it as a mount option.
    
    For compatibility with versions of the "bcachefs fsck" command that try
    to pass the "read_only" mount opt, remove it from the mount options
    string prior to parsing when it is present.
    
    Signed-off-by: Thomas Bertschinger <tahbertschinger@gmail.com>
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    bertschingert authored and Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    87c521d View commit details
    Browse the repository at this point in the history
  10. bcachefs: don't expose "read_only" as a mount option

    When "read_only" is exposed as a mount option, it is redundant with the
    standard option "ro" and gives users multiple ways to specify that a
    bcachefs filesystem should be mounted read-only. This presents the risk
    of having inconsistent options specified.
    
    This can be seen when remounting a read-only filesystem in read-write
    mode, using mount(8) from util-linux. Because mount(8) parses the
    existing mount options from `/proc/mounts` and applies them when
    remounting, it can end up applying both "read_only" and "rw":
    
    $ mount img -o ro /mnt
    $ strace mount -o remount,rw /mnt
    ...
    fsconfig(4, FSCONFIG_SET_FLAG, "read_only", NULL, 0) = 0
    fsconfig(4, FSCONFIG_SET_FLAG, "rw", NULL, 0) = 0
    ...
    
    Making "read_only" no longer a mount option means this edge case cannot
    occur.
    
    Fixes: 62719cf ("bcachefs: Fix nochanges/read_only interaction")
    Signed-off-by: Thomas Bertschinger <tahbertschinger@gmail.com>
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    bertschingert authored and Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    2ae1eed View commit details
    Browse the repository at this point in the history
  11. bcachefs: bch2_printbuf_strip_trailing_newline()

    Add a new helper to fix inode_to_text()
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    090ade9 View commit details
    Browse the repository at this point in the history
  12. bcachefs: allow passing full device path for target options

    The output of mount options such as "metadata_target" in `/proc/mounts`
    uses the full path to the device.
    
    mount(8) from util-linux uses the output from `/proc/mounts` to pass
    existing mount options when performing a remount, so bcachefs should
    accept as input the same form that it prints as output.
    
    Without this change:
    
    $ mount -t bcachefs -o metadata_target=vdb /dev/vdb /mnt
    $ strace mount -o remount /mnt
    ...
    fsconfig(4, FSCONFIG_SET_STRING, "metadata_target", "/dev/vdb", 0) = -1 EINVAL (Invalid argument)
    ...
    
    Signed-off-by: Thomas Bertschinger <tahbertschinger@gmail.com>
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    bertschingert authored and Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    cad8924 View commit details
    Browse the repository at this point in the history
  13. bcachefs: check_key_has_inode()

    Consolidate duplicated checks for extents/dirents/xattrs - these keys
    should all have a corresponding inode of the correct type.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    69d433a View commit details
    Browse the repository at this point in the history
  14. bcachefs: bch_alloc->stripe_sectors

    Add a separate counter to bch_alloc_v4 for amount of striped data; this
    lets us separately track striped and unstriped data in a bucket, which
    lets us see when erasure coding has failed to update extents with stripe
    pointers, and also find buckets to continue updating if we crash mid way
    through creating a new stripe.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    282e932 View commit details
    Browse the repository at this point in the history
  15. bcachefs: BCH_DATA_unstriped

    Add a new pseudo data type, to track buckets that are members of a
    stripe, but have unstriped data in them.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    f02276e View commit details
    Browse the repository at this point in the history
  16. bcachefs: metadata version bucket_stripe_sectors

    New on disk format version for bch_alloc->stripe_sectors and
    BCH_DATA_unstriped - accounting for unstriped data in stripe buckets.
    
    Upgrade/downgrade requires regenerating alloc info - but only if erasure
    coding is in use.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    6ec2ee3 View commit details
    Browse the repository at this point in the history
  17. bcachefs: add printbuf arg to bch2_parse_mount_opts()

    Mount options that take the name of a device that may be part of a
    filesystem, for example "metadata_target", cannot be validated until
    after the filesystem has been opened. However, an attempt to parse those
    options may be made prior to the filesystem being opened.
    
    This change adds a printbuf parameter to bch2_parse_mount_opts() which
    will be used to save those mount options, when they are supplied prior
    to the FS being opened, so that they can be parsed later.
    
    This functionality is not currently needed, but will be used after
    bcachefs starts using the new mount API to parse mount options. This is
    because using the new mount API, we will process mount options prior to
    opening the FS, but the new API doesn't provide a convenient way to
    "replay" mount option parsing. So we save these options ourselves to
    accomplish this.
    
    This change also splits out the code to parse a single option into
    bch2_parse_one_mount_opt(), which will be useful when using the new
    mount API which deals with a single mount option at a time.
    
    Signed-off-by: Thomas Bertschinger <tahbertschinger@gmail.com>
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    bertschingert authored and Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    a25851d View commit details
    Browse the repository at this point in the history
  18. bcachefs: Add error code to defer option parsing

    This introduces a new error code, option_needs_open_fs, which is used to
    indicate that an attempt was made to parse a mount option prior to
    opening a filesystem, when that mount option requires an open filesystem
    in order to be validated.
    
    Returning this error results in bch2_parse_one_mount_opt() saving that
    option for later parsing, after the filesystem is opened.
    
    Signed-off-by: Thomas Bertschinger <tahbertschinger@gmail.com>
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    bertschingert authored and Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    30302eb View commit details
    Browse the repository at this point in the history
  19. bcachefs: use new mount API

    This updates bcachefs to use the new mount API:
    
    - Update the file_system_type to use the new init_fs_context()
      function.
    
    - Define the new fs_context_operations functions.
    
    - No longer register bch2_mount() and bch2_remount(); these are now
      called via the new fs_context functions.
    
    - Define a new helper type, bch2_opts_parse that includes a struct
      bch_opts and additionally a printbuf used to save options that can't
      be parsed until after the FS is opened. This enables us to parse as
      many options as possible prior to opening the filesystem while saving
      those options that need the open FS for later parsing.
    
    Signed-off-by: Thomas Bertschinger <tahbertschinger@gmail.com>
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    bertschingert authored and Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    756494a View commit details
    Browse the repository at this point in the history
  20. bcachefs: KEY_TYPE_accounting

    New key type for the disk space accounting rewrite.
    
     - Holds a variable sized array of u64s (may be more than one for
       accounting e.g. compressed and uncompressed size, or buckets and
       sectors for a given data type)
    
     - Updates are deltas, not new versions of the key: this means updates
       to accounting can happen via the btree write buffer, which we'll be
       teaching to accumulate deltas.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    3f6e210 View commit details
    Browse the repository at this point in the history
  21. bcachefs: Accumulate accounting keys in journal replay

    Until accounting keys hit the btree, they are deltas, not new versions
    of the existing key; this means we have to teach journal replay to
    accumulate them.
    
    Additionally, the journal doesn't track precisely which entries have
    been flushed to the btree; it only tracks a range of entries that may
    possibly still need to be flushed.
    
    That means we need to compare accounting keys against the version in the
    btree and only flush updates that are newer.
    
    There's another wrinkle with the write buffer: if the write buffer
    starts flushing accounting keys before journal replay has finished
    flushing accounting keys, journal replay will see the version number
    from the new updates and updates from the journal will be lost.
    
    To avoid this, journal replay has to flush accounting keys first, and
    we'll be adding a flag so that write buffer flush knows to hold
    accounting keys until then.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    8b37579 View commit details
    Browse the repository at this point in the history
  22. bcachefs: btree write buffer knows how to accumulate bch_accounting keys

    Teach the btree write buffer how to accumulate accounting keys - instead
    of having the newer key overwrite the older key as we do with other
    updates, we need to add them together.
    
    Also, add a flag so that write buffer flush knows when journal replay is
    finished flushing accounting, and teach it to hold accounting keys until
    that flag is set.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    a56f8cf View commit details
    Browse the repository at this point in the history
  23. bcachefs: Disk space accounting rewrite

    Main part of the disk accounting rewrite.
    
    This is a wholesale rewrite of the existing disk space accounting, which
    relies on percepu counters that are sharded by journal buffer, and
    rolled up and added to each journal write.
    
    With the new scheme, every set of counters is a distinct key in the
    accounting btree; this fixes scaling limitations of the old scheme,
    where counters took up space in each journal entry and required multiple
    percpu counters.
    
    Now, in memory accounting requires a single set of percpu counters - not
    multiple for each in flight journal buffer - and in the future we'll
    probably also have counters that don't use in memory percpu counters,
    they're not strictly required.
    
    An accounting update is now a normal btree update, using the btree write
    buffer path. At transaction commit time, we apply accounting updates to
    the in memory counters, which are percpu counters indexed in an
    eytzinger tree by the accounting key.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    35afc30 View commit details
    Browse the repository at this point in the history
  24. bcachefs: Coalesce accounting keys before journal replay

    This fixes a performance regression in journal replay; without
    colaescing accounting keys we have multiple keys at the same position,
    which means journal_keys_peek_upto() has to skip past many overwritten
    keys - turning journal replay into an O(n^2) algorithm.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    1c42770 View commit details
    Browse the repository at this point in the history
  25. bcachefs: dev_usage updated by new accounting

    Reading disk accounting now requires an eytzinger lookup (see:
    bch2_accounting_mem_read()), but the per-device counters are used
    frequently enough that we'd like to still be able to read them with just
    a percpu sum, as in the old code.
    
    This patch special cases the device counters; when we update in-memory
    accounting we also update the old style percpu counters if it's a deice
    counter update.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    9062045 View commit details
    Browse the repository at this point in the history
  26. bcachefs: Kill bch2_fs_usage_initialize()

    Deleting code for the old disk accounting scheme.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    5891ad3 View commit details
    Browse the repository at this point in the history
  27. bcachefs: Convert bch2_ioctl_fs_usage() to new accounting

    This converts bch2_ioctl_fs_usage() to read from the new disk
    accounting, via bch2_fs_replicas_usage_read().
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    b9525f4 View commit details
    Browse the repository at this point in the history
  28. bcachefs: kill bch2_fs_usage_read()

    With bch2_ioctl_fs_usage(), this is now dead code.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    e4d21c9 View commit details
    Browse the repository at this point in the history
  29. bcachefs: Kill writing old accounting to journal

    More ripping out of the old disk space accounting.
    
    Note that the new disk space accounting is incompatible with the old,
    and writing out old style disk space accounting with the new code is
    infeasible.
    
    This means upgrading and downgrading past this version requires
    regenerating accounting.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    0f19bc7 View commit details
    Browse the repository at this point in the history
  30. bcachefs: Delete journal-buf-sharded old style accounting

    More deletion of dead code.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    dd33d3c View commit details
    Browse the repository at this point in the history
  31. bcachefs: Kill bch2_fs_usage_to_text()

    Dead code.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    68e3486 View commit details
    Browse the repository at this point in the history
  32. bcachefs: Kill fs_usage_online

    More dead code deletion.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    f0ff2b7 View commit details
    Browse the repository at this point in the history
  33. bcachefs: Kill replicas_journal_res

    More dead code deletion
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    aab369a View commit details
    Browse the repository at this point in the history
  34. bcachefs: Convert gc to new accounting

    Rewrite fsck/gc for the new accounting scheme.
    
    This adds a second set of in-memory accounting counters for gc to use;
    like with other parts of gc we run all trigger in TRIGGER_GC mode, then
    compare what we calculated to existing in-memory accounting at the end.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    974eae5 View commit details
    Browse the repository at this point in the history
  35. bcachefs: Convert bch2_replicas_gc2() to new accounting

    bch2_replicas_gc2() is used for garbage collection superblock replicas
    entries that are empty - this converts it to the new accounting scheme.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    1aa4f7d View commit details
    Browse the repository at this point in the history
  36. bcachefs: bch2_verify_accounting_clean()

    Verify that the in-memory accounting verifies the on-disk accounting
    after a clean shutdown.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    2b8cdc1 View commit details
    Browse the repository at this point in the history
  37. bcachefs: bch_acct_compression

    This adds per-compression-type accounting of compressed and uncompressed
    size as well as number of extents - meaning we can now see compression
    ratio (without walking the whole filesystem).
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    7cdd5ad View commit details
    Browse the repository at this point in the history
  38. bcachefs: Convert bch2_compression_stats_to_text() to new accounting

    We no longer have to walk the whole btree to calculate compression
    stats.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    72bbac1 View commit details
    Browse the repository at this point in the history
  39. bcachefs: bch2_fs_accounting_to_text()

    Helper to show raw accounting in sysfs, mainly for debugging.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    9cb6b03 View commit details
    Browse the repository at this point in the history
  40. bcachefs: bch2_fs_usage_base_to_text()

    Helper to show raw accounting in sysfs, mainly for debugging.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    fee751f View commit details
    Browse the repository at this point in the history
  41. bcachefs: bch_acct_snapshot

    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    3f828b2 View commit details
    Browse the repository at this point in the history
  42. bcachefs: bch_acct_btree

    Add counters for how much disk space we're using per btree.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    7a35214 View commit details
    Browse the repository at this point in the history
  43. bcachefs: bch_acct_rebalance_work

    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    9791390 View commit details
    Browse the repository at this point in the history
  44. bcachefs: Eytzinger accumulation for accounting keys

    The btree write buffer takes as input keys from the journal, sorts them,
    deduplicates them, and flushes them back to the btree in sorted order.
    
    The disk space accounting rewrite is moving accounting to normal btree
    keys, with update (in this case deltas) accumulated in the write buffer
    and then flushed to the btree; but this is going to increase the number
    of keys handled by the write buffer by perhaps as much as a factor of
    3x-5x.
    
    The overhead from copying around and sorting this many keys would cause
    a significant performance regression, but: there is huge locality in
    updates to accounting keys that we can take advantage of.
    
    Instead of appending accounting keys to the list of keys to be sorted,
    this patch adds an eytzinger search tree of recently seen accounting
    keys. We look up the accounting key in the eytzinger search tree and
    apply the delta directly, adding it if it doesn't exist, and
    periodically prune the eytzinger tree of unused entries.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    8091277 View commit details
    Browse the repository at this point in the history
  45. bcachefs: Kill bch2_mount()

    Fold into bch2_fs_get_tree()
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    43c6a0c View commit details
    Browse the repository at this point in the history
  46. bcachefs: bch2_fs_get_tree() cleanup

    - improve error paths
    - call bch2_fs_start() separately, after applying late-parsed options
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    1205c8d View commit details
    Browse the repository at this point in the history
  47. bcachefs: Don't block journal when finishing check_allocations()

    Blocking the journal was needed to finish checking old style accounting,
    but that code is gone and it's not needed in the alloc rewrite,
    mark_lock is sufficient for synchronization.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    78edd4e View commit details
    Browse the repository at this point in the history
  48. bcachefs: Walk leaf to root in btree_gc

    Next change will move gc_alloc_start initialization into the alloc
    trigger, so we have to mark those first.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    5f55c9b View commit details
    Browse the repository at this point in the history
  49. bcachefs: Initialize gc buckets in alloc trigger

    Needed for online fsck; we need the trigger to initialize newly
    allocated buckets and generation number changes while gc is running.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    cd89843 View commit details
    Browse the repository at this point in the history
  50. bcachefs: Delete old assertion for online fsck

    the order in which btree_gc walks keys have changed, so we no longer
    have the sort of issues with online fsck this assertion was warning
    about.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    9a2e3b3 View commit details
    Browse the repository at this point in the history
  51. bcachefs: btree_types bitmask cleanups

    Make things more consistent and ensure that we're using u64 bitfields -
    key types and btree ids are already around 32 bits.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    6e1499f View commit details
    Browse the repository at this point in the history
  52. bcachefs: fsck_err() may now take a btree_trans

    fsck_err() now optionally takes a btree_trans; if the current thread has
    one, it is required that it be passed.
    
    The next patch will use this to unlock when waiting for user input.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    f75ad70 View commit details
    Browse the repository at this point in the history
  53. bcachefs: Plumb more logging through stdio redirect

    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    f9df073 View commit details
    Browse the repository at this point in the history
  54. bcachefs: twf: convert bch2_stdio_redirect_readline() to darray

    We now read the line from the buffer atomically, which means we have to
    allow the buffer to grow past STDIO_REDIRECT_BUFSIZE if we're waiting
    for a full line - this behaviour is necessary for
    stdio_redirect_readline_timeout() in the next patch.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    9b687b3 View commit details
    Browse the repository at this point in the history
  55. bcachefs: bch2_stdio_redirect_readline_timeout()

    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    1bc1ed6 View commit details
    Browse the repository at this point in the history
  56. bcachefs: twf: delete dead struct fields

    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    350132d View commit details
    Browse the repository at this point in the history
  57. bcachefs: bch2_dir_emit() - fix directory reads in the fuse driver

    Commit 0c0cbfd dropped the ctx->pos
    update before the call to dir_emit. This breaks the userspace
    implementation, causing the directory reads to be stuck in an infinite
    loop. This doesn't happen in the kernel because the vfs handles the
    updates to ctx->pos, but in the fuse implementation nobody updates
    it.
    
    Signed-off-by: Ariel Miculas <ariel.miculas@gmail.com>
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    ariel-miculas authored and Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    a1d1983 View commit details
    Browse the repository at this point in the history
  58. bcachefs: track writeback errors using the generic tracking infrastru…

    …cture
    
    We already using mapping_set_error() in bch2_writepage_io_done(), so all
    we need to do is to use file_check_and_advance_wb_err() when handling
    fsync() requests in bch2_fsync().
    
    Signed-off-by: Youling Tang <tangyouling@kylinos.cn>
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Youling Tang authored and Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    3ab5d8c View commit details
    Browse the repository at this point in the history
  59. bcachefs: Add tracepoints for bch2_sync_fs() and bch2_fsync()

    Add trace_bch2_sync_fs() and trace_bch2_fsync() implementations.
    
    The output in trace is as follows:
      sync-29779   [000] .....   193.700935: bch2_sync_fs: dev 254,16 wait 1
      <...>-40027  [002] .....   342.535227: bch2_fsync: dev 254,32 ino 4099 parent 4096 datasync 1
    
    Signed-off-by: Youling Tang <tangyouling@kylinos.cn>
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Youling Tang authored and Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    e179998 View commit details
    Browse the repository at this point in the history
  60. bcachefs: Clear trans->last_unlock_ip when setting trans->locked

    We're seeing a funny looking trans->locked assertion pop; we should be
    after a bch2_trans_begin() call.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    f810ac9 View commit details
    Browse the repository at this point in the history
  61. bcachefs: Unlock trans when waiting for user input in fsck

    We can't hold locks while waiting for user input, that's a deadlock.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    944e7cf View commit details
    Browse the repository at this point in the history
  62. bcachefs: implement FS_IOC_GETVERSION to support lsattr

    In this patch we add the FS_IOC_GETVERSION ioctl for getting
    i_generation from inode, after that, users can list file's
    generation number by using "lsattr".
    
    Signed-off-by: Hongbo Li <lihongbo22@huawei.com>
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Hongbo Li authored and Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    73f9d2f View commit details
    Browse the repository at this point in the history
  63. bcachefs: support get fs label

    Implement support for FS_IOC_GETFSLABEL ioctl to read filesystem
    label.
    
    Signed-off-by: Hongbo Li <lihongbo22@huawei.com>
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Hongbo Li authored and Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    b0f9af9 View commit details
    Browse the repository at this point in the history
  64. bcachefs: support FS_IOC_SETFSLABEL

    Implement support for FS_IOC_SETFSLABEL ioctl to set filesystem
    label.
    
    Signed-off-by: Hongbo Li <lihongbo22@huawei.com>
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Hongbo Li authored and Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    e796a5d View commit details
    Browse the repository at this point in the history
  65. bcachefs: support REMAP_FILE_DEDUP in bch2_remap_file_range

    By removing the early-exit when REMAP_FILE_DEDUP is set, we should be
    able to support the fideduperange ioctl, albeit less efficiently than if
    we handled some of the extent locking and comparison logic inside
    bcachefs.  Extent comparison logic already exists inside of
    `__generic_remap_file_range_prep`.
    
    Signed-off-by: Reed Riley <reed@riley.engineer>
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    reedriley authored and Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    ba185f8 View commit details
    Browse the repository at this point in the history
  66. bcachefs: BCH_IOCTL_QUERY_ACCOUNTING

    Add a new ioctl that can return the new accounting counter types; it
    takes as input a bitmask of accounting types to return.
    
    This will be used for returning e.g. compression accounting and
    rebalance_work accounting.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    cf67e99 View commit details
    Browse the repository at this point in the history
  67. bcachefs: bch2_btree_insert() - add btree iter flags

    The commit 65bd442 ("bcachefs: bch2_btree_insert_trans() no longer
    specifies BTREE_ITER_cached") removes BTREE_ITER_cached from
    bch2_btree_insert_trans, which causes the update_inode function from
    bcachefs-tools to take a long time (~20s).  Add an iter_flags parameter
    to bch2_btree_insert, so the users can specify iter update trigger
    flags, such as BTREE_ITER_cached.
    
    Signed-off-by: Ariel Miculas <ariel.miculas@gmail.com>
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    ariel-miculas authored and Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    fc19f9a View commit details
    Browse the repository at this point in the history
  68. bcachefs: Fix race in bch2_accounting_mem_insert()

    bch2_accounting_mem_insert() drops and retakes mark_lock; thus, we need
    to check if the entry in question has already been inserted.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    9b0104b View commit details
    Browse the repository at this point in the history
  69. bcachefs: fix smatch data leak warning in fs usage ioctl

    smatch warns that the copy of arg to userspace is a potential data
    leak by virtue of arg.pad not being checked or zeroed. This was
    introduced by the commit referenced below that switched arg from
    being a zeroed runtime allocation to living on the stack. Fix by
    simply zero initializing the structure.
    
    Fixes: cde738a ("bcachefs: Convert bch2_ioctl_fs_usage() to new accounting")
    Reported-by: Dan Carpenter <dan.carpenter@linaro.org>
    Signed-off-by: Brian Foster <bfoster@redhat.com>
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Brian Foster authored and Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    0d8872a View commit details
    Browse the repository at this point in the history
  70. bcachefs: Refactor disk accounting data structures

    Break up the percpu counter allocations into individual allocations for
    each disk accounting counter; this fixes an issue on large systems where
    we have too many replica entries to for the percpu allocator's max
    practical size.
    
    Also, use just one eytzinger tree for the normal set of counters and the
    gc counters; this simplifies accounting_gc_done() where we need the same
    set of counters to be present in both tables.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    3ec9d95 View commit details
    Browse the repository at this point in the history
  71. bcachefs: bch2_accounting_mem_gc()

    Add a new helper to free zeroed out accounting entries, and use it in
    bch2_replicas_gc2(); bch2_replicas_gc2() was killing superblock replicas
    entries if their corresponding accounting counters were nonzero, but
    that's incorrect - the superblock replicas entry needs to exist if the
    accounting entry exists, not if it's nonzero, because we check and
    create the replicas entry when creating the new accounting entry - we
    don't know when it's becoming nonzero.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    628059e View commit details
    Browse the repository at this point in the history
  72. bcachefs: Fix bch2_gc_accounting_done() locking

    The transaction commit path takes mark_lock, so we shouldn't be holding
    it; use a bpos as an iterator so that we can drop and retake.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    f5be443 View commit details
    Browse the repository at this point in the history
  73. bcachefs: Kill gc_pos_btree_node()

    gc_pos is now based on keys, not nodes, for invariantness w.r.t. splits
    and merges
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    8c59906 View commit details
    Browse the repository at this point in the history
  74. bcachefs: bch2_btree_id_to_text()

    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    6abbcfc View commit details
    Browse the repository at this point in the history
  75. bcachefs: bch2_gc_pos_to_text()

    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    f6ba909 View commit details
    Browse the repository at this point in the history
  76. bcachefs: btree_node_unlock() assert

    we have a separate helper for releasing write locks
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    75904d7 View commit details
    Browse the repository at this point in the history
  77. bcachefs: btree_path_cached_set()

    new helper - small refactoring
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    c79209e View commit details
    Browse the repository at this point in the history
  78. bcachefs: kill key cache arg to bch2_assert_pos_locked()

    this is an internal implementation detail - and we're improving key
    cache coherency
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    8942cc1 View commit details
    Browse the repository at this point in the history
  79. MAINTAINERS: remove Brian Foster as a reviewer for bcachefs

    Signed-off-by: Brian Foster <bfoster@redhat.com>
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Brian Foster authored and Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    9594def View commit details
    Browse the repository at this point in the history
  80. bcachefs: per_cpu_sum()

    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    3683245 View commit details
    Browse the repository at this point in the history
  81. bcachefs: Reduce the scope of gc_lock

    gc_lock is now only for synchronization between check_alloc_info and
    interior btree updates - nothing else
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    21ed629 View commit details
    Browse the repository at this point in the history
  82. bcachefs: use FGP_WRITEBEGIN instead of combining individual flags

    Use FGP_WRITEBEGIN to avoid repeating the individual FGP flags before
    starting a buffered write.
    
    Signed-off-by: Pankaj Raghav <p.raghav@samsung.com>
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Panky-codes authored and Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    f8f9232 View commit details
    Browse the repository at this point in the history
  83. bcachefs: set fgf order hint before starting a buffered write

    Set the preferred folio order in the fgp_flags by calling
    fgf_set_order(). Page cache will try to allocate large folio of the
    preferred order whenever possible instead of allocating multiple 0 order
    folios.
    
    This improves the buffered write performance up to 1.25x with default
    mount options and up to 1.57x when mounted with no_data_io option with
    the following fio workload:
    
    fio --name=bcachefs --filename=/mnt/test  --size=100G \
         --ioengine=io_uring --iodepth=16 --rw=write --bs=128k
    
    Signed-off-by: Pankaj Raghav <p.raghav@samsung.com>
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Panky-codes authored and Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    b4061d3 View commit details
    Browse the repository at this point in the history
  84. bcachefs: bch2_btree_key_cache_drop() now evicts

    As part of improving btree key cache coherency, the bkey_cached.valid
    flag is going away.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    001f34c View commit details
    Browse the repository at this point in the history
  85. bcachefs: split out lru_format.h

    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    4e8dec0 View commit details
    Browse the repository at this point in the history
  86. bcachefs: support STATX_DIOALIGN for statx file

    Add support for STATX_DIOALIGN to bcachefs, so that direct I/O alignment
    restrictions are exposed to userspace in a generic way.
    
    [Before]
    ```
    ./statx_test /mnt/bcachefs/test
    statx(/mnt/bcachefs/test) = 0
    dio mem align:0
    dio offset align:0
    ```
    
    [After]
    ```
    ./statx_test /mnt/bcachefs/test
    statx(/mnt/bcachefs/test) = 0
    dio mem align:1
    dio offset align:512
    ```
    
    Signed-off-by: Hongbo Li <lihongbo22@huawei.com>
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Hongbo Li authored and Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    f01cc71 View commit details
    Browse the repository at this point in the history
  87. bcachefs: Discard, invalidate workers are now per device

    There's no reason for discards to be single threaded across all devices;
    this will also help with fixing a race between the discard path and
    device removal.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    2059216 View commit details
    Browse the repository at this point in the history
  88. bcachefs: Ensure buffered writes write as much as they can

    This adds a new helper, bch2_folio_reservation_get_partial(), which
    reserves as many blocks as possible and may return partial success.
    
    __bch2_buffered_write() is switched to the new helper - this fixes
    fstests generic/275, the write until -ENOSPC test.
    
    generic/230 now fails: this appears to be a test bug, where xfs_io isn't
    looping after a partial write to get the error code.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    5500322 View commit details
    Browse the repository at this point in the history
  89. bcachefs: Fix missing BTREE_TRIGGER_bucket_invalidate flag

    This fixes an accounting mismatch for cached data.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    51f4149 View commit details
    Browse the repository at this point in the history
  90. bcachefs: Improve "unable to allocate journal write" message

    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    e5c368e View commit details
    Browse the repository at this point in the history

Commits on Jun 24, 2024

  1. bcachefs: Simplify btree key cache fill path

    Don't allocate the new bkey_cached until after we've done the btree
    lookup; this means we can kill bkey_cached.valid.
    
    Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
    Kent Overstreet committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    d6d793a View commit details
    Browse the repository at this point in the history

Commits on Jul 6, 2024

  1. build(deps): bump certifi in /drivers/gpu/drm/ci/xfails

    Bumps [certifi](https://github.com/certifi/python-certifi) from 2023.7.22 to 2024.7.4.
    - [Commits](certifi/python-certifi@2023.07.22...2024.07.04)
    
    ---
    updated-dependencies:
    - dependency-name: certifi
      dependency-type: direct:production
    ...
    
    Signed-off-by: dependabot[bot] <support@github.com>
    dependabot[bot] authored Jul 6, 2024
    Configuration menu
    Copy the full SHA
    9a83b15 View commit details
    Browse the repository at this point in the history