Skip to content

Commit

Permalink
fscache: update fscache to be thread specific instead of global
Browse files Browse the repository at this point in the history
The threading model for fscache has been to have a single, global cache.
This puts requirements on it to be thread safe so that callers like
preload-index can call it from multiple threads.  This was implemented
with a single mutex and completion events which introduces contention
between the calling threads.

Simplify the threading model by making fscache thread specific.  This allows
us to remove the global mutex and synchronization events entirely and instead
associate a fscache with every thread that requests one. This works well with
the current multi-threading which divides the cache entries into blocks with
a separate thread processing each block.

At the end of each worker thread, if there is a fscache on the primary
thread, merge the cached results from the worker into the primary thread
cache. This enables us to reuse the cache later especially when scanning for
untracked files.

In testing, this reduced the time spent in preload_index() by about 25% and
also reduced the CPU utilization significantly.  On a repo with ~200K files,
it reduced overall status times by ~12%.

Signed-off-by: Ben Peart <benpeart@microsoft.com>
  • Loading branch information
benpeart authored and dscho committed Sep 24, 2024
1 parent 1746116 commit db35cf5
Show file tree
Hide file tree
Showing 4 changed files with 215 additions and 121 deletions.
Loading

0 comments on commit db35cf5

Please sign in to comment.