Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sparse Index: log why the index is being expanded #691

Merged
merged 6 commits into from
Oct 9, 2024
Merged

Sparse Index: log why the index is being expanded #691

merged 6 commits into from
Oct 9, 2024

Commits on Oct 1, 2024

  1. sparse-index: add ensure_full_index_with_reason() 

    It is sometimes difficult to support users who are hitting issues with
sparse index expansion because it is unclear why the index needs to expand
from logs alone. It is too invasive to set up a debugging scenario on the
user's machine, so let's improve the logging.

Create a new ensure_full_index_with_reason() method that takes a formatting
string and parameters. If the index is not fully expanded, then apply the
formatting logic to create the logged string and log it before calling
ensure_full_index(). This should assist with discovering why an index is
expanded from trace2 logs alone.

Signed-off-by: Derrick Stolee <stolee@gmail.com>
    @derrickstolee
    derrickstolee committed Oct 1, 2024
    Configuration menu
    Copy the full SHA
    5489d58 View commit details
    Browse the repository at this point in the history

  2. treewide: add reasons for expanding index 

    These locations that previously called ensure_full_index() are now
updated to call the ..._with_reason() varation using fixed strings that
should be enough to identify the reason for the expansion.

This will help users use tracing to determine why the index is expanding
in their scenarios.

Signed-off-by: Derrick Stolee <stolee@gmail.com>
    @derrickstolee
    derrickstolee committed Oct 1, 2024
    Configuration menu
    Copy the full SHA
    bee6d26 View commit details
    Browse the repository at this point in the history

  3. treewide: custom reasons for expanding index 

    These cases that call ensure_full_index() are likely to be due to a data
shape issue on a user's machine, so take the extra time to format a
message that can be placed in their trace2 output and hopefully identify
the problem that is leading to this slow behavior.

Signed-off-by: Derrick Stolee <stolee@gmail.com>
    @derrickstolee
    derrickstolee committed Oct 1, 2024
    Configuration menu
    Copy the full SHA
    7f061d8 View commit details
    Browse the repository at this point in the history

  4. sparse-index: add macro for unaudited expansions 

    For safety, areas of code that iterate over the cache entries in the
index were guarded with ensure_full_index() and labeled with a comment.
Replace these with a macro that calls ensure_full_index_with_reason()
using the line number of the caller to help identify the situation that
is causing the index expansion.

Signed-off-by: Derrick Stolee <stolee@gmail.com>
    @derrickstolee
    derrickstolee committed Oct 1, 2024
    Configuration menu
    Copy the full SHA
    53e2668 View commit details
    Browse the repository at this point in the history

  5. Docs: update sparse index plan with logging 

    The recent changes update the callers of ensure_full_index() to call
variants that will log extra information. This should assist developers
assisting users who are hitting the sparse index expansion message.

Signed-off-by: Derrick Stolee <stolee@gmail.com>
    @derrickstolee
    derrickstolee committed Oct 1, 2024
    Configuration menu
    Copy the full SHA
    dc7b1e0 View commit details
    Browse the repository at this point in the history

  6. sparse-index: log failure to clear skip-worktree 

    The clear_skip_worktree_from_present_files_sparse() method attempts to
clear the skip worktree bit from cache entries in the index depending on
when they exist in the workdir. When this comes across a sparse
directory that actually exists in the workdir, then this method fails
and signals that the index needs expansion.

The index expansion already logs a reason, but this reason is separate
from the path that caused this failure.

Add logging to demonstrate this situation for full clarity.

Signed-off-by: Derrick Stolee <stolee@gmail.com>
    @derrickstolee
    derrickstolee committed Oct 1, 2024
    Configuration menu
    Copy the full SHA
    e751308 View commit details
    Browse the repository at this point in the history