macOS: how to ensure untracked cache triggers

[derrickstolee]

I was doing some performance tests on macOS and my git status times were slower than expected. It turns out that the untracked cache was not kicking in. A git reset --hard followed by a (slow) git status triggered the untracked cache and following git status commands were very fast.

How can we help ensure the untracked cache is up-to-date? Should we run things in the background?

[github-actions]

Labeling this issue as stale. There has been no activity for 30 days. Remove stale label or comment or this issue will be closed in 7 days.

[derrickstolee]

Thanks for reminding me!

[github-actions]

Labeling this issue as stale. There has been no activity for 30 days. Remove stale label or comment or this issue will be closed in 7 days.

[derrickstolee]

@jeffhostetler: does this seem FSMonitor adjacent to you? It might be something you know how to handle.

[jeffhostetler]

Let me take a look. I know there were some untracked-cache changes in 2.30 in this area, but I haven't studied them.

[github-actions]

Labeling this issue as stale. There has been no activity for 30 days. Remove stale label or comment or this issue will be closed in 7 days.

[TaoK]

Fwiw, this sounds like it could be related to the reproduction steps described at https://lore.kernel.org/git/CAPMMpogjr43111HfPzk23c98JBaiEF2YA_OTkQVG1a63zyeiOw@mail.gmail.com/ - I have encountered the same symptoms when doing manual performance testing.

I just retested, and I get this behavior (if an empty untracked cache is written to the index file, then the untracked cache "doesn't work" until something else causes an index file write) with or without fsmonitor enabled.

I tried correcting this by avoiding writing an empty untracked cache into the index file altogether (in read-cache.c#do_write_index(), where we check "istate->untracked", add an extra check on "istate->untracked->root"), because it was not clear to me when or why doing so would make sense, but that rapidly fell afoul of a bunch of test cases in t7063. Fundamentally, when you do "git update-index --untracked-cache" the tests expect to get precisely that - an empty untracked cache written into the index file, even though this should not happen in "natural" usage.

@derrickstolee seems to suggest here that this could be a "gradual creep" problem that might benefit from background maintenance; given that in real usage index files get regularly rewritten anyway, is it not more likely that these symptoms were from an "unlucky" sequence of events involving an index file with an empty untracked cache?

If so, isn't the right approach either to disallow having an empty cache in the file altogether, or to recognize "I just ended up doing all the same work as if the untracked cache had been completely absent" as a situation where "I should probably write my updated index file" (in git, rather than scalar)?

[TaoK]

Insofar as this "empty but present untracked cache causes same performance as no untracked cache" issue might be what @derrickstolee encountered above, I should mention I've now submitted a patch for this: https://lore.kernel.org/git/pull.986.git.1624559401.gitgitgadget@gmail.com/

[derrickstolee]

@TaoK: thank you for your interest in this topic! We will take a close look at your upstream submission.

[github-actions]

Labeling this issue as stale. There has been no activity for 30 days. Remove stale label or comment or this issue will be closed in 7 days.

[derrickstolee]

This was fixed by microsoft/git#415.