Sparse index: `git reset` #417

vdye · 2021-09-01T13:23:37Z

Changes

In addition to the usual changes a sparse index integration needs, there were some modifications needed to make sure reset could handle sparse directory entries and keep the index (especially the cache tree) in a valid state:

for reset --mixed, add change and add_remove functions in diff settings, as those functions are applied to sparse directories in oneway_diff (the unpacking function used by mixed reset)
- related to this, the recursive diff setting is enabled to merge in nested outside-of-cone files
for reset --merge and reset --hard, the cache tree is manually reconstructed (from a given tree, not from the index itself, as it is in cache_tree_update) in prime_cache_tree. In order for the cache tree to be accurate, directory contents are only excluded if that directory exists in the index as a sparse directory
- this check feels inefficient (each time a directory outside the sparse checkout cone is reached, the entire index is searched for a matching entry). If there's a function that does this search more efficiently, please let me know!
tests added for all non-soft reset modes (since --soft doesn't touch the index), include tests ensuring the index isn't expanded

Note: this doesn't address the bug found in the existing sparse reset - if it's alright with you (reviewers), I'll create a separate issue for that.

cache-tree.c

ldennington

Looks really good to me! I will wait on others who are more familiar with cache-tree and the options you added to give their approval, though (I'll admit I got a bit lost with those changes).

derrickstolee

I took a look through the code and left some comments that are mostly about crafting a good patch series. The biggest one is to make your commits as focused as possible: do only one thing at a time. But that also means, demonstrate how that one thing changes functionality through tests, if possible. The test dump at the end should be replaced with a sprinkling of tests throughout.

The tests could probably use some cases that specify a pathspec, both in and outside of the sparse-checkout cone.

I installed your version of Git and tried a bunch of git reset shenanigans to see if I could trigger an index expansion or other slow behavior. I did see a really slow (5s) loop inside unpack_trees() during a git reset --hard -- <path>, but that's unrelated to the sparse index. It happens to be the loop conditioned by this if in unpack_trees():

	/* Any left-over entries in the index? */
	if (o->merge) {

Turns out that this loops through all of the index entries in a very slow way. This might be something to investigate separately (and much later than the current work).

t/t1092-sparse-checkout-compatibility.sh

cache-tree.c

Mixed reset in a sparse checkout (cone mode or otherwise) will, if a file outside of the sparse definition has differences between the working tree and target, write the file to the local working copy of the repository. Since this behavior conflicts with the typical expectations of a strict checkout definition/cone, a test is added demonstrating (and explaining) it. Signed-off-by: Victoria Dye <vdye@github.com>

THIS CHANGE IS NOT IN FINAL FORM AND IS CURRENTLY JUST INTENDED FOR FEEDBACK. To understand the context laid out below I have purposefully included both source and test changes in this initial commit. Based on the feedback I get, I will either split this commit or create an entirely new branch with the necessary changes when it is ready to officially submit. This change enables running the `diff` builtin command without expanding the full index in a cone-mode sparse checkout. It is an attempt to add the basic infrastructure to "light up" sparse index for the command. However, based on some testing, this may need to change. After reviewing microsoft#417, I decided to add `ensure_full_index` to this initial commit in an attempt to maintain current behavior. However, as I started building and testing with and without this update, I noticed unexpected results. The `ensure_not_expanded` tests I added are passing with `ensure_full_index` both enabled and disabled. That felt wrong, so I used `TRACE2` to try and understand what was happening and have attached the results. It looks as though no code paths in `diff.c` are actually being used. On the bright side, however, it doesn't look as though `ensure_full_index` is ever being called. I recognize my lack of knowledge wrt `cache-tree` and `read-cache` may be the reason I'm unsure what the correct path forward is. In light of this, any guidance is appreciated.

derrickstolee

Great job reorganizing the commits!

builtin/reset.c

derrickstolee · 2021-09-02T01:04:28Z

t/t1092-sparse-checkout-compatibility.sh

+# TODO: reset with a file pathspec outside of the sparse checkout definition
+# does not currently work in cone-mode checkouts. This is due to the tree's
+# directory entry (that contains the file) not being expanded before the
+# pathspec check is performed (in the case of this test, "folder1/" is compared
+# to the pathspec "folder1/a", and does not match).


Thanks for pointing out how this is failing. We should debug into this tomorrow to see what logic is happening. Something like index_name_pos(istate, "folder1/a") would expand a sparse index, so something different must be happening. Perhaps it's related to how reset uses the cache-tree structure.

I did find the the source of the failure, but wasn't sure whether I should have pointed out the specifics in the TODO comment: https://github.com/microsoft/git/blob/vfs-2.33.0/diff-lib.c#L505-L507

In the linked ce_path_match check, idx->name is folder1/ (since it's initially a sparse entry), but revs->prune_data (set to the input pathspec) is folder1/a. The ce_path_match check fails, so do_oneway_diff is never called and that part of the tree is effectively ignored. This also explains why a sparse directory pathspec (like folder1) does work - the match succeeds, and the relevant entries are expanded in do_oneway_diff.

The part I'm not sure about is the right way to fix it (if it should be fixed at all). Should the full index be expanded right at the beginning in builtin/reset.c if the pathspec is outside the sparse checkout cone (although how would you handle a wildcard pathspec like *.c)? Or should pathspec usage be blocked by default, but expanding the full index would be allowed using a --sparse input flag? There are definitely other ways of doing it, but I imagine we probably want something that doesn't preemptively expand index entries if we can avoid it.

The fact that we are getting into oneway_diff() with a folder1/ entry an a folder1/a entry makes me think that we need more updates to unpack_trees() to catch this case during that update. When you debug into that line, is there anything in the stack trace that looks like a good place to identify that we need to expand the index here?

As for preventing the user request, there is some precedent in builtin/add.c using matches_skip_worktree() and path_in_sparse_checkout() that might be illuminating. Take a look and see if you are inspired for an approach in git reset. There is absolutely momentum building to prevent these kinds of interactions outside of the sparse-checkout cone.

Sorry, I should clarify - folder1/a isn't an entry (as in a tree or cache entry), it's a pathspec pulled directly from command-line input that's compared to idx (a sparse directory cache entry). Both idx and tree (the two cache entries compared in oneway_diff) are sparse directory entries pointing at folder1/. The only indicator of needing to expand the index before that (as far as I can tell) is the pathspec itself, which could be used as early as cmd_reset.

I'll check out matches_skip_worktree() and path_in_sparse_checkout() - if they can interpret wildcards in the pathspec (or can be updated to do that), it should be straightforward to expand the index before even entering unpack_trees()

I've just added two fixups (for readability in this PR - I'll squash them before merging) that should fix this. The first adds more tests for pathspec resets, and the second adds checks to expand the index if the pathspec isn't obviously for things in the index (and corresponding tests for when the index should not be expanded).

derrickstolee

I have a tiny nit for you to fixup while you rebase. Otherwise this looks great!

cache-tree.c

derrickstolee · 2021-09-07T16:11:06Z

t/t1092-sparse-checkout-compatibility.sh

+	# Although folder1 is outside the sparse definition, it exists as a
+	# directory entry in the index, so it will be reset without needing to
+	# expand the full index.
+	ensure_not_expanded reset --hard update-folder1 &&


nice! I'm excited to see that this is possible.

Add tests for `--merge` and `--keep` modes, as well as (mixed) reset with pathspecs both inside and outside of the sparse checkout definition. Signed-off-by: Victoria Dye <vdye@github.com>

In anticipation of `reset` being fully integrated with sparse index, the test for index expand and collapse for non-sparse index integrated commands is updated to use `read-tree`. Signed-off-by: Victoria Dye <vdye@github.com>

This change enables running the `reset` builtin command without expanding the full index in a cone-mode sparse checkout. `reset --soft` does not modify the index, so no compatibility changes are needed for it to function without expanding the index. For all other reset modes (`--mixed`, `--hard`, `--keep`, `--merge`), the full index is explicitly expanded with `ensure_full_index` to maintain current behavior. Additionally, the `read_cache()` check verifying an uncorrupted index is moved after argument parsing and preparing the repo settings. The index is not used by the preceding argument handling, but `read_cache()` does need to be run after enabling sparse index for the command and before using the index for the reset. Signed-off-by: Victoria Dye <vdye@github.com>

This change allows non-mixed resets to no longer require that the full index is expanded. This is accomplished by updating the `prime_cache_tree_rec` function to have it reconstruct the cache tree "aware" of whether a directory is sparse in the index or not. This check must be done by verifying the contents of the cache itself (rather than checking whether a directory is inside of the sparse checkout cone) as entries may have been added to the sparse index outside of the checkout cone. Signed-off-by: Victoria Dye <vdye@github.com>

Due to the introduction of sparse directory entries in the `diff_cache` execution (used internally by `reset --mixed`), explicit `change` and `add_remove` functions are specified for merging of the interior contents of those sparse directories. Additionally, to handle cases in which `reset --mixed` must merge files nested multiple levels deep in directories outside the sparse checkout cone, the `recursive` diff option is added. Signed-off-by: Victoria Dye <vdye@github.com>

One thing I forgot when talking about the sparse index is that we have a performance test: `t/perf/p2000-sparse-operations.sh`. This test wasn't helpful for commands like `git merge` that need a particular set of input, but work for more read-only operations. Here is a quick demonstration of how this performance test works so we could have a definitive measure of how your previous updates improved performance. To get these results, I ran the following command in `t/perf`: ``` ./run 4bcd533 f9255a5 f28fc01 b713582 -- p2000-sparse-operations.sh ``` The short-shas correspond to the merge commits for these PRs: * #410 * #421 * #417 * #419 The test takes a copy of the Git repository and creates several copies within a nested directory heirarchy. ``` Test 4bcd533 f9255a5 f28fc01 b713582 ------------------------------------------------------------------------------------------------------------------------------------------------- 2000.2: git status (full-v3) 0.19(0.15+0.05) 0.19(0.16+0.05) +0.0% 0.20(0.18+0.03) +5.3% 0.19(0.17+0.04) +0.0% 2000.3: git status (full-v4) 0.20(0.18+0.04) 0.19(0.15+0.06) -5.0% 0.21(0.18+0.05) +5.0% 0.18(0.18+0.02) -10.0% 2000.4: git status (sparse-v3) 0.04(0.04+0.04) 0.05(0.07+0.04) +25.0% 0.04(0.04+0.05) +0.0% 0.04(0.06+0.04) +0.0% 2000.5: git status (sparse-v4) 0.04(0.03+0.06) 0.04(0.05+0.05) +0.0% 0.05(0.05+0.04) +25.0% 0.05(0.06+0.04) +25.0% 2000.6: git add -A (full-v3) 0.36(0.29+0.05) 0.38(0.28+0.07) +5.6% 0.36(0.31+0.05) +0.0% 0.37(0.31+0.05) +2.8% 2000.7: git add -A (full-v4) 0.34(0.27+0.06) 0.34(0.29+0.05) +0.0% 0.34(0.29+0.04) +0.0% 0.35(0.28+0.06) +2.9% 2000.8: git add -A (sparse-v3) 0.06(0.07+0.04) 0.06(0.05+0.06) +0.0% 0.06(0.09+0.01) +0.0% 0.06(0.08+0.03) +0.0% 2000.9: git add -A (sparse-v4) 0.05(0.05+0.04) 0.05(0.05+0.07) +0.0% 0.05(0.04+0.06) +0.0% 0.06(0.06+0.05) +20.0% 2000.10: git add . (full-v3) 0.38(0.31+0.05) 0.37(0.29+0.06) -2.6% 0.37(0.30+0.07) -2.6% 0.37(0.29+0.06) -2.6% 2000.11: git add . (full-v4) 0.35(0.31+0.04) 0.35(0.29+0.07) +0.0% 0.35(0.29+0.05) +0.0% 0.34(0.29+0.06) -2.9% 2000.12: git add . (sparse-v3) 0.06(0.06+0.05) 0.06(0.05+0.06) +0.0% 0.06(0.07+0.05) +0.0% 0.06(0.09+0.03) +0.0% 2000.13: git add . (sparse-v4) 0.06(0.06+0.06) 0.06(0.07+0.04) +0.0% 0.05(0.06+0.05) -16.7% 0.05(0.05+0.07) -16.7% 2000.14: git commit -a -m A (full-v3) 0.48(0.37+0.08) 0.45(0.36+0.08) -6.2% 0.45(0.35+0.09) -6.2% 0.44(0.36+0.07) -8.3% 2000.15: git commit -a -m A (full-v4) 0.45(0.40+0.06) 0.43(0.34+0.07) -4.4% 0.45(0.37+0.06) +0.0% 0.42(0.36+0.05) -6.7% 2000.16: git commit -a -m A (sparse-v3) 0.05(0.05+0.06) 0.05(0.05+0.03) +0.0% 0.05(0.06+0.06) +0.0% 0.05(0.04+0.06) +0.0% 2000.17: git commit -a -m A (sparse-v4) 0.05(0.06+0.03) 0.05(0.06+0.04) +0.0% 0.06(0.07+0.05) +20.0% 0.05(0.04+0.06) +0.0% 2000.18: git checkout -f - (full-v3) 0.55(0.43+0.08) 0.54(0.46+0.05) -1.8% 0.55(0.46+0.07) +0.0% 0.54(0.40+0.10) -1.8% 2000.19: git checkout -f - (full-v4) 0.55(0.41+0.09) 0.50(0.40+0.09) -9.1% 0.51(0.46+0.05) -7.3% 0.51(0.44+0.06) -7.3% 2000.20: git checkout -f - (sparse-v3) 0.06(0.09+0.03) 0.06(0.08+0.03) +0.0% 0.06(0.06+0.05) +0.0% 0.07(0.09+0.03) +16.7% 2000.21: git checkout -f - (sparse-v4) 0.06(0.08+0.04) 0.05(0.07+0.05) -16.7% 0.05(0.07+0.04) -16.7% 0.06(0.09+0.03) +0.0% ``` All of the above were already integrated. ``` 2000.22: git reset (full-v3) 0.41(0.32+0.06) 0.40(0.31+0.06) -2.4% 0.41(0.33+0.05) +0.0% 0.42(0.34+0.04) +2.4% 2000.23: git reset (full-v4) 0.37(0.32+0.05) 0.35(0.30+0.05) -5.4% 0.37(0.30+0.05) +0.0% 0.35(0.31+0.03) -5.4% 2000.24: git reset (sparse-v3) 0.68(0.65+0.05) 0.55(0.52+0.04) -19.1% 0.04(0.05+0.04) -94.1% 0.04(0.05+0.04) -94.1% 2000.25: git reset (sparse-v4) 0.70(0.65+0.05) 0.54(0.50+0.06) -22.9% 0.04(0.07+0.01) -94.3% 0.03(0.05+0.05) -95.7% 2000.26: git reset --hard (full-v3) 0.54(0.43+0.07) 0.53(0.43+0.06) -1.9% 0.55(0.46+0.05) +1.9% 0.55(0.44+0.06) +1.9% 2000.27: git reset --hard (full-v4) 0.50(0.45+0.03) 0.50(0.43+0.05) +0.0% 0.49(0.41+0.06) -2.0% 0.50(0.42+0.05) +0.0% 2000.28: git reset --hard (sparse-v3) 0.83(0.76+0.06) 0.68(0.62+0.05) -18.1% 0.07(0.05+0.02) -91.6% 0.07(0.05+0.02) -91.6% 2000.29: git reset --hard (sparse-v4) 0.80(0.75+0.05) 0.69(0.62+0.06) -13.8% 0.07(0.04+0.02) -91.2% 0.07(0.04+0.03) -91.2% ``` As expected, `git reset [--hard]` improves with the sparse index integration, but remains constant across the full index case. ``` 2000.30: git update-index --add --remove (full-v3) 0.03(0.01+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.02+0.01) +0.0% 0.03(0.01+0.01) +0.0% 2000.31: git update-index --add --remove (full-v4) 0.03(0.02+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.03+0.00) +0.0% 0.03(0.02+0.01) +0.0% 2000.32: git update-index --add --remove (sparse-v3) 0.57(0.54+0.02) 0.43(0.42+0.00) -24.6% 0.44(0.41+0.03) -22.8% 0.44(0.42+0.01) -22.8% 2000.33: git update-index --add --remove (sparse-v4) 0.56(0.52+0.04) 0.43(0.42+0.01) -23.2% 0.44(0.42+0.02) -21.4% 0.42(0.41+0.01) -25.0% ``` These do not change significantly because #423 is not merged. ``` 2000.34: git diff (full-v3) 0.07(0.05+0.03) 0.06(0.05+0.03) -14.3% 0.07(0.05+0.03) +0.0% 0.06(0.05+0.03) -14.3% 2000.35: git diff (full-v4) 0.06(0.05+0.03) 0.06(0.05+0.02) +0.0% 0.06(0.05+0.02) +0.0% 0.06(0.06+0.02) +0.0% 2000.36: git diff (sparse-v3) 0.25(0.23+0.03) 0.17(0.17+0.02) -32.0% 0.18(0.18+0.02) -28.0% 0.01(0.03+0.03) -96.0% 2000.37: git diff (sparse-v4) 0.25(0.22+0.05) 0.16(0.16+0.01) -36.0% 0.18(0.15+0.04) -28.0% 0.01(0.04+0.02) -96.0% 2000.38: git diff --staged (full-v3) 0.03(0.01+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.02+0.01) +0.0% 0.03(0.02+0.00) +0.0% 2000.39: git diff --staged (full-v4) 0.04(0.03+0.01) 0.03(0.02+0.01) -25.0% 0.03(0.03+0.00) -25.0% 0.03(0.03+0.00) -25.0% 2000.40: git diff --staged (sparse-v3) 0.21(0.19+0.01) 0.15(0.13+0.01) -28.6% 0.15(0.14+0.01) -28.6% 0.01(0.01+0.00) -95.2% 2000.41: git diff --staged (sparse-v4) 0.22(0.21+0.01) 0.14(0.11+0.03) -36.4% 0.15(0.13+0.02) -31.8% 0.01(0.01+0.00) -95.5% ``` The `git diff` improvements are measurable. ``` 2000.42: git sparse-checkout reapply (full-v3) 0.63(0.54+0.05) 0.56(0.48+0.04) -11.1% 0.57(0.48+0.03) -9.5% 0.59(0.48+0.05) -6.3% 2000.43: git sparse-checkout reapply (full-v4) 0.60(0.54+0.02) 0.51(0.46+0.03) -15.0% 0.54(0.48+0.02) -10.0% 0.50(0.44+0.04) -16.7% 2000.44: git sparse-checkout reapply (sparse-v3) 0.91(0.86+0.05) 0.05(0.05+0.00) -94.5% 0.06(0.05+0.01) -93.4% 0.06(0.06+0.00) -93.4% 2000.45: git sparse-checkout reapply (sparse-v4) 0.92(0.88+0.04) 0.05(0.05+0.00) -94.6% 0.05(0.05+0.01) -94.6% 0.05(0.04+0.01) -94.6% ``` Finally, the `git sparse-checkout` measurements are also present. This test script is particularly valuable when contributing changes upstream. It can be good to start by adding the lines to the performance test in an early commit, then demonstrating the performance change by copying the necessary lines from the output table into your commit message.

Sparse index: `git reset`

One thing I forgot when talking about the sparse index is that we have a performance test: `t/perf/p2000-sparse-operations.sh`. This test wasn't helpful for commands like `git merge` that need a particular set of input, but work for more read-only operations. Here is a quick demonstration of how this performance test works so we could have a definitive measure of how your previous updates improved performance. To get these results, I ran the following command in `t/perf`: ``` ./run 4bcd533 f9255a5 f28fc01 b713582 -- p2000-sparse-operations.sh ``` The short-shas correspond to the merge commits for these PRs: * #410 * #421 * #417 * #419 The test takes a copy of the Git repository and creates several copies within a nested directory heirarchy. ``` Test 4bcd533 f9255a5 f28fc01 b713582 ------------------------------------------------------------------------------------------------------------------------------------------------- 2000.2: git status (full-v3) 0.19(0.15+0.05) 0.19(0.16+0.05) +0.0% 0.20(0.18+0.03) +5.3% 0.19(0.17+0.04) +0.0% 2000.3: git status (full-v4) 0.20(0.18+0.04) 0.19(0.15+0.06) -5.0% 0.21(0.18+0.05) +5.0% 0.18(0.18+0.02) -10.0% 2000.4: git status (sparse-v3) 0.04(0.04+0.04) 0.05(0.07+0.04) +25.0% 0.04(0.04+0.05) +0.0% 0.04(0.06+0.04) +0.0% 2000.5: git status (sparse-v4) 0.04(0.03+0.06) 0.04(0.05+0.05) +0.0% 0.05(0.05+0.04) +25.0% 0.05(0.06+0.04) +25.0% 2000.6: git add -A (full-v3) 0.36(0.29+0.05) 0.38(0.28+0.07) +5.6% 0.36(0.31+0.05) +0.0% 0.37(0.31+0.05) +2.8% 2000.7: git add -A (full-v4) 0.34(0.27+0.06) 0.34(0.29+0.05) +0.0% 0.34(0.29+0.04) +0.0% 0.35(0.28+0.06) +2.9% 2000.8: git add -A (sparse-v3) 0.06(0.07+0.04) 0.06(0.05+0.06) +0.0% 0.06(0.09+0.01) +0.0% 0.06(0.08+0.03) +0.0% 2000.9: git add -A (sparse-v4) 0.05(0.05+0.04) 0.05(0.05+0.07) +0.0% 0.05(0.04+0.06) +0.0% 0.06(0.06+0.05) +20.0% 2000.10: git add . (full-v3) 0.38(0.31+0.05) 0.37(0.29+0.06) -2.6% 0.37(0.30+0.07) -2.6% 0.37(0.29+0.06) -2.6% 2000.11: git add . (full-v4) 0.35(0.31+0.04) 0.35(0.29+0.07) +0.0% 0.35(0.29+0.05) +0.0% 0.34(0.29+0.06) -2.9% 2000.12: git add . (sparse-v3) 0.06(0.06+0.05) 0.06(0.05+0.06) +0.0% 0.06(0.07+0.05) +0.0% 0.06(0.09+0.03) +0.0% 2000.13: git add . (sparse-v4) 0.06(0.06+0.06) 0.06(0.07+0.04) +0.0% 0.05(0.06+0.05) -16.7% 0.05(0.05+0.07) -16.7% 2000.14: git commit -a -m A (full-v3) 0.48(0.37+0.08) 0.45(0.36+0.08) -6.2% 0.45(0.35+0.09) -6.2% 0.44(0.36+0.07) -8.3% 2000.15: git commit -a -m A (full-v4) 0.45(0.40+0.06) 0.43(0.34+0.07) -4.4% 0.45(0.37+0.06) +0.0% 0.42(0.36+0.05) -6.7% 2000.16: git commit -a -m A (sparse-v3) 0.05(0.05+0.06) 0.05(0.05+0.03) +0.0% 0.05(0.06+0.06) +0.0% 0.05(0.04+0.06) +0.0% 2000.17: git commit -a -m A (sparse-v4) 0.05(0.06+0.03) 0.05(0.06+0.04) +0.0% 0.06(0.07+0.05) +20.0% 0.05(0.04+0.06) +0.0% 2000.18: git checkout -f - (full-v3) 0.55(0.43+0.08) 0.54(0.46+0.05) -1.8% 0.55(0.46+0.07) +0.0% 0.54(0.40+0.10) -1.8% 2000.19: git checkout -f - (full-v4) 0.55(0.41+0.09) 0.50(0.40+0.09) -9.1% 0.51(0.46+0.05) -7.3% 0.51(0.44+0.06) -7.3% 2000.20: git checkout -f - (sparse-v3) 0.06(0.09+0.03) 0.06(0.08+0.03) +0.0% 0.06(0.06+0.05) +0.0% 0.07(0.09+0.03) +16.7% 2000.21: git checkout -f - (sparse-v4) 0.06(0.08+0.04) 0.05(0.07+0.05) -16.7% 0.05(0.07+0.04) -16.7% 0.06(0.09+0.03) +0.0% ``` All of the above were already integrated. ``` 2000.22: git reset (full-v3) 0.41(0.32+0.06) 0.40(0.31+0.06) -2.4% 0.41(0.33+0.05) +0.0% 0.42(0.34+0.04) +2.4% 2000.23: git reset (full-v4) 0.37(0.32+0.05) 0.35(0.30+0.05) -5.4% 0.37(0.30+0.05) +0.0% 0.35(0.31+0.03) -5.4% 2000.24: git reset (sparse-v3) 0.68(0.65+0.05) 0.55(0.52+0.04) -19.1% 0.04(0.05+0.04) -94.1% 0.04(0.05+0.04) -94.1% 2000.25: git reset (sparse-v4) 0.70(0.65+0.05) 0.54(0.50+0.06) -22.9% 0.04(0.07+0.01) -94.3% 0.03(0.05+0.05) -95.7% 2000.26: git reset --hard (full-v3) 0.54(0.43+0.07) 0.53(0.43+0.06) -1.9% 0.55(0.46+0.05) +1.9% 0.55(0.44+0.06) +1.9% 2000.27: git reset --hard (full-v4) 0.50(0.45+0.03) 0.50(0.43+0.05) +0.0% 0.49(0.41+0.06) -2.0% 0.50(0.42+0.05) +0.0% 2000.28: git reset --hard (sparse-v3) 0.83(0.76+0.06) 0.68(0.62+0.05) -18.1% 0.07(0.05+0.02) -91.6% 0.07(0.05+0.02) -91.6% 2000.29: git reset --hard (sparse-v4) 0.80(0.75+0.05) 0.69(0.62+0.06) -13.8% 0.07(0.04+0.02) -91.2% 0.07(0.04+0.03) -91.2% ``` As expected, `git reset [--hard]` improves with the sparse index integration, but remains constant across the full index case. ``` 2000.30: git update-index --add --remove (full-v3) 0.03(0.01+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.02+0.01) +0.0% 0.03(0.01+0.01) +0.0% 2000.31: git update-index --add --remove (full-v4) 0.03(0.02+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.03+0.00) +0.0% 0.03(0.02+0.01) +0.0% 2000.32: git update-index --add --remove (sparse-v3) 0.57(0.54+0.02) 0.43(0.42+0.00) -24.6% 0.44(0.41+0.03) -22.8% 0.44(0.42+0.01) -22.8% 2000.33: git update-index --add --remove (sparse-v4) 0.56(0.52+0.04) 0.43(0.42+0.01) -23.2% 0.44(0.42+0.02) -21.4% 0.42(0.41+0.01) -25.0% ``` These do not change significantly because #423 is not merged. ``` 2000.34: git diff (full-v3) 0.07(0.05+0.03) 0.06(0.05+0.03) -14.3% 0.07(0.05+0.03) +0.0% 0.06(0.05+0.03) -14.3% 2000.35: git diff (full-v4) 0.06(0.05+0.03) 0.06(0.05+0.02) +0.0% 0.06(0.05+0.02) +0.0% 0.06(0.06+0.02) +0.0% 2000.36: git diff (sparse-v3) 0.25(0.23+0.03) 0.17(0.17+0.02) -32.0% 0.18(0.18+0.02) -28.0% 0.01(0.03+0.03) -96.0% 2000.37: git diff (sparse-v4) 0.25(0.22+0.05) 0.16(0.16+0.01) -36.0% 0.18(0.15+0.04) -28.0% 0.01(0.04+0.02) -96.0% 2000.38: git diff --staged (full-v3) 0.03(0.01+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.02+0.01) +0.0% 0.03(0.02+0.00) +0.0% 2000.39: git diff --staged (full-v4) 0.04(0.03+0.01) 0.03(0.02+0.01) -25.0% 0.03(0.03+0.00) -25.0% 0.03(0.03+0.00) -25.0% 2000.40: git diff --staged (sparse-v3) 0.21(0.19+0.01) 0.15(0.13+0.01) -28.6% 0.15(0.14+0.01) -28.6% 0.01(0.01+0.00) -95.2% 2000.41: git diff --staged (sparse-v4) 0.22(0.21+0.01) 0.14(0.11+0.03) -36.4% 0.15(0.13+0.02) -31.8% 0.01(0.01+0.00) -95.5% ``` The `git diff` improvements are measurable. ``` 2000.42: git sparse-checkout reapply (full-v3) 0.63(0.54+0.05) 0.56(0.48+0.04) -11.1% 0.57(0.48+0.03) -9.5% 0.59(0.48+0.05) -6.3% 2000.43: git sparse-checkout reapply (full-v4) 0.60(0.54+0.02) 0.51(0.46+0.03) -15.0% 0.54(0.48+0.02) -10.0% 0.50(0.44+0.04) -16.7% 2000.44: git sparse-checkout reapply (sparse-v3) 0.91(0.86+0.05) 0.05(0.05+0.00) -94.5% 0.06(0.05+0.01) -93.4% 0.06(0.06+0.00) -93.4% 2000.45: git sparse-checkout reapply (sparse-v4) 0.92(0.88+0.04) 0.05(0.05+0.00) -94.6% 0.05(0.05+0.01) -94.6% 0.05(0.04+0.01) -94.6% ``` Finally, the `git sparse-checkout` measurements are also present. This test script is particularly valuable when contributing changes upstream. It can be good to start by adding the lines to the performance test in an early commit, then demonstrating the performance change by copying the necessary lines from the output table into your commit message.

Sparse index: `git reset`

One thing I forgot when talking about the sparse index is that we have a performance test: `t/perf/p2000-sparse-operations.sh`. This test wasn't helpful for commands like `git merge` that need a particular set of input, but work for more read-only operations. Here is a quick demonstration of how this performance test works so we could have a definitive measure of how your previous updates improved performance. To get these results, I ran the following command in `t/perf`: ``` ./run 4bcd533 f9255a5 f28fc01 b713582 -- p2000-sparse-operations.sh ``` The short-shas correspond to the merge commits for these PRs: * #410 * #421 * #417 * #419 The test takes a copy of the Git repository and creates several copies within a nested directory heirarchy. ``` Test 4bcd533 f9255a5 f28fc01 b713582 ------------------------------------------------------------------------------------------------------------------------------------------------- 2000.2: git status (full-v3) 0.19(0.15+0.05) 0.19(0.16+0.05) +0.0% 0.20(0.18+0.03) +5.3% 0.19(0.17+0.04) +0.0% 2000.3: git status (full-v4) 0.20(0.18+0.04) 0.19(0.15+0.06) -5.0% 0.21(0.18+0.05) +5.0% 0.18(0.18+0.02) -10.0% 2000.4: git status (sparse-v3) 0.04(0.04+0.04) 0.05(0.07+0.04) +25.0% 0.04(0.04+0.05) +0.0% 0.04(0.06+0.04) +0.0% 2000.5: git status (sparse-v4) 0.04(0.03+0.06) 0.04(0.05+0.05) +0.0% 0.05(0.05+0.04) +25.0% 0.05(0.06+0.04) +25.0% 2000.6: git add -A (full-v3) 0.36(0.29+0.05) 0.38(0.28+0.07) +5.6% 0.36(0.31+0.05) +0.0% 0.37(0.31+0.05) +2.8% 2000.7: git add -A (full-v4) 0.34(0.27+0.06) 0.34(0.29+0.05) +0.0% 0.34(0.29+0.04) +0.0% 0.35(0.28+0.06) +2.9% 2000.8: git add -A (sparse-v3) 0.06(0.07+0.04) 0.06(0.05+0.06) +0.0% 0.06(0.09+0.01) +0.0% 0.06(0.08+0.03) +0.0% 2000.9: git add -A (sparse-v4) 0.05(0.05+0.04) 0.05(0.05+0.07) +0.0% 0.05(0.04+0.06) +0.0% 0.06(0.06+0.05) +20.0% 2000.10: git add . (full-v3) 0.38(0.31+0.05) 0.37(0.29+0.06) -2.6% 0.37(0.30+0.07) -2.6% 0.37(0.29+0.06) -2.6% 2000.11: git add . (full-v4) 0.35(0.31+0.04) 0.35(0.29+0.07) +0.0% 0.35(0.29+0.05) +0.0% 0.34(0.29+0.06) -2.9% 2000.12: git add . (sparse-v3) 0.06(0.06+0.05) 0.06(0.05+0.06) +0.0% 0.06(0.07+0.05) +0.0% 0.06(0.09+0.03) +0.0% 2000.13: git add . (sparse-v4) 0.06(0.06+0.06) 0.06(0.07+0.04) +0.0% 0.05(0.06+0.05) -16.7% 0.05(0.05+0.07) -16.7% 2000.14: git commit -a -m A (full-v3) 0.48(0.37+0.08) 0.45(0.36+0.08) -6.2% 0.45(0.35+0.09) -6.2% 0.44(0.36+0.07) -8.3% 2000.15: git commit -a -m A (full-v4) 0.45(0.40+0.06) 0.43(0.34+0.07) -4.4% 0.45(0.37+0.06) +0.0% 0.42(0.36+0.05) -6.7% 2000.16: git commit -a -m A (sparse-v3) 0.05(0.05+0.06) 0.05(0.05+0.03) +0.0% 0.05(0.06+0.06) +0.0% 0.05(0.04+0.06) +0.0% 2000.17: git commit -a -m A (sparse-v4) 0.05(0.06+0.03) 0.05(0.06+0.04) +0.0% 0.06(0.07+0.05) +20.0% 0.05(0.04+0.06) +0.0% 2000.18: git checkout -f - (full-v3) 0.55(0.43+0.08) 0.54(0.46+0.05) -1.8% 0.55(0.46+0.07) +0.0% 0.54(0.40+0.10) -1.8% 2000.19: git checkout -f - (full-v4) 0.55(0.41+0.09) 0.50(0.40+0.09) -9.1% 0.51(0.46+0.05) -7.3% 0.51(0.44+0.06) -7.3% 2000.20: git checkout -f - (sparse-v3) 0.06(0.09+0.03) 0.06(0.08+0.03) +0.0% 0.06(0.06+0.05) +0.0% 0.07(0.09+0.03) +16.7% 2000.21: git checkout -f - (sparse-v4) 0.06(0.08+0.04) 0.05(0.07+0.05) -16.7% 0.05(0.07+0.04) -16.7% 0.06(0.09+0.03) +0.0% ``` All of the above were already integrated. ``` 2000.22: git reset (full-v3) 0.41(0.32+0.06) 0.40(0.31+0.06) -2.4% 0.41(0.33+0.05) +0.0% 0.42(0.34+0.04) +2.4% 2000.23: git reset (full-v4) 0.37(0.32+0.05) 0.35(0.30+0.05) -5.4% 0.37(0.30+0.05) +0.0% 0.35(0.31+0.03) -5.4% 2000.24: git reset (sparse-v3) 0.68(0.65+0.05) 0.55(0.52+0.04) -19.1% 0.04(0.05+0.04) -94.1% 0.04(0.05+0.04) -94.1% 2000.25: git reset (sparse-v4) 0.70(0.65+0.05) 0.54(0.50+0.06) -22.9% 0.04(0.07+0.01) -94.3% 0.03(0.05+0.05) -95.7% 2000.26: git reset --hard (full-v3) 0.54(0.43+0.07) 0.53(0.43+0.06) -1.9% 0.55(0.46+0.05) +1.9% 0.55(0.44+0.06) +1.9% 2000.27: git reset --hard (full-v4) 0.50(0.45+0.03) 0.50(0.43+0.05) +0.0% 0.49(0.41+0.06) -2.0% 0.50(0.42+0.05) +0.0% 2000.28: git reset --hard (sparse-v3) 0.83(0.76+0.06) 0.68(0.62+0.05) -18.1% 0.07(0.05+0.02) -91.6% 0.07(0.05+0.02) -91.6% 2000.29: git reset --hard (sparse-v4) 0.80(0.75+0.05) 0.69(0.62+0.06) -13.8% 0.07(0.04+0.02) -91.2% 0.07(0.04+0.03) -91.2% ``` As expected, `git reset [--hard]` improves with the sparse index integration, but remains constant across the full index case. ``` 2000.30: git update-index --add --remove (full-v3) 0.03(0.01+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.02+0.01) +0.0% 0.03(0.01+0.01) +0.0% 2000.31: git update-index --add --remove (full-v4) 0.03(0.02+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.03+0.00) +0.0% 0.03(0.02+0.01) +0.0% 2000.32: git update-index --add --remove (sparse-v3) 0.57(0.54+0.02) 0.43(0.42+0.00) -24.6% 0.44(0.41+0.03) -22.8% 0.44(0.42+0.01) -22.8% 2000.33: git update-index --add --remove (sparse-v4) 0.56(0.52+0.04) 0.43(0.42+0.01) -23.2% 0.44(0.42+0.02) -21.4% 0.42(0.41+0.01) -25.0% ``` These do not change significantly because #423 is not merged. ``` 2000.34: git diff (full-v3) 0.07(0.05+0.03) 0.06(0.05+0.03) -14.3% 0.07(0.05+0.03) +0.0% 0.06(0.05+0.03) -14.3% 2000.35: git diff (full-v4) 0.06(0.05+0.03) 0.06(0.05+0.02) +0.0% 0.06(0.05+0.02) +0.0% 0.06(0.06+0.02) +0.0% 2000.36: git diff (sparse-v3) 0.25(0.23+0.03) 0.17(0.17+0.02) -32.0% 0.18(0.18+0.02) -28.0% 0.01(0.03+0.03) -96.0% 2000.37: git diff (sparse-v4) 0.25(0.22+0.05) 0.16(0.16+0.01) -36.0% 0.18(0.15+0.04) -28.0% 0.01(0.04+0.02) -96.0% 2000.38: git diff --staged (full-v3) 0.03(0.01+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.02+0.01) +0.0% 0.03(0.02+0.00) +0.0% 2000.39: git diff --staged (full-v4) 0.04(0.03+0.01) 0.03(0.02+0.01) -25.0% 0.03(0.03+0.00) -25.0% 0.03(0.03+0.00) -25.0% 2000.40: git diff --staged (sparse-v3) 0.21(0.19+0.01) 0.15(0.13+0.01) -28.6% 0.15(0.14+0.01) -28.6% 0.01(0.01+0.00) -95.2% 2000.41: git diff --staged (sparse-v4) 0.22(0.21+0.01) 0.14(0.11+0.03) -36.4% 0.15(0.13+0.02) -31.8% 0.01(0.01+0.00) -95.5% ``` The `git diff` improvements are measurable. ``` 2000.42: git sparse-checkout reapply (full-v3) 0.63(0.54+0.05) 0.56(0.48+0.04) -11.1% 0.57(0.48+0.03) -9.5% 0.59(0.48+0.05) -6.3% 2000.43: git sparse-checkout reapply (full-v4) 0.60(0.54+0.02) 0.51(0.46+0.03) -15.0% 0.54(0.48+0.02) -10.0% 0.50(0.44+0.04) -16.7% 2000.44: git sparse-checkout reapply (sparse-v3) 0.91(0.86+0.05) 0.05(0.05+0.00) -94.5% 0.06(0.05+0.01) -93.4% 0.06(0.06+0.00) -93.4% 2000.45: git sparse-checkout reapply (sparse-v4) 0.92(0.88+0.04) 0.05(0.05+0.00) -94.6% 0.05(0.05+0.01) -94.6% 0.05(0.04+0.01) -94.6% ``` Finally, the `git sparse-checkout` measurements are also present. This test script is particularly valuable when contributing changes upstream. It can be good to start by adding the lines to the performance test in an early commit, then demonstrating the performance change by copying the necessary lines from the output table into your commit message.

Sparse index: `git reset`

One thing I forgot when talking about the sparse index is that we have a performance test: `t/perf/p2000-sparse-operations.sh`. This test wasn't helpful for commands like `git merge` that need a particular set of input, but work for more read-only operations. Here is a quick demonstration of how this performance test works so we could have a definitive measure of how your previous updates improved performance. To get these results, I ran the following command in `t/perf`: ``` ./run 4bcd533 f9255a5 f28fc01 b713582 -- p2000-sparse-operations.sh ``` The short-shas correspond to the merge commits for these PRs: * #410 * #421 * #417 * #419 The test takes a copy of the Git repository and creates several copies within a nested directory heirarchy. ``` Test 4bcd533 f9255a5 f28fc01 b713582 ------------------------------------------------------------------------------------------------------------------------------------------------- 2000.2: git status (full-v3) 0.19(0.15+0.05) 0.19(0.16+0.05) +0.0% 0.20(0.18+0.03) +5.3% 0.19(0.17+0.04) +0.0% 2000.3: git status (full-v4) 0.20(0.18+0.04) 0.19(0.15+0.06) -5.0% 0.21(0.18+0.05) +5.0% 0.18(0.18+0.02) -10.0% 2000.4: git status (sparse-v3) 0.04(0.04+0.04) 0.05(0.07+0.04) +25.0% 0.04(0.04+0.05) +0.0% 0.04(0.06+0.04) +0.0% 2000.5: git status (sparse-v4) 0.04(0.03+0.06) 0.04(0.05+0.05) +0.0% 0.05(0.05+0.04) +25.0% 0.05(0.06+0.04) +25.0% 2000.6: git add -A (full-v3) 0.36(0.29+0.05) 0.38(0.28+0.07) +5.6% 0.36(0.31+0.05) +0.0% 0.37(0.31+0.05) +2.8% 2000.7: git add -A (full-v4) 0.34(0.27+0.06) 0.34(0.29+0.05) +0.0% 0.34(0.29+0.04) +0.0% 0.35(0.28+0.06) +2.9% 2000.8: git add -A (sparse-v3) 0.06(0.07+0.04) 0.06(0.05+0.06) +0.0% 0.06(0.09+0.01) +0.0% 0.06(0.08+0.03) +0.0% 2000.9: git add -A (sparse-v4) 0.05(0.05+0.04) 0.05(0.05+0.07) +0.0% 0.05(0.04+0.06) +0.0% 0.06(0.06+0.05) +20.0% 2000.10: git add . (full-v3) 0.38(0.31+0.05) 0.37(0.29+0.06) -2.6% 0.37(0.30+0.07) -2.6% 0.37(0.29+0.06) -2.6% 2000.11: git add . (full-v4) 0.35(0.31+0.04) 0.35(0.29+0.07) +0.0% 0.35(0.29+0.05) +0.0% 0.34(0.29+0.06) -2.9% 2000.12: git add . (sparse-v3) 0.06(0.06+0.05) 0.06(0.05+0.06) +0.0% 0.06(0.07+0.05) +0.0% 0.06(0.09+0.03) +0.0% 2000.13: git add . (sparse-v4) 0.06(0.06+0.06) 0.06(0.07+0.04) +0.0% 0.05(0.06+0.05) -16.7% 0.05(0.05+0.07) -16.7% 2000.14: git commit -a -m A (full-v3) 0.48(0.37+0.08) 0.45(0.36+0.08) -6.2% 0.45(0.35+0.09) -6.2% 0.44(0.36+0.07) -8.3% 2000.15: git commit -a -m A (full-v4) 0.45(0.40+0.06) 0.43(0.34+0.07) -4.4% 0.45(0.37+0.06) +0.0% 0.42(0.36+0.05) -6.7% 2000.16: git commit -a -m A (sparse-v3) 0.05(0.05+0.06) 0.05(0.05+0.03) +0.0% 0.05(0.06+0.06) +0.0% 0.05(0.04+0.06) +0.0% 2000.17: git commit -a -m A (sparse-v4) 0.05(0.06+0.03) 0.05(0.06+0.04) +0.0% 0.06(0.07+0.05) +20.0% 0.05(0.04+0.06) +0.0% 2000.18: git checkout -f - (full-v3) 0.55(0.43+0.08) 0.54(0.46+0.05) -1.8% 0.55(0.46+0.07) +0.0% 0.54(0.40+0.10) -1.8% 2000.19: git checkout -f - (full-v4) 0.55(0.41+0.09) 0.50(0.40+0.09) -9.1% 0.51(0.46+0.05) -7.3% 0.51(0.44+0.06) -7.3% 2000.20: git checkout -f - (sparse-v3) 0.06(0.09+0.03) 0.06(0.08+0.03) +0.0% 0.06(0.06+0.05) +0.0% 0.07(0.09+0.03) +16.7% 2000.21: git checkout -f - (sparse-v4) 0.06(0.08+0.04) 0.05(0.07+0.05) -16.7% 0.05(0.07+0.04) -16.7% 0.06(0.09+0.03) +0.0% ``` All of the above were already integrated. ``` 2000.22: git reset (full-v3) 0.41(0.32+0.06) 0.40(0.31+0.06) -2.4% 0.41(0.33+0.05) +0.0% 0.42(0.34+0.04) +2.4% 2000.23: git reset (full-v4) 0.37(0.32+0.05) 0.35(0.30+0.05) -5.4% 0.37(0.30+0.05) +0.0% 0.35(0.31+0.03) -5.4% 2000.24: git reset (sparse-v3) 0.68(0.65+0.05) 0.55(0.52+0.04) -19.1% 0.04(0.05+0.04) -94.1% 0.04(0.05+0.04) -94.1% 2000.25: git reset (sparse-v4) 0.70(0.65+0.05) 0.54(0.50+0.06) -22.9% 0.04(0.07+0.01) -94.3% 0.03(0.05+0.05) -95.7% 2000.26: git reset --hard (full-v3) 0.54(0.43+0.07) 0.53(0.43+0.06) -1.9% 0.55(0.46+0.05) +1.9% 0.55(0.44+0.06) +1.9% 2000.27: git reset --hard (full-v4) 0.50(0.45+0.03) 0.50(0.43+0.05) +0.0% 0.49(0.41+0.06) -2.0% 0.50(0.42+0.05) +0.0% 2000.28: git reset --hard (sparse-v3) 0.83(0.76+0.06) 0.68(0.62+0.05) -18.1% 0.07(0.05+0.02) -91.6% 0.07(0.05+0.02) -91.6% 2000.29: git reset --hard (sparse-v4) 0.80(0.75+0.05) 0.69(0.62+0.06) -13.8% 0.07(0.04+0.02) -91.2% 0.07(0.04+0.03) -91.2% ``` As expected, `git reset [--hard]` improves with the sparse index integration, but remains constant across the full index case. ``` 2000.30: git update-index --add --remove (full-v3) 0.03(0.01+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.02+0.01) +0.0% 0.03(0.01+0.01) +0.0% 2000.31: git update-index --add --remove (full-v4) 0.03(0.02+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.03+0.00) +0.0% 0.03(0.02+0.01) +0.0% 2000.32: git update-index --add --remove (sparse-v3) 0.57(0.54+0.02) 0.43(0.42+0.00) -24.6% 0.44(0.41+0.03) -22.8% 0.44(0.42+0.01) -22.8% 2000.33: git update-index --add --remove (sparse-v4) 0.56(0.52+0.04) 0.43(0.42+0.01) -23.2% 0.44(0.42+0.02) -21.4% 0.42(0.41+0.01) -25.0% ``` These do not change significantly because #423 is not merged. ``` 2000.34: git diff (full-v3) 0.07(0.05+0.03) 0.06(0.05+0.03) -14.3% 0.07(0.05+0.03) +0.0% 0.06(0.05+0.03) -14.3% 2000.35: git diff (full-v4) 0.06(0.05+0.03) 0.06(0.05+0.02) +0.0% 0.06(0.05+0.02) +0.0% 0.06(0.06+0.02) +0.0% 2000.36: git diff (sparse-v3) 0.25(0.23+0.03) 0.17(0.17+0.02) -32.0% 0.18(0.18+0.02) -28.0% 0.01(0.03+0.03) -96.0% 2000.37: git diff (sparse-v4) 0.25(0.22+0.05) 0.16(0.16+0.01) -36.0% 0.18(0.15+0.04) -28.0% 0.01(0.04+0.02) -96.0% 2000.38: git diff --staged (full-v3) 0.03(0.01+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.02+0.01) +0.0% 0.03(0.02+0.00) +0.0% 2000.39: git diff --staged (full-v4) 0.04(0.03+0.01) 0.03(0.02+0.01) -25.0% 0.03(0.03+0.00) -25.0% 0.03(0.03+0.00) -25.0% 2000.40: git diff --staged (sparse-v3) 0.21(0.19+0.01) 0.15(0.13+0.01) -28.6% 0.15(0.14+0.01) -28.6% 0.01(0.01+0.00) -95.2% 2000.41: git diff --staged (sparse-v4) 0.22(0.21+0.01) 0.14(0.11+0.03) -36.4% 0.15(0.13+0.02) -31.8% 0.01(0.01+0.00) -95.5% ``` The `git diff` improvements are measurable. ``` 2000.42: git sparse-checkout reapply (full-v3) 0.63(0.54+0.05) 0.56(0.48+0.04) -11.1% 0.57(0.48+0.03) -9.5% 0.59(0.48+0.05) -6.3% 2000.43: git sparse-checkout reapply (full-v4) 0.60(0.54+0.02) 0.51(0.46+0.03) -15.0% 0.54(0.48+0.02) -10.0% 0.50(0.44+0.04) -16.7% 2000.44: git sparse-checkout reapply (sparse-v3) 0.91(0.86+0.05) 0.05(0.05+0.00) -94.5% 0.06(0.05+0.01) -93.4% 0.06(0.06+0.00) -93.4% 2000.45: git sparse-checkout reapply (sparse-v4) 0.92(0.88+0.04) 0.05(0.05+0.00) -94.6% 0.05(0.05+0.01) -94.6% 0.05(0.04+0.01) -94.6% ``` Finally, the `git sparse-checkout` measurements are also present. This test script is particularly valuable when contributing changes upstream. It can be good to start by adding the lines to the performance test in an early commit, then demonstrating the performance change by copying the necessary lines from the output table into your commit message.

Sparse index: `git reset`

One thing I forgot when talking about the sparse index is that we have a performance test: `t/perf/p2000-sparse-operations.sh`. This test wasn't helpful for commands like `git merge` that need a particular set of input, but work for more read-only operations. Here is a quick demonstration of how this performance test works so we could have a definitive measure of how your previous updates improved performance. To get these results, I ran the following command in `t/perf`: ``` ./run 4bcd533 f9255a5 f28fc01 b713582 -- p2000-sparse-operations.sh ``` The short-shas correspond to the merge commits for these PRs: * #410 * #421 * #417 * #419 The test takes a copy of the Git repository and creates several copies within a nested directory heirarchy. ``` Test 4bcd533 f9255a5 f28fc01 b713582 ------------------------------------------------------------------------------------------------------------------------------------------------- 2000.2: git status (full-v3) 0.19(0.15+0.05) 0.19(0.16+0.05) +0.0% 0.20(0.18+0.03) +5.3% 0.19(0.17+0.04) +0.0% 2000.3: git status (full-v4) 0.20(0.18+0.04) 0.19(0.15+0.06) -5.0% 0.21(0.18+0.05) +5.0% 0.18(0.18+0.02) -10.0% 2000.4: git status (sparse-v3) 0.04(0.04+0.04) 0.05(0.07+0.04) +25.0% 0.04(0.04+0.05) +0.0% 0.04(0.06+0.04) +0.0% 2000.5: git status (sparse-v4) 0.04(0.03+0.06) 0.04(0.05+0.05) +0.0% 0.05(0.05+0.04) +25.0% 0.05(0.06+0.04) +25.0% 2000.6: git add -A (full-v3) 0.36(0.29+0.05) 0.38(0.28+0.07) +5.6% 0.36(0.31+0.05) +0.0% 0.37(0.31+0.05) +2.8% 2000.7: git add -A (full-v4) 0.34(0.27+0.06) 0.34(0.29+0.05) +0.0% 0.34(0.29+0.04) +0.0% 0.35(0.28+0.06) +2.9% 2000.8: git add -A (sparse-v3) 0.06(0.07+0.04) 0.06(0.05+0.06) +0.0% 0.06(0.09+0.01) +0.0% 0.06(0.08+0.03) +0.0% 2000.9: git add -A (sparse-v4) 0.05(0.05+0.04) 0.05(0.05+0.07) +0.0% 0.05(0.04+0.06) +0.0% 0.06(0.06+0.05) +20.0% 2000.10: git add . (full-v3) 0.38(0.31+0.05) 0.37(0.29+0.06) -2.6% 0.37(0.30+0.07) -2.6% 0.37(0.29+0.06) -2.6% 2000.11: git add . (full-v4) 0.35(0.31+0.04) 0.35(0.29+0.07) +0.0% 0.35(0.29+0.05) +0.0% 0.34(0.29+0.06) -2.9% 2000.12: git add . (sparse-v3) 0.06(0.06+0.05) 0.06(0.05+0.06) +0.0% 0.06(0.07+0.05) +0.0% 0.06(0.09+0.03) +0.0% 2000.13: git add . (sparse-v4) 0.06(0.06+0.06) 0.06(0.07+0.04) +0.0% 0.05(0.06+0.05) -16.7% 0.05(0.05+0.07) -16.7% 2000.14: git commit -a -m A (full-v3) 0.48(0.37+0.08) 0.45(0.36+0.08) -6.2% 0.45(0.35+0.09) -6.2% 0.44(0.36+0.07) -8.3% 2000.15: git commit -a -m A (full-v4) 0.45(0.40+0.06) 0.43(0.34+0.07) -4.4% 0.45(0.37+0.06) +0.0% 0.42(0.36+0.05) -6.7% 2000.16: git commit -a -m A (sparse-v3) 0.05(0.05+0.06) 0.05(0.05+0.03) +0.0% 0.05(0.06+0.06) +0.0% 0.05(0.04+0.06) +0.0% 2000.17: git commit -a -m A (sparse-v4) 0.05(0.06+0.03) 0.05(0.06+0.04) +0.0% 0.06(0.07+0.05) +20.0% 0.05(0.04+0.06) +0.0% 2000.18: git checkout -f - (full-v3) 0.55(0.43+0.08) 0.54(0.46+0.05) -1.8% 0.55(0.46+0.07) +0.0% 0.54(0.40+0.10) -1.8% 2000.19: git checkout -f - (full-v4) 0.55(0.41+0.09) 0.50(0.40+0.09) -9.1% 0.51(0.46+0.05) -7.3% 0.51(0.44+0.06) -7.3% 2000.20: git checkout -f - (sparse-v3) 0.06(0.09+0.03) 0.06(0.08+0.03) +0.0% 0.06(0.06+0.05) +0.0% 0.07(0.09+0.03) +16.7% 2000.21: git checkout -f - (sparse-v4) 0.06(0.08+0.04) 0.05(0.07+0.05) -16.7% 0.05(0.07+0.04) -16.7% 0.06(0.09+0.03) +0.0% ``` All of the above were already integrated. ``` 2000.22: git reset (full-v3) 0.41(0.32+0.06) 0.40(0.31+0.06) -2.4% 0.41(0.33+0.05) +0.0% 0.42(0.34+0.04) +2.4% 2000.23: git reset (full-v4) 0.37(0.32+0.05) 0.35(0.30+0.05) -5.4% 0.37(0.30+0.05) +0.0% 0.35(0.31+0.03) -5.4% 2000.24: git reset (sparse-v3) 0.68(0.65+0.05) 0.55(0.52+0.04) -19.1% 0.04(0.05+0.04) -94.1% 0.04(0.05+0.04) -94.1% 2000.25: git reset (sparse-v4) 0.70(0.65+0.05) 0.54(0.50+0.06) -22.9% 0.04(0.07+0.01) -94.3% 0.03(0.05+0.05) -95.7% 2000.26: git reset --hard (full-v3) 0.54(0.43+0.07) 0.53(0.43+0.06) -1.9% 0.55(0.46+0.05) +1.9% 0.55(0.44+0.06) +1.9% 2000.27: git reset --hard (full-v4) 0.50(0.45+0.03) 0.50(0.43+0.05) +0.0% 0.49(0.41+0.06) -2.0% 0.50(0.42+0.05) +0.0% 2000.28: git reset --hard (sparse-v3) 0.83(0.76+0.06) 0.68(0.62+0.05) -18.1% 0.07(0.05+0.02) -91.6% 0.07(0.05+0.02) -91.6% 2000.29: git reset --hard (sparse-v4) 0.80(0.75+0.05) 0.69(0.62+0.06) -13.8% 0.07(0.04+0.02) -91.2% 0.07(0.04+0.03) -91.2% ``` As expected, `git reset [--hard]` improves with the sparse index integration, but remains constant across the full index case. ``` 2000.30: git update-index --add --remove (full-v3) 0.03(0.01+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.02+0.01) +0.0% 0.03(0.01+0.01) +0.0% 2000.31: git update-index --add --remove (full-v4) 0.03(0.02+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.03+0.00) +0.0% 0.03(0.02+0.01) +0.0% 2000.32: git update-index --add --remove (sparse-v3) 0.57(0.54+0.02) 0.43(0.42+0.00) -24.6% 0.44(0.41+0.03) -22.8% 0.44(0.42+0.01) -22.8% 2000.33: git update-index --add --remove (sparse-v4) 0.56(0.52+0.04) 0.43(0.42+0.01) -23.2% 0.44(0.42+0.02) -21.4% 0.42(0.41+0.01) -25.0% ``` These do not change significantly because #423 is not merged. ``` 2000.34: git diff (full-v3) 0.07(0.05+0.03) 0.06(0.05+0.03) -14.3% 0.07(0.05+0.03) +0.0% 0.06(0.05+0.03) -14.3% 2000.35: git diff (full-v4) 0.06(0.05+0.03) 0.06(0.05+0.02) +0.0% 0.06(0.05+0.02) +0.0% 0.06(0.06+0.02) +0.0% 2000.36: git diff (sparse-v3) 0.25(0.23+0.03) 0.17(0.17+0.02) -32.0% 0.18(0.18+0.02) -28.0% 0.01(0.03+0.03) -96.0% 2000.37: git diff (sparse-v4) 0.25(0.22+0.05) 0.16(0.16+0.01) -36.0% 0.18(0.15+0.04) -28.0% 0.01(0.04+0.02) -96.0% 2000.38: git diff --staged (full-v3) 0.03(0.01+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.02+0.01) +0.0% 0.03(0.02+0.00) +0.0% 2000.39: git diff --staged (full-v4) 0.04(0.03+0.01) 0.03(0.02+0.01) -25.0% 0.03(0.03+0.00) -25.0% 0.03(0.03+0.00) -25.0% 2000.40: git diff --staged (sparse-v3) 0.21(0.19+0.01) 0.15(0.13+0.01) -28.6% 0.15(0.14+0.01) -28.6% 0.01(0.01+0.00) -95.2% 2000.41: git diff --staged (sparse-v4) 0.22(0.21+0.01) 0.14(0.11+0.03) -36.4% 0.15(0.13+0.02) -31.8% 0.01(0.01+0.00) -95.5% ``` The `git diff` improvements are measurable. ``` 2000.42: git sparse-checkout reapply (full-v3) 0.63(0.54+0.05) 0.56(0.48+0.04) -11.1% 0.57(0.48+0.03) -9.5% 0.59(0.48+0.05) -6.3% 2000.43: git sparse-checkout reapply (full-v4) 0.60(0.54+0.02) 0.51(0.46+0.03) -15.0% 0.54(0.48+0.02) -10.0% 0.50(0.44+0.04) -16.7% 2000.44: git sparse-checkout reapply (sparse-v3) 0.91(0.86+0.05) 0.05(0.05+0.00) -94.5% 0.06(0.05+0.01) -93.4% 0.06(0.06+0.00) -93.4% 2000.45: git sparse-checkout reapply (sparse-v4) 0.92(0.88+0.04) 0.05(0.05+0.00) -94.6% 0.05(0.05+0.01) -94.6% 0.05(0.04+0.01) -94.6% ``` Finally, the `git sparse-checkout` measurements are also present. This test script is particularly valuable when contributing changes upstream. It can be good to start by adding the lines to the performance test in an early commit, then demonstrating the performance change by copying the necessary lines from the output table into your commit message.

Sparse index: `git reset`

…tests One thing I forgot when talking about the sparse index is that we have a performance test: `t/perf/p2000-sparse-operations.sh`. This test wasn't helpful for commands like `git merge` that need a particular set of input, but work for more read-only operations. Here is a quick demonstration of how this performance test works so we could have a definitive measure of how your previous updates improved performance. To get these results, I ran the following command in `t/perf`: ``` ./run 4bcd533 f9255a5 f28fc01 b713582 -- p2000-sparse-operations.sh ``` The short-shas correspond to the merge commits for these PRs: * microsoft#410 * microsoft#421 * microsoft#417 * microsoft#419 The test takes a copy of the Git repository and creates several copies within a nested directory heirarchy. ``` Test 4bcd533 f9255a5 f28fc01 b713582 ------------------------------------------------------------------------------------------------------------------------------------------------- 2000.2: git status (full-v3) 0.19(0.15+0.05) 0.19(0.16+0.05) +0.0% 0.20(0.18+0.03) +5.3% 0.19(0.17+0.04) +0.0% 2000.3: git status (full-v4) 0.20(0.18+0.04) 0.19(0.15+0.06) -5.0% 0.21(0.18+0.05) +5.0% 0.18(0.18+0.02) -10.0% 2000.4: git status (sparse-v3) 0.04(0.04+0.04) 0.05(0.07+0.04) +25.0% 0.04(0.04+0.05) +0.0% 0.04(0.06+0.04) +0.0% 2000.5: git status (sparse-v4) 0.04(0.03+0.06) 0.04(0.05+0.05) +0.0% 0.05(0.05+0.04) +25.0% 0.05(0.06+0.04) +25.0% 2000.6: git add -A (full-v3) 0.36(0.29+0.05) 0.38(0.28+0.07) +5.6% 0.36(0.31+0.05) +0.0% 0.37(0.31+0.05) +2.8% 2000.7: git add -A (full-v4) 0.34(0.27+0.06) 0.34(0.29+0.05) +0.0% 0.34(0.29+0.04) +0.0% 0.35(0.28+0.06) +2.9% 2000.8: git add -A (sparse-v3) 0.06(0.07+0.04) 0.06(0.05+0.06) +0.0% 0.06(0.09+0.01) +0.0% 0.06(0.08+0.03) +0.0% 2000.9: git add -A (sparse-v4) 0.05(0.05+0.04) 0.05(0.05+0.07) +0.0% 0.05(0.04+0.06) +0.0% 0.06(0.06+0.05) +20.0% 2000.10: git add . (full-v3) 0.38(0.31+0.05) 0.37(0.29+0.06) -2.6% 0.37(0.30+0.07) -2.6% 0.37(0.29+0.06) -2.6% 2000.11: git add . (full-v4) 0.35(0.31+0.04) 0.35(0.29+0.07) +0.0% 0.35(0.29+0.05) +0.0% 0.34(0.29+0.06) -2.9% 2000.12: git add . (sparse-v3) 0.06(0.06+0.05) 0.06(0.05+0.06) +0.0% 0.06(0.07+0.05) +0.0% 0.06(0.09+0.03) +0.0% 2000.13: git add . (sparse-v4) 0.06(0.06+0.06) 0.06(0.07+0.04) +0.0% 0.05(0.06+0.05) -16.7% 0.05(0.05+0.07) -16.7% 2000.14: git commit -a -m A (full-v3) 0.48(0.37+0.08) 0.45(0.36+0.08) -6.2% 0.45(0.35+0.09) -6.2% 0.44(0.36+0.07) -8.3% 2000.15: git commit -a -m A (full-v4) 0.45(0.40+0.06) 0.43(0.34+0.07) -4.4% 0.45(0.37+0.06) +0.0% 0.42(0.36+0.05) -6.7% 2000.16: git commit -a -m A (sparse-v3) 0.05(0.05+0.06) 0.05(0.05+0.03) +0.0% 0.05(0.06+0.06) +0.0% 0.05(0.04+0.06) +0.0% 2000.17: git commit -a -m A (sparse-v4) 0.05(0.06+0.03) 0.05(0.06+0.04) +0.0% 0.06(0.07+0.05) +20.0% 0.05(0.04+0.06) +0.0% 2000.18: git checkout -f - (full-v3) 0.55(0.43+0.08) 0.54(0.46+0.05) -1.8% 0.55(0.46+0.07) +0.0% 0.54(0.40+0.10) -1.8% 2000.19: git checkout -f - (full-v4) 0.55(0.41+0.09) 0.50(0.40+0.09) -9.1% 0.51(0.46+0.05) -7.3% 0.51(0.44+0.06) -7.3% 2000.20: git checkout -f - (sparse-v3) 0.06(0.09+0.03) 0.06(0.08+0.03) +0.0% 0.06(0.06+0.05) +0.0% 0.07(0.09+0.03) +16.7% 2000.21: git checkout -f - (sparse-v4) 0.06(0.08+0.04) 0.05(0.07+0.05) -16.7% 0.05(0.07+0.04) -16.7% 0.06(0.09+0.03) +0.0% ``` All of the above were already integrated. ``` 2000.22: git reset (full-v3) 0.41(0.32+0.06) 0.40(0.31+0.06) -2.4% 0.41(0.33+0.05) +0.0% 0.42(0.34+0.04) +2.4% 2000.23: git reset (full-v4) 0.37(0.32+0.05) 0.35(0.30+0.05) -5.4% 0.37(0.30+0.05) +0.0% 0.35(0.31+0.03) -5.4% 2000.24: git reset (sparse-v3) 0.68(0.65+0.05) 0.55(0.52+0.04) -19.1% 0.04(0.05+0.04) -94.1% 0.04(0.05+0.04) -94.1% 2000.25: git reset (sparse-v4) 0.70(0.65+0.05) 0.54(0.50+0.06) -22.9% 0.04(0.07+0.01) -94.3% 0.03(0.05+0.05) -95.7% 2000.26: git reset --hard (full-v3) 0.54(0.43+0.07) 0.53(0.43+0.06) -1.9% 0.55(0.46+0.05) +1.9% 0.55(0.44+0.06) +1.9% 2000.27: git reset --hard (full-v4) 0.50(0.45+0.03) 0.50(0.43+0.05) +0.0% 0.49(0.41+0.06) -2.0% 0.50(0.42+0.05) +0.0% 2000.28: git reset --hard (sparse-v3) 0.83(0.76+0.06) 0.68(0.62+0.05) -18.1% 0.07(0.05+0.02) -91.6% 0.07(0.05+0.02) -91.6% 2000.29: git reset --hard (sparse-v4) 0.80(0.75+0.05) 0.69(0.62+0.06) -13.8% 0.07(0.04+0.02) -91.2% 0.07(0.04+0.03) -91.2% ``` As expected, `git reset [--hard]` improves with the sparse index integration, but remains constant across the full index case. ``` 2000.30: git update-index --add --remove (full-v3) 0.03(0.01+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.02+0.01) +0.0% 0.03(0.01+0.01) +0.0% 2000.31: git update-index --add --remove (full-v4) 0.03(0.02+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.03+0.00) +0.0% 0.03(0.02+0.01) +0.0% 2000.32: git update-index --add --remove (sparse-v3) 0.57(0.54+0.02) 0.43(0.42+0.00) -24.6% 0.44(0.41+0.03) -22.8% 0.44(0.42+0.01) -22.8% 2000.33: git update-index --add --remove (sparse-v4) 0.56(0.52+0.04) 0.43(0.42+0.01) -23.2% 0.44(0.42+0.02) -21.4% 0.42(0.41+0.01) -25.0% ``` These do not change significantly because microsoft#423 is not merged. ``` 2000.34: git diff (full-v3) 0.07(0.05+0.03) 0.06(0.05+0.03) -14.3% 0.07(0.05+0.03) +0.0% 0.06(0.05+0.03) -14.3% 2000.35: git diff (full-v4) 0.06(0.05+0.03) 0.06(0.05+0.02) +0.0% 0.06(0.05+0.02) +0.0% 0.06(0.06+0.02) +0.0% 2000.36: git diff (sparse-v3) 0.25(0.23+0.03) 0.17(0.17+0.02) -32.0% 0.18(0.18+0.02) -28.0% 0.01(0.03+0.03) -96.0% 2000.37: git diff (sparse-v4) 0.25(0.22+0.05) 0.16(0.16+0.01) -36.0% 0.18(0.15+0.04) -28.0% 0.01(0.04+0.02) -96.0% 2000.38: git diff --staged (full-v3) 0.03(0.01+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.02+0.01) +0.0% 0.03(0.02+0.00) +0.0% 2000.39: git diff --staged (full-v4) 0.04(0.03+0.01) 0.03(0.02+0.01) -25.0% 0.03(0.03+0.00) -25.0% 0.03(0.03+0.00) -25.0% 2000.40: git diff --staged (sparse-v3) 0.21(0.19+0.01) 0.15(0.13+0.01) -28.6% 0.15(0.14+0.01) -28.6% 0.01(0.01+0.00) -95.2% 2000.41: git diff --staged (sparse-v4) 0.22(0.21+0.01) 0.14(0.11+0.03) -36.4% 0.15(0.13+0.02) -31.8% 0.01(0.01+0.00) -95.5% ``` The `git diff` improvements are measurable. ``` 2000.42: git sparse-checkout reapply (full-v3) 0.63(0.54+0.05) 0.56(0.48+0.04) -11.1% 0.57(0.48+0.03) -9.5% 0.59(0.48+0.05) -6.3% 2000.43: git sparse-checkout reapply (full-v4) 0.60(0.54+0.02) 0.51(0.46+0.03) -15.0% 0.54(0.48+0.02) -10.0% 0.50(0.44+0.04) -16.7% 2000.44: git sparse-checkout reapply (sparse-v3) 0.91(0.86+0.05) 0.05(0.05+0.00) -94.5% 0.06(0.05+0.01) -93.4% 0.06(0.06+0.00) -93.4% 2000.45: git sparse-checkout reapply (sparse-v4) 0.92(0.88+0.04) 0.05(0.05+0.00) -94.6% 0.05(0.05+0.01) -94.6% 0.05(0.04+0.01) -94.6% ``` Finally, the `git sparse-checkout` measurements are also present. This test script is particularly valuable when contributing changes upstream. It can be good to start by adding the lines to the performance test in an early commit, then demonstrating the performance change by copying the necessary lines from the output table into your commit message.

Sparse index: `git reset`

…tests One thing I forgot when talking about the sparse index is that we have a performance test: `t/perf/p2000-sparse-operations.sh`. This test wasn't helpful for commands like `git merge` that need a particular set of input, but work for more read-only operations. Here is a quick demonstration of how this performance test works so we could have a definitive measure of how your previous updates improved performance. To get these results, I ran the following command in `t/perf`: ``` ./run 4bcd533 f9255a5 f28fc01 b713582 -- p2000-sparse-operations.sh ``` The short-shas correspond to the merge commits for these PRs: * microsoft#410 * microsoft#421 * microsoft#417 * microsoft#419 The test takes a copy of the Git repository and creates several copies within a nested directory heirarchy. ``` Test 4bcd533 f9255a5 f28fc01 b713582 ------------------------------------------------------------------------------------------------------------------------------------------------- 2000.2: git status (full-v3) 0.19(0.15+0.05) 0.19(0.16+0.05) +0.0% 0.20(0.18+0.03) +5.3% 0.19(0.17+0.04) +0.0% 2000.3: git status (full-v4) 0.20(0.18+0.04) 0.19(0.15+0.06) -5.0% 0.21(0.18+0.05) +5.0% 0.18(0.18+0.02) -10.0% 2000.4: git status (sparse-v3) 0.04(0.04+0.04) 0.05(0.07+0.04) +25.0% 0.04(0.04+0.05) +0.0% 0.04(0.06+0.04) +0.0% 2000.5: git status (sparse-v4) 0.04(0.03+0.06) 0.04(0.05+0.05) +0.0% 0.05(0.05+0.04) +25.0% 0.05(0.06+0.04) +25.0% 2000.6: git add -A (full-v3) 0.36(0.29+0.05) 0.38(0.28+0.07) +5.6% 0.36(0.31+0.05) +0.0% 0.37(0.31+0.05) +2.8% 2000.7: git add -A (full-v4) 0.34(0.27+0.06) 0.34(0.29+0.05) +0.0% 0.34(0.29+0.04) +0.0% 0.35(0.28+0.06) +2.9% 2000.8: git add -A (sparse-v3) 0.06(0.07+0.04) 0.06(0.05+0.06) +0.0% 0.06(0.09+0.01) +0.0% 0.06(0.08+0.03) +0.0% 2000.9: git add -A (sparse-v4) 0.05(0.05+0.04) 0.05(0.05+0.07) +0.0% 0.05(0.04+0.06) +0.0% 0.06(0.06+0.05) +20.0% 2000.10: git add . (full-v3) 0.38(0.31+0.05) 0.37(0.29+0.06) -2.6% 0.37(0.30+0.07) -2.6% 0.37(0.29+0.06) -2.6% 2000.11: git add . (full-v4) 0.35(0.31+0.04) 0.35(0.29+0.07) +0.0% 0.35(0.29+0.05) +0.0% 0.34(0.29+0.06) -2.9% 2000.12: git add . (sparse-v3) 0.06(0.06+0.05) 0.06(0.05+0.06) +0.0% 0.06(0.07+0.05) +0.0% 0.06(0.09+0.03) +0.0% 2000.13: git add . (sparse-v4) 0.06(0.06+0.06) 0.06(0.07+0.04) +0.0% 0.05(0.06+0.05) -16.7% 0.05(0.05+0.07) -16.7% 2000.14: git commit -a -m A (full-v3) 0.48(0.37+0.08) 0.45(0.36+0.08) -6.2% 0.45(0.35+0.09) -6.2% 0.44(0.36+0.07) -8.3% 2000.15: git commit -a -m A (full-v4) 0.45(0.40+0.06) 0.43(0.34+0.07) -4.4% 0.45(0.37+0.06) +0.0% 0.42(0.36+0.05) -6.7% 2000.16: git commit -a -m A (sparse-v3) 0.05(0.05+0.06) 0.05(0.05+0.03) +0.0% 0.05(0.06+0.06) +0.0% 0.05(0.04+0.06) +0.0% 2000.17: git commit -a -m A (sparse-v4) 0.05(0.06+0.03) 0.05(0.06+0.04) +0.0% 0.06(0.07+0.05) +20.0% 0.05(0.04+0.06) +0.0% 2000.18: git checkout -f - (full-v3) 0.55(0.43+0.08) 0.54(0.46+0.05) -1.8% 0.55(0.46+0.07) +0.0% 0.54(0.40+0.10) -1.8% 2000.19: git checkout -f - (full-v4) 0.55(0.41+0.09) 0.50(0.40+0.09) -9.1% 0.51(0.46+0.05) -7.3% 0.51(0.44+0.06) -7.3% 2000.20: git checkout -f - (sparse-v3) 0.06(0.09+0.03) 0.06(0.08+0.03) +0.0% 0.06(0.06+0.05) +0.0% 0.07(0.09+0.03) +16.7% 2000.21: git checkout -f - (sparse-v4) 0.06(0.08+0.04) 0.05(0.07+0.05) -16.7% 0.05(0.07+0.04) -16.7% 0.06(0.09+0.03) +0.0% ``` All of the above were already integrated. ``` 2000.22: git reset (full-v3) 0.41(0.32+0.06) 0.40(0.31+0.06) -2.4% 0.41(0.33+0.05) +0.0% 0.42(0.34+0.04) +2.4% 2000.23: git reset (full-v4) 0.37(0.32+0.05) 0.35(0.30+0.05) -5.4% 0.37(0.30+0.05) +0.0% 0.35(0.31+0.03) -5.4% 2000.24: git reset (sparse-v3) 0.68(0.65+0.05) 0.55(0.52+0.04) -19.1% 0.04(0.05+0.04) -94.1% 0.04(0.05+0.04) -94.1% 2000.25: git reset (sparse-v4) 0.70(0.65+0.05) 0.54(0.50+0.06) -22.9% 0.04(0.07+0.01) -94.3% 0.03(0.05+0.05) -95.7% 2000.26: git reset --hard (full-v3) 0.54(0.43+0.07) 0.53(0.43+0.06) -1.9% 0.55(0.46+0.05) +1.9% 0.55(0.44+0.06) +1.9% 2000.27: git reset --hard (full-v4) 0.50(0.45+0.03) 0.50(0.43+0.05) +0.0% 0.49(0.41+0.06) -2.0% 0.50(0.42+0.05) +0.0% 2000.28: git reset --hard (sparse-v3) 0.83(0.76+0.06) 0.68(0.62+0.05) -18.1% 0.07(0.05+0.02) -91.6% 0.07(0.05+0.02) -91.6% 2000.29: git reset --hard (sparse-v4) 0.80(0.75+0.05) 0.69(0.62+0.06) -13.8% 0.07(0.04+0.02) -91.2% 0.07(0.04+0.03) -91.2% ``` As expected, `git reset [--hard]` improves with the sparse index integration, but remains constant across the full index case. ``` 2000.30: git update-index --add --remove (full-v3) 0.03(0.01+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.02+0.01) +0.0% 0.03(0.01+0.01) +0.0% 2000.31: git update-index --add --remove (full-v4) 0.03(0.02+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.03+0.00) +0.0% 0.03(0.02+0.01) +0.0% 2000.32: git update-index --add --remove (sparse-v3) 0.57(0.54+0.02) 0.43(0.42+0.00) -24.6% 0.44(0.41+0.03) -22.8% 0.44(0.42+0.01) -22.8% 2000.33: git update-index --add --remove (sparse-v4) 0.56(0.52+0.04) 0.43(0.42+0.01) -23.2% 0.44(0.42+0.02) -21.4% 0.42(0.41+0.01) -25.0% ``` These do not change significantly because microsoft#423 is not merged. ``` 2000.34: git diff (full-v3) 0.07(0.05+0.03) 0.06(0.05+0.03) -14.3% 0.07(0.05+0.03) +0.0% 0.06(0.05+0.03) -14.3% 2000.35: git diff (full-v4) 0.06(0.05+0.03) 0.06(0.05+0.02) +0.0% 0.06(0.05+0.02) +0.0% 0.06(0.06+0.02) +0.0% 2000.36: git diff (sparse-v3) 0.25(0.23+0.03) 0.17(0.17+0.02) -32.0% 0.18(0.18+0.02) -28.0% 0.01(0.03+0.03) -96.0% 2000.37: git diff (sparse-v4) 0.25(0.22+0.05) 0.16(0.16+0.01) -36.0% 0.18(0.15+0.04) -28.0% 0.01(0.04+0.02) -96.0% 2000.38: git diff --staged (full-v3) 0.03(0.01+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.02+0.01) +0.0% 0.03(0.02+0.00) +0.0% 2000.39: git diff --staged (full-v4) 0.04(0.03+0.01) 0.03(0.02+0.01) -25.0% 0.03(0.03+0.00) -25.0% 0.03(0.03+0.00) -25.0% 2000.40: git diff --staged (sparse-v3) 0.21(0.19+0.01) 0.15(0.13+0.01) -28.6% 0.15(0.14+0.01) -28.6% 0.01(0.01+0.00) -95.2% 2000.41: git diff --staged (sparse-v4) 0.22(0.21+0.01) 0.14(0.11+0.03) -36.4% 0.15(0.13+0.02) -31.8% 0.01(0.01+0.00) -95.5% ``` The `git diff` improvements are measurable. ``` 2000.42: git sparse-checkout reapply (full-v3) 0.63(0.54+0.05) 0.56(0.48+0.04) -11.1% 0.57(0.48+0.03) -9.5% 0.59(0.48+0.05) -6.3% 2000.43: git sparse-checkout reapply (full-v4) 0.60(0.54+0.02) 0.51(0.46+0.03) -15.0% 0.54(0.48+0.02) -10.0% 0.50(0.44+0.04) -16.7% 2000.44: git sparse-checkout reapply (sparse-v3) 0.91(0.86+0.05) 0.05(0.05+0.00) -94.5% 0.06(0.05+0.01) -93.4% 0.06(0.06+0.00) -93.4% 2000.45: git sparse-checkout reapply (sparse-v4) 0.92(0.88+0.04) 0.05(0.05+0.00) -94.6% 0.05(0.05+0.01) -94.6% 0.05(0.04+0.01) -94.6% ``` Finally, the `git sparse-checkout` measurements are also present. This test script is particularly valuable when contributing changes upstream. It can be good to start by adding the lines to the performance test in an early commit, then demonstrating the performance change by copying the necessary lines from the output table into your commit message.

Sparse index: `git reset`

…tests One thing I forgot when talking about the sparse index is that we have a performance test: `t/perf/p2000-sparse-operations.sh`. This test wasn't helpful for commands like `git merge` that need a particular set of input, but work for more read-only operations. Here is a quick demonstration of how this performance test works so we could have a definitive measure of how your previous updates improved performance. To get these results, I ran the following command in `t/perf`: ``` ./run 4bcd533 f9255a5 f28fc01 b713582 -- p2000-sparse-operations.sh ``` The short-shas correspond to the merge commits for these PRs: * microsoft#410 * microsoft#421 * microsoft#417 * microsoft#419 The test takes a copy of the Git repository and creates several copies within a nested directory heirarchy. ``` Test 4bcd533 f9255a5 f28fc01 b713582 ------------------------------------------------------------------------------------------------------------------------------------------------- 2000.2: git status (full-v3) 0.19(0.15+0.05) 0.19(0.16+0.05) +0.0% 0.20(0.18+0.03) +5.3% 0.19(0.17+0.04) +0.0% 2000.3: git status (full-v4) 0.20(0.18+0.04) 0.19(0.15+0.06) -5.0% 0.21(0.18+0.05) +5.0% 0.18(0.18+0.02) -10.0% 2000.4: git status (sparse-v3) 0.04(0.04+0.04) 0.05(0.07+0.04) +25.0% 0.04(0.04+0.05) +0.0% 0.04(0.06+0.04) +0.0% 2000.5: git status (sparse-v4) 0.04(0.03+0.06) 0.04(0.05+0.05) +0.0% 0.05(0.05+0.04) +25.0% 0.05(0.06+0.04) +25.0% 2000.6: git add -A (full-v3) 0.36(0.29+0.05) 0.38(0.28+0.07) +5.6% 0.36(0.31+0.05) +0.0% 0.37(0.31+0.05) +2.8% 2000.7: git add -A (full-v4) 0.34(0.27+0.06) 0.34(0.29+0.05) +0.0% 0.34(0.29+0.04) +0.0% 0.35(0.28+0.06) +2.9% 2000.8: git add -A (sparse-v3) 0.06(0.07+0.04) 0.06(0.05+0.06) +0.0% 0.06(0.09+0.01) +0.0% 0.06(0.08+0.03) +0.0% 2000.9: git add -A (sparse-v4) 0.05(0.05+0.04) 0.05(0.05+0.07) +0.0% 0.05(0.04+0.06) +0.0% 0.06(0.06+0.05) +20.0% 2000.10: git add . (full-v3) 0.38(0.31+0.05) 0.37(0.29+0.06) -2.6% 0.37(0.30+0.07) -2.6% 0.37(0.29+0.06) -2.6% 2000.11: git add . (full-v4) 0.35(0.31+0.04) 0.35(0.29+0.07) +0.0% 0.35(0.29+0.05) +0.0% 0.34(0.29+0.06) -2.9% 2000.12: git add . (sparse-v3) 0.06(0.06+0.05) 0.06(0.05+0.06) +0.0% 0.06(0.07+0.05) +0.0% 0.06(0.09+0.03) +0.0% 2000.13: git add . (sparse-v4) 0.06(0.06+0.06) 0.06(0.07+0.04) +0.0% 0.05(0.06+0.05) -16.7% 0.05(0.05+0.07) -16.7% 2000.14: git commit -a -m A (full-v3) 0.48(0.37+0.08) 0.45(0.36+0.08) -6.2% 0.45(0.35+0.09) -6.2% 0.44(0.36+0.07) -8.3% 2000.15: git commit -a -m A (full-v4) 0.45(0.40+0.06) 0.43(0.34+0.07) -4.4% 0.45(0.37+0.06) +0.0% 0.42(0.36+0.05) -6.7% 2000.16: git commit -a -m A (sparse-v3) 0.05(0.05+0.06) 0.05(0.05+0.03) +0.0% 0.05(0.06+0.06) +0.0% 0.05(0.04+0.06) +0.0% 2000.17: git commit -a -m A (sparse-v4) 0.05(0.06+0.03) 0.05(0.06+0.04) +0.0% 0.06(0.07+0.05) +20.0% 0.05(0.04+0.06) +0.0% 2000.18: git checkout -f - (full-v3) 0.55(0.43+0.08) 0.54(0.46+0.05) -1.8% 0.55(0.46+0.07) +0.0% 0.54(0.40+0.10) -1.8% 2000.19: git checkout -f - (full-v4) 0.55(0.41+0.09) 0.50(0.40+0.09) -9.1% 0.51(0.46+0.05) -7.3% 0.51(0.44+0.06) -7.3% 2000.20: git checkout -f - (sparse-v3) 0.06(0.09+0.03) 0.06(0.08+0.03) +0.0% 0.06(0.06+0.05) +0.0% 0.07(0.09+0.03) +16.7% 2000.21: git checkout -f - (sparse-v4) 0.06(0.08+0.04) 0.05(0.07+0.05) -16.7% 0.05(0.07+0.04) -16.7% 0.06(0.09+0.03) +0.0% ``` All of the above were already integrated. ``` 2000.22: git reset (full-v3) 0.41(0.32+0.06) 0.40(0.31+0.06) -2.4% 0.41(0.33+0.05) +0.0% 0.42(0.34+0.04) +2.4% 2000.23: git reset (full-v4) 0.37(0.32+0.05) 0.35(0.30+0.05) -5.4% 0.37(0.30+0.05) +0.0% 0.35(0.31+0.03) -5.4% 2000.24: git reset (sparse-v3) 0.68(0.65+0.05) 0.55(0.52+0.04) -19.1% 0.04(0.05+0.04) -94.1% 0.04(0.05+0.04) -94.1% 2000.25: git reset (sparse-v4) 0.70(0.65+0.05) 0.54(0.50+0.06) -22.9% 0.04(0.07+0.01) -94.3% 0.03(0.05+0.05) -95.7% 2000.26: git reset --hard (full-v3) 0.54(0.43+0.07) 0.53(0.43+0.06) -1.9% 0.55(0.46+0.05) +1.9% 0.55(0.44+0.06) +1.9% 2000.27: git reset --hard (full-v4) 0.50(0.45+0.03) 0.50(0.43+0.05) +0.0% 0.49(0.41+0.06) -2.0% 0.50(0.42+0.05) +0.0% 2000.28: git reset --hard (sparse-v3) 0.83(0.76+0.06) 0.68(0.62+0.05) -18.1% 0.07(0.05+0.02) -91.6% 0.07(0.05+0.02) -91.6% 2000.29: git reset --hard (sparse-v4) 0.80(0.75+0.05) 0.69(0.62+0.06) -13.8% 0.07(0.04+0.02) -91.2% 0.07(0.04+0.03) -91.2% ``` As expected, `git reset [--hard]` improves with the sparse index integration, but remains constant across the full index case. ``` 2000.30: git update-index --add --remove (full-v3) 0.03(0.01+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.02+0.01) +0.0% 0.03(0.01+0.01) +0.0% 2000.31: git update-index --add --remove (full-v4) 0.03(0.02+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.03+0.00) +0.0% 0.03(0.02+0.01) +0.0% 2000.32: git update-index --add --remove (sparse-v3) 0.57(0.54+0.02) 0.43(0.42+0.00) -24.6% 0.44(0.41+0.03) -22.8% 0.44(0.42+0.01) -22.8% 2000.33: git update-index --add --remove (sparse-v4) 0.56(0.52+0.04) 0.43(0.42+0.01) -23.2% 0.44(0.42+0.02) -21.4% 0.42(0.41+0.01) -25.0% ``` These do not change significantly because microsoft#423 is not merged. ``` 2000.34: git diff (full-v3) 0.07(0.05+0.03) 0.06(0.05+0.03) -14.3% 0.07(0.05+0.03) +0.0% 0.06(0.05+0.03) -14.3% 2000.35: git diff (full-v4) 0.06(0.05+0.03) 0.06(0.05+0.02) +0.0% 0.06(0.05+0.02) +0.0% 0.06(0.06+0.02) +0.0% 2000.36: git diff (sparse-v3) 0.25(0.23+0.03) 0.17(0.17+0.02) -32.0% 0.18(0.18+0.02) -28.0% 0.01(0.03+0.03) -96.0% 2000.37: git diff (sparse-v4) 0.25(0.22+0.05) 0.16(0.16+0.01) -36.0% 0.18(0.15+0.04) -28.0% 0.01(0.04+0.02) -96.0% 2000.38: git diff --staged (full-v3) 0.03(0.01+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.02+0.01) +0.0% 0.03(0.02+0.00) +0.0% 2000.39: git diff --staged (full-v4) 0.04(0.03+0.01) 0.03(0.02+0.01) -25.0% 0.03(0.03+0.00) -25.0% 0.03(0.03+0.00) -25.0% 2000.40: git diff --staged (sparse-v3) 0.21(0.19+0.01) 0.15(0.13+0.01) -28.6% 0.15(0.14+0.01) -28.6% 0.01(0.01+0.00) -95.2% 2000.41: git diff --staged (sparse-v4) 0.22(0.21+0.01) 0.14(0.11+0.03) -36.4% 0.15(0.13+0.02) -31.8% 0.01(0.01+0.00) -95.5% ``` The `git diff` improvements are measurable. ``` 2000.42: git sparse-checkout reapply (full-v3) 0.63(0.54+0.05) 0.56(0.48+0.04) -11.1% 0.57(0.48+0.03) -9.5% 0.59(0.48+0.05) -6.3% 2000.43: git sparse-checkout reapply (full-v4) 0.60(0.54+0.02) 0.51(0.46+0.03) -15.0% 0.54(0.48+0.02) -10.0% 0.50(0.44+0.04) -16.7% 2000.44: git sparse-checkout reapply (sparse-v3) 0.91(0.86+0.05) 0.05(0.05+0.00) -94.5% 0.06(0.05+0.01) -93.4% 0.06(0.06+0.00) -93.4% 2000.45: git sparse-checkout reapply (sparse-v4) 0.92(0.88+0.04) 0.05(0.05+0.00) -94.6% 0.05(0.05+0.01) -94.6% 0.05(0.04+0.01) -94.6% ``` Finally, the `git sparse-checkout` measurements are also present. This test script is particularly valuable when contributing changes upstream. It can be good to start by adding the lines to the performance test in an early commit, then demonstrating the performance change by copying the necessary lines from the output table into your commit message.

Sparse index: `git reset`

…tests One thing I forgot when talking about the sparse index is that we have a performance test: `t/perf/p2000-sparse-operations.sh`. This test wasn't helpful for commands like `git merge` that need a particular set of input, but work for more read-only operations. Here is a quick demonstration of how this performance test works so we could have a definitive measure of how your previous updates improved performance. To get these results, I ran the following command in `t/perf`: ``` ./run 4bcd533 f9255a5 f28fc01 b713582 -- p2000-sparse-operations.sh ``` The short-shas correspond to the merge commits for these PRs: * microsoft#410 * microsoft#421 * microsoft#417 * microsoft#419 The test takes a copy of the Git repository and creates several copies within a nested directory heirarchy. ``` Test 4bcd533 f9255a5 f28fc01 b713582 ------------------------------------------------------------------------------------------------------------------------------------------------- 2000.2: git status (full-v3) 0.19(0.15+0.05) 0.19(0.16+0.05) +0.0% 0.20(0.18+0.03) +5.3% 0.19(0.17+0.04) +0.0% 2000.3: git status (full-v4) 0.20(0.18+0.04) 0.19(0.15+0.06) -5.0% 0.21(0.18+0.05) +5.0% 0.18(0.18+0.02) -10.0% 2000.4: git status (sparse-v3) 0.04(0.04+0.04) 0.05(0.07+0.04) +25.0% 0.04(0.04+0.05) +0.0% 0.04(0.06+0.04) +0.0% 2000.5: git status (sparse-v4) 0.04(0.03+0.06) 0.04(0.05+0.05) +0.0% 0.05(0.05+0.04) +25.0% 0.05(0.06+0.04) +25.0% 2000.6: git add -A (full-v3) 0.36(0.29+0.05) 0.38(0.28+0.07) +5.6% 0.36(0.31+0.05) +0.0% 0.37(0.31+0.05) +2.8% 2000.7: git add -A (full-v4) 0.34(0.27+0.06) 0.34(0.29+0.05) +0.0% 0.34(0.29+0.04) +0.0% 0.35(0.28+0.06) +2.9% 2000.8: git add -A (sparse-v3) 0.06(0.07+0.04) 0.06(0.05+0.06) +0.0% 0.06(0.09+0.01) +0.0% 0.06(0.08+0.03) +0.0% 2000.9: git add -A (sparse-v4) 0.05(0.05+0.04) 0.05(0.05+0.07) +0.0% 0.05(0.04+0.06) +0.0% 0.06(0.06+0.05) +20.0% 2000.10: git add . (full-v3) 0.38(0.31+0.05) 0.37(0.29+0.06) -2.6% 0.37(0.30+0.07) -2.6% 0.37(0.29+0.06) -2.6% 2000.11: git add . (full-v4) 0.35(0.31+0.04) 0.35(0.29+0.07) +0.0% 0.35(0.29+0.05) +0.0% 0.34(0.29+0.06) -2.9% 2000.12: git add . (sparse-v3) 0.06(0.06+0.05) 0.06(0.05+0.06) +0.0% 0.06(0.07+0.05) +0.0% 0.06(0.09+0.03) +0.0% 2000.13: git add . (sparse-v4) 0.06(0.06+0.06) 0.06(0.07+0.04) +0.0% 0.05(0.06+0.05) -16.7% 0.05(0.05+0.07) -16.7% 2000.14: git commit -a -m A (full-v3) 0.48(0.37+0.08) 0.45(0.36+0.08) -6.2% 0.45(0.35+0.09) -6.2% 0.44(0.36+0.07) -8.3% 2000.15: git commit -a -m A (full-v4) 0.45(0.40+0.06) 0.43(0.34+0.07) -4.4% 0.45(0.37+0.06) +0.0% 0.42(0.36+0.05) -6.7% 2000.16: git commit -a -m A (sparse-v3) 0.05(0.05+0.06) 0.05(0.05+0.03) +0.0% 0.05(0.06+0.06) +0.0% 0.05(0.04+0.06) +0.0% 2000.17: git commit -a -m A (sparse-v4) 0.05(0.06+0.03) 0.05(0.06+0.04) +0.0% 0.06(0.07+0.05) +20.0% 0.05(0.04+0.06) +0.0% 2000.18: git checkout -f - (full-v3) 0.55(0.43+0.08) 0.54(0.46+0.05) -1.8% 0.55(0.46+0.07) +0.0% 0.54(0.40+0.10) -1.8% 2000.19: git checkout -f - (full-v4) 0.55(0.41+0.09) 0.50(0.40+0.09) -9.1% 0.51(0.46+0.05) -7.3% 0.51(0.44+0.06) -7.3% 2000.20: git checkout -f - (sparse-v3) 0.06(0.09+0.03) 0.06(0.08+0.03) +0.0% 0.06(0.06+0.05) +0.0% 0.07(0.09+0.03) +16.7% 2000.21: git checkout -f - (sparse-v4) 0.06(0.08+0.04) 0.05(0.07+0.05) -16.7% 0.05(0.07+0.04) -16.7% 0.06(0.09+0.03) +0.0% ``` All of the above were already integrated. ``` 2000.22: git reset (full-v3) 0.41(0.32+0.06) 0.40(0.31+0.06) -2.4% 0.41(0.33+0.05) +0.0% 0.42(0.34+0.04) +2.4% 2000.23: git reset (full-v4) 0.37(0.32+0.05) 0.35(0.30+0.05) -5.4% 0.37(0.30+0.05) +0.0% 0.35(0.31+0.03) -5.4% 2000.24: git reset (sparse-v3) 0.68(0.65+0.05) 0.55(0.52+0.04) -19.1% 0.04(0.05+0.04) -94.1% 0.04(0.05+0.04) -94.1% 2000.25: git reset (sparse-v4) 0.70(0.65+0.05) 0.54(0.50+0.06) -22.9% 0.04(0.07+0.01) -94.3% 0.03(0.05+0.05) -95.7% 2000.26: git reset --hard (full-v3) 0.54(0.43+0.07) 0.53(0.43+0.06) -1.9% 0.55(0.46+0.05) +1.9% 0.55(0.44+0.06) +1.9% 2000.27: git reset --hard (full-v4) 0.50(0.45+0.03) 0.50(0.43+0.05) +0.0% 0.49(0.41+0.06) -2.0% 0.50(0.42+0.05) +0.0% 2000.28: git reset --hard (sparse-v3) 0.83(0.76+0.06) 0.68(0.62+0.05) -18.1% 0.07(0.05+0.02) -91.6% 0.07(0.05+0.02) -91.6% 2000.29: git reset --hard (sparse-v4) 0.80(0.75+0.05) 0.69(0.62+0.06) -13.8% 0.07(0.04+0.02) -91.2% 0.07(0.04+0.03) -91.2% ``` As expected, `git reset [--hard]` improves with the sparse index integration, but remains constant across the full index case. ``` 2000.30: git update-index --add --remove (full-v3) 0.03(0.01+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.02+0.01) +0.0% 0.03(0.01+0.01) +0.0% 2000.31: git update-index --add --remove (full-v4) 0.03(0.02+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.03+0.00) +0.0% 0.03(0.02+0.01) +0.0% 2000.32: git update-index --add --remove (sparse-v3) 0.57(0.54+0.02) 0.43(0.42+0.00) -24.6% 0.44(0.41+0.03) -22.8% 0.44(0.42+0.01) -22.8% 2000.33: git update-index --add --remove (sparse-v4) 0.56(0.52+0.04) 0.43(0.42+0.01) -23.2% 0.44(0.42+0.02) -21.4% 0.42(0.41+0.01) -25.0% ``` These do not change significantly because microsoft#423 is not merged. ``` 2000.34: git diff (full-v3) 0.07(0.05+0.03) 0.06(0.05+0.03) -14.3% 0.07(0.05+0.03) +0.0% 0.06(0.05+0.03) -14.3% 2000.35: git diff (full-v4) 0.06(0.05+0.03) 0.06(0.05+0.02) +0.0% 0.06(0.05+0.02) +0.0% 0.06(0.06+0.02) +0.0% 2000.36: git diff (sparse-v3) 0.25(0.23+0.03) 0.17(0.17+0.02) -32.0% 0.18(0.18+0.02) -28.0% 0.01(0.03+0.03) -96.0% 2000.37: git diff (sparse-v4) 0.25(0.22+0.05) 0.16(0.16+0.01) -36.0% 0.18(0.15+0.04) -28.0% 0.01(0.04+0.02) -96.0% 2000.38: git diff --staged (full-v3) 0.03(0.01+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.02+0.01) +0.0% 0.03(0.02+0.00) +0.0% 2000.39: git diff --staged (full-v4) 0.04(0.03+0.01) 0.03(0.02+0.01) -25.0% 0.03(0.03+0.00) -25.0% 0.03(0.03+0.00) -25.0% 2000.40: git diff --staged (sparse-v3) 0.21(0.19+0.01) 0.15(0.13+0.01) -28.6% 0.15(0.14+0.01) -28.6% 0.01(0.01+0.00) -95.2% 2000.41: git diff --staged (sparse-v4) 0.22(0.21+0.01) 0.14(0.11+0.03) -36.4% 0.15(0.13+0.02) -31.8% 0.01(0.01+0.00) -95.5% ``` The `git diff` improvements are measurable. ``` 2000.42: git sparse-checkout reapply (full-v3) 0.63(0.54+0.05) 0.56(0.48+0.04) -11.1% 0.57(0.48+0.03) -9.5% 0.59(0.48+0.05) -6.3% 2000.43: git sparse-checkout reapply (full-v4) 0.60(0.54+0.02) 0.51(0.46+0.03) -15.0% 0.54(0.48+0.02) -10.0% 0.50(0.44+0.04) -16.7% 2000.44: git sparse-checkout reapply (sparse-v3) 0.91(0.86+0.05) 0.05(0.05+0.00) -94.5% 0.06(0.05+0.01) -93.4% 0.06(0.06+0.00) -93.4% 2000.45: git sparse-checkout reapply (sparse-v4) 0.92(0.88+0.04) 0.05(0.05+0.00) -94.6% 0.05(0.05+0.01) -94.6% 0.05(0.04+0.01) -94.6% ``` Finally, the `git sparse-checkout` measurements are also present. This test script is particularly valuable when contributing changes upstream. It can be good to start by adding the lines to the performance test in an early commit, then demonstrating the performance change by copying the necessary lines from the output table into your commit message.

Sparse index: `git reset`

…tests One thing I forgot when talking about the sparse index is that we have a performance test: `t/perf/p2000-sparse-operations.sh`. This test wasn't helpful for commands like `git merge` that need a particular set of input, but work for more read-only operations. Here is a quick demonstration of how this performance test works so we could have a definitive measure of how your previous updates improved performance. To get these results, I ran the following command in `t/perf`: ``` ./run 4bcd533 f9255a5 f28fc01 b713582 -- p2000-sparse-operations.sh ``` The short-shas correspond to the merge commits for these PRs: * microsoft#410 * microsoft#421 * microsoft#417 * microsoft#419 The test takes a copy of the Git repository and creates several copies within a nested directory heirarchy. ``` Test 4bcd533 f9255a5 f28fc01 b713582 ------------------------------------------------------------------------------------------------------------------------------------------------- 2000.2: git status (full-v3) 0.19(0.15+0.05) 0.19(0.16+0.05) +0.0% 0.20(0.18+0.03) +5.3% 0.19(0.17+0.04) +0.0% 2000.3: git status (full-v4) 0.20(0.18+0.04) 0.19(0.15+0.06) -5.0% 0.21(0.18+0.05) +5.0% 0.18(0.18+0.02) -10.0% 2000.4: git status (sparse-v3) 0.04(0.04+0.04) 0.05(0.07+0.04) +25.0% 0.04(0.04+0.05) +0.0% 0.04(0.06+0.04) +0.0% 2000.5: git status (sparse-v4) 0.04(0.03+0.06) 0.04(0.05+0.05) +0.0% 0.05(0.05+0.04) +25.0% 0.05(0.06+0.04) +25.0% 2000.6: git add -A (full-v3) 0.36(0.29+0.05) 0.38(0.28+0.07) +5.6% 0.36(0.31+0.05) +0.0% 0.37(0.31+0.05) +2.8% 2000.7: git add -A (full-v4) 0.34(0.27+0.06) 0.34(0.29+0.05) +0.0% 0.34(0.29+0.04) +0.0% 0.35(0.28+0.06) +2.9% 2000.8: git add -A (sparse-v3) 0.06(0.07+0.04) 0.06(0.05+0.06) +0.0% 0.06(0.09+0.01) +0.0% 0.06(0.08+0.03) +0.0% 2000.9: git add -A (sparse-v4) 0.05(0.05+0.04) 0.05(0.05+0.07) +0.0% 0.05(0.04+0.06) +0.0% 0.06(0.06+0.05) +20.0% 2000.10: git add . (full-v3) 0.38(0.31+0.05) 0.37(0.29+0.06) -2.6% 0.37(0.30+0.07) -2.6% 0.37(0.29+0.06) -2.6% 2000.11: git add . (full-v4) 0.35(0.31+0.04) 0.35(0.29+0.07) +0.0% 0.35(0.29+0.05) +0.0% 0.34(0.29+0.06) -2.9% 2000.12: git add . (sparse-v3) 0.06(0.06+0.05) 0.06(0.05+0.06) +0.0% 0.06(0.07+0.05) +0.0% 0.06(0.09+0.03) +0.0% 2000.13: git add . (sparse-v4) 0.06(0.06+0.06) 0.06(0.07+0.04) +0.0% 0.05(0.06+0.05) -16.7% 0.05(0.05+0.07) -16.7% 2000.14: git commit -a -m A (full-v3) 0.48(0.37+0.08) 0.45(0.36+0.08) -6.2% 0.45(0.35+0.09) -6.2% 0.44(0.36+0.07) -8.3% 2000.15: git commit -a -m A (full-v4) 0.45(0.40+0.06) 0.43(0.34+0.07) -4.4% 0.45(0.37+0.06) +0.0% 0.42(0.36+0.05) -6.7% 2000.16: git commit -a -m A (sparse-v3) 0.05(0.05+0.06) 0.05(0.05+0.03) +0.0% 0.05(0.06+0.06) +0.0% 0.05(0.04+0.06) +0.0% 2000.17: git commit -a -m A (sparse-v4) 0.05(0.06+0.03) 0.05(0.06+0.04) +0.0% 0.06(0.07+0.05) +20.0% 0.05(0.04+0.06) +0.0% 2000.18: git checkout -f - (full-v3) 0.55(0.43+0.08) 0.54(0.46+0.05) -1.8% 0.55(0.46+0.07) +0.0% 0.54(0.40+0.10) -1.8% 2000.19: git checkout -f - (full-v4) 0.55(0.41+0.09) 0.50(0.40+0.09) -9.1% 0.51(0.46+0.05) -7.3% 0.51(0.44+0.06) -7.3% 2000.20: git checkout -f - (sparse-v3) 0.06(0.09+0.03) 0.06(0.08+0.03) +0.0% 0.06(0.06+0.05) +0.0% 0.07(0.09+0.03) +16.7% 2000.21: git checkout -f - (sparse-v4) 0.06(0.08+0.04) 0.05(0.07+0.05) -16.7% 0.05(0.07+0.04) -16.7% 0.06(0.09+0.03) +0.0% ``` All of the above were already integrated. ``` 2000.22: git reset (full-v3) 0.41(0.32+0.06) 0.40(0.31+0.06) -2.4% 0.41(0.33+0.05) +0.0% 0.42(0.34+0.04) +2.4% 2000.23: git reset (full-v4) 0.37(0.32+0.05) 0.35(0.30+0.05) -5.4% 0.37(0.30+0.05) +0.0% 0.35(0.31+0.03) -5.4% 2000.24: git reset (sparse-v3) 0.68(0.65+0.05) 0.55(0.52+0.04) -19.1% 0.04(0.05+0.04) -94.1% 0.04(0.05+0.04) -94.1% 2000.25: git reset (sparse-v4) 0.70(0.65+0.05) 0.54(0.50+0.06) -22.9% 0.04(0.07+0.01) -94.3% 0.03(0.05+0.05) -95.7% 2000.26: git reset --hard (full-v3) 0.54(0.43+0.07) 0.53(0.43+0.06) -1.9% 0.55(0.46+0.05) +1.9% 0.55(0.44+0.06) +1.9% 2000.27: git reset --hard (full-v4) 0.50(0.45+0.03) 0.50(0.43+0.05) +0.0% 0.49(0.41+0.06) -2.0% 0.50(0.42+0.05) +0.0% 2000.28: git reset --hard (sparse-v3) 0.83(0.76+0.06) 0.68(0.62+0.05) -18.1% 0.07(0.05+0.02) -91.6% 0.07(0.05+0.02) -91.6% 2000.29: git reset --hard (sparse-v4) 0.80(0.75+0.05) 0.69(0.62+0.06) -13.8% 0.07(0.04+0.02) -91.2% 0.07(0.04+0.03) -91.2% ``` As expected, `git reset [--hard]` improves with the sparse index integration, but remains constant across the full index case. ``` 2000.30: git update-index --add --remove (full-v3) 0.03(0.01+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.02+0.01) +0.0% 0.03(0.01+0.01) +0.0% 2000.31: git update-index --add --remove (full-v4) 0.03(0.02+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.03+0.00) +0.0% 0.03(0.02+0.01) +0.0% 2000.32: git update-index --add --remove (sparse-v3) 0.57(0.54+0.02) 0.43(0.42+0.00) -24.6% 0.44(0.41+0.03) -22.8% 0.44(0.42+0.01) -22.8% 2000.33: git update-index --add --remove (sparse-v4) 0.56(0.52+0.04) 0.43(0.42+0.01) -23.2% 0.44(0.42+0.02) -21.4% 0.42(0.41+0.01) -25.0% ``` These do not change significantly because microsoft#423 is not merged. ``` 2000.34: git diff (full-v3) 0.07(0.05+0.03) 0.06(0.05+0.03) -14.3% 0.07(0.05+0.03) +0.0% 0.06(0.05+0.03) -14.3% 2000.35: git diff (full-v4) 0.06(0.05+0.03) 0.06(0.05+0.02) +0.0% 0.06(0.05+0.02) +0.0% 0.06(0.06+0.02) +0.0% 2000.36: git diff (sparse-v3) 0.25(0.23+0.03) 0.17(0.17+0.02) -32.0% 0.18(0.18+0.02) -28.0% 0.01(0.03+0.03) -96.0% 2000.37: git diff (sparse-v4) 0.25(0.22+0.05) 0.16(0.16+0.01) -36.0% 0.18(0.15+0.04) -28.0% 0.01(0.04+0.02) -96.0% 2000.38: git diff --staged (full-v3) 0.03(0.01+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.02+0.01) +0.0% 0.03(0.02+0.00) +0.0% 2000.39: git diff --staged (full-v4) 0.04(0.03+0.01) 0.03(0.02+0.01) -25.0% 0.03(0.03+0.00) -25.0% 0.03(0.03+0.00) -25.0% 2000.40: git diff --staged (sparse-v3) 0.21(0.19+0.01) 0.15(0.13+0.01) -28.6% 0.15(0.14+0.01) -28.6% 0.01(0.01+0.00) -95.2% 2000.41: git diff --staged (sparse-v4) 0.22(0.21+0.01) 0.14(0.11+0.03) -36.4% 0.15(0.13+0.02) -31.8% 0.01(0.01+0.00) -95.5% ``` The `git diff` improvements are measurable. ``` 2000.42: git sparse-checkout reapply (full-v3) 0.63(0.54+0.05) 0.56(0.48+0.04) -11.1% 0.57(0.48+0.03) -9.5% 0.59(0.48+0.05) -6.3% 2000.43: git sparse-checkout reapply (full-v4) 0.60(0.54+0.02) 0.51(0.46+0.03) -15.0% 0.54(0.48+0.02) -10.0% 0.50(0.44+0.04) -16.7% 2000.44: git sparse-checkout reapply (sparse-v3) 0.91(0.86+0.05) 0.05(0.05+0.00) -94.5% 0.06(0.05+0.01) -93.4% 0.06(0.06+0.00) -93.4% 2000.45: git sparse-checkout reapply (sparse-v4) 0.92(0.88+0.04) 0.05(0.05+0.00) -94.6% 0.05(0.05+0.01) -94.6% 0.05(0.04+0.01) -94.6% ``` Finally, the `git sparse-checkout` measurements are also present. This test script is particularly valuable when contributing changes upstream. It can be good to start by adding the lines to the performance test in an early commit, then demonstrating the performance change by copying the necessary lines from the output table into your commit message.

Sparse index: `git reset`

…tests One thing I forgot when talking about the sparse index is that we have a performance test: `t/perf/p2000-sparse-operations.sh`. This test wasn't helpful for commands like `git merge` that need a particular set of input, but work for more read-only operations. Here is a quick demonstration of how this performance test works so we could have a definitive measure of how your previous updates improved performance. To get these results, I ran the following command in `t/perf`: ``` ./run 4bcd533 f9255a5 f28fc01 b713582 -- p2000-sparse-operations.sh ``` The short-shas correspond to the merge commits for these PRs: * microsoft#410 * microsoft#421 * microsoft#417 * microsoft#419 The test takes a copy of the Git repository and creates several copies within a nested directory heirarchy. ``` Test 4bcd533 f9255a5 f28fc01 b713582 ------------------------------------------------------------------------------------------------------------------------------------------------- 2000.2: git status (full-v3) 0.19(0.15+0.05) 0.19(0.16+0.05) +0.0% 0.20(0.18+0.03) +5.3% 0.19(0.17+0.04) +0.0% 2000.3: git status (full-v4) 0.20(0.18+0.04) 0.19(0.15+0.06) -5.0% 0.21(0.18+0.05) +5.0% 0.18(0.18+0.02) -10.0% 2000.4: git status (sparse-v3) 0.04(0.04+0.04) 0.05(0.07+0.04) +25.0% 0.04(0.04+0.05) +0.0% 0.04(0.06+0.04) +0.0% 2000.5: git status (sparse-v4) 0.04(0.03+0.06) 0.04(0.05+0.05) +0.0% 0.05(0.05+0.04) +25.0% 0.05(0.06+0.04) +25.0% 2000.6: git add -A (full-v3) 0.36(0.29+0.05) 0.38(0.28+0.07) +5.6% 0.36(0.31+0.05) +0.0% 0.37(0.31+0.05) +2.8% 2000.7: git add -A (full-v4) 0.34(0.27+0.06) 0.34(0.29+0.05) +0.0% 0.34(0.29+0.04) +0.0% 0.35(0.28+0.06) +2.9% 2000.8: git add -A (sparse-v3) 0.06(0.07+0.04) 0.06(0.05+0.06) +0.0% 0.06(0.09+0.01) +0.0% 0.06(0.08+0.03) +0.0% 2000.9: git add -A (sparse-v4) 0.05(0.05+0.04) 0.05(0.05+0.07) +0.0% 0.05(0.04+0.06) +0.0% 0.06(0.06+0.05) +20.0% 2000.10: git add . (full-v3) 0.38(0.31+0.05) 0.37(0.29+0.06) -2.6% 0.37(0.30+0.07) -2.6% 0.37(0.29+0.06) -2.6% 2000.11: git add . (full-v4) 0.35(0.31+0.04) 0.35(0.29+0.07) +0.0% 0.35(0.29+0.05) +0.0% 0.34(0.29+0.06) -2.9% 2000.12: git add . (sparse-v3) 0.06(0.06+0.05) 0.06(0.05+0.06) +0.0% 0.06(0.07+0.05) +0.0% 0.06(0.09+0.03) +0.0% 2000.13: git add . (sparse-v4) 0.06(0.06+0.06) 0.06(0.07+0.04) +0.0% 0.05(0.06+0.05) -16.7% 0.05(0.05+0.07) -16.7% 2000.14: git commit -a -m A (full-v3) 0.48(0.37+0.08) 0.45(0.36+0.08) -6.2% 0.45(0.35+0.09) -6.2% 0.44(0.36+0.07) -8.3% 2000.15: git commit -a -m A (full-v4) 0.45(0.40+0.06) 0.43(0.34+0.07) -4.4% 0.45(0.37+0.06) +0.0% 0.42(0.36+0.05) -6.7% 2000.16: git commit -a -m A (sparse-v3) 0.05(0.05+0.06) 0.05(0.05+0.03) +0.0% 0.05(0.06+0.06) +0.0% 0.05(0.04+0.06) +0.0% 2000.17: git commit -a -m A (sparse-v4) 0.05(0.06+0.03) 0.05(0.06+0.04) +0.0% 0.06(0.07+0.05) +20.0% 0.05(0.04+0.06) +0.0% 2000.18: git checkout -f - (full-v3) 0.55(0.43+0.08) 0.54(0.46+0.05) -1.8% 0.55(0.46+0.07) +0.0% 0.54(0.40+0.10) -1.8% 2000.19: git checkout -f - (full-v4) 0.55(0.41+0.09) 0.50(0.40+0.09) -9.1% 0.51(0.46+0.05) -7.3% 0.51(0.44+0.06) -7.3% 2000.20: git checkout -f - (sparse-v3) 0.06(0.09+0.03) 0.06(0.08+0.03) +0.0% 0.06(0.06+0.05) +0.0% 0.07(0.09+0.03) +16.7% 2000.21: git checkout -f - (sparse-v4) 0.06(0.08+0.04) 0.05(0.07+0.05) -16.7% 0.05(0.07+0.04) -16.7% 0.06(0.09+0.03) +0.0% ``` All of the above were already integrated. ``` 2000.22: git reset (full-v3) 0.41(0.32+0.06) 0.40(0.31+0.06) -2.4% 0.41(0.33+0.05) +0.0% 0.42(0.34+0.04) +2.4% 2000.23: git reset (full-v4) 0.37(0.32+0.05) 0.35(0.30+0.05) -5.4% 0.37(0.30+0.05) +0.0% 0.35(0.31+0.03) -5.4% 2000.24: git reset (sparse-v3) 0.68(0.65+0.05) 0.55(0.52+0.04) -19.1% 0.04(0.05+0.04) -94.1% 0.04(0.05+0.04) -94.1% 2000.25: git reset (sparse-v4) 0.70(0.65+0.05) 0.54(0.50+0.06) -22.9% 0.04(0.07+0.01) -94.3% 0.03(0.05+0.05) -95.7% 2000.26: git reset --hard (full-v3) 0.54(0.43+0.07) 0.53(0.43+0.06) -1.9% 0.55(0.46+0.05) +1.9% 0.55(0.44+0.06) +1.9% 2000.27: git reset --hard (full-v4) 0.50(0.45+0.03) 0.50(0.43+0.05) +0.0% 0.49(0.41+0.06) -2.0% 0.50(0.42+0.05) +0.0% 2000.28: git reset --hard (sparse-v3) 0.83(0.76+0.06) 0.68(0.62+0.05) -18.1% 0.07(0.05+0.02) -91.6% 0.07(0.05+0.02) -91.6% 2000.29: git reset --hard (sparse-v4) 0.80(0.75+0.05) 0.69(0.62+0.06) -13.8% 0.07(0.04+0.02) -91.2% 0.07(0.04+0.03) -91.2% ``` As expected, `git reset [--hard]` improves with the sparse index integration, but remains constant across the full index case. ``` 2000.30: git update-index --add --remove (full-v3) 0.03(0.01+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.02+0.01) +0.0% 0.03(0.01+0.01) +0.0% 2000.31: git update-index --add --remove (full-v4) 0.03(0.02+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.03+0.00) +0.0% 0.03(0.02+0.01) +0.0% 2000.32: git update-index --add --remove (sparse-v3) 0.57(0.54+0.02) 0.43(0.42+0.00) -24.6% 0.44(0.41+0.03) -22.8% 0.44(0.42+0.01) -22.8% 2000.33: git update-index --add --remove (sparse-v4) 0.56(0.52+0.04) 0.43(0.42+0.01) -23.2% 0.44(0.42+0.02) -21.4% 0.42(0.41+0.01) -25.0% ``` These do not change significantly because microsoft#423 is not merged. ``` 2000.34: git diff (full-v3) 0.07(0.05+0.03) 0.06(0.05+0.03) -14.3% 0.07(0.05+0.03) +0.0% 0.06(0.05+0.03) -14.3% 2000.35: git diff (full-v4) 0.06(0.05+0.03) 0.06(0.05+0.02) +0.0% 0.06(0.05+0.02) +0.0% 0.06(0.06+0.02) +0.0% 2000.36: git diff (sparse-v3) 0.25(0.23+0.03) 0.17(0.17+0.02) -32.0% 0.18(0.18+0.02) -28.0% 0.01(0.03+0.03) -96.0% 2000.37: git diff (sparse-v4) 0.25(0.22+0.05) 0.16(0.16+0.01) -36.0% 0.18(0.15+0.04) -28.0% 0.01(0.04+0.02) -96.0% 2000.38: git diff --staged (full-v3) 0.03(0.01+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.02+0.01) +0.0% 0.03(0.02+0.00) +0.0% 2000.39: git diff --staged (full-v4) 0.04(0.03+0.01) 0.03(0.02+0.01) -25.0% 0.03(0.03+0.00) -25.0% 0.03(0.03+0.00) -25.0% 2000.40: git diff --staged (sparse-v3) 0.21(0.19+0.01) 0.15(0.13+0.01) -28.6% 0.15(0.14+0.01) -28.6% 0.01(0.01+0.00) -95.2% 2000.41: git diff --staged (sparse-v4) 0.22(0.21+0.01) 0.14(0.11+0.03) -36.4% 0.15(0.13+0.02) -31.8% 0.01(0.01+0.00) -95.5% ``` The `git diff` improvements are measurable. ``` 2000.42: git sparse-checkout reapply (full-v3) 0.63(0.54+0.05) 0.56(0.48+0.04) -11.1% 0.57(0.48+0.03) -9.5% 0.59(0.48+0.05) -6.3% 2000.43: git sparse-checkout reapply (full-v4) 0.60(0.54+0.02) 0.51(0.46+0.03) -15.0% 0.54(0.48+0.02) -10.0% 0.50(0.44+0.04) -16.7% 2000.44: git sparse-checkout reapply (sparse-v3) 0.91(0.86+0.05) 0.05(0.05+0.00) -94.5% 0.06(0.05+0.01) -93.4% 0.06(0.06+0.00) -93.4% 2000.45: git sparse-checkout reapply (sparse-v4) 0.92(0.88+0.04) 0.05(0.05+0.00) -94.6% 0.05(0.05+0.01) -94.6% 0.05(0.04+0.01) -94.6% ``` Finally, the `git sparse-checkout` measurements are also present. This test script is particularly valuable when contributing changes upstream. It can be good to start by adding the lines to the performance test in an early commit, then demonstrating the performance change by copying the necessary lines from the output table into your commit message.

One thing I forgot when talking about the sparse index is that we have a performance test: `t/perf/p2000-sparse-operations.sh`. This test wasn't helpful for commands like `git merge` that need a particular set of input, but work for more read-only operations. Here is a quick demonstration of how this performance test works so we could have a definitive measure of how your previous updates improved performance. To get these results, I ran the following command in `t/perf`: ``` ./run 4bcd533 f9255a5 f28fc01 b713582 -- p2000-sparse-operations.sh ``` The short-shas correspond to the merge commits for these PRs: * #410 * #421 * #417 * #419 The test takes a copy of the Git repository and creates several copies within a nested directory heirarchy. ``` Test 4bcd533 f9255a5 f28fc01 b713582 ------------------------------------------------------------------------------------------------------------------------------------------------- 2000.2: git status (full-v3) 0.19(0.15+0.05) 0.19(0.16+0.05) +0.0% 0.20(0.18+0.03) +5.3% 0.19(0.17+0.04) +0.0% 2000.3: git status (full-v4) 0.20(0.18+0.04) 0.19(0.15+0.06) -5.0% 0.21(0.18+0.05) +5.0% 0.18(0.18+0.02) -10.0% 2000.4: git status (sparse-v3) 0.04(0.04+0.04) 0.05(0.07+0.04) +25.0% 0.04(0.04+0.05) +0.0% 0.04(0.06+0.04) +0.0% 2000.5: git status (sparse-v4) 0.04(0.03+0.06) 0.04(0.05+0.05) +0.0% 0.05(0.05+0.04) +25.0% 0.05(0.06+0.04) +25.0% 2000.6: git add -A (full-v3) 0.36(0.29+0.05) 0.38(0.28+0.07) +5.6% 0.36(0.31+0.05) +0.0% 0.37(0.31+0.05) +2.8% 2000.7: git add -A (full-v4) 0.34(0.27+0.06) 0.34(0.29+0.05) +0.0% 0.34(0.29+0.04) +0.0% 0.35(0.28+0.06) +2.9% 2000.8: git add -A (sparse-v3) 0.06(0.07+0.04) 0.06(0.05+0.06) +0.0% 0.06(0.09+0.01) +0.0% 0.06(0.08+0.03) +0.0% 2000.9: git add -A (sparse-v4) 0.05(0.05+0.04) 0.05(0.05+0.07) +0.0% 0.05(0.04+0.06) +0.0% 0.06(0.06+0.05) +20.0% 2000.10: git add . (full-v3) 0.38(0.31+0.05) 0.37(0.29+0.06) -2.6% 0.37(0.30+0.07) -2.6% 0.37(0.29+0.06) -2.6% 2000.11: git add . (full-v4) 0.35(0.31+0.04) 0.35(0.29+0.07) +0.0% 0.35(0.29+0.05) +0.0% 0.34(0.29+0.06) -2.9% 2000.12: git add . (sparse-v3) 0.06(0.06+0.05) 0.06(0.05+0.06) +0.0% 0.06(0.07+0.05) +0.0% 0.06(0.09+0.03) +0.0% 2000.13: git add . (sparse-v4) 0.06(0.06+0.06) 0.06(0.07+0.04) +0.0% 0.05(0.06+0.05) -16.7% 0.05(0.05+0.07) -16.7% 2000.14: git commit -a -m A (full-v3) 0.48(0.37+0.08) 0.45(0.36+0.08) -6.2% 0.45(0.35+0.09) -6.2% 0.44(0.36+0.07) -8.3% 2000.15: git commit -a -m A (full-v4) 0.45(0.40+0.06) 0.43(0.34+0.07) -4.4% 0.45(0.37+0.06) +0.0% 0.42(0.36+0.05) -6.7% 2000.16: git commit -a -m A (sparse-v3) 0.05(0.05+0.06) 0.05(0.05+0.03) +0.0% 0.05(0.06+0.06) +0.0% 0.05(0.04+0.06) +0.0% 2000.17: git commit -a -m A (sparse-v4) 0.05(0.06+0.03) 0.05(0.06+0.04) +0.0% 0.06(0.07+0.05) +20.0% 0.05(0.04+0.06) +0.0% 2000.18: git checkout -f - (full-v3) 0.55(0.43+0.08) 0.54(0.46+0.05) -1.8% 0.55(0.46+0.07) +0.0% 0.54(0.40+0.10) -1.8% 2000.19: git checkout -f - (full-v4) 0.55(0.41+0.09) 0.50(0.40+0.09) -9.1% 0.51(0.46+0.05) -7.3% 0.51(0.44+0.06) -7.3% 2000.20: git checkout -f - (sparse-v3) 0.06(0.09+0.03) 0.06(0.08+0.03) +0.0% 0.06(0.06+0.05) +0.0% 0.07(0.09+0.03) +16.7% 2000.21: git checkout -f - (sparse-v4) 0.06(0.08+0.04) 0.05(0.07+0.05) -16.7% 0.05(0.07+0.04) -16.7% 0.06(0.09+0.03) +0.0% ``` All of the above were already integrated. ``` 2000.22: git reset (full-v3) 0.41(0.32+0.06) 0.40(0.31+0.06) -2.4% 0.41(0.33+0.05) +0.0% 0.42(0.34+0.04) +2.4% 2000.23: git reset (full-v4) 0.37(0.32+0.05) 0.35(0.30+0.05) -5.4% 0.37(0.30+0.05) +0.0% 0.35(0.31+0.03) -5.4% 2000.24: git reset (sparse-v3) 0.68(0.65+0.05) 0.55(0.52+0.04) -19.1% 0.04(0.05+0.04) -94.1% 0.04(0.05+0.04) -94.1% 2000.25: git reset (sparse-v4) 0.70(0.65+0.05) 0.54(0.50+0.06) -22.9% 0.04(0.07+0.01) -94.3% 0.03(0.05+0.05) -95.7% 2000.26: git reset --hard (full-v3) 0.54(0.43+0.07) 0.53(0.43+0.06) -1.9% 0.55(0.46+0.05) +1.9% 0.55(0.44+0.06) +1.9% 2000.27: git reset --hard (full-v4) 0.50(0.45+0.03) 0.50(0.43+0.05) +0.0% 0.49(0.41+0.06) -2.0% 0.50(0.42+0.05) +0.0% 2000.28: git reset --hard (sparse-v3) 0.83(0.76+0.06) 0.68(0.62+0.05) -18.1% 0.07(0.05+0.02) -91.6% 0.07(0.05+0.02) -91.6% 2000.29: git reset --hard (sparse-v4) 0.80(0.75+0.05) 0.69(0.62+0.06) -13.8% 0.07(0.04+0.02) -91.2% 0.07(0.04+0.03) -91.2% ``` As expected, `git reset [--hard]` improves with the sparse index integration, but remains constant across the full index case. ``` 2000.30: git update-index --add --remove (full-v3) 0.03(0.01+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.02+0.01) +0.0% 0.03(0.01+0.01) +0.0% 2000.31: git update-index --add --remove (full-v4) 0.03(0.02+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.03+0.00) +0.0% 0.03(0.02+0.01) +0.0% 2000.32: git update-index --add --remove (sparse-v3) 0.57(0.54+0.02) 0.43(0.42+0.00) -24.6% 0.44(0.41+0.03) -22.8% 0.44(0.42+0.01) -22.8% 2000.33: git update-index --add --remove (sparse-v4) 0.56(0.52+0.04) 0.43(0.42+0.01) -23.2% 0.44(0.42+0.02) -21.4% 0.42(0.41+0.01) -25.0% ``` These do not change significantly because #423 is not merged. ``` 2000.34: git diff (full-v3) 0.07(0.05+0.03) 0.06(0.05+0.03) -14.3% 0.07(0.05+0.03) +0.0% 0.06(0.05+0.03) -14.3% 2000.35: git diff (full-v4) 0.06(0.05+0.03) 0.06(0.05+0.02) +0.0% 0.06(0.05+0.02) +0.0% 0.06(0.06+0.02) +0.0% 2000.36: git diff (sparse-v3) 0.25(0.23+0.03) 0.17(0.17+0.02) -32.0% 0.18(0.18+0.02) -28.0% 0.01(0.03+0.03) -96.0% 2000.37: git diff (sparse-v4) 0.25(0.22+0.05) 0.16(0.16+0.01) -36.0% 0.18(0.15+0.04) -28.0% 0.01(0.04+0.02) -96.0% 2000.38: git diff --staged (full-v3) 0.03(0.01+0.01) 0.03(0.02+0.01) +0.0% 0.03(0.02+0.01) +0.0% 0.03(0.02+0.00) +0.0% 2000.39: git diff --staged (full-v4) 0.04(0.03+0.01) 0.03(0.02+0.01) -25.0% 0.03(0.03+0.00) -25.0% 0.03(0.03+0.00) -25.0% 2000.40: git diff --staged (sparse-v3) 0.21(0.19+0.01) 0.15(0.13+0.01) -28.6% 0.15(0.14+0.01) -28.6% 0.01(0.01+0.00) -95.2% 2000.41: git diff --staged (sparse-v4) 0.22(0.21+0.01) 0.14(0.11+0.03) -36.4% 0.15(0.13+0.02) -31.8% 0.01(0.01+0.00) -95.5% ``` The `git diff` improvements are measurable. ``` 2000.42: git sparse-checkout reapply (full-v3) 0.63(0.54+0.05) 0.56(0.48+0.04) -11.1% 0.57(0.48+0.03) -9.5% 0.59(0.48+0.05) -6.3% 2000.43: git sparse-checkout reapply (full-v4) 0.60(0.54+0.02) 0.51(0.46+0.03) -15.0% 0.54(0.48+0.02) -10.0% 0.50(0.44+0.04) -16.7% 2000.44: git sparse-checkout reapply (sparse-v3) 0.91(0.86+0.05) 0.05(0.05+0.00) -94.5% 0.06(0.05+0.01) -93.4% 0.06(0.06+0.00) -93.4% 2000.45: git sparse-checkout reapply (sparse-v4) 0.92(0.88+0.04) 0.05(0.05+0.00) -94.6% 0.05(0.05+0.01) -94.6% 0.05(0.04+0.01) -94.6% ``` Finally, the `git sparse-checkout` measurements are also present. This test script is particularly valuable when contributing changes upstream. It can be good to start by adding the lines to the performance test in an early commit, then demonstrating the performance change by copying the necessary lines from the output table into your commit message.

vdye requested review from dscho, derrickstolee and ldennington September 1, 2021 13:23

vdye self-assigned this Sep 1, 2021

derrickstolee reviewed Sep 1, 2021

View reviewed changes

cache-tree.c Outdated Show resolved Hide resolved

ldennington reviewed Sep 1, 2021

View reviewed changes

derrickstolee reviewed Sep 1, 2021

View reviewed changes

This was referenced Sep 1, 2021

sparse-index diff integration: initial attempt #418

Closed

diff: enable and test the sparse index #419

Merged

derrickstolee reviewed Sep 2, 2021

View reviewed changes

derrickstolee approved these changes Sep 7, 2021

View reviewed changes

vdye added 5 commits September 7, 2021 13:07

reset: expand test coverage in sparse checkout

e934f4f

Add tests for `--merge` and `--keep` modes, as well as (mixed) reset with pathspecs both inside and outside of the sparse checkout definition. Signed-off-by: Victoria Dye <vdye@github.com>

sparse-index: use read-tree for expand/collapse test

8d8143a

In anticipation of `reset` being fully integrated with sparse index, the test for index expand and collapse for non-sparse index integrated commands is updated to use `read-tree`. Signed-off-by: Victoria Dye <vdye@github.com>

vdye merged commit f28fc01 into microsoft:vfs-2.33.0 Sep 7, 2021

vdye deleted the sparse-index/git-reset branch September 7, 2021 20:29

derrickstolee mentioned this pull request Sep 13, 2021

Sparse index: add more performance tests #425

Merged

dscho pushed a commit that referenced this pull request Oct 30, 2021

Merge pull request #417 from vdye/sparse-index/git-reset

69e6c5c

Sparse index: `git reset`

derrickstolee pushed a commit that referenced this pull request Oct 30, 2021

Merge pull request #417 from vdye/sparse-index/git-reset

ac2e52d

Sparse index: `git reset`

derrickstolee pushed a commit that referenced this pull request Oct 31, 2021

Merge pull request #417 from vdye/sparse-index/git-reset

1cddfde

Sparse index: `git reset`

derrickstolee pushed a commit that referenced this pull request Nov 4, 2021

Merge pull request #417 from vdye/sparse-index/git-reset

279ee77

Sparse index: `git reset`

ldennington pushed a commit to ldennington/git that referenced this pull request Jan 12, 2022

Merge pull request microsoft#417 from vdye/sparse-index/git-reset

ba36888

Sparse index: `git reset`

ldennington pushed a commit to ldennington/git that referenced this pull request Jan 19, 2022

Merge pull request microsoft#417 from vdye/sparse-index/git-reset

9c89a83

Sparse index: `git reset`

ldennington pushed a commit to ldennington/git that referenced this pull request Jan 20, 2022

Merge pull request microsoft#417 from vdye/sparse-index/git-reset

bfa6d9d

Sparse index: `git reset`

ldennington pushed a commit to ldennington/git that referenced this pull request Jan 20, 2022

Merge pull request microsoft#417 from vdye/sparse-index/git-reset

9a24428

Sparse index: `git reset`

ldennington pushed a commit to ldennington/git that referenced this pull request Jan 20, 2022

Merge pull request microsoft#417 from vdye/sparse-index/git-reset

5cb1841

Sparse index: `git reset`

ldennington pushed a commit to ldennington/git that referenced this pull request Jan 20, 2022

Merge pull request microsoft#417 from vdye/sparse-index/git-reset

b88753f

Sparse index: `git reset`

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sparse index: `git reset` #417

Sparse index: `git reset` #417

vdye commented Sep 1, 2021 •

edited

Loading

ldennington left a comment

derrickstolee left a comment

derrickstolee left a comment

derrickstolee Sep 2, 2021

vdye Sep 2, 2021 •

edited

Loading

derrickstolee Sep 2, 2021

vdye Sep 2, 2021

vdye Sep 3, 2021

derrickstolee left a comment

derrickstolee Sep 7, 2021

Sparse index: git reset #417

Sparse index: git reset #417

Conversation

vdye commented Sep 1, 2021 • edited Loading

Changes

ldennington left a comment

Choose a reason for hiding this comment

derrickstolee left a comment

Choose a reason for hiding this comment

derrickstolee left a comment

Choose a reason for hiding this comment

derrickstolee Sep 2, 2021

Choose a reason for hiding this comment

vdye Sep 2, 2021 • edited Loading

Choose a reason for hiding this comment

derrickstolee Sep 2, 2021

Choose a reason for hiding this comment

vdye Sep 2, 2021

Choose a reason for hiding this comment

vdye Sep 3, 2021

Choose a reason for hiding this comment

derrickstolee left a comment

Choose a reason for hiding this comment

derrickstolee Sep 7, 2021

Choose a reason for hiding this comment

Sparse index: `git reset` #417

Sparse index: `git reset` #417

vdye commented Sep 1, 2021 •

edited

Loading

vdye Sep 2, 2021 •

edited

Loading