Skip to content

Commit

Permalink
refs: skip hashing when writing packed-refs v2
Browse files Browse the repository at this point in the history
The 'skip_hash' option in 'struct hashfile' indicates that we want to
use the hashfile API as a buffered writer, and not use the hash function
to create a trailing hash. We still write a trailing null hash to
indicate that we do not have a checksum at the end. This feature is
enabled for index writes using the 'index.computeHash' config key.

Create a similar (currently hidden) option for the packed-refs v2 file
format: refs.hashPackedRefs. This defaults to false because performance
is compared to the packed-refs v1 file format which does have a checksum
anywhere.

This change results in improvements to p1401 when using a repository
with a 42 MB packed-refs file (600,000+ refs).

Test                        HEAD~1            HEAD
--------------------------------------------------------------------
1401.1: git pack-refs (v1)  0.38(0.31+0.52)   0.37(0.28+0.52) -2.6%
1401.5: git pack-refs (v2)  0.39(0.33+0.52)   0.30(0.28+0.46) -23.1%

Note that these tests update a ref and then repack the packed-refs file.
The following benchmarks are from a hyperfine experiment that only ran
the 'git pack-refs --all' command for the two formats, but also compared
the effect when refs.hashPackedRefs=true.

Benchmark 1: v1
  Time (mean ± σ):     163.5 ms ±  18.1 ms    [User: 117.8 ms, System: 38.1 ms]
  Range (min … max):   131.3 ms … 190.4 ms    50 runs

Benchmark 2: v2-no-hash
  Time (mean ± σ):      95.8 ms ±  15.1 ms    [User: 72.5 ms, System: 23.0 ms]
  Range (min … max):    82.9 ms … 131.2 ms    50 runs

Benchmark 3: v2-hashing
  Time (mean ± σ):     100.8 ms ±  16.4 ms    [User: 77.2 ms, System: 23.1 ms]
  Range (min … max):    83.0 ms … 131.1 ms    50 runs

Summary
  'v2-no-hash' ran
    1.05 ± 0.24 times faster than 'v2-hashing'
    1.71 ± 0.33 times faster than 'v1'

In this case of repeatedly rewriting the same refs seems to demonstrate
a smaller improvement than the p1401 test. However, the overall
reduction from v1 matches the expected reduction in file size. In my
tests, the 42 MB packed-refs (v1) file was compacted to 28 MB in the v2
format.

Signed-off-by: Derrick Stolee <derrickstolee@github.com>
  • Loading branch information
derrickstolee committed Nov 7, 2022
1 parent 4a4f230 commit 873b5bb
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 0 deletions.
7 changes: 7 additions & 0 deletions refs/packed-format-v2.c
Original file line number Diff line number Diff line change
Expand Up @@ -417,6 +417,7 @@ struct write_packed_refs_v2_context *create_v2_context(struct packed_ref_store *
struct strbuf *err)
{
struct write_packed_refs_v2_context *ctx;
int do_skip_hash;
CALLOC_ARRAY(ctx, 1);

ctx->refs = refs;
Expand All @@ -430,6 +431,12 @@ struct write_packed_refs_v2_context *create_v2_context(struct packed_ref_store *
}

ctx->f = hashfd(refs->tempfile->fd, refs->tempfile->filename.buf);

/* Default to true, so skip_hash if not set. */
if (git_config_get_maybe_bool("refs.hashpackedrefs", &do_skip_hash) ||
do_skip_hash)
ctx->f->skip_hash = 1;

ctx->cf = init_chunkfile(ctx->f);

return ctx;
Expand Down
5 changes: 5 additions & 0 deletions t/perf/p1401-ref-operations.sh
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,11 @@ test_perf 'git pack-refs (v2)' '
git pack-refs --all
'

test_perf 'git pack-refs (v2;hashing)' '
git commit --allow-empty -m "change one ref" &&
git -c refs.hashPackedRefs=true pack-refs --all
'

test_perf 'git for-each-ref (v2)' '
git for-each-ref --format="%(refname)" >/dev/null
'
Expand Down

0 comments on commit 873b5bb

Please sign in to comment.