Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conclude gitoxide to git2 transition #1285

Merged
merged 3 commits into from
Mar 16, 2024
Merged

Conversation

Byron
Copy link
Collaborator

@Byron Byron commented Mar 13, 2024

Fixes #628

I had to wait a long time for this PR to become possible, and finally here it is :)!

Tasks

  • transition the status computation
  • remove git2 dependency
  • Switch to new public release of gix

Review Notes

  • Deviation: I think it's better to not write the index back if any stat-changes are picked up to make onefetch truly read-only.
    If there is disagreement, the index could be updated and written back - it will be valid, but won't write back all extensions that might
    have been present before. git2 most certainly also didn't do it in a way that is 100% faithful to what Git would do.
  • renames_head_to_index() was never functional, as the status was only obtained between the index and the worktree. If this must be present
    with the switch, then I have to hold off with this update as gix status doesn't yet support HEAD->index status.
  • The binary size isn't positively affected by this change, unfortunately. The unstripped binary is now 17710984, up from 17023992, with the build settings of this project.

Performance

In short, onefetch got a little bit faster, and speeds up with the size of the repositories measured by the amount of files in their worktrees.

WebKit

Here the faster status implementation is definitely visible.

❯ hyperfine -w1 -N -r3 onefetch /Users/byron/dev/github.com/o2sh/onefetch/target/release/onefetch
Benchmark 1: onefetch
  Time (mean ± σ):     22.839 s ±  1.328 s    [User: 80.878 s, System: 17.448 s]
  Range (min … max):   21.898 s … 24.358 s    3 runs

Benchmark 2: /Users/byron/dev/github.com/o2sh/onefetch/target/release/onefetch
  Time (mean ± σ):     18.506 s ±  0.145 s    [User: 77.921 s, System: 26.274 s]
  Range (min … max):   18.378 s … 18.663 s    3 runs

Summary
  /Users/byron/dev/github.com/o2sh/onefetch/target/release/onefetch ran
    1.23 ± 0.07 times faster than onefetch
Linux

For linux, the commit-aggregation is the most stressful part, and there is much less of a difference as status is just a small part of the overall runtime.

linux (ffc2532)
❯ hyperfine -w1 -N -r3 onefetch /Users/byron/dev/github.com/o2sh/onefetch/target/release/onefetch
Benchmark 1: onefetch
  Time (mean ± σ):     12.166 s ±  0.049 s    [User: 23.918 s, System: 2.509 s]
  Range (min … max):   12.110 s … 12.200 s    3 runs

Benchmark 2: /Users/byron/dev/github.com/o2sh/onefetch/target/release/onefetch
  Time (mean ± σ):     11.828 s ±  0.031 s    [User: 24.007 s, System: 2.904 s]
  Range (min … max):   11.801 s … 11.862 s    3 runs

Summary
  /Users/byron/dev/github.com/o2sh/onefetch/target/release/onefetch ran
    1.03 ± 0.00 times faster than onefetch
Git

Git is interesting as it doesn't have too many files in the worktree, yet it seems to greatly benefit from the new status implementation. Unfortunately, at ~200ms, the difference isn't really perceivable by a user.

❯ hyperfine -w1 -N -r3 onefetch /Users/byron/dev/github.com/o2sh/onefetch/target/release/onefetch
Benchmark 1: onefetch
  Time (mean ± σ):     237.5 ms ±   4.5 ms    [User: 883.7 ms, System: 223.4 ms]
  Range (min … max):   234.2 ms … 242.6 ms    3 runs

Benchmark 2: /Users/byron/dev/github.com/o2sh/onefetch/target/release/onefetch
  Time (mean ± σ):     213.7 ms ±   3.5 ms    [User: 872.6 ms, System: 235.9 ms]
  Range (min … max):   210.6 ms … 217.5 ms    3 runs

Summary
  /Users/byron/dev/github.com/o2sh/onefetch/target/release/onefetch ran
    1.11 ± 0.03 times faster than onefetch

Summary::Removed => (added, deleted + 1, modified),
Summary::Added => (added + 1, deleted, modified),
Summary::Modified | Summary::TypeChange => (added, deleted, modified + 1),
Summary::Renamed | Summary::Copied => (added + 1, deleted + 1, modified),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Watching the conclusion of switching from git2 to gix is awesome!

Should deleted really be incremented when the status is Copied? AFAIK a rename is always a delete + add, while a copy just means a new file was created with extremely similar contents to an existing file.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a great catch right there! Copied should only count as added file.

This Summary was added just for onefetch and it was interesting to see that the possible states indeed are best represented by an enum, even though libgit2 tries to convince you otherwise.

@Byron
Copy link
Collaborator Author

Byron commented Mar 14, 2024

While I prepare the gix release to ready this PR for review, I recommend trying it out on repos with modifications. I noticed that in my case on the linux kernel, the git2 version showed removals where there were none. Thus, gitoxide seems to be (more) inline with Git.

❯ gst
HEAD detached at v6.6
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   include/uapi/linux/netfilter/xt_DSCP.h
        modified:   include/uapi/linux/netfilter/xt_MARK.h
        modified:   include/uapi/linux/netfilter/xt_RATEEST.h
        modified:   include/uapi/linux/netfilter/xt_TCPMSS.h
        modified:   include/uapi/linux/netfilter/xt_connmark.h
        modified:   include/uapi/linux/netfilter_ipv4/ipt_ECN.h
        modified:   include/uapi/linux/netfilter_ipv4/ipt_TTL.h
        modified:   include/uapi/linux/netfilter_ipv6/ip6t_HL.h
        modified:   net/netfilter/xt_DSCP.c
        modified:   net/netfilter/xt_HL.c
        modified:   net/netfilter/xt_RATEEST.c
        modified:   net/netfilter/xt_TCPMSS.c
        modified:   tools/memory-model/litmus-tests/Z6.0+pooncelock+poonceLock+pombonce.litmus

no changes added to commit (use "git add" and/or "git commit -a")

linux (ffc2532) +369 -819 [!]
❯ onefetch && /Users/byron/dev/github.com/o2sh/onefetch/target/release/onefetch
                 ++++++                    Sebastian Thiel ~ git version 2.39.3 (Apple Git-146)
              ++++++++++++                 ----------------------------------------------------
          ++++++++++++++++++++             Project: linux (2 branches, 820 tags)
       ++++++++++++++++++++++++++          HEAD: ffc253263a13
    ++++++++++++++++++++++++++++++++       Pending: 12+- 13-
 +++++++++++++************+++++++++++++    Version: v6.7-rc8
+++++++++++******************++++++++;;;   Created: 18 years ago
+++++++++**********************++;;;;;;;   Languages:
++++++++*********++++++******;;;;;;;;;;;              ● C (98.4 %) ● Shell (0.7 %)
+++++++********++++++++++**;;;;;;;;;;;;;              ● Python (0.3 %) ● Makefile (0.3 %)
+++++++*******+++++++++;;;;;;;;;;;;;;;;;              ● Perl (0.2 %) ● Rust (0.1 %)
+++++++******+++++++;;;;;;;;;;;;;;;;;;;;              ● Other (0.0 %)
+++++++*******+++:::::;;;;;;;;;;;;;;;;;;   Authors: 3% Linus Torvalds 35041
+++++++********::::::::::**;;;;;;;;;;;;;            1% David S. Miller 14200
++++++++*********::::::******;;;;;;;;;;;            1% Arnd Bergmann 10387
++++++:::**********************::;;;;;;;   Last change: 4 months ago
+++::::::::******************::::::::;;;   Contributors: 35615
 :::::::::::::************:::::::::::::    URL: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
    ::::::::::::::::::::::::::::::::       Commits: 1217245
       ::::::::::::::::::::::::::          Churn (491): …/usb/r8152.c 8
          ::::::::::::::::::::                          …/netfilter/nf_tables_api.c 5
              ::::::::::::                              MAINTAINERS 4
                 ::::::                    Lines of code: 17644823
                                           Size: 1.26 GiB (81766 files)



                 ++++++                    Sebastian Thiel ~ git version 2.39.3 (Apple Git-146)
              ++++++++++++                 ----------------------------------------------------
          ++++++++++++++++++++             Project: linux (2 branches, 820 tags)
       ++++++++++++++++++++++++++          HEAD: ffc253263a13
    ++++++++++++++++++++++++++++++++       Pending: 13+-
 +++++++++++++************+++++++++++++    Version: v6.7-rc8
+++++++++++******************++++++++;;;   Created: 18 years ago
+++++++++**********************++;;;;;;;   Languages:
++++++++*********++++++******;;;;;;;;;;;              ● C (98.4 %) ● Shell (0.7 %)
+++++++********++++++++++**;;;;;;;;;;;;;              ● Python (0.3 %) ● Makefile (0.3 %)
+++++++*******+++++++++;;;;;;;;;;;;;;;;;              ● Perl (0.2 %) ● Rust (0.1 %)
+++++++******+++++++;;;;;;;;;;;;;;;;;;;;              ● Other (0.0 %)
+++++++*******+++:::::;;;;;;;;;;;;;;;;;;   Authors: 3% Linus Torvalds 35041
+++++++********::::::::::**;;;;;;;;;;;;;            1% David S. Miller 14200
++++++++*********::::::******;;;;;;;;;;;            1% Arnd Bergmann 10387
++++++:::**********************::;;;;;;;   Last change: 4 months ago
+++::::::::******************::::::::;;;   Contributors: 35615
 :::::::::::::************:::::::::::::    URL: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
    ::::::::::::::::::::::::::::::::       Commits: 1217245
       ::::::::::::::::::::::::::          Churn (445): …/usb/r8152.c 8
          ::::::::::::::::::::                          Makefile 4
              ::::::::::::                              …/misc/fastrpc.c 4
                 ::::::                    Lines of code: 17644823
                                           Size: 1.26 GiB (81766 files)

Summary::Removed => (added, deleted + 1, modified),
Summary::Added | Summary::Copied => (added + 1, deleted, modified),
Summary::Modified | Summary::TypeChange => (added, deleted, modified + 1),
Summary::Renamed => (added + 1, deleted + 1, modified),
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By the way, it won't run into Renamed and Copied as rename tracking isn't activated by default.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you expand on that, what are the implications ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would mean the code could call unreachable!() in case of Copied and Renamed and be right about it. The implementation/settings have the same effect as git2 previously.

Does that make more sense?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you refering to git config --get diff.renames ? Whether it's enable doesn't change the output for onefetch, right? a rename is always shown as an addition and an deletion.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a great point to bring up! I decided not to let it affect renames between index and worktree as Git itself doesn't even expose that functionality to the user (only git2 seems to have it). Everything that is affected by configuration, really only emit_untracked at this time, is overridden though.

@Byron Byron force-pushed the gitoxide-for-status branch from dfc0831 to 9c15105 Compare March 14, 2024 20:42
@Byron Byron marked this pull request as ready for review March 14, 2024 20:50
@Byron Byron requested a review from o2sh as a code owner March 14, 2024 20:50
Copy link
Owner

@o2sh o2sh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Finally, onefetch is fully oxidized 🎉

The missing piece finally came through.

Just tested it and #978 seems to be resolved

Can the PR be merged?

@Byron
Copy link
Collaborator Author

Byron commented Mar 16, 2024

Great to hear that another issue is fixed :).

This PR is good to go from my side.

@o2sh o2sh merged commit 8fcaf08 into o2sh:main Mar 16, 2024
12 checks passed
@o2sh o2sh added the chore label Mar 16, 2024
@Byron Byron deleted the gitoxide-for-status branch March 17, 2024 06:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Replace libgit2 with gitoxide
3 participants