feat: shallow repo clone in merge checkout strategy #2758

nitrocode · 2022-12-07T01:58:32Z

what

Allow shallow cloning for merge checkout strategy
The default check out is set to 0

why

Faster clones for repos with a long history using the merge strategy. If the checkout cannot do the merge with a shallow clone then the full clone is checked out.
The new flag to set the --checkout-depth for the merge check out strategy continues to default to 0 to avoid breaking any existing users
The branch strategy already uses shallow cloning by cloning with a depth of 1 (unchanged)
This new PR was opened in order to resolve conflicts. The old PR did not allow maintainer updates.

Line 275 in d33bd0f

    
           "git", "clone", "--branch", p.HeadBranch, "--depth=1", "--single-branch", headCloneURL, cloneDir,

references

Previous PR from Joom from @jilyaluk Implement shallow repo cloning in merge checkout strategy #2387
Notable comment Implement shallow repo cloning in merge checkout strategy #2387 (comment)
Relates to Resolve CodeQL issues #2884

When the repository is quite large clones are becoming slower. Previously Atlantis fetched full repository in --checkout-strategy=merge mode. However, that's not required, we only should have enough commits in our clone to perform a merge between base and feature branches, not full history. This implements rough guessing: fetch 50 (user-configurable) commits from each of base and feature branches, and if their merge base is not in these commits, fetch full repo history. This is not ideal solution and could be improved (e.g. increase the number of fetched commits in several attempts before full fetch), but I think it should be good enough in most cases.

runatlantis.io/docs/server-configuration.md

server/events/working_dir_test.go

nitrocode · 2022-12-07T02:22:30Z

@rationalex @ibonder @Radcriminal @nikolaik @jilyaluk

You folks all upvoted the previous PR. I tried my best to bring this back to life but I'm not completely sure why the original change by @jilyaluk was only for the non-default checkout strategy of merge instead of branch or instead of both. If you folks can explain that, it would help because this change does modify a lot of how atlantis clones repos.

A review would also be helpful by maintainers and anyone else willing to look this over. I'm not so savvy and don't have a huge vested interest in this so I do not plan to make additional changes as I do not use the merge checkout strategy.

Edit: it's already enabled for branch strategy

atlantis/server/events/working_dir.go

Line 275 in d33bd0f

    
           "git", "clone", "--branch", p.HeadBranch, "--depth=1", "--single-branch", headCloneURL, cloneDir,

runatlantis.io/docs/checkout-strategy.md

Co-authored-by: Nikolai Røed Kristiansen <nikolaik@gmail.com>

runatlantis.io/docs/server-configuration.md

Co-authored-by: Ken Kaizu <k.kaizu38@gmail.com>

cmd/server.go

* Implement shallow repo clone in merge checkout strategy --------- Co-authored-by: Ilya Lukyanov <il@joom.com> Co-authored-by: Nikolai Røed Kristiansen <nikolaik@gmail.com> Co-authored-by: PePe Amengual <jose.amengual@gmail.com> Co-authored-by: Ken Kaizu <k.kaizu38@gmail.com>

nitrocode requested a review from a team as a code owner December 7, 2022 01:58