Use git protocol v2 for shallow git_repository fetches

[michaeleisel]

Without this, the git fetch was failing for me. I believe git operations run with protocol v2 should work seamlessly with other git operations, but I'm not an expert. v2 is the default in git 2.26+, but e.g. MacOS isn't there yet. The global config analogy would be git config --global protocol.version 2.

[michaeleisel]

@HackAttack can you take a look?

[philwo]

Hi @michaeleisel, thanks for this fix!

I tested this manually on:

• CentOS 7 with git 1.8.3.1,
• Ubuntu 16.04 with git 2.7.4,
• Ubuntu 18.04 with git 2.17.1,
• Ubuntu 20.04 with git 2.25.1.

It works fine everywhere except on Ubuntu 18.04:

$ docker run --rm -it ubuntu:18.04 bash
$ apt update && apt install -y git
$ git --version
git version 2.17.1

$ git clone --depth 1 https://github.com/bazelbuild/bazelisk.git
Cloning into 'bazelisk'...
remote: Enumerating objects: 48, done.
remote: Counting objects: 100% (48/48), done.
remote: Compressing objects: 100% (41/41), done.
remote: Total 48 (delta 5), reused 20 (delta 3), pack-reused 0
Unpacking objects: 100% (48/48), done.

$ git -c protocol.version=2 clone --depth 1 https://github.com/bazelbuild/bazelisk.git
Cloning into 'bazelisk'...
fatal: unknown value for config 'protocol.version': 2

So, this would be a regression for users of Ubuntu 18.04, which is a supported OS for Bazel. 😞

Could you explain more about the git invocation where you got an error without this additional flag? What was the error message?

[HackAttack]

Any reason to not make this change to start above, so it applies everywhere?

[michaeleisel]

@philwo this is just a fast path for which we handle failure for anyways (see directly below the change), and my understanding is that protocol v2 is needed for fetching arbitrary shas. so, it's not a regression in terms of stability. the example where you succeeded without it is one where it's not fetching an arbitrary sha. for specific cases like that where the sha is being advertised, it is indeed a regression in performance, and perhaps we could run this command twice, once with protocol v2, and once without

@HackAttack the reason to only use it here was to limit the impact of this flag, which seems to be good since there's this regression

[HackAttack]

my understanding is that protocol v2 is needed for fetching arbitrary shas.

I don’t think this is true.

$ docker run -h test --rm -ti ubuntu:16.04 bash
root@test:/# apt update -qq && apt install -y git &> /dev/null
All packages are up to date.
root@test:/# mkdir bazel && cd bazel && git init
Initialized empty Git repository in /bazel/.git/
root@test:/bazel# git remote add origin https://github.com/bazelbuild/bazel.git
root@test:/bazel# git fetch origin 4a27ebbd48b82a3768476cd70fe6925510bde52f
remote: Enumerating objects: 64, done.
remote: Counting objects: 100% (64/64), done.
remote: Compressing objects: 100% (47/47), done.
remote: Total 507977 (delta 13), reused 36 (delta 9), pack-reused 507913
Receiving objects: 100% (507977/507977), 706.54 MiB | 38.69 MiB/s, done.
Resolving deltas: 100% (316795/316795), done.
From https://github.com/bazelbuild/bazel
 * branch            4a27ebbd48b82a3768476cd70fe6925510bde52f -> FETCH_HEAD
root@test:/bazel# git --version
git version 2.7.4

[michaeleisel]

It's needed for GHE on macOS at least, up until a very recent version (also note that fetching an individual sha can succeed if it just happens to be tied to a branch or something)

[philwo]

Sorry, I didn't have time to look into this again yet. I'm still confused why Git is not smart enough to negotiate the correct protocol automatically :(

perhaps we could run this command twice, once with protocol v2, and once without

Did we figure out whether that's a good idea?

[thii]

@michaeleisel What kind of failing did you see? Have you seen this: #10292

[dfreese]

Wanted to check in on this, is Ubuntu 18.04 still considered a supported platform?