Nix 2.4 assumes refs are branch names and fails on tags (regression)

[lionello]

Describe the bug

Nix 2.4 assumes refs are branch names and fails on tags.

Steps To Reproduce

Try this with Nix 2.4:

$ nix repl
nix-repl> builtins.fetchGit { url = "https://github.com/lionello/onedrive-cli.git"; ref = "1.0.5"; }
fetching Git repository 'https://github.com/lionello/onedrive-cli.git'fatal: couldn't find remote ref refs/heads/1.0.5
error: program 'git' failed with exit code 128

Expected behavior

This worked with Nix 2.3.11:

$ nix repl
nix-repl> builtins.fetchGit { url = "https://github.com/lionello/onedrive-cli.git"; ref = "1.0.5"; }
{ outPath = "/nix/store/8h1zfi0k5qcxpx0rjiwljfsg145hgvyw-source"; rev = "b87142e9d1c2d2ebd088975b11fff433cdfba6e1"; revCount = 72; shortRev = "b87142e"; }

nix-env --version output

nix-env (Nix) 2.4pre20210908_3c56f62

Additional context

Changing to ref = "refs/tags/1.0.5"; works as a workaround. (In my experience it's way more common to use tags for versioning than branches.)

[bjornfor]

Cross-reference (duplicate?): #5128

[stale]

I marked this as stale due to inactivity. → More info

[jakubgs]

This issue can be worked around by adding a refs/tags/ prefix to tags. But it is a regression anyway.

[figsoda]

Still important to me

[IreneKnapp]

As I said over on #8790 I believe this is the fault of this line of code:

nix/src/libfetchers/git.cc

Line 565 in 635df5e

: "refs/heads/" + *ref;

I'd be happy to submit a patch that just... doesn't try to prefix the ref name and gives it to git as-is. However, I don't understand why the fetcher is doing that in the first place, so I don't know what other regressions that might cause.

[IreneKnapp]

ha! found it. this was added in fb38459

which says

Ensure we restrict refspec interpretation while fetching
As git fetch may chose to interpret refspec to it's liking, ensure that we
only pass refs that begin with refs/ as is, otherwise, prepend them with
refs/heads. Otherwise, branches named heads/foo (I know it's bad, but it's
allowed), would be fetched as foo, instead of heads/foo.

Additionally, some but not all of the nuances to this logic have been properly documented: https://nixos.org/manual/nix/stable/language/builtins.html#builtins-fetchGit

ref (default: HEAD)

The Git reference under which to look for the requested revision. This is often a branch or tag name.

By default, the ref value is prefixed with refs/heads/. As of 2.3.0, Nix will not prefix refs/heads/ if ref starts with refs/.

I note that the choice to link to upstream git documentation to define what a git reference is, along with the remark that it "is often a branch or tag name", seem to suggest that users should be able to pass anything that git would accept at the command line. Unfortunately this is directly at odds with prefixing it with refs/heads/, since the branch name is the part that comes after that, and the tag name is the part that comes after refs/tags/.

None of those sources say this, but to me the best argument to sanitize the ref is that we don't want nix's behavior to depend too much on specific versions of git and their quirks.

This leaves the question of how we should fix it. If we stick with the current logic that only makes one call to git, we can't know for sure whether we've got a branch or a tag, so turning off the sanitization is the only option. There is a chance that would break someone downstream.

Alternatively, there could be fallback logic which first attempts to retrieve a tag, then a branch if that fails. (Surely tags are more common, right?)

Unfortunately the code is a little more convoluted than I realized at first, I'm not sure I can be the one to implement this.

[IreneKnapp]

oh um just to be more explicit, my preferred solution is the two-calls-to-git approach, trying it first as a tag and then as a branch.