Can't compile the nix library since da66431

[JohanLorenzo]

I wish I could have a better title 😐

A few days ago, I came across a compiler issue when I tried to compile nix.

Steps to reproduce

1. git clone https://github.com/nix-rust/nix.git
2. cd nix
3. cargo build

Results

rustc: /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/llvm/include/llvm/Support/Casting.h:237: typename llvm::cast_retty<X, Y*>::ret_type llvm::cast(Y*) [with X = llvm::Function; Y = llvm::Value; typename llvm::cast_retty<X, Y*>::ret_type = llvm::Function*]: Assertion `isa<X>(Val) && "cast<Ty>() argument of incompatible type!"' failed.
error: Could not compile `nix`.

nix-rust/nix#339 was filed to track it. With @pnkfelix 's help, I got some debug traces. However, it didn't really help knowing which function of nix was impacted.

Regression window

A bisection narrowed down the issue to #32080, and more precisely these commits. I have not been able to bisect down further, because none of these commits compile on my machine.

Investigation

In the meantime, @pnkfelix found the function likely impacted. You can see the results of his investigation.

@eddyb, what information would you think we need to isolate the problem?

[eddyb]

Ah, I have been looking for backtraces for the assertion hit in servo/servo#10173 (I think it's the same as this).

It appears we're misusing LLVMGetOrInsertFunction which can return a cast pointer instead of a definition, if the existing type and the desired one differ, but we're only casting at a later time manually (and assuming we have a definition with no cast in between).

Looking back at my commits, it appears that we used to cache imports by symbol name, instead of ID, so that was actually bypassing the behavior of LLVMGetOrInsertFunction.