Long build times caused by full rebuild of rustc and llvm

[kjhoerr]

After #147 was merged I noticed system upgrades were taking a long time and using a lot of resources. Locally this pushes my upgrade times from ~2 minutes to an hour or more, depending on the machine. Per GH actions doing a build of all systems used to take about 20 minutes but now times out at 6 hours.

https://github.com/kjhoerr/dotfiles/actions/runs/5251351652/jobs/9486211349

I was able to revert the selected lanzaboote revision to prior #147 and build times went back to normal. All my inputs are available here in case this is an isolated issue.

Looking at the derivations it looks like it's building x86_64-windows-rustc as a dependency for lanzaboote - cross dependency issue potentially?

[SuperSandro2000]

You want to use nixos-23.05 instead of the small variant to avoid cache misses. Also using nixos-unstable could also increase cache hits.

It could still be that this is not cached but using those channels could improve the situation.

[kjhoerr]

Oh duh! Forgot I switched it to the stable release. I should have checked that first.

Looks like llvm is covered by nixos-unstable-small at least. Might need to switch over to nixos-unstable still.

Thanks!

[RaitoBezarius]

Also, @kjhoerr I advise you to track a stable tag of Lanzaboote rather than the master branch for now. :)

[SuperSandro2000]

That is in flakes how? There is not stable branch.

[RaitoBezarius]

This is a stable reference: https://github.com/nix-community/lanzaboote/releases/tag/v0.3.0

[SuperSandro2000]

So I can just use github:nix-community/lanzaboote/v0.3.0?

[RaitoBezarius]

Correct!

[SuperSandro2000]

TIL

[Quant-ux]

Please correct me if I'm wrong but it's been a month since this was reported so binary cache for both LLVM and rustc should be available by now.

And although I can confirm that this is the case (I run nix build nixpkgs#llvm_{14,15} on an out-of-date NixOS Unstable installation), an attempt to build LLVM is still made for me.

After telling Nix to use github:nix-community/lanzaboote/v0.3.0, I could rebuild my system and successfully switch to NixOS 23.05, without having to wait for LLVM to get built. Now, I've booted into the new generation and if I try reverting to github:nix-community/lanzaboote, the attempt is being made, again. This begs the question, is this issue actually solved?

These are my two cents.

[RaitoBezarius]

Please correct me if I'm wrong but it's been a month since this was reported so binary cache for both LLVM and rustc should be available by now.

It was always from the very start, but not in cache.nixos.org. It is in the nix-community's cache.

And although I can confirm that this is the case (I run nix build nixpkgs#llvm_{14,15} on an out-of-date NixOS Unstable installation), an attempt to build LLVM is still made for me.

Yes, this is expected.

After telling Nix to use github:nix-community/lanzaboote/v0.3.0, I could rebuild my system and successfully switch to NixOS 23.05, without having to wait for LLVM to get built. Now, I've booted into the new generation and if I try reverting to github:nix-community/lanzaboote, the attempt is being made, again. This begs the question, is this issue actually solved?

This is solved by not using the master branch of lanzaboote, except if you are a developer and you know what you are doing, in that case, you know how to add the nix-community cache and sidesteps this problem. Otherwise, you should avoid running the master branch in the time being.