Deep runtime dependency substitutions

Sometimes there's a need to replace a runtime dependency, but if that dependency is compiled deep into a dependency tree, it becomes really expensive since the whole tree needs to be recompiled. Meanwhile, the alternative of not hard-coding the dependency, but instead discovering it at runtime is impure. We should find a solution for that.

Use cases

• Third-party glibc NSS modules for user name lookups and co. Currently essentially the entirety of nixpkgs needs to be recompiled if a custom module is desired
• SSL certificate updates to cacert: A lot of packages depend on the cacert package providing SSL root certificates. These certificates can get compromised and updated over time though. It should be possible to immediately update the certificates for all packages that depend on them. In addition, past versions of nixpkgs shouldn't use past versions of cacert, they should also be able to use the latest version
• Similarly, timezones change frequently in the tzdata package, old versions of nixpkgs shouldn't run with old timezone information. Also locales
• If packages depend on setuid wrappers, they are currently hardcoded to the NixOS-specific /run/wrappers directory, which breaks these packages on non-NixOS distros and when the wrappers aren't installed.
• Similarly OpenGL drivers which tend to be hardcoded to the NixOS-specific /run/opengl-driver

Possible solutions

Nixpkgs pivots / generic impure runtime dependencies

A rough description of this idea is here. The idea is to mark derivations as needing an impure runtime dependency. For an entire closure we might have many derivations needing such an impure runtime dependency. These dependencies are only resolved at the end by either looking for them in a hardcoded system-wide path, or at a path specified in an environment variable. To ensure reasonable purity, derivations only look for impure runtime dependencies in paths suffixed with their own hash, so e.g. /nix/store/k56...mf6-glibc-2.34-210 would look for NSS modules in

/nix/var/nix/profiles/per-user/root/pivot/k56...mf6-glibc-2.34-210/nss-modules

Shipping Security Updates PR

Shipping Security Updates was inspired Guix grafting, but with the intent to make fixing security update exactly like packaging as usual. A user who fixed the hash of a base nixpkgs, would maintain another branch with applied patches. Then deep dependencies are handled by patching packages as follow:

let
  nixpkgsPatched = patch {
    what = nixpkgsWithImportedFixes (fix nixpkgsBase);
    with = nixpkgsWithImportedFixes nixpkgsPatched;
  };
in nixpkgsPatched

This means that nixpkgsPatched is created by building patches using nixpkgsWithImportedFixes, when building nixpkgsWithImportedFixes on top of fix nixpkgsBase differs from when built on top of nixpkgsPatched.

Thus, if one dependency extremely deep is recompiled, because imported fixes where added in nixpkgsWithImportedFixes, but not in fix nixpkgsBase, then all packages which are depending on it would get patched in nixpkgsPatched, including packages which are newer in nixpkgsWithImportedFixes.

In both cases, we build nixpkgsWithImportedFixes, if the derivation differs, then we know that one of the input differs (i.e. (fix nixpkgsBase)."${pkg}" != nixpkgsPatche."${pkg}"), or transitively if input's inputs were patched. Rewrite of input's inputs can be carried around using passtru attribute. This logic is implemented within the patch function implementation.

Thus replacing all hashes as expected, even if extremely deep.