pkgs.root inside devshells.<name>.devshell.packagesFrom raises "not of type package or string convertible to it" error

[ShamrockLee]

Describe the bug

The root derivation inside Nixpkgs is the package of CERN ROOT, a C++-based data analysis framework.

When using devshell.flakeModule and specify devshells.root-devenv.devshell.packagesFrom = [ pkgs.root ];, where pkgs is from the input of the function assigned to perSystem flake-part attribute, the following error is triggered

       error: A definition for option `perSystem.x86_64-linux.devshells.root-devenv.devshell.packages."[definition 1-entry 39]"' is not of type `package or string convertible to it'. Definition values:
       - In `/nix/store/8rz17fv8rxpzzb7qglhvsk3p034g5hmf-source/modules/devshell.nix': /nix/store/jq8mrs1gldmvkfi3fap5gvj59gxqj4zs-source/pkgs/build-support/setup-hooks/separate-debug-info.sh

To Reproduce

Steps to reproduce the behavior:

1. Prepare flake.nix with the following content

   {
     inputs.devshell.url = "github:numtide/devshell";
     inputs.devshell.inputs = {
       nixpkgs.follows = "nixpkgs";
       systems.follows = "systems";
     };
     inputs.flake-parts.url = "github:hercules-ci/flake-parts";
     inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-23.05";
     inputs.systems.url = "github:nix-systems/default";
     inputs.systems.flake = false;
   
     outputs = inputs:
       inputs.flake-parts.lib.mkFlake { inherit inputs; }
       {
         imports = [
           inputs.devshell.flakeModule
         ];
         systems = import inputs.systems;
         perSystem = { config, self', inputs', pkgs, system, ... }: {
           devshells.root-devenv = {
             devshell = {
               packagesFrom = [
                 pkgs.root
               ];
             };
           };
         };
       };
   }

2. Prepare flake.lock with the following content

   {
     "nodes": {
       "devshell": {
         "inputs": {
           "nixpkgs": [
             "nixpkgs"
           ],
           "systems": [
             "systems"
           ]
         },
         "locked": {
           "lastModified": 1688380630,
           "narHash": "sha256-8ilApWVb1mAi4439zS3iFeIT0ODlbrifm/fegWwgHjA=",
           "owner": "numtide",
           "repo": "devshell",
           "rev": "f9238ec3d75cefbb2b42a44948c4e8fb1ae9a205",
           "type": "github"
         },
         "original": {
           "owner": "numtide",
           "repo": "devshell",
           "type": "github"
         }
       },
       "flake-parts": {
         "inputs": {
           "nixpkgs-lib": "nixpkgs-lib"
         },
         "locked": {
           "lastModified": 1690933134,
           "narHash": "sha256-ab989mN63fQZBFrkk4Q8bYxQCktuHmBIBqUG1jl6/FQ=",
           "owner": "hercules-ci",
           "repo": "flake-parts",
           "rev": "59cf3f1447cfc75087e7273b04b31e689a8599fb",
           "type": "github"
         },
         "original": {
           "owner": "hercules-ci",
           "repo": "flake-parts",
           "type": "github"
         }
       },
       "nixpkgs": {
         "locked": {
           "lastModified": 1690927903,
           "narHash": "sha256-D5gCaCROnjEKDOel//8TO/pOP87pAEtT0uT8X+0Bj/U=",
           "owner": "NixOS",
           "repo": "nixpkgs",
           "rev": "bd836ac5e5a7358dea73cb74a013ca32864ccb86",
           "type": "github"
         },
         "original": {
           "owner": "NixOS",
           "ref": "nixos-23.05",
           "repo": "nixpkgs",
           "type": "github"
         }
       },
       "nixpkgs-lib": {
         "locked": {
           "dir": "lib",
           "lastModified": 1690881714,
           "narHash": "sha256-h/nXluEqdiQHs1oSgkOOWF+j8gcJMWhwnZ9PFabN6q0=",
           "owner": "NixOS",
           "repo": "nixpkgs",
           "rev": "9e1960bc196baf6881340d53dccb203a951745a2",
           "type": "github"
         },
         "original": {
           "dir": "lib",
           "owner": "NixOS",
           "ref": "nixos-unstable",
           "repo": "nixpkgs",
           "type": "github"
         }
       },
       "root": {
         "inputs": {
           "devshell": "devshell",
           "flake-parts": "flake-parts",
           "nixpkgs": "nixpkgs",
           "systems": "systems"
         }
       },
       "systems": {
         "locked": {
           "lastModified": 1681028828,
           "narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=",
           "owner": "nix-systems",
           "repo": "default",
           "rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e",
           "type": "github"
         },
         "original": {
           "owner": "nix-systems",
           "repo": "default",
           "type": "github"
         }
       }
     },
     "root": "root",
     "version": 7
   }

3. Place them in the current working directory.
4. Run nix flake check path:. and see error.
5. Run nix develop path:. root-devenv and see error.
6. Replace the above pkgs.root with pkgs.hello, and the error will disappear.

Expected behavior

The above flake.nix and flake.lock run without error, as if the pkgs.root were replaced with pkgs.hello.

System information

nix-info -m

• system: "x86_64-linux"
• host os: Linux 6.1.31, NixOS, 23.05 (Stoat), 23.05.20230604.e7603eb
• multi-user?: yes
• sandbox: yes
• version: nix-env (Nix) 2.15.0
• nixpkgs: /nix/store/yd0l7092ykql0k60fn1jxlqxl16wp4vn-source

Additional context

$ nix flake check --show-trace path:.
error:
       … while checking flake output 'devShells'

         at «none»:0: (source not available)

       … while checking the derivation 'devShells.aarch64-linux.root-devenv'

         at «none»:0: (source not available)

       … while calling the 'derivationStrict' builtin

         at //builtin/derivation.nix:9:12: (source not available)

       … while evaluating derivation 'devshell'
         whose name attribute is located at /nix/store/8rz17fv8rxpzzb7qglhvsk3p034g5hmf-source/nix/mkNakedShell.nix:30:10

       … while evaluating attribute 'args' of derivation 'devshell'

         at /nix/store/8rz17fv8rxpzzb7qglhvsk3p034g5hmf-source/nix/mkNakedShell.nix:36:3:

           35|   # Bring in the dependencies on `nix-build`
           36|   args = [ "-ec" "${coreutils}/bin/ln -s ${profile} $out; exit 0" ];
             |   ^
           37|

       … while evaluating derivation 'devshell-dir'
         whose name attribute is located at /nix/store/jq8mrs1gldmvkfi3fap5gvj59gxqj4zs-source/pkgs/stdenv/generic/make-derivation.nix:303:7

       … while evaluating attribute 'passAsFile' of derivation 'devshell-dir'

         at /nix/store/jq8mrs1gldmvkfi3fap5gvj59gxqj4zs-source/pkgs/build-support/trivial-builders/default.nix:88:7:

           87|       inherit buildCommand name;
           88|       passAsFile = [ "buildCommand" ]
             |       ^
           89|         ++ (derivationArgs.passAsFile or []);

       … while calling anonymous lambda

         at /nix/store/fc0ajx4015sh8i65iwb8qd3vap3wswpr-source/lib/types.nix:509:14:

          508|       merge = loc: defs:
          509|         map (x: x.value) (filter (x: x ? value) (concatLists (imap1 (n: def:
             |              ^
          510|           imap1 (m: def':

       error: A definition for option `perSystem.aarch64-linux.devshells.root-devenv.devshell.packages."[definition 1-entry 39]"' is not of type `package or string convertible to it'. Definition values:
       - In `/nix/store/8rz17fv8rxpzzb7qglhvsk3p034g5hmf-source/modules/devshell.nix': /nix/store/jq8mrs1gldmvkfi3fap5gvj59gxqj4zs-source/pkgs/build-support/setup-hooks/separate-debug-info.sh

[sigprof]

Apparently the same underlying error can be reproduced using this simple devshell.toml (after nix flake init -t github:numtide/devshell):

[devshell]
packagesFrom = ["i3"]

Error output

error:
       … while calling the 'derivationStrict' builtin

         at /builtin/derivation.nix:9:12: (source not available)

       … while evaluating derivation 'devshell'
         whose name attribute is located at /nix/store/ycrg6ch2z4pbrpf0pwk875j8ipyg99pp-source/nix/mkNakedShell.nix:30:10

       … while evaluating attribute 'args' of derivation 'devshell'

         at /nix/store/ycrg6ch2z4pbrpf0pwk875j8ipyg99pp-source/nix/mkNakedShell.nix:36:3:

           35|   # Bring in the dependencies on `nix-build`
           36|   args = [ "-ec" "${coreutils}/bin/ln -s ${profile} $out; exit 0" ];
             |   ^
           37|

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: A definition for option `devshell.packages."[definition 1-entry 33]"' is not of type `package or string convertible to it'. Definition values:
       - In `/nix/store/ycrg6ch2z4pbrpf0pwk875j8ipyg99pp-source/modules/devshell.nix': /nix/store/rkkiiyv317i5c9iwqz8cz7prvd04k08k-source/pkgs/build-support/setup-hooks/separate-debug-info.sh

Looks like the error is triggered by separateDebugInfo = true; in the package; specifying the devshell configuration in Nix and using overrideAttrs may be used as a workaround:

          devshell.packagesFrom = [
            (pkgs.i3.overrideAttrs { separateDebugInfo = false; })
          ];

Apparently the problem comes from https://github.com/NixOS/nixpkgs/blob/7c5f52e2bf743401ab9253bab048883d66ad357f/pkgs/stdenv/generic/make-derivation.nix#L282 — that code adds a path instead of a proper derivation to nativeBuildInputs, which then shows up in the package attributes (the last one):

$ nix repl --extra-experimental-features repl-flake github:NixOS/nixpkgs/nixpkgs-unstable
nix-repl> legacyPackages.x86_64-linux.i3.nativeBuildInputs
[ «derivation /nix/store/9by54ylsfw1a617njnldvhn8dr5j01wx-pkg-config-wrapper-0.29.2.drv» «derivation /nix/store/97yalyh89bbizl1v8vimw2wzj523kxpy-make-shell-wrapper-hook.drv» «derivation /nix/store/pydn180r1ydq95qvpajqj4nqiz99zzk7-meson-1.3.2.drv» «derivation /nix/store/4risxx9frn02fraphwrb0jbcpcx78a0g-ninja-1.11.1.drv» «derivation /nix/store/6znlk8vzvs6p344kh139n864rl9rgw18-install-shell-files.drv» «derivation /nix/store/wp1k91z46bnkp1vg99r654km760ridj3-perl-5.38.2.drv» «derivation /nix/store/asni7p54vgbyjdrbl4jrmc96svk1yxj1-asciidoc-10.2.0.drv» «derivation /nix/store/1grx2q3s0k31cdawwdssanxpn0psjy8y-xmlto-0.0.28.drv» «derivation /nix/store/9ngb4s4qc7b1dlkpjdivvi8i4mdm3cas-docbook-xml-4.5.drv» «derivation /nix/store/hac09wcvx8i3xfvc646zdjjpnkf1hiad-docbook-xsl-nons-1.79.2.drv» «derivation /nix/store/fjn8b7x8gdz7hhav0qyvjr8yzmjaicsc-find-xml-catalogs-hook.drv» /nix/store/43fgdg04gbrjh8ww8q8zgbqxn4sb35py-source/pkgs/build-support/setup-hooks/separate-debug-info.sh ]

Maybe

devshell/modules/devshell.nix

Lines 200 to 205 in 5ddecd6

	inputsOf = drv:
	(drv.buildInputs or [ ]) ++
	(drv.nativeBuildInputs or [ ]) ++
	(drv.propagatedBuildInputs or [ ]) ++
	(drv.propagatedNativeBuildInputs or [ ])
	;

needs to filter the attribute values instead of accepting everything.

[sigprof]

However, the current implementation of devshell.packagesFrom is not ideal even with a possible fix, because it gets only the direct build dependencies of the specified packages, but does not follow the chain of propagatedBuildInputs and propagatedNativeBuildInputs, therefore some build dependencies actually end up being unusable without their own dependencies. In theory, applying lib.misc.closePropagation to the list of packages should solve that problem by bringing in all required dependencies recursively; however, because devshell tries to build a single environment with all specified packages, it runs into conflicts like

error: collision between `/nix/store/5zv2bbd3y7s08qcnj6j7w39i26gqjg9b-xorgproto-2023.2/include/X11/extensions/XKBgeom.h' and `/nix/store/853h1w10rfpvhky02l8a2z07z21sqbhx-libX11-1.8.7-dev/include/X11/extensions/XKBgeom.h'

With the stock mkShell such conflicts are not really a problem, because the conflicting packages don't get combined into a single environment (you get a long list of options in NIX_CFLAGS_COMPILE and NIX_LDFLAGS instead), therefore the conflicts are resolved silently (but one of those conflicting include files would end up shadowing the others).