Skip to content

Commit

Permalink
Create outputOf primop.
Browse files Browse the repository at this point in the history
In the Nix language, given a drv path, we should be able to construct
another string referencing to one of its output. We can do this today
with `(import drvPath).output`, but this only works for derivations we
already have.

With dynamic derivations, however, that doesn't work well because the
`drvPath` isn't yet built: importing it like would need to trigger IFD,
when the whole point of this feature is to do "dynamic build graph"
without IFD!

Instead, what we want to do is create a placeholder value with the right
string context to refer to the output of the as-yet unbuilt derivation.
A new primop in the language, analogous to `builtins.placeholder` can be
used to create one. This will achieve all the right properties. The
placeholder machinery also will match out the `outPath` attribute for CA
derivations works.

In 60b7121 we added that type of placeholder,
and the derived path and string holder changes necessary to support it.
Now, we can wire up the primop.

Part of RFC 92: dynamic derivations (tracking issue NixOS#6316)
  • Loading branch information
Ericson2314 committed Aug 11, 2023
1 parent 010dc79 commit 0d26a45
Show file tree
Hide file tree
Showing 5 changed files with 82 additions and 1 deletion.
33 changes: 33 additions & 0 deletions src/libexpr/primops.cc
Original file line number Diff line number Diff line change
Expand Up @@ -1836,6 +1836,39 @@ static RegisterPrimOp primop_readDir({
.fun = prim_readDir,
});

/* Extend single element string context with another output. */
static void prim_outputOf(EvalState & state, const PosIdx pos, Value * * args, Value & v)
{
SingleDerivedPath drvPath = state.coerceToSingleDerivedPath(pos, *args[0], "while evaluating the first argument to builtins.outputOf");

std::string_view outputName = state.forceStringNoCtx(*args[1], pos, "while evaluating teh second argument to builtins.outputOf");

NixStringContextElem::Built path2 {
.drvPath = make_ref<SingleDerivedPath>(drvPath),
.output = std::string { outputName },
};

v.mkString(
DownstreamPlaceholder::fromSingleDerivedPathBuilt(path2).render(),
{ path2 });
}

static RegisterPrimOp primop_outputOf({
.name = "__outputOf",
.args = {"drv path", "output name"},
.doc = R"(
Return path (actually placeholder path) to the output of the given drv.
The drv path may itself be a placeholder, which permits chaining this primop.
For instance, `builtins.outputOf (builtins.outputOf myDrv "out) "out"`
will return a placeholder for the output of the output of `myDrv`,
interpreted as a derivation.
It may help to compare to the `->` operator in C, which can also by
chained. E.g. compare the above example to `drv->out->out`.
)",
.fun = prim_outputOf,
});

/*************************************************************
* Creating files
Expand Down
9 changes: 9 additions & 0 deletions tests/dyn-drv/dep-built-drv.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
#!/usr/bin/env bash

source common.sh

out1=$(nix-build ./text-hashed-output.nix -A hello --no-out-link)

clearStore

expectStderr 1 nix-build ./text-hashed-output.nix -A wrapper --no-out-link | grepQuiet "Dependencies on the outputs of dynamic derivations are not yet supported"
27 changes: 27 additions & 0 deletions tests/dyn-drv/eval-outputOf.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
#!/usr/bin/env bash

source ./common.sh

nix eval --impure --expr \
'with (import ./text-hashed-output.nix); let
a = hello.outPath;
b = builtins.outputOf (builtins.unsafeDiscardOutputDependency hello.drvPath) "out";
in builtins.trace a
(builtins.trace b
(assert a == b; null))'

nix eval --impure --expr \
'with (import ./text-hashed-output.nix); let
a = producingDrv.outPath;
b = builtins.outputOf (builtins.builtins.unsafeDiscardOutputDependency producingDrv.drvPath) "out";
in builtins.trace a
(builtins.trace b
(assert a == b; null))'

nix eval --impure --expr \
'with (import ./text-hashed-output.nix); let
a = builtins.outputOf producingDrv.out.outPath "out";
b = builtins.outputOf (builtins.outputOf (builtins.unsafeDiscardOutputDependency producingDrv.drvPath) "out") "out";
in builtins.trace a
(builtins.trace b
(assert a == b; null))'
4 changes: 3 additions & 1 deletion tests/dyn-drv/local.mk
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
dyn-drv-tests := \
$(d)/text-hashed-output.sh \
$(d)/recursive-mod-json.sh \
$(d)/build-built-drv.sh
$(d)/build-built-drv.sh \
$(d)/eval-outputOf.sh \
$(d)/dep-built-drv.sh

install-tests-groups += dyn-drv

Expand Down
10 changes: 10 additions & 0 deletions tests/dyn-drv/text-hashed-output.nix
Original file line number Diff line number Diff line change
Expand Up @@ -26,4 +26,14 @@ rec {
outputHashMode = "text";
outputHashAlgo = "sha256";
};
wrapper = mkDerivation {
name = "put-it-all-together";
buildCommand = ''
echo "Copying the output of the dynamic derivation"
cp -r ${builtins.outputOf producingDrv.out.outPath "out"} $out
'';
__contentAddressed = true;
outputHashMode = "recursive";
outputHashAlgo = "sha256";
};
}

0 comments on commit 0d26a45

Please sign in to comment.