Skip to content
This repository has been archived by the owner on May 14, 2024. It is now read-only.

Commit

Permalink
Document string context (NixOS#8595)
Browse files Browse the repository at this point in the history
* Document string context

Now what we have enough primops, we can document how string contexts
work.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Co-authored-by: Felix Uhl <iFreilicht@users.noreply.github.com>
  • Loading branch information
5 people authored and pull[bot] committed May 9, 2024
1 parent fd3bad6 commit 70b1e5b
Show file tree
Hide file tree
Showing 4 changed files with 161 additions and 4 deletions.
1 change: 1 addition & 0 deletions doc/manual/src/SUMMARY.md.in
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
- [Language Constructs](language/constructs.md)
- [String interpolation](language/string-interpolation.md)
- [Lookup path](language/constructs/lookup-path.md)
- [String context](language/string-context.md)
- [Operators](language/operators.md)
- [Derivations](language/derivations.md)
- [Advanced Attributes](language/advanced-attributes.md)
Expand Down
11 changes: 11 additions & 0 deletions doc/manual/src/glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -218,6 +218,17 @@
- [output closure]{#gloss-output-closure}\
The [closure] of an [output path]. It only contains what is [reachable] from the output.

- [deriving path]{#gloss-deriving-path}

Deriving paths are a way to refer to [store objects][store object] that ar not yet [realised][realise].
This is necessary because, in general and particularly for [content-addressed derivations][content-addressed derivation], the [output path] of an [output] is not known in advance.
There are two forms:

- *constant*: just a [store path]
It can be made [valid][validity] by copying it into the store: from the evaluator, command line interface or another store.

- *output*: a pair of a [store path] to a [derivation] and an [output] name.

- [deriver]{#gloss-deriver}

The [store derivation] that produced an [output path].
Expand Down
134 changes: 134 additions & 0 deletions doc/manual/src/language/string-context.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
# String context

> **Note**
>
> This is an advanced topic.
> The Nix language is designed to be used without the programmer consciously dealing with string contexts or even knowing what they are.
A string in the Nix language is not just a sequence of characters like strings in other languages.
It is actually a pair of a sequence of characters and a *string context*.
The string context is an (unordered) set of *string context elements*.

The purpose of string contexts is to collect non-string values attached to strings via
[string concatenation](./operators.md#string-concatenation),
[string interpolation](./string-interpolation.md),
and similar operations.
The idea is that a user can combine together values to create a build instructions for derivations without manually keeping track of where they come from.
Then the Nix language implicitly does that bookkeeping to efficiently obtain the closure of derivation inputs.

> **Note**
>
> String contexts are *not* explicitly manipulated in idiomatic Nix language code.
String context elements come in different forms:

- [deriving path]{#string-context-element-derived-path}

A string context element of this type is a [deriving path](@docroot@/glossary.md#gloss-deriving-path).
They can be either of type [constant](#string-context-constant) or [output](#string-context-output), which correspond to the types of deriving paths.

- [Constant string context elements]{#string-context-constant}

> **Example**
>
> [`builtins.storePath`] creates a string with a single constant string context element:
>
> ```nix
> builtins.getContext (builtins.storePath "/nix/store/wkhdf9jinag5750mqlax6z2zbwhqb76n-hello-2.10")
> ```
> evaluates to
> ```nix
> {
> "/nix/store/wkhdf9jinag5750mqlax6z2zbwhqb76n-hello-2.10" = {
> path = true;
> };
> }
> ```
[deriving path]: @docroot@/glossary.md#gloss-deriving-path
[store path]: @docroot@/glossary.md#gloss-store-path
[`builtins.storePath`]: ./builtins.md#builtins-storePath
- [Output string context elements]{#string-context-output}
> **Example**
>
> The behavior of string contexts are best demonstrated with a built-in function that is still experimental: [`builtins.outputOf`].
> This example will *not* work with stable Nix!
>
> ```nix
> builtins.getContext
> (builtins.outputOf
> (builtins.storePath "/nix/store/fvchh9cvcr7kdla6n860hshchsba305w-hello-2.12.drv")
> "out")
> ```
> evaluates to
> ```nix
> {
> "/nix/store/fvchh9cvcr7kdla6n860hshchsba305w-hello-2.12.drv" = {
> outputs = [ "out" ];
> };
> }
> ```
[`builtins.outputOf`]: ./builtins.md#builtins-outputOf
- [*derivation deep*]{#string-context-element-derivation-deep}
*derivation deep* is an advanced feature intended to be used with the
[`exportReferencesGraph` derivation attribute](./advanced-attributes.html#adv-attr-exportReferencesGraph).
A *derivation deep* string context element is a derivation path, and refers to both its outputs and the entire build closure of that derivation:
all its outputs, all the other derivations the given derivation depends on, and all the outputs of those.
> **Example**
>
> The best way to illustrate *derivation deep* string contexts is with [`builtins.addDrvOutputDependencies`].
> Take a regular constant string context element pointing to a derivation, and transform it into a "Derivation deep" string context element.
>
> ```nix
> builtins.getContext
> (builtins.addDrvOutputDependencies
> (builtins.storePath "/nix/store/fvchh9cvcr7kdla6n860hshchsba305w-hello-2.12.drv"))
> ```
> evaluates to
> ```nix
> {
> "/nix/store/fvchh9cvcr7kdla6n860hshchsba305w-hello-2.12.drv" = {
> allOutputs = true;
> };
> }
> ```
[`builtins.addDrvOutputDependencies`]: ./builtins.md#builtins-addDrvOutputDependencies
[`builtins.unsafeDiscardOutputDependency`]: ./builtins.md#builtins-unsafeDiscardOutputDependency
## Inspecting string contexts
Most basically, [`builtins.hasContext`] will tell whether a string has a non-empty context.
When more granular information is needed, [`builtins.getContext`] can be used.
It creates an [attribute set] representing the string context, which can be inspected as usual.
[`builtins.hasContext`]: ./builtins.md#builtins-hasContext
[`builtins.getContext`]: ./builtins.md#builtins-getContext
[attribute set]: ./values.md#attribute-set
## Clearing string contexts
[`buitins.unsafeDiscardStringContext`](./builtins.md#builtins-unsafeDiscardStringContext) will make a copy of a string, but with an empty string context.
The returned string can be used in more ways, e.g. by operators that require the string context to be empty.
The requirement to explicitly discard the string context in such use cases helps ensure that string context elements are not lost by mistake.
The "unsafe" marker is only there to remind that Nix normally guarantees that dependencies are tracked, whereas the returned string has lost them.
## Constructing string contexts
[`builtins.appendContext`] will create a copy of a string, but with additional string context elements.
The context is specified explicitly by an [attribute set] in the format that [`builtins.hasContext`] produces.
A string with arbitrary contexts can be made like this:
1. Create a string with the desired string context elements.
(The contents of the string do not matter.)
2. Dump its context with [`builtins.getContext`].
3. Combine it with a base string and repeated [`builtins.appendContext`] calls.
[`builtins.appendContext`]: ./builtins.md#builtins-appendContext
19 changes: 15 additions & 4 deletions src/libexpr/primops/context.cc
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,11 @@ static void prim_unsafeDiscardStringContext(EvalState & state, const PosIdx pos,

static RegisterPrimOp primop_unsafeDiscardStringContext({
.name = "__unsafeDiscardStringContext",
.arity = 1,
.fun = prim_unsafeDiscardStringContext
.args = {"s"},
.doc = R"(
Discard the [string context](@docroot@/language/string-context.md) from a value that can be coerced to a string.
)",
.fun = prim_unsafeDiscardStringContext,
});


Expand Down Expand Up @@ -75,7 +78,11 @@ static RegisterPrimOp primop_unsafeDiscardOutputDependency({
.name = "__unsafeDiscardOutputDependency",
.args = {"s"},
.doc = R"(
Create a copy of the given string where every "derivation deep" string context element is turned into a constant string context element.
Create a copy of the given string where every
[derivation deep](@docroot@/language/string-context.md#string-context-element-derivation-deep)
string context element is turned into a
[constant](@docroot@/language/string-context.md#string-context-element-constant)
string context element.
This is the opposite of [`builtins.addDrvOutputDependencies`](#builtins-addDrvOutputDependencies).
Expand Down Expand Up @@ -137,7 +144,11 @@ static RegisterPrimOp primop_addDrvOutputDependencies({
.name = "__addDrvOutputDependencies",
.args = {"s"},
.doc = R"(
Create a copy of the given string where a single constant string context element is turned into a "derivation deep" string context element.
Create a copy of the given string where a single
[constant](@docroot@/language/string-context.md#string-context-element-constant)
string context element is turned into a
[derivation deep](@docroot@/language/string-context.md#string-context-element-derivation-deep)
string context element.
The store path that is the constant string context element should point to a valid derivation, and end in `.drv`.
Expand Down

0 comments on commit 70b1e5b

Please sign in to comment.