Merge pull request #9348 from obsidiansystems/json-formats

Document JSON formats

doc/manual/src/SUMMARY.md.in

@@ -104,6 +104,9 @@
     - [Channels](command-ref/files/channels.md)
     - [Default Nix expression](command-ref/files/default-nix-expression.md)
 - [Architecture and Design](architecture/architecture.md)
+- [JSON Formats](json/index.md)
+  - [Store Object Info](json/store-object-info.md)
+  - [Derivation](json/derivation.md)
 - [Protocols](protocols/index.md)
   - [Serving Tarball Flakes](protocols/tarball-fetcher.md)
   - [Derivation "ATerm" file format](protocols/derivation-aterm.md)

doc/manual/src/glossary.md

@@ -127,7 +127,7 @@
   non-[fixed-output](#gloss-fixed-output-derivation)
   derivation.
 
-- [output-addressed store object]{#gloss-output-addressed-store-object}
+- [content-addressed store object]{#gloss-content-addressed-store-object}
 
   A [store object] whose [store path] is determined by its contents.
   This includes derivations, the outputs of [content-addressed derivations](#gloss-content-addressed-derivation), and the outputs of [fixed-output derivations](#gloss-fixed-output-derivation).

doc/manual/src/json/derivation.md

@@ -0,0 +1,71 @@
+# Derivation JSON Format
+
+> **Warning**
+>
+> This JSON format is currently
+> [**experimental**](@docroot@/contributing/experimental-features.md#xp-feature-nix-command)
+> and subject to change.
+
+The JSON serialization of a
+[derivations](@docroot@/glossary.md#gloss-store-derivation)
+is a JSON object with the following fields:
+
+* `name`:
+  The name of the derivation.
+  This is used when calculating the store paths of the derivation's outputs.
+
+* `outputs`:
+  Information about the output paths of the derivation.
+  This is a JSON object with one member per output, where the key is the output name and the value is a JSON object with these fields:
+
+  * `path`: The output path.
+
+  * `hashAlgo`:
+    For fixed-output derivations, the hashing algorithm (e.g. `sha256`), optionally prefixed by `r:` if `hash` denotes a NAR hash rather than a flat file hash.
+
+  * `hash`:
+    For fixed-output derivations, the expected content hash in base-16.
+
+  > **Example**
+  >
+  > ```json
+  > "outputs": {
+  >   "out": {
+  >     "path": "/nix/store/2543j7c6jn75blc3drf4g5vhb1rhdq29-source",
+  >     "hashAlgo": "r:sha256",
+  >     "hash": "6fc80dcc62179dbc12fc0b5881275898f93444833d21b89dfe5f7fbcbb1d0d62"
+  >   }
+  > }
+  > ```
+
+* `inputSrcs`:
+  A list of store paths on which this derivation depends.
+
+* `inputDrvs`:
+  A JSON object specifying the derivations on which this derivation depends, and what outputs of those derivations.
+
+  > **Example**
+  >
+  > ```json
+  > "inputDrvs": {
+  >   "/nix/store/6lkh5yi7nlb7l6dr8fljlli5zfd9hq58-curl-7.73.0.drv": ["dev"],
+  >   "/nix/store/fn3kgnfzl5dzym26j8g907gq3kbm8bfh-unzip-6.0.drv": ["out"]
+  > }
+  > ```
+
+  specifies that this derivation depends on the `dev` output of `curl`, and the `out` output of `unzip`.
+
+* `system`:
+  The system type on which this derivation is to be built
+  (e.g. `x86_64-linux`).
+
+* `builder`:
+  The absolute path of the program to be executed to run the build.
+  Typically this is the `bash` shell
+  (e.g. `/nix/store/r3j288vpmczbl500w6zz89gyfa4nr0b1-bash-4.4-p23/bin/bash`).
+
+* `args`:
+  The command-line arguments passed to the `builder`.
+
+* `env`:
+  The environment passed to the `builder`.

doc/manual/src/json/store-object-info.md

@@ -0,0 +1,97 @@
+# Store object info JSON format
+
+> **Warning**
+>
+> This JSON format is currently
+> [**experimental**](@docroot@/contributing/experimental-features.md#xp-feature-nix-command)
+> and subject to change.
+
+Info about a [store object].
+
+* `path`:
+
+  [Store path][store path] to the given store object.
+
+* `narHash`:
+
+  Hash of the [file system object] part of the store object when serialized as a [Nix Archive](#gloss-nar).
+
+* `narSize`:
+
+  Size of the [file system object] part of the store object when serialized as a [Nix Archive](#gloss-nar).
+
+* `references`:
+
+  An array of [store paths][store path], possibly including this one.
+
+* `ca` (optional):
+
+  Content address of this store object's file system object, used to compute its store path.
+
+[store path]: @docroot@/glossary.md#gloss-store-path
+[file system object]: @docroot@/store/file-system-object.md
+
+## Impure fields
+
+These are not intrinsic properties of the store object.
+In other words, the same store object residing in different store could have different values for these properties.
+
+* `deriver` (optional):
+
+  The path to the [derivation] from which this store object is produced.
+
+  [derivation]: @docroot@/glossary.md#gloss-store-derivation
+
+* `registrationTime` (optional):
+
+  When this derivation was added to the store.
+
+* `ultimate` (optional):
+
+  Whether this store object is trusted because we built it ourselves, rather than substituted a build product from elsewhere.
+
+* `signatures` (optional):
+
+  Signatures claiming that this store object is what it claims to be.
+  Not relevant for [content-addressed] store objects,
+  but useful for [input-addressed] store objects.
+
+  [content-addressed]: @docroot@/glossary.md#gloss-content-addressed-store-object
+  [input-addressed]: @docroot@/glossary.md#gloss-input-addressed-store-object
+
+### `.narinfo` extra fields
+
+This meta data is specific to the "binary cache" family of Nix store types.
+This information is not intrinsic to the store object, but about how it is stored.
+
+* `url`:
+
+  Where to download a compressed archive of the file system objects of this store object.
+
+* `compression`:
+
+  The compression format that the archive is in.
+
+* `fileHash`:
+
+  A digest for the compressed archive itself, as opposed to the data contained within.
+
+* `fileSize`:
+
+  The size of the compressed archive itself.
+
+## Computed closure fields
+
+These fields are not stored at all, but computed by traverising the other other fields across all the store objects in a [closure].
+
+* `closureSize`:
+
+  The total size of the compressed archive itself for this object, and the compressed archive of every object in this object's [closure].
+
+### `.narinfo` extra fields
+
+* `closureSize`:
+
+  The total size of this store object and every other object in its [closure].
+
+[closure]: @docroot@/glossary.md#gloss-closure

src/libstore/globals.hh

@@ -635,7 +635,7 @@ public:
 
           - the store object has been signed using a key in the trusted keys list
           - the [`require-sigs`](#conf-require-sigs) option has been set to `false`
-          - the store object is [output-addressed](@docroot@/glossary.md#gloss-output-addressed-store-object)
+          - the store object is [content-addressed](@docroot@/glossary.md#gloss-content-addressed-store-object)
         )",
         {"binary-cache-public-keys"}};
 

src/nix/derivation-add.md

@@ -9,10 +9,11 @@ Store derivations are used internally by Nix. They are store paths with
 extension `.drv` that represent the build-time dependency graph to which
 a Nix expression evaluates.
 
-[store derivation]: ../../glossary.md#gloss-store-derivation
 
-The JSON format is documented under the [`derivation show`] command.
+[store derivation]: @docroot@/glossary.md#gloss-store-derivation
 
-[`derivation show`]: ./nix3-derivation-show.md
+`nix derivation add` takes a single derivation in the following format:
+
+{{#include ../../json/derivation.md}}
 
 )""

src/nix/derivation-show.md

@@ -5,8 +5,6 @@ R""(
 * Show the [store derivation] that results from evaluating the Hello
   package:
 
-  [store derivation]: ../../glossary.md#gloss-store-derivation
-
   ```console
   # nix derivation show nixpkgs#hello
   {
@@ -48,62 +46,12 @@ a Nix expression evaluates.
 By default, this command only shows top-level derivations, but with
 `--recursive`, it also shows their dependencies.
 
-The JSON output is a JSON object whose keys are the store paths of the
-derivations, and whose values are a JSON object with the following
-fields:
-
-* `name`: The name of the derivation. This is used when calculating the
-  store paths of the derivation's outputs.
-
-* `outputs`: Information about the output paths of the
-  derivation. This is a JSON object with one member per output, where
-  the key is the output name and the value is a JSON object with these
-  fields:
-
-  * `path`: The output path.
-  * `hashAlgo`: For fixed-output derivations, the hashing algorithm
-    (e.g. `sha256`), optionally prefixed by `r:` if `hash` denotes a
-    NAR hash rather than a flat file hash.
-  * `hash`: For fixed-output derivations, the expected content hash in
-    base-16.
-
-  Example:
-
-  ```json
-  "outputs": {
-    "out": {
-      "path": "/nix/store/2543j7c6jn75blc3drf4g5vhb1rhdq29-source",
-      "hashAlgo": "r:sha256",
-      "hash": "6fc80dcc62179dbc12fc0b5881275898f93444833d21b89dfe5f7fbcbb1d0d62"
-    }
-  }
-  ```
-
-* `inputSrcs`: A list of store paths on which this derivation depends.
-
-* `inputDrvs`: A JSON object specifying the derivations on which this
-  derivation depends, and what outputs of those derivations. For
-  example,
-
-  ```json
-  "inputDrvs": {
-    "/nix/store/6lkh5yi7nlb7l6dr8fljlli5zfd9hq58-curl-7.73.0.drv": ["dev"],
-    "/nix/store/fn3kgnfzl5dzym26j8g907gq3kbm8bfh-unzip-6.0.drv": ["out"]
-  }
-  ```
-
-  specifies that this derivation depends on the `dev` output of
-  `curl`, and the `out` output of `unzip`.
-
-* `system`: The system type on which this derivation is to be built
-  (e.g. `x86_64-linux`).
+[store derivation]: @docroot@/glossary.md#gloss-store-derivation
 
-* `builder`: The absolute path of the program to be executed to run
-  the build. Typically this is the `bash` shell
-  (e.g. `/nix/store/r3j288vpmczbl500w6zz89gyfa4nr0b1-bash-4.4-p23/bin/bash`).
+`nix derivation show` outputs a JSON map of [store path]s to derivations in the following format:
 
-* `args`: The command-line arguments passed to the `builder`.
+[store path]: @docroot@/glossary.md#gloss-store-path
 
-* `env`: The environment passed to the `builder`.
+{{#include ../../json/derivation.md}}
 
 )""