Merge branch 'main' into tx-watch

.github/workflows/book.yml

@@ -28,7 +28,7 @@ jobs:
         with:
           name: book
           path: ./book
-      - uses: peaceiris/actions-gh-pages@068dc23d9710f1ba62e86896f84735d869951305 # v3.8.0
+      - uses: peaceiris/actions-gh-pages@373f7f263a76c20808c831209c920827a82a2847 # v3.9.3
         with:
           github_token: ${{ secrets.GITHUB_TOKEN }}
           publish_dir: ./book

README.md

@@ -12,12 +12,3 @@ It aims at being a reference point. If the text in this specification doesn't ma
 
 You are encouraged to open issues if you see issues or would like to suggest improvements to this specification, or if some elements are missing clarity.
 This specification can be modified (in a backwards-compatible way) by opening pull requests.
-
-## Note
-
-*This section should be removed in the future.*
-
-At the moment, this specification isn't implemented by Substrate or any other client implementation.
-It is planned to be implemented in Substrate once this specification has been fully fleshed out.
-
-[The smoldot repository](https://github.com/paritytech/smoldot) aims to implement this specification while it is still in progress, in order to experiment with it in practice.

src/SUMMARY.md

@@ -15,13 +15,11 @@
     - [archive_unstable_hashByHeight](api/archive_unstable_hashByHeight.md)
     - [archive_unstable_header](api/archive_unstable_header.md)
     - [archive_unstable_storage](api/archive_unstable_storage.md)
-    - [archive_unstable_storageContinue](api/archive_unstable_storageContinue.md)
   - [chainHead](api/chainHead.md)
     - [chainHead_unstable_body](api/chainHead_unstable_body.md)
     - [chainHead_unstable_call](api/chainHead_unstable_call.md)
     - [chainHead_unstable_continue](api/chainHead_unstable_continue.md)
     - [chainHead_unstable_follow](api/chainHead_unstable_follow.md)
-    - [chainHead_unstable_genesisHash](api/chainHead_unstable_genesisHash.md)
     - [chainHead_unstable_header](api/chainHead_unstable_header.md)
     - [chainHead_unstable_stopOperation](api/chainHead_unstable_stopOperation.md)
     - [chainHead_unstable_storage](api/chainHead_unstable_storage.md)
@@ -37,6 +35,9 @@
     - [sudo_unstable_p2pDiscover](api/sudo_unstable_p2pDiscover.md)
     - [sudo_unstable_pendingTransactions](api/sudo_unstable_pendingTransactions.md)
     - [sudo_unstable_version](api/sudo_unstable_version.md)
+  - [sudo_network]()
+    - [sudo_network_unstable_unwatch](api/sudo_network_unstable_unwatch.md)
+    - [sudo_network_unstable_watch](api/sudo_network_unstable_watch.md)
   - [sudo_sessionKeys]()
     - [sudo_sessionKeys_unstable_generate](api/sudo_sessionKeys_unstable_generate.md)
   - [transaction](api/transaction.md)

src/api.md

@@ -8,3 +8,4 @@ Any missing parameter, or parameter with an invalid format, should result in a J
 
 - "hexadecimal-encoded" designates a binary value encoded as hexadecimal. The value must either be empty, or start with `"0x"` and contain an even number of characters.
 - "SCALE-encoded" designates a value encoded using [the SCALE codec](https://docs.substrate.io/v3/advanced/scale-codec/).
+- "Merkle value" is described in the [Polkadot specification](https://spec.polkadot.network/chap-state#defn-merkle-value).

src/api/archive_unstable_body.md

@@ -2,7 +2,7 @@
 
 **Parameters**:
 
-- `hash`: String containing the hexadecimal-encoded hash of the header of the block whose body must be retrieved.
+- `hash`: String containing a hexadecimal-encoded hash of the header of the block whose body will be retrieved.
 
 **Return value**: If a block with that hash is found, an array of strings containing the hexadecimal-encoded SCALE-codec-encoded transactions in that block. If no block with that hash is found, `null`.
 

src/api/archive_unstable_finalizedHeight.md

@@ -2,7 +2,7 @@
 
 **Parameters**: *none*
 
-**Return value**: String containing the hexadecimal-encoded height of the current finalized block of the chain.
+**Return value**: An integer height of the current finalized block of the chain.
 
 The value returned by this function must only ever increase over time. In other words, if calling this function returns `N`, then calling it again later must return a value superior or equal to `N`.
 

src/api/archive_unstable_hashByHeight.md

@@ -2,9 +2,9 @@
 
 **Parameters**:
 
-- `height`: String containing an hexadecimal-encoded integer.
+- `height`: Integer representing the height of the block.
 
-**Return value**: Array (possibly empty) of strings containing an hexadecimal-encoded hash of a block header.
+**Return value**: Array (possibly empty) of strings containing a hexadecimal-encoded hash of a block header.
 
 The JSON-RPC client must find the blocks (zero, one, or more) whose height is the one passed as parameter. If the `height` is inferior or equal to the finalized block height, then only finalized blocks must be fetched and returned.
 

src/api/archive_unstable_storage.md

@@ -2,92 +2,120 @@
 
 **Parameters**:
 
-- `hash`: String containing an hexadecimal-encoded hash of the header of the block whose storage to fetch.
-- `key`: String containing the hexadecimal-encoded key to fetch in the storage.
+- `hash`: String containing a hexadecimal-encoded hash of the header of the block whose storage to fetch.
+- `items`: Array of objects. The structure of these objects is found below.
 - `childTrie`: `null` for main storage look-ups, or a string containing the hexadecimal-encoded key of the child trie of the "default" namespace.
-- `includeDescendants`: Boolean indicating whether the key-values of all the descendants of the `key` should be returned as well.
-
-**Return value**: String containing an opaque value representing the operation, or `null` if no block with that `hash` exists.
 
-The JSON-RPC server must obtain the value of the entry with the given `key` from the storage, either from the main trie of from `childTrie`. If `includeDescendants` is `true`, then the values of all the descendants must be obtained as well.
+Each element in `items` must be an object containing the following fields:
 
-If the block was previously returned by `archive_unstable_hashByHeight` at a height inferior or equal to the current finalized block height (as indicated by `archive_unstable_finalizedHeight`), then calling this method multiple times is guaranteed to always return non-null and always the same results.
-
-If the block was previously returned by `archive_unstable_hashByHeight` at a height strictly superior to the current finalized block height (as indicated by `archive_unstable_finalizedHeight`), then the block might "disappear" and calling this function might return `null` at any point.
+- `key`: String containing the hexadecimal-encoded key to fetch in the storage.
+- `type`: String equal to one of: `value`, `hash`, `closestDescendantMerkleValue`, `descendantsValues`, `descendantsHashes`.
+- `paginationStartKey`: This parameter is optional and should be a string containing the hexadecimal-encoded key from which the storage iteration should resume. This parameter is only valid in the context of `descendantsValues` and `descendantsHashes`.
 
-## Notifications format
+**Return value**: A JSON object.
 
-This function will later generate notifications in the following format:
+The JSON object returned by this function has the following format:
 
-```json
+```
 {
-    "jsonrpc": "2.0",
-    "method": "archive_unstable_storageEvent",
-    "params": {
-        "subscription": "...",
-        "result": ...
-    }
+    "result": [
+        {
+            ...
+        },
+        ...
+    ],
+    "discardedItems": ...
 }
 ```
 
-Where `subscription` is equal to the value returned by this function, and `result` can be one of:
+Where:
+
+- `result` contains a vector of JSON objects (possibly empty) that were found in the storage.
+- `discardedItems` is an integer indicating the number of items at the back of the array of the `items` parameters that couldn't be processed.
+
+The JSON objects in the `"result"` field can have one of the following formats based on their type:
 
-### item
+### Value
 
-```json
+```
 {
-    "event": "item",
     "key": "0x0000000...",
     "value": "0x0000000...",
 }
 ```
 
-Yields an item that was found in the storage.
+The JSON object corresponds to one of the requested items whose `type` was `"value"` or `"descendantsValues"`.
+
+If the `key` is not associated with a storage value in the trie, then no response is generated in the `"result"` vector for this item.
+
+Returned when the `type` of the query was `"value"`:
+
+- `key` is guaranteed to be equal to one of the `key`s provided.
+- `value` is a string containing the hexadecimal-encoded value of the storage entry.
 
-The `key` field is a string containing the hexadecimal-encoded key of the value that was found.
-If the `includeDescendants` parameter was `true`, this `key` is guaranteed to start with the `key` provided as parameter.
-If the `includeDescendants` parameter was `false`, then it is also guaranteed to be equal to the `key` provided as parameter.
+Returned when the `type` of the query was `"descendantsValues"`:
 
-The `value` field is a string containing the hexadecimal-encoded value of the storage item.
+- `key` is guaranteed to start with one of the `key`s provided.
+- `value` is a string containing the hexadecimal-encoded value of the storage entry.
 
-### waiting-for-continue
+### Hash
 
-```json
+```
 {
-    "event": "waiting-for-continue"
+    "key": "0x0000000...",
+    "hash": "0x0000000...",
 }
 ```
 
-The `waiting-for-continue` event is generated after at least one `"item"` event has been generated, and indicates that the JSON-RPC client must call `archive_unstable_storageContinue` before more events are generated.
+The JSON object corresponds to one of the requested items whose `type` was `"hash"` or `"descendantsHashes"`.
+
+If the `key` is not associated with a storage value in the trie, then no response is generated in the `"result"` vector for this item.
 
-This event only ever happens if the `includeDescendants` parameter was `true`.
+Returned when the `type` of the query was `"hash"`:
 
-While the JSON-RPC server is waiting for `archive_unstable_storageContinue` to be called, it can generate a `stop` event indicating that it can no longer proceed with that storage access.
+- `key` is guaranteed to be equal to one of the `key`s provided.
+- `hash` is a string containing the hexadecimal-encoded hash of the storage entry.
 
-### done
+Returned when the `type` of the query was `"descendantsHashes"`:
 
-```json
+- `key` is guaranteed to start with one of the `key`s provided.
+- `hash` is a string containing the hexadecimal-encoded cryptographic hash of the storage entry.
+
+
+### ClosestDescendantMerkleValue
+
+```
 {
-    "event": "done"
+    "key": "0x0000000...",
+    "closestDescendantMerkleValue": "0x000000..."
 }
 ```
 
-The `done` event indicates that everything went well and all values have been provided through `item` events in the past.
+The JSON object corresponds to one of the requested items whose `type` was `"closestDescendantMerkleValue"`.
 
-If no `item` event was yielded, then the storage doesn't contain a value at the given key.
+If the `key` doesn't exist in the trie, then the Merkle value of the closest descendant of `key` (including branch nodes) is provided. If `key` doesn't have any descendant in the trie, then no response is generated in the `"result"` vector for this item.
 
-No more event will be generated with this `subscription`.
+- `key` is guaranteed to be equal to one of the `key`s provided.
+- `closestDescendantMerkleValue` is the closest trie Merkle value of the `key`.
 
-### stop
+The trie node whose Merkle value is indicated in `closestDescendantMerkleValue` is not indicated, as determining the key of this node might incur an overhead for the JSON-RPC server. The Merkle value is equal to either the node value or the hash of the node value, as defined in the [Polkadot specification](https://spec.polkadot.network/chap-state#defn-merkle-value).
 
-```json
-{
-    "event": "stop"
-}
-```
+## Overview
+
+For each item in `items`, the JSON-RPC server must start obtaining the value of the entry with the given `key` from the storage, either from the main trie or from `childTrie`. If `type` is `descendantsValues` or `descendantsHashes`, then it must also obtain the values of all the descendants of the entry.
+
+For the purpose of storage requests, the trie root hash of the child tries of the storage can be found in the main trie at keys starting the bytes of the ASCII string `:child_storage:`. This behaviour is consistent with all the other storage-request-alike mechanisms of Polkadot and Substrate-based chains, such as host functions or libp2p network requests.
+
+If the height of the block hash provided is less than or equal to the current finalized block height (which can be obtained via archive_unstable_finalizedHeight), then calling this method with the same parameters will always return the same response.
+If the height of the block hash provided is greater than the current finalized block height, then the block might be pruned at any time and calling this method may return null.
+
+This function should be used when the target block is older than the blocks reported by `chainHead_unstable_follow`.
+Use `chainHead_unstable_storage` if instead you want to retrieve the storage of a block obtained by the `chainHead_unstable_follow`.
+
+If `items` contains multiple identical or overlapping queries, the JSON-RPC server can choose whether to merge or not the items in the result. For example, if the request contains two items with the same key, one with `hash` and one with `value`, the JSON-RPC server can choose whether to generate two `item` objects, one with the value and one with the hash, or only a single `item` object with both `hash` and `value` set.
 
-The `stop` event can be generated after a `waiting-for-continue` event in order to indicate that the JSON-RPC server can't continue. The JSON-RPC client should simply try again.
+It is allowed (but discouraged) for the JSON-RPC server to provide the same information multiple times in the result, for example providing the `value` field of the same `key` twice. Forcing the JSON-RPC server to de-duplicate items in the result might lead to unnecessary overhead.
 
-No more event will be generated with this `subscription`.
+## Possible errors
 
-**Note**: This event is generated in very niche situations, such as a node doing a clean shutdown of all its active subscriptions before shutting down.
+- A JSON-RPC error is generated if `type` isn't one of the allowed values (similarly to a missing parameter or an invalid parameter type).

src/api/chainHead_unstable_body.md

@@ -3,7 +3,7 @@
 **Parameters**:
 
 - `followSubscription`: An opaque string that was returned by `chainHead_unstable_follow`.
-- `hash`: String containing an hexadecimal-encoded hash of the header of the block whose body to fetch.
+- `hash`: String containing a hexadecimal-encoded hash of the header of the block whose body will be retrieved.
 
 **Return value**: A JSON object.
 
@@ -32,10 +32,10 @@ This return value indicates that the request has successfully started.
 
 This return value indicates the request couldn't be started because the server is overloaded, or that the `followSubscription` is invalid or stale.
 
-The JSON-RPC client should try again after an on-going [`chainHead_unstable_storage`], [`chainHead_unstable_body`], or [`chainHead_unstable_call`] operation finishes.
+The JSON-RPC client should try again after an on-going `chainHead_unstable_storage`, `chainHead_unstable_body`, or `chainHead_unstable_call` operation finishes.
 
-The JSON-RPC server must accept at least 16 concurrent operations for any given [`chainHead_unstable_follow`] subscription. In other words, as long as the JSON-RPC client makes sure that no more than 16 operations are in progress at any given item, it is guaranteed that all of its operations will be accepted by the JSON-RPC server.
-For this purpose, each item requested through [`chainHead_unstable_storage`] counts as one operation, and each call to [`chainHead_unstable_body`] and [`chainHead_unstable_call`] counts as one operation.
+The JSON-RPC server must accept at least 16 concurrent operations for any given `chainHead_unstable_follow` subscription. In other words, as long as the JSON-RPC client makes sure that no more than 16 operations are in progress at any given item, it is guaranteed that all of its operations will be accepted by the JSON-RPC server.
+For this purpose, each item requested through `chainHead_unstable_storage` counts as one operation, and each call to `chainHead_unstable_body` and `chainHead_unstable_call` counts as one operation.
 
 ## Overview
 
@@ -51,5 +51,6 @@ This function should be seen as a complement to `chainHead_unstable_follow`, all
 
 - If the networking part of the behaviour fails, then a `{"event": "operationInaccessible"}` notification is generated (as explained above).
 - If the `followSubscription` is invalid or stale, then `"result": "limitReached"` is returned (as explained above).
-- A JSON-RPC error is generated if the block hash passed as parameter doesn't correspond to any block that has been reported by `chainHead_unstable_follow`.
-- A JSON-RPC error is generated if the `followSubscription` is valid but the block hash passed as parameter has already been unpinned.
+- A JSON-RPC error with error code `-32801` is generated if the block hash passed as parameter doesn't correspond to any block that has been reported by `chainHead_unstable_follow`, or the block hash has been unpinned.
+- A JSON-RPC error with error code `-32602` is generated if one of the parameters doesn't correspond to the expected type (similarly to a missing parameter or an invalid parameter type).
+- A JSON-RPC error with error code `-32603` is generated if the JSON-RPC server cannot interpret the block (hardware issues, corrupted database, disk failure etc).