Build guide: zome functions and lifecycle callbacks (#512)

* lifecycle callbacks and zome functions pages

* add new pages to navs

* link up all references to two new pages

* remove redundant descriptions of lifecycles, add examples for relaxed
ordering

* add under-the-hood for CreateLink and DeleteLink

* add snapshotted to dict

* fix broken JSON

* fix broken links

* fix hdk_entry_defs, whoops

* add example for post_commit

* simplify/elaborate post_commit example

* improve language around callbacks and lifecycle hooks, plus a couple tiny edits

* little bit more of the same

* test/fix all code samples in callbacks page

* small edits to callbacks page

* link from identifiers to post_commit page (plus a typo fix)

* reference/further reading for zomes and callbacks pages

* make post_commit infallible

* don't mention entry_defs callback

* Apply suggestions from code review

Co-authored-by: ThetaSinner <ThetaSinner@users.noreply.github.com>

* warn about spam/abuse vectors for unrestricted grants and recv_remote_signal

* missing semicolon

it's always a missing semicolon

Co-authored-by: ThetaSinner <ThetaSinner@users.noreply.github.com>

* simplify users anchor pattern

* simplify language about users anchor pattern

* fix some mistakes re:  callback

* fix typo

* remove reject-all validation example and bad advice

* fix broken fragment identifier

* change wording re: remote signal routing

* correct mistakes about the definition of genesis records

* make genesis_self_check less scary

* tiny text tweaks

---------

Co-authored-by: ThetaSinner <ThetaSinner@users.noreply.github.com>

.cspell/words-that-should-exist.txt

@@ -25,6 +25,7 @@ runtimes
 sandboxed
 sandboxing
 scaffolder
+snapshotted
 spacebar
 todo
 todos

code_test_4/web-happ.yaml

@@ -0,0 +1,6 @@
+manifest_version: '1'
+name: blup
+ui:
+  bundled: ./path/to/my/ui.zip
+happ_manifest:
+  bundled: ./path/to/my/happ-bundle.happ

src/pages/_data/navigation/mainNav.json5

@@ -26,7 +26,10 @@
     },
     { title: "Build", url: "/build/", children: [
         { title: "Application Structure", url: "/build/application-structure/", children: [
-          { title: "Zomes", url: "/build/zomes/" },
+          { title: "Zomes", url: "/build/zomes/", children: [
+            { title: "Lifecycle Events and Callbacks", url: "/build/callbacks-and-lifecycle-hooks/" },
+            { title: "Zome Functions", url: "/build/zome-functions/" },
+          ] },
         ]},
         { title: "Working with Data", url: "/build/working-with-data/", children: [
           { title: "Identifiers", url: "/build/identifiers/" },

src/pages/build/application-structure.md

@@ -7,6 +7,8 @@ title: Application Structure
 
 * Application Structure (this page)
     * [Zomes](/build/zomes/) --- integrity vs coordinator, how to structure and compile
+        * [Lifecycle Events and Callbacks](/build/callbacks-and-lifecycle-hooks/) --- writing functions that respond to events in a hApp's lifecycle
+        * [Zome Functions](/build/zome-functions/) --- writing your hApp's back-end API
     * DNAs (coming soon) --- what they're used for, how to specify and bundle
     * hApps (coming soon) --- headless vs UI-based, how to bundle and distribute
 :::

src/pages/build/entries.md

@@ -43,13 +43,13 @@ pub struct Movie {
 
 This implements a host of [`TryFrom` conversions](https://docs.rs/hdi/latest/src/hdi/entry.rs.html#120-209) that your type is expected to implement, along with serialization and deserialization functions.
 
-In order to dispatch validation to the proper integrity zome, Holochain needs to know about all the entry types that your integrity zome defines. This is done by implementing a callback in your zome called `entry_defs`, but it's easier to use the [`hdi::prelude::hdk_entry_defs`](https://docs.rs/hdi/latest/hdi/prelude/attr.hdk_entry_defs.html) macro on an enum of all the entry types:
+In order to dispatch validation to the proper integrity zome, Holochain needs to know about all the entry types that your integrity zome defines. This is done by implementing a callback in your zome called `entry_defs`, but it's easier to use the [`hdi::prelude::hdk_entry_types`](https://docs.rs/hdi/latest/hdi/prelude/attr.hdk_entry_types.html) macro on an enum of all the entry types:
 
 ```rust
 use hdi::prelude::*;
 
-#[hdk_entry_defs]
-// This macro is required by hdk_entry_defs.
+#[hdk_entry_types]
+// This macro is required by hdk_entry_types.
 #[unit_enum(UnitEntryTypes)]
 enum EntryTypes {
     Director(Director),
@@ -70,7 +70,7 @@ Each variant in the enum should hold the Rust type that corresponds to it, and i
 ```rust
 use hdi::prelude::*;
 
-#[hdk_entry_defs]
+#[hdk_entry_types]
 #[unit_enum(UnitEntryTypes)]
 enum EntryTypes {
     Director(Director),
@@ -91,7 +91,7 @@ enum EntryTypes {
 
 Most of the time you'll want to define your create, read, update, and delete (CRUD) functions in a [**coordinator zome**](/resources/glossary/#coordinator-zome) rather than the integrity zome that defines it. This is because a coordinator zome is easier to update in the wild than an integrity zome.
 
-Create an entry by calling [`hdk::prelude::create_entry`](https://docs.rs/hdk/latest/hdk/entry/fn.create_entry.html). If you used `hdk_entry_helper` and `hdk_entry_defs` macro in your integrity zome (see [Define an entry type](#define-an-entry-type)), you can use the entry types enum you defined, and the entry will be serialized and have the correct integrity zome and entry type indexes added to it.
+Create an entry by calling [`hdk::prelude::create_entry`](https://docs.rs/hdk/latest/hdk/entry/fn.create_entry.html). If you used `hdk_entry_helper` and `hdk_entry_types` macro in your integrity zome (see [Define an entry type](#define-an-entry-type)), you can use the entry types enum you defined, and the entry will be serialized and have the correct integrity zome and entry type indexes added to it.
 
 ```rust
 use hdk::prelude::*;
@@ -113,42 +113,65 @@ let movie = Movie {
 let create_action_hash = create_entry(
     // The value you pass to `create_entry` needs a lot of traits to tell
     // Holochain which entry type from which integrity zome you're trying to
-    // create. The `hdk_entry_defs` macro will have set this up for you, so all
+    // create. The `hdk_entry_types` macro will have set this up for you, so all
     // you need to do is wrap your movie in the corresponding enum variant.
     &EntryTypes::Movie(movie),
 )?;
 ```
 
+### Create with relaxed chain top ordering
+
+If your entry doesn't have any dependencies on other data, you can use [relaxed chain top ordering](/build/zome-functions/#relaxed-chain-top-ordering) to prevent possible transaction rollbacks (we'll let that page explain when this could happen and how to design around it).
+
+To use this feature, you'll need to use the more low-level [`create`](https://docs.rs/hdk/latest/hdk/entry/fn.create.html) host function, which requires you to build a more complex input. This example batches updates to director entries, which don't have to reference other data including each other, so they're a good candidate for relaxed ordering.
+
+```rust
+use movie_integrity::{Director, EntryTypes};
+use hdk::prelude::*;
+
+let directors = vec![/* construct a vector of `Director` structs here */];
+for director in directors.iter() {
+    // To specify chain top ordering other than the default Strict, we
+    // need to use the `create` host function which requires a bit more
+    // setup.
+    let entry = EntryTypes::Director(director);
+    let ScopedEntryDefIndex {
+        zome_index,
+        zome_type: entry_def_index,
+    } = (&entry).try_into()?;
+    let visibility = EntryVisibility::from(&entry);
+    let create_input = CreateInput::new(
+        EntryDefLocation::app(zome_index, entry_def_index),
+        visibility,
+        entry.try_into()?,
+        ChainTopOrdering::Relaxed,
+    );
+    create(create_input))?;
+}
+```
+
 ### Create under the hood
 
-When the client calls a zome function that calls `create_entry`, Holochain does the following:
+When a zome function calls `create`, Holochain does the following:
 
-1. Prepare a **scratch space** for making an atomic set of changes to the source chain for the agent's cell.
-2. Build an entry creation action called [`Create`](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/action/struct.Create.html) that includes:
+1. Build an entry creation action called [`Create`](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/action/struct.Create.html) that includes:
     * the author's public key,
     * a timestamp,
     * the action's sequence in the source chain and the previous action's hash,
     * the entry type (integrity zome index and entry type index), and
     * the hash of the serialized entry data.
     <!-- * a calculated weight value for rate limiting -->
-3. Write the `Create` action and the serialized entry data to the scratch space.
-4. Return the `ActionHash` of the `Create` action to the calling zome function. (At this point, the action hasn't been persisted to the source chain.)
-5. Wait for the zome function to complete.
-6. Convert the action to DHT operations.
-7. Run the validation callback for all DHT operations.
-    * If successful, continue.
-    * If unsuccessful, return the validation error to the client instead of the zome function's return value.
-8. Compare the scratch space against the actual state of the source chain.
-    * If the source chain has diverged from the scratch space, and the write specified strict chain top ordering, the scratch space is discarded and a `HeadMoved` error is returned to the caller.
-    * If the source chain has diverged and the write specified relaxed chain top ordering, the data in the scratch space is 'rebased' on top of the new source chain state as it's being written.
-    * If the source chain has not diverged, the data in the scratch space is written to the source chain state.
-9. Return the zome function's return value to the client.
-10. In the background, publish all newly created DHT operations to their respective authority agents.
+2. Write the `Create` action and the serialized entry data to the scratch space.
+3. Return the `ActionHash` of the pending `Create` action to the calling zome function.
+
+At this point, the action hasn't been persisted to the source chain. Read the [zome function call lifecycle](/build/zome-functions/#zome-function-call-lifecycle) section to find out more about persistence.
 
 ## Update an entry
 
 Update an entry creation action by calling [`hdk::prelude::update_entry`](https://docs.rs/hdk/latest/hdk/prelude/fn.update_entry.html) with the old action hash and the new entry data:
 
+<!-- FIXME: this won't compile as written; write something that gets the old movie and its action hash from somewhere. -->
+
 ```rust
 use hdk::prelude::*;
 use movie_integrity::*;
@@ -174,28 +197,57 @@ let update_action_hash = update_entry(
 
 An [`Update` action](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/action/struct.Update.html) operates on an entry creation action (either a `Create` or an `Update`), not just an entry by itself. It also doesn't remove the original data from the DHT; instead, it gets attached to both the original entry and its entry creation action. As an entry creation action itself, it references the hash of the new entry so it can be retrieved from the DHT.
 
+### Update with relaxed chain top ordering
+
+If you want to use relaxed chain top ordering, use the low-level [`update`](https://docs.rs/hdk/latest/hdk/entry/fn.update.html) instead:
+
+```rust
+use hdk::prelude::*;
+use movie_integrity::*;
+use chrono::DateTime;
+
+// A simple struct to keep a mapping to an old director action hash to new
+// entry content.
+struct OldToNewDirector {
+    old_action_hash: ActionHash,
+    new_entry: Director,
+}
+
+let old_to_new_directors = vec![
+    /* construct a vector of old director action hashes and updated content */
+];
+
+for director in old_to_new_directors.iter() {
+    // To specify chain top ordering other than the default Strict, we
+    // need to use the `create` host function which requires a bit more
+    // setup.
+    let entry = EntryTypes::Director(&director.new_entry);
+    let ScopedEntryDefIndex {
+        zome_index,
+        zome_type: entry_def_index,
+    } = (&entry).try_into()?;
+    let visibility = EntryVisibility::from(&entry);
+    let update_input: UpdateInput = {
+        original_action_address: &director.old_action_hash,
+        entry: entry.try_into()?,
+        chain_top_ordering: ChainTopOrdering::Relaxed,
+    };
+    update(update_input)?;
+}
+```
+
 ### Update under the hood
 
-Calling `update_entry` does the following:
+When a zome function calls `create`, Holochain does the following:
 
-1. Prepare a **scratch space** for making an atomic set of changes to the source chain for the agent's cell.
-2. Build an `Update` action that contains everything in a `Create` action, plus:
+1. Build an entry creation action called `Update` that contains everything in a `Create` action, plus:
     * the hash of the original action and
     * the hash of the original action's serialized entry data.
-    (Note that the entry type is automatically retrieved from the original action.)
-3. Write an `Update` action to the scratch space.
-4. Return the `ActionHash` of the `Update` action to the calling zome function. (At this point, the action hasn't been persisted to the source chain.)
-5. Wait for the zome function to complete.
-6. Convert the action to DHT operations.
-7. Run the validation callback for all DHT operations.
-    * If successful, continue.
-    * If unsuccessful, return the validation error to the client instead of the zome function's return value.
-8. Compare the scratch space against the actual state of the source chain.
-    * If the source chain has diverged from the scratch space, and the write specified strict chain top ordering, the scratch space is discarded and a `HeadMoved` error is returned to the caller.
-    * If the source chain has diverged and the write specified relaxed chain top ordering, the data in the scratch space is 'rebased' on top of the new source chain state as it's being written.
-    * If the source chain has not diverged, the data in the scratch space is written to the source chain state.
-9. Return the zome function's return value to the client.
-10. In the background, publish all newly created DHT operations to their respective authority agents.
+    (Note that the entry type and visibility are automatically retrieved from the original action.)
+2. Write the `Update` action and the serialized entry data to the scratch space.
+3. Return the `ActionHash` of the pending `Update` action to the calling zome function.
+
+As with `Create`, the action hasn't been persisted to the source chain yet. Read the [zome function call lifecycle](/build/zome-functions/#zome-function-call-lifecycle) section to find out more about persistence.
 
 ### Update patterns
 
@@ -248,24 +300,31 @@ In the future we plan to include a 'purge' functionality. This will give agents
 
 Remember that, even once purge is implemented, it is impossible to force another person to delete data once they have seen it. Be deliberate about choosing what data becomes public in your app.
 
+### Delete with relaxed chain top ordering
+
+To delete with relaxed chain top ordering, use the low-level [`delete`](https://docs.rs/hdk/latest/hdk/entry/fn.delete.html) instead.
+
+```rust
+use hdk::prelude::*;
+
+let actions_to_delete: Vec<ActionHash> = vec![/* construct vector here */];
+for action in actions_to_delete.iter() {
+    let delete_input: DeleteInput = {
+        deletes_action_hash: action,
+        chain_top_ordering: ChainTopOrdering::Relaxed,
+    }
+    delete(delete_input)?;
+}
+```
+
 ### Delete under the hood
 
 Calling `delete_entry` does the following:
 
-1. Prepare a **scratch space** for making an atomic set of changes to the source chain for the agent's cell.
-2. Write a `Delete` action to the scratch space.
-3. Return the `ActionHash` of the `Delete` action to the calling zome function. (At this point, the action hasn't been persisted to the source chain.)
-4. Wait for the zome function to complete.
-5. Convert the action to DHT operations.
-6. Run the validation callback for all DHT operations.
-    * If successful, continue.
-    * If unsuccessful, return the validation error to the client instead of the zome function's return value.
-7. Compare the scratch space against the actual state of the source chain.
-    * If the source chain has diverged from the scratch space, and the write specified strict chain top ordering, the scratch space is discarded and a `HeadMoved` error is returned to the caller.
-    * If the source chain has diverged and the write specified relaxed chain top ordering, the data in the scratch space is 'rebased' on top of the new source chain state as it's being written.
-    * If the source chain has not diverged, the data in the scratch space is written to the source chain state.
-8. Return the zome function's return value to the client.
-9. In the background, publish all newly created DHT operations to their respective authority agents.
+1. Write a `Delete` action to the scratch space.
+2. Return the pending `ActionHash` of the `Delete` action to the calling zome function.
+
+As with `Create` and `Delete`, the action hasn't been persisted to the source chain yet. Read the [zome function call lifecycle](/build/zome-functions/#zome-function-call-lifecycle) section to find out more about persistence.
 
 ## Identifiers on the DHT
 
@@ -404,7 +463,7 @@ There are some community-maintained libraries that offer opinionated and high-le
 ## Reference
 
 * [`hdi::prelude::hdk_entry_helper`](https://docs.rs/hdi/latest/hdi/attr.hdk_entry_helper.html)
-* [`hdi::prelude::hdk_entry_defs`](https://docs.rs/hdi/latest/hdi/prelude/attr.hdk_entry_defs.html)
+* [`hdi::prelude::hdk_entry_types`](https://docs.rs/hdi/latest/hdi/prelude/attr.hdk_entry_types.html)
 * [`hdi::prelude::entry_def`](https://docs.rs/hdi/latest/hdi/prelude/entry_def/index.html)
 * [`hdk::prelude::create_entry`](https://docs.rs/hdk/latest/hdk/entry/fn.create_entry.html)
 * [`hdk::prelude::update_entry`](https://docs.rs/hdk/latest/hdk/entry/fn.update_entry.html)

src/pages/build/identifiers.md

@@ -226,14 +226,14 @@ Read more about [entries](/build/entries/) and [links](/build/links-paths-and-an
 There are a few important things to know about action hashes:
 
 * You can't know an action's hash until you've written the action, because the action contains the current system time at the moment of writing.
-* When you write an action, you can specify "relaxed chain top ordering". We won't go into the details here, <!-- TODO: fill this in when I write about zome call lifecycles -->but when you use it, the action hash may change after the function completes.
+* When you write an action, you can specify "relaxed chain top ordering". We won't go into the details here (see [the section in the Zome Functions page](/build/zome-functions/#relaxed-chain-top-ordering), but when you use it, the action hash may change after the function completes.
 * A function that writes actions is _atomic_, which means that all writes fail or succeed together.
 
 Because of these three things, it's unsafe to depend on the value or even existence of an action hash within the same function that writes it. Here are some 'safe usage' notes:
 
 * You may safely use the hash of an action you've just written as data in another action in the same function (e.g., in a link or an entry that contains the hash in a field), as long as you're not using relaxed chain top ordering.
 * The same is also true of action hashes in your function's return value.
-* Don't communicate the action hash with the front end, another cell, or another peer on the network via a remote function call or [signal](/concepts/9_signals/) _from within the same function that writes it_, in case the write fails. Instead, do your communicating in a follow-up step. The easiest way to do this is by implementing [a callback called `post_commit`](https://docs.rs/hdk/latest/hdk/#internal-callbacks) which receives a vector of all the actions that the function wrote.
+* Don't communicate the action hash with the front end, another cell, or another peer on the network via a remote function call or [signal](/concepts/9_signals/) _from within the same function that writes it_, in case the write fails. Instead, do your communicating in a follow-up step. The easiest way to do this is by [implementing a callback called `post_commit`](/build/callbacks-and-lifecycle-hooks/#define-a-post-commit-callback) which receives a vector of all the actions that the function wrote.
 
 <!-- TODO: write about the front end -->
 

src/pages/build/index.md

@@ -27,6 +27,8 @@ Now that you've got some basic concepts and the terms we use for them, it's time
 ::: topic-list
 * [Overview](/build/application-structure/) --- an overview of Holochain's modularity and composability units
     * [Zomes](/build/zomes/) --- integrity vs coordinator, how to structure and compile
+        * [Lifecycle Events and Callbacks](/build/callbacks-and-lifecycle-hooks/) --- writing functions that respond to events in a hApp's lifecycle
+        * [Zome Functions](/build/zome-functions/) --- writing your hApp's back-end API
 :::
 
 ## Working with data