docs(swingset): Improve "delivery" documentation #4717

gibson042 · 2022-03-02T05:21:51Z

Description

General improvements to documentation (fixing typos, improving internal consistency, clarifying concepts, etc.). It might make sense to review commit by commit and/or in rich-text mode.

Security Considerations

n/a

Documentation Considerations

See above.

Testing Considerations

n/a

…romise table"

…rences

gibson042 · 2022-03-02T05:22:40Z

packages/SwingSet/docs/delivery.md

@@ -146,7 +149,7 @@ enum CapSlot {
    Object(ObjectID),
 }
 struct CapData {
-    body: Vec<u8>,
+    body: String,


Vec<u8> appeared to be a typo, especially with other page content referring to body as a string and providing examples of such.

Hm, I believe my original attempt at a Rust implementation used Vec<u8> (the Rust name for bytes, as opposed to unicode), because I think that's what comes out of the serde JSON+UTF8 serializer and it's what can be sent over a wire. But String (i.e. the Rust name for unicode strings) is more meaningful to a JS audience, and is equally plausible, and is closer to what we actually do now in JS, depending upon where you look (there's one point where syscall.send includes a string .body, then there's a later point where the whole syscall gets serialized into bytes to send over the wire). So String sounds fine.

(#2589 is a long-neglected aspiration to move away from JSON and use something denser, maybe CBOR, which would also accomodate bytes arguments better, and if/when we figure that out, .body will probably need to be bytes instead of String)

gibson042 · 2022-03-02T05:24:38Z

packages/SwingSet/docs/delivery.md

 but occasionally a batch of mutually-referencing Promises must be resolved in
 a single syscall because their identifiers are aggressively retired
-immediately after translation).
+immediately after translation). \[TODO: resolve inconsistency between this claim
+and [Syscall/Dispatch API](#syscalldispatch-api)]


The inconsistency is documentation of trait Syscall including fn resolve(subject: PromiseID, to: Resolution), which seems unable to support resolving multiple promises in a batch.

oh yeah, when we switched to batch resolution, I didn't attempt to update the pseudo-rust writeup

The new signature is more like:

struct OneResolution { subject: PromiseID, rejected: bool, resolution: CapData, } fn resolve(resolutions: Vec<OneResolution>) {}

and dispatch.notify takes a batch of resolutions too

gibson042 · 2022-03-02T05:25:30Z

packages/SwingSet/docs/delivery.md

-generally subscribe themselves, since they are the ones causing the Promise
-to become resolved, so they have no need to hear about it from the kernel.
+about the resolution of that Promise [TODO: document ordering
+relevance/constraints]. The Decider vat for a Promise will not generally


This is a question from me about whether the deliver order of notifications to multiple subscribing vats is deterministic, and if not whether that is guaranteed to be safe.

We don't make any particular guarantees about that. Currently we probably accidentally deliver notifies in the order of subscription, but 1: vats would be hard pressed to tell, because we don't make any guarantees about the order of vat execution, also they only talk through queues, and those are the same queues that the subscribe/notify messages travel, so unless they both have access to a device that reveals ordering differences they have no reliable way to perform a test. And 2: the ordering is about to change a bunch with #3465 .

Is that a concern w.r.t. to nondeterminism and subsequent loss of consensus, or is every instance guaranteed to deliver in the same (unpredictable) order?

All consensus instances get the same order: they're all using the same kernel, and it's that kernel that decides how to service the queues. The term we use / property we provide is "arrival order non-determinism": until the thing arrives, you don't know the ordering, but once happens, the ordering is fixed and nobody will see any other order.

At a different level, vats shouldn't be able to sense much about other vats, and if the kernel has the flexibility to do things in various orders, its choices might be influenced by its knowledge of all vats, and thus the kernel might reveal a side-channel that we'd rather keep closed. Some of this is inevitable, since vats talk to each other. And it's probably the case that if vats A and B only talk to each other, and vats C and D only talk to each other, then neither pair's event ordering will be influenced by the other pair. So I'm not worried about our current ordering.

At an even different level, within a single vat, when it sends a bunch of messages out and is holding promises for their resolution, we're also trying to avoid making claims about the order in which they resolve on that particular vat. But those resolutions arrive as dispatch.notify events, which go into the vat's transcript, and all consensus instances run their vats with the same transcripts. So at the point of entry to the vat (dispatch.*), we have arrival-order non-determinism: the order is deterministic across all consensus instances, but the vat has no idea what that order is (and shouldn't depend upon any particular one), but the vat will never observe two different orders (in fact, orthogonal persistence means the vat will never even know that it was restarted and the transcript re-executed, as it has no access to anything which could prove otherwise).

warner

Good changes, minor update suggested. Land after considering them (of course squash before landing).

warner · 2022-03-02T17:01:43Z

packages/SwingSet/docs/delivery.md

+code, which effectively perceives its memory data as eternal) and is written in
+the SES subset of Javascript, employing native platform Promises and making
+eventual-send calls to local or remote objects with either the `E()` wrapper
+(`resultPromise=E(x).foo(args)`) or (eventually) the wavy dot syntax


let's drop mentions of wavy-dot, it's not coming back any time soon

warner · 2022-03-02T17:11:57Z

packages/SwingSet/docs/delivery.md

+Each vat is represented in the kernel as a `dispatch` object with methods that
+close over its internal state (cf.
+[Vat-Inbound Slot Translation](#vat-inbound-slot-translation)). A vat's code
+executes when the kernel initiates a **turn** by invoking one of these methods,


We're using "crank" for this, instead of "turn". A turn is one step of the engine's microtask queue, e.g. a single Promise's .then callback. A crank is 1 or more turns initiated by a dispatch invocation and finishing when the microtask queue is empty. Since vat code doesn't get access to timers or other forms of IO, they can only hold onto control until the end of the crank, at which point they cannot regain agency until the kernel sends them a new dispatch.

In fact, we're a bit casual about the difference between what vat code experiences and what the vat as a whole (including liveslots, and the worker process supervisor) experiences, and indeed what the kernel sees. Also our terminology has shifted over time.

If we're talking about the kernel, these days we use "crank" to talk about the processing of an event from the run-queue, which may or may not perform a delivery to a vat (e.g. the crank might merely queue a message on some unresolved kernel promise). If it does, we send a delivery message to the worker, which begins the "crank" that the vat knows about. Liveslots gets control first, deserializes the arguments, then invoke vat code, which can run for multiple turns until it loses agency. Then liveslots regains control, does some cleanup, then sends the results back to the waiting kernel.

For the purposes of this paragraph, I guess we should say the kernel initiates a "delivery". If "crank" comes up elsewhere, we could differentiate between a "kernel crank" (that may or may perform a delivery) and a "vat crank" which is the multi-turn execution triggered by a delivery).

warner · 2022-03-02T17:17:30Z

packages/SwingSet/docs/delivery.md

@@ -146,7 +149,7 @@ enum CapSlot {
    Object(ObjectID),
 }
 struct CapData {
-    body: Vec<u8>,
+    body: String,


Hm, I believe my original attempt at a Rust implementation used Vec<u8> (the Rust name for bytes, as opposed to unicode), because I think that's what comes out of the serde JSON+UTF8 serializer and it's what can be sent over a wire. But String (i.e. the Rust name for unicode strings) is more meaningful to a JS audience, and is equally plausible, and is closer to what we actually do now in JS, depending upon where you look (there's one point where syscall.send includes a string .body, then there's a later point where the whole syscall gets serialized into bytes to send over the wire). So String sounds fine.

(#2589 is a long-neglected aspiration to move away from JSON and use something denser, maybe CBOR, which would also accomodate bytes arguments better, and if/when we figure that out, .body will probably need to be bytes instead of String)

warner · 2022-03-02T17:20:55Z

packages/SwingSet/docs/delivery.md

-* `Fulfilled`: includes an ObjectID (an Export) to which it was resolved
-* `Data`: includes resolution data (body+slots)
-* `Rejected`: includes the rejection data (body+slots, maybe an Error object)
+* `Fulfilled`: includes the ObjectID with which it was fulfilled


We no longer distinguish between fulfilled (fulfilled to a callable object) and data (fulfilled to anything else). The current states (look for resolveKernelPromise in src/kernel/state/kernelKeeper.js) are unresolved or fulfilled or rejected, and the last two mean there's a .data property on the promise record that contains capdata, plus a .rejected boolean. We wait until we're trying to send a message to the resolved promise before we look at .data to distinguish between something that accepts messages and a data record that cannot.

warner · 2022-03-02T17:25:44Z

packages/SwingSet/docs/delivery.md

@@ -375,19 +383,18 @@ prefer to avoid deserializing the messages until their resolution is known,
 to avoid a wasteful reserialization cycle.

 If the deciding Vat has *not* opted into pipelining, the messages are queued
-in the kernel's Promise Table entry instead. They remain there until the
+in the kernel's Promise table entry instead. They remain there until the


I go back and forth about how to reinforce the idea that a "kernel promise" is not a javascript Promise object (but does correspond to Promises in vats). Sometimes I use "kernel promise" vs "promise", sometimes I use "kernel promise" vs "Promise" (i.e. use the font and capitalization to evoke the JS syntax for the non-kernel thing). I've used "Promise Table" (both caps) to sort of glue the two words together and avoid making it sound like the kernel has a table of Promises (it has a table of kernel promises, but no actual Promise objects).

Eh, words, they're hard. No recommendations here.

warner · 2022-03-02T17:46:30Z

packages/SwingSet/docs/delivery.md

-generally subscribe themselves, since they are the ones causing the Promise
-to become resolved, so they have no need to hear about it from the kernel.
+about the resolution of that Promise [TODO: document ordering
+relevance/constraints]. The Decider vat for a Promise will not generally


We don't make any particular guarantees about that. Currently we probably accidentally deliver notifies in the order of subscription, but 1: vats would be hard pressed to tell, because we don't make any guarantees about the order of vat execution, also they only talk through queues, and those are the same queues that the subscribe/notify messages travel, so unless they both have access to a device that reveals ordering differences they have no reliable way to perform a test. And 2: the ordering is about to change a bunch with #3465 .

warner · 2022-03-02T17:47:20Z

packages/SwingSet/docs/delivery.md

@@ -826,16 +833,17 @@ A Vat might be informed that one Promise has been resolved to another Promise
 ## Sample Message Delivery

 This section follows a variety of messages are they are sent from Vat-1 to
-Vat-2.
+Vat-2. Syntax like `foo!bar(baz)` is used to indicate a message to Object or


Ah, bang notation, even older than tildot/wavydot. This might be worth replacing with E(), except I really like e.g. x~.foo()~.bar() over E(E(x).foo()).bar()

I figure this note allows us to keep it here as a not-actually-executable shorthand.

Yeah, good point, and I do that all the time in unit test comments. Let's update it to tildot then, so at least we only have one archaic dialect instead of two.

warner · 2022-03-02T17:54:15Z

packages/SwingSet/docs/delivery.md

@@ -966,19 +974,19 @@ receiving pipelined messages.

 The two `send` calls will look like:

-* `syscall.send(target=o-1001, msg={method: "foo", .. result=p+104})`
-* `syscall.send(target=p+104, msg={method: "bar", .. result=p+105})`
+* `syscall.send(target=o-1001, msg={method: "foo", …, result=p+104})`


You're determined to drag me kicking and screaming into the modern unicode world, aren't you? :)

I don't have a button on my keyboard for that symbol. I'd kinda prefer to keep this document plain ASCII, but if you think there's an ambiguity caused by U+002E,U+002E (I know .. looks like ... which looks like a real JS operator, and .. is an operator in other languages), then stick with U+2026 and I'll cut-and-paste this symbol if I ever need to edit this doc.

That was indeed the concern. Hopefully changes won't be needed too often. :P (or more properly 😜, although that one I had to copy from Emojipedia).

P.S. I don't know this part of Mac yet, but I think every Linux input method has a default <Multi_key> <period> <period> .XCompose sequence for "…" (which is how I type it, where my Multi_key is configured to be the right-side Ctrl).

warner · 2022-03-02T17:55:03Z

packages/SwingSet/docs/delivery.md

 a kernel-facing C-List (however kernel "messages", really syscalls, are
 delivered immediately, whereas messages to remote machines are asynchronous).
 The Comms Vat does not need to manage a run-queue (`dispatch()` causes an
 immediate external message), nor does each unresolved promise have a queue of
 messages (these are pipelined immediately).

-The one wrinkle is that Vat-Vat connections are symmetric, which impacts the
+The one wrinkle is that Vat→Vat connections are symmetric, which impacts the


warner · 2022-03-02T17:56:26Z

packages/SwingSet/docs/delivery.md


-The message names are also different. In the local-machine Vat-Kernel-Vat
+The message names are also different. In the local-machine Vat→Kernel→Vat


ok here's a case where the unicode arrows are a tiny bit nicer than Vat->Kernel->Vat, I'l begrudgingly admit :)

gibson042 · 2022-03-03T01:59:05Z

@warner The followup changes were sufficiently substantial that I'd like another round of review before merging.

warner

Good, a few small recommendations remaining.

warner · 2022-03-03T19:54:28Z

packages/SwingSet/docs/delivery.md

  ordering properties better.

 In some places, `dispatch.deliver()` is named `message`: we're still in the
 process of refactoring and unifying the codebase.

-## Outbound (Vat->Kernel) translation
+## Vat-Outbound Slot Translation


I'd recommend against this change. Each new person I've explained this to has gotten confused about the directions of "outbound" vs "inbound" for the first few days.. we have a coherent metaphor ("vats are egocentric"), but it isn't obvious, and I'd like to reinforce the scheme whereever we can, especially in a document like this one. So for section titles, having both the word "outbound" and the "vat->kernel" direction expressed at the same time feels important to me.

Feel free to rephrase it, and "Vat-Outbound" does make more sense than just "Outbound", but I think it's important to indicate the direction in the same place as introducing "outbound". Maybe "Vat Outbound Slot Translation (from Vat into Kernel)"? "Vat Outbound Slot Translation (Vat->Kernel)"?

Also maybe I'm being paranoid but I've seen enough <unrenderable glyph> boxes in documents I read, that I don't trust that all readers will render a unicode arrow, and if that happens then we're losing critical information. If you do want to use a unicode arrow in the title, maybe add a few words to be beginning of the section, "As messages are sent "outbound" from the vat into the kernel, ..." for the hapless folks reading the .md in a terminal and cursing about newfangled character sets when the old one worked just fine.

I agree with keeping special characters out of section headings, even including ASCII punctuation (which is actually what motivated this change—I want predictable and stable auto-generated anchor ids). But I just pushed a fix for that by adding anchor elements directly.

warner · 2022-03-03T19:56:31Z

packages/SwingSet/docs/delivery.md


-## Inbound (Kernel->Vat) translation
+## Vat-Inbound Slot Translation


warner · 2022-03-03T20:02:11Z

packages/SwingSet/docs/delivery.md

 translated just like `Message.args`.

 TODO: After a Vat receives notice of a Promise being resolved, we might
-choose to remove that promise from the vat's c-list, and forbid the Vat from
+choose to remove that promise from the vat's C-List, and forbid the Vat from


Let's change this paragraph: we do indeed remove the promise from the c-list ("c-list promise entries are retired upon resolution")

We also use the word "retired" to describe removing c-list object entries when one side no longer needs to talk about the object, which is somewhat after the "drop" that one side does when it can no longer reach that object, because of WeakMaps. So let's be sure to use "retire" for promises here, and not "drop".

We don't exactly forbid the vat from mentioning the vpid again (we don't keep a history of retired IDs), but if it did, the kernel would treat it as a brand new promise. Liveslots won't, because it uses a persistent incrementing counter to generate IDs, so it will never re-generate the old one.. oh, huh, ok that's a problem for vat-upgrade, I'll make a ticket #4730 about that.

gibson042 added 20 commits February 28, 2022 14:47

docs(swingset): Capitalize Unix

cb773b4

docs(swingset): Replace outdated syscall claim with authoritative link

4673804

docs(swingset): Fix typo

6b1cb9b

docs(swingset): Reword Vat Code introduction paragraph for clarity

f9214ac

docs(swingset): Reword dispatch introduction paragraph for clarity

4c9879d

docs(swingset): Explain "orthogonal peristence"

55a1fd7

docs(swingset): Normalize the spelling and capitalization of "C-List"

e0a305f

docs(swingset): Normalize the capitalization of "Object table" and "P…

698ef2a

…romise table"

docs(swingset): Tweak "Data Types" sections

42612e7

docs(swingset): Tweak "Vat Message Types" section

90bd254

docs(swingset): Tweak "Pipelining" section

22979db

docs(swingset): Normalize parentheses in syscall/dispatch method refe…

b20eacd

…rences

docs(swingset): Tweak "Descriptive Conventions" section

27c9c8e

docs(swingset): Clarify that setImmediate is specific to Javascript

993e903

docs(swingset): Tweak "Syscall/Dispatch API" section

f79d2b7

docs(swingset): Replace "legal" with "valid"

bed0a46

docs(swingset): Tweak "Slot Translation" sections

d9d2cff

docs(swingset): Use proper ellipses

6f1c12f

docs(swingset): Tweak "Sample Message Delivery" section

aa4709c

docs(swingset): Tweak "Comms Protocol" section

9c907c9

gibson042 added documentation Improvements or additions to documentation SwingSet package: SwingSet labels Mar 2, 2022

gibson042 requested a review from warner March 2, 2022 05:21

gibson042 self-assigned this Mar 2, 2022

gibson042 commented Mar 2, 2022

View reviewed changes

warner approved these changes Mar 2, 2022

View reviewed changes

docs(swingset): Correct uses of "turn" that should be "crank"

407a117

gibson042 changed the title ~~Improve SwingSet "delivery" documentation~~ docs(swingset): Improve "delivery" documentation Mar 2, 2022

gibson042 added 2 commits March 2, 2022 20:43

docs(swingset): Collapse together promise states "fulfilled" and "data"

b4849df

docs(swingset): Improve the promise pipelining example

3ca2492

gibson042 added 4 commits March 2, 2022 20:43

docs(swingset): Update pseudo-Rust blocks for batch resolution

c172471

docs(swingset): Document that forwarding is not yet implemented

bb63e02

docs(swingset): Remove TODO paragraph

37d6790

docs(swingset): Remove mention of wavy dot

6bf3167

warner self-requested a review March 3, 2022 19:37

warner mentioned this pull request Mar 3, 2022

exported object ID counter vs upgrade #4730

Closed

warner approved these changes Mar 3, 2022

View reviewed changes

gibson042 added 3 commits March 3, 2022 22:21

docs(swingset): Restore section heading text

205e310

docs(swingset): Document retirement of C-List Promise entries

ae9750d

docs(swingset): Replace foo!bar() shorthand with foo~.bar()

8fb69f0

gibson042 added the automerge:squash Automatically squash merge label Mar 4, 2022

Merge branch 'master' into gibson-swingset-delivery-docs-editorial

6c2c4dc

mergify bot merged commit 55155d3 into master Mar 4, 2022

mergify bot deleted the gibson-swingset-delivery-docs-editorial branch March 4, 2022 04:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

docs(swingset): Improve "delivery" documentation #4717

docs(swingset): Improve "delivery" documentation #4717

gibson042 commented Mar 2, 2022

gibson042 Mar 2, 2022

warner Mar 2, 2022

gibson042 Mar 2, 2022

warner Mar 2, 2022

gibson042 Mar 2, 2022

warner Mar 2, 2022

gibson042 Mar 3, 2022

warner Mar 3, 2022

warner left a comment

warner Mar 2, 2022

warner Mar 2, 2022

warner Mar 2, 2022

warner Mar 2, 2022

warner Mar 2, 2022

warner Mar 2, 2022

warner Mar 2, 2022

gibson042 Mar 3, 2022

warner Mar 3, 2022

warner Mar 2, 2022

gibson042 Mar 3, 2022

warner Mar 2, 2022

warner Mar 2, 2022

gibson042 commented Mar 3, 2022

warner left a comment

warner Mar 3, 2022

gibson042 Mar 4, 2022

warner Mar 3, 2022

warner Mar 3, 2022


		The message names are also different. In the local-machine Vat-Kernel-Vat
		The message names are also different. In the local-machine Vat→Kernel→Vat


		## Inbound (Kernel->Vat) translation
		## Vat-Inbound Slot Translation

docs(swingset): Improve "delivery" documentation #4717

docs(swingset): Improve "delivery" documentation #4717

Conversation

gibson042 commented Mar 2, 2022

Description

Security Considerations

Documentation Considerations

Testing Considerations

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

warner left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gibson042 commented Mar 3, 2022

warner left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment