SwingSet / cosmic-swingset roadmap

[warner]

The next several things I'm thinking we need to add to swingset / cosmic-swingset.

• Improved persistence. The kernel side is probably done (more efficient/transactional state storage #54), but we need host-side support (like more efficient state storage #254). This is currently blocked on finding a synchronous LevelDB library, or a workaround for an async version. In the longer run, we might want dynamic paging of Vats.
• Dynamic Vats (create addVat() device #24) : Currently all Vats must be "genesis vats", defined by the host at startup. We need to add a way (probably via a Device) to create new Vats at runtime. Ideally the code these new Vats run will be defined by a module spec (the root of a module graph, and some policy to control how import names are mapped to code), rather than one giant flattened/rolluped string. The creator of the Vat gets a certain amount of control over the Vat, include the authority to destroy it.
• Escalator Scheduler (implement kernel-side escalator scheduler #23) : Define and implement Meters and Keepers, expand the kernel run-queue into a more general scheduler, define some sort of token with which to pay for the escalators. Related to (but separate from) Gas-like metering within a computation: the escalators decide when a message is delivered, Gas decides when execution is terminated early.
• Gas : maybe inject code to decrement a Gas counter, terminate execution if the meter underflows. Decide what happens next (does a Keeper get notified? can the computation be resumed? cheaply?).
• Refactor the vattp layer into something that accepts connection-type plugins, specifically IBC (for comms between chain-like SwingSet machines), and something as similar to IBC as possible (for everything involving non-chains). I think the "Mailbox" device we have now is pretty close, but we could use some libraries surrounding it for the networking parts (starting with moving code from cosmic-swingset out into a cosmos-sdk -specific connector library).
• Offline bootstrap/introduction: back in the TLS/SwissNum-based VatTP days, we could create a single secret string that would serve as a bearer token, granting access to a specific object for anyone who learns it. This was (I believe) the preferred/only mechanism to bootstrap connection between two Vats. Now that we have transparent Vats (specifically tranparent comms), we can't use secret tokens, and we need to build a new bootstrap mechanism. We'll probably need different tools for introducing chains to solo machines, chains to other chains, solo to solo, etc. I think many scenarios will require us to pass data in the opposite direction of the authority transfer (e.g. take the pubkey of a solo machine and sign it into a chain transaction, to give a solo vat access to a chain vat's object).
• Three-Party Handoff: we can probably get away without this for a while, but sooner or later we'll have multiple solo machines that need to create direct connections (unlikely between "client" machines, but I might have a mobile wallet machine as well as a personal non-chain server machine, and I'll want to move authority between my personal machines). We've done this before, but it's fussy, and complicated by the lack of secrets, so it's going to require considerable work.
• Monitoring: We need chain-explorer tools that can show us exactly which Mailbox/Comms messages got processed in each block, along with all the syscalls and console.log messages they provoked. Ideally this is integrated into the "deep stacks" work that MarkM has been doing, and we can have a nice Causeway-like display. Bonus points for local view that merge the chain-side operations with information from private local machines.
• Debugging Support: We can't stop the chain for someone's debugger, but since everything is deterministic, we should be able to point at any comms message and have a script spin up a local copy of that Vat, with a debugger() breakpoint that fires as the message is being processed. Likewise we should have a way to simulate the processing of a new message: given a message we're thinking about sending, the tool should fetch the current chain's state (or the necessary subset of it), and give us a debugger view of the chain Vats plus our local message (without actually signing and delivering anything).
• Switch to XS: We know that we don't want to rely upon Node.js for memory security, and we want to switch to a simpler JavaScript engine that we can audit better.
• Heap Snapshots: this requires a lot of support from the JavaScript engine, but it's vital to enable long-lived Vats. Our current transcript-based persistence means the state vector grows with the number of transcations, whereas we'd like it to be a function of the actual size of the Vat (roughly the number of live objects).
• Promise Forwarding (add promise forwarding #50) : if one method returns a known Promise (in particular, a Promise that was obtained as the result of some earlier outgoing message), the kernel should be able to replace the new result Promise with the other. This requires liveslots changes (to detect this case without waiting for the first promise to resolve), a new syscall.forward(), kernel changes to update the promise table correctly, and a new dispatch.notifyForward. It could save some time/effort and allow messages to be pipelined earlier.
• Non-orthogonal Persistence: another way to wrangle bulky state vectors is to have some Vats use explicit load/store operations to manage their state. We've brainstormed about some React-shaped ORM-ish approaches that might work, or some ways to automatically extract a schema from the code in a way that maintains readability. There may be a tension between using objects and Maps as state storage, and getting efficient non-transcript-based persistence, so we have to experiment.
• Garbage Collection: We currently have no way to detect when Vat Code dereferences a Presence, nor a way for Vats to voluntarily release an imported Object or Promise. When we get the ability to destroy dynamic vats, we'll need to add code to the kernel to drop the old vat's references, which might offer a halfway solution (akin to Tyler Close's "Live Fast, Die Young, and Leave a Good-Looking Corpse" paper): use short-lived Vats that are destroyed at the end of the operation, and rely on kernel-level GC. But in the long run we want to have Vat-level GC. There are proposals in TC39 to add "weakrefs" to the language, which include finalizers that are run after the references go away (and could be used by our liveslots layer to make a syscall.release() and notify the kernel that a reference is no longer needed). However this is likely to suffer from nondeterminism. We might use a periodic "heap snapshot and restore" cycle to build a set of objects that are reachable from the liveslots tables but not from the c-list, and then purge these objects in a deterministic way.
• Vat Migration: One benefit of our "nano-kernel" scheme is that Vats do not talk directly to each other, so we can hopefully migrate them between machines. We need to figure out how comity will work and make this a reality.

[warner]

in the old repo. this was cosmic-swingset issue 93

[michaelfig]

Intro

From the bottom:

0. Host top-level controller (*ag-chain-cosmos, *ag-solo, Agoric Wallet)
1. Host Storage (state with committable transactions, *on-chain state)
2. I/O devices (timer backend, *messages, *HTTP, *WebSocket, *IBC)
3. SwingSet kernel
4. Vats

Upgrade process

#1479 explains what we need from @warner @FUDCo

Cosmic swingset is concerned with upgradability of 0-2, vs. upgradability of 3 (SwingSet group) + 4 (Contract people).

Upgrade to Cosmos Stargate

• cosmic-swingset PR Stargate upgrade #1634 @kriskowal
• IBC async acks (Cosmos SDK feature)
• IBC relayer (@jackzampolin's Golang project)
• Stargate block explorer (like https://explorer.testnet.agoric.com), this will come with time