The intent of this module is to provide a primitive for building on-chain applications with many clients all following a state stream with immutable history, then those clients independently submit transactions (or other i/o occurs) to affect the future of the stream.
Here are some use cases accommodated by this architecture:
- Query the last published value - for polling the current value
- Get future updates, preferring to skip intermediate values - for getting the most recent values without having to iterate through all of the published values
- Get all values published since last seen - for following individual publications from a client with persistent state
- Query history in reverse chronological order - the only useful way to consume history that may be indefinitely long.
A "stream" is an evolving structure with the following properties:
- Publishing: A single value can be published to a stream at a given time, and becomes part of the "stream cell" of that block.
- Consistency: Each stream has a monotonically increasing "sequence number", starting from 1. This enables stream cell relationship consistency checking.
- Batching: in order to preserve all publications to a stream, if there are more than one publication per block they are captured in order within that block's stream cell.
- History: the stream cell can be queried as of a particular block. Each cell contains the position of the prior value, namely information to query the prior relevant stream cell and the value offset within that cell.
- Forking: stream cells can be shared between multiple streams if the application wishes to publish a divergent stream with the old history.
- Only one key per stream: each stream consists of a single KVStore key/value entry at a given height. Stream history is queried via historical queries made for the prior cell, which is usually the same key at an earlier height. So, the on-chain storage requirements for a stream are only for the latest stream cell.
- Provable queries: a given block's committed stream cell can be queried with a single provable Tendermint store query.
- Update events: besides history, a stream also logs an event to indicate to queriers that the stream cell has been updated.
The above stream properties enable several applications:
- query the stream cell at the current height
- and extract the last update from that cell's publications list
This algorithm is only useful if the published values are complete states (with no data dependencies on other states):
- subscribe to stream cell update events to prevent losing future updates
- query the current height's stream cell
- consume only the last update in the cell
- when an update event has arrived, repeat from step 2
This algorithm is useful for consuming the entire history of the stream, such as when the published values are logical deltas that cannot be meaningfully skipped. It requires client-side persistent storage of the "last seen" cell (starting with "none"):
- subscribe to stream cell update events to prevent losing future updates
- query the current height's stream cell
- use the previous blockheight information to issue backward queries for historical states until we reach the "last seen" cell
- consume the queried history in chronological order, updating the "last seen" cell as we go
- when an update event has arrived, repeat from step 2
- query the stream cell at the specified height
- display the published values for that height in reverse order
- if the stream cell contains a "last published height", query the stream cell and repeat from step 2
For forward iteration via an Interchain Query without prohibitively costly polling, there must be enforcement of three criteria specified by the Querying chain:
- Query height must be later than a specified height.
- Query result must not match a specified value.
- Query will time out at another specified block height or time.
These compose to ensure the returned stream cell is not the same as the latest seen result, and thus that it represents a forward evolution of the stream.
At the time of this writing, the draft Interchain Query specification does not enable the option to enforce those criteria.
An on-chain publisher of stream values can invoke three possible operations for a given storage key:
- UpdateState: add a new value to the tail of the stream
- Finish: add a terminal value to the tail of the stream and mark it as done to prevent further publications.
- Fail: attach a terminal error to the tail of the stream and mark it as done
It is the responsibility of this module to enforce these update/finish/fail semantics.