Consistency model of JetStream

[tekumara]

Which of the formal consistency models does JetStream offer for messages in a stream?

The docs mention immediate consistency:

JetStream uses a NATS optimized RAFT distributed quorum algorithm to distribute the persistence service between nats servers in a cluster while maintaining immediate consistency even in the face of Byzantine failures.

But I'm not sure which of the formal consistency models the NATS optimized RAFT distributed quorum algorithm represents.

[jnmoyne]

When the doc mentions immediate consistency, it is in contrast to eventual consistency. It is about how 'writes' (i.e. publishing a message to a stream).

JetStream is an immediately consistent distributed storage system in that every new message stored in the stream is done so in a unique order (when those messages reach the stream leader) and that the acknowledgement that the storing of the message has been successful only happens as the result of a RAFT vote between the NATS JetStream servers (e.g. 3 of them if replicas=3) handling the stream.

This means that when a publishing application receives the positive acknowledgement to it's publication to the stream you are guaranteed that everyone will see that new message in their updates in the same order (and with the same sequence number and time stamp).

This 'non-eventual' consistency is what enables 'compare and set' (i.e. compare and publish to a stream) operations on streams: because there can only be one new message added to a stream at a time.

To map back to those formal consistency models it means that for writes, NATS JetStream is Linearizable.

On the read side the models don't really apply because JetStream doesn't support atomic batching of multiple operations together so the only kind of 'transaction' is the persisting, replicating and voting of a single operation on the stream, but in essence JetStream is serializable because messages are added to a stream in one global order (which you can control using compare and publish).

[derekcollison]

Ack with sync option and AckAll ack policy does provide multiple messages to be all acked at once.