KV duplication without a cluster

[computinglife]

Hi,

We are trying to design an application using NATS, jetstream feature (btw its awesome)

We cannot use a cluster since we have just 2 nodes and we want the changes to KV store from one of the nodes to be replicated to another in an eventually consistent manner

Is there anyway we can do these without writing explicit code to -

1. To dump the KV store from 1st to the 2nd server upon startup - maybe a CLI command ?
2. To automatically watch the KV store from one and use the data to update the 2nd server?

Thanks in advance
Arun

[ripienaar]

2 node jetstream works and native replication can be used. Your data is replicated but you have HA since any outage is a full outage.

there are other options also that can be used depending on needs.

So the question is what are you trying to achieve - how would the replicated data be used.

[computinglife]

Thanks for replying.

I am trying to have HA without using the native clustering - i was under the impression native replication can be used only with 3 nodes.

I want to make it possible for my clients to fall back to either of the 2 servers and continue making requests for which configuration and past event processin data is needed. This is why i need replication.

Can you point me to the relevant docs that explain the 2 node jetstream replication ? (without a shared data storage)

[ripienaar]

2 nodes provide no HA only data duplication so this won’t work for you.

The reason for more than 2 is there needs to be consensus to avoid 2 leaders and and so both being active leading to duplication

so you would need to think how you would deal with the scenario of both being active. There are ways to achieve this but I suspect you would have to arrange a situation where you pick a bucket based on the node you are connected to. In that case you could do circular replication but you would end up with 2 distinct buckets with distinct names. They would end up with same data eventually but your application would need to pick a bucket name based on where he is connected.

You could use leafnodes to connect to seperate nats servers on different domains and replicate that way. But would need different buckets.

same bucket on 2 sides with no real awareness of a leader would cause corruption if data is updated on both sides. If all you want to do is read during an outage that would also help

[computinglife]

Thanks

so you would need to think how you would deal with the scenario of both being active

I guess i dont understand why this an issue. I will read up some more

You could use leafnodes to connect to seperate nats servers on different domains and replicate that way. But would need different buckets.

How is this done? Yes i dont want to update the KV on one side

[jnmoyne]

You have one server (let's call it the hub server) that is configured to accept connections from leaf node servers, and another server that runs as a leaf node and connects to the hub server. Then you create your KV bucket on the hub server and create another KV on the leaf node server that is a mirror of the KV bucket on the hub. You can then have application connected to the hub server do puts, deletes and gets on that KV bucket, and applications that are connected to the leaf node server do gets (or watch, but no puts or deletes) on the mirror KV.

If the leaf node server goes down, applications connected to the hub server can still do their puts, deletes and gets on the KV bucket on the hub, and when the leaf node server comes back up and re-connects to the hub it will then re-synchronize with the KV bucket on the hub.