FAQs

Why do I see "Ignoring common conflict" errors when I enable debug-level logging?

These errors are completely normal and are nothing with which to be concerned. You will only see these errors when you have multiple instances of the same process, e.g. 2 change-listeners, running simultaneously. These errors occur when an instance is trying to lock an item that is already locked and are typical when there aren't a lot of changes to process. In this case, multiple instances can retrieve a similar list of items to process. This is the nature of redundant batch processing as batches are only fresh with respect to a small snapshot in time. Specifically, the errors occur because the first instance holds a lock for an item, while a second instance is trying to lock the same item. When this second instance receives the conflict, it moves on to the next item. This locking mechanism introduces some extra noise on the database, but allows the instances to be redundant, which is an essential part in making Spiegel fault tolerant and redundant.

Why do we still need Spiegel even though CouchDB 2.1 has a replicator scheduler?

Great question! The summary is that you'll want to use Spiegel if you have a lot of databases, e.g. over 500, and only have a select segment of these databases changing at any given time. The new _replicator scheduler in CouchDB 2.1 is really cool, but:

1. It is not yet intelligent enough to only replicate the databases that are changing. So, if you have 10,000 databases and only say 100 are changing at any given time then it will still try to replicate all 10,000 databases. This leads to a lot of latency if you are expecting the 100 active databases to receive timely replications.
2. This scheduler uses a round robin selection, which means that it can take a long time for the replicator to randomly select the segment of databases (say the 100) that you are really interested in. The CouchDB team is planning to enhance the _replicator scheduler in the future by adding an option that will make it only replicate those databases that are changing, but this work is not yet scheduled so it is hard to say when it will be ready.

And another thing to mention is that there is no notion of "scalable change listening" in CouchDB. Sure, you could use the _global_changes and then _changes feed of specific DBs to get a list of changes, but to scale that design when there are a lot of databases and a lot of changes takes work. Spiegel's change listening, via on_change docs make this scalable listening easier as Spiegel does a lot of the hard work for you ;)