Distributed Logs

[haadcode]

In this issue, we should collect the research, articles and implementations on and of "Distributed Logs".

While the term is easy to confuse with other types of logs, what I mean specifically is logs that many databases use under the hood, for example Apache Kafka.

Please add anything relevant to this thread! Especially research papers related to the topic.

Eventually, we'd like to collect all these into a document, perhaps the README in this repo.

[haadcode]

Gonna start with what OrbitDB project has developed, a trustless p2p log: ipfs-log.

[haadcode]

And the canonical article (Kafka): The Log: What every software engineer should know about real-time data's unifying abstraction

[haadcode]

Building a Distributed Log from Scratch, Part 1: Storage Mechanics

(This article was posted on HN today, which inspired me to start this issue)

[haadcode]

Implementations:

• Kafka
• NATS
• Tank
• Pulsar

[haadcode]

LogDevice: a distributed data store for logs - Facebook's log storage

[haadcode]

Efficient Solutions to the Replicated Log and Dictionary Problems

[haadcode]

Turning the database inside-out with Apache Samza

[haadcode]

Using logs to build a solid data infrastructure (or: why dual writes are a bad idea)

[haadcode]

Apache DistributedLog