From 5009afa08d5a225f451e20e97f9589a706675f32 Mon Sep 17 00:00:00 2001 From: Jose Luis Lucas Date: Tue, 11 Dec 2018 00:41:38 +0100 Subject: [PATCH] Improve README --- CONTRIBUTING.md | 2 +- docs/advanced_usage.md | 47 ++++++++++++++++++++++++------------- docs/architecture/gossip.md | 23 ++++++++++++++++++ docs/architecture/raft.md | 13 ++++++++++ 4 files changed, 68 insertions(+), 17 deletions(-) create mode 100644 docs/architecture/gossip.md create mode 100644 docs/architecture/raft.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 1276ac888..3924887cc 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -6,4 +6,4 @@ You can contribute in a few different ways: * If you wish to make code changes, or contribute something new, please follow the [GitHub Forks / Pull requests model](https://help.github.com/articles/fork-a-repo/): fork the repo, make the change and propose it back by submitting a pull request. -* Useful notes used by the team are in [docs/development.md]() file +* Useful notes used by the team are in [docs/development.md](docs/development.md) file diff --git a/docs/advanced_usage.md b/docs/advanced_usage.md index 66708340d..0e9b631ff 100644 --- a/docs/advanced_usage.md +++ b/docs/advanced_usage.md @@ -2,19 +2,20 @@ ## Overview -Besides the standalone example given in the README.md, QED are created to be a +Besides the standalone example given in the [README](../README.md), QED are created to be a production-ready cluster. Here you can find some detailed examples. ## QED cluster In order to garantee reliability and High Availabity QED storage servers are created around hashicorp's [raft](https://github.com/hashicorp/raft) implementation. +An architectural perspective can be found at [raft](architecture/raft.md) doc. file. -### Starting cluster mode - -To have identified the leader beforehand, we launch a server and then some +To have identified the leader beforehand, it is recommended to launch a server and then some disposable followers. +### Starting cluster mode + ```bash go run main.go start \ -k my-key \ @@ -51,20 +52,32 @@ any follower (and it's the way to go). A Quick example could be use the README standalone client example changing the endpoint port `--endpoint http://localhost:8081` in the verify event command. + ## Agents In order to allow public `auditors`, we need to ensure a public storage in which QED server sends the snapshot information. `publisher` agents are designed to do it. Finally `monitors` agents are check internal consitency. -## test_service -To use a demo public log-storage QED provides a small helper wich uses a in-memory -structure to store signed snapshots. +QED sends signed snapshots to, at least, one agent of each type. +QED uses Gossip protocol for message passing between server and agents, and between +agents themselves. +An architectural perspective can be found at [gossip](architecture/gossip.md) doc. file. + + +### Aux service + +For demo purposes, QED provides an auxiliary service which uses +an in-memory structure to store signed snapshots that acts as a public log-storage. + +Moreover, it provides an alert endpoint to allow agents register its alerts. ```bash go run tests/e2e/gossip/test_service.go ``` +To be production-ready, both services must be developed and deployed separatelly. + ### Publisher ```bash @@ -87,7 +100,8 @@ go run main.go agent \ -l info ``` -### auditor +### Auditor + ```bash go run main.go agent \ --alertsUrls $alertsEndpoint \ @@ -101,15 +115,16 @@ go run main.go agent \ -l info ``` -### monitor +### Monitor + ```bash go run main.go agent \ --alertsUrls $alertsEndpoint \ - monitor \ - -k my-key \ - --bind 127.0.0.1:920$i \ - --join $masterEndpoint \ - --endpoints $qedEndpoint \ - --node monitor0 \ - -l info + monitor \ + -k my-key \ + --bind 127.0.0.1:9200 \ + --join $masterEndpoint \ + --endpoints $qedEndpoint \ + --node monitor0 \ + -l info ``` diff --git a/docs/architecture/gossip.md b/docs/architecture/gossip.md new file mode 100644 index 000000000..2ea60f623 --- /dev/null +++ b/docs/architecture/gossip.md @@ -0,0 +1,23 @@ +# Message passing. + +QED receives events as input, and outputs signed snapshots. These snapshots +are inputs to any agent (publisher, monitor and auditor), so QED needs to +pass batchs of signed snapshots to the agents. + +## Gossip + +QED server and agents use the [memberslist](https://github.com/hashicorp/memberlist) +package from HashiCorp to create lists of servers, publishers, monitors, and +auditors. + +Then, QED sends a batch of signed snapshots to a configurable number `N` of +each agent type vía memberlist `send reliable` tcp connection, adding a TTL +to each batch. + +Agents receive a batch of signed snapshots, perform their particular task +using it, and send the batch again to other agents (not to QED), reducing the +message TTL. Message passing ends when TTL is equal to 0. + +## Alternatives +Besides of Gossip, HTTP protocol can be used for passing messages, but +syncronous requests make QED to not perform as expected. \ No newline at end of file diff --git a/docs/architecture/raft.md b/docs/architecture/raft.md new file mode 100644 index 000000000..538c1b093 --- /dev/null +++ b/docs/architecture/raft.md @@ -0,0 +1,13 @@ +## Raft +QED uses `raft` for a leader election within a cluster of servers. +Agents do not use raft. + +The leader is able to add new events or verify proofs, while +followers only perform the verifiying option, to cope with read +scalability requirements. + +Once there is a leader and some followers, QED leader replicate the +finite state machine (FMI) to the followers before performing the +insert operation. +Only insert operations (not query operations) are stored in the FMI, +since QED uses them to recover from server failures.