Update dispute-coordinator documentation

Include changes made in paritytech#4134 and paritytech#4854

Fixes paritytech#4397

roadmap/implementers-guide/src/node/disputes/dispute-coordinator.md

@@ -23,30 +23,43 @@ The meta information that we track per-candidate is defined as the `CandidateVot
 This draws on the [dispute statement types][DisputeTypes]
 
 ```rust
-struct CandidateVotes {
-    // The receipt of the candidate itself.
-    candidate_receipt: CandidateReceipt,
-    // Sorted by validator index.
-    valid: Vec<(ValidDisputeStatementKind, ValidatorIndex, ValidatorSignature)>,
-    // Sorted by validator index.
-    invalid: Vec<(InvalidDisputeStatementKind, ValidatorIndex, ValidatorSignature)>,
+/// Tracked votes on candidates, for the purposes of dispute resolution.
+#[derive(Debug, Clone, Encode, Decode)]
+pub struct CandidateVotes {
+	/// The receipt of the candidate itself.
+	pub candidate_receipt: CandidateReceipt,
+	/// Votes of validity, sorted by validator index.
+	pub valid: Vec<(ValidDisputeStatementKind, ValidatorIndex, ValidatorSignature)>,
+	/// Votes of invalidity, sorted by validator index.
+	pub invalid: Vec<(InvalidDisputeStatementKind, ValidatorIndex, ValidatorSignature)>,
 }
 
-// The status of the dispute.
-enum DisputeStatus {
-  // Dispute is still active.
-  Active,
-  // Dispute concluded positive (2/3 supermajority) along with what
-  // timestamp it concluded at.
-  ConcludedPositive(Timestamp),
-  // Dispute concluded negative (2/3 supermajority, takes precedence over
-  // positive in the case of many double-votes).
-  ConcludedNegative(Timestamp),
-}
-
-struct RecentDisputes {
-    // sorted by session index and then by candidate hash.
-    disputed: Vec<(SessionIndex, CandidateHash, DisputeStatus)>,
+/// The mapping for recent disputes; any which have not yet been pruned for being ancient.
+pub type RecentDisputes = std::collections::BTreeMap<(SessionIndex, CandidateHash), DisputeStatus>;
+
+/// The status of dispute. This is a state machine which can be altered by the
+/// helper methods.
+#[derive(Debug, Clone, Copy, Encode, Decode, PartialEq)]
+pub enum DisputeStatus {
+	/// The dispute is active and unconcluded.
+	#[codec(index = 0)]
+	Active,
+	/// The dispute has been concluded in favor of the candidate
+	/// since the given timestamp.
+	#[codec(index = 1)]
+	ConcludedFor(Timestamp),
+	/// The dispute has been concluded against the candidate
+	/// since the given timestamp.
+	///
+	/// This takes precedence over `ConcludedFor` in the case that
+	/// both are true, which is impossible unless a large amount of
+	/// validators are participating on both sides.
+	#[codec(index = 2)]
+	ConcludedAgainst(Timestamp),
+	/// Dispute has been confirmed (more than `byzantine_threshold` have already participated/ or
+	/// we have seen the candidate included already/participated successfully ourselves).
+	#[codec(index = 3)]
+	Confirmed,
 }
 ```
 
@@ -65,82 +78,75 @@ Ephemeral in-memory state:
 
 ```rust
 struct State {
-    keystore: KeyStore,
-    highest_session: SessionIndex,
+  keystore: Arc<LocalKeystore>,
+	rolling_session_window: RollingSessionWindow,
+	highest_session: SessionIndex,
+	spam_slots: SpamSlots,
+	participation: Participation,
+	ordering_provider: OrderingProvider,
+	participation_receiver: WorkerMessageReceiver,
+	metrics: Metrics,
+	// This tracks only rolling session window failures.
+	// It can be a `Vec` if the need to track more arises.
+	error: Option<SessionsUnavailable>,
+	/// Latest relay blocks that have been successfully scraped.
+	last_scraped_blocks: LruCache<Hash, ()>,
 }
 ```
 
 ### On startup
+Wait for first leaf from Runtime.
+Check DB for recorded votes for non concluded disputes we have not yet recorded a local statement for.
+For all of those initiate dispute participation. This involves the following steps:
+
+* Scrape on chain votes by issuing `RuntimeApiRequest::FetchOnChainVotes` to Runtime.
+* Save votes for candidates.
+* Process the baking votes.
+* Process the disputes.
+
+### The main loop
+After the initialisation the main loop is started. It's job is to react to various incoming messages.
+The behaviour for each message is described in individual section.
+
+### On `MuxedMessage::Participation`
 
-Check DB for recorded votes for non concluded disputes we have not yet
-recorded a local statement for.
-For all of those initiate dispute participation.
+Loads votes for the candidate from DB and sends `DisputeDistributionMessage::SendDispute` messages for
+each eligible candidate (ones with `DisputeStatement::Invalid`).
+### On `OverseerSignal::ActiveLeaves`
 
-### On `OverseerSignal::ActiveLeavesUpdate`
+The processing is done in `process_active_leaves_update()`. The `ActiveLeavesUpdate` message is first
+passed to the ordering provider's (instance of `OrderingProvider`) `process_active_leaves_update` function.
+Generally it performs the following operations:
 
-For each leaf in the leaves update:
+* Gets the latest finalized block number via `ChainApiMessage::FinalizedBlockNumber` message.
+* Gets the ancestors of the latest finalized block via `ChainApiMessage::Ancestors` message.
+* Updates local state of the `OrderingProvider`  (`included_candidates`
+  `and candidates_by_block_number`) with the included candidates from the obtainet ancestors.
 
-* Fetch the session index for the child of the block with a [`RuntimeApiMessage::SessionIndexForChild`][RuntimeApiMessage].
-* If the session index is higher than `state.highest_session`:
-  * update `state.highest_session`
-  * remove everything with session index less than `state.highest_session - DISPUTE_WINDOW` from the `"recent-disputes"` in the DB.
-  * Use `iter_with_prefix` to remove everything from `"earliest-session"` up to `state.highest_session - DISPUTE_WINDOW` from the DB under `"candidate-votes"`.
-  * Update `"earliest-session"` to be equal to `state.highest_session - DISPUTE_WINDOW`.
-* For each new block, explicitly or implicitly, under the new leaf, scan for a dispute digest which indicates a rollback. If a rollback is detected, use the `ChainApi` subsystem to blacklist the chain.
-* For each new block, use the `RuntimeApi` to obtain a `ScrapedOnChainVotes` and handle them as if they were provided by means of a incoming `DisputeCoordinatorMessage::ImportStatement` message.
-  * In the case of a concluded dispute, there are some cases that do not guarantee the presence of a `CandidateReceipt`, where handling has to be defered <https://github.com/paritytech/polkadot/issues/4011>.
+Then for the updated leaf the onchain votes are scraped.
 
 ### On `OverseerSignal::Conclude`
 
 Exit gracefully.
 
 ### On `OverseerSignal::BlockFinalized`
 
-Do nothing.
-
-### On `DisputeCoordinatorMessage::ImportStatement`
-
-1. Deconstruct into parts `{ candidate_hash, candidate_receipt, session, statements }`.
-2. If the session is earlier than `state.highest_session - DISPUTE_WINDOW`,
-   respond with `ImportStatementsResult::InvalidImport` and return.
-3. Load from underlying DB by querying `("candidate-votes", session,
-   candidate_hash)`.  If that does not exist, create fresh with the given
-   candidate receipt.
-4. If candidate votes is empty and the statements only contain dispute-specific
-   votes, respond with `ImportStatementsResult::InvalidImport` and return.
-5. Otherwise, if there is already an entry from the validator in the respective
-  `valid` or `invalid` field of the `CandidateVotes`,  respond with
-  `ImportStatementsResult::ValidImport` and return.
-6. Add an entry to the respective `valid` or `invalid` list of the
-   `CandidateVotes` for each statement in `statements`.
-7. If the both `valid` and `invalid` lists now became non-zero length where
-   previously one or both had zero length, the candidate is now freshly
-   disputed.
-8. If the candidate is not freshly disputed as determined by 7, continue with
-   10. If it is freshly disputed now, load `"recent-disputes"` and add the
-   candidate hash and session index. Then, if we have local statements with
-   regards to that candidate,  also continue with 10. Otherwise proceed with 9.
-9. Issue a
-   [`DisputeParticipationMessage::Participate`][DisputeParticipationMessage].
-   Wait for response on the `report_availability` oneshot. If available, continue
-   with 10. If not send back `ImportStatementsResult::InvalidImport` and return.
-10. Write the `CandidateVotes` to the underyling DB.
-11. Send back `ImportStatementsResult::ValidImport`.
-12. If the dispute now has supermajority votes in the "valid" direction,
-    according to the `SessionInfo` of the dispute candidate's session, the
-    `DisputeStatus` should be set to `ConcludedPositive(now)` unless it was
-    already `ConcludedNegative`.
-13. If the dispute now has supermajority votes in the "invalid" direction,
-    the `DisputeStatus` should be set to `ConcludedNegative(now)`. If it
-    was `ConcludedPositive` before, the timestamp `now` should be copied
-    from the previous status. It will be pruned after some time and all chains
-    containing the disputed block will be reverted by the runtime and
-    chain-selection subsystem.
-14. Write `"recent-disputes"`
+Performs cleanup of the finalized candidate.
+
+### On `DisputeCoordinatorMessage::ImportStatements`
+
+* The votes for each candidate (and corresponding session) are loaded from the DB.
+* Save all incoming statements that are 'fresh' (meaning seen for first time).
+* Perform spam detection by checking if a specific validator is generating too much disputes.
+* Saves the newly seen votes in the database.
+
+### On `DisputeCoordinatorMessage::RecentDisputes`
+
+Returns all recent disputes saved in the DB.
 
 ### On `DisputeCoordinatorMessage::ActiveDisputes`
 
-* Load `"recent-disputes"` and filter out any disputes which have been concluded for over 5 minutes. Return the filtered data
+* Load `"recent-disputes"` and filter out any disputes which have been concluded for over 5 minutes. Return the filtered data.
 
 ### On `DisputeCoordinatorMessage::QueryCandidateVotes`