Proposal: Introduce Follower Replication in Raft

[Fullstop000]

Summary

In the origin raft protocol, a follower can only receive new raft logs and snapshots from the leader, which could be insufficient in some situations. For example, when a raft cluster is distributed in different data centers, log replication can is expensive due to the frequent cross-DC network transmission. In this case, internal follower-to-follower transfer in one data center can be far more efficient than the traditionally stipulated leader-to-follower transfer.

Relate docs

tikv/rfcs#33

[siddontang]

@Fullstop000 maybe it is better for us to paste the benchmark result here.

[siddontang]

Another benefit I think is to reduce the pressure of the leader when the leader has more followers or learners.

[xiang90]

This is an OK idea. Just need to justify the complexity from some benchmark results, ideally from practical experience/use cases.

[Fullstop000]

@xiang90 is it ok to add some extra fields in Message like below?

message Message {
	optional MessageType type        = 1  [(gogoproto.nullable) = false];
	optional uint64      to          = 2  [(gogoproto.nullable) = false];
	optional uint64      from        = 3  [(gogoproto.nullable) = false];
	optional uint64      term        = 4  [(gogoproto.nullable) = false];
	optional uint64      logTerm     = 5  [(gogoproto.nullable) = false];
	optional uint64      index       = 6  [(gogoproto.nullable) = false];
	repeated Entry       entries     = 7  [(gogoproto.nullable) = false];
	optional uint64      commit      = 8  [(gogoproto.nullable) = false];
	optional Snapshot    snapshot    = 9  [(gogoproto.nullable) = false];
	optional bool        reject      = 10 [(gogoproto.nullable) = false];
	optional uint64      rejectHint  = 11 [(gogoproto.nullable) = false];
	optional bytes       context     = 12;

	repeated uint64  members = 13 [(gogoproto.nullable) = false];
	optional uint64  groupID = 14 [(gogoproto.nullable) = false];
	optional bool isBroadcasting = 15 [(gogoproto.nullable) = false];
	optional uint64 fromDelegate = 16 [(gogoproto.nullable) = false];
}

Not all of new fields will be used in different message types.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Fullstop000]

@xiang90 @tbg
We've tested this feature in a tikv cluster with 5nodes and the network traffic throughput was reduced without obvious performance regression

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 21 days if no further activity occurs. Thank you for your contributions.