Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: Introduce Follower Replication in Raft #11357

Closed
Fullstop000 opened this issue Nov 14, 2019 · 7 comments
Closed

Proposal: Introduce Follower Replication in Raft #11357

Fullstop000 opened this issue Nov 14, 2019 · 7 comments

Comments

@Fullstop000
Copy link
Contributor

Summary

In the origin raft protocol, a follower can only receive new raft logs and snapshots from the leader, which could be insufficient in some situations. For example, when a raft cluster is distributed in different data centers, log replication can is expensive due to the frequent cross-DC network transmission. In this case, internal follower-to-follower transfer in one data center can be far more efficient than the traditionally stipulated leader-to-follower transfer.

Relate docs

tikv/rfcs#33

@siddontang
Copy link
Contributor

@Fullstop000 maybe it is better for us to paste the benchmark result here.

@siddontang
Copy link
Contributor

Another benefit I think is to reduce the pressure of the leader when the leader has more followers or learners.

@xiang90
Copy link
Contributor

xiang90 commented Nov 15, 2019

This is an OK idea. Just need to justify the complexity from some benchmark results, ideally from practical experience/use cases.

@Fullstop000
Copy link
Contributor Author

Fullstop000 commented Dec 3, 2019

@xiang90 is it ok to add some extra fields in Message like below?

message Message {
	optional MessageType type        = 1  [(gogoproto.nullable) = false];
	optional uint64      to          = 2  [(gogoproto.nullable) = false];
	optional uint64      from        = 3  [(gogoproto.nullable) = false];
	optional uint64      term        = 4  [(gogoproto.nullable) = false];
	optional uint64      logTerm     = 5  [(gogoproto.nullable) = false];
	optional uint64      index       = 6  [(gogoproto.nullable) = false];
	repeated Entry       entries     = 7  [(gogoproto.nullable) = false];
	optional uint64      commit      = 8  [(gogoproto.nullable) = false];
	optional Snapshot    snapshot    = 9  [(gogoproto.nullable) = false];
	optional bool        reject      = 10 [(gogoproto.nullable) = false];
	optional uint64      rejectHint  = 11 [(gogoproto.nullable) = false];
	optional bytes       context     = 12;

	repeated uint64  members = 13 [(gogoproto.nullable) = false];
	optional uint64  groupID = 14 [(gogoproto.nullable) = false];
	optional bool isBroadcasting = 15 [(gogoproto.nullable) = false];
	optional uint64 fromDelegate = 16 [(gogoproto.nullable) = false];
}

Not all of new fields will be used in different message types.

@stale
Copy link

stale bot commented Apr 6, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@Fullstop000
Copy link
Contributor Author

@xiang90 @tbg
We've tested this feature in a tikv cluster with 5nodes and the network traffic throughput was reduced without obvious performance regression
image

@stale stale bot removed the stale label Apr 22, 2020
@stale
Copy link

stale bot commented Jul 21, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 21 days if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Jul 21, 2020
@stale stale bot closed this as completed Aug 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

3 participants