Skip to content

Commit

Permalink
kvcoord: MuxRangeFeed client uses 1 go routine per node
Browse files Browse the repository at this point in the history
Rewrite MuxRangeFeed client to use 1 Go routine per node,
instead of 1 Go routine per range.

Prior to this change, MuxRangeFeed client was structured
so that it was entirely compatible with the execution
model of the regular range feed.  As a result,
1 Go routine was used per range.  This rewrite replaces
old implementation with an almost clean slate implementation
which uses 1 Go routine per node.

Where possible, relatively small and targetted modifications
to the rangefeed library were made to extract common methods
(such as range splitting).

The reduction in the number of Go routines created by rangefeed
has direct impact on the cluster performance, and most importantly
SQL latency.  This is mostly due to the fact that with this PR,
the number of Go routines started by MuxRangeFeed is down to
2 per range (on the rangefeed server side) vs 5 for the regular
rangefeed.  When running changefeeds against tables with
10s-100s of thousands of ranges, this significant difference
in the Go routine count has direct impact on Go scheduler latency,
the number of runnable Go routines, and ultimately, on the SQL
latency.

Epic: none

Release note (enterprise change) : MuxRangeFeed client (enabled via
`changefeed.mux_rangefeed.enabled` setting) is more efficient
when running against large scale workloads.
  • Loading branch information
Yevgeniy Miretskiy committed Mar 16, 2023
1 parent 439b1e3 commit 4fbdabb
Show file tree
Hide file tree
Showing 6 changed files with 659 additions and 406 deletions.
1 change: 1 addition & 0 deletions pkg/kv/kvclient/kvcoord/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@ go_library(
"//pkg/util/ctxgroup",
"//pkg/util/envutil",
"//pkg/util/errorutil/unimplemented",
"//pkg/util/future",
"//pkg/util/grpcutil",
"//pkg/util/hlc",
"//pkg/util/iterutil",
Expand Down
Loading

0 comments on commit 4fbdabb

Please sign in to comment.