[xCluster] Slow SetupReplication requests when using too many tables #10611
Assignees
Labels
area/docdb
YugabyteDB core features
Comments
|
It looks like the culprit here is Update: This is handle in batch, so we're fine here. We were creating new network connections per table within this code, which became expensive when connection costs are high. |
nspiegelberg
added a commit
that referenced
this issue
Feb 16, 2022
… tables Summary: Currently, setup_replication times out when using too many tables. The workaround is to batch into multiple entries with ~10 tables. When investigating, found that we were serially creating a YBClient object, per table, that required 5 sec to startup. Modified to use a cached object and added 2 basic unit tests to run the setup_replication command with 100 tables and verify a reasonable runtime. Test Plan: TwoDCTestParams/TwoDCTest.BootstrapAndSetupLargeTableCount/0 -n 10 TwoDCTestParams/TwoDCTest.SetupUniverseReplicationLargeTableCount/0 -n 10 Reviewers: jhe, rahuldesirazu Reviewed By: rahuldesirazu Subscribers: ybase, bogdan Differential Revision: https://phabricator.dev.yugabyte.com/D15256
jayant07-yb
pushed a commit
to jayant07-yb/yugabyte-db
that referenced
this issue
Mar 8, 2022
…too many tables Summary: Currently, setup_replication times out when using too many tables. The workaround is to batch into multiple entries with ~10 tables. When investigating, found that we were serially creating a YBClient object, per table, that required 5 sec to startup. Modified to use a cached object and added 2 basic unit tests to run the setup_replication command with 100 tables and verify a reasonable runtime. Test Plan: TwoDCTestParams/TwoDCTest.BootstrapAndSetupLargeTableCount/0 -n 10 TwoDCTestParams/TwoDCTest.SetupUniverseReplicationLargeTableCount/0 -n 10 Reviewers: jhe, rahuldesirazu Reviewed By: rahuldesirazu Subscribers: ybase, bogdan Differential Revision: https://phabricator.dev.yugabyte.com/D15256
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Description
Similar to #10065,
yb-admin setup_replicationalso times out when using too many tables. We had a customer who used 100 tables as command input. The current workaround is to batch into multiple entries with ~10 tables. A performance issue with the command, not the feature.The text was updated successfully, but these errors were encountered: