partitionccl: enhance partition test to use leaseholders and more nodes #42758
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Prior to this commit the partitioning tests worked by creating a 3 node cluster and then expressed constraints over the three nodes. It then validates that the cluster conforms to the constraints by querying data and examining the trace to determine which node held the data. This is problematic for one because it is succeptible to cockroachdb#40333. In rare cases we'll down-replicate to the wrong single node (e.g. if the right one is not live) and we won't ever fix it. It also doesn't exercise leaseholder preferences. This PR adds functionality to configure clusters with larger numbers of nodes where each expectation in the config can now refer to a leaseholder_preference rather than a constraint and we'll allocate the additional nodes to 3 datacenters. This larger test creates dramatically more data movement and has been useful when testing cockroachdb#40892. The PR also adds a flag to control how many of these subtests to run. Release justification: Only touches testing and is useful for testing a release blocker. Release note: None
|
This change is |
|
@darinpp I hear you're interested in testing rebalancing in 1x replication scenarios. I believe this expanded version of the RepartitioningTest pushes on that. cc @andreimatei |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Prior to this commit the partitioning tests worked by creating a 3 node cluster
and then expressed constraints over the three nodes. It then validates that
the cluster conforms to the constraints by querying data and examining the
trace to determine which node held the data.
This is problematic for one because it is susceptible to #40333. In rare
cases we'll down-replicate to the wrong single node (e.g. if the right one
is not live) and we won't ever fix it.
It also doesn't exercise leaseholder preferences.
This PR adds functionality to configure clusters with larger numbers of nodes
where each expectation in the config can now refer to a leaseholder_preference
rather than a constraint and we'll allocate the additional nodes to 3
datacenters.
This larger test creates dramatically more data movement and has been useful
when testing #40892.
The PR also adds a flag to control how many of these subtests to run.
Release justification: Only touches testing and is useful for testing a
release blocker.
Release note: None