Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kvserver: support atomic promotions, demotions and swaps of NON_VOTERs and VOTERs #58499

Closed
aayushshah15 opened this issue Jan 6, 2021 · 2 comments · Fixed by #58627
Closed

kvserver: support atomic promotions, demotions and swaps of NON_VOTERs and VOTERs #58499

aayushshah15 opened this issue Jan 6, 2021 · 2 comments · Fixed by #58627
Assignees
@aayushshah15
Labels
A-kv-replication-constraints C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)

Comments

@aayushshah15
Copy link
Contributor

aayushshah15 commented Jan 6, 2021 •
edited
Loading

In order to support #56197 and #57184, we'd like to be able to relocate the replicas of a range such that a store that has a VOTER should have a NON_VOTER or vice-versa. Currently, this would require multiple AdminChangeReplicas requests. For instance, in order to promote a NON_VOTER to a VOTER on store X, we would have to first remove the NON_VOTER and then add a new VOTER to X. This entails unnecessary data movement and potentially leaves a range in violation of its constraints in the interim.

We already have the machinery in place to be able to perform such promotions, demotions, and even VOTER <-> NON_VOTER swaps atomically as we support these things for the LEARNER replica type. We should support the same functionality for NON_VOTERs.

#58627 adds support in AdminRelocateRange to be able to execute such a swap, much in the same way we implicitly swap LEARNERs with VOTER_INCOMINGs during rebalancing. [DONE]

The final piece of the puzzle will be to get the allocator to incorporate this ability into its rebalancing decisions. It will always be preferable to promote/swap an existing non-voter to a voting replica over adding a new voter, as the new voter has to get upreplicated via a snapshot whereas the existing non-voter just needs to catch up to the raft log index corresponding to the promotion.
@aayushshah15 aayushshah15 self-assigned this Jan 6, 2021
@blathers-crl
Copy link

blathers-crl bot commented Jan 6, 2021

Hi @aayushshah15, I've guessed the C-ategory of your issue and suitably labeled it. Please re-label if inaccurate.

While you're here, please consider adding an A- label to help keep our repository tidy.

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan.

@blathers-crl blathers-crl bot added the C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) label Jan 6, 2021
@aayushshah15 aayushshah15 mentioned this issue Jan 8, 2021
Merged
aayushshah15 added a commit to aayushshah15/cockroach that referenced this issue Jan 8, 2021
@aayushshah15
kvserver: support atomic promotions and demotions of non-voting replicas
58231e4 
This PR teaches `AdminChangeReplicas` to atomically promote voters to
non-voters, demote voters to non-voters or swap voters with non-voters
via joint consensus.

Fixes cockroachdb#58499
Informs cockroachdb#51943

Release note: None
aayushshah15 added a commit to aayushshah15/cockroach that referenced this issue Jan 12, 2021
@aayushshah15
kvserver: support atomic promotions and demotions of non-voting replicas
98e9e29 
This PR teaches `AdminChangeReplicas` to atomically promote voters to
non-voters, demote voters to non-voters or swap voters with non-voters
via joint consensus.

Fixes cockroachdb#58499
Informs cockroachdb#51943

Release note: None
aayushshah15 added a commit to aayushshah15/cockroach that referenced this issue Jan 18, 2021
@aayushshah15
kvserver: support atomic promotions and demotions of non-voting replicas
09ce463 
This PR teaches `AdminChangeReplicas` to atomically promote voters to
non-voters, demote voters to non-voters or swap voters with non-voters
via joint consensus.

Fixes cockroachdb#58499
Informs cockroachdb#51943

Release note: None
@aayushshah15 aayushshah15 mentioned this issue Feb 2, 2021
Closed
aayushshah15 added a commit to aayushshah15/cockroach that referenced this issue Feb 15, 2021
@aayushshah15
kvserver: support atomic promotions and demotions of non-voting replicas
e28d39e 
This PR teaches `AdminChangeReplicas` to atomically promote voters to
non-voters, demote voters to non-voters or swap voters with non-voters
via joint consensus.

Fixes cockroachdb#58499
Informs cockroachdb#51943

Release note: None
aayushshah15 added a commit to aayushshah15/cockroach that referenced this issue Feb 16, 2021
@aayushshah15
kvserver: support atomic promotions and demotions of non-voting replicas
a1dd966 
This PR teaches `AdminChangeReplicas` to atomically promote voters to
non-voters, demote voters to non-voters or swap voters with non-voters
via joint consensus.

Fixes cockroachdb#58499
Informs cockroachdb#51943

Release note: None
aayushshah15 added a commit to aayushshah15/cockroach that referenced this issue Feb 18, 2021
@aayushshah15
kvserver: support atomic promotions and demotions of non-voting replicas
6d88a62 
This PR teaches `AdminChangeReplicas` to atomically promote voters to
non-voters, demote voters to non-voters or swap voters with non-voters
via joint consensus.

Fixes cockroachdb#58499
Informs cockroachdb#51943

Release note: None
aayushshah15 added a commit to aayushshah15/cockroach that referenced this issue Feb 22, 2021
@aayushshah15
kvserver: support atomic promotions and demotions of non-voting replicas
01e99d0 
This PR teaches `AdminChangeReplicas` to atomically promote voters to
non-voters, demote voters to non-voters or swap voters with non-voters
via joint consensus.

Fixes cockroachdb#58499
Informs cockroachdb#51943

Release note: None
aayushshah15 added a commit to aayushshah15/cockroach that referenced this issue Feb 22, 2021
@aayushshah15
kvserver: support atomic promotions and demotions of non-voting replicas
fcda057 
This PR teaches `AdminChangeReplicas` to atomically promote voters to
non-voters, demote voters to non-voters or swap voters with non-voters
via joint consensus.

Fixes cockroachdb#58499
Informs cockroachdb#51943

Release note: None
aayushshah15 added a commit to aayushshah15/cockroach that referenced this issue Feb 22, 2021
@aayushshah15
kvserver: support atomic promotions and demotions of non-voting replicas
eea4767 
This PR teaches `AdminChangeReplicas` to atomically promote voters to
non-voters, demote voters to non-voters or swap voters with non-voters
via joint consensus.

Fixes cockroachdb#58499
Informs cockroachdb#51943

Release note: None
craig bot pushed a commit that referenced this issue Feb 22, 2021
@aayushshah15 @rafiss
@arulajmani @irfansharif
Merge #58627 #60799 #60938 #60939
eab1ae3 
58627: kvserver: support atomic promotions and demotions of non-voting replicas r=aayushshah15 a=aayushshah15

This PR teaches `AdminChangeReplicas` to atomically promote voters to non-voters, demote voters to non-voters or swap voters with non-voters via joint consensus.

The approach followed by this PR tries to coalesce `ReplicationChanges` of types `ADD_VOTER` and `REMOVE_NON_VOTER` on a given target as promotions of non-voters into voters and likewise, `ADD_NON_VOTER` and `REMOVE_VOTER` changes for a given target as demotions of voters into non-voters. When all 4 of these operations are simultaneously passed into a `ChangeReplicasRequest`, the patch will try to execute them as one atomic swap of a voter with a non-voter. 

Fixes #58499
Informs #51943

Release note: None


60799: sql: include virtual table names in crash reports r=knz a=rafiss

This uses the functionality that we already had in place for telemetry
data. This might potentially make some crash reports more useful without
compromising privacy.

Release note (general change): Crash reports that are sent to Cockroach Labs
now no longer redact the names of builtin virtual tables from the
crdb_internal, information_schema, and pg_catalog schemas.

60938: sql: drop partitions and zone configs from RBR tables on region drop r=otan a=arulajmani

Closes #58340

Release note (sql change): `ALTER DATABASE ... DROP REGION ...`
repartitions regional by row tables to remove the partition for the
removed region and removes the zone congfiguration for the partition
as well.

60939: bazel: correct BUILD file generation order r=irfansharif a=irfansharif

pkg/BUILD.bazel has the (auto-generated) directive that instructs bazel
to strip out the /pkg prefix. If we start off without
pkg/BUILD.bazel[1], the earlier gazelle invocations don't learn about
the prefix strip.

[1]: Maybe we shouldn't be able to, I was testing to see if there were
     no extraneous diffs by re-generating the file entirely.

Release note: None

Co-authored-by: Aayush Shah <aayush.shah15@gmail.com>
Co-authored-by: Rafi Shamim <rafi@cockroachlabs.com>
Co-authored-by: arulajmani <arulajmani@gmail.com>
Co-authored-by: irfan sharif <irfanmahmoudsharif@gmail.com>
@craig craig bot closed this as completed in e924d91 Feb 22, 2021
@aayushshah15 aayushshah15 reopened this Feb 22, 2021
@andreimatei
Copy link
Contributor

We got these swaps; remaining work tracked in #62370.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@aayushshah15 aayushshah15

Labels
A-kv-replication-constraints C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

kvserver: support atomic promotions and demotions of non-voting replicas aayushshah15/cockroach
2 participants
@andreimatei @aayushshah15