-
Notifications
You must be signed in to change notification settings - Fork 5.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
*: Truncate partition with global index improvement #55831
*: Truncate partition with global index improvement #55831
Conversation
Hi @mjonss. Thanks for your PR. PRs from untrusted users cannot be marked as trusted with I understand the commands that are listed here. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #55831 +/- ##
================================================
+ Coverage 72.8328% 75.2357% +2.4028%
================================================
Files 1672 1721 +49
Lines 462993 476569 +13576
================================================
+ Hits 337211 358550 +21339
+ Misses 104974 95919 -9055
- Partials 20808 22100 +1292
Flags with carried forward coverage won't be shown. Click here to find out more.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, please move some lines back into StateDeleteReorganization
[LGTM Timeline notifier]Timeline:
|
@mjonss: The following test failed, say
Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
/retest |
@mjonss: Cannot trigger testing until a trusted user reviews the PR and leaves an In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: Defined2014, tangenta, winoros The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
/retest |
@mjonss: Cannot trigger testing until a trusted user reviews the PR and leaves an In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
/retest |
@mjonss: Cannot trigger testing until a trusted user reviews the PR and leaves an In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
/retest |
@mjonss: Cannot trigger testing until a trusted user reviews the PR and leaves an In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
What problem does this PR solve?
Issue Number: close #55819
Problem Summary:
During TRUNCATE PARTITION writes was blocked to partitions that was being truncated, also during
StateDeleteReorganization
, meaning the DDL would block writes to all truncated partitions for the full duration of the DDL, in case of a Global Index.What changed and how does it work?
Instead of simply blocking writes to partitions being truncated, we are hiding the new partitions from Global Indexes
StateWriteOnly
and old partitions fromStateDeleteOnly
so that they cannot be seen, and not cause any anomaly, where you can find data or not depending on access method, i.e. it may be found if using Global Index, but not for table scan.A side effect of this is that we are still blocking duplicate keys, if a write to a global unique index from a
StateDeleteOnly
client that collides with a not-yet cleaned up Global Index entry, or a write to a global unique index fromStateWriteOnly
(old partition, to be truncated) collides with a newly written entry inStateDeleteOnly
(using the new emptied partition).To not generate these kind of duplicate errors for too long period, duplicate key errors are not generated during
StateDeleteReorganization
if the duplicate comes from an old truncated partition, instead that global unique index entry is overwritten.Also refactored/deduplicated code for cleaning up the global index entries from dropped/truncated partitions, so both DROP and TRUNCATE PARTITION uses the same code.
Check List
Tests
Side effects
Documentation
Release note
Please refer to Release Notes Language Style Guide to write a quality release note.