Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sql: adding columns with new column families with concurrent DML can lead to corruption #99950

Closed
fqazi opened this issue Mar 29, 2023 · 1 comment
Closed

sql: adding columns with new column families with concurrent DML can lead to corruption #99950

fqazi opened this issue Mar 29, 2023 · 1 comment
Assignees
@fqazi
Labels
branch-release-22.2 Used to mark GA and release blockers, technical advisories, and bugs for 22.2 branch-release-23.1 Used to mark GA and release blockers, technical advisories, and bugs for 23.1 C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. GA-blocker T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions)

Comments

@fqazi
Copy link
Collaborator

fqazi commented Mar 29, 2023 •
edited
Loading

When a table is modified to add a new column with a new column family, and this addition needs the creation of a new primary index, its possible for INSERT/UPDATE statements to write the new column family inside the old primary index. If a rollback happens, then the old table can be led in an inaccessible state since there will be KV pairs for the new column family that runtime can't use.

This was specifically more problematic when a rollback occurs, since we will switch back to the old primary index, which will no longer be properly readable.

Jira issue: CRDB-26152
@fqazi fqazi added C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. T-sql-schema-deprecated Use T-sql-foundations instead branch-release-22.2 Used to mark GA and release blockers, technical advisories, and bugs for 22.2 branch-release-23.1 Used to mark GA and release blockers, technical advisories, and bugs for 23.1 labels Mar 29, 2023
@fqazi fqazi self-assigned this Mar 29, 2023
fqazi added a commit to fqazi/cockroach that referenced this issue Mar 29, 2023
@fqazi
sql: avoid writing to column families that do not exist in the primar…
088b492 
…y index

Previously, if a new column family was added during an add
column and an update/insert occurred concurrently, we could
end up writing to this new column family in any primary index.
This was inadequate because if the primary index did not store
the column, then runtime will have trouble reading data from this
table, since after a rollback the added column / column family
will get cleaned up from the table descriptor. To address this,
this patch avoids writing any columns not stored within an index
descriptor.

Fixes: cockroachdb#99950

Release note (bug fix): Concurrent DML while adding
a new column with a new column family can lead to
corruption in the existing primary index. If a rollback
occurs the table may no longer be accessible.
fqazi added a commit to fqazi/cockroach that referenced this issue Mar 29, 2023
@fqazi
sql: avoid writing to column families that do not exist in the primar…
710752e 
…y index

Previously, if a new column family was added during an add
column and an update/insert occurred concurrently, we could
end up writing to this new column family in any primary index.
This was inadequate because if the primary index did not store
the column, then runtime will have trouble reading data from this
table, since after a rollback the added column / column family
will get cleaned up from the table descriptor. To address this,
this patch avoids writing any columns not stored within an index
descriptor.

Fixes: cockroachdb#99950

Release note (bug fix): Concurrent DML while adding
a new column with a new column family can lead to
corruption in the existing primary index. If a rollback
occurs the table may no longer be accessible.
@craig craig bot closed this as completed in b3f5177 Mar 30, 2023
blathers-crl bot pushed a commit that referenced this issue Mar 30, 2023
@fqazi
sql: avoid writing to column families that do not exist in the primar…
192299b 
…y index

Previously, if a new column family was added during an add
column and an update/insert occurred concurrently, we could
end up writing to this new column family in any primary index.
This was inadequate because if the primary index did not store
the column, then runtime will have trouble reading data from this
table, since after a rollback the added column / column family
will get cleaned up from the table descriptor. To address this,
this patch avoids writing any columns not stored within an index
descriptor.

Fixes: #99950

Release note (bug fix): Concurrent DML while adding
a new column with a new column family can lead to
corruption in the existing primary index. If a rollback
occurs the table may no longer be accessible.
@fqazi fqazi reopened this Mar 30, 2023
@fqazi
Copy link
Collaborator Author

fqazi commented Mar 30, 2023

I will close this with the backport getting merged

fqazi added a commit to fqazi/cockroach that referenced this issue Mar 30, 2023
@fqazi
sql: avoid writing to column families that do not exist in the primar…
5dff0e5 
…y index

Previously, if a new column family was added during an add
column and an update/insert occurred concurrently, we could
end up writing to this new column family in any primary index.
This was inadequate because if the primary index did not store
the column, then runtime will have trouble reading data from this
table, since after a rollback the added column / column family
will get cleaned up from the table descriptor. To address this,
this patch avoids writing any columns not stored within an index
descriptor.

Fixes: cockroachdb#99950

Release note (bug fix): Concurrent DML while adding
a new column with a new column family can lead to
corruption in the existing primary index. If a rollback
occurs the table may no longer be accessible.
@fqazi fqazi mentioned this issue Mar 30, 2023
Merged
@fqazi fqazi closed this as completed Mar 30, 2023
@exalate-issue-sync exalate-issue-sync bot added T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) and removed T-sql-schema-deprecated Use T-sql-foundations instead labels May 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@fqazi fqazi

Labels
branch-release-22.2 Used to mark GA and release blockers, technical advisories, and bugs for 22.2 branch-release-23.1 Used to mark GA and release blockers, technical advisories, and bugs for 23.1 C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. GA-blocker T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions)
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@fqazi