-
Notifications
You must be signed in to change notification settings - Fork 3.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
sql: computed columns are not validated on existing data #81698
Comments
Same problem for decimals and other things of that ilk. This goes way way back. I think the same problem applies for default values. |
default values while adding Computed columns with default values are not validated while adding them, this was causing issues when data is inserted. Same validation mechanism as insert line is implemented. Resolves: cockroachdb#81698 Signed-off-by: Kivanc Bilen <kivanc_bilen@hotmail.com>
default values while adding Computed columns with default values are not validated while adding them, this was causing issues when data is inserted. Same validation mechanism as insert line is implemented. Release Note: Previously while adding computed columns with constants they were not validated, validation is implemented. Resolves: cockroachdb#81698 Signed-off-by: Kivanc Bilen <kivanc_bilen@hotmail.com> enum tests are added, fixed the code accordingly Signed-off-by: Kivanc Bilen <kivanc_bilen@hotmail.com> error message is changed in test Signed-off-by: Kivanc Bilen <kivanc_bilen@hotmail.com> validation is moved from alter_table_add_column -> computed_column, tests are added to computed, create table tests are added Signed-off-by: Kivanc Bilen <kivanc_bilen@hotmail.com> failing virtual columns are fixed Signed-off-by: Kivanc Bilen <kivanc_bilen@hotmail.com>
default values while adding Computed columns with default values are not validated while adding them, this was causing issues when data is inserted. Same validation mechanism as insert line is implemented. Resolves: cockroachdb#81698 Release Note: Previously while adding computed columns with constants they were not validated, validation is implemented. Signed-off-by: Kivanc Bilen <kivanc_bilen@hotmail.com>
Postgres (on 14.4, at least) has a weird inconsistency: a table can be created with a column like
So it looks like we handle the CREATE TABLE case correctly, but need special logic for the ALTER TABLE case. I originally thought the error was the result of an assignment cast performed while backfilling the column, but the error occurs for an empty table when nothing is backfilled... unless they always evaluate the expression even if the table has no rows. Maybe we should solve this similarly? |
Computed columns with constant value are not validated while adding them, this was causing issues when data is inserted. Same validation mechanism as insert line is implemented. Resolves: cockroachdb#81698 Release note (bug fix): Computed columns with constant value expressions are now validated when they are added to the table. If the constant value does not conform to the column type, an error is returned. Previously, the column would be successfully added and any subsequent INSERTs would fail. Signed-off-by: Kivanc Bilen <kivanc_bilen@hotmail.com>
optbuilder wraps an assignment cast around a virtual computed column expression when writing to the table if the expression does not have the same type as the virtual column. This matches the behavior of all writes to columns of expressions that don't match the target column type. However, the same cast was not applied to the projection of the virtual column expression when reading from the table. This cast is necessary in order to produce the correct the value of the column - the projection must produce the same value that would have been written to the table if the column was a stored computed column. This commit corrects optbuilder so that the cast is correctly added to the projection. Note that this commit adds a standard cast, not an assignment cast, as a temporary workaround until cockroachdb#81698 is addressed. This is because an assignment cast can fail in cases when a regular cast will not. Because we don't validate that all existing rows can successfully produce a new virtual column expression during a backfill, adding an assignment cast to the projection of the virtual column could cause future reads to error. Once cockroachdb#81698 is addressed, we can change these casts to assignment casts so that they directly match the values that would be written to the same column if it were stored. Fixes cockroachdb#91817 Informs cockroachdb#81698 Release note (bug fix): A bug has been fixed that could produce incorrect values for virtual computed columns in rare cases. The bug only presented when the virtual column expression's type did not match the type of the virtual column.
optbuilder wraps an assignment cast around a virtual computed column expression when writing to the table if the expression does not have the same type as the virtual column. This matches the behavior of all writes to columns of expressions that don't match the target column type. However, the same cast was not applied to the projection of the virtual column expression when reading from the table. This cast is necessary in order to produce the correct the value of the column - the projection must produce the same value that would have been written to the table if the column was a stored computed column. This commit corrects optbuilder so that the cast is correctly added to the projection. Note that this commit adds a standard cast, not an assignment cast, as a temporary workaround until cockroachdb#81698 is addressed. This is because an assignment cast can fail in cases when a regular cast will not. Because we don't validate that all existing rows can successfully produce a new virtual column expression during a backfill, adding an assignment cast to the projection of the virtual column could cause future reads to error. Once cockroachdb#81698 is addressed, we can change these casts to assignment casts so that they directly match the values that would be written to the same column if it were stored. Fixes cockroachdb#91817 Informs cockroachdb#81698 Release note (bug fix): A bug has been fixed that could produce incorrect values for virtual computed columns in rare cases. The bug only presented when the virtual column expression's type did not match the type of the virtual column.
optbuilder wraps an assignment cast around a virtual computed column expression when writing to the table if the expression does not have the same type as the virtual column. This matches the behavior of all writes to columns of expressions that don't match the target column type. However, the same cast was not applied to the projection of the virtual column expression when reading from the table. This cast is necessary in order to produce the correct value of the column - the projection must produce the same value that would have been written to the table if the column was a stored computed column. This commit corrects optbuilder so that the cast is correctly added to the projection. Note that this commit adds a standard cast, not an assignment cast, as a temporary workaround until cockroachdb#81698 is addressed. This is because an assignment cast can fail in cases when a regular cast will not. Because we don't validate that all existing rows can successfully produce a new virtual column expression during a backfill, adding an assignment cast to the projection of the virtual column could cause future reads to error. Once cockroachdb#81698 is addressed, we can change these casts to assignment casts so that they directly match the values that would be written to the same column if it were stored. Fixes cockroachdb#91817 Informs cockroachdb#81698 Release note (bug fix): A bug has been fixed that could produce incorrect values for virtual computed columns in rare cases. The bug only presented when the virtual column expression's type did not match the type of the virtual column.
105736: opt: wrap virtual computed column projections in a cast r=mgartner a=mgartner #### sql/logictest: remove assignment cast TODO This commit removes a TODO that was partially addressed by #82022. Informs #73743 Release note: None #### sql/types: fix T.Identical This commit fixes a bug in `types.T.Identical` which caused it to return false for collated string types with a different string-representation of locales that represents the same locale logically. For example, collated string types with locales `en_US` and `en_us` would not be identical, even though these are both valid representations of the same locale. There is no release note because this has not caused any known bugs. Release note: None #### opt: wrap virtual computed column projections in a cast optbuilder wraps an assignment cast around a virtual computed column expression when writing to the table if the expression does not have the same type as the virtual column. This matches the behavior of all writes to columns of expressions that don't match the target column type. However, the same cast was not applied to the projection of the virtual column expression when reading from the table. This cast is necessary in order to produce the correct value of the column - the projection must produce the same value that would have been written to the table if the column was a stored computed column. This commit corrects optbuilder so that the cast is correctly added to the projection. Note that this commit adds a standard cast, not an assignment cast, as a temporary workaround until #81698 is addressed. This is because an assignment cast can fail in cases when a regular cast will not. Because we don't validate that all existing rows can successfully produce a new virtual column expression during a backfill, adding an assignment cast to the projection of the virtual column could cause future reads to error. Once #81698 is addressed, we can change these casts to assignment casts so that they directly match the values that would be written to the same column if it were stored. Fixes #91817 Informs #81698 Release note (bug fix): A bug has been fixed that could produce incorrect values for virtual computed columns in rare cases. The bug only presented when the virtual column expression's type did not match the type of the virtual column. 105932: opt/exec: add passthrough cols to DELETE USING result cols in explain r=yuzefovich,DrewKimball a=michae2 Now that we support `DELETE USING`, delete nodes can have passthrough columns. Add these to the result columns used by `EXPLAIN (TYPES)`. Fixes: #105803 Release note (sql change): Fix an internal error when using `EXPLAIN (TYPES)` on a `DELETE FROM ... USING ... RETURNING` statement. Co-authored-by: Marcus Gartner <marcus@cockroachlabs.com> Co-authored-by: Michael Erickson <michae2@cockroachlabs.com>
optbuilder wraps an assignment cast around a virtual computed column expression when writing to the table if the expression does not have the same type as the virtual column. This matches the behavior of all writes to columns of expressions that don't match the target column type. However, the same cast was not applied to the projection of the virtual column expression when reading from the table. This cast is necessary in order to produce the correct value of the column - the projection must produce the same value that would have been written to the table if the column was a stored computed column. This commit corrects optbuilder so that the cast is correctly added to the projection. Note that this commit adds a standard cast, not an assignment cast, as a temporary workaround until cockroachdb#81698 is addressed. This is because an assignment cast can fail in cases when a regular cast will not. Because we don't validate that all existing rows can successfully produce a new virtual column expression during a backfill, adding an assignment cast to the projection of the virtual column could cause future reads to error. Once cockroachdb#81698 is addressed, we can change these casts to assignment casts so that they directly match the values that would be written to the same column if it were stored. Fixes cockroachdb#91817 Informs cockroachdb#81698 Release note (bug fix): A bug has been fixed that could produce incorrect values for virtual computed columns in rare cases. The bug only presented when the virtual column expression's type did not match the type of the virtual column.
Describe the problem
We don't currently evaluate computed expressions for new virtual computed columns to ensure that they
are indeed valid for all of the data in the table. This is a major oversight.
To Reproduce
Expected behavior
We should fail to add these columns.
Jira issue: CRDB-16484
Epic CRDB-31282
The text was updated successfully, but these errors were encountered: