INSERT/REPLACE complex target column types are validated against source input expressions #16223
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
10 tasks
kgyrtkirk
reviewed
Apr 12, 2024
| catch (Types.IncompatibleTypeException e) { | ||
| incompatible = true; | ||
| } | ||
| if (incompatible) { |
There was a problem hiding this comment.
- throw a
Types.IncompatibleTypeExceptionexception based on the result of equals - you could place these things inside the
catch - remove the
incompatibleboolean
There was a problem hiding this comment.
Good suggestion, fixed.
Comment on lines
479
to
494
| RelDataType relType; | ||
| if (sqlTypeName != null) { | ||
| relType = typeFactory.createSqlType(sqlTypeName); | ||
| } else { | ||
| fields.add(Pair.of( | ||
| colName, | ||
| typeFactory.createTypeWithNullability(relType, true) | ||
| )); | ||
| ColumnType columnType = ColumnType.fromString(definedCol.sqlStorageType()); | ||
| if (columnType != null && columnType.getType().equals(ValueType.COMPLEX)) { | ||
| relType = RowSignatures.makeComplexType(typeFactory, columnType, sourceField.getType().isNullable()); | ||
| } else { | ||
| relType = RowSignatures.columnTypeToRelDataType( | ||
| typeFactory, | ||
| columnType, | ||
| // this nullability is ignored for complex types for some reason, hence the check for complex above. | ||
| sourceField.getType().isNullable() | ||
| ); | ||
| } | ||
| } |
There was a problem hiding this comment.
this seems to me that it tries to compute the RelDataType for definedCol ; can we have method or something for that? maybe that could even make it easier to write more direct tests
...nsions-core/druid-catalog/src/test/java/org/apache/druid/catalog/sql/CatalogReplaceTest.java
Show resolved
Hide resolved
10 tasks
kgyrtkirk
approved these changes
Apr 16, 2024
abhishekrb19
approved these changes
Apr 16, 2024
| .authentication(CalciteTests.SUPER_USER_AUTH_RESULT) | ||
| .expectValidationError( | ||
| DruidException.class, | ||
| StringUtils.format("Operation [%s] requires a PARTITIONED BY to be explicitly defined, but none was found.", operationName) |
There was a problem hiding this comment.
Perhaps be more explicit in the error message (the actual code also needs to change):
Suggested change
| StringUtils.format("Operation [%s] requires a PARTITIONED BY to be explicitly defined, but none was found.", operationName) | |
| StringUtils.format("Operation [%s] requires a PARTITIONED BY clause to be explicitly defined in the query or the catalog, but none was found.", operationName) |
| * validation error. | ||
| */ | ||
| @Test | ||
| public void testInsertNoPartitonedByFromCatalog() |
There was a problem hiding this comment.
typo:
Suggested change
| public void testInsertNoPartitonedByFromCatalog() | |
| public void testErrorWhenNoPartitionedBy() |
| * value from the catalog. | ||
| */ | ||
| @Test | ||
| public void testInsertHourGrainPartitonedByFromCatalog() |
There was a problem hiding this comment.
typo:
Suggested change
| public void testInsertHourGrainPartitonedByFromCatalog() | |
| public void testUsePartitionedByFromCatalog() |
| * the query value is used. | ||
| */ | ||
| @Test | ||
| public void testInsertHourGrainWithDayPartitonedByFromQuery() |
There was a problem hiding this comment.
Suggested change
| public void testInsertHourGrainWithDayPartitonedByFromQuery() | |
| public void testUsePartitionedByFromQuery() |
| * the query value is used. | ||
| */ | ||
| @Test | ||
| public void testInsertNoPartitonedByWithDayPartitonedByFromQuery() |
There was a problem hiding this comment.
Suggested change
| public void testInsertNoPartitonedByWithDayPartitonedByFromQuery() | |
| public void testUsePartitionedByFromQuery() |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This change allows for tables that are defined in the catalog to have their complex defined column types validated against source input expressions mapped to them during DML INSERT/REPLACE operations. The catalog dml unit tests have also been greatly simplified, and a lot of duplicate test code simplified or removed.
This PR has: