PoC for Projection implementation for API Tables #1315
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,87 @@ | ||
| package io.stargate.sgv2.jsonapi.service.operation.tables; | ||
|
|
||
| import com.datastax.oss.driver.api.core.cql.Row; | ||
| import com.datastax.oss.driver.api.core.metadata.schema.ColumnMetadata; | ||
| import com.datastax.oss.driver.api.querybuilder.select.Select; | ||
| import com.datastax.oss.driver.api.querybuilder.select.SelectFrom; | ||
| import com.fasterxml.jackson.databind.ObjectMapper; | ||
| import com.fasterxml.jackson.databind.node.ObjectNode; | ||
| import io.stargate.sgv2.jsonapi.exception.ErrorCode; | ||
| import io.stargate.sgv2.jsonapi.service.cqldriver.executor.TableSchemaObject; | ||
| import io.stargate.sgv2.jsonapi.service.operation.DocumentSource; | ||
| import io.stargate.sgv2.jsonapi.service.operation.filters.table.codecs.JSONCodec; | ||
| import io.stargate.sgv2.jsonapi.service.operation.filters.table.codecs.JSONCodecRegistry; | ||
| import io.stargate.sgv2.jsonapi.service.operation.filters.table.codecs.MissingJSONCodecException; | ||
| import io.stargate.sgv2.jsonapi.service.operation.filters.table.codecs.ToJSONCodecException; | ||
| import io.stargate.sgv2.jsonapi.service.projection.TableProjectionDefinition; | ||
| import java.util.HashMap; | ||
| import java.util.List; | ||
| import java.util.Map; | ||
|
|
||
| /** | ||
| * Projection used for Table Rows (as opposed to Collection Documents), built from command API | ||
| * projection definitions (expressed in JSON). | ||
| */ | ||
| public record TableRowProjection( | ||
| ObjectMapper objectMapper, TableSchemaObject table, List<ColumnMetadata> columns) | ||
| implements OperationProjection { | ||
| /** | ||
| * Factory method for construction projection instance, given a projection definition and table | ||
| * schema. | ||
| */ | ||
| public static TableRowProjection fromDefinition( | ||
| ObjectMapper objectMapper, | ||
| TableProjectionDefinition projectionDefinition, | ||
| TableSchemaObject table) { | ||
| Map<String, ColumnMetadata> columnsByName = new HashMap<>(); | ||
| // TODO: This can also be cached as part of TableSchemaObject than resolving it for every query. | ||
| table | ||
tatu-at-datastax marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| .tableMetadata | ||
| .getColumns() | ||
| .forEach((id, column) -> columnsByName.put(id.asInternal(), column)); | ||
|
|
||
| List<ColumnMetadata> columns = projectionDefinition.extractSelectedColumns(columnsByName); | ||
|
|
||
| // TODO: A table can't be with empty columns. Think a redundant check. | ||
| if (columns.isEmpty()) { | ||
|
There was a problem hiding this comment. A table can't be with empty columns. Think a redundant check. There was a problem hiding this comment. Left a TODO, just in case it breaks anything There was a problem hiding this comment. We are not checking whether table has no columns but whether set of columns specified by projection definition matching actual columns is empty. |
||
| throw ErrorCode.UNSUPPORTED_PROJECTION_DEFINITION.toApiException( | ||
| "did not include any Table columns"); | ||
| } | ||
|
|
||
| return new TableRowProjection(objectMapper, table, columns); | ||
| } | ||
|
|
||
| @Override | ||
| public Select forSelect(SelectFrom selectFrom) { | ||
| return selectFrom.columnsIds(columns.stream().map(ColumnMetadata::getName).toList()); | ||
| } | ||
|
|
||
| @Override | ||
| public DocumentSource toDocument(Row row) { | ||
| ObjectNode result = objectMapper.createObjectNode(); | ||
| for (int i = 0, len = columns.size(); i < len; ++i) { | ||
| final ColumnMetadata column = columns.get(i); | ||
| final String columnName = column.getName().asInternal(); | ||
| JSONCodec codec; | ||
|
|
||
| // TODO: maybe optimize common case of String, Boolean to avoid conversions, lookups | ||
| try { | ||
| codec = JSONCodecRegistry.codecToJSON(table.tableMetadata, column); | ||
| } catch (MissingJSONCodecException e) { | ||
| throw ErrorCode.UNSUPPORTED_PROJECTION_PARAM.toApiException( | ||
| "Column '%s' has unsupported type '%s'", columnName, column.getType().toString()); | ||
| } | ||
| try { | ||
| result.put(columnName, codec.toJSON(objectMapper, row.getObject(i))); | ||
| } catch (ToJSONCodecException e) { | ||
| throw ErrorCode.UNSUPPORTED_PROJECTION_PARAM.toApiException( | ||
| e, | ||
| "Column '%s' has invalid value of type '%s': failed to convert to JSON: %s", | ||
| columnName, | ||
| column.getType().toString(), | ||
| e.getMessage()); | ||
| } | ||
| } | ||
| return () -> result; | ||
| } | ||
| } | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,152 @@ | ||
| package io.stargate.sgv2.jsonapi.service.projection; | ||
|
|
||
| import com.fasterxml.jackson.databind.JsonNode; | ||
| import io.stargate.sgv2.jsonapi.exception.ErrorCode; | ||
| import java.math.BigDecimal; | ||
| import java.util.ArrayList; | ||
| import java.util.Collections; | ||
| import java.util.List; | ||
| import java.util.Map; | ||
|
|
||
| /** | ||
| * Class similar to {@link DocumentProjector} but used for Table API, for non-nested | ||
| * inclusion/exclusion projections. | ||
| */ | ||
| public class TableProjectionDefinition { | ||
| // Include-all is "exclude nothing" | ||
| private static final TableProjectionDefinition INCLUDE_ALL_PROJECTOR = | ||
| new TableProjectionDefinition(false, Collections.emptyList()); | ||
|
|
||
| // Exclude-all is "include nothing" | ||
| private static final TableProjectionDefinition EXCLUDE_ALL_PROJECTOR = | ||
| new TableProjectionDefinition(true, Collections.emptyList()); | ||
|
|
||
| private final boolean inclusion; | ||
|
|
||
| private final List<String> columnNames; | ||
|
|
||
| private TableProjectionDefinition(boolean inclusion, List<String> columnNames) { | ||
| this.inclusion = inclusion; | ||
| this.columnNames = columnNames; | ||
| } | ||
|
|
||
| public static TableProjectionDefinition createFromDefinition(JsonNode projectionDefinition) { | ||
| // First special case: "simple" default projection; "include all" | ||
| if (projectionDefinition == null || projectionDefinition.isEmpty()) { | ||
| return INCLUDE_ALL_PROJECTOR; | ||
| } | ||
| if (!projectionDefinition.isObject()) { | ||
| throw ErrorCode.UNSUPPORTED_PROJECTION_DEFINITION.toApiException( | ||
| "must be OBJECT, was %s", projectionDefinition.getNodeType()); | ||
| } | ||
| // Special cases: "star-include/exclude" | ||
| if (projectionDefinition.size() == 1) { | ||
| Map.Entry<String, JsonNode> entry = projectionDefinition.fields().next(); | ||
| if ("*".equals(entry.getKey())) { | ||
| boolean includeAll = extractIncludeOrExclude(entry.getKey(), entry.getValue()); | ||
| if (includeAll) { | ||
| return INCLUDE_ALL_PROJECTOR; | ||
| } | ||
| return EXCLUDE_ALL_PROJECTOR; | ||
| } | ||
| } | ||
| return createFromNonEmpty(projectionDefinition); | ||
| } | ||
|
|
||
| private static TableProjectionDefinition createFromNonEmpty(JsonNode projectionDefinition) { | ||
| List<String> columnNames = new ArrayList<>(); | ||
| boolean inclusionProjection = false; | ||
|
|
||
| var it = projectionDefinition.fields(); | ||
| while (it.hasNext()) { | ||
| var entry = it.next(); | ||
| String path = entry.getKey(); | ||
|
|
||
| if (path.isEmpty()) { | ||
| throw ErrorCode.UNSUPPORTED_PROJECTION_PARAM.toApiException( | ||
| "empty paths (and path segments) not allowed"); | ||
| } | ||
|
|
||
| // Special rule for "*": only allowed as single root-level entry; | ||
| if ("*".equals(path)) { | ||
| throw ErrorCode.UNSUPPORTED_PROJECTION_PARAM.toApiException( | ||
| "wildcard ('*') only allowed as the only root-level path"); | ||
| } | ||
| JsonNode value = entry.getValue(); | ||
| boolean addInclusion = extractIncludeOrExclude(path, value); | ||
|
|
||
| // If the first entry, we know inclusion/exclusion; if other, need to ensure | ||
| // there's no mixing of inclusion/exclusion | ||
| if (columnNames.isEmpty()) { | ||
| inclusionProjection = addInclusion; | ||
| } else if (inclusionProjection != addInclusion) { | ||
| if (addInclusion) { | ||
| throw ErrorCode.UNSUPPORTED_PROJECTION_PARAM.toApiException( | ||
| "cannot include '%s' on exclusion projection", path); | ||
| } else { | ||
| throw ErrorCode.UNSUPPORTED_PROJECTION_PARAM.toApiException( | ||
| "cannot exclude '%s' on inclusion projection", path); | ||
| } | ||
| } | ||
| columnNames.add(path); | ||
| } | ||
| return new TableProjectionDefinition(inclusionProjection, columnNames); | ||
| } | ||
|
|
||
| /** | ||
| * Method that selects columns from a map of column definitions, based on this projection | ||
| * definition. | ||
| * | ||
| * @param columnDefs Column definitions by matching name to proper identifier | ||
| * @return Filtered List of matching columns | ||
| * @param <T> Actual column identifier type | ||
| */ | ||
| public <T> List<T> extractSelectedColumns(Map<String, T> columnDefs) { | ||
| // "missing" root layer used as short-cut for include-all/exclude-all | ||
| if (columnNames.isEmpty()) { | ||
| if (inclusion) { // exclude-all | ||
| return Collections.emptyList(); | ||
| } | ||
| // include-all | ||
| return columnDefs.values().stream().toList(); | ||
| } | ||
|
|
||
| // Otherwise need to actually determine | ||
| List<T> included = new ArrayList<>(); | ||
|
|
||
| if (inclusion) { | ||
| for (String columnName : columnNames) { | ||
| T columnDef = columnDefs.get(columnName); | ||
| if (columnDef != null) { | ||
| included.add(columnDef); | ||
| } | ||
| } | ||
| } else { | ||
| for (Map.Entry<String, T> entry : columnDefs.entrySet()) { | ||
| if (!columnNames.contains(entry.getKey())) { | ||
| included.add(entry.getValue()); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| return included; | ||
| } | ||
|
|
||
| private static boolean extractIncludeOrExclude(String path, JsonNode value) { | ||
| if (value.isNumber()) { | ||
| // "0" means exclude (like false); any other number include | ||
| return !BigDecimal.ZERO.equals(value.decimalValue()); | ||
| } | ||
| if (value.isBoolean()) { | ||
| return value.booleanValue(); | ||
| } | ||
| if (value.isObject()) { | ||
| throw ErrorCode.UNSUPPORTED_PROJECTION_PARAM.toApiException( | ||
| "path ('%s') value cannot be OBJECT: nesting not supported for Tables", path); | ||
| } | ||
|
|
||
| // Unknown JSON node type; error | ||
| throw ErrorCode.UNSUPPORTED_PROJECTION_PARAM.toApiException( | ||
| "path ('%s') value must be NUMBER or BOOLEAN, was %s", path, value.getNodeType()); | ||
| } | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't these be handled in the resolvers? These throws unhandled errors.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Depends on how we view this: if there is no way to trigger it from outside (which I think is the case), these are assertions for internal state and should be ok this way.
But if there was a way to get null here from request it should throw
JsonApiException.