Merge branch 'main' into ajm/tables

README.md

@@ -28,13 +28,15 @@ The quickest way to test out the Data API directly is to start a local copy of t
 
 ```shell
 cd docker-compose
-./start_dse_next_dev_mode.sh
+./start_hcd.sh
+# or
+./start_dse69.sh
 ```
 
-This starts an instance of the Data API along with a Stargate coordinator node in "developer mode" (with DataStax Enterprise 6.8 embedded). 
+This starts an instance of the Data API along with a backend database (HCD or DSE 6.9)
 
 > **Warning**
-> Running this script with no options will use the latest `v1` tagged version of Data API and latest `v2` tagged version of the Stargate coordinator. Therefore, if you have these tags already present in your local Docker from other development/testing, those are the images that will be used. See our Docker compose [README](docker-compose/README.md) to see additional options.
+> Running this script with no options will use the latest `v1` tagged version of Data API. Therefore, if you have these tags already present in your local Docker from other development/testing, those are the images that will be used. See our Docker compose [README](docker-compose/README.md) to see additional options.
 
 Once the services are up, you can access the Swagger endpoint at: http://localhost:8181/swagger-ui/
 
@@ -73,7 +75,9 @@ You can run your application in dev mode that enables live coding using:
 ```shell script
 docker run -d --rm -e CLUSTER_NAME=dse-cluster -e CLUSTER_VERSION=6.8 -e ENABLE_AUTH=true -e DEVELOPER_MODE=true -e DS_LICENSE=accept -e DSE=true -p 8081:8081 -p 8091:8091 -p 9042:9042 stargateio/coordinator-dse-next:v2
 
-./mvnw compile quarkus:dev
+./mvnw compile quarkus:dev -Dstargate.data-store.ignore-bridge=true \
+  -Dstargate.jsonapi.operations.vectorize-enabled=true \
+  -Dstargate.jsonapi.operations.database-config.local-datacenter=dc1
 ```
 
 The command above will first spin the single Stargate DSE coordinator in dev that the API would communicate to.

docker-compose/README.md

@@ -1,6 +1,6 @@
-# Docker Compose scripts for Data API with DSE-6.9
+# Docker Compose scripts for Data API with HCD or DSE-6.9
 
-This directory provides two ways to start the Data API with DSE-6.9 or HCD using `docker compose`.
+This directory provides two ways to start the Data API with HCD or DSE-6.9 using `docker compose`.
 
 ## Prerequisites
 
@@ -19,21 +19,22 @@ You can control the platform using the `-Dquarkus.docker.buildx.platform=linux/a
 
 Follow instructions under the [Script options](#script-options) section to use the locally built image.
 
-## Data API with DSE-6.9 cluster
+## Data API with HCD or DSE-6.9 cluster
 
-You can start a simple configuration with DSE 6.9 with the following command:
+You can start a simple configuration with HCD with the following command:
 
 ```
-./start_dse69.sh
+./start_hcd.sh
 ``` 
 
-You can start a simple configuration with HCD with the following command:
+You can start a simple configuration with DSE 6.9 with the following command:
 
 ```
-./start_hcd.sh
+./start_dse69.sh
 ``` 
 
-This convenience script verifies your Docker installation meets minimum requirements and brings up the configuration described in the `docker-compose.yml` file. The configuration includes health criteria for each container that is used to ensure the containers come up in the correct order.
+This convenience script verifies your Docker installation meets minimum requirements and brings up the configuration described in the `docker-compose.yml` (DSE-6.9) or `docker-compose-hcd.yml` (HCD) file.
+The configuration includes health criteria for each container that is used to ensure the containers come up in the correct order.
 
 The convenience script uses the `-d` and `--wait` options to track the startup progress, so that the compose command exits when all containers have started and reported healthy status within a specified timeout. 
 
@@ -42,6 +43,24 @@ We recommend doing a `docker compose pull` periodically to ensure you always hav
 
 Once done using the containers, you can stop them using the command `docker compose down`.
 
+## HCD or DSE-6.9 cluster WITHOUT Data API
+
+To run local non-Docker instance of Data API, use `-d` with the script: it will prevent running of Data API container:
+
+```
+./start_hcd.sh -d
+# OR
+./start_dse69.sh -d
+```
+
+To run locally built Data API in Quarkus Development mode, you will use:
+
+```
+/mvnw compile quarkus:dev -Dstargate.data-store.ignore-bridge=true \
+  -Dstargate.jsonapi.operations.vectorize-enabled=true \
+  -Dstargate.jsonapi.operations.database-config.local-datacenter=dc1
+```
+
 ## Script options
 
 Both convenience scripts support the following options:
@@ -54,10 +73,22 @@ Both convenience scripts support the following options:
 
 * You can enable reguest logging for the Data API using `-q`: if so, each request is logged under category `io.quarkus.http.access-log`
 
-* You can start dse/hcd node only using `-d` option. This is useful when you want to start Data API locally for development.
+* You can start DSE/HCD node only using `-d` option. This is useful when you want to start Data API locally for development.
 
 * You can specify the number of cassandra node required using `-n [NUMBER]`. Default is 1.
 
+## CQLSH
+
+A convenient way to run `cqlsh` to connect locally is to use one of:
+
+```
+# HCD:
+docker exec -it data-api-hcd-1 cqlsh -u cassandra -p cassandra
+
+# DSE 6.9:
+docker exec -it data-api-dse-1 cqlsh -u cassandra -p cassandra
+```
+
 ## Notes
 
 * The `.env` file defines variables for the docker compose project name (`COMPOSE_PROJECT_NAME`),

src/main/java/io/stargate/sgv2/jsonapi/StargateJsonApi.java

@@ -44,7 +44,7 @@
                       `Cassandra:Base64(username):Base64(password)`. For example,
                       assuming a username of `cassandra` and password of
                       `cassandra` the `Token` header would be set to
-                      `Cassandra:Y2Fzc2FuZHJhCg==:Y2Fzc2FuZHJhCg==`.
+                      `Cassandra:Y2Fzc2FuZHJh:Y2Fzc2FuZHJh`.
                       """),
             },
 

src/main/java/io/stargate/sgv2/jsonapi/api/configuration/CommandObjectMapperHandler.java

@@ -61,11 +61,11 @@ public JavaType handleUnknownTypeId(
     // CollectionCommand
     //            interface io.stargate.sgv2.jsonapi.api.model.command.GeneralCommand ->
     // GeneralCommand
-    final String rawCommandClassString = baseType.getRawClass().toString();
+    final String rawCommandClassString = baseType.getRawClass().getName();
     final String baseCommand =
         rawCommandClassString.substring(rawCommandClassString.lastIndexOf('.') + 1);
-    throw new JsonApiException(
-        ErrorCode.NO_COMMAND_MATCHED,
-        String.format("No \"%s\" command found as \"%s\"", subTypeId, baseCommand));
+    throw ErrorCode.COMMAND_UNKNOWN.toApiException(
+        "\"%s\" not one of \"%s\"s: known commands are %s",
+        subTypeId, baseCommand, idResolver.getDescForKnownTypeIds());
   }
 }

src/main/java/io/stargate/sgv2/jsonapi/api/model/command/CommandResult.java

@@ -194,11 +194,9 @@ public record Error(
    * @return
    */
   public RestResponse map() {
-    if (null != this.errors() && !this.errors().isEmpty()) {
+    if (null != this.errors()) {
       final Optional<Error> first =
-          this.errors().stream()
-              .filter(error -> error.status().getStatusCode() != Response.Status.OK.getStatusCode())
-              .findFirst();
+          this.errors().stream().filter(error -> error.status() != Response.Status.OK).findFirst();
       if (first.isPresent()) {
         return RestResponse.ResponseBuilder.create(first.get().status(), this).build();
       }

src/main/java/io/stargate/sgv2/jsonapi/api/model/command/NoOptionsCommand.java

@@ -20,10 +20,6 @@ default void setOptions(JsonNode value) throws JsonApiException {
     if (value.isNull() || (value.isObject() && value.isEmpty())) {
       return;
     }
-    final String msg =
-        String.format(
-            "%s: %s",
-            ErrorCode.COMMAND_ACCEPTS_NO_OPTIONS.getMessage(), getClass().getSimpleName());
-    throw new JsonApiException(ErrorCode.COMMAND_ACCEPTS_NO_OPTIONS, msg);
+    throw ErrorCode.COMMAND_ACCEPTS_NO_OPTIONS.toApiException("`%s`", getClass().getSimpleName());
   }
 }

src/main/java/io/stargate/sgv2/jsonapi/api/model/command/clause/filter/ComparisonExpression.java

@@ -1,7 +1,6 @@
 package io.stargate.sgv2.jsonapi.api.model.command.clause.filter;
 
 import io.stargate.sgv2.jsonapi.exception.ErrorCode;
-import io.stargate.sgv2.jsonapi.exception.JsonApiException;
 import io.stargate.sgv2.jsonapi.service.operation.model.filters.DBFilterBase;
 import io.stargate.sgv2.jsonapi.service.shredding.model.DocumentId;
 import jakarta.validation.Valid;
@@ -139,9 +138,8 @@ private static JsonLiteral<?> getLiteral(Object value) {
     if (value instanceof UUID || value instanceof ObjectId) {
       return new JsonLiteral<>(value.toString(), JsonType.STRING);
     }
-    throw new JsonApiException(
-        ErrorCode.FILTER_UNRESOLVABLE,
-        String.format("Unsupported filter value type %s", value.getClass()));
+    throw ErrorCode.SERVER_INTERNAL_ERROR.toApiException(
+        "Unsupported filter value type `%s`", value.getClass().getName());
   }
 
   public List<FilterOperation<?>> match(

src/main/java/io/stargate/sgv2/jsonapi/api/model/command/clause/update/UpdateOperator.java

@@ -2,7 +2,6 @@
 
 import com.fasterxml.jackson.databind.node.ObjectNode;
 import io.stargate.sgv2.jsonapi.exception.ErrorCode;
-import io.stargate.sgv2.jsonapi.exception.JsonApiException;
 import java.util.HashMap;
 import java.util.Map;
 
@@ -106,9 +105,8 @@ public String operator() {
   }
 
   public UpdateOperation resolveOperation(ObjectNode arguments) {
-    throw new JsonApiException(
-        ErrorCode.UNSUPPORTED_UPDATE_OPERATION,
-        "Unsupported update operator '%s'".formatted(operator));
+    throw ErrorCode.UNSUPPORTED_UPDATE_OPERATION.toApiException(
+        "Unsupported update operator '%s'", operator);
   }
 
   private static Map<String, UpdateOperator> operatorMap = new HashMap<>();

src/main/java/io/stargate/sgv2/jsonapi/api/model/command/deserializers/UpdateClauseDeserializer.java

@@ -38,15 +38,13 @@ public UpdateClause deserialize(
       Map.Entry<String, JsonNode> entry = fieldIter.next();
       final String operName = entry.getKey();
       if (!operName.startsWith("$")) {
-        throw new JsonApiException(
-            ErrorCode.UNSUPPORTED_UPDATE_OPERATION,
-            "Invalid update operator '%s' (must start with '$')".formatted(operName));
+        throw ErrorCode.UNSUPPORTED_UPDATE_OPERATION.toApiException(
+            "Invalid update operator '%s' (must start with '$')", operName);
       }
       UpdateOperator oper = UpdateOperator.getUpdateOperator(operName);
       if (oper == null) {
-        throw new JsonApiException(
-            ErrorCode.UNSUPPORTED_UPDATE_OPERATION,
-            "Unrecognized update operator '%s'".formatted(operName));
+        throw ErrorCode.UNSUPPORTED_UPDATE_OPERATION.toApiException(
+            "Unrecognized update operator '%s'", operName);
       }
       JsonNode operationArg = entry.getValue();
       if (!operationArg.isObject()) {

src/main/java/io/stargate/sgv2/jsonapi/exception/ErrorCode.java

@@ -4,9 +4,8 @@
 public enum ErrorCode {
   /** Command error codes. */
   COUNT_READ_FAILED("Unable to count documents"),
-  COMMAND_NOT_IMPLEMENTED("The provided command is not implemented."),
+  COMMAND_UNKNOWN("Provided command unknown"),
   INVALID_CREATE_COLLECTION_OPTIONS("The provided options are invalid"),
-  NO_COMMAND_MATCHED("Unable to find the provided command"),
   COMMAND_ACCEPTS_NO_OPTIONS("Command accepts no options"),
 
   /**
@@ -33,8 +32,6 @@ public enum ErrorCode {
   EMBEDDING_PROVIDER_TIMEOUT("The Embedding Provider timed out"),
   EMBEDDING_PROVIDER_UNEXPECTED_RESPONSE("The Embedding Provider returned an unexpected response"),
 
-  FILTER_UNRESOLVABLE("Unable to resolve the filter"),
-
   FILTER_MULTIPLE_ID_FILTER(
       "Cannot have more than one _id equals filter clause: use $in operator instead"),
 
@@ -67,23 +64,16 @@ public enum ErrorCode {
 
   SHRED_BAD_DOCID_EMPTY_STRING("Bad value for '_id' property: empty String not allowed"),
 
-  SHRED_INTERNAL_NO_PATH("Internal: path being built does not point to a property or element"),
-
-  SHRED_NO_MD5("MD5 Hash algorithm not available"),
-
-  SHRED_UNRECOGNIZED_NODE_TYPE("Unrecognized JSON node type in input document"),
-
-  SHRED_DOC_LIMIT_VIOLATION("Document size limitation violated"),
-
-  SHRED_DOC_KEY_NAME_VIOLATION("Document field name invalid"),
-
   SHRED_BAD_EJSON_VALUE("Bad JSON Extension value"),
 
   SHRED_BAD_VECTOR_SIZE("$vector value can't be empty"),
 
   SHRED_BAD_VECTOR_VALUE("$vector value needs to be array of numbers"),
   SHRED_BAD_VECTORIZE_VALUE("$vectorize search clause needs to be non-blank text value"),
 
+  SHRED_DOC_KEY_NAME_VIOLATION("Document field name invalid"),
+  SHRED_DOC_LIMIT_VIOLATION("Document size limitation violated"),
+
   EXISTING_COLLECTION_DIFFERENT_SETTINGS("Collection already exists"),
 
   INVALID_VECTORIZE_VALUE_TYPE("$vectorize value needs to be text value"),
@@ -108,8 +98,6 @@ public enum ErrorCode {
 
   INVALID_USAGE_OF_VECTORIZE("`$vectorize` and `$vector` can't be used together"),
 
-  UNSUPPORTED_OPERATION("Unsupported operation class"),
-
   UNSUPPORTED_PROJECTION_PARAM("Unsupported projection parameter"),
 
   UNSUPPORTED_UPDATE_DATA_TYPE("Unsupported update data type"),
@@ -160,9 +148,7 @@ public enum ErrorCode {
   INVALID_ID_TYPE("Invalid Id type"),
   INVALID_QUERY("Invalid query"),
   NO_INDEX_ERROR("Faulty collection (missing indexes). Recommend re-creating the collection"),
-  UNSUPPORTED_CQL_QUERY_TYPE("Unsupported cql query type"),
   MISSING_VECTOR_VALUE("Missing the vector value when building cql"),
-  INVALID_LOGIC_OPERATOR("Invalid logical operator"),
 
   // Driver failure codes
   /** Error codes related to driver exceptions. */
@@ -172,6 +158,11 @@ public enum ErrorCode {
   SERVER_DRIVER_FAILURE("Driver failed"),
   /** Driver timeout failure. */
   SERVER_DRIVER_TIMEOUT("Driver timeout"),
+  /**
+   * Error code used for "should never happen" style problems. Suffix part needs to include details
+   * of actual issue.
+   */
+  SERVER_INTERNAL_ERROR("Server internal error"),
   SERVER_NO_NODE_AVAILABLE("No node was available to execute the query"),
   SERVER_QUERY_CONSISTENCY_FAILURE("Database query consistency failed"),
   SERVER_QUERY_EXECUTION_FAILURE("Database query execution failed"),
@@ -196,6 +187,10 @@ public JsonApiException toApiException(String format, Object... args) {
     return new JsonApiException(this, message + ": " + String.format(format, args));
   }
 
+  public JsonApiException toApiException(Throwable cause, String format, Object... args) {
+    return new JsonApiException(this, message + ": " + String.format(format, args), cause);
+  }
+
   public JsonApiException toApiException() {
     return new JsonApiException(this, message);
   }

src/main/java/io/stargate/sgv2/jsonapi/exception/mappers/ConstraintViolationExceptionMapper.java

@@ -39,8 +39,7 @@ public RestResponse<CommandResult> constraintViolationException(
         violations.stream().map(ConstraintViolationExceptionMapper::getError).distinct().toList();
 
     // return result
-    CommandResult commandResult = new CommandResult(errors);
-    return RestResponse.ok(commandResult);
+    return new CommandResult(errors).map();
   }
 
   private static CommandResult.Error getError(ConstraintViolation<?> violation) {

src/main/java/io/stargate/sgv2/jsonapi/exception/mappers/GenericExceptionMapper.java

@@ -15,6 +15,6 @@ public class GenericExceptionMapper {
   @ServerExceptionMapper({Exception.class, MismatchedInputException.class})
   public RestResponse<CommandResult> genericExceptionMapper(Throwable e) {
     CommandResult commandResult = new ThrowableCommandResultSupplier(e).get();
-    return RestResponse.ok(commandResult);
+    return commandResult.map();
   }
 }

src/main/java/io/stargate/sgv2/jsonapi/service/cql/ExpressionUtils.java

@@ -15,11 +15,10 @@
  */
 package io.stargate.sgv2.jsonapi.service.cql;
 
-import static io.stargate.sgv2.jsonapi.exception.ErrorCode.INVALID_LOGIC_OPERATOR;
-
 import com.bpodgursky.jbool_expressions.And;
 import com.bpodgursky.jbool_expressions.Expression;
 import com.bpodgursky.jbool_expressions.Or;
+import io.stargate.sgv2.jsonapi.exception.ErrorCode;
 import java.util.List;
 
 /**
@@ -61,7 +60,9 @@ public static <K> Expression<K> buildExpression(
       case "$or" -> {
         return orOf(expressions);
       }
-      default -> throw INVALID_LOGIC_OPERATOR.toApiException();
+      default ->
+          throw ErrorCode.SERVER_INTERNAL_ERROR.toApiException(
+              "Invalid logical operator '%s'", logicOperator);
     }
   }
 

src/main/java/io/stargate/sgv2/jsonapi/service/cql/builder/QueryBuilder.java

@@ -115,7 +115,8 @@ public Query build() {
     if (isSelect) {
       return selectQuery();
     }
-    throw ErrorCode.UNSUPPORTED_CQL_QUERY_TYPE.toApiException();
+    throw ErrorCode.SERVER_INTERNAL_ERROR.toApiException(
+        "Unsupported cql query type in QueryBuilder (isSelect=false)");
   }
 
   private Query selectQuery() {