feat: Cloud Spanner support (#8730)

* working on Cloud Spanner driver implementation * working on DDL synchronization * working on DDL synchronization * fixed failing test * working on VIEW implementation * fixed query parameters * lint * added transaction support; added streaming support; * fixed column types * fixes after merge * prettier * added support for generated columns * added escaping for distinct alias * working on generated columns * changed failing test * updated tests for Spanner; bugfixes; * updated tests for Spanner; bugfixes; * updated tests for Spanner; bugfixes; * fixed failing test * fixed failing test * fixing failing tests * fixing failing tests * fixing failing tests * added support for typeorm-generated uuid; fixed caching; * fixing failing tests * fixing failing tests * fixing failing tests * fixing failing tests * fixing failing tests * fixing failing tests * debugging failing test * debugging failing test * fixed bug in @PrimaryColumn decorator * fixed failing tests * fixed VIEW functionality; fixed failing tests; * updated docs
typeorm · Apr 12, 2022 · 62518ae · 62518ae
1 parent 4687be8
commit 62518ae
Show file tree

Hide file tree

Showing 233 changed files with 5,466 additions and 582 deletions.
diff --git a/.github/ISSUE_TEMPLATE/bug-report.md b/.github/ISSUE_TEMPLATE/bug-report.md
@@ -98,6 +98,7 @@ assignees: ''
 | `postgres`        | no            |
 | `react-native`    | no            |
 | `sap`             | no            |
+| `spanner`         | no            |
 | `sqlite`          | no            |
 | `sqlite-abstract` | no            |
 | `sqljs`           | no            |

diff --git a/.github/ISSUE_TEMPLATE/feature-request.md b/.github/ISSUE_TEMPLATE/feature-request.md
@@ -76,6 +76,7 @@ assignees: ''
 | `postgres`        | no            |
 | `react-native`    | no            |
 | `sap`             | no            |
+| `spanner`         | no            |
 | `sqlite`          | no            |
 | `sqlite-abstract` | no            |
 | `sqljs`           | no            |

diff --git a/README.md b/README.md
@@ -214,12 +214,41 @@ await timber.remove()
     - for **SAP Hana**
 
         ```
-        npm i @sap/hana-client
-        npm i hdb-pool
+        npm install @sap/hana-client
+        npm install hdb-pool
         ```
 
         _SAP Hana support made possible by the sponsorship of [Neptune Software](https://www.neptune-software.com/)._
 
+    - for **Google Cloud Spanner**
+
+        ```
+        npm install @google-cloud/spanner --save
+        ```
+
+        Provide authentication credentials to your application code
+        by setting the environment variable `GOOGLE_APPLICATION_CREDENTIALS`:
+
+        ```shell
+        # Linux/macOS
+        export GOOGLE_APPLICATION_CREDENTIALS="KEY_PATH"
+
+        # Windows
+        set GOOGLE_APPLICATION_CREDENTIALS=KEY_PATH
+
+        # Replace KEY_PATH with the path of the JSON file that contains your service account key.
+        ```
+
+        To use Spanner with the emulator you should set `SPANNER_EMULATOR_HOST` environment variable:
+
+        ```shell
+        # Linux/macOS
+        export SPANNER_EMULATOR_HOST=localhost:9010
+
+        # Windows
+        set SPANNER_EMULATOR_HOST=localhost:9010
+        ```
+
     - for **MongoDB** (experimental)
 
         `npm install mongodb@^3.6.0 --save`
@@ -255,7 +284,7 @@ npx typeorm init --name MyProject --database postgres
 ```
 
 Where `name` is the name of your project and `database` is the database you'll use.
-Database can be one of the following values: `mysql`, `mariadb`, `postgres`, `cockroachdb`, `sqlite`, `mssql`, `oracle`, `mongodb`,
+Database can be one of the following values: `mysql`, `mariadb`, `postgres`, `cockroachdb`, `sqlite`, `mssql`, `sap`, `spanner`, `oracle`, `mongodb`,
 `cordova`, `react-native`, `expo`, `nativescript`.
 
 This command will generate a new project in the `MyProject` directory with the following files:
@@ -553,7 +582,7 @@ AppDataSource.initialize()
 
 We are using Postgres in this example, but you can use any other supported database.
 To use another database, simply change the `type` in the options to the database type you are using:
-`mysql`, `mariadb`, `postgres`, `cockroachdb`, `sqlite`, `mssql`, `oracle`, `cordova`, `nativescript`, `react-native`,
+`mysql`, `mariadb`, `postgres`, `cockroachdb`, `sqlite`, `mssql`, `oracle`, `sap`, `spanner`, `cordova`, `nativescript`, `react-native`,
 `expo`, or `mongodb`.
 Also make sure to use your own host, port, username, password and database settings.
 

diff --git a/docker-compose.yml b/docker-compose.yml
@@ -79,6 +79,14 @@ services:
       - default
       - typeorm
 
+  # google cloud spanner
+  spanner:
+    image: alexmesser/spanner-emulator
+    container_name: "typeorm-spanner"
+    ports:
+      - "9010:9010"
+      - "9020:9020"
+
   # sap hana (works only on linux)
   # hanaexpress:
   #   image: "store/saplabs/hanaexpress:2.00.040.00.20190729.1"

diff --git a/docs/data-source-options.md b/docs/data-source-options.md
@@ -25,7 +25,7 @@ Different RDBMS-es have their own specific options.
 
 -   `type` - RDBMS type. You must specify what database engine you use.
     Possible values are:
-    "mysql", "postgres", "cockroachdb", "sap", "mariadb", "sqlite", "cordova", "react-native", "nativescript", "sqljs", "oracle", "mssql", "mongodb", "aurora-mysql", "aurora-postgres", "expo", "better-sqlite3", "capacitor".
+    "mysql", "postgres", "cockroachdb", "sap", "spanner", "mariadb", "sqlite", "cordova", "react-native", "nativescript", "sqljs", "oracle", "mssql", "mongodb", "aurora-mysql", "aurora-postgres", "expo", "better-sqlite3", "capacitor".
     This option is **required**.
 
 -   `extra` - Extra options to be passed to the underlying driver.

diff --git a/docs/entities.md b/docs/entities.md
@@ -337,6 +337,10 @@ or
 `timestamp with local time zone`, `interval year to month`, `interval day to second`, `bfile`, `blob`, `clob`,
 `nclob`, `rowid`, `urowid`
 
+### Column types for `spanner`
+
+`bool`, `int64`, `float64`, `numeric`, `string`, `json`, `bytes`, `date`, `timestamp`, `array`
+
 ### `enum` column type
 
 `enum` column type is supported by `postgres` and `mysql`. There are various possible column definitions:

diff --git a/package.json b/package.json
@@ -74,12 +74,15 @@
     "postgresql-orm",
     "mariadb",
     "mariadb-orm",
+    "spanner",
     "sqlite",
     "sqlite-orm",
     "sql-server",
     "sql-server-orm",
     "oracle",
-    "oracle-orm"
+    "oracle-orm",
+    "cloud-spanner",
+    "cloud-spanner-orm"
   ],
   "devDependencies": {
     "@types/app-root-path": "^1.2.4",
@@ -121,6 +124,7 @@
     "mysql2": "^2.2.5",
     "pg": "^8.5.1",
     "pg-query-stream": "^4.0.0",
+    "prettier": "^2.5.1",
     "redis": "^3.1.1",
     "remap-istanbul": "^0.13.0",
     "rimraf": "^3.0.2",
@@ -131,10 +135,10 @@
     "sqlite3": "^5.0.2",
     "ts-node": "^10.7.0",
     "typeorm-aurora-data-api-driver": "^2.0.0",
-    "typescript": "^4.6.2",
-    "prettier": "^2.5.1"
+    "typescript": "^4.6.2"
   },
   "peerDependencies": {
+    "@google-cloud/spanner": "^5.18.0",
     "@sap/hana-client": "^2.11.14",
     "better-sqlite3": "^7.1.2",
     "hdb-pool": "^0.1.6",
@@ -153,6 +157,9 @@
     "typeorm-aurora-data-api-driver": "^2.0.0"
   },
   "peerDependenciesMeta": {
+    "@google-cloud/spanner": {
+      "optional": true
+    },
     "@sap/hana-client": {
       "optional": true
     },
@@ -208,6 +215,7 @@
     "buffer": "^6.0.3",
     "chalk": "^4.1.0",
     "cli-highlight": "^2.1.11",
+    "date-fns": "^2.28.0",
     "debug": "^4.3.3",
     "dotenv": "^16.0.0",
     "glob": "^7.2.0",
@@ -218,8 +226,7 @@
     "tslib": "^2.3.1",
     "uuid": "^8.3.2",
     "xml2js": "^0.4.23",
-    "yargs": "^17.3.1",
-    "date-fns": "^2.28.0"
+    "yargs": "^17.3.1"
   },
   "scripts": {
     "test": "rimraf ./build && tsc && mocha --file ./build/compiled/test/utils/test-setup.js --bail --recursive --timeout 60000 ./build/compiled/test",

diff --git a/sample/sample1-simple-entity/entity/Post.ts b/sample/sample1-simple-entity/entity/Post.ts
@@ -4,7 +4,7 @@ import { Generated } from "../../../src/decorator/Generated"
 
 @Entity("sample01_post")
 export class Post {
-    @PrimaryColumn("integer")
+    @PrimaryColumn()
     @Generated()
     id: number
 

diff --git a/src/cache/DbQueryResultCache.ts b/src/cache/DbQueryResultCache.ts
@@ -5,6 +5,7 @@ import { QueryRunner } from "../query-runner/QueryRunner"
 import { Table } from "../schema-builder/table/Table"
 import { QueryResultCache } from "./QueryResultCache"
 import { QueryResultCacheOptions } from "./QueryResultCacheOptions"
+import { v4 as uuidv4 } from "uuid"
 
 /**
  * Caches query result into current database, into separate table called "query-result-cache".
@@ -80,7 +81,10 @@ export class DbQueryResultCache implements QueryResultCache {
                         type: driver.normalizeType({
                             type: driver.mappedDataTypes.cacheId,
                         }),
-                        generationStrategy: "increment",
+                        generationStrategy:
+                            driver.options.type === "spanner"
+                                ? "uuid"
+                                : "increment",
                         isGenerated: true,
                     },
                     {
@@ -256,6 +260,14 @@ export class DbQueryResultCache implements QueryResultCache {
 
             await qb.execute()
         } else {
+            // Spanner does not support auto-generated columns
+            if (
+                this.connection.driver.options.type === "spanner" &&
+                !insertedValues.id
+            ) {
+                insertedValues.id = uuidv4()
+            }
+
             // otherwise insert
             await queryRunner.manager
                 .createQueryBuilder()

diff --git a/src/data-source/DataSourceOptions.ts b/src/data-source/DataSourceOptions.ts
@@ -15,6 +15,7 @@ import { SapConnectionOptions } from "../driver/sap/SapConnectionOptions"
 import { AuroraPostgresConnectionOptions } from "../driver/aurora-postgres/AuroraPostgresConnectionOptions"
 import { BetterSqlite3ConnectionOptions } from "../driver/better-sqlite3/BetterSqlite3ConnectionOptions"
 import { CapacitorConnectionOptions } from "../driver/capacitor/CapacitorConnectionOptions"
+import { SpannerConnectionOptions } from "../driver/spanner/SpannerConnectionOptions"
 
 /**
  * DataSourceOptions is an interface with settings and options for specific DataSource.
@@ -37,3 +38,4 @@ export type DataSourceOptions =
     | ExpoConnectionOptions
     | BetterSqlite3ConnectionOptions
     | CapacitorConnectionOptions
+    | SpannerConnectionOptions
diff --git a/src/decorator/Index.ts b/src/decorator/Index.ts
@@ -135,6 +135,7 @@ export function Index(
             unique: options && options.unique ? true : false,
             spatial: options && options.spatial ? true : false,
             fulltext: options && options.fulltext ? true : false,
+            nullFiltered: options && options.nullFiltered ? true : false,
             parser: options ? options.parser : undefined,
             sparse: options && options.sparse ? true : false,
             background: options && options.background ? true : false,

diff --git a/src/decorator/columns/PrimaryColumn.ts b/src/decorator/columns/PrimaryColumn.ts
@@ -41,8 +41,13 @@ export function PrimaryColumn(
     return function (object: Object, propertyName: string) {
         // normalize parameters
         let type: ColumnType | undefined
-        if (typeof typeOrOptions === "string") {
-            type = typeOrOptions
+        if (
+            typeof typeOrOptions === "string" ||
+            typeOrOptions === String ||
+            typeOrOptions === Boolean ||
+            typeOrOptions === Number
+        ) {
+            type = typeOrOptions as ColumnType
         } else {
             options = Object.assign({}, <PrimaryColumnOptions>typeOrOptions)
         }

diff --git a/src/decorator/options/IndexOptions.ts b/src/decorator/options/IndexOptions.ts
@@ -19,6 +19,15 @@ export interface IndexOptions {
      */
     fulltext?: boolean
 
+    /**
+     * NULL_FILTERED indexes are particularly useful for indexing sparse columns, where most rows contain a NULL value.
+     * In these cases, the NULL_FILTERED index can be considerably smaller and more efficient to maintain than
+     * a normal index that includes NULL values.
+     *
+     * Works only in Spanner.
+     */
+    nullFiltered?: boolean
+
     /**
      * Fulltext parser.
      * Works only in MySQL.

diff --git a/src/driver/DriverFactory.ts b/src/driver/DriverFactory.ts
@@ -18,6 +18,7 @@ import { DataSource } from "../data-source/DataSource"
 import { SapDriver } from "./sap/SapDriver"
 import { BetterSqlite3Driver } from "./better-sqlite3/BetterSqlite3Driver"
 import { CapacitorDriver } from "./capacitor/CapacitorDriver"
+import { SpannerDriver } from "./spanner/SpannerDriver"
 
 /**
  * Helps to create drivers.
@@ -65,6 +66,8 @@ export class DriverFactory {
                 return new AuroraPostgresDriver(connection)
             case "capacitor":
                 return new CapacitorDriver(connection)
+            case "spanner":
+                return new SpannerDriver(connection)
             default:
                 throw new MissingDriverError(type, [
                     "aurora-mysql",
@@ -85,6 +88,7 @@ export class DriverFactory {
                     "sap",
                     "sqlite",
                     "sqljs",
+                    "spanner",
                 ])
         }
     }

diff --git a/src/driver/spanner/SpannerConnectionCredentialsOptions.ts b/src/driver/spanner/SpannerConnectionCredentialsOptions.ts
@@ -0,0 +1,19 @@
+/**
+ * Spanner specific connection credential options.
+ */
+export interface SpannerConnectionCredentialsOptions {
+    /**
+     * Connection url where perform connection to.
+     */
+    readonly instanceId?: string
+
+    /**
+     * Database host.
+     */
+    readonly projectId?: string
+
+    /**
+     * Database host port.
+     */
+    readonly databaseId?: string
+}