Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add batch API for neon-http driver #1895

Merged
merged 8 commits into from
Feb 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/release-feature-branch.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,7 @@ jobs:
PG_CONNECTION_STRING: postgres://postgres:postgres@localhost:5432/drizzle
MYSQL_CONNECTION_STRING: mysql://root:root@localhost:3306/drizzle
PLANETSCALE_CONNECTION_STRING: ${{ secrets.PLANETSCALE_CONNECTION_STRING }}
NEON_CONNECTION_STRING: ${{ secrets.NEON_CONNECTION_STRING }}
LIBSQL_URL: file:local.db
run: |
if [[ ${{ github.event_name }} != "push" && "${{ github.event.pull_request.head.repo.full_name }}" != "${{ github.repository }}" ]]; then
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/release-latest.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,7 @@ jobs:
PG_CONNECTION_STRING: postgres://postgres:postgres@localhost:5432/drizzle
MYSQL_CONNECTION_STRING: mysql://root:root@localhost:3306/drizzle
PLANETSCALE_CONNECTION_STRING: ${{ secrets.PLANETSCALE_CONNECTION_STRING }}
NEON_CONNECTION_STRING: ${{ secrets.NEON_CONNECTION_STRING }}
LIBSQL_URL: file:local.db
run: |
if [[ "${{ matrix.package }}" == "drizzle-orm" ]]; then
Expand Down
85 changes: 85 additions & 0 deletions changelogs/drizzle-orm/0.29.4.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
## New Features

### 🎉 **Neon HTTP Batch**

For more info you can check [Neon docs](https://neon.tech/docs/serverless/serverless-driver#issue-multiple-queries-with-the-transaction-function)

**Example**

```ts
const batchResponse: BatchType = await db.batch([
db.insert(usersTable).values({ id: 1, name: 'John' }).returning({
id: usersTable.id,
}),
db.insert(usersTable).values({ id: 2, name: 'Dan' }),
db.query.usersTable.findMany({}),
db.query.usersTable.findFirst({}),
]);
```

```ts
type BatchType = [
{
id: number;
}[],
NeonHttpQueryResult<never>,
{
id: number;
name: string;
verified: number;
invitedBy: number | null;
}[],
{
id: number;
name: string;
verified: number;
invitedBy: number | null;
} | undefined,
];
```

## Improvements

Thanks to the `database-js` and `PlanetScale` teams, we have updated the default behavior and instances of `database-js`.

As suggested by the `database-js` core team, you should use the `Client` instance instead of `connect()`:

```typescript
import { Client } from '@planetscale/database';
import { drizzle } from 'drizzle-orm/planetscale-serverless';

// create the connection
const client = new Client({
host: process.env['DATABASE_HOST'],
username: process.env['DATABASE_USERNAME'],
password: process.env['DATABASE_PASSWORD'],
});

const db = drizzle(client);
```

> Warning: In this version, there are no breaking changes, but starting from version `0.30.0`, you will encounter an error if you attempt to use anything other than a `Client` instance.
>
> We suggest starting to change connections to PlanetScale now to prevent any runtime errors in the future.

Previously our docs stated to use `connect()` and only this function was can be passed to drizzle. In this realase we are adding support for `new Client()` and deprecating `connect()`, by suggesting from `database-js` team. In this release you will see a `warning` when trying to pass `connect()` function result:

**Warning text**

```mdx
Warning: You need to pass an instance of Client:

import { Client } from "@planetscale/database";

const client = new Client({
host: process.env["DATABASE_HOST"],
username: process.env["DATABASE_USERNAME"],
password: process.env["DATABASE_PASSWORD"],
});

const db = drizzle(client);

Starting from version 0.30.0, you will encounter an error if you attempt to use anything other than a Client instance.

Please make the necessary changes now to prevent any runtime errors in the future
```
2 changes: 1 addition & 1 deletion drizzle-orm/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "drizzle-orm",
"version": "0.29.3",
"version": "0.29.4",
"description": "Drizzle ORM package for SQL databases",
"type": "module",
"scripts": {
Expand Down
2 changes: 1 addition & 1 deletion drizzle-orm/src/alias.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ import type { AnyColumn } from './column.ts';
import { Column } from './column.ts';
import { entityKind, is } from './entity.ts';
import type { Relation } from './relations.ts';
import type { View} from './sql/sql.ts';
import type { View } from './sql/sql.ts';
import { SQL, sql } from './sql/sql.ts';
import { Table } from './table.ts';
import { ViewBaseConfig } from './view-common.ts';
Expand Down
8 changes: 4 additions & 4 deletions drizzle-orm/src/aws-data-api/pg/session.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,10 @@ import { entityKind } from '~/entity.ts';
import type { Logger } from '~/logger.ts';
import {
type PgDialect,
PgPreparedQuery,
PgSession,
PgTransaction,
type PgTransactionConfig,
PreparedQuery,
type PreparedQueryConfig,
type QueryResultHKT,
} from '~/pg-core/index.ts';
Expand All @@ -24,7 +24,7 @@ import { getValueFromDataApi, toValueParam } from '../common/index.ts';

export type AwsDataApiClient = RDSDataClient;

export class AwsDataApiPreparedQuery<T extends PreparedQueryConfig> extends PreparedQuery<T> {
export class AwsDataApiPreparedQuery<T extends PreparedQueryConfig> extends PgPreparedQuery<T> {
static readonly [entityKind]: string = 'AwsDataApiPreparedQuery';

private rawQuery: ExecuteStatementCommand;
Expand All @@ -40,7 +40,7 @@ export class AwsDataApiPreparedQuery<T extends PreparedQueryConfig> extends Prep
readonly transactionId: string | undefined,
private customResultMapper?: (rows: unknown[][]) => T['execute'],
) {
super();
super({ sql: queryString, params });
this.rawQuery = new ExecuteStatementCommand({
sql: queryString,
parameters: [],
Expand Down Expand Up @@ -151,7 +151,7 @@ export class AwsDataApiSession<
fields: SelectedFieldsOrdered | undefined,
transactionId?: string,
customResultMapper?: (rows: unknown[][]) => T['execute'],
): PreparedQuery<T> {
): PgPreparedQuery<T> {
return new AwsDataApiPreparedQuery(
this.client,
query.sql,
Expand Down
8 changes: 4 additions & 4 deletions drizzle-orm/src/d1/session.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import type { Logger } from '~/logger.ts';
import { NoopLogger } from '~/logger.ts';
import type { RelationalSchemaConfig, TablesRelationalConfig } from '~/relations.ts';
import type { PreparedQuery } from '~/session.ts';
import { type Query, sql, fillPlaceholders } from '~/sql/sql.ts';
import { fillPlaceholders, type Query, sql } from '~/sql/sql.ts';
import type { SQLiteAsyncDialect } from '~/sqlite-core/dialect.ts';
import { SQLiteTransaction } from '~/sqlite-core/index.ts';
import type { SelectedFieldsOrdered } from '~/sqlite-core/query-builders/select.types.ts';
Expand Down Expand Up @@ -57,7 +57,7 @@ export class SQLiteD1Session<
const builtQueries: D1PreparedStatement[] = [];

for (const query of queries) {
const preparedQuery = query.prepare();
const preparedQuery = query._prepare();
const builtQuery = preparedQuery.getQuery();
preparedQueries.push(preparedQuery);
if (builtQuery.params.length > 0) {
Expand Down Expand Up @@ -154,8 +154,8 @@ export class SQLiteD1Session<
// return res;
}

override extractRawAllValueFromBatchResult(_result: unknown): unknown {
return (_result as D1Result).results;
override extractRawAllValueFromBatchResult(result: unknown): unknown {
return (result as D1Result).results;
}

override extractRawGetValueFromBatchResult(result: unknown): unknown {
Expand Down
2 changes: 1 addition & 1 deletion drizzle-orm/src/libsql/session.ts
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ export class LibSQLSession<
const builtQueries: InStatement[] = [];

for (const query of queries) {
const preparedQuery = query.prepare();
const preparedQuery = query._prepare();
const builtQuery = preparedQuery.getQuery();
preparedQueries.push(preparedQuery);
builtQueries.push({ sql: builtQuery.sql, args: builtQuery.params as InArgs });
Expand Down
35 changes: 24 additions & 11 deletions drizzle-orm/src/neon-http/driver.ts
Original file line number Diff line number Diff line change
@@ -1,15 +1,13 @@
import type { NeonQueryFunction } from '@neondatabase/serverless';
import { types } from '@neondatabase/serverless';
import type { BatchItem, BatchResponse } from '~/batch.ts';
import { entityKind } from '~/entity.ts';
import type { Logger } from '~/logger.ts';
import { DefaultLogger } from '~/logger.ts';
import { PgDatabase } from '~/pg-core/db.ts';
import { PgDialect } from '~/pg-core/dialect.ts';
import {
createTableRelationsHelpers,
extractTablesRelationalConfig,
type RelationalSchemaConfig,
type TablesRelationalConfig,
} from '~/relations.ts';
import { createTableRelationsHelpers, extractTablesRelationalConfig } from '~/relations.ts';
import type { ExtractTablesWithRelations, RelationalSchemaConfig, TablesRelationalConfig } from '~/relations.ts';
import type { DrizzleConfig } from '~/utils.ts';
import { type NeonHttpClient, type NeonHttpQueryResultHKT, NeonHttpSession } from './session.ts';

Expand Down Expand Up @@ -41,12 +39,23 @@ export class NeonHttpDriver {
}
}

export type NeonHttpDatabase<
export class NeonHttpDatabase<
TSchema extends Record<string, unknown> = Record<string, never>,
> = PgDatabase<NeonHttpQueryResultHKT, TSchema>;
> extends PgDatabase<NeonHttpQueryResultHKT, TSchema> {
static readonly [entityKind]: string = 'NeonHttpDatabase';

/** @internal */
declare readonly session: NeonHttpSession<TSchema, ExtractTablesWithRelations<TSchema>>;

async batch<U extends BatchItem<'pg'>, T extends Readonly<[U, ...U[]]>>(
batch: T,
): Promise<BatchResponse<T>> {
return this.session.batch(batch) as Promise<BatchResponse<T>>;
}
}

export function drizzle<TSchema extends Record<string, unknown> = Record<string, never>>(
client: NeonHttpClient,
client: NeonQueryFunction<boolean, boolean>,
config: DrizzleConfig<TSchema> = {},
): NeonHttpDatabase<TSchema> {
const dialect = new PgDialect();
Expand All @@ -70,8 +79,12 @@ export function drizzle<TSchema extends Record<string, unknown> = Record<string,
};
}

const driver = new NeonHttpDriver(client, dialect, { logger });
const driver = new NeonHttpDriver(client as unknown as NeonHttpClient, dialect, { logger });
const session = driver.createSession(schema);

return new PgDatabase(dialect, session, schema) as NeonHttpDatabase<TSchema>;
return new NeonHttpDatabase(
dialect,
session,
schema as RelationalSchemaConfig<ExtractTablesWithRelations<TSchema>> | undefined,
);
}
Loading
Loading