Epic: local dev setup for serverless proxy

[kelvich]

Motivation

With more adoption of serverless driver we got more requests for local and CI setups for proxy.

DoD

We have local setup that is:

• fast to spin up and has small footprint
• works without internet access (except for the setup)
• provides working local connection string in one command

Implementation ideas

So far we had two ideas:

• provide docker image that runs proxy+postgres
• built proxy in our CLI, so that it can start listening for connections on neon dev command

Second approach is intriguing but it is way more work (e.g. we should distribute binaries in npm, like esbuild does) so it seems better to proceed with docker image approach for now.

For the local experience to be end-to-end we need the following things:

• serverless driver supports unencrypted localhost conections both for WS and http
• proxy can serve WS and http without TLS
• build and publish docker image from the CI
• add docs
• mention loca
• (potentially) write a blog post

So we can target for the following flow:

> docker run -p 4432:4432 neondatabase/proxy_local
Listening on 4432. You can use `postgres://test:test@localhost/testdb` for local development.

To make things easier to use I suggest to embed postgres into the image. At the same time it would be nice to support env var that can supply external postgres URL for cases when postgres is already configured.

[janvorwerk]

Indeed, this would be awsome! 🚀
I tried to play with the wsproxy alone for a couple hours (https://github.com/neondatabase/wsproxy) but I suppose this is a bit outdated as it only serves websockets (and prometheus metrics) but the neon serverless client now seem to do fetch requests rather than websocket connections...

Something very simple for a start would be great (for instance, Docker images are great, but if you target development, this should be flexible enough as to permit all dev setups).

--edit: found additional info in this issue... In particular, the fact that websockets are only used if you use Client or Pool, not the neon() function

[TimoWilhelm]

I already commend on the closed thread (neondatabase/serverless#33 (comment)) but I think it makes sense to share it here too.

I've taken the steps from neondatabase/serverless#33 (comment) and created a Docker compose project to simplify the setup process for a local neon proxy + PostgreSQL DB when using the neon() http proxy setup.

You can find the repo here: https://github.com/TimoWilhelm/local-neon-http-proxy.

[venkatd]

@kelvich I would consider this a high impact.

We almost didn't consider neon because our developers prefer to have a fully offline localhost development option. Installing docker and running docker is a huge burden for those of us who have simple infrastructure (node+postgres only in our case).

For a lot of folks considering neon, having neon appear like a drop-in replacement for postgres would remove a lot of hesitation. It's almost there except for the localhost experience :)

Perhaps this could be designed to work outside of the box? For example, the library could auto-detect whether a connection string is a neon string or a regular postgres string and use a built-in proxy to reduce the number of moving parts. Maybe wsproxy could be ported to javascript to reduce the number of moving parts and steps to installation.

[kelvich]

A bit more context: neondatabase/serverless#33

[jacobdr]

I tried to get pretty far on this with a local postgres database running in docker with a NextJS edge function, using a bunch of different ws:// -> tcp:// proxy implementations but ended up getting FATAL: invalid frontend message type 112 errors back on the postgres side... I haven't figured out yet where things got tripped up, but wanted to leave this here in case its useful for anyone else

[jacobdr]

For those arriving here, I did get this working... Here is my setup (big thanks for @kincaidoneil who cracked the code with disabling pipelineConnect):

import {
    Pool as NeonPool,
    neonConfig,
} from "@neondatabase/serverless";
import {
    PrismaNeon,
} from "@prisma/adapter-neon";
import { PrismaClient } from "@prisma/client";
import ws from "ws";

neonConfig.webSocketConstructor = typeof WebSocket !== "undefined" ? WebSocket : ws;

function buildNeonClient(databaseUrl: string, isLocalDevelopment: boolean): PrismaClient {
    // https://gist.github.com/kincaidoneil/bc2516111f0ec8850cd6020b8191b27b
    if (isLocalDevelopment) {
        neonConfig.pipelineConnect = false;
        neonConfig.useSecureWebSocket = false;
        neonConfig.wsProxy = () => "localhost:5433";
    }
    const neon = new NeonPool({ connectionString: databaseUrl });
    const adapter = new PrismaNeon(neon);
    const prisma = new PrismaClient({ adapter });
    return prisma;
}

Then, to make this work, run a websocket -> TCP proxy like this:

# Your database URL would point to this proxy, assuming postgres is at localhost:5432
# example: DATABASE_URL=postgresql://postgres:password@localhost:5433/postgres
wstcp --log-level debug --bind-addr 127.0.0.1:5433 127.0.0.1:5432

[jahabeebs]

For those arriving here, I did get this working... Here is my setup (big thanks for @kincaidoneil who cracked the code with disabling pipelineConnect):

import {
    Pool as NeonPool,
    neonConfig,
} from "@neondatabase/serverless";
import {
    PrismaNeon,
} from "@prisma/adapter-neon";
import { PrismaClient } from "@prisma/client";
import ws from "ws";

neonConfig.webSocketConstructor = typeof WebSocket !== "undefined" ? WebSocket : ws;

function buildNeonClient(databaseUrl: string, isLocalDevelopment: boolean): PrismaClient {
    // https://gist.github.com/kincaidoneil/bc2516111f0ec8850cd6020b8191b27b
    if (isLocalDevelopment) {
        neonConfig.pipelineConnect = false;
        neonConfig.useSecureWebSocket = false;
        neonConfig.wsProxy = () => "localhost:5433";
    }
    const neon = new NeonPool({ connectionString: databaseUrl });
    const adapter = new PrismaNeon(neon);
    const prisma = new PrismaClient({ adapter });
    return prisma;
}

Then, to make this work, run a websocket -> TCP proxy like this:

# Your database URL would point to this proxy, assuming postgres is at localhost:5432
# example: DATABASE_URL=postgresql://postgres:password@localhost:5433/postgres
wstcp --log-level debug --bind-addr 127.0.0.1:5433 127.0.0.1:5432

I've noticed this works well when accessing the db from the front end (for example taking some action on the front end that calls an API route) but for some reason when I use Postman to hit the api endpoints directly it doesn't proxy the request correctly--the db call fails in the middleware.ts file. This is what the error looks like:

 ⨯ Error: All attempts to open a WebSocket to connect to the database failed. Please refer to https://github.com/neondatabase/serverless/blob/main/CONFIG.md#websocketconstructor-typeof-websocket--undefined. Details: fetch failed
    at eval (webpack-internal:///(middleware)/./node_modules/@neondatabase/serverless/index.mjs:1359:74)
    at async PrismaNeon.performIO (webpack-internal:///(middleware)/./node_modules/@prisma/adapter-neon/dist/index.mjs:385:79)
    at async PrismaNeon.queryRaw (webpack-internal:///(middleware)/./node_modules/@prisma/adapter-neon/dist/index.mjs:347:17)
    at async eval (webpack-internal:///(middleware)/./node_modules/@prisma/client/runtime/wasm.js:2:7537)

proxy logs:

Jun 20 04:58:13.460 DEBG Header: host: localhost:5433
connection: keep-alive
upgrade: websocket
user-agent: Next.js Middleware
x-middleware-subrequest: src/middleware
accept: */*
accept-language: *
sec-fetch-mode: cors
accept-encoding: gzip, deflate

, client_addr: 127.0.0.1:50588, server_addr: 127.0.0.1:55433, proxy_addr: 127.0.0.1:5433
Jun 20 04:58:13.460 WARN Invalid WebSocket handshake request: InvalidInput (cause; assertion failed: `values.any(|v| v.trim() == "Upgrade")`; value="keep-alive")
HISTORY:
  [0] at /Users/beebs/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wstcp-0.2.1/src/channel.rs:162
  [1] at /Users/beebs/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wstcp-0.2.1/src/channel.rs:81
, client_addr: 127.0.0.1:50588, server_addr: 127.0.0.1:55433, proxy_addr: 127.0.0.1:5433

I don't have a solution yet but I'll update this if I figure something out

[jswhisperer]

For those arriving here, I did get this working... Here is my setup (big thanks for @kincaidoneil who cracked the code with disabling pipelineConnect):

import {
Pool as NeonPool,
neonConfig,
} from "@neondatabase/serverless";
import {
PrismaNeon,
} from "@prisma/adapter-neon";
import { PrismaClient } from "@prisma/client";
import ws from "ws";

neonConfig.webSocketConstructor = typeof WebSocket !== "undefined" ? WebSocket : ws;

function buildNeonClient(databaseUrl: string, isLocalDevelopment: boolean): PrismaClient {
// https://gist.github.com/kincaidoneil/bc2516111f0ec8850cd6020b8191b27b
if (isLocalDevelopment) {
neonConfig.pipelineConnect = false;
neonConfig.useSecureWebSocket = false;
neonConfig.wsProxy = () => "localhost:5433";
}
const neon = new NeonPool({ connectionString: databaseUrl });
const adapter = new PrismaNeon(neon);
const prisma = new PrismaClient({ adapter });
return prisma;
}
Then, to make this work, run a websocket -> TCP proxy like this:

Your database URL would point to this proxy, assuming postgres is at localhost:5432

example: DATABASE_URL=postgresql://postgres:password@localhost:5433/postgres

wstcp --log-level debug --bind-addr 127.0.0.1:5433 127.0.0.1:5432

followed all your steps and still get "Error: PrismaClient is unable to run in this browser environment, or has been bundled for the browser (running in unknown)." any tips