Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

16 latest #117

Merged
merged 3 commits into from
Apr 4, 2024
Merged

16 latest #117

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
001ad1f
@pgsql/parser for PG16 — no deparse yet supported
pyramation Apr 4, 2024
769da98
readme
pyramation Apr 4, 2024
701ca23
workflows
pyramation Apr 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .github/workflows/run-tests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,14 @@ jobs:
POSTGRES_HOST: localhost
POSTGRES_PORT: 5432

- name: pgsql-parser
run: cd ./packages/pgsql-parser && yarn test
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: password
POSTGRES_HOST: localhost
POSTGRES_PORT: 5432

- name: deparser
run: cd ./packages/deparser && yarn test
env:
Expand All @@ -65,6 +73,9 @@ jobs:
POSTGRES_HOST: localhost
POSTGRES_PORT: 5432

- name: pgsql-enums
run: cd ./packages/pgsql-enums && yarn test

- name: enums
run: cd ./packages/enums && yarn test

Expand Down
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,7 @@ Our latest is built with `13-latest` branch from libpg_query

* [pgsql-parser](https://github.com/launchql/pgsql-parser): The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration.
* [pgsql-deparser](https://github.com/launchql/pgsql-parser/tree/main/packages/deparser): A streamlined tool designed for converting PostgreSQL ASTs back into SQL queries, focusing solely on deparser functionality to complement `pgsql-parser`.
* [pgsql-enums](https://github.com/launchql/pgsql-parser/tree/main/packages/enums-json): A utility package offering easy access to PostgreSQL enumeration types in JSON format, aiding in string and integer conversions of enums used within ASTs to compliment `pgsql-parser`
* [pgsql-enums](https://github.com/launchql/pgsql-parser/tree/main/packages/pgsql-enums): A utility package offering easy access to PostgreSQL enumeration types in JSON format, aiding in string and integer conversions of enums used within ASTs to compliment `pgsql-parser`
* [@pgsql/enums](https://github.com/launchql/pgsql-parser/tree/main/packages/enums): Provides PostgreSQL AST enums in TypeScript, enhancing type safety and usability in projects interacting with PostgreSQL AST nodes.
* [@pgsql/types](https://github.com/launchql/pgsql-parser/tree/main/packages/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs.
* [@pgsql/utils](https://github.com/launchql/pgsql-parser/tree/main/packages/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs.
Expand Down
2 changes: 1 addition & 1 deletion packages/deparser/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ Our latest is built with `13-latest` branch from libpg_query

* [pgsql-parser](https://github.com/launchql/pgsql-parser): The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration.
* [pgsql-deparser](https://github.com/launchql/pgsql-parser/tree/main/packages/deparser): A streamlined tool designed for converting PostgreSQL ASTs back into SQL queries, focusing solely on deparser functionality to complement `pgsql-parser`.
* [pgsql-enums](https://github.com/launchql/pgsql-parser/tree/main/packages/enums-json): A utility package offering easy access to PostgreSQL enumeration types in JSON format, aiding in string and integer conversions of enums used within ASTs to compliment `pgsql-parser`
* [pgsql-enums](https://github.com/launchql/pgsql-parser/tree/main/packages/pgsql-enums): A utility package offering easy access to PostgreSQL enumeration types in JSON format, aiding in string and integer conversions of enums used within ASTs to compliment `pgsql-parser`
* [@pgsql/enums](https://github.com/launchql/pgsql-parser/tree/main/packages/enums): Provides PostgreSQL AST enums in TypeScript, enhancing type safety and usability in projects interacting with PostgreSQL AST nodes.
* [@pgsql/types](https://github.com/launchql/pgsql-parser/tree/main/packages/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs.
* [@pgsql/utils](https://github.com/launchql/pgsql-parser/tree/main/packages/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs.
Expand Down
2 changes: 1 addition & 1 deletion packages/enums/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ In this example, `A_Expr_Kind` is an enum imported from `@pgsql/enums`, and expr

* [pgsql-parser](https://github.com/launchql/pgsql-parser): The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration.
* [pgsql-deparser](https://github.com/launchql/pgsql-parser/tree/main/packages/deparser): A streamlined tool designed for converting PostgreSQL ASTs back into SQL queries, focusing solely on deparser functionality to complement `pgsql-parser`.
* [pgsql-enums](https://github.com/launchql/pgsql-parser/tree/main/packages/enums-json): A utility package offering easy access to PostgreSQL enumeration types in JSON format, aiding in string and integer conversions of enums used within ASTs to compliment `pgsql-parser`
* [pgsql-enums](https://github.com/launchql/pgsql-parser/tree/main/packages/pgsql-enums): A utility package offering easy access to PostgreSQL enumeration types in JSON format, aiding in string and integer conversions of enums used within ASTs to compliment `pgsql-parser`
* [@pgsql/enums](https://github.com/launchql/pgsql-parser/tree/main/packages/enums): Provides PostgreSQL AST enums in TypeScript, enhancing type safety and usability in projects interacting with PostgreSQL AST nodes.
* [@pgsql/types](https://github.com/launchql/pgsql-parser/tree/main/packages/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs.
* [@pgsql/utils](https://github.com/launchql/pgsql-parser/tree/main/packages/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs.
Expand Down
65 changes: 12 additions & 53 deletions packages/parser/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# pgsql-parser
# @pgsql/parser

<p align="center" width="100%">
<img height="120" src="https://github.com/launchql/pgsql-parser/assets/545047/6440fa7d-918b-4a3b-8d1b-755d85de8bea" />
Expand All @@ -14,12 +14,14 @@
<a href="https://www.npmjs.com/package/pgsql-parser"><img height="20" src="https://img.shields.io/github/package-json/v/launchql/pgsql-parser?filename=packages%2Fparser%2Fpackage.json"/></a>
</p>

The real PostgreSQL parser for Node.js, `pgsql-parser` provides symmetric parsing and deparsing of SQL statements using the actual [PostgreSQL parser](https://github.com/pganalyze/libpg_query). It allows you to parse SQL queries into AST and modify or reconstruct SQL queries from the AST.
**NOTE:** This version of `@pgsql/parser` only contains the parsing functionality, optimized for users who do not require deparsing capabilities. This version is specifically tailored for PostgreSQL 16.

The real PostgreSQL parser for Node.js, `@pgsql/parser` offers the ability to parse SQL statements into ASTs using the actual [PostgreSQL parser](https://github.com/pganalyze/libpg_query). It enables users to parse SQL queries into AST format for further manipulation and analysis. Deparsing functionality is (yet) not included in this package.

## Installation

```sh
npm install pgsql-parser
npm install @pgsql/parser
```

## Key Features
Expand All @@ -33,61 +35,15 @@ npm install pgsql-parser
Rewrite part of a SQL query:

```js
import { parse, deparse } from 'pgsql-parser';
import { parse } from '@pgsql/parser';

const stmts = parse('SELECT * FROM test_table');

// Assuming the structure of stmts is known and matches the expected type
stmts[0].RawStmt.stmt.SelectStmt.fromClause[0].RangeVar.relname = 'another_table';

console.log(deparse(stmts));

// SELECT * FROM "another_table"
```

## Deparser Example

The `pgsql-deparser` module serializes ASTs to SQL in pure TypeScript, avoiding the full parser's native dependencies. It's useful when only SQL string conversion from ASTs is needed, and is written in pure TypeScript for easy cross-environment deployment.

Here's how you can use the deparser in your TypeScript code, using [`@pgsql/utils`](https://github.com/launchql/pgsql-parser/tree/main/packages/utils) to create an AST for `deparse`:

```ts
import ast, { SelectStmt } from '@pgsql/utils';
import { deparse } from 'pgsql-deparser';

// This could have been obtained from any JSON or AST, not necessarily @pgsql/utils
const stmt: SelectStmt = ast.selectStmt({
targetList: [
ast.resTarget({
val: ast.columnRef({
fields: [ast.aStar()]
})
})
],
fromClause: [
ast.rangeVar({
relname: 'some_table',
inh: true,
relpersistence: 'p'
})
],
limitOption: 'LIMIT_OPTION_DEFAULT',
op: 'SETOP_NONE'
});

// Modify the AST if needed
stmt.SelectStmt.fromClause[0].RangeVar.relname = 'another_table';

// Deparse the modified AST back to a SQL string
console.log(deparse(stmts));

// Output: SELECT * FROM another_table
```

## CLI

```
npm install -g pgsql-parser
npm install -g @pgsql/parser
```

### usage
Expand Down Expand Up @@ -140,7 +96,10 @@ Our latest is built with `13-latest` branch from libpg_query

| PostgreSQL Major Version | libpg_query | Status | npm
|--------------------------|-------------|---------------------|---------|
| 13 | 13-latest | Active development | `latest`
| 16 | 16-latest | Active development | `latest`
| 15 | (n/a) | Not supported |
| 14 | (n/a) | Not supported |
| 13 | 13-latest | use `pgsql-parser` |
| 12 | (n/a) | Not supported |
| 11 | (n/a) | Not supported |
| 10 | 10-latest | Not supported | `@1.3.1` ([tree](https://github.com/launchql/pgsql-parser/tree/39b7b1adc8914253226e286a48105785219a81ca)) |
Expand All @@ -150,7 +109,7 @@ Our latest is built with `13-latest` branch from libpg_query

* [pgsql-parser](https://github.com/launchql/pgsql-parser): The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration.
* [pgsql-deparser](https://github.com/launchql/pgsql-parser/tree/main/packages/deparser): A streamlined tool designed for converting PostgreSQL ASTs back into SQL queries, focusing solely on deparser functionality to complement `pgsql-parser`.
* [pgsql-enums](https://github.com/launchql/pgsql-parser/tree/main/packages/enums-json): A utility package offering easy access to PostgreSQL enumeration types in JSON format, aiding in string and integer conversions of enums used within ASTs to compliment `pgsql-parser`
* [pgsql-enums](https://github.com/launchql/pgsql-parser/tree/main/packages/pgsql-enums): A utility package offering easy access to PostgreSQL enumeration types in JSON format, aiding in string and integer conversions of enums used within ASTs to compliment `pgsql-parser`
* [@pgsql/enums](https://github.com/launchql/pgsql-parser/tree/main/packages/enums): Provides PostgreSQL AST enums in TypeScript, enhancing type safety and usability in projects interacting with PostgreSQL AST nodes.
* [@pgsql/types](https://github.com/launchql/pgsql-parser/tree/main/packages/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs.
* [@pgsql/utils](https://github.com/launchql/pgsql-parser/tree/main/packages/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs.
Expand Down
22 changes: 9 additions & 13 deletions packages/parser/__tests__/__snapshots__/async.test.ts.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ exports[`works 1`] = `
"fields": [
{
"String": {
"str": "health_grade",
"sval": "health_grade",
},
},
],
Expand All @@ -57,17 +57,15 @@ exports[`works 1`] = `
"name": [
{
"String": {
"str": "=",
"sval": "=",
},
},
],
"rexpr": {
"A_Const": {
"location": undefined,
"val": {
"String": {
"str": "A",
},
"sval": {
"sval": "A",
},
},
},
Expand All @@ -81,7 +79,7 @@ exports[`works 1`] = `
"fields": [
{
"String": {
"str": "average_rating",
"sval": "average_rating",
},
},
],
Expand All @@ -92,18 +90,16 @@ exports[`works 1`] = `
"name": [
{
"String": {
"str": ">=",
"sval": ">=",
},
},
],
"rexpr": {
"A_Const": {
"location": undefined,
"val": {
"Float": {
"str": "4.5",
},
"fval": {
"fval": "4.5",
},
"location": undefined,
},
},
},
Expand Down
24 changes: 16 additions & 8 deletions packages/parser/__tests__/__snapshots__/funcs.test.ts.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@ exports[`works 1`] = `
"PLpgSQL_stmt_assign": {
"expr": {
"PLpgSQL_expr": {
"query": "SELECT ast.jsonb_set(result, '{IndexElem, name}', to_jsonb(v_name))",
"parseMode": 3,
"query": "result = ast.jsonb_set(result, '{IndexElem, name}', to_jsonb(v_name))",
},
},
"lineno": 5,
Expand All @@ -22,7 +23,8 @@ exports[`works 1`] = `
"PLpgSQL_stmt_assign": {
"expr": {
"PLpgSQL_expr": {
"query": "SELECT ast.jsonb_set(result, '{IndexElem, ordering}', to_jsonb(v_ordering))",
"parseMode": 3,
"query": "result = ast.jsonb_set(result, '{IndexElem, ordering}', to_jsonb(v_ordering))",
},
},
"lineno": 6,
Expand All @@ -33,7 +35,8 @@ exports[`works 1`] = `
"PLpgSQL_stmt_assign": {
"expr": {
"PLpgSQL_expr": {
"query": "SELECT ast.jsonb_set(result, '{IndexElem, nulls_ordering}', to_jsonb(v_nulls_ordering))",
"parseMode": 3,
"query": "result = ast.jsonb_set(result, '{IndexElem, nulls_ordering}', to_jsonb(v_nulls_ordering))",
},
},
"lineno": 7,
Expand All @@ -44,7 +47,8 @@ exports[`works 1`] = `
"PLpgSQL_stmt_assign": {
"expr": {
"PLpgSQL_expr": {
"query": "SELECT ast.jsonb_set(result, '{IndexElem, expr}', v_expr)",
"parseMode": 3,
"query": "result = ast.jsonb_set(result, '{IndexElem, expr}', v_expr)",
},
},
"lineno": 8,
Expand All @@ -55,7 +59,8 @@ exports[`works 1`] = `
"PLpgSQL_stmt_assign": {
"expr": {
"PLpgSQL_expr": {
"query": "SELECT ast.jsonb_set(result, '{IndexElem, opclass}', v_opclass)",
"parseMode": 3,
"query": "result = ast.jsonb_set(result, '{IndexElem, opclass}', v_opclass)",
},
},
"lineno": 9,
Expand All @@ -66,7 +71,8 @@ exports[`works 1`] = `
"PLpgSQL_stmt_assign": {
"expr": {
"PLpgSQL_expr": {
"query": "SELECT ast.jsonb_set(result, '{IndexElem, collation}', v_collation)",
"parseMode": 3,
"query": "result = ast.jsonb_set(result, '{IndexElem, collation}', v_collation)",
},
},
"lineno": 10,
Expand All @@ -77,7 +83,8 @@ exports[`works 1`] = `
"PLpgSQL_stmt_return": {
"expr": {
"PLpgSQL_expr": {
"query": "SELECT result",
"parseMode": 2,
"query": "result",
},
},
"lineno": 11,
Expand Down Expand Up @@ -167,7 +174,8 @@ exports[`works 1`] = `
},
"default_val": {
"PLpgSQL_expr": {
"query": "SELECT '{"IndexElem":{}}'::jsonb",
"parseMode": 2,
"query": "'{"IndexElem":{}}'::jsonb",
},
},
"lineno": 3,
Expand Down
9 changes: 4 additions & 5 deletions packages/parser/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "pgsql-parser",
"version": "13.15.0",
"name": "@pgsql/parser",
"version": "16.0.0-testing",
"description": "The real PostgreSQL query parser",
"author": "Dan Lynch <pyramation@gmail.com>",
"homepage": "https://github.com/launchql/pgsql-parser",
Expand Down Expand Up @@ -84,9 +84,8 @@
"typescript": "^5.0.4"
},
"dependencies": {
"libpg-query": "13.3.2",
"libpg-query": "16.1.0",
"minimist": "^1.2.6",
"pgsql-deparser": "^13.14.0",
"pgsql-enums": "^13.9.0"
"@pgsql/types": "13.8.0"
}
}
7 changes: 4 additions & 3 deletions packages/parser/src/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { Deparser, deparse } from 'pgsql-deparser';
import { Node } from '@pgsql/types';

import {
parseQuery,
parseQuerySync,
Expand All @@ -15,7 +16,7 @@ function mapStmt({ stmt, stmt_len, stmt_location }) {
};
}

export const parse = (sql) => {
export const parse = (sql): Node => {
if (!sql) throw new Error('no SQL provided to parser');
const result = parseQuerySync(sql);
return result.stmts.map(mapStmt);
Expand All @@ -27,4 +28,4 @@ export const parseAsync = async (sql) => {
return result.stmts.map(mapStmt);
};

export { deparse, Deparser, parseFunction };
export { parseFunction };
0 packages/enums-json/.editorconfig → packages/pgsql-enums/.editorconfig
View file
Open in desktop
File renamed without changes.
0 packages/enums-json/.eslintignore → packages/pgsql-enums/.eslintignore
View file
Open in desktop
File renamed without changes.
0 packages/enums-json/.eslintrc.js → packages/pgsql-enums/.eslintrc.js
View file
Open in desktop
File renamed without changes.
0 packages/enums-json/.gitignore → packages/pgsql-enums/.gitignore
View file
Open in desktop
File renamed without changes.
0 packages/enums-json/.npmignore → packages/pgsql-enums/.npmignore
View file
Open in desktop
File renamed without changes.
0 packages/enums-json/.npmrc → packages/pgsql-enums/.npmrc
View file
Open in desktop
File renamed without changes.
0 packages/enums-json/CHANGELOG.md → packages/pgsql-enums/CHANGELOG.md
View file
Open in desktop
File renamed without changes.
0 packages/enums-json/LICENSE → packages/pgsql-enums/LICENSE
View file
Open in desktop
File renamed without changes.
Loading
Loading