Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add json traversal support. #440

Merged
merged 72 commits into from
Jul 7, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
72 commits
Select commit Hold shift + click to select a range
5036a74
add JSON operators.
igalklebanov Apr 22, 2023
ea53e33
add json operator parser types.
igalklebanov Apr 22, 2023
195c6ed
add `jxp` @ expression builder [unimplemented].
igalklebanov Apr 22, 2023
6046642
simplify and extract json operation parser types.
igalklebanov Apr 22, 2023
b26a2cd
fix jxp types / imports.
igalklebanov Apr 22, 2023
0c2ab48
fix json type performance?
igalklebanov Apr 24, 2023
2113725
Merge branch 'master' into json
igalklebanov Apr 24, 2023
eee1f86
add json path builder.
igalklebanov Apr 26, 2023
e0adc1d
support column->JSON path.
igalklebanov Apr 26, 2023
a2df0ac
Merge branch 'master' into json
igalklebanov Apr 28, 2023
6811557
Merge branch 'master' into json
igalklebanov May 13, 2023
a02f624
Merge branch 'master' into json
igalklebanov May 17, 2023
68d9b42
extend reference node.
igalklebanov May 22, 2023
4911911
make string reference parsing always return a reference node.
igalklebanov May 22, 2023
17de399
string reference builder.
igalklebanov May 22, 2023
b1864bb
reshaping json path builder.
igalklebanov May 22, 2023
200c206
json traversal operation was a bad idea.
igalklebanov May 22, 2023
2a6138d
add json-path-reference-node.
igalklebanov May 23, 2023
6fdb3ac
compile json paths.
igalklebanov May 23, 2023
0dff03e
house cleaning
igalklebanov May 23, 2023
4b62343
parse references with json paths.
igalklebanov May 23, 2023
8937b7c
build json paths.
igalklebanov May 23, 2023
94366d1
Merge branch 'experimenting-with-ref-builder' into json
igalklebanov May 23, 2023
208be66
Merge branch 'master' into json
igalklebanov May 23, 2023
2646326
fix type error.
igalklebanov May 23, 2023
8e7d71f
fix type error following parseStringReference return type widening.
igalklebanov May 25, 2023
838531b
Merge branch 'master' into json
igalklebanov May 26, 2023
7e78ad0
Merge branch 'master' into json
igalklebanov Jun 13, 2023
e059b0a
Merge branch 'master' into json
igalklebanov Jun 15, 2023
23a79e3
Merge branch 'master' into json
igalklebanov Jun 19, 2023
1c28cc0
introduce ->>$ instead of per dialect implementation.
igalklebanov Jun 19, 2023
fe87265
support ColumnType in json path builder.
igalklebanov Jun 19, 2023
9ab7ff6
Merge branch 'master' into json
igalklebanov Jun 20, 2023
a2f8041
simplify and fix json types.
igalklebanov Jun 21, 2023
2157bd5
fix pg style traversal keys compilation.
igalklebanov Jun 21, 2023
a018d33
Merge branch 'master' into json
igalklebanov Jun 21, 2023
47c0f03
Merge branch 'master' into json
igalklebanov Jun 21, 2023
a804834
array traversal should be nullish at all times.
igalklebanov Jun 23, 2023
e28b4f3
postgres only supports ->> in last leg.
igalklebanov Jun 24, 2023
0264084
finish json traversal unit tests.
igalklebanov Jun 24, 2023
cde4239
Merge branch 'master' into json
igalklebanov Jun 25, 2023
b265c12
add typings types.
igalklebanov Jun 25, 2023
c6b9243
.
igalklebanov Jun 25, 2023
5097e11
positive typings cases.
igalklebanov Jun 25, 2023
7502704
some negative typings tests.
igalklebanov Jun 25, 2023
2cfdb6d
fix wrong expected type.
igalklebanov Jun 25, 2023
4443748
Merge branch 'master' into json
igalklebanov Jun 25, 2023
4128ebd
finalize typings test.
igalklebanov Jun 26, 2023
a2f6248
document json ref @ expression builder.
igalklebanov Jun 26, 2023
8620db3
Merge branch 'master' into json
igalklebanov Jun 27, 2023
bf73830
Merge branch 'master' into json
igalklebanov Jun 27, 2023
aa97a6d
add json path builder js docs.
igalklebanov Jun 28, 2023
798007e
properties
igalklebanov Jun 28, 2023
52282bd
Merge branch 'master' into json
igalklebanov Jun 28, 2023
37ad51e
add `ParseJSONResultsPlugin`.
igalklebanov Jun 30, 2023
915689a
remove console.log
igalklebanov Jun 30, 2023
82dfa87
add additional JSON operators.
igalklebanov Jun 30, 2023
b667841
-> for pg, ->$ for mysql, ->> / ->>$ for sqlite.
igalklebanov Jun 30, 2023
a7eef99
Merge branch 'master' into json
igalklebanov Jun 30, 2023
f05bef9
add `JSONColumnType`.
igalklebanov Jun 30, 2023
a162355
update jsdocs.
igalklebanov Jun 30, 2023
e711587
Merge branch 'master' into json
igalklebanov Jul 2, 2023
6f561ec
Merge branch 'master' into json
igalklebanov Jul 3, 2023
16417ca
Merge branch 'master' into json
igalklebanov Jul 6, 2023
713e97e
omit all $ operators properly.
igalklebanov Jul 6, 2023
3f13b50
remove reference to jxp in jsdocs.
igalklebanov Jul 6, 2023
af8aae2
export json-path-reference-node.
igalklebanov Jul 6, 2023
ec4a26d
add comment about empty catch block.
igalklebanov Jul 6, 2023
4ab875d
simplify json path builder internal node type for now.
igalklebanov Jul 6, 2023
d12e447
rework json reference nodes and compilation.
igalklebanov Jul 7, 2023
9bed305
Merge branch 'master' into json
igalklebanov Jul 7, 2023
e3874f8
Merge branch 'master' into json
igalklebanov Jul 7, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@
"test:node:postgres": "DIALECT=postgres npm run test:node:run",
"test:node:mysql": "DIALECT=mysql npm run test:node:run",
"test:node:sqlite": "DIALECT=sqlite npm run test:node:run",
"test:node:run": "mocha --timeout 15000 test/node/dist/**/*.test.js",
"test:node:run": "mocha --reporter min --timeout 15000 test/node/dist/**/*.test.js",
"test:browser:build": "rm -rf test/browser/bundle.js && esbuild test/browser/main.ts --bundle --outfile=test/browser/bundle.js",
"test:browser": "npm run build && npm run test:browser:build && node test/browser/test.js",
"test:bun": "npm run build && cd test/bun && bun install && bun run test",
Expand Down
74 changes: 61 additions & 13 deletions src/expression/expression-builder.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import {
} from '../query-builder/function-module.js'
import {
ExtractTypeFromReferenceExpression,
parseJSONReference,
parseReferenceExpression,
parseStringReference,
ReferenceExpression,
Expand All @@ -40,6 +41,7 @@ import { ParensNode } from '../operation-node/parens-node.js'
import { ExpressionWrapper } from './expression-wrapper.js'
import {
ComparisonOperator,
JSONOperatorWith$,
UnaryOperator,
} from '../operation-node/operator-node.js'
import { SqlBool } from '../util/type-utils.js'
Expand All @@ -53,6 +55,7 @@ import { ValueNode } from '../operation-node/value-node.js'
import { CaseBuilder } from '../query-builder/case-builder.js'
import { CaseNode } from '../operation-node/case-node.js'
import { isUndefined } from '../util/object-utils.js'
import { JSONPathBuilder } from '../query-builder/json-path-builder.js'

export interface ExpressionBuilder<DB, TB extends keyof DB> {
/**
Expand Down Expand Up @@ -298,7 +301,11 @@ export interface ExpressionBuilder<DB, TB extends keyof DB> {
case<O>(expression: Expression<O>): CaseBuilder<DB, TB, O>

/**
* This can be used to reference columns.
* This method can be used to reference columns within the query's context. For
* a non-type-safe version of this method see {@link sql}'s version.
*
* Additionally, this method can be used to reference nested JSON properties or
* array elements. See {@link JSONPathBuilder} for more information.
*
* ### Examples
*
Expand All @@ -314,9 +321,8 @@ export interface ExpressionBuilder<DB, TB extends keyof DB> {
* ]))
* ```
*
* In the next example we use the `ref` method to reference
* columns of the virtual table `excluded` in a type-safe way
* to create an upsert operation:
* In the next example we use the `ref` method to reference columns of the virtual
* table `excluded` in a type-safe way to create an upsert operation:
*
* ```ts
* db.insertInto('person')
Expand All @@ -330,19 +336,60 @@ export interface ExpressionBuilder<DB, TB extends keyof DB> {
* )
* ```
*
* In the next example we use `ref` in a raw sql expression. Unless you
* want to be as type-safe as possible, this is probably overkill:
* In the next example we use `ref` in a raw sql expression. Unless you want
* to be as type-safe as possible, this is probably overkill:
*
* ```ts
* db.update('pet').set((eb) => ({
* name: sql`concat(${eb.ref('pet.name')}, ${suffix})`
* }))
* ```
*
* In the next example we use `ref` to reference a nested JSON property:
*
* ```ts
* db.selectFrom('person')
* .where(({ cmpr, ref }) => cmpr(
* ref('address', '->').key('state').key('abbr'),
* '=',
* 'CA'
* ))
* .selectAll()
* ```
*
* The generated SQL (PostgreSQL):
*
* ```sql
* select * from "person" where "address"->'state'->'abbr' = $1
* ```
*
* You can also compile to a JSON path expression by using the `->$`or `->>$` operator:
*
* ```ts
* db.selectFrom('person')
* .select(({ ref }) =>
* ref('experience', '->$')
* .at('last')
* .key('title')
* .as('current_job')
* )
* ```
*
* The generated SQL (MySQL):
*
* ```sql
* select `experience`->'$[last].title' as `current_job` from `person`
* ```
*/
ref<RE extends StringReference<DB, TB>>(
reference: RE
): ExpressionWrapper<DB, TB, ExtractTypeFromReferenceExpression<DB, TB, RE>>

ref<RE extends StringReference<DB, TB>>(
reference: RE,
op: JSONOperatorWith$
): JSONPathBuilder<ExtractTypeFromReferenceExpression<DB, TB, RE>>

/**
* Returns a value expression.
*
Expand Down Expand Up @@ -690,13 +737,14 @@ export function createExpressionBuilder<DB, TB extends keyof DB>(
},

ref<RE extends StringReference<DB, TB>>(
reference: RE
): ExpressionWrapper<
DB,
TB,
ExtractTypeFromReferenceExpression<DB, TB, RE>
> {
return new ExpressionWrapper(parseStringReference(reference))
reference: RE,
op?: JSONOperatorWith$
): any {
if (isUndefined(op)) {
return new ExpressionWrapper(parseStringReference(reference))
}

return new JSONPathBuilder(parseJSONReference(reference, op))
},

val<VE>(
Expand Down
6 changes: 6 additions & 0 deletions src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ export * from './query-builder/update-result.js'
export * from './query-builder/on-conflict-builder.js'
export * from './query-builder/aggregate-function-builder.js'
export * from './query-builder/case-builder.js'
export * from './query-builder/json-path-builder.js'

export * from './raw-builder/raw-builder.js'
export * from './raw-builder/sql.js'
Expand Down Expand Up @@ -97,6 +98,7 @@ export * from './plugin/kysely-plugin.js'
export * from './plugin/camel-case/camel-case-plugin.js'
export * from './plugin/deduplicate-joins/deduplicate-joins-plugin.js'
export * from './plugin/with-schema/with-schema-plugin.js'
export * from './plugin/parse-json-results/parse-json-results-plugin.js'

export * from './operation-node/add-column-node.js'
export * from './operation-node/add-constraint-node.js'
Expand Down Expand Up @@ -183,6 +185,10 @@ export * from './operation-node/set-operation-node.js'
export * from './operation-node/binary-operation-node.js'
export * from './operation-node/unary-operation-node.js'
export * from './operation-node/using-node.js'
export * from './operation-node/json-reference-node.js'
export * from './operation-node/json-path-leg-node.js'
export * from './operation-node/json-path-node.js'
igalklebanov marked this conversation as resolved.
Show resolved Hide resolved
export * from './operation-node/json-operator-chain-node.js'

export * from './util/column-type.js'
export * from './util/compilable.js'
Expand Down
37 changes: 37 additions & 0 deletions src/operation-node/json-operator-chain-node.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
import { freeze } from '../util/object-utils.js'
import { OperationNode } from './operation-node.js'
import { OperatorNode } from './operator-node.js'
import { ValueNode } from './value-node.js'

export interface JSONOperatorChainNode extends OperationNode {
readonly kind: 'JSONOperatorChainNode'
readonly operator: OperatorNode
readonly values: readonly ValueNode[]
}

/**
* @internal
*/
export const JSONOperatorChainNode = freeze({
is(node: OperationNode): node is JSONOperatorChainNode {
return node.kind === 'JSONOperatorChainNode'
},

create(operator: OperatorNode): JSONOperatorChainNode {
return freeze({
kind: 'JSONOperatorChainNode',
operator,
values: freeze([]),
})
},

cloneWithValue(
node: JSONOperatorChainNode,
value: ValueNode
): JSONOperatorChainNode {
return freeze({
...node,
values: freeze([...node.values, value]),
})
},
})
27 changes: 27 additions & 0 deletions src/operation-node/json-path-leg-node.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
import { freeze } from '../util/object-utils.js'
import { OperationNode } from './operation-node.js'

export type JSONPathLegType = 'Member' | 'ArrayLocation'

export interface JSONPathLegNode extends OperationNode {
readonly kind: 'JSONPathLegNode'
readonly type: JSONPathLegType
readonly value: string | number
}

/**
* @internal
*/
export const JSONPathLegNode = freeze({
is(node: OperationNode): node is JSONPathLegNode {
return node.kind === 'JSONPathLegNode'
},

create(type: JSONPathLegType, value: string | number): JSONPathLegNode {
return freeze({
kind: 'JSONPathLegNode',
type,
value,
})
},
})
37 changes: 37 additions & 0 deletions src/operation-node/json-path-node.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
import { freeze } from '../util/object-utils.js'
import { JSONPathLegNode } from './json-path-leg-node.js'
import { OperationNode } from './operation-node.js'
import { OperatorNode } from './operator-node.js'

export interface JSONPathNode extends OperationNode {
readonly kind: 'JSONPathNode'
readonly inOperator?: OperatorNode
readonly pathLegs: ReadonlyArray<JSONPathLegNode>
}

/**
* @internal
*/
export const JSONPathNode = freeze({
is(node: OperationNode): node is JSONPathNode {
return node.kind === 'JSONPathNode'
},

create(inOperator?: OperatorNode): JSONPathNode {
return freeze({
kind: 'JSONPathNode',
inOperator,
pathLegs: freeze([]),
})
},

cloneWithLeg(
jsonPathNode: JSONPathNode,
pathLeg: JSONPathLegNode
): JSONPathNode {
return freeze({
...jsonPathNode,
pathLegs: freeze([...jsonPathNode.pathLegs, pathLeg]),
})
},
})
41 changes: 41 additions & 0 deletions src/operation-node/json-reference-node.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
import { freeze } from '../util/object-utils.js'
import { JSONOperatorChainNode } from './json-operator-chain-node.js'
import { JSONPathNode } from './json-path-node.js'
import { OperationNode } from './operation-node.js'
import { ReferenceNode } from './reference-node.js'

export interface JSONReferenceNode extends OperationNode {
readonly kind: 'JSONReferenceNode'
readonly reference: ReferenceNode
readonly traversal: JSONPathNode | JSONOperatorChainNode
}

/**
* @internal
*/
export const JSONReferenceNode = freeze({
is(node: OperationNode): node is JSONReferenceNode {
return node.kind === 'JSONReferenceNode'
},

create(
reference: ReferenceNode,
traversal: JSONPathNode | JSONOperatorChainNode
): JSONReferenceNode {
return freeze({
kind: 'JSONReferenceNode',
reference,
traversal,
})
},

cloneWithTraversal(
node: JSONReferenceNode,
traversal: JSONPathNode | JSONOperatorChainNode
): JSONReferenceNode {
return freeze({
...node,
traversal,
})
},
})
44 changes: 43 additions & 1 deletion src/operation-node/operation-node-transformer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,10 @@ import { UsingNode } from './using-node.js'
import { FunctionNode } from './function-node.js'
import { CaseNode } from './case-node.js'
import { WhenNode } from './when-node.js'
import { JSONReferenceNode } from './json-reference-node.js'
import { JSONPathNode } from './json-path-node.js'
import { JSONPathLegNode } from './json-path-leg-node.js'
import { JSONOperatorChainNode } from './json-operator-chain-node.js'

/**
* Transforms an operation node tree into another one.
Expand Down Expand Up @@ -198,6 +202,10 @@ export class OperationNodeTransformer {
FunctionNode: this.transformFunction.bind(this),
CaseNode: this.transformCase.bind(this),
WhenNode: this.transformWhen.bind(this),
JSONReferenceNode: this.transformJSONReference.bind(this),
JSONPathNode: this.transformJSONPath.bind(this),
JSONPathLegNode: this.transformJSONPathLeg.bind(this),
JSONOperatorChainNode: this.transformJSONOperatorChain.bind(this),
})

transformNode<T extends OperationNode | undefined>(node: T): T {
Expand Down Expand Up @@ -286,8 +294,8 @@ export class OperationNodeTransformer {
protected transformReference(node: ReferenceNode): ReferenceNode {
return requireAllProps<ReferenceNode>({
kind: 'ReferenceNode',
table: this.transformNode(node.table),
column: this.transformNode(node.column),
table: this.transformNode(node.table),
})
}

Expand Down Expand Up @@ -920,6 +928,40 @@ export class OperationNodeTransformer {
})
}

protected transformJSONReference(node: JSONReferenceNode): JSONReferenceNode {
return requireAllProps<JSONReferenceNode>({
kind: 'JSONReferenceNode',
reference: this.transformNode(node.reference),
traversal: this.transformNode(node.traversal),
})
}

protected transformJSONPath(node: JSONPathNode): JSONPathNode {
return requireAllProps<JSONPathNode>({
kind: 'JSONPathNode',
inOperator: this.transformNode(node.inOperator),
pathLegs: this.transformNodeList(node.pathLegs),
})
}

protected transformJSONPathLeg(node: JSONPathLegNode): JSONPathLegNode {
return requireAllProps<JSONPathLegNode>({
kind: 'JSONPathLegNode',
type: node.type,
value: node.value,
})
}

protected transformJSONOperatorChain(
node: JSONOperatorChainNode
): JSONOperatorChainNode {
return requireAllProps<JSONOperatorChainNode>({
kind: 'JSONOperatorChainNode',
operator: this.transformNode(node.operator),
values: this.transformNodeList(node.values),
})
}

protected transformDataType(node: DataTypeNode): DataTypeNode {
// An Object.freezed leaf node. No need to clone.
return node
Expand Down
Loading