Skip to content

Commit

Permalink
Add support for decimal values without leading digits (#794)
Browse files Browse the repository at this point in the history
Encountered issue while handling decimal values without leading digits
(for example .00457). Lexer was breaking, updated number token regex to
support the similar cases and added a corresponding test case.

Sample SQL query: `SELECT employee_id FROM employees WHERE salary > .456
* 1000000 AND bonus < .0000239 * salary;`
```
An Unexpected Error Occurred
Parse error at token: 456 at line 1 column 51 Unexpected NUMBER token: {"type":"NUMBER","raw":"456","text":"456","start":50}
```
  • Loading branch information
nene authored Oct 25, 2024
2 parents 2ad4028 + a8435b5 commit 472e17b
Show file tree
Hide file tree
Showing 2 changed files with 16 additions and 1 deletion.
2 changes: 1 addition & 1 deletion src/lexer/Tokenizer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ export default class Tokenizer {
{
type: TokenType.NUMBER,
regex:
/(?:0x[0-9a-fA-F]+|0b[01]+|(?:-\s*)?[0-9]+(?:\.[0-9]*)?(?:[eE][-+]?[0-9]+(?:\.[0-9]+)?)?)(?![\w\p{Alphabetic}])/uy,
/(?:0x[0-9a-fA-F]+|0b[01]+|(?:-\s*)?(?:[0-9]*\.[0-9]+|[0-9]+(?:\.[0-9]*)?)(?:[eE][-+]?[0-9]+(?:\.[0-9]+)?)?)(?![\w\p{Alphabetic}])/uy,
},
// RESERVED_PHRASE is matched before all other keyword tokens
// to e.g. prioritize matching "TIMESTAMP WITH TIME ZONE" phrase over "WITH" clause.
Expand Down
15 changes: 15 additions & 0 deletions test/behavesLikeSqlFormatter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -279,4 +279,19 @@ export default function behavesLikeSqlFormatter(format: FormatFn) {
tbl;
`);
});

it('supports decimal values without leading digits', () => {
const result = format(`
SELECT employee_id FROM employees WHERE salary > .456 * 1000000 AND bonus < .0000239 * salary;
`);
expect(result).toBe(dedent`
SELECT
employee_id
FROM
employees
WHERE
salary > .456 * 1000000
AND bonus < .0000239 * salary;
`);
});
}

0 comments on commit 472e17b

Please sign in to comment.