Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exasol: InvalidSyntaxException when using multiple CTEs with the same base table and LOCAL #646

Open
Briimbo opened this issue Aug 22, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@Briimbo
Copy link

Briimbo commented Aug 22, 2024

Describe the bug
The SQL statement below is valid for Exasol and produces the desired results when used in production (in a more complex scenario).
However, sqllineage throws an error when processing the file.
I think this is because COL_A appears in multiple CTEs and the correct base table can not be determined when LOCAL is used as well, maybe because of a wrong evaluation order.

Interestingly, the error is not thrown if there are no parenthesis in the last line, e.g. LOCAL.COL_A - LOCAL.COL_B AS COL_C works fine. Also, using other dialects like ansi work as well. However, this is not a possible workaround in our scenario.

SQL

INSERT INTO TABLE_A
(COL_A,
 COL_B,
 COL_C)

WITH CTE1 AS (SELECT COL_A, ID FROM TABLE_B WHERE COND_A),
     CTE2 AS (SELECT COL_A, ID FROM TABLE_C WHERE COND_B),

SELECT CTE1.COL_A                      AS COL_A,
       CTE2.COL_A                      AS COL_B,
       (LOCAL.COL_A - LOCAL.COL_B) / 5 AS COL_C
FROM CTE1 
    LEFT JOIN CTE2 ON CTE1.ID = CTE2.ID;

To Reproduce

  • if CLI (Command Line Interface): provide the command you're calling and the output.
    For example:
sqllineage -f test.sql --dialect=exasol
 File "/home/vscode/venv/bin/sqllineage", line 8, in <module>
    sys.exit(main())
  File "/home/vscode/venv/lib/python3.10/site-packages/sqllineage/cli.py", line 109, in main
    runner.print_table_lineage()
  File "/home/vscode/venv/lib/python3.10/site-packages/sqllineage/runner.py", line 176, in print_table_lineage
    print(str(self))
  File "/home/vscode/venv/lib/python3.10/site-packages/sqllineage/runner.py", line 26, in wrapper
    self._eval()
  File "/home/vscode/venv/lib/python3.10/site-packages/sqllineage/runner.py", line 194, in _eval
    self._sql_holder = SQLLineageHolder.of(
  File "/home/vscode/venv/lib/python3.10/site-packages/sqllineage/core/holders.py", line 349, in of
    g = SQLLineageHolder._build_digraph(metadata_provider, *args)
  File "/home/vscode/venv/lib/python3.10/site-packages/sqllineage/core/holders.py", line 330, in _build_digraph
    raise InvalidSyntaxException(
sqllineage.exceptions.InvalidSyntaxException: col_a is not allowed from more than one table or subquery

Expected behavior
The SQL should be processed correctly, the output should identical to the working case of using --dialect=ansi.
The result should be:

Statements(#): 1
Source Tables:
.cte1
.cte2
.table_b
.table_c
Target Tables:
.table_a

Python version (available via python --version)
3.10.12

SQLLineage version (available via sqllineage --version):
1.4.9

Additional context
Dialect: Exasol

@Briimbo Briimbo added the bug Something isn't working label Aug 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant