Skip to content

Commit

Permalink
some models for dbt-impala-example are not appearing in Hue when adap…
Browse files Browse the repository at this point in the history
…ted to run with dbt-spark-livy adapter

Internal Ticket: https://jira.cloudera.com/projects/DBT/issues/DBT-124

Synopsis: When debuging the above ticket, the dbt logs indicated that there was an error in list_relations_without_caching indicating a NoneType was passed to the macro.
Further it was observed that when an actual error by SQL executing in the warehouse was thrown, the error was siliently ignored.
This PR addresses this issue.

Testplan:
Clone and sucessfully execute dbt seed and dbt run for the project https://github.com/TapasSenapati/dbt-spark-example
  • Loading branch information
tovganesh committed Jun 30, 2022
1 parent 203d0a7 commit 59fdac3
Showing 1 changed file with 9 additions and 3 deletions.
12 changes: 9 additions & 3 deletions dbt/adapters/spark_livy/livysession.py
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,8 @@ def execute(self, sql: str, *parameters: Any) -> None:

# print(res)

res = self._getLivyResult(self._submitLivyCode(self._getLivySQL(sql)))

if (res['output']['status'] == 'ok'):
# values = res['output']['data']['application/json']
values = res['output']['data']['application/json']
Expand All @@ -191,11 +193,15 @@ def execute(self, sql: str, *parameters: Any) -> None:
# print("rows", self._rows)
# print("schema", self._schema)
else:
self._rows = None
self._schema = None
self._rows = []
self._schema = []
else:
self._rows = None
self._schema = None
self._schema = None

raise dbt.exceptions.raise_database_error(
'Error while executing query: ' + res['output']['evalue']
)

def fetchall(self):
"""
Expand Down

0 comments on commit 59fdac3

Please sign in to comment.