Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

better error handling for run hooks #456

Closed
drewbanin opened this issue Jun 12, 2017 · 2 comments
Closed

better error handling for run hooks #456

drewbanin opened this issue Jun 12, 2017 · 2 comments
Assignees
Labels
bug Something isn't working
Milestone

Comments

@drewbanin
Copy link
Contributor

query errors in on-run-start and on-run-end hooks aren't appropriately caught/handled.

Example backtrace:

Traceback (most recent call last):
  File "/usr/local/bin/dbt", line 11, in <module>
    load_entry_point('dbt==0.8.2', 'console_scripts', 'dbt')()
  File "/usr/local/Cellar/dbt/0.8.2/libexec/lib/python3.6/site-packages/dbt/main.py", line 37, in main
    handle(args)
  File "/usr/local/Cellar/dbt/0.8.2/libexec/lib/python3.6/site-packages/dbt/main.py", line 63, in handle
    res = run_from_args(parsed)
  File "/usr/local/Cellar/dbt/0.8.2/libexec/lib/python3.6/site-packages/dbt/main.py", line 117, in run_from_args
    result = task.run()
  File "/usr/local/Cellar/dbt/0.8.2/libexec/lib/python3.6/site-packages/dbt/task/run.py", line 18, in run
    results = runner.run_models(self.args.models, self.args.exclude)
  File "/usr/local/Cellar/dbt/0.8.2/libexec/lib/python3.6/site-packages/dbt/runner.py", line 800, in run_models
    should_run_hooks=True)
  File "/usr/local/Cellar/dbt/0.8.2/libexec/lib/python3.6/site-packages/dbt/runner.py", line 770, in run_types_from_graph
    should_execute)
  File "/usr/local/Cellar/dbt/0.8.2/libexec/lib/python3.6/site-packages/dbt/runner.py", line 627, in execute_nodes
    self.run_hooks(profile, flat_graph, dbt.utils.RunHookType.End)
  File "/usr/local/Cellar/dbt/0.8.2/libexec/lib/python3.6/site-packages/dbt/runner.py", line 496, in run_hooks
    adapter.execute_all(profile=profile, sqls=compiled_hooks)
  File "/usr/local/Cellar/dbt/0.8.2/libexec/lib/python3.6/site-packages/dbt/adapters/default.py", line 547, in execute_all
    connection, _ = cls.add_query(profile, sql, model_name)
  File "/usr/local/Cellar/dbt/0.8.2/libexec/lib/python3.6/site-packages/dbt/adapters/default.py", line 526, in add_query
    cursor.execute(sql)
psycopg2.ProgrammingError: relation "dbt_jthandy.some_table" does not exist
@drewbanin drewbanin added the bug Something isn't working label Jun 12, 2017
@drewbanin drewbanin self-assigned this Jun 12, 2017
@drewbanin drewbanin modified the milestone: 0.8.3 Jun 20, 2017
@drewbanin
Copy link
Contributor Author

This particular error shouldn't happen anymore. If there's an error in an on-run-start or on-run-end hook, dbt will hard exit, but will not show a backtrace.

$ dbt run
Encountered an error:
schema "bad_schema" does not exist
LINE 1: create table if not exists bad_schema.hook_test (id int)
                                   ^
$

@jthandy what do you think should happen if an on-run-start hook fails? I don't think we can skip dependents as we do with models, since every model is a dependent! My instinct is that cancelling the run makes sense, but the above output would benefit from more info for the end-user.

@jthandy
Copy link
Member

jthandy commented Jul 4, 2017

I don't think error messages (even ones that are passed through from the database) are a bad thing here. The only thing that would be helpful is if dbt could tell the user what it was attempting to run when it encountered that error message. Could it say:

$ dbt run
Encountered an error **while running pre-hook**:
schema "bad_schema" does not exist
LINE 1: create table if not exists bad_schema.hook_test (id int)
                                   ^

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants