Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix statements in on run start hooks #693

Merged
merged 4 commits into from
Apr 9, 2018

Conversation

cmcarthur
Copy link
Member

statements don't really work in on run start hooks, because the connection name used doesn't match. there's some code that uses master and some that uses the node name. this makes everything use the node name, and releases the connection after compilation in case a statement is run.

@drewbanin
Copy link
Contributor

@cmcarthur nice! This should also fix the issue with connections being closed in on-run-end hooks, right?

@cmcarthur
Copy link
Member Author

cmcarthur commented Mar 14, 2018

@drewbanin yep, that's right. if you have a test case for that, can you run this branch and see if it fixes it?

@drewbanin
Copy link
Contributor

@cmcarthur I don't have a test case handy -- let me see if I can whip one up

auto_begin=False)
adapter.release_connection(profile, conn_name)

if len(sql) > 0:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we do sql.strip() here? Unsure if it will make any difference, but I know sometimes people put conditional statements into hooks, and the else clause might contain some empty space

Copy link
Contributor

@drewbanin drewbanin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cmcarthur this LGTM if you've tested on Redshift + Snowflake with and without hooks!

# configuring psycopg2 (and other adapters?) to ensure that a
# transaction is only created if dbt initiates it.
conn_name = adapter.clear_transaction(profile)
adapter.clear_transaction(profile)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cmcarthur this will use the master connection, right? I think this can still cause issues with the on-run-end hook after a long dbt run. Do we need to clear this transaction here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess not? good call


ordered_hooks = sorted(compiled_hooks, key=lambda h: h.get('index', 0))

for hook in ordered_hooks:
model_name = compiled.get('name')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be hook, not compiled I think

@drewbanin drewbanin added this to the Adapter Parity milestone Apr 4, 2018
@drewbanin
Copy link
Contributor

This also fixes #620

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants