Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add runtime per-model warehouse config on snowflake models (#1358) #1788

Merged
merged 2 commits into from
Sep 27, 2019

Conversation

beckjake
Copy link
Contributor

Fixes #1358

  • Add warehouse config to snowflake
  • Add concept of adapter-level pre/post model hooks
  • Use those hooks to optionally set a warehouse on a per-model basis in snowflake
  • Added some silly integration tests
    • one tests that the warehouse is applied by setting a bad one and failing
    • one that tests that overriding the warehouse to a good one is ok

@cla-bot cla-bot bot added the cla:yes label Sep 27, 2019
@beckjake beckjake force-pushed the feature/warehouse-model-config branch 3 times, most recently from 0f9eb1d to 6748057 Compare September 27, 2019 16:27
Add warehouse config to snowflake
Add concept of adapter-level pre/post model hooks
Use those hooks to optionally set a warehouse on a per-model basis
Added some integration tests
 - one tests that the warehouse is applied by setting a bad one and failing
 - one that tests that overriding the warehouse to a good one is ok
logging.getLogger('requests').setLevel(logging.ERROR)
logging.getLogger('urllib3').setLevel(logging.ERROR)
logging.getLogger('google').setLevel(logging.ERROR)
logging.getLogger('snowflake.connector').setLevel(logging.ERROR)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙌 🙏

plugins/snowflake/dbt/adapters/snowflake/impl.py Outdated Show resolved Hide resolved
Copy link
Contributor

@drewbanin drewbanin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ship it

warehouse = config.get('warehouse', default_warehouse)
if warehouse == default_warehouse or warehouse is None:
return None
previous = self._get_warehouse()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i like that this cost is only incurred if a non-default warehouse is specified!

@beckjake beckjake merged commit 812c549 into dev/louisa-may-alcott Sep 27, 2019
@beckjake beckjake deleted the feature/warehouse-model-config branch September 27, 2019 20:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Snowflake: Enable choosing a warehouse at runtime
2 participants