Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Impl] Call adapter.pre_model_hook + adapter.post_model_hook within TestTask #10198

Closed
1 task done
jtcohen6 opened this issue May 21, 2024 · 1 comment · Fixed by #10258
Closed
1 task done

[Impl] Call adapter.pre_model_hook + adapter.post_model_hook within TestTask #10198

jtcohen6 opened this issue May 21, 2024 · 1 comment · Fixed by #10258
Assignees

Comments

@jtcohen6
Copy link
Contributor

Housekeeping

  • I am a maintainer of dbt-core

Short description

Before running models, dbt runs an adapter-specified "hook." Note that this is different from user-configured model hooks. In dbt-snowflake, this is the mechanism we use for calling use warehouse {snowflake_warehouse}, and it's what we should also use for custom secondary roles (and other things in the future).

Here's where that happens within RunTask:

hook_ctx = self.adapter.pre_model_hook(context_config)
try:
result = MacroGenerator(
materialization_macro, context, stack=context["context_macro_stack"]
)()
finally:
self.adapter.post_model_hook(context_config, hook_ctx)

We want the same thing to happen within TestTask, for both data + unit tests:

# generate materialization macro
macro_func = MacroGenerator(materialization_macro, context)
# execute materialization macro
macro_func()

# generate materialization macro
macro_func = MacroGenerator(materialization_macro, context)
# execute materialization macro
try:
macro_func()
except DbtBaseException as e:
raise DbtRuntimeError(
f"An error occurred during execution of unit test '{unit_test_def.name}'. "
f"There may be an error in the unit test definition: check the data types.\n {e}"
)

This way, unit + data tests can use custom Snowflake warehouses.

Acceptance criteria

adapter.pre_model_hook + adapter.post_model_hook are being called within TestTask for unit + data tests.

Suggested Tests

I don't think we're doing a great job of testing that pre_model_hook and post_model_hook are called today.

Ideas:

Impact to Other Teams

Impact: Adapters

Will backports be required?

no

Context

Original issue:

However, compilation is out of scope. This issue is focused only on adding these hook calls to TestTask.

@adrian-pasek-prv
Copy link

adrian-pasek-prv commented Sep 18, 2024

@jtcohen6 @McKnight-42 I'm not sure it resolved the issue with tests using pre/post hooks. When I configure +snowflake_warehouse under tests, it does not seem to take affect. Tests are still being executed with default WH set in the environment. I'm using the latest version.

#dbt_project.yml

tests:
    my_project:
      +snowflake_warehouse: "MY_WH_XS"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants