Skip to content

Commit

Permalink
Example of custom macros in get_where_subquery (#2418)
Browse files Browse the repository at this point in the history
* Example of custom macros in get_where_subquery

* Close tag

* Apply suggestions from code review

Just picked some options so we can get this merged. Nothing stopping us from continuing to make updates if we change our minds.

Co-authored-by: mirnawong1 <89008547+mirnawong1@users.noreply.github.com>
Co-authored-by: Doug Beatty <44704949+dbeatty10@users.noreply.github.com>
  • Loading branch information
3 people authored Dec 1, 2022
1 parent bd26d40 commit bca2272
Showing 1 changed file with 45 additions and 0 deletions.
45 changes: 45 additions & 0 deletions website/docs/reference/resource-configs/where.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,8 @@ tests:
### Custom logic
The rendering context for the `where` config is the same as for all configurations defined in `.yml` files. You have access to `{{ var() }}` and `{{ env_var() }}`, but you **do not** have access to custom macros for setting this config. If you do want to use custom macros to template out the `where` filter for certain tests, there is a workaround.

As of v0.21, dbt defines a [`get_where_subquery` macro](https://github.com/dbt-labs/dbt-core/blob/main/core/dbt/include/global_project/macros/materializations/tests/where_subquery.sql).

dbt replaces `{{ model }}` in generic test definitions with `{{ get_where_subquery(relation) }}`, where `relation` is a `ref()` or `source()` for the resource being tested. The default implementation of this macro returns:
Expand All @@ -132,3 +134,46 @@ dbt replaces `{{ model }}` in generic test definitions with `{{ get_where_subque
You can override this behavior by:
- Defining a custom `get_where_subquery` in your root project
- Defining a custom `<adapter>__get_where_subquery` [dispatch candidate](dispatch) in your package or adapter plugin

Within this macro definition, you can reference whatever custom macros you want, based on static inputs from the configuration. At simplest, this enables you to DRY up code that you'd otherwise need to repeat across many different `.yml` files. Because the `get_where_subquery` macro is resolved at runtime, your custom macros can also include [fetching the results of introspective database queries](https://docs.getdbt.com/reference/dbt-jinja-functions/run_query).

**Example:** Filter your test to the past three days of data, using dbt's cross-platform [`dateadd()`](https://docs.getdbt.com/reference/dbt-jinja-functions/cross-database-macros#dateadd) utility macro.

<File name='models/config.yml'>

```yml
version: 2
models:
- name: my_model
columns:
- name: id
tests:
- unique:
config:
where: "date_column > __last_three_days__" # placeholder string for static config
```

</File>

<File name='macros/custom_get_where_subquery.sql'>

```sql
{% macro get_where_subquery(relation) -%}
{% set where = config.get('where', '') %}
{% if "__three_days_ago__" in where %}
{# replace placeholder string with result of custom macro #}
{% set three_days_ago = dbt.dateadd('day', -3, current_timestamp()) %}
{% set where = where | replace("__three_days_ago__", three_days_ago) %}
{% endif %}
{% if where %}
{%- set filtered -%}
(select * from {{ relation }} where {{ where }}) dbt_subquery
{%- endset -%}
{% do return(filtered) %}
{%- else -%}
{% do return(relation) %}
{%- endif -%}
{%- endmacro %}
```

</File>

0 comments on commit bca2272

Please sign in to comment.