Skip to content

Commit

Permalink
Add any value (#501)
Browse files Browse the repository at this point in the history
* Add link for fewer_rows_than schema test in docs (#465)

* Update get_query_results_as_dict example to demonstrate accessing columnar results as dictionary values (#474)

* Update get_qu
ery_results_as_dict example to demonstrate accessing columnar results as dictionary values

* Use slugify in example

* Fix slugify example with dbt_utils. package prefix

Co-authored-by: Elize Papineau <elize.papineau@dbtlabs.com>

* Add note about not_null_where deprecation to Readme (#477)

* Add note about not_null_where deprecation to Readme

* Add docs to unique_where test

* Update pull_request_template.md to reference `main` vs `master` (#496)

* Correct coalesce -> concatenation typo (#495)

* add any_value cross-db macro

* Missing colon in test

* Update CHANGELOG.md

Co-authored-by: José Coto <jlcoto@users.noreply.github.com>
Co-authored-by: Elize Papineau <elizepapineau@gmail.com>
Co-authored-by: Elize Papineau <elize.papineau@dbtlabs.com>
Co-authored-by: Joe Ste.Marie <stemarie.joe@gmail.com>
Co-authored-by: Niall Woodward <niall@niallrees.com>
  • Loading branch information
6 people authored Feb 18, 2022
1 parent 14127aa commit 46fcdf6
Show file tree
Hide file tree
Showing 6 changed files with 50 additions and 1 deletion.
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
# dbt-utils v0.8.1

## New features
- A cross-database implementation of `any_value()` ([#497](https://github.com/dbt-labs/dbt-utils/issues/497), [#501](https://github.com/dbt-labs/dbt-utils/pull/501))

## Under the hood
- also ignore `dbt_packages/` directory [#463](https://github.com/dbt-labs/dbt-utils/pull/463)

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -461,7 +461,7 @@ in isolation.
We generally recommend testing this uniqueness condition by either:
* generating a [surrogate_key](#surrogate_key-source) for your model and testing
the uniqueness of said key, OR
* passing the `unique` test a coalesce of the columns (as discussed [here](https://docs.getdbt.com/docs/building-a-dbt-project/testing-and-documentation/testing/#testing-expressions)).
* passing the `unique` test a concatenation of the columns (as discussed [here](https://docs.getdbt.com/docs/building-a-dbt-project/testing-and-documentation/testing/#testing-expressions)).

However, these approaches can become non-perfomant on large data sets, in which
case we recommend using this test instead.
Expand Down
4 changes: 4 additions & 0 deletions integration_tests/data/cross_db/data_any_value_expected.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
key_name,static_col,num_rows
abc,dbt,2
jkl,dbt,3
xyz,test,1
6 changes: 6 additions & 0 deletions integration_tests/models/cross_db_utils/schema.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,12 @@
version: 2

models:
- name: test_any_value
tests:
- dbt_utils.equality:
compare_model: ref('data_any_value_expected')


- name: test_concat
tests:
- assert_equal:
Expand Down
19 changes: 19 additions & 0 deletions integration_tests/models/cross_db_utils/test_any_value.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
with some_model as (
select 1 as id, 'abc' as key_name, 'dbt' as static_col union all
select 2 as id, 'abc' as key_name, 'dbt' as static_col union all
select 3 as id, 'jkl' as key_name, 'dbt' as static_col union all
select 4 as id, 'jkl' as key_name, 'dbt' as static_col union all
select 5 as id, 'jkl' as key_name, 'dbt' as static_col union all
select 6 as id, 'xyz' as key_name, 'test' as static_col
),

final as (
select
key_name,
{{ dbt_utils.any_value('static_col') }} as static_col,
count(id) as num_rows
from some_model
group by key_name
)

select * from final
17 changes: 17 additions & 0 deletions macros/cross_db_utils/any_value.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{% macro any_value(expression) -%}
{{ return(adapter.dispatch('any_value', 'dbt_utils') (expression)) }}
{% endmacro %}


{% macro default__any_value(expression) -%}

any_value({{ expression }})

{%- endmacro %}


{% macro postgres__any_value(expression) -%}
{#- /*Postgres doesn't support any_value, so we're using min() to get the same result*/ -#}
min({{ expression }})

{%- endmacro %}

0 comments on commit 46fcdf6

Please sign in to comment.