Add optional where clause to get_column_values (#583)

* Add optional where clause to get_column_values * Add new argument for get_column_values to README * Add an entry to CHANGELOG.md * Reorder args * Fix typo0 * Fix typo * Add integration test for get_column_values where argument * Update where arg test to use data_get_column_values_where_expected * Update CHANGELOG.md Co-authored-by: Doug Beatty <44704949+dbeatty10@users.noreply.github.com> * Reorder changelog entry Co-authored-by: Doug Beatty <44704949+dbeatty10@users.noreply.github.com>
dbt-labs · May 12, 2022 · e09fa7d · e09fa7d
1 parent 50f1d48
commit e09fa7d
Show file tree

Hide file tree

Showing 7 changed files with 40 additions and 3 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,4 +1,6 @@
 # Unreleased
+## New features
+- Add an optional `where` clause parameter to `get_column_values()` to filter values returned ([#511](https://github.com/dbt-labs/dbt-utils/issues/511), [#583](https://github.com/dbt-labs/dbt-utils/pull/583)) 
 
 ## Quality of life
 - Documentation about listagg macro ([#544](https://github.com/dbt-labs/dbt-utils/issues/544), [#560](https://github.com/dbt-labs/dbt-utils/pull/560))
@@ -12,6 +14,7 @@
 - [@graciegoheen](https://github.com/graciegoheen) (#545)
 - [@judahrand](https://github.com/judahrand) (#552)
 - [@clausherther](https://github.com/clausherther) (#555)
+- [@epapineau](https://github.com/epapineau) (#583)
 
 # dbt-utils v0.8.4
 ## Fixes

diff --git a/README.md b/README.md
@@ -553,6 +553,7 @@ This macro returns the unique values for a column in a given [relation](https://
 **Args:**
 - `table` (required): a [Relation](https://docs.getdbt.com/reference/dbt-classes#relation) (a `ref` or `source`) that contains the list of columns you wish to select from
 - `column` (required): The name of the column you wish to find the column values of
+- `where` (optional, default=`none`): A where clause to filter the column values by. 
 - `order_by` (optional, default=`'count(*) desc'`): How the results should be ordered. The default is to order by `count(*) desc`, i.e. decreasing frequency. Setting this as `'my_column'` will sort alphabetically, while `'min(created_at)'` will sort by when thevalue was first observed.
 - `max_records` (optional, default=`none`): The maximum number of column values you want to return
 - `default` (optional, default=`[]`): The results this macro should return if the relation has not yet been created (and therefore has no column values).

diff --git a/integration_tests/data/sql/data_get_column_values_where.csv b/integration_tests/data/sql/data_get_column_values_where.csv
@@ -0,0 +1,12 @@
+field,condition
+a,left
+b,right
+c,left
+d,right
+e,left
+f,right
+g,left
+g,right
+g,left
+g,right
+g,left
diff --git a/integration_tests/data/sql/data_get_column_values_where_expected.csv b/integration_tests/data/sql/data_get_column_values_where_expected.csv
@@ -0,0 +1,5 @@
+field
+a
+c
+e
+g
diff --git a/integration_tests/models/sql/schema.yml b/integration_tests/models/sql/schema.yml
@@ -50,6 +50,11 @@ models:
               values:
                 - '5'
 
+  - name: test_get_column_values_where
+    tests:
+      - dbt_utils.equality:
+          compare_model: ref('data_get_column_values_where_expected')
+
   - name: test_get_filtered_columns_in_relation
     tests:
       - dbt_utils.equality:

diff --git a/integration_tests/models/sql/test_get_column_values_where.sql b/integration_tests/models/sql/test_get_column_values_where.sql
@@ -0,0 +1,6 @@
+{% set column_values = dbt_utils.get_column_values(ref('data_get_column_values_where'), 'field', where="condition = 'left'") %}
+
+-- Create a relation using the values
+{% for val in column_values -%}
+select {{ dbt_utils.string_literal(val) }} as field {% if not loop.last %}union all{% endif %} 
+{% endfor %}
diff --git a/macros/sql/get_column_values.sql b/macros/sql/get_column_values.sql
@@ -1,8 +1,8 @@
-{% macro get_column_values(table, column, order_by='count(*) desc', max_records=none, default=none) -%}
-    {{ return(adapter.dispatch('get_column_values', 'dbt_utils')(table, column, order_by, max_records, default)) }}
+{% macro get_column_values(table, column, order_by='count(*) desc', max_records=none, default=none, where=none) -%}
+    {{ return(adapter.dispatch('get_column_values', 'dbt_utils')(table, column, order_by, max_records, default, where)) }}
 {% endmacro %}
 
-{% macro default__get_column_values(table, column, order_by='count(*) desc', max_records=none, default=none) -%}
+{% macro default__get_column_values(table, column, order_by='count(*) desc', max_records=none, default=none, where=none) -%}
     {#-- Prevent querying of db in parsing mode. This works because this macro does not create any new refs. #}
     {%- if not execute -%}
         {% set default = [] if not default %}
@@ -37,6 +37,11 @@
                 {{ column }} as value
 
             from {{ target_relation }}
+
+            {% if where is not none %}
+            where {{ where }}
+            {% endif %}
+
             group by {{ column }}
             order by {{ order_by }}
-Original file line number
+Diff line change
@@ -0,0 +1,5 @@
+    field
+    a
+    c
+    e
+    g