Skip to content

Commit

Permalink
Schema Test - Not accepted values
Browse files Browse the repository at this point in the history
  • Loading branch information
JavierMonton authored and clrcrl committed Dec 23, 2020
1 parent 7a3e37d commit 10bf540
Show file tree
Hide file tree
Showing 4 changed files with 65 additions and 0 deletions.
16 changes: 16 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -300,6 +300,22 @@ models:
where: "_deleted = false"
```

#### not_accepted_values ([source](macros/schema_tests/not_accepted_values.sql))
This test validates that there are no rows that match the given values.

Usage:
```yaml
version: 2
models:
- name: my_model
columns:
- name: city
tests:
- dbt_utils.not_accepted_values:
values: ['Barcelona', 'New York']
```

#### relationships_where ([source](macros/schema_tests/relationships_where.sql))
This test validates the referential integrity between two relations (same as the core relationships schema test) with an added predicate to filter out some rows from the test. This is useful to exclude records such as test entities, rows created in the last X minutes/hours to account for temporary gaps due to ETL limitations, etc.

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
id,city
1,Barcelona
2,London
3,Paris
4,New York
7 changes: 7 additions & 0 deletions integration_tests/models/schema_tests/schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,13 @@ models:
- dbt_utils.not_null_where:
where: "_deleted = false"

- name: data_test_not_accepted_values
columns:
- name: city
tests:
- dbt_utils.not_accepted_values:
values: ['Barcelona', 'New York', 'Berlin']

- name: data_test_relationships_where_table_2
columns:
- name: id
Expand Down
37 changes: 37 additions & 0 deletions macros/schema_tests/not_accepted_values.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
{% macro test_not_accepted_values(model, values) %}

{% set column_name = kwargs.get('column_name', kwargs.get('field')) %}
{% set quote_values = kwargs.get('quote', True) %}

with all_values as (

select distinct
{{ column_name }} as value_field

from {{ model }}

),

validation_errors as (

select
value_field

from all_values
where value_field in (
{% for value in values -%}
{% if quote_values -%}
'{{ value }}'
{%- else -%}
{{ value }}
{%- endif -%}
{%- if not loop.last -%},{%- endif %}
{%- endfor %}
)

)

select count(*) as validation_errors
from validation_errors

{% endmacro %}

0 comments on commit 10bf540

Please sign in to comment.