Skip to content

Commit

Permalink
Add not_accepted_values schema test (#284)
Browse files Browse the repository at this point in the history
Schema Test - Not accepted values, test fixed

empty lines at the end

Update changelog
  • Loading branch information
JavierMonton authored and clrcrl committed May 18, 2021
1 parent 230d329 commit 8388f76
Show file tree
Hide file tree
Showing 5 changed files with 66 additions and 1 deletion.
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
## Features
* Add new `accepted_range` test ([#276](https://github.com/fishtown-analytics/dbt-utils/pull/276) [@joellabes](https://github.com/joellabes))
* Make `expression_is_true` work as a column test (code originally in [#226](https://github.com/fishtown-analytics/dbt-utils/pull/226/) from [@elliottohara](https://github.com/elliottohara), merged via [#313])
* Add new schema test, `not_accepted_values` ([#284](https://github.com/fishtown-analytics/dbt-utils/pull/284) [@JavierMonton](https://github.com/JavierMonton))

## Fixes
* Handle booleans gracefully in the unpivot macro ([#305](https://github.com/fishtown-analytics/dbt-utils/pull/305) [@avishalom](https://github.com/avishalom))
Expand All @@ -24,7 +25,6 @@

- Bump `require-dbt-version` to `[">=0.18.0", "<0.20.0"]` to support dbt v0.19.0 ([#308](https://github.com/fishtown-analytics/dbt-utils/pull/308), [#309](https://github.com/fishtown-analytics/dbt-utils/pull/309))


# dbt-utils v0.6.2

## Fixes
Expand Down
16 changes: 16 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -300,6 +300,22 @@ models:
where: "_deleted = false"
```

#### not_accepted_values ([source](macros/schema_tests/not_accepted_values.sql))
This test validates that there are no rows that match the given values.

Usage:
```yaml
version: 2
models:
- name: my_model
columns:
- name: city
tests:
- dbt_utils.not_accepted_values:
values: ['Barcelona', 'New York']
```

#### relationships_where ([source](macros/schema_tests/relationships_where.sql))
This test validates the referential integrity between two relations (same as the core relationships schema test) with an added predicate to filter out some rows from the test. This is useful to exclude records such as test entities, rows created in the last X minutes/hours to account for temporary gaps due to ETL limitations, etc.

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
id,city
1,Barcelona
2,London
3,Paris
4,New York
7 changes: 7 additions & 0 deletions integration_tests/models/schema_tests/schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,13 @@ models:
- dbt_utils.not_null_where:
where: "_deleted = false"

- name: data_test_not_accepted_values
columns:
- name: city
tests:
- dbt_utils.not_accepted_values:
values: ['Madrid', 'Berlin']

- name: data_test_relationships_where_table_2
columns:
- name: id
Expand Down
37 changes: 37 additions & 0 deletions macros/schema_tests/not_accepted_values.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
{% macro test_not_accepted_values(model, values) %}

{% set column_name = kwargs.get('column_name', kwargs.get('field')) %}
{% set quote_values = kwargs.get('quote', True) %}

with all_values as (

select distinct
{{ column_name }} as value_field

from {{ model }}

),

validation_errors as (

select
value_field

from all_values
where value_field in (
{% for value in values -%}
{% if quote_values -%}
'{{ value }}'
{%- else -%}
{{ value }}
{%- endif -%}
{%- if not loop.last -%},{%- endif %}
{%- endfor %}
)

)

select count(*) as validation_errors
from validation_errors

{% endmacro %}

0 comments on commit 8388f76

Please sign in to comment.