-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
use test materialization for schema/generic tests #3286
Conversation
b368d60
to
ac8cd78
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is looking really good!
One thing we can think about before / as part of #2593: What data should each generic test query return? For the not_null
test, the entire row (*
) makes sense, but I could imagine the unique
and accepted_values
test including the count of each duplicated/unaccepted value, too. That does not need to block merging this PR, just something on my mind.
select 'Expected {{ expected }}, but got {{ actual }}' as validation_error | ||
where {{ success }} = 0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nicely done! I like this a lot
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The where
hack actually doesn't work on bigquery :(
Query without FROM clause cannot have a WHERE clause at [..]
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is definitely a perfect example of a where
test config.. any thoughts what to do for the time being or for this instance specifically?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Heh, ok, try this:
select 'Expected {{ expected }}, but got {{ actual }}' as validation_error
from (select true)
where {{ success }} = 0
The where
test config should work a little differently—that would be filtering/rewriting the actual {{ model }}
reference (as a subquery). There would definitely be a from
clause.
@@ -15,4 +15,4 @@ | |||
select 1 as error | |||
{% endif %} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this just need the same treatment as above?
select '{{ table.type }} does not match expected value {{ type }}'
where '{{ table.type }}' != '{{ type }}'
@@ -65,5 +64,5 @@ | |||
{% endif %} | |||
{% endfor %} | |||
{% do log('bad columns: ' ~ bad_columns, info=True) %} | |||
select {{ bad_columns | length }} as pass_fail | |||
{% endmacro %} | |||
select * from unnest(string_to_array('{{ bad_columns|join(',') }}', ',')) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fancy! :)
This works well on Postgres. If we needed to run it on any database, I think we could do:
{% for bad_column in bad_columns %}
select '{{ bad_column }}' as bad_column
{{ 'union all' if not loop.last }}
{% endfor %}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I totally agree! The piece here that I'm struggling to handle correctly is if there are no bad columns (bad_columns == []
) In this case, no sql will be rendered.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ohh right! So this is a bit jankier, but perhaps something like:
{% for bad_column in bad_columns %}
select '{{ bad_column }}' as bad_column union all
{% endfor %}
select * from (select 1 limit 0)
If we only run this test on postgres, I like your answer better :) All the more reason for adapter-specific implementations of generic tests...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🙏
@@ -20,7 +21,7 @@ def models(self): | |||
def test_postgres_column_types(self): | |||
self.run_and_test() | |||
|
|||
|
|||
@pytest.mark.skip("Temporarily skipping while implementing the handling of test results.") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
15fa927
to
1d7b4c0
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+78 −422
LGr8TM!
resolves #3192
Description
test
jijna tagChecklist
CHANGELOG.md
and added information about my change to the "dbt next" section.