You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
we were using the dbt_valid_from and dbt_valid_to ranges to determine which row represents the correct values at a point in time, but now it means 2 rows would be selected for finding a point-in-time between the 3 seconds overlap of the example i posted
dbt should instead ensure that snapshot change records for the check strategy are non-overlapping.
Sample code:
{% macro snapshot_check_fixed_strategy(node, snapshotted_rel, current_rel, config, target_exists) %}
{% set check_cols_config = config['check_cols'] %}
{% set primary_key = config['unique_key'] %}
{# -------------- NEW ------------------ #}
{% set select_current_time -%}
select {{ snapshot_get_time() }} as snapshot_start
{%- endset %}
{% set now = run_query(select_current_time)[0].snapshot_start %}
{% set updated_at = "'" ~ now ~ "'::timestamp" %}
{# -------------- NEW ------------------ #}
{% if check_cols_config == 'all' %}
{% set check_cols = get_columns_in_query(node['injected_sql']) %}
{% elif check_cols_config is iterable and (check_cols_config | length) > 0 %}
{% set check_cols = check_cols_config %}
{% else %}
{% do exceptions.raise_compiler_error("Invalid value for 'check_cols': " ~ check_cols_config) %}
{% endif %}
{% set row_changed_expr -%}
(
{% for col in check_cols %}
{{ snapshotted_rel }}.{{ col }} != {{ current_rel }}.{{ col }}
or
({{ snapshotted_rel }}.{{ col }} is null) != ({{ current_rel }}.{{ col }} is null)
{%- if not loop.last %} or {% endif %}
{% endfor %}
)
{%- endset %}
{% set scd_id_expr = snapshot_hash_arguments([primary_key, updated_at]) %}
{% do return({
"unique_key": primary_key,
"updated_at": updated_at,
"row_changed": row_changed_expr,
"scd_id": scd_id_expr
}) %}
{% endmacro %}
The sample code shown above will need to be updated to:
use database-specific quotes (not ' everywhere)
use a database-specific timestamp type
The text was updated successfully, but these errors were encountered:
From Josh P and Claus H in Slack:
dbt should instead ensure that snapshot change records for the
check
strategy are non-overlapping.Sample code:
The sample code shown above will need to be updated to:
'
everywhere)The text was updated successfully, but these errors were encountered: