You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Given an incremental model that lists multiple columns as the unique key, the columns should be treated as a single composite unique key.
Example: If an incremental model generates rows with composite keys [(1, 2), (2, 1)], only the rows with composite key (1, 2), and (2, 1) should be deleted.
Actual behavior
Given an incremental model that lists multiple columns as the unique key, the columns are treated as separate unique keys.
Example: If an incremental model generates rows with composite keys [(1, 2), (2, 1)], rows with composite keys of (1, 2), and (2, 1) are deleted, but rows with composite key (1, 1) and (2, 2) are also deleted.
The behaviour in dbt-core, dbt-mysql, dbt-postgres, and dbt-snowflake matches that of the expected behaviour, not the actual behaviour of dbt-trino.
Steps To Reproduce
Consider the below table.
Quotes
Id, Version
1, 1
1, 2
2, 1
2, 2
After running an incremental model that generates the following rows.
Id, Version
1, 2
2, 1
{{
config
(
materialized='incremental',
incremental_strategy='delete+insert',
unique_key=['id', 'version']
)
}}
select
*
from
{{ source('source', 'quote_test') }}
{% if is_incremental() %}
where
(id = 1 and version = 2) or
(id = 2 and version = 1)
{% endif %}
All of the rows from Quotes will be deleted, before reinserting the generated rows. The final result will look like the following, which does not seem correct nor beneficial.
Quotes
Id, Version
1, 2
2, 1
Log output/Screenshots
No response
Operating System
Linux
dbt version
1.9.0
Trino Server version
443
Python version
3.11.9
Are you willing to submit PR?
Yes I am willing to submit a PR!
The text was updated successfully, but these errors were encountered:
HaydenNess
changed the title
Incorrect behaviour for incremental deletions
Incremental unique keys treated as separate rather than composite
Jan 22, 2025
Expected behavior
Given an incremental model that lists multiple columns as the unique key, the columns should be treated as a single composite unique key.
Example: If an incremental model generates rows with composite keys [(1, 2), (2, 1)], only the rows with composite key (1, 2), and (2, 1) should be deleted.
Actual behavior
Given an incremental model that lists multiple columns as the unique key, the columns are treated as separate unique keys.
Example: If an incremental model generates rows with composite keys [(1, 2), (2, 1)], rows with composite keys of (1, 2), and (2, 1) are deleted, but rows with composite key (1, 1) and (2, 2) are also deleted.
The behaviour in dbt-core, dbt-mysql, dbt-postgres, and dbt-snowflake matches that of the expected behaviour, not the actual behaviour of dbt-trino.
Steps To Reproduce
Consider the below table.
After running an incremental model that generates the following rows.
All of the rows from Quotes will be deleted, before reinserting the generated rows. The final result will look like the following, which does not seem correct nor beneficial.
Log output/Screenshots
No response
Operating System
Linux
dbt version
1.9.0
Trino Server version
443
Python version
3.11.9
Are you willing to submit PR?
The text was updated successfully, but these errors were encountered: