Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Correctly join calculated number of parcels per sale to sales view #334

Merged
merged 2 commits into from
Mar 8, 2024
Merged

Correctly join calculated number of parcels per sale to sales view #334

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b043a3c
Adjust joining for calculated number of parcels per sale
wrridgeway Mar 8, 2024
b70eab5
Add dbt test to detect NULL values for num_parcels_sale
wrridgeway Mar 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion aws-athena/views/default-vw_pin_sale.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,8 @@ unique_sales AS (
) AS sale_filter_deed_type
FROM {{ source('iasworld', 'sales') }} AS sales
LEFT JOIN calculated
ON sales.instruno = calculated.instruno
ON NULLIF(REPLACE(sales.instruno, 'D', ''), '')
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

calculated AS (
    SELECT
        instruno,
        COUNT(*) AS nopar_calculated
    FROM (
        SELECT DISTINCT
            parid,
            NULLIF(REPLACE(instruno, 'D', ''), '') AS instruno
        FROM {{ source('iasworld', 'sales') }}
        WHERE deactivat IS NULL
            AND cur = 'Y'
    )
    GROUP BY instruno
)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeeep, that'll do it. We'll need to re-ingest sales for modeling.

= calculated.instruno
LEFT JOIN
town_class AS tc
ON sales.parid = tc.parid
Expand Down
3 changes: 3 additions & 0 deletions dbt/models/default/schema/default.vw_pin_sale.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,9 @@ models:
- row_count:
name: default_vw_pin_sale_row_count
above: 2477674 # as of 2023-11-22
- not_null:
name: default_vw_pin_sale_num_parcels_sale_not_null
column_name: num_parcels_sale
# TODO: Sale is validated (after sales validation has been added to
# iasworld)
# TODO: Validation is catching obvious outliers
Expand Down
Loading