Skip to content

Commit

Permalink
Update input with additional sales (#225)
Browse files Browse the repository at this point in the history
* Checkpoint export splice progress

* Revert export splice

* Update ingest to mark excluded sales

* Finalize ad-hoc DR sheet with new sales

* Update input data with newly flagged sales

* Revert "Finalize ad-hoc DR sheet with new sales"

This reverts commit 47fb94a.

* Revert "Update ingest to mark excluded sales"

This reverts commit 8ab984c.

* Revert "Revert export splice"

This reverts commit b54758c.

* Revert "Checkpoint export splice progress"

This reverts commit beb7992.

* Update final run ID

* Add kludge to remove small sales on multi-card PINs
  • Loading branch information
dfsnow authored Mar 28, 2024
1 parent 5a33b6e commit 148e56d
Showing 5 changed files with 18 additions and 13 deletions.
20 changes: 10 additions & 10 deletions dvc.lock
Original file line number Diff line number Diff line change
@@ -33,20 +33,20 @@ stages:
outs:
- path: input/assessment_data.parquet
hash: md5
md5: b32480d27f75180149afe1b2de908037
size: 310404298
md5: c5c5b10a62f815a8b47de9101424dea6
size: 309845333
- path: input/char_data.parquet
hash: md5
md5: 537c278255d5721744ad8f180df22526
size: 618205566
md5: 95b41b06f03b055c8f1ba77bf80c8d30
size: 616908148
- path: input/complex_id_data.parquet
hash: md5
md5: 0492bec3b81f48d814bfedb9d9f893e0
size: 703076
md5: d0b2c6cb1dbf8ae90de2ccee2e99568d
size: 702334
- path: input/hie_data.parquet
hash: md5
md5: 2e049a011cae0a45d2072406e95b78b7
size: 1925258
md5: 675edaaee68b015e15a9d753a3531124
size: 1924257
- path: input/land_nbhd_rate_data.parquet
hash: md5
md5: e508daf5790982c303d6503fe1cb8e2b
@@ -56,8 +56,8 @@ stages:
size: 2109
- path: input/training_data.parquet
hash: md5
md5: 292caca708c332ecb5acc7886ce46e80
size: 156692811
md5: 787d2c901d1d957b87ecaee83663e8e5
size: 163303649
train:
cmd: Rscript pipeline/01-train.R
deps:
2 changes: 1 addition & 1 deletion params.yaml
Original file line number Diff line number Diff line change
@@ -445,4 +445,4 @@ comp:
# upload
export:
triad_code: "1"
run_id: "2024-02-06-relaxed-tristan"
run_id: "2024-03-17-stupefied-maya"
5 changes: 5 additions & 0 deletions pipeline/02-assess.R
Original file line number Diff line number Diff line change
@@ -350,6 +350,11 @@ sales_data_ratio_study <- sales_data %>%
# For ratio studies, we don't want to include outliers
filter(!sv_is_outlier) %>%
filter(meta_year == params$assessment$data_year) %>%
# Kludge to remove some sales that somehow appear to be for a single card
# on a multi-card PIN. Will need to go back and hand validate these
filter(
!meta_sale_document_num %in% c("2335646020", "2312245016")
) %>%
group_by(meta_pin) %>%
filter(meta_sale_date == max(meta_sale_date)) %>%
distinct(
2 changes: 1 addition & 1 deletion reports/_setup.qmd
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
params:
run_id: "2024-02-06-relaxed-tristan"
run_id: "2024-03-17-stupefied-maya"
year: "2024"
---

2 changes: 1 addition & 1 deletion reports/performance/performance.qmd
Original file line number Diff line number Diff line change
@@ -20,7 +20,7 @@ knitr:
out.width: "100%"
editor: source
params:
run_id: "2024-02-06-relaxed-tristan"
run_id: "2024-03-17-stupefied-maya"
year: "2024"
---

0 comments on commit 148e56d

Please sign in to comment.