Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update ratio_stats sales chasing detection methods #506

Closed
dfsnow opened this issue Jun 12, 2024 · 1 comment
Closed

Update ratio_stats sales chasing detection methods #506

dfsnow opened this issue Jun 12, 2024 · 1 comment

Comments

@dfsnow
Copy link
Member

dfsnow commented Jun 12, 2024

We disabled the sales chasing detection in the reporting.ratio_stats table when it was converted to a Python model in #422. The reasons to disable it were enumerated in that PR, but to recap:

  • The detect_chasing() function specifically did not play well with the Spark environment for some reason (probably using ECDF()).
  • The results of the chasing detection were not heavily used anywhere.
  • The detect_chasing() function combines two methods using an AND operator. At least one of those methods was highly sensitive to/created lots of false positives for areas with extremely homogenous housing stock.

However, the chasing detection output is valuable. We should update the ratio_stats Python model to reimplement chasing with improved methods. "Improved" here could mean many things, e.g.:

  • Simply changing some of the parameters set by default for the original methods.
  • Trying to refine or tweak the existing methods
  • Creating entirely new methods, incorporating them into assesspy, and then integrating them into the Python model
@dfsnow
Copy link
Member Author

dfsnow commented Nov 26, 2024

This issue is closed as of #521. The updated sales chasing detection methods added to assesspy 2.0.0 are less sensitive and work within dbt Python models.

@dfsnow dfsnow closed this as completed Nov 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant