Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Summarize date columns #35

Open
ccunningham101 opened this issue Aug 19, 2022 · 0 comments
Open

Summarize date columns #35

ccunningham101 opened this issue Aug 19, 2022 · 0 comments
Labels
enhancement New feature or request

Comments

@ccunningham101
Copy link

In the dataset-report notebook, date columns are summarized with describe() and a bar plot

One use case is for identifying impossible dates. 1900-01-01 is a common impossible date, but impossible dates may differ between data sources. Or checking that dynamic dates, particularly when extracted monthly, fell within an expected range.

The bmi report groups dates by year and then redacts and rounds. This is helpful for identifying common systematic errors (such as a null date as 1900-01-01), rather than individual outliers or issues that may require use of another data column (such as diagnosis before someone's birth date).

Initial implementation in dataset-report format here [Note: the redaction should probably include 0's]

To consider:

  1. The ideal grouping time granularity (Y/M/D) probably is some balance of what gives the most information without everything being redacted; could be specified by the user, but may differ between variables
  2. Are there potential disclosure issues if multiple variables are querying the same codelist in similar but slightly different ways? Should this be covered by rounding?
@iaindillingham iaindillingham added the enhancement New feature or request label Jan 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants