Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add finite constraint for numbers or number.nanValues, number.infValues #644

Open
micimize opened this issue Oct 6, 2019 · 4 comments
Open

Comments

@micimize
Copy link
Contributor

micimize commented Oct 6, 2019

Similar to #393, but proposing a constraint rather than a blanket policy.
I think it should be { "finite": true }. There are many problem domains where NaN, INF, -INF are undesirable.

Also related: frictionlessdata/tableschema-py#249

@roll
Copy link
Member

roll commented Oct 7, 2019

I would even make it stricter having InF/Nan disabled by default with an option (instead of the constraint) like:

number.<allow Inf/Nan - not sure regarding the prop's name> = True
...
number.bareNumber = True
...

@roll roll added this to the v2 milestone Jan 3, 2024
@roll roll changed the title add finite constraint for numbers Add finite constraint for numbers Jan 3, 2024
@roll roll self-assigned this Jan 6, 2024
@roll roll added the proposal label Jan 6, 2024
@nichtich
Copy link
Contributor

nichtich commented Jan 9, 2024

The lexical values of NaN, +Inf, -Inf may also differ. We have trueValues and falseValues for boolean, so nanValues, positiveInfValues, negativeInfValues could be a good idea.

Note that XML Schema datatypes float and double include NaN, +Inf, -Inf but other numeric datatypes don't.

@micimize
Copy link
Contributor Author

@nichtich that's an excellent idea as well, though I'd suggest having both as options as having to specify three empty lists for the common finite case feels a bit fragile, even if it means a mutual exclusion validation.

@roll roll changed the title Add finite constraint for numbers Add finite constraint for numbers or nanValues, infValues Jan 25, 2024
@roll roll changed the title Add finite constraint for numbers or nanValues, infValues Add finite constraint for numbers or number.nanValues, number.infValues Jan 25, 2024
@roll
Copy link
Member

roll commented Jan 25, 2024

I think nanValues and infValues are really nice proposal! It's not supported by pandas or polars but can be easily added on the post-processing stage.

Currently, I'm removing it from v2. Let's first get a demand metrics using this instrument - #863

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: No status
Development

Successfully merging a pull request may close this issue.

3 participants