Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sel(method='nan') option filling missing values with NaN #1990

Closed
frexvahi opened this issue Mar 15, 2018 · 4 comments
Closed

sel(method='nan') option filling missing values with NaN #1990

frexvahi opened this issue Mar 15, 2018 · 4 comments

Comments

@frexvahi
Copy link

frexvahi commented Mar 15, 2018

Rationale

I would find it helpful to be able to specify "give me NaNs when the selector is not in the corresponding coordinate rather than raising an IndexError". This could go as a method option e.g. method='nan'.

>>> data.dims
 ('time', 'feature')
>>> time_array.dims
 ('base_time', 'offset')
>>> data.sel(time=time_array, method='nan')
 <array with NaNs in place of missing times>

Currently I work around the problem by appending a row to the data array with time coordinate NaT and all values NaN, then replacing missing times from time_array with NaT. However, this gives a FutureWarning from xarray/core/duck_array_ops.py:138: In the future, 'NAT == x' and 'x == NAT' will always be False.

If this is something you would like to see then I could have a go at making a pull request for this.

@shoyer
Copy link
Member

shoyer commented Mar 15, 2018

This is exactly what the .reindex() method is designed for. You can use it like sel, but it gives the appropriate NaN/NaT value instead of raising an error.

@frexvahi
Copy link
Author

frexvahi commented Mar 15, 2018

I tried .reindex() and got ValueError: IndexVariable objects must be 1-dimensional

so allowing reindex with a two-dimensional selector would be another solution.

@shoyer
Copy link
Member

shoyer commented Mar 15, 2018

so allowing reindex with a two-dimensional selector would be another solution.

Yes, we want to support that. See #1553.

We're already written most of the underlying machinery, so it should be pretty easy to add now. There are some backwards compatibility concerns with adjusting this for 1D arrays prior to a breaking release (see this warning), but I don't see any reason why we can't enable this for higher dimensional indexers right now.

@frexvahi
Copy link
Author

OK, that should solve my problem. I will go on using the NaT hack for now and refer back here as and when that stops working because NaT != NaT

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants