Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Masked arrays break units #983

Closed
dopplershift opened this issue Dec 14, 2018 · 5 comments
Closed

Masked arrays break units #983

dopplershift opened this issue Dec 14, 2018 · 5 comments
Labels
Area: Calc Pertains to calculations Type: Bug Something is not working like it should
Milestone

Comments

@dopplershift
Copy link
Member

Feeding masked arrays to the CAPE/CIN functions results in an error about masked points. Sample data can be pull for NUCAPS and/or taken from #973.

@dopplershift dopplershift added Type: Bug Something is not working like it should Area: Calc Pertains to calculations labels Dec 14, 2018
@dopplershift dopplershift added this to the 0.11 milestone Dec 14, 2018
@zbruick zbruick self-assigned this Jun 10, 2019
@zbruick
Copy link
Contributor

zbruick commented Jun 11, 2019

So if you read in NUCAPS data with xarray, nothing is masked and everything works just fine. However, if you use netcdf4 to read in the data, all variables are returned as masked arrays. Units are unable to be assigned to these, which then breaks calculations (more than just CAPE/CIN due to lack of units). How do we go about solving this, since it seems that the issue is on the data import side and not on the calculation side?

@dopplershift
Copy link
Member Author

I'm kind of embarrassed I didn't see that at the time since we've hit that in the past. I'm going to rename the issue to cover that part, but I don't think it's something we should be trying to fix for the next release.

@dopplershift dopplershift changed the title Masked arrays break CAPE/CIN Masked arrays break units Jun 12, 2019
@dopplershift dopplershift modified the milestones: 0.11, 1.0 Jun 12, 2019
@jthielen
Copy link
Collaborator

jthielen commented Dec 31, 2019

I'm not sure if this should go in this issue or a new one, but yet again there are breaks with the current master branch of Pint, this time with MaskedArrays. Since MaskedArrays do not currently defer properly on ufuncs, Pint's implementation of __array_ufunc__ on pint.Unit to broaden compatibility with duck arrays unfortunately broke creation of a MaskedArray Quantity by multiplication by a unit on the right (see hgrecco/pint#963 (comment)). Instead, for now MaskedArray Quantities should only be created by multiplication on the left or directly through the Quantity constructor.

Test Log

Also, I'm not sure if I'll have the time to patch MetPy with respect to this before I have to leave for travels tomorrow and be away from regular internet access for the next week, so I wanted to make sure I got this out there. If it isn't easy enough to fix quick, perhaps MetPy v0.12 needs to pin Pint to <0.10? I do apologize for creating a bit of a time bomb and not fixing it!

@dopplershift
Copy link
Member Author

No worries! It's good to know. Should be able to clean those up I think...

@dopplershift
Copy link
Member Author

We've done everything we can do to fix this internally in #1271, #1282, #1286 . You can use masked arrays just fine with Quantity(masked_array, units). Everything else is left upstream. I'm closing.

joernu76 added a commit to joernu76/MetPy that referenced this issue Mar 30, 2020
joernu76 added a commit to joernu76/MetPy that referenced this issue Mar 31, 2020
The issue occurs only, when "x" in first_derivative is a masked array
with units. As all arrays stemming from NetCDF access are basically
masked arrays, we do not have non-masked arrays in our code, even though
the mask is almost always 'False'.

I added a test case demonstrating the problem, it fails without the
contained fix.

See issue Unidata#983
dopplershift pushed a commit to dopplershift/MetPy that referenced this issue Apr 17, 2020
The issue occurs only, when "x" in first_derivative is a masked array
with units. As all arrays stemming from NetCDF access are basically
masked arrays, we do not have non-masked arrays in our code, even though
the mask is almost always 'False'.

I added a test case demonstrating the problem, it fails without the
contained fix.

See issue Unidata#983
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: Calc Pertains to calculations Type: Bug Something is not working like it should
Projects
None yet
Development

No branches or pull requests

3 participants