Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve mapping for images containing NaN values #383

Merged
merged 7 commits into from
Jul 18, 2024

Conversation

ortk95
Copy link
Owner

@ortk95 ortk95 commented Jul 18, 2024

Improved mapping with BodyXY.map_img when using spline interpolation for images containing NaN pixels. The spline interpolator does not accept NaN values, so any NaN pixels must be replaced with finite values before performing the interpolation.

Although we propagate NaN values to the map (by default at least), the replacement values can still affect the interpolated values elsewhere if the replacement values are very different to the surrounding values. Previously, we just used np.nan_to_num to replaced any NaNs with 0, which could introduce large artefacts when the data values were large, due to the jump between 0 and the data (see below).

In the updated code, we now replace NaN (or infinite) values with the mean of surrounding non-NaN pixels (with a 3x3 footprint). All other NaN pixels (which don't have neighbouring non-NaN pixels) are replaced with the median of all the pixels in the original image. This generally minimises the delta between replaced NaN pixels, and the surrounding pixels, massively reducing (or even preventing) any artefacts around the NaN regions.

Other minor mapping changes:

  • For spline interpolations, areas outside the convex hull of pixel centres are automatically set to NaN, for consistency with the handling of NaN pixels. This can be controlled with the propagate_nan parameter.
  • propagate_nan can now be controlled from other Observation methods, and its value is recorded in mapped FITS headers.
  • It is now possible to directly pass a cube to map_img, where each slice will be automatically mapped and a mapped cube is returned. This saves having to manually iterate with np.array([body.map_img(img) for img in cube]).

Closes #382

Example

This is a very extreme example of the artefacts that could be introduced around NaN regions with the old behaviour with cubic interpolation. The circles show the locations and values for the image pixels. The old behaviour had artefacts introduced along the top edge of the mapped area and the NaN pixel on the left, caused by the large jump from the filled-in 0 values and the data values. The new behaviour prevents any large jumps between the pixels, preventing any artefacts.

image
image

Pull request checklist

  • Add a clear description of the change
  • Add any new tests needed
  • Run spell check on new text visible to user (documentation, GUI etc.)
  • Check any changes to requirements.txt are reflected in setup.py and conda-forge feedstock
  • Check code passes CI checks (run run_ci.sh or check GitHub Actions)

See CONTRIBUTING.md for more details.

ortk95 added 6 commits July 16, 2024 16:19
Non-finite values are replaced with the average of neighbouring pixels before mapping, rather than 0. This should reduce artefacts when using spline interpolation.

#382
Mapping interpolation seems to be sensitive to the system it's run on, so increase tolerance for tests on mapped image data
`scipy.interpolate.griddata` was very slow, and could produce unstable results, so it seems much more sensible to stick with `RectBivariateSpline` with the newer more sophisticated NaN replacement.

#382 (comment)
@coveralls
Copy link

coveralls commented Jul 18, 2024

Pull Request Test Coverage Report for Build 9991348244

Details

  • 46 of 46 (100.0%) changed or added relevant lines in 2 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.003%) to 99.615%

Totals Coverage Status
Change from base Build 9714025732: 0.003%
Covered Lines: 3135
Relevant Lines: 3147

💛 - Coveralls

@ortk95 ortk95 merged commit c36b8e9 into dev Jul 18, 2024
38 checks passed
@ortk95 ortk95 deleted the 382-mapping-interpolation branch July 18, 2024 12:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants