Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add parallax correction via new ParallaxCorrectionModifier #1904

Merged
merged 129 commits into from
Jul 28, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
129 commits
Select commit Hold shift + click to select a range
8f1257f
Parallax correction
May 5, 2021
58a960a
Added projection
May 6, 2021
342030f
Added projection
May 6, 2021
a49855c
better error message
uhamann Aug 23, 2021
2a796de
parallax example script
uhamann Aug 24, 2021
77f5e50
Merge branch 'master' into parallax
uhamann Aug 24, 2021
daf4400
Started unit test for parallax correction
gerritholl Nov 29, 2021
eebe910
Merge remote-tracking branch 'remotes/tmp/parallax' into parallax-cor…
gerritholl Nov 29, 2021
db6ec84
Merge parallax.py into satpy
gerritholl Nov 29, 2021
ae72e98
Merge remote-tracking branch 'remotes/tmp/parallax' into parallax-cor…
gerritholl Nov 30, 2021
a9d5668
Adapt parallax dependency from meteoswiss
gerritholl Nov 30, 2021
c3918bf
Complement AUTHORS.md
gerritholl Nov 30, 2021
1b38fb8
Add Satpy GPL headers to new source files
gerritholl Nov 30, 2021
c321b18
Extend unit tests for parallax correction.
gerritholl Nov 30, 2021
5dae1e6
More unit tests for parallax correction
gerritholl Nov 30, 2021
70292ae
Remove some unused code
gerritholl Dec 1, 2021
2cf131a
Extend parallax unit tests
gerritholl Dec 2, 2021
a9aebd0
Cleaner parallax correction implementation
Dec 2, 2021
913adaf
Merge Jussis simplification back in parallax.py
gerritholl Dec 3, 2021
5ffc618
Improve parameterised unit tests
gerritholl Dec 3, 2021
1077d7a
Add more unit tests for parallax correction
gerritholl Dec 3, 2021
eed47ec
Extend unit tests
gerritholl Dec 3, 2021
b0a2c68
Merge branch 'main' into parallax-correction
gerritholl Dec 3, 2021
0850114
Switch to nearest neighbour
gerritholl Dec 6, 2021
7960f1b
Add more tests for parallax correction
gerritholl Dec 6, 2021
e017866
Restore slant cloudy test
gerritholl Dec 7, 2021
714c110
Use Simons reference value
gerritholl Dec 7, 2021
b30f8bc
Merge branch 'main' into parallax-correction
gerritholl Dec 14, 2021
0c45d39
Fix swapped lon/lat in unit test
gerritholl Dec 14, 2021
06a5716
Add parameterised fixture for generating a cloud.
gerritholl Dec 14, 2021
b54a431
Merge branch 'main' into parallax-correction
gerritholl Jan 3, 2022
e713e89
Add more detailed tests
gerritholl Jan 3, 2022
9c1404b
Add tolerance to distance comparison
gerritholl Jan 3, 2022
6518d19
Add tests for no or partly overlapping CTH
gerritholl Jan 5, 2022
c2de3da
Allow costumisation of resampling method in parallax correction
gerritholl Jan 6, 2022
4ed3aa9
Add tests with dask and same areas
gerritholl Jan 7, 2022
7854191
Point out that CompositeBase is also for modifiers
gerritholl Jan 7, 2022
90d57e1
Started work on ParallaxCorrectionModifier class
gerritholl Jan 7, 2022
f149eba
Add modifier interface for parallax correction
gerritholl Jan 7, 2022
ee97200
Mark partly/no overlap as xfail
gerritholl Jan 7, 2022
f7d692f
Merge branch 'main' into parallax-correction
gerritholl Jan 7, 2022
07e7731
Improve parallax-correction docs
gerritholl Jan 7, 2022
d987fc1
Adapt to pre-3.9 syntax
gerritholl Jan 7, 2022
25da335
Merge branch 'improve-modifier-docs' into parallax-correction
gerritholl Jan 7, 2022
33df7c1
Add documentation on parallax correction modifier
gerritholl Jan 7, 2022
cc01a86
Improve parallax correction documentation
gerritholl Jan 7, 2022
d6a9cc0
Merge branch 'improve-modifier-docs' into parallax-correction
gerritholl Jan 7, 2022
7df874b
Remove unused function and TODO
gerritholl Jan 7, 2022
6e94b77
Repair forward parallax after elevation change
gerritholl Jan 7, 2022
a9022e0
More specific parameter failure marking
gerritholl Jan 10, 2022
bf56335
Mark one more case as xfail
gerritholl Jan 10, 2022
b5340da
Write log message when doing parallax correction
gerritholl Jan 11, 2022
688e2c7
Add backup calculation of orbital parameters
gerritholl Jan 12, 2022
17479d9
Adapt to Python 3.7 syntax
gerritholl Jan 12, 2022
0a8e515
Add missing dependencies for continuous integration
gerritholl Jan 12, 2022
75f2e2d
Try unstable test with latest astropy
gerritholl Jan 12, 2022
0ea5bbb
Improve example and documentation for parallax composite
gerritholl Jan 12, 2022
fc3c94e
More informative debug message
gerritholl Jan 17, 2022
e43c499
Add failing test for NWCSAF confetti bug
gerritholl Jan 20, 2022
6236340
Add debug mode to parallax correction
gerritholl Jan 26, 2022
c75681f
Consistent use metres above Earth's surface
gerritholl Jan 27, 2022
eeb52a4
Remember that satellite altitude reported in km
gerritholl Jan 27, 2022
5c4089c
Remove destructive preprocess method
gerritholl Jan 27, 2022
e20ad8f
Verify cloud moves toward observer
gerritholl Jan 28, 2022
c96dd78
Avoid creating a scene when resampling
gerritholl Jan 28, 2022
c4ae9d1
Improve test confirm cloud moves south
gerritholl Jan 28, 2022
65e2533
Fix cloud direction movement test
gerritholl Jan 28, 2022
8047bc5
Perform test with different radii of influence
gerritholl Jan 28, 2022
ff40e6a
Clarify comments
gerritholl Feb 2, 2022
6e4e3d1
Set cloudfree to 0 CTH and try other things
gerritholl Feb 3, 2022
a73b31a
Try with bucket resampler
gerritholl Feb 3, 2022
22ae499
Change parallax resampling approach
gerritholl Feb 4, 2022
bd8b5a9
Add test confirming new cloud area fully cloudy
gerritholl Feb 4, 2022
a67a206
Fix get_abs_max implementation
gerritholl Feb 4, 2022
cffb1d2
Resample CTH to target area first
gerritholl Feb 4, 2022
e539415
Ensure test case has valid polar area
gerritholl Feb 4, 2022
568b551
Also use 3575 at antimeridian
gerritholl Feb 4, 2022
2f18c60
Fix tests for 15 km cloud
gerritholl Feb 7, 2022
7b67560
Add test with dask arrays
gerritholl Feb 7, 2022
6766f90
make parallax correction more dask-friendly
gerritholl Feb 7, 2022
17d7468
Re-arrange and extend parallax unit tests
gerritholl Feb 7, 2022
fd3a2ac
Rename variable
gerritholl Feb 8, 2022
4e4ce01
In debug mode, store counts as dataset not ndarray
gerritholl Feb 8, 2022
bea71a3
Fix radius of influence in final resampling step
gerritholl Feb 8, 2022
e9e8b20
Merge branch 'main' into parallax-correction
gerritholl Mar 18, 2022
a3765c6
Change version added for parallax correction
gerritholl Mar 18, 2022
8a23932
Refactor parallax correction tests
gerritholl Mar 22, 2022
c452e7d
Merge branch 'main' into parallax-correction
gerritholl Mar 23, 2022
1297774
Merge branch 'main' into parallax-correction
gerritholl Apr 6, 2022
e4fd0c9
Merge branch 'main' into parallax-correction
gerritholl Apr 7, 2022
fe4f938
Merge branch 'main' into parallax-correction
gerritholl Apr 11, 2022
173ed44
Make sure to call apply_modifier_info
gerritholl Apr 11, 2022
ddfebcb
Add test for double-load bug
gerritholl Apr 12, 2022
d4edc4c
Add test confirming no compute occurs.
gerritholl Apr 12, 2022
933dac6
Fix example and test YAML config
gerritholl Apr 12, 2022
f60b0ad
Downgrade syntax to support Python 3.8
gerritholl Apr 12, 2022
cb15600
refactor unit test test_modifier_interface_cloud_moves_to_observer
gerritholl Apr 12, 2022
11fee22
refactor test_correct_area_clearsky
gerritholl Apr 12, 2022
d585dbd
Merge branch 'main' into parallax-correction
gerritholl Apr 14, 2022
58f55b6
Change versionadded to 0.37
gerritholl Apr 14, 2022
c3e393d
Make sure we're getting dask arrays from get_lonlats
gerritholl May 4, 2022
5c3a4f0
Refactoring and docstring improvements
gerritholl May 5, 2022
5323363
Add more prominent "experimental" warning
gerritholl May 5, 2022
259a0a2
Fix filename for pc docstring example
gerritholl May 6, 2022
ed38cda
Point out pc only supported for GEO
gerritholl May 6, 2022
26564f5
Move get_satpos_from_name to satpy.utils
gerritholl May 6, 2022
48690ca
Needs pyresample 1.24
gerritholl May 6, 2022
bb2040a
Change variable name for bucket resampler instance
gerritholl Jun 17, 2022
afd5951
Merge branch 'main' into parallax-correction
gerritholl Jul 11, 2022
0b3ea1c
Mock TLE downloading in parallax correction unit tests
gerritholl Jul 11, 2022
1230822
add another missing socket mock
gerritholl Jul 11, 2022
02bbda3
Merge branch 'main' into parallax-correction
gerritholl Jul 11, 2022
e78a9f8
Minor change to satisfy codebeat
gerritholl Jul 11, 2022
2aac451
Add extras dependency for skyfield and astropy
gerritholl Jul 25, 2022
1d953a8
Describe version, not only year
gerritholl Jul 25, 2022
41e6ec5
Move units note from comment to docstring
gerritholl Jul 25, 2022
7d8ff7b
Write "degrees" rather than "°"
gerritholl Jul 25, 2022
d874994
typo: or → on
gerritholl Jul 25, 2022
5e37359
Change opening line of class docstring.
gerritholl Jul 25, 2022
9509274
Add some more flexibility in parallax correction
gerritholl Jul 25, 2022
4e97767
Improve configurability in parallax correction
gerritholl Jul 26, 2022
567d27a
Add images with and without parallax correction
gerritholl Jul 26, 2022
2c3a5d7
Fix duplicate doc and rename parameter
gerritholl Jul 27, 2022
5ca49c1
Parallax docstring clarification
gerritholl Jul 27, 2022
66f02f1
Adapt unit test in test_utils
gerritholl Jul 27, 2022
a8fce07
Simplify docstring for forward_parallax
gerritholl Jul 27, 2022
5130cdb
Utility function for parallax displacement + docs
gerritholl Jul 28, 2022
dc352e3
Small docstring fixes
gerritholl Jul 28, 2022
9b5e980
Add image showing parallax magnitude for GOES-East
gerritholl Jul 28, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,8 @@ jobs:
git+https://github.com/Unidata/cftime \
git+https://github.com/mapbox/rasterio \
git+https://github.com/pydata/bottleneck \
git+https://github.com/pydata/xarray;
git+https://github.com/pydata/xarray \
git+https://github.com/astropy/astropy;

- name: Install satpy
shell: bash -l {0}
Expand Down
7 changes: 4 additions & 3 deletions AUTHORS.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,13 +25,13 @@ The following people have made contributions to this project:
- [Adam Dybbroe (adybbroe)](https://github.com/adybbroe)
- [Ulrik Egede (egede)](https://github.com/egede)
- [Joleen Feltz (joleenf)](https://github.com/joleenf)
- [Stephan Finkensieper (sfinkens)](https://github.com/sfinkens)
- [Stephan Finkensieper (sfinkens)](https://github.com/sfinkens) - Deutscher Wetterdienst
mraspaud marked this conversation as resolved.
Show resolved Hide resolved
- [Andrea Grillini (AppLEaDaY)](https://github.com/AppLEaDaY)
- [Blanka Gvozdikova (gvozdikb)](https://github.com/gvozdikb)
- [Nina Håkansson (ninahakansson)](https://github.com/ninahakansson)
- [Ulrich Hamann](https://github.com/)
- [Mitch Herbertson (mherbertson)](https://github.com/mherbertson)
- [Gerrit Holl (gerritholl)](https://github.com/gerritholl)
- [Gerrit Holl (gerritholl)](https://github.com/gerritholl) - Deutscher Wetterdienst
- [David Hoese (djhoese)](https://github.com/djhoese)
- [Marc Honnorat (honnorat)](https://github.com/honnorat)
- [Mikhail Itkin (mitkin)](https://github.com/mitkin)
Expand All @@ -42,7 +42,8 @@ The following people have made contributions to this project:
- [Janne Kotro (jkotro)](https://github.com/jkotro)
- [Ralph Kuehn (ralphk11)](https://github.com/ralphk11)
- [Panu Lahtinen (pnuu)](https://github.com/pnuu)
- [Thomas Leppelt (m4sth0)](https://github.com/m4sth0)
- [Jussi Leinonen (jleinonen)](https://github.com/jleinonen) - meteoswiss
- [Thomas Leppelt (m4sth0)](https://github.com/m4sth0) - Deutscher Wetterdienst
- [Lu Liu (yukaribbba)](https://github.com/yukaribbba)
- [Andrea Meraner (ameraner)](https://github.com/ameraner)
- [Aronne Merrelli (aronnem)](https://github.com/aronnem)
Expand Down
4 changes: 3 additions & 1 deletion continuous_integration/environment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ dependencies:
- scipy
- pyyaml
- pyproj
- pyresample
- pyresample>=1.24
- coveralls
- coverage
- codecov
Expand Down Expand Up @@ -49,6 +49,8 @@ dependencies:
- python-geotiepoints
- pooch
- pip
- skyfield
- astropy
- pip:
- trollsift
- trollimage
Expand Down
81 changes: 81 additions & 0 deletions doc/source/modifiers.rst
Original file line number Diff line number Diff line change
Expand Up @@ -55,3 +55,84 @@ on those modifiers can be found in the linked API documentation.
A complete list can be found in the `etc/composites
<https://github.com/pytroll/satpy/tree/main/satpy/etc/composites>`_
source code and in the :mod:`~satpy.modifiers` module documentation.

Parallax correction
-------------------
gerritholl marked this conversation as resolved.
Show resolved Hide resolved

.. warning::

The Satpy parallax correction is experimental and subject to change.

Since version 0.37 (mid 2022), Satpy has included a
modifier for parallax correction, implemented in the
:class:`~satpy.modifiers.parallax.ParallaxCorrectionModifier` class.
This modifier is important for some applications, but not applied
by default to any Satpy datasets or composites, because it can be
applied to any input dataset and used with any source of (cloud top)
height. Therefore, users wishing to apply the parallax correction
semi-automagically have to define their own modifier and then apply
that modifier for their datasets. An example is included
with the :class:`~satpy.modifiers.parallax.ParallaxCorrectionModifier`
API documentation. Note that Satpy cannot apply modifiers to
composites, so users wishing to apply parallax correction to a composite
will have to use a lower level API or duplicate an existing composite
recipe to use modified inputs.

The parallax correction is directly calculated from the cloud top height.
Information on satellite position is obtained from cloud top height
metadata. If no orbital parameters are present in the cloud top height
metadata, Satpy will attempt to calculate orbital parameters from the
platform name and start time. The backup calculation requires skyfield
and astropy to be installed. If the metadata include neither orbital
parameters nor platform name and start time, parallax calculation will
fail. Because the cloud top height metadata are used, it is essential
that the cloud top height data are derived from the same platform as
the measurements to be corrected are taken by.

The parallax error moves clouds away from the observer. Therefore, the
parallax correction shifts clouds in the direction of the observer. The
space left behind by the cloud will be filled with fill values. As the
cloud is shifted toward the observer, it may occupy less pixels than before,
because pixels closer to the observer have a smaller surface area. It can
also be deformed (a "rectangular" cloud may get the shape of a parallelogram).

.. figure:: https://figshare.com/ndownloader/files/36422616/preview/36422616/preview.jpg
:width: 512
:height: 512
:alt: Satellite image without parallax correction.

SEVIRI view of southern Sweden, 2021-11-30 12:15Z, without parallax correction.
This is the ``natural_color`` composite as built into Satpy.


.. figure:: https://figshare.com/ndownloader/files/36422613/preview/36422613/preview.jpg
:width: 512
:height: 512
:alt: Satellite image with parallax correction.

The same satellite view with parallax correction. The most obvious change
are the gaps left behind by the parallax correction, shown as black pixels.
Otherwise it shows that clouds have "moved" south-south-west in the direction
of the satellite. To view the images side-by-side or alternating, look at
`the figshare page <https://figshare.com/articles/figure/20211130121510-Meteosat-11-seviri-sswe-parallax_corrected_natural_color_jpg/20377203>`_

The utility function :func:`~satpy.modifiers.parallax.get_surface_parallax_displacement` allows to calculate the magnitude of the parallax error. For a cloud with a cloud top height of 10 km:

.. figure:: https://figshare.com/ndownloader/files/36462435/preview/36462435/preview.jpg
:width: 512
:height: 512
:alt: Figure showing magnitude of parallax effect.

Magnitude of the parallax error for a fictitious cloud with a cloud top
height of 10 km for the GOES-East (GOES-16) full disc.

The parallax correction is currently experimental and subject to change.
Although it is covered by tests, there may be cases that yield unexpected
or incorrect results. It does not yet perform any checks that the
provided (cloud top) height covers the area of the dataset for which
the parallax correction shall be applied.

For more general background information and web routines related to the
parallax effect, see also `this collection at the CIMSS website <https://cimss.ssec.wisc.edu/goes/webapps/parallax/>_`.

gerritholl marked this conversation as resolved.
Show resolved Hide resolved
.. versionadded:: 0.37
Loading