Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ StackSTACMosaicIterDataPipe to mosaic tiles into one piece #63

Merged
merged 4 commits into from
Sep 22, 2022

Conversation

weiji14
Copy link
Owner

@weiji14 weiji14 commented Sep 22, 2022

An iterable-style DataPipe to mosaic tiles in an xarray.DataArray! Uses stackstac.mosaic to do the mosaic operation.

Preview at https://zen3geo--63.org.readthedocs.build/en/63/api.html#zen3geo.datapipes.StackSTACMosaic

Usage:

import pystac

from torchdata.datapipes.iter import IterableWrapper
from zen3geo.datapipes import StackSTACMosaic, StackSTACStacker

# Get list of ALOS DEM tiles to mosaic together later
item_urls = [
    "https://planetarycomputer.microsoft.com/api/stac/v1/collections/alos-dem/items/ALPSMLC30_N022E113_DSM",
    "https://planetarycomputer.microsoft.com/api/stac/v1/collections/alos-dem/items/ALPSMLC30_N022E114_DSM",
]

stac_items = [pystac.Item.from_file(href=url) for url in item_urls]
dp = IterableWrapper(iterable=[stac_items])

# Mosaic different tiles in an xarray.DataArray using DataPipe
dp_mosaic = dp.stack_stac_items().mosaic_dataarray()

# Loop or iterate over the DataPipe stream
it = iter(dp_mosaic)
dataarray = next(it)

print(dataarray.sizes)
# Frozen({'band': 1, 'y': 3600, 'x': 7200})

print(dataarray.coords)
# Coordinates:
#   * band         (band) <U4 'data'
#   * x            (x) float64 113.0 113.0 113.0 113.0 ... 115.0 115.0 115.0 115.0
#   * y            (y) float64 23.0 23.0 23.0 23.0 23.0 ... 22.0 22.0 22.0 22.0
#     proj:epsg    int64 4326
#     instruments  <U5 'prism'
#     platform     <U4 'alos'
#     gsd          int64 30
#     proj:shape   object {3600}
#     epsg         int64 4326

print(dataarray.attrs["spec"])
# RasterSpec(epsg=4326, bounds=(113.0, 22.0, 115.0, 23.0), resolutions_xy=(0.0002777777777777778, 0.0002777777777777778))

TODO:

  • Initial implementation with a doctest
  • Add unit test

Needed for #62

An iterable-style DataPipe to mosaic tiles in a DataArray! Uses stackstac to do the mosaic operation. Included a doctest that mosaicks two ALOS DEM tiles (stored at different positions in the time index of an xarray.DataArray) into an xarray.DataArray with dimensions (band, y, x). Added a new section in the API docs too!
@weiji14 weiji14 added the feature New feature or request label Sep 22, 2022
@weiji14 weiji14 added this to the 0.5.0 milestone Sep 22, 2022
@weiji14 weiji14 self-assigned this Sep 22, 2022
Ensure that tiles within a 4D xarray.DataArray can be mosaicked into a 3D xarray.DataArray.
@weiji14 weiji14 marked this pull request as ready for review September 22, 2022 18:02
Looks a bit nicer without the triple >>> showing up.
@weiji14 weiji14 merged commit 3c7a88b into main Sep 22, 2022
@weiji14 weiji14 deleted the stackstac/mosaic branch September 22, 2022 18:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant