Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: get_source_data #485

Merged
merged 9 commits into from
Aug 3, 2021
Merged

Feature: get_source_data #485

merged 9 commits into from
Aug 3, 2021

Conversation

jmilloy
Copy link
Collaborator

@jmilloy jmilloy commented Aug 2, 2021

Datasource

>>> node = podpac.data.Array(
...             source=np.ones((3, 4)),
...             coordinates=podpac.Coordinates([range(3), range(4)], ["lat", "lon"]),
...         )
>>> 
>>> node.get_source_data()
<xarray.UnitsDataArray (lat: 3, lon: 4)>
array([[1., 1., 1., 1.],
       [1., 1., 1., 1.],
       [1., 1., 1., 1.]])
Coordinates:
  * lat      (lat) float64 0.0 1.0 2.0
  * lon      (lon) float64 0.0 1.0 2.0 3.0
Attributes:
    layer_style:   <podpac.core.style.Style object at 0x7f7c09fcb730>
    crs:           +proj=longlat +datum=WGS84 +no_defs +vunits=m
    geotransform:  (-0.5, 1.0, 0.0, -0.5, 0.0, 1.0)

TileCompositor

>>> a = ArrayRaw(source=np.arange(5) + 100, coordinates=podpac.Coordinates([[0, 1, 2, 3, 4]], dims=["lat"]))
>>> b = ArrayRaw(source=np.arange(5) + 200, coordinates=podpac.Coordinates([[5, 6, 7, 8, 9]], dims=["lat"]))
>>> c = ArrayRaw(source=np.arange(5) + 300, coordinates=podpac.Coordinates([[10, 11, 12, 13, 14]], dims=["lat"]))
>>> node = TileCompositorRaw(sources=[a, b, c])
>>> node.get_source_data()
<xarray.UnitsDataArray (lat: 15)>
array([100., 101., 102., 103., 104., 200., 201., 202., 203., 204., 300.,
       301., 302., 303., 304.])
Coordinates:
  * lat      (lat) float64 0.0 1.0 2.0 3.0 4.0 5.0 ... 10.0 11.0 12.0 13.0 14.0
Attributes:
    layer_style:  <podpac.core.style.Style object at 0x7f7bb627a340>
    crs:          +proj=longlat +datum=WGS84 +no_defs +vunits=m

Bounds

>>> node.get_source_data({"lon": (1.5, 4.5)})
<xarray.UnitsDataArray (lat: 3, lon: 2)>
array([[1., 1.],
       [1., 1.],
       [1., 1.]])
Coordinates:
  * lat      (lat) float64 0.0 1.0 2.0
  * lon      (lon) float64 2.0 3.0
Attributes:
    layer_style:   <podpac.core.style.Style object at 0x7f7bb6566190>
    crs:           +proj=longlat +datum=WGS84 +no_defs +vunits=m
    geotransform:  (1.5, 1.0, 0.0, -0.5, 0.0, 1.0)

closes #468

@jmilloy jmilloy self-assigned this Aug 2, 2021
@jmilloy jmilloy requested a review from mpu-creare August 2, 2021 19:08
@jmilloy jmilloy added the enhancement New feature or request label Aug 2, 2021
@jmilloy
Copy link
Collaborator Author

jmilloy commented Aug 2, 2021

@mpu-creare This is a simple enhancement. It is just a general method implemented in the DataSource class and the TileCompositor class. Neither method calls eval, so it is a true enhancement and not just an API wrapper. However, some child datasource or datalib nodes could probably override get_source_data to be more efficient, such as perhaps the Rasterio node. At this point, I haven't done that (we can do that as needed as optimization), but I want to know if you would prefer to do some of that for this PR.

Copy link
Contributor

@mpu-creare mpu-creare left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine approving this. I did have a few suggestions on wording and potential improvements, but I'll leave it up to you.

The major thing is the OGR node which will NOT work with this implementation. It will throw an exception. Any ideas?

doc/source/overview.md Show resolved Hide resolved
doc/source/overview.md Outdated Show resolved Hide resolved
podpac/core/compositor/tile_compositor.py Outdated Show resolved Hide resolved
podpac/core/data/datasource.py Show resolved Hide resolved
podpac/core/data/datasource.py Show resolved Hide resolved
podpac/core/data/datasource.py Show resolved Hide resolved
jmilloy and others added 4 commits August 2, 2021 17:15
Co-authored-by: mpu-creare <mpu-creare@users.noreply.github.com>
Co-authored-by: mpu-creare <mpu-creare@users.noreply.github.com>
@jmilloy jmilloy merged commit 180af3d into develop Aug 3, 2021
@jmilloy jmilloy deleted the feature/get_source_data branch August 3, 2021 13:07
@mpu-creare mpu-creare mentioned this pull request Dec 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

get source data for Datasource and TileCompositors
2 participants