Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🚸 Walkthrough on creating batches of data chips #20

Merged
merged 5 commits into from
Jul 16, 2022
Merged

Commits on Jun 18, 2022

  1. 🚧 Walkthrough on creating batches of data

    Initial draft tutorial on creating batches of chipped data from full-size satellite scenes! Will be working with Sentinel-1 GRD GeoTIFFs, let's see how far this will go.
    weiji14 committed Jun 18, 2022
    Configuration menu
    Copy the full SHA
    5fa7efe View commit details
    Browse the repository at this point in the history

Commits on Jul 6, 2022

  1. 🔀 Merge branch 'main' into batching

    Bump minimum pyogrio version from 0.4.0a1 to 0.4.0 and include new XbatcherSlicer feature! Also get the refactored RioXarrayReader in.
    weiji14 committed Jul 6, 2022
    Configuration menu
    Copy the full SHA
    9a3ef3a View commit details
    Browse the repository at this point in the history

Commits on Jul 7, 2022

  1. 💡 Demo XbatcherSlicer to get 512x512 chips from larger scene

    Walkthrough how to cut up a large satellite scene into multiple smaller chips of size 512 pixels by 512 pixels. Heavy lifting done by xbatcher which handles slicing along dimensions and overlapping strides. Needed a hacky workaround in XbatcherSlicer to fix a ValueError due to the xarray.DataArray name not being set (though it should be).
    weiji14 committed Jul 7, 2022
    Configuration menu
    Copy the full SHA
    147f270 View commit details
    Browse the repository at this point in the history
  2. 💚 Install xbatcher for documentation build

    Fix readthedocs build failure because xbatcher was not installed.
    weiji14 committed Jul 7, 2022
    Configuration menu
    Copy the full SHA
    b575930 View commit details
    Browse the repository at this point in the history

Commits on Jul 16, 2022

  1. 🗃️ Collate chips into mini-batches

    Finalize tutorial by converting chips from xarray.Dataset to torch.Tensor and stacking them per mini-batch! Debated on whether to have the xarray collate function in the codebase, but let's wait for updates on xbatcher's end (xarray-contrib/xbatcher#71). Also renamed the tutorial file from batching to chipping and added more emojis to the intro section.
    weiji14 committed Jul 16, 2022
    Configuration menu
    Copy the full SHA
    de85f4a View commit details
    Browse the repository at this point in the history