Skip to content

Releases: databrickslabs/mosaic

v0.4.2

14 May 17:08
66a0bc1
Compare
Choose a tag to compare

v0.4.2 [DBR 13.3 LTS]

  • Geopandas now set to "<0.14.4,>=0.14" due to conflict with minimum numpy version in geopandas 0.14.4.
  • H3 python changed from "==3.7.0" to "<4.0,>=3.7" to pick up patches.
  • Fixes an issue with fallback logic when deserializing subdatasets from a zip.
  • Adjusts data used to speed up a long-running test.
  • Streamlines setup_gdal and setup_fuse_install functions:
    • init script and resource copy logic fixed to repo "main" (.so) / "latest" (.jar).
    • added apt-get lock handling for native installs.
    • removed support for UbuntuGIS PPA as GDAL version no longer compatible with jammy default (3.4.x).

v0.4.1

15 Mar 17:44
a64da84
Compare
Choose a tag to compare

v0.4.1 [DBR 13.3 LTS]

  • Fixed python bindings for MosaicAnalyzer functions.
  • Added tiller functions, ST_AsGeoJSONTile and ST_AsMVTTile, for creating GeoJSON and MVT tiles as aggregations of geometries.
  • Added filter and convolve functions for raster data.
  • Raster tile schema changed to be <tile:struct<index_id:bigint, tile:binary, metadata:map<string, string>>.
  • Raster tile metadata will contain driver, parentPath and path.
  • Raster tile metadata will contain warnings and errors in case of failures.
  • All raster functions ensure rasters are TILED and not STRIPED when appropriate.
  • GDAL cache memory has been decreased to 512MB to reduce memory usage and competition with Spark.
  • Add RST_MakeTiles that allows for different raster creations.
  • Rasters can now be passed as file pointers using checkpoint location.
  • Added logic to handle zarr format for raster data.
  • Added RST_SeparateBands to separate bands from a raster for NetCDF and Zarr formats.

v0.4.0

15 Jan 19:49
1621d25
Compare
Choose a tag to compare
  • First release for DBR 13.3 LTS which is Ubuntu Jammy and Spark 3.4.1. Not backwards compatible, meaning it will not run on prior DBRs; requires either a Photon DBR or a ML Runtime (Standard, non-Photon DBR no longer allowed).
  • New setup_fuse_install function to meet various requirements arising with Unity Catalog + Shared Access clusters; removed the scala equivalent function, making artifact setup and install python-first for scala and Spark SQL.
  • Removed OSS ESRI Geometry API for 0.4 series, JTS now the only vector provider.
  • MosaicAnalyzer functions now accept Spark DataFrames instead of MosaicFrame, which has been removed.
  • Docs for 0.3.x have been archived and linked from current docs; notebooks for 0.3.x have been separated from current notebooks.
  • This release targets Assigned (vs Shared Access) clusters and offers python and scala language bindings; SQL expressions will not register in this release within Unity Catalog.

v0.3.14

14 Dec 17:54
3b5b597
Compare
Choose a tag to compare
  • Fixes for Warning and Error messages on mosaic_enable call.
  • Performance improvements for raster functions.
  • Fix support for GDAL configuration via spark config (use 'spark.databricks.labs.mosaic.gdal.' prefix).

v0.3.13

12 Dec 14:12
aad7db4
Compare
Choose a tag to compare

R bindings generation fixed and improved.
Remove usage of /vsimem/ drivers for GDAL due to memory leaks.
Add support for MapAlgebra expressions via RST_MapAlgebra.
Add support for custom combine python functions via RST_DerivedBand.
Improve test coverage.
Allow for GDAL configuration via spark config (use 'spark.databricks.labs.mosaic.gdal.' prefix).

v0.3.12

31 Oct 16:58
67a0cdc
Compare
Choose a tag to compare
  • Make JTS default Geometry Provider
  • Updated R bindings for sparkr and sparklyr
  • Add raster tile functions.
  • Expand the support for raster manipulation.
  • Add abstractions for running distributed gdal_translate, gdalwarp, gdalcalc, etc.
  • Add RST_BoundingBox, RST_Clip, RST_CombineAvg, RST_CombineAvgAgg, RST_FromBands, RST_FromFile, RST_GetNoData,
    RST_InitNoData, RST_Merge, RST_MergeAgg, RST_NDVI, RST_ReTile, RST_SetNoData, RST_Subdivide
  • Add RST_Tessellate that generates H3 tiles from rasters.
  • Add RST_ToOverlappingTiles that generates tiles with overlapping areas from rasters.
  • Add GDAL generic format reader.

This release is for DBR < 13, mainly DBR 12.2 LTS. Stay tuned for a release very soon that targets DBR 13.3 LTS.

v0.3.11

01 Jun 08:35
0aec1f8
Compare
Choose a tag to compare
  • Update the CONTRIBUTING.md to follow the standard process.
  • Fix for issue 383: grid_pointascellid fails with a Java type error when run on an already instantiated point.
  • Bump maven-assembly-plugin from 3.5.0 to 3.6.0.
  • Fix the cluster side init script generation.
  • Fixed photon check for DBR warnings.
  • Bump maven-surefire-plugin from 3.0.0 to 3.1.0.
  • Fix the bug described in issue 360: incomplete coverage from grid_geometrykring and grid_tessellate.
  • Add default value for script location path to init script.

v0.3.10

06 May 20:07
00d54ee
Compare
Choose a tag to compare
  • Fixed k-ring logic for BNG grid close to the edge of the grid
  • Fixed deprecated st_centroid2D expression
  • Documentation improvements
  • Fix handling null geometry fields for OGRFileFormat reader
  • Added warning for future DBR environment support
  • Added support for GeometryCollection
  • Fix intersection operations with ESRI geometry APIs
  • Fixed custom grid issues for grids not multiple of the root size resolution
  • Fixed python binding for rst_georeference
  • Fixed ESRI create polygon with correct path order with ESRI APIs
  • Fixed automatic SQL registration with GDAL

v0.3.9

27 Mar 10:32
5c4c047
Compare
Choose a tag to compare
  • Fixed k-ring interpolation on raster data read

v0.3.8

20 Mar 12:31
050e43a
Compare
Choose a tag to compare
  • Added readers for default GDAL raster drivers (https://gdal.org/drivers/raster/index.html)
    • TIFF
    • COG
    • NetCDF
    • ... And more
  • Added readers for default GDAL vector drivers (https://gdal.org/drivers/vector/index.html)
    • Shapefiles
    • Geodatabase (File GDB)
    • ... And more
  • Added custom grid index system for arbitrary CRS
  • Added Spatial KNN example
  • Refactored and simplified Mosaic expressions definition
  • Documentation updates and improvements