Releases: onthegomap/planetiler
v0.8.3
What's Changed
This release includes a number of bug fixes and YAML profile improvements, including utilities for computing feature size in non-tile-based units like meters or acres.
To run this release from Java, download the attached planetiler.jar
, or from docker use ghcr.io/onthegomap/planetiler:0.8.3
New Features and Improvements
- Add utilities for computing feature size in meters by @msbarry in #1078
- Shortcut through simplification step if distanceTolerance is negative by @davecraig in #1093
- Expose vector tile feature ID method on
RelationInfo
by @msbarry in #960 - Update old wikidata items by @phanecak-maptiler in #971
- Fix handling of string typed null values by @phanecak-maptiler in #987
- YAML Schema improvements:
- Add length/area units and CEL expression geometry accessors by @msbarry in #1084
- Add line midpoint geometry type by @msbarry in #1072
- Load large yaml files by @msbarry in #1039
- Support yaml merge operator by @msbarry in #1040
- Allow dynamic expressions on left hand side of match statements by @msbarry in #1063
- Convert
${feature.source/source_layer}: [ ... ]
match expression toMatchSource
/MatchSourceLayer
by @msbarry in #1065 - Fix expression issues by @msbarry in #1030
- Make projection configurable for yaml sources by @msbarry in #1066
Cleanup & Documentation improvements
- Clean-up of
LanguageUtils
by @phanecak-maptiler in #1068 - Expose caresAboutLayer() and new ForwardingProfile(config) ... by @phanecak-maptiler in #968
- added
dependsOnLayer()
... by @phanecak-maptiler in #991 - rename LayerPostProcesser to LayerPostProcessor by @bdon in #1031
- Update README.md to reflect change in demo tile hosting by @quincylvania in #955
- Fix
min_tile_cover_size
docs by @msbarry in #967 - Spelling correction in bug_report.md by @zstadler in #1037
- Clarify
max_point_buffer
by @zstadler in #1046 - Update the
if
/value
expression description by @zstadler in #1081 - Fix broken link by @msbarry in #1023
- Fix comment about needing admin.osc.bz2 by @Zverik in #1034
Dependencies
Dependency upgrades in this release
- 0.8.2 was released, so let's use 0.8.3 for SNAPSHOT by @phanecak-maptiler in #970
- update openmaptiles dependency by @Strubbl in #947
- Bump jackson.version from 2.17.1 to 2.17.2 by @dependabot in #948
- Bump org.roaringbitmap:RoaringBitmap from 1.1.0 to 1.2.0 by @dependabot in #949
- Bump org.apache.maven.plugins:maven-failsafe-plugin from 3.3.0 to 3.3.1 by @dependabot in #951
- Bump org.apache.maven.plugins:maven-surefire-plugin from 3.3.0 to 3.3.1 in /.github/planetiler-examples-dependabot by @dependabot in #950
- Bump org.apache.maven.plugins:maven-surefire-plugin from 3.3.0 to 3.3.1 by @dependabot in #952
- Bump org.apache.maven.plugins:maven-failsafe-plugin from 3.3.0 to 3.3.1 in /.github/planetiler-examples-dependabot by @dependabot in #953
- Bump org.roaringbitmap:RoaringBitmap from 1.2.0 to 1.2.1 by @dependabot in #954
- Bump geotools.version from 31.2 to 31.3 by @dependabot in #956
- Bump org.apache.maven.plugins:maven-javadoc-plugin from 3.7.0 to 3.8.0 by @dependabot in #958
- Bump com.google.protobuf:protobuf-java from 3.25.3 to 3.25.4 by @dependabot in #962
- Bump org.tukaani:xz from 1.9 to 1.10 by @dependabot in #966
- Bump org.xerial:sqlite-jdbc from 3.46.0.0 to 3.46.0.1 by @dependabot in #969
- Bump org.projectnessie.cel:cel-bom from 0.4.4 to 0.4.5 by @dependabot in #972
- Bump org.xerial.snappy:snappy-java from 1.1.10.5 to 1.1.10.6 by @dependabot in #973
- Bump org.slf4j:slf4j-api from 2.0.13 to 2.0.14 by @dependabot in #974
- Bump org.slf4j:slf4j-api from 2.0.14 to 2.0.15 by @dependabot in #977
- Bump org.junit.jupiter:junit-jupiter-api from 5.10.3 to 5.11.0 in /.github/planetiler-examples-dependabot by @dependabot in #980
- Bump org.slf4j:slf4j-api from 2.0.15 to 2.0.16 by @dependabot in #978
- Bump org.apache.maven.plugins:maven-surefire-plugin from 3.3.1 to 3.4.0 by @dependabot in #985
- Bump org.apache.maven.plugins:maven-surefire-plugin from 3.3.1 to 3.4.0 in /.github/planetiler-examples-dependabot by @dependabot in #986
- Bump org.apache.maven.plugins:maven-deploy-plugin from 3.1.2 to 3.1.3 by @dependabot in #988
- Bump org.apache.maven.plugins:maven-failsafe-plugin from 3.3.1 to 3.4.0 in /.github/planetiler-examples-dependabot by @dependabot in #989
- Bump com.google.guava:guava from 33.2.1-jre to 33.3.0-jre by @dependabot in #990
- Bump org.apache.maven.plugins:maven-failsafe-plugin from 3.3.1 to 3.4.0 by @dependabot in #994
- Bump org.projectnessie.cel:cel-bom from 0.4.5 to 0.5.0 and protobuf from 3.25.4 to 4.27.3 by @dependabot in #997
- Bump org.xerial:sqlite-jdbc from 3.46.0.1 to 3.46.1.0 by @dependabot in #998
- Bump org.apache.maven.plugins:maven-gpg-plugin from 3.2.4 to 3.2.5 by @dependabot in #999
- Bump org.apache.maven.plugins:maven-failsafe-plugin from 3.4.0 to 3.5.0 in /.github/planetiler-examples-dependabot by @dependabot in #1001
- Bump junit.version from 5.10.3 to 5.11.0 by @dependabot in #1000
- Bump org.apache.maven.plugins:maven-surefire-plugin from 3.4.0 to 3.5.0 in /.github/planetiler-examples-dependabot by @dependabot in #1004
- Bump org.apache.maven.plugins:maven-surefire-plugin from 3.4.0 to 3.5.0 by @dependabot in #1003
- Bump org.locationtech.jts:jts-core from 1.19.0 to 1.20.0 by @dependabot in #1005
- Bump blue.strategic.parquet:parquet-floor from 1.45 to 1.46 by @dependabot in #1006
- Bump org.codehaus.mojo:buildnumber-maven-plugin from 3.2.0 to 3.2.1 by @dependabot in #1007
- Bump org.yaml:snakeyaml from 2.2 to 2.3 by @dependabot in #1008
- Bump org.apache.maven.plugins:maven-failsafe-plugin from 3.4.0 to 3.5.0 by @dependabot in #1009
- Bump org.apache.maven.plugins:maven-javadoc-plugin from 3.8.0 to 3.10.0 by @dependabot in #1010
- Bump io.github.zlika:reproducible-build-maven-plugin from 0.16 to 0.17 by @dependabot in #1011
- Bump log4j.version from 2.23.1 to 2.24.0 by @dependabot in #1012
- Bump com.google.protobuf:protobuf-java from 4.27.3 to 4.28.1 by @dependabot in #1013
- Bump org.mockito:mockito-core from 5.12.0 to 5.13.0 by @dependabot in #1014
- Bump org.apache.maven.plugins:maven-gpg-plugin from 3.2.5 to 3.2.6 by @dependabot in #1016
- Bump org.projectnessie.cel:cel-bom from 0.5.0 to 0.5.1 by @dependabot in #1018
- Bump org.commonmark:commonmark from 0.22.0 to 0.23.0 by @dependabot in #1019
- Bump org.snakeyaml:snakeyaml-engine from 2.7 to 2.8 by @dependabot in #1020
- Bump org.xerial.snappy:snappy-java from 1.1.10.6 to 1.1.10.7 by @dependabot in #1022
- Bump com.google.protobuf:protobuf-java from 4.28.1 to 4.28.2 in /planetiler-core by @dependabot ...
v0.8.2
v0.8.1
What's Changed
Minor improvements since 0.8.0 release:
- More efficient geoparquet geoarrow conversion by @msbarry in #933, #934
- Encode OSM node/way/relation in vector tile feature ids by @msbarry in #826
- Fallback when tile weights are missing by @msbarry in #937
And dependency upgrades:
- Bump org.apache.maven.plugins:maven-failsafe-plugin from 3.2.5 to 3.3.0 in /.github/planetiler-examples-dependabot by @dependabot in #925
- Bump org.apache.maven.plugins:maven-surefire-plugin from 3.2.5 to 3.3.0 by @dependabot in #926
- Bump org.apache.maven.plugins:maven-failsafe-plugin from 3.2.5 to 3.3.0 by @dependabot in #927
- Bump blue.strategic.parquet:parquet-floor from 1.44 to 1.45 by @dependabot in #929
- Bump com.carrotsearch:hppc from 0.9.1 to 0.10.0 by @dependabot in #930
- Bump org.roaringbitmap:RoaringBitmap from 1.0.6 to 1.1.0 by @dependabot in #931
- Bump org.apache.maven.plugins:maven-jar-plugin from 3.4.1 to 3.4.2 by @dependabot in #932
- Bump org.junit.jupiter:junit-jupiter-api from 5.10.2 to 5.10.3 in /.github/planetiler-examples-dependabot by @dependabot in #936
- Bump junit.version from 5.10.2 to 5.10.3 by @dependabot in #935
Full Changelog: v0.8.0...v0.8.1
v0.8.0
What's Changed
This release adds support for geoparquet sources like Overture Maps. It also adds support for writing custom Java profiles without any build tools. See https://github.com/onthegomap/planetiler-examples for instructions getting started and examples using OpenStreetMap and Overture data.
To run this release from Java, download the attached planetiler.jar
, or from docker use ghcr.io/onthegomap/planetiler:0.8.0
There is ongoing discussion about the project in the #planetiler channel on OSM-US slack. If you are interested in joining, get an invite here!
New Features and Improvements
- Add geoparquet/Overture support
- Iniitial geoparquet support by @msbarry in #888
- Remove primary geometry from parquet feature tags by @msbarry in #893
- Attach parquet schema to features by @msbarry in #897
- Avoid deserializing entire parquet geometry just to determine type by @msbarry in #898
- Make sure
FeatureCollector
can handle structs by @msbarry in #904 - Structured tag api by @msbarry in #895
- Add support for linear-referenced tags for Overture Maps Data by @msbarry in #900
- Java API improvements
- Min polygon area centroid by @msbarry in #720
- Add API for maximum inscribed circle/pole of inaccessibility centerpoint of a polygon by @msbarry in #723
- Add
setAttrWithMinSize
to feature API by @msbarry in #725 - add extraMetadata to profile interface [#794] by @bdon in #795
- Add whole-tile postprocess hook by @msbarry in #802
- Add omit() feature collector api and
--refresh-sources
arg by @msbarry in #810 - Add tile post-process hook to forwarding profile by @msbarry in #813
- Add firstCoordinate vector tile feature helper by @msbarry in #814
- Add polygon index intersection utility by @msbarry in #866
- Attach sources to fat jar to improve no-build IDE experience by @msbarry in #892
- Make VectorTile.Feature implement WithTags by @msbarry in #896
- Parquet/no-build-tool java profile usability improvements by @msbarry in #914
- Handle immutable output from post process layer/tile feature methods by @msbarry in #920
- YAML Schema improvements
- Generate deterministic output for the same input
- Add detailed jts debugging info by @msbarry in #703
- Archive compare by @msbarry in #780
- Deterministic merging by @msbarry in #785
- Make OSM polygon construction deterministic by @msbarry in #788
- Fix feature merge consistent ordering by @msbarry in #789
- Deterministic IDs on SimpleFeatures by @msbarry in #791
- Check for deterministic output in CI by @msbarry in #792
- Upgrade to OpenMapTiles 3.15 by @msbarry and @phanecak-maptiler in #764, #764, #835, #880, #923
- Use maven build time for docker image by @msbarry in #744
- Use push-down bbox filter for shapefiles by @msbarry in #757
- Add support for "files"-archive by @bbilger in #761
- Use snappy compression for
--compress-temp
flag by @msbarry in #905 - Log more details on feature processing errors by @msbarry in #884
Bugs
- Fix log4j log format by @msbarry in #697
- Logging fix by @msbarry in #699
- Add detailed jts debugging info by @msbarry in #703
- Fix buffer union unbuffer exceptions by @msbarry in #713
- Compute layer attr stats from actual vector tile features by @msbarry in #752
- fix center coordinates of pmtiles writer by @bdon in #769
- Remove log4j usage by @cmahnke in #776
- remove log4j-CloseShieldOutputStream usage by @bbilger in #779
- Fix comparison of multipolygons bounds check [#780] by @bdon in #793
- Attempt to fix IOException/GOAWAY happening within parseResults() by @phanecak-maptiler in #818
- Filter out null MultiExpression entries by @phanecak-maptiler in #822
- Safer natural earth unzip by @msbarry in #825
- Fix tile size stats memory leak by @msbarry in #861
- exclude slf4j-nop from geopackage to avoid SLF4J warning by @phanecak-maptiler in #874
- Ignore duplicate multipolygon relation members by @msbarry in #876
- Log more details on feature processing errors by @msbarry in #884
- Prevent double MultExpression matching by @msbarry in #916
Contributor/Documentation improvements
- Upgrade to Java 21 by @msbarry in #693
- Use some new Java 21 features by @msbarry in #695
- Upgrade eclipse formatter by @msbarry in #696
- Rewrite downloader using virtual threads by @msbarry in #702
- Updated benchmarks by @msbarry in #798
- Small benchmarks by @msbarry in #801
- Update GCP benchmarks by @msbarry in #803
- Add note about java update process by @wipfli in #807
- Remove quoting side-bars from planetiler-custommap/README.md by @zstadler in #844
- Fix typo by @wipfli in #907
Dependencies
Once again, @dependabot has been busy keeping dependencies up-to-date.
Dependency upgrades in this release
- Bump com.diffplug.spotless:spotless-maven-plugin from 2.39.0 to 2.40.0 by @dependabot in #678
- Bump styfle/cancel-workflow-action from 0.11.0 to 0.12.0 by @dependabot in #680
- Bump mil.nga.geopackage:geopackage from 6.6.2 to 6.6.3 by @dependabot in #681
- Bump com.google.protobuf:protobuf-java from 3.24.3 to 3.24.4 by @dependabot in #682
- Bump org.mockito:mockito-core from 5.5.0 to 5.6.0 by @dependabot in #683
- Bump com.google.guava:guava from 32.1.2-jre to 32.1.3-jre by @dependabot in #684
- Bump geotools.version from 29.2 to 30.0 by @dependabot in #685
- Bump org.apache.maven.plugins:maven-failsafe-plugin from 3.1.2 to 3.2.1 in /.github/planetiler-examples-dependabot by @dependabot in #689
- Bump actions/setup-node from 3 to 4 by @dependabot in #688
- Bump jackson.version from 2.15.2 to 2.15.3 by @dependabot in #687
- Bump org.xerial:sqlite-jdbc from 3.43.0.0 to 3.43.2.1 by @dependabot in #692
- Bump org.apache.maven.plugins:maven-surefire-plugin from 3.1.2 to 3.2.1 in /.github/planetiler-examples-dependabot by @dependabot in #691
- Bump log4j.version from 2.20.0 to 2.21.1 by @dependabot in #694
- Bump org.apache.maven.plugins:maven-failsafe-plugin from 3.1.2 to 3.2.1 by @...
v0.8.0-pre.2
Built a few no-build-tool java profiles using Overture and OSM data in https://github.com/onthegomap/planetiler-examples and addressed a few bugs and usability improvements in #914
v0.8.0-pre.1
Bug fixes, performance improvements, dependency upgrades, plus:
- Add
--layer-stats
option to output a layer stats tsv - Make output deterministic for the same input
- Upgrade to openmaptiles 3.15 (including ~50% tile size reduction from layer stats debugging)
- Add support for geoparquet (ie. overture) sources and structured feature attributes
- Improve support for writing a java profile against planetiler.jar without any build tools (will add an example repo)
- profile API improvements: omit(), whole tile post-process hook, partial-line attributes (used by overture)
Putting out a pre-release to test no-build-tool java profiles against attached planetiler.jar
v0.7.0
What's Changed
This release reduces openmaptiles tile sizes by 40-50% and includes tools other profiles can use to use to optimize their tile sizes:
--max-point-buffer=4
argument to remove points outside the tile boundary if your client handles label collisions between tiles (web/native but not raster)--output-layerstats
to write a layer stats tsv.gz file you can analyze with duckdb to look for improvementsFeatureMerge
Utilities to to merge geometries with the same attributes into multigeometries. See openmaptiles/planetiler-openmaptiles#112 which applies these improvements to openmaptiles profile for close to 90% savings on some dense housenumber/building layers.
This release also adds support for writing plugins in other languages to send tiles to other data stores. See #639 for examples of how to write to mysql, postgres, or s3.
To run this release from Java, download the attached planetiler.jar, or from docker use ghcr.io/onthegomap/planetiler:0.7.0
There is ongoing discussion about the project in the #planetiler channel on OSM-US slack. If you are interested in joining, get an invite here!
New Features and Improvements
- Tile size improvements:
- Output more detailed tile size statistics using
--output-layerstats
flag by @msbarry in #656 - Utilities to reduce tile size by @msbarry in #669
- Sort merged vector tile features by hilbert order by @msbarry in #673
- Upgrade planetiler-openmaptiles to pull in openmaptiles/planetiler-openmaptiles#112 by @msbarry in #672
- Output more detailed tile size statistics using
- Add support for simple file output streams: CSV, JSON, protobuf by @bbilger in #639
- Add support for Overture maps 2023-04-02-alpha release by @erik in #599
- Expose
min_size
in configurable schema by @wipfli in #421 - Permit post-process merging in custommap schemas by @zhibek in #626
- Upgrade openmaptiles snapshot by @msbarry in #676
Bug Fixes
- Handle empty geopackage geometries by @msbarry in #561
- Fix NPE on way by @msbarry in #562
- Exclude relations with invalid multipolygon geometry by @erik in #564
- Fix invalid polygons before snapping to tile coordinates by @msbarry in #566
- Fallback to lenient shapefile parsing by @msbarry in #570
- Page through S3 index by @msbarry in #608
- Fix regex escaping in polygon file parsing by @mikaelhg in #627
- Fix downloading tile weights in CI by @msbarry in #668
Contributor/Documentation Improvements
- Improve docs for generating custom vector tiles with a schema yaml by @zhibek in #622
- remove the old TileEncodingResult by @bbilger in #642
- Fix macro in layerstats readme by @msbarry in #671
- Bump version to 0.7.0 by @msbarry in #677
Dependencies
Once again, @dependabot has been busy keeping dependencies up-to-date.
Dependency upgrades in this release
- Bump jacoco-maven-plugin from 0.8.8 to 0.8.9 by @dependabot in #545
- Bump maven-enforcer-plugin from 3.2.1 to 3.3.0 by @dependabot in #547
- Bump spotless-maven-plugin from 2.35.0 to 2.36.0 by @dependabot in #548
- Bump mockito-core from 5.2.0 to 5.3.0 by @dependabot in #549
- Bump protobuf-java from 3.22.2 to 3.22.3 by @dependabot in #553
- Bump icu4j from 72.1 to 73.1 by @dependabot in #554
- Bump cel-bom from 0.3.13 to 0.3.15 by @dependabot in #557
- Bump jackson.version from 2.14.2 to 2.15.0 by @dependabot in #563
- Bump junit-jupiter-api from 5.9.2 to 5.9.3 in /.github/planetiler-examples-dependabot by @dependabot in #568
- Bump jacoco-maven-plugin from 0.8.9 to 0.8.10 by @dependabot in #567
- Bump junit.version from 5.9.2 to 5.9.3 by @dependabot in #569
- Bump RoaringBitmap from 0.9.39 to 0.9.40 by @dependabot in #571
- Bump mockito-core from 5.3.0 to 5.3.1 by @dependabot in #572
- Bump protobuf-java from 3.22.3 to 3.22.4 by @dependabot in #574
- Bump maven-failsafe-plugin from 3.0.0 to 3.1.0 in /.github/planetiler-examples-dependabot by @dependabot in #576
- Bump cel-bom from 0.3.13 to 0.3.16 by @dependabot in #575
- Bump maven-surefire-plugin from 3.0.0 to 3.1.0 in /.github/planetiler-examples-dependabot by @dependabot in #578
- Bump maven-gpg-plugin from 3.0.1 to 3.1.0 by @dependabot in #577
- Bump maven-surefire-plugin from 3.0.0 to 3.1.0 by @dependabot in #580
- Bump maven-failsafe-plugin from 3.0.0 to 3.1.0 by @dependabot in #581
- Bump protobuf-java from 3.22.4 to 3.23.0 by @dependabot in #582
- Bump maven-assembly-plugin from 3.5.0 to 3.6.0 by @dependabot in #583
- Bump maven-assembly-plugin from 3.5.0 to 3.6.0 in /.github/planetiler-examples-dependabot by @dependabot in #584
- Bump RoaringBitmap from 0.9.40 to 0.9.44 by @dependabot in #585
- Bump jackson.version from 2.15.0 to 2.15.1 by @dependabot in #586
- Bump jib-maven-plugin from 3.3.1 to 3.3.2 by @dependabot in #588
- Bump protobuf-java from 3.23.0 to 3.23.1 by @dependabot in #589
- Bump sqlite-jdbc from 3.41.2.1 to 3.41.2.2 in /planetiler-core by @dependabot in #590
- Bump geotools.version from 29.0 to 29.1 by @dependabot in #591
- Bump sqlite-jdbc from 3.41.2.2 to 3.42.0.0 by @dependabot in #592
- Bump cel-bom from 0.3.16 to 0.3.17 by @dependabot in #593
- Bump guava from 31.1-jre to 32.0.0-jre by @dependabot in #594
- Bump buildnumber-maven-plugin from 3.0.0 to 3.1.0 by @dependabot in #595
- Bump flatten-maven-plugin from 1.4.1 to 1.5.0 by @dependabot in #596
- Bump jackson.version from 2.15.1 to 2.15.2 by @dependabot in #598
- Bump maven-failsafe-plugin from 3.1.0 to 3.1.2 in /.github/planetiler-examples-dependabot by @dependabot in #600
- Bump maven-surefire-plugin from 3.1.0 to 3.1.2 by @dependabot in #602
- Bump maven-surefire-plugin from 3.1.0 to 3.1.2 in /.github/planetiler-examples-dependabot by @dependabot in #603
- Bump spotless-maven-plugin from 2.36.0 to 2.37.0 by @dependabot in #604
- Bump jnr-ffi from 2.2.12 to 2.2.14 by @dependabot in #605
- Bump maven-source-plugin from 3.2.1 to 3.3.0 by @dependabot in #606
- Bump maven-failsafe-plugin from 3.1.0 to 3.1.2 by @dependabot in #607
- Bump guava from 32.0.0-jre to 32.0.1-jre by @dependabot in #609
- DON'T Bump protobuf-java from 3.23.1 to 3.23.3 by @dependabot in #610
- Bump buildnumber-maven-plugin from 3.1.0 to 3.2.0 by @dependabot in #611
- Bump mockito-core from 5.3.1 to 5.4.0 by @dependabot in #612
- Bump icu4j from 73.1 to 73.2 by @dependabot in #613
- Bump RoaringBitmap from 0.9.44 to 0.9.45 by @dependabot in https://github.com/onthegomap/p...
v0.6.0
What's Changed
This release adds several new features:
- YAML-driven configuration schemas
- Generate Shortbread Schema with shortbread.yml that uses the new YAML configuration language
- Generate OSM QA Tiles by running
planetiler.jar osm-qa
- Switch between PMTiles or MBTiles output by specifying
--output=output.pmtiles
or--output=output.mbtiles
(see #520 for more usage details) - Increase
--maxzoom
limit from 14 to 15 - Extract openmaptiles profile to openmaptiles/planetiler-openmaptiles repo, but
planetiler.jar
and the docker container still generate openmaptiles 3.14 by default or with theopenmaptiles
argument - Filter output by polygon by running with
--polygon bounds.poly
- Support for GeoPackage input files. Interested in adding support for more input formats? Use #413 as a template.
To run this release from Java, download the attached planetiler.jar, or from docker use ghcr.io/onthegomap/planetiler:0.6.0
There is ongoing discussion about the project in the #planetiler channel on OSM-US slack. If you are interested in joining, get an invite here!
New Features and Improvements
- YAML configuration schemas
- PMTiles output format
- Tile archive refactor by @bdon in #443
- add VarInt encoder for unsigned longs [#98] by @bdon in #449
- refactor VectorLayers from mbtiles into util.LayerStats [#98] by @bdon in #456
- FeatureGroup can emit in either TMS or Hilbert order. [#98] by @bdon in #455
- Speed up VarInt encoding, remove emitTilesInOrder option [#98] by @bdon in #460
- Pmtiles package low level header and directory serialization/deserialization [#98] by @bdon in #466
- Finish PMTiles write implementation and basic reader by @bdon in #502
- Expose pmtiles writer by @msbarry in #520
- Increase max zoom from z14 to z15
- Extract OpenMapTiles schema to openmpatiles/planetiler-openmaptiles and upgrade to 3.14
- Decrease power sample minzoom from 12 to 7 by @wipfli in #268
- Allow more than 256 attribute keys by @ZeLonewolf in #275
- feat: added
maxzoomForRendering
andminzoomForRendering
by @farfromrefug in #272 - OSM QA Tiles Example Profile by @msbarry in #278
- feat:
--polygon
argument to constrain mbtiles to a poly shape by @farfromrefug in #280 - Cli add version argument by @ttomasz in #376
- Add shapefile directory source by @erik in #406
- Add planetiler-specific metadata by @msbarry in #428
- Add layer to shapefile by @msbarry in #429
- Add basic support for reading GeoPackage files. by @erik in #413
- Add geopackage to supported data sources by @wipfli in #442
- Support unzipping GeoPackage sources at runtime by @erik in #430
- Add
--download-max-bandwidth
option by @msbarry in #467 - Support disabling colors in terminal output by @msbarry in #501
- Expose setId in FeatureCollector [#377] by @bdon in #514
Performance Improvements
- Memory-mapped lazy reads by default by @msbarry in #300
- Lazy primitive group deserialization by @msbarry in #309
- Add --keep-unzipped option to avoid unzipping sources each time by @msbarry in #522
Bug Fixes
- fix: merge line if it needs to be simplified by @farfromrefug in #244
- Fix memoize bug by @msbarry in #279
- Fix bounds check on dense nodes by @msbarry in #301
- Improve shapefile zip reads by @msbarry in #314
- Fix NPE for
OsmReader#estimateMemoryUsageBytes()
by @erik in #389 - Move NumberFormat to thread-local variable by @erik in #387
- Fix max tile size in stats output. by @erik in #399
- Switch ordering of translation providers by @msbarry in #403
- Fix timeout issues by @msbarry in #435
- Tile hashing fix by @phanecak-maptiler in #436
- Revert "fix is convex negated (#241)" by @erik in #458
- Improve is convex logic by @msbarry in #459
- Detect certain kinds of invalid polygons when slicing into tiles by @msbarry in #499
- Fix madvise bug by @msbarry in #523
- Fix geometry errors by @msbarry in #526
- Fall back to slower transliterator by @msbarry in #543
Contributor/Documentation Improvements
- Fix flaky windows test by @msbarry in #264
- Rename README subtitle from custom map to examples by @wipfli in #267
- small readme cleanups by @bdon in #265
- Add instructions for using Planetiler with gradle by @ZeLonewolf in #274
- Generate protobuf during build by @msbarry in #310
- Publish test zip files by @msbarry in #313
- Submodule followups by @msbarry in #312
- Add
quickstart.sh
script to run with recommended settings by @msbarry in #318 - Bump version to 0.6-SNAPSHOT by @msbarry in #342
- Fix build by @msbarry in #344
- Custommap readme tweaks by @msbarry in #346
- Describe anchors and aliases in readme by @msbarry in #354
- Logging deps by @msbarry in #327
- use current repo for update-pr/dawidd6/action-download-artifact by @phanecak-maptiler in #380
- Fix linux install link by @msbarry in #395
- Remove outdated reference to "relation_member" type by @erik in #397
- Update README.md by @msbarry in #400
- Link to pyosmium-up-to-date by @wipfli in #402
- Fix typo by @wipfli in #420
- Extract ZIP file walker out of ShapefileReader. by @erik in #423
- fix CloseableConsumer class name by @bdon in #454
- Convert get all tile coords to iterator by @msbarry in #463
- Remove snakeyaml dependency by @msbarry ...
v0.5.0
What's Changed
This release improves performance on high-end machines by 20% (from 47 to 38 minutes with 64 cpus/128GB ram) by addressing several single-threaded bottlenecks, reduces mbtiles output size by 25% (from 104 to 78GB) by deduplicating repeated ocean tiles, and upgrades the basemap profile to OpenMapTiles 3.13.1.
To run this release from Java, download the attached planetiler.jar, or from docker use ghcr.io/onthegomap/planetiler:0.5.0
.
There is ongoing discussion about the project in the #planetiler channel on OSM-US slack. If you are interested in joining, get an invite here!
Performance Improvements
- Reduce synchronized work in
transportation_name
layer by @msbarry in #180 - Minimize number of temp feature chunk files by @msbarry in #199
- Add
--mmap-temp
option to use memory-mapped IO for temp feature files by @msbarry in #209 - Enable
--mmap-temp
by default by @msbarry in #236 - Parallel temp feature writes using
--write-threads
argument by @msbarry in #213 - Use custom min-heap to improve k-way merge by 30-50% by @msbarry in #217
- add compact DB mode (--compact-db) to de-duplicate mbtiles output by @bbilger in #219
- Improve DB Indexes and Fix Deferred Index Logging in Compact DB Mode by @bbilger in #245
- Use fnv1a64 for tile hash by @msbarry in #253
- Add
--feature-read-threads
option to read features in parallel when writing tiles by @msbarry in #225 - Unsynchronize ICU4j transliterate by @msbarry in #246
- Cheap isFill check and add
--skip-filled-tiles
option by @msbarry in #234 - Gather v0.5.0 benchmarks by @msbarry in #256
Improvements and Bug Fixes
- Upgrade basemap profile to OpenMapTiles v3.13.1 by @msbarry in #227
- Reduce forest/wood landcover min zoom to 7 by @msbarry in #176
- Handle elevations in units besides meters by @msbarry in #226
- Merge overlapping water polygons by @msbarry in #235
- Data type support for Expression / MultiExpression by @ZeLonewolf in #190
- Implement matchField testing for Expression.not by @ZeLonewolf in #201
- More robust multiexpression filtering by @msbarry in #206
- Change terminology around "rendering" to be consistent with MapLibre vision by @ZeLonewolf in #238
- Fix is convex negated by @msbarry in #241
- Fall back to parsing bounds from all nodes by @msbarry in #251
Contributor Improvements
- Run
spotless:check
before verify actions in CI by @msbarry in #169 - Add sonar for static analysis by @msbarry in #177
- Inline sonar token by @msbarry in #186
- Fix sonar warnings by @msbarry in #181
- Fix sonar warnings in tests by @msbarry in #191
- Fix sonar issues with unclosed resources by @msbarry in #195
- Review thread local warnings by @msbarry in #196
- Ignore generated code from sonar analysis by @msbarry in #197
- Run nested tests by @msbarry in #218
- Upload artifact during build by @msbarry in #222
- Also run java 18 in CI by @msbarry in #249
Dependencies
Once again, @dependabot has been busy keeping dependencies up-to-date.
Dependency upgrades in this release
- Bump jnr-ffi from 2.2.11 to 2.2.12 by @dependabot in #154
- Bump maven-surefire-plugin from 3.0.0-M5 to 3.0.0-M6 by @dependabot in #155
- Bump maven-surefire-plugin from 3.0.0-M5 to 3.0.0-M6 in /.github/planetiler-examples-dependabot by @dependabot in #156
- Bump maven-failsafe-plugin from 3.0.0-M5 to 3.0.0-M6 in /.github/planetiler-examples-dependabot by @dependabot in #158
- Bump maven-failsafe-plugin from 3.0.0-M5 to 3.0.0-M6 by @dependabot in #157
- Bump spotless-maven-plugin from 2.22.0 to 2.22.1 by @dependabot in #159
- Bump icu4j from 70.1 to 71.1 by @dependabot in #161
- Bump actions/upload-artifact from 2 to 3 by @dependabot in #170
- Bump geotools.version from 26.3 to 26.4 by @dependabot in #172
- Bump actions/setup-java from 2 to 3 by @dependabot in #171
- Bump actions/cache from 2 to 3 by @dependabot in #182
- Bump mockito-core from 4.4.0 to 4.5.0 by @dependabot in #183
- Bump maven-javadoc-plugin from 3.3.2 to 3.4.0 by @dependabot in #188
- Bump mockito-core from 4.5.0 to 4.5.1 by @dependabot in #189
- Bump spotless-maven-plugin from 2.22.1 to 2.22.2 by @dependabot in #194
- Bump spotless-maven-plugin from 2.22.2 to 2.22.3 by @dependabot in #198
- Bump spotless-maven-plugin from 2.22.3 to 2.22.4 by @dependabot in #210
- Bump spotless-maven-plugin from 2.22.4 to 2.22.5 by @dependabot in #221
- Bump jackson.version from 2.13.2 to 2.13.3 by @dependabot in #223
- Bump geotools.version from 26.4 to 27.0 by @dependabot in #230
- Bump mockito-core from 4.5.1 to 4.6.0 by @dependabot in #239
- Bump commonmark from 0.18.2 to 0.19.0 by @dependabot in #248
- Bump mockito-core from 4.6.0 to 4.6.1 by @dependabot in #252
New Contributors
What's Next?
After this release, we'll merge #160 which is the first step to defining custom profiles through a config file, stay tuned!
Full Changelog: v0.4.0...v0.5.0
v0.4.0
What's Changed
This release improves performance and reduces memory requirements:
- Generating a map of the world on a high-end machine (64 cpu/128GB RAM) now takes as little as 47 minutes instead of an hour
- You can now run with as little as 32GB of RAM instead of 128 using
--storage=mmap
to store temporary data in memory-mapped files
To run this release from Java, download the attached planetiler.jar, or from docker use ghcr.io/onthegomap/planetiler:0.4.0
.
There is ongoing discussion about the project in the #planetiler channel on OSM-US slack. If you are interested in joining, get an invite here!
Performance Improvements
- Custom PBF parsing by @msbarry in #82
- Memory-mapped file improvements by @msbarry in #103
- Node location cache: off-heap storage and “array” implementation that supports parallel inserts by @msbarry in #131
- Offload multipolygon storage to disk (reduce memory usage by 10-15gb) by @msbarry in #141
Improvements and Bug Fixes
- Log thread stats by @msbarry in #85
- Download automatically from quickstart by @msbarry in #93
- Warn when running with insufficient memory or disk space by @msbarry in #73
- Log direct memory usage by @msbarry in #108
- Retry wikidata requests by @msbarry in #115
- Use newer lake centerlines from @acalcutt by @msbarry in #116
- Check disk space and memory only with OSM input file by @wipfli in #138
- Read shapefiles in UTF-8 mode by @wipfli in #137
- Handle more than one centerline for a lake by @msbarry in #139
- Update README.md by @SomeoneElseOSM in #140
- Allow changing Basemap Base-Url by @boldtrn in #143
- Improve name:latin logic by @msbarry in #147
- Don't get transportation_name ref values from minor networks by @msbarry in #148
- Linting - spelling, tiny cleanups by @nyurik in #150
- 0.4.0 tweaks by @msbarry in #153
Contributor Improvements
- Support for each loop in workers by @msbarry in #83
- Add intellij file settings instructions on Linux by @wipfli in #89
- Update adoptium links by @wipfli in #94
- Remove graphhopper dependency by @msbarry in #101
- Test for --free-osm-after-read by @msbarry in #107
- Migrate to eclipse formatter to support multiple IDEs by @msbarry in #122
Dependencies
Once again, @dependabot has been busy keeping dependencies up-to-date.
Dependency upgrades in this release
- Bump geotools.version from 26.2 to 26.3 by @dependabot in #84
- Bump commonmark from 0.18.1 to 0.18.2 by @dependabot in #96
- Bump actions/setup-node from 2 to 3 by @dependabot in #95
- Bump log4j.version from 2.17.1 to 2.17.2 by @dependabot in #100
- Bump guava from 31.0.1-jre to 31.1-jre by @dependabot in #102
- Bump actions/checkout from 2 to 3 by @dependabot in #106
- Bump jackson.version from 2.13.1 to 2.13.2 by @dependabot in #121
- Bump msgpack-core from 0.9.0 to 0.9.1 by @dependabot in #123
- Bump mockito-core from 4.3.1 to 4.4.0 by @dependabot in #130
- Bump spotless-maven-plugin from 2.21.0 to 2.22.0 by @dependabot in #151
- Bump jib-maven-plugin from 3.2.0 to 3.2.1 by @dependabot in #152
New Contributors
- @wipfli made their first contribution in #89
- @SomeoneElseOSM made their first contribution in #140
- @boldtrn made their first contribution in #143
- @nyurik made their first contribution in #150
Full Changelog: v0.3.0...v0.4.0