-
Notifications
You must be signed in to change notification settings - Fork 190
Support Full OSM Stack Toolchain
This proposal was brought about by GEOWAVE-158 The intent of this proposal is to enable a full OSM data management stack - comparable to what currently exists on top of postgis/postgresql + supporting infrastructure.
These capabilities should cover the span of reading full dumps, reading diffs, editing interface, providing diffs, and configurable simple feature transformations.
- Jan 20th, 2015
- Initial Draft
- Bulk Ingest
- Support for bulk OSM dumps in both the PBF and OSM (xml) format
- Dumps: http://wiki.openstreetmap.org/wiki/Planet.osm
- PBF Spec: http://wiki.openstreetmap.org/wiki/PBF
- OSM XML Spec: http://wiki.openstreetmap.org/wiki/OSM_XML
- Support for bulk OSM dumps in both the PBF and OSM (xml) format
- Diff Ingest
- Capable of ingesting changesets in OSM XML and PBF format (don't need to support o5m, or osmconvert one-off).
- Should be able to handle small batches or pure stream
- Ability to generate some sort of change/dirty notification for what has changed (may need "re-rendering/materializing") in a given timespan
- Node/Way/Relation persistence
- Persistence nodes, ways, relations in a format which which maintains all native OSM tags and relations.
- Ability to integrate OSM diffs into the system
- OSM diffs (in PBF or OSM XML format) should be able to be created for a given timespan
- Editing API
- Implementation of the OSM v0.6 API on top of the Node/Way/Relation persistence model
- Edits should integrate into the change/dirty notification system called out under the Diff Ingest header
- Authorization portion will be extended to support PKI based auth
- Simple Feature Generation
- A separate map-reduce (or similar - spark, etc.) job will be provided to convert the Node/Way/Relation persistence data in to OGC Simple Features in GeoWave
- The spec called out in IMPOSM3 will be supported as a means of defining the mapping
- Ref: https://github.com/omniscale/imposm3/blob/master/example-mapping.json
- The sql_filter values will instead be interpreted as CQL
- invalid cql values can be ignored (a log message should be output)
- A new set of supported CQL functions/projections will be documented
- The conversion method will be able to tap into the change notification system mentioned above to only generate updated/modified/deleted features.
- Multiple conversions runs to different formats should be fully supported
The system, when complete, should be capable of ingesting an existing OSM dump, ingesting updates, supporting an ID based map rendering (using the 0.6 API). The user should be able to user geoserver (with appropriate SLD) to render the simple feature representation, or optionally mapnik/tilemill as well (if/when the mapnik plugin - ref: https://github.com/ngageoint/geowave/issues/13 is done)
(...)
(...)
(...)