Support offline maps and POIs

[HarelM]

Feature

The following might be a good solution to offline tiles without requiring the user to download a huge file - which sometimes fail.
https://github.com/allartk/leaflet.offline
Or this:
https://github.com/robertomlsoares/leaflet-offline

[zstadler]

If we want to support Oruxmaps's offline maps, with the known download issues, the following are starting points:

• The MBTiles leaflet plugin has implemented fetching tiles from an SQLite DB. Clearly, the DB scheme are different than Oruxmaps offline maps scheme.
• This comment describes how to use 512x512 pixel tiles, as is the case in Oruxmaps offline maps. However, this approach is not directly applicable to Oruxmaps, as its offline tiles are not necessarily aligned with the tile boundaries the the lower-level zoom.

[HarelM]

Another thing to consider here is that if we implement rotate we'll need to switch to OpenLayers according to #669 - which has a cache layer built in.
http://workshops.boundlessgeo.com/openlayers3/layers/cached.html

[HarelM]

This feature will facilitate the entry point to looking into what's needed in order to work offline properly - it should work both with the regular browser and with the mobile app the same way.
In order to facilitate offline work the first thing that we need it to better hold the current state of the app so that we initially fetch it from local storage and then hydrate it from the server.
To do that we need a good framework.
Redux seems like it is battle proof and is used with react so it seems like a good idea to use it.
It also supports undo and redo almost out of the box which is needed for #402

[HarelM]

This is the angular implementation of it:
https://github.com/angular-redux/platform/tree/master/packages/store

[HarelM]

This is an interesting answer related to vector tiles, openlayers and ionic (which is a superset of cordova)
https://gis.stackexchange.com/questions/237407/openlayers-support-for-vector-tiles-pbf-split-and-crop

[HarelM]

Looking into database solution for offline usage the following is what I came across:
http://www.diva-portal.org/smash/get/diva2:1215529/FULLTEXT01.pdf
Need to investigate IndexDB with spatial capabilities.

[HarelM]

Here's another interesting option: PouchDB - Seems like we need a solution to state management persistence for #669 anyway so it might be a good idea to take this issue into consideration.
The following page explains how this is possible, I think:
https://pouchdb.com/adapters.html
This might allow using Oruxmaps sqlite database file to be download to the device - and wrap it using Cordova and a OpenLayer custom XYZ Tile image layer:
See the following issue: openlayers/openlayers#3707
There's a lot of code, research and bugs involve in doing this, but it might work :-)

[HarelM]

Need to investigate this further due to performance issues with PouchDB and chrome.
https://malcoded.com/posts/angular-service-worker
https://angular.io/guide/service-worker-intro
https://angular.io/guide/service-worker-getting-started

[HarelM]

Performance issues were fixed using worker-pouch repository.

Here's what I think is needed in order to make OruxMaps work with IHM app:

1. Be able to query a sqlite file that is one the device, probably using this plug-in. There's an example app that can be used for initial testing on the device which should allow basic operations to the database here.
2. Reverse engineer the OruxMaps Sqlite format and understand how to get a specific tile according to a given X,Y,Z coordinates.
3. Once we can read the file there will be a need to implement a tile provider for openlayers, described somewhat here.
4. If all the above are working there will be a need to wire this through the layers sidebar navigation to allow users to download a file or use a file that is already on the device.

[HarelM]

https://www.google.com/amp/s/mobilegeo.wordpress.com/2013/06/07/mbtiles-and-openlayers/amp/
This is an interesting piece on exactly what we need, I think.

[zstadler]

As far as I understand, this example needs to be converted from raster tiles to vector tiles.

[HarelM]

Seems about right, this explains why I couldn't find where the style is being applied. I'm not sure how to use this code for vector tiles, but it will be fun to research :-)

[HarelM]

The following issue might show a way to use vector tiles with custom source, there's a codepen there that might be good:

openlayers/openlayers#7342
openlayers/openlayers#5198

[HarelM]

In order to insert a mbtiles to websql and "fool" chrome to think it is part of the site the following steps are needed:

1. write the following line in the initialization function of the app:

let db = openDatabase("tiles", 1.0, "tiles_db", 100 * 1024 * 1024, (database) => {
            console.log("DB opened!");
        });

This will create a database in chrome's application data folder, somewhere here:
C:\Users\my-user\AppData\Local\Google\Chrome\User Data\Default\databases\http_localhost_58231
Now the file can be replaced with the mbtiles file and chrome will be able to access it using the method above.

When this will be matured the following library maybe of use:
https://github.com/orbitaloop/WebSqlSync

[HarelM]

Things I would like to finish as part of this issue, other items probably should have their own issue in order to discuss them there since this feature is already too long:

1. Support offline POI images - low resolution.
2. Make sure add/update POI was not broken as part of this change.
3. Define UX for offline - i.e. what the user would see when the app knows it is offline (now supported).

[HarelM]

Remaining for this issue before closing it:

1. Make sure add/update POI was not broken as part of this change.
2. Make sure server interval update is keeping the database up-to-date (so that rebuild results are identical).
3. Add elevation and ITM grid data to POIs (should work the same for online and offline and reduce server calls).
4. Source image link should be local and not remote to present the relevant icon for link or hide links.

Other issues:

1. Allow saving to regular file when offline (currently the file is created in the server since it uploads images to imgur and does the conversion to GPX there).
2. Conversion from of public POIs' features without using the server ("+" sign) - this should be more simple.
3. Refresh map when returning from offline to online - I'm not sure this is easily possible...
4. Offline editing of public POIs and sync when going back to online.
5. Offline routing - huge effort, not sure what's the use case that justify it.
6. Offline search - large effort, not sure the use case of searching while offline justifies it.
7. Offline save as - I don't think it's needed.
8. Offline share creation - not sure how this will work, probably not needed.
9. Offline user shares/traces interaction - medium effort - not sure it is justified.

I believe these are all the abilities of the site that needs the server right now.

[HarelM]

Current implementation allows initial testing of offline maps and POIs, no further development is expected as part of this feature. Further development will be split to other issues as needed.

[zstadler]

I don't know if this is related to the refactoring, but...

The English version of the site has a Wikipedia POI here for which there is no OSM node. There is no such POI in the Hebrew version of the site. Both Wikipedia languages have an Image.

Ref:

• English Wikipedia entry
  
• Hebrew Wikipedia entry

[HarelM]

The refactoring might have affected this, but this issue is now resolved. It will be better to open another issue on this subject so this won't get lost.
There are actually two OSM elements that should be linked to the above Wikipedia entries:
Not really sure how to map fix this right:
https://www.openstreetmap.org/node/5605113504
https://www.openstreetmap.org/way/218656641
Feel free to fix the mapping first so we can see how the merge behaves with a correct mapping.