CORE: Support for Offline Tilesets?

[nitrag]

I just discovered how powerful a Mapbox Tileset (Studio) can be and I'm adding it to my map via a MGLVectorSource (de-attached from a base Style layer (styleUrl) so I can retain the data source between map style changes or switch to raster map layer).

MGLTilePyramidOfflineRegion only supports styleUrl seemingly only being able to download a base Style layer. Which I guess means that if I want my Tileset available for offline use I need to bake it into my custom Mapbox Style which isn't convenient.

Shouldn't it be straightforward to support downloading a Tileset for offline?

• ETA? If far out, I can regrettably switch to baking it into my Map Style.
• Maybe give me some pointers on files to edit and I can test/submit a PR?

Platform: iOS + Android

@1ec5 @jfirebaugh

[jfirebaugh]

Hi @nitrag -- good question. Offline features are likely always going to be driven by styles, since they are the unifying top-level concept in the Mapbox GL ecosystem. I can think of a couple ways to accomplish your use case given the existing features:

• Include the source in your base style and retrieve it at runtime with -sourceWithIdentifier: rather than creating it from scratch. The source doesn't necessarily need to be used by any layers in the base style.
• Create an auxiliary style that includes just that source, and create offline regions with this style mirroring the offline regions you create for the base style. Because offline resources are shared and used opportunistically, offline tiles from this source will be used even if it's added at runtime to the base style.

Will one of those options work for you?

[nitrag]

Thanks for the fast response and glad to know I'm not totally out of luck here.

Include the source in your base style and retrieve it at runtime with -sourceWithIdentifier: rather than creating it from scratch. The source doesn't necessarily need to be used by any layers in the base style.

This sounds easiest. I didn't realize there was a getter on the style. My concern though is hitting the 6000 "resource" limit for offline download. What exactly is a resource anyway?

Create an auxiliary style that includes just that source, and create offline regions with this style mirroring the offline regions you create for the base style. Because offline resources are shared and used opportunistically, offline tiles from this source will be used even if it's added at runtime to the base style.

I understand the download part but I don't understand how to access the data. I need to provide a source layer like so:

let testSource = style.source(withIdentifier: "auxiliarystyle-tilesetname")
//Returns a source with the given identifier in the current style. 
//Source identifiers are not guaranteed to exist across styles or different versions of the same style. 

Are you saying that after I download the offline pack for the auxiliary style, I'll be able to access those source identifiers with the above command? Also what about the warning?

[1ec5]

Are you saying that after I download the offline pack for the auxiliary style, I'll be able to access those source identifiers with the above command? Also what about the warning?

You'll only deal with the auxiliary style in conjunction with MGLTilePyramidOfflineRegion, not MGLMapView. Once the offline pack is downloaded, however, you can create a new source that refers to the same tile URL templates as the downloaded offline pack and add that source to the map view. MGLMapView will automatically fetch the downloaded tiles while offline, as if you had used map view's current style with MGLTilePyramidOfflineRegion. I haven't actually tried this approach, but it's worth a shot.

[nitrag]

So I went with @jfirebaugh second option, created an auxiliary style with only my custom tilesets, and downloaded that style into a second pack. All good there.

I'm adding my tileset to the map via the following (I don't know how to add additional styleUrls to the map):

self.trailVector = MGLVectorSource(identifier: "trail-data", configurationURL: URL(string: "mapbox://userId.tilesetId")!) 

Is this what you meant @1ec5 ? Using mapbox://styles/userId/styleId URL in the vector source doesn't load anything online or offline.

It works when online.
It does not work when offline, even though I've added the tileset to my auxiliary style which was saved as a separate offline pack:

2017-02-23 00:10:32.609587 offroad2[3148:1575558] [ERROR] {}[Style]: Failed to load tile 3/1/2=>3 for source composite: The Internet connection appears to be offline.

Looking into the first option, I don't see a way in studio to add a tileset to a style without adding it as a layer too. I can try manipulating the style.json and uploading a custom style with the source attached, but I would have to do this every time I modified the style...

[1ec5]

I was describing the second approach of creating a new vector source, which seems to be what you’re attempting. 👍

I don't know how to add additional styleUrls to the map
Using mapbox://styles/userId/styleId URL in the vector source doesn't load anything online or offline.

There can only be one active style at a time. configurationURL takes the form mapbox://userId.tilesetId, not a style URL like mapbox://styles/userId/styleId.

It does not work when offline, even though I've added the tileset to my auxiliary style which was saved as a separate offline pack:

What if you force the SDK to open that pack (without displaying it in a map view), by calling -requestProgress on it?

[nitrag]

Yea, no dice. Yes, I'm using the tilesetId, thanks for confirming that I'm not crazy and styleId shouldn't be possible.

requestProgress doesn't work. Tried it in viewDidLoad() and mapViewDidFinishLoadingMap()

if let packs = MGLOfflineStorage.shared().packs {
    for pack in packs {
        guard let context = NSKeyedUnarchiver.unarchiveObject(with: pack.context) as? [String:String] else {
            print("Error retrieving offline pack context")
            return
        }
        if( context["type"] == "data") {
            print("Forcing load of pack: \(context["name"])")
            pack.requestProgress()
        }
    }
}

I guess I'll attempt option 1.

[nitrag]

Can't get option 1 to work either. App crashes because the trailVector winds up being nil
@jfirebaugh

self.trailVector = style.source(withIdentifier: "trail-approved") //tried both MGLSource and MGLVectorSource

I also have the layer, within the style called trail-approved

[nitrag]

@jfirebaugh @1ec5 next steps?

[jfirebaugh]

Turn off compositing in Studio:

[nitrag]

Aha!

style.sources.forEach({ print($0.identifier) })
//results:
composite
com.mapbox.annotations

Per your suggestion, in Studio for my Auxiliary Style (custom tileset), I tried turning off Compositing. No dice. I removed the layers and re-added. No dice. I wound up having to create a completely new/blank style, then turn off compositing before adding the layers. Bingo, that worked. When I load the map with the Auxiliary Style, I am able to see style.sources listed as the mapbox://username.tileId. Once I figured that out I switched back to my the standard style, downloaded the 2 offline packs. I still get the same results from style.sources (as above) but you are correct, it will load the resources from the Auxiliary offline pack automagically!

Can you submit a Bug Fix request to Studio team? To fix the bug which will fix:
"Turning off compositing on the fly" - Toggling on an existing map style doesn't do anything.

I'm really lucky I discovered that^, that could really trip the users up.

WOOT! WOOT! WOOT!
Cheers and thank you!! @jfirebaugh @1ec5
🍺 🍻

[1ec5]

That’s great to hear, @nitrag!

Can you submit a Bug Fix request to Studio team? To fix the bug which will fix:
"Turning off compositing on the fly" - Toggling on an existing map style doesn't do anything.

The Studio team is aware of this issue. Ultimately, however, we’re hoping to remove the auto-compositing feature in Studio once mapbox/mapbox-gl-js#2703 is implemented across all the Mapbox GL libraries. (@samanpwbb, correct me if I got any of this wrong.)

[nitrag]

@jfirebaugh I'm having a very difficult time getting in touch with sales. As I've tried to explain to them, downloading two offline packs at once effectively halves my tile ceiling to 3k. My users are those who might be off grid for 1-2 weeks at a time. Prior to finding out about this composition issue I was hoping to request double the normal tiles (12k) already but now I technically need 24k to have effectively double. While doubling the tile limit is critical, certainly getting back to the 6000 (12k) limit would support 75% of my users needs.

The size of my 2nd tileset (covering the entire USA) is only 100mb. "Enterprise Sales" says raising the tile count isn't possible unless you are on the Enterprise Plan ($499????) due to "technical resources involved in packaging up the tiles" (doesn't make sense - aren't they already packed? Isn't the resources just the download bandwidth?). I think there's a huge disconnect here, I asked for a meeting a week ago. Hoping you can help me out. I was talking to Erin Q.

[AidenQuinn]

@nitrag, increasing the tile ceiling of the Mapbox SDKs is licensed annually as an Enterprise feature - we do not have flexibility on pricing.

[jfirebaugh]

aren't they already packed? Isn't the resources just the download bandwidth?

No, tiles for offline packs are requested using the same API as normal map views, and incur the same cost of goods for Mapbox. In addition to bandwidth, this includes cloud compute time and per-request CDN and other networking costs.

[nitrag]

Really? That's a shame it's either $0 or $500 and with no flexibility. I'd like to know why the $50 was removed too. I started developing with Mapbox because you didn't seem corporate focused due to open source and scalable pricing. Guess not. I'll be posting an article on Medium about this.

[nitrag]

@jfirebaugh I have ~1000 app users and am hitting 50k web map views because my app was using non-Mapbox SDK and the raster URL. I'd like to know how many map views 50k app *users use*, few million? That pricing model has always confused me.

[nitrag]

Wait I read the pricing page wrong. I thought $499 was enterprise but it's only premium. So one would need pay more than $500/mo if you want to raise the tile limit? That's ludicrous! Even if I had 20k mobile app users, that's nowhere near the bandwidth and costs of supporting 200k+ mobile app users. Cmon guys, this just doesn't make any sense.

[nitrag]

So you aren't caught off guard (I respect you guys), here's my Medium article.
@incanus

https://goo.gl/qQ6H1j

[hjhimanshu01]

@jfirebaugh @1ec5 , does mapbox have support for offline tilesets now? also is there a way to add markers on offline maps, I'm able to store tileRegions and use layer via hosting custom styles though having a bit difficulty in adding layers on the fly or maybe manipulating layers on the map(marker layer or tileset layer), is it even possible on offline maps?