Local storage

[dauwhe]

Local storage is not reliably supported in most reading systems.

What can be done to help with this? We do have some spec issues around origins. There are bugs in reading systems. What do reading systems have to say about this?

[GarthConboy]

Well, as a Reading System that doesn't support scripting, I have very little to say. It's difficult to make the case to deal with the plethora of issues around scripting, given the dearth of high-value scripted content and the prohibitive cost of creating such content.

[teytag]

I'm an EPUB3 designer. I think the "local storage" is an important support for making enhanced EPUB3. This support is also important for the development of the dynamic structure of the EPUB3 format. I want to give two examples that I designed: First, I permanently change the font colors (with local stroage support) that I read in a poetry book. The publisher finds it interesting ... The second example is the preparation of institutional training books. The last page of the e-book for each company employee has "content read performance". This information is very important for the "human resources department". It is hard to expect "Local Storage" support from every RS. But can the "read and write TXT" file be added to the EPUB3 content structure to work like LS? After that, "script samples" can be presented for designers (or EPUB Author apps) Best regards. N. Erhan Uzumcu Art Director and EPUB Designer TEYTAG Founder (Turkish Electronic Publishing Design and Research Group - Istanbul, Turkey) https://independent.academia.edu/NErhanÜzümcü http://dbookmarksblog.blogspot.com.tr http://teytagblog.blogspot.com.tr @ePub3Lib #teytag

…

On Jan 24, 2019, at 7:47 PM, Dave Cramer ***@***.***> wrote: Local storage is not reliably supported in most reading systems. What can be done to help with this? We do have some spec issues around origins. There are bugs in reading systems. What do reading systems have to say about this? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#76>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGmy-nmlQTtf8Flk1PKRWtdJvx4C2WzMks5vGeOTgaJpZM4aRRwC>.

[llemeurfr]

Readium projects are based of Web rendering engines, which support LS;
LS is shared between publications if nothing is made to avoid it: this is a security issue;
On our desktop project, care is taken to sandbox storage areas via a virtual publication origin;
This is not the case for most reading systems on the market; those which do not rely on Web rendering engines don't support LS, and most of those which support LS don't sandbox by publication;
LS is not robust: the webview can decide to delete content if it needs space.

[teytag]

Thank you for the information. N. Erhan Uzumcu Art Director and EPUB Designer

…

On Feb 5, 2019, at 6:11 PM, L. Le Meur ***@***.***> wrote: Readium projects are based of Web rendering engines, which support LS; LS is shared between publications if nothing is made to avoid it: this is a security issue; On our desktop project, care is taken to sandbox storage areas via a virtual publication origin; This is not the case for most reading systems on the market; those which do not rely on Web rendering engines don't support LS, and most of those which support LS don't sandbox by publication; LS is not robust: the webview can decide to delete content if it need space.

[JayPanoz]

Adding to Laurent’s answer:

• “LS is not robust”
  • depending on the implementation, localStorage can even turn into sessionStorage (e.g. Reading System is using a local server with a random port) so don’t count on persistence too much;
  • also, if the origin is global, one localStorage.clear() in one publication = all items are cleared, for all publications ;
  • and if you have a collection of publications making use of localStorage but don’t prefix your items, then you’re constantly overwriting them when the user switches from one to another.
• “LS is shared between publications if nothing is made to avoid it” → web has no suborigin concept (yet) so that’s a tough challenge for cloud readers;
• on top of that, Reading Systems may (re)load several or all spine items to compute the total number of reflowable pages for instance, so…
  • expect localStorage to be populated frantically if setting items on DOMContentLoaded or load;
  • don’t assume localStorage.getItem("key") to be reliable on load, it the RS does that.
• finally, platform web views handle localStorage differently i.e. opt-in vs. opt-out – on Android for instance, you must enable it while it is by default on iOS.

[llemeurfr]

In summary the real issue is not that LS support by reading systems is usually bad, it is that LS support by underlying rendering engines is suboptimal.

[JayPanoz]

Note we’ve been working on that use case under an “origin” umbrella here in order to list issues and their possible solutions.

(Our bandwidths are currently low though, so it is not necessarily evolving as fast as we’d like to considering some details are sparse.)

[danielweck]

@llemeurfr

In summary the real issue is not that LS support by reading systems is usually bad, it is that LS support by underlying rendering engines is suboptimal.

Well, indeed a typical reading system (RS) makes use of a browser engine (webview), but the RS effectively also runs a "web server", even if an IP address / HTTP(S) protocol is not explicitly used. Internally, the RS controls caching, browsing sessions, resource cleanup, etc. Crucially ; as mentioned before ; the RS is also responsible for serving web documents using an arbitrary, possibly temporary/transient (i.e. not persistent) origin. Or multiple origins, in supporting implementations.

Historically, this problem space has arguably not been addressed sufficiently (not just in terms of the EPUB spec. itself not being clear or thorough enough), resulting in significant disparities across RS solutions. Although, this has not been a major concern for the vast majority of "ebook"-like publications, which typically do not have mission-critical Javascript needs.

Perhaps the EPUB format specification cannot / should not have conformance requirements for such things. Perhaps it is too late anyway: RS vendors / implementors are only going to rewrite / upgrade their software if it makes sense financially, if there is a huge demand for an alignment with the web security model, etc. driven by the needs of consumers / content producers.

So, perhaps these kinds of issues are best articulated in "production practices / guidelines" (for content authors), and tested / validated (for implementors) through RS evaluation methods such as epub-test.org. We know that content creators struggle to find reliable, up-to-date "can-I-use?" information about web features like localStorage in EPUB (the default assumption is "probably not"). Consequently, they either fallback to the lowest common denominator (e.g. no scripting), or they resort to targeting a tiny selection of vendor-bound RS that support the rich functionality they need in their products.

In the web community there is a fairly good degree of awareness regarding graceful degradation / fallbacks vs. progressive enhancements (etc.), because of accessibility requirements, the necessity to adapt content to desktop / mobile (etc.). In the e-book space / digital publishing, there are competent EPUB "developers" too, but they do not have the luxury of a consistent and reliable runtime: i.e. the web platform (servers + clients) which follows a common set of open standards and conventions.

Would it be more effective to have a "living document" outside of the specification process, that would somehow articulate the technical gap between "the web" and a typical RS runtime? (baring in mind that there are many variants of this thing we call "RS"). Or perhaps categorized additions to epub-test.org in order to specifically check for web / origin-related features? (e.g. localStorage, indexedDB, etc.)

[RachelComerford]

This was an interesting conversation but is out of scope for the EPUB CG to resolve. If it's still of interest, it could be taken to the newly formed Publishing CG for incubation.