Skip to content

Implementation notes: EPUB3 Media Overlays, Readium WebPub Manifest and W3C "Sync Narration"

Jump to bottom
Daniel Weck edited this page Jun 1, 2021 · 5 revisions

Step-by-step instructions to easily convert EPUB3 Media Overlays to RWPM “sync narration” (the transformation generates a static fileset, so can be open with a text editor):

manifest.json (truncated):

{
  "@context": "https://readium.org/webpub-manifest/context.jsonld",
  "metadata": {
    "@type": "http://schema.org/Book",
    "title": "Moby-Dick",
    ...
    "narrator": "Stuart Wills",
    "duration": 1403.5,
    "media-overlay": {
      "active-class": "-epub-media-overlay-active"
    },
    "media:duration": "0:09:03.000",
    ...
  },
  "readingOrder": [
    {
      "type": "application/xhtml+xml",
      "href": "OPS/titlepage.xhtml"
    },
    ...
    {
      "type": "application/xhtml+xml",
      "alternate": [
        {
          "type": "application/vnd.syncnarr+json",
          "duration": 860.5,
          "href": "media-overlays_0.json"
        }
      ],
      "href": "OPS/chapter_001.xhtml"
    },
    {
      "type": "application/xhtml+xml",
      "alternate": [
        {
          "type": "application/vnd.syncnarr+json",
          "duration": 543,
          "href": "media-overlays_1.json"
        }
      ],
      "href": "OPS/chapter_002.xhtml"
    },
    {
      "type": "application/xhtml+xml",
      "href": "OPS/chapter_003.xhtml"
    },
    ...
  ],
  "resources": [
    ...
  ],
  "toc": [
    ...
  ],
  "landmarks": [
    ...
  ]
}

media-overlays_0.json (usually, one SMIL per spine item / reading order XHTML document):

{
  "role": "section",
  "narration": [
    {
      "text": "OPS/chapter_001.xhtml",
      "role": [
        "section",
        "bodymatter",
        "chapter"
      ],
      "narration": [
        {
          "text": "OPS/chapter_001.xhtml#c01h01",
          "audio": "OPS/audio/mobydick_001_002_melville.mp4#t=24.5,29.268"
        },
        {
          "text": "OPS/chapter_001.xhtml#c01w00001",
          "audio": "OPS/audio/mobydick_001_002_melville.mp4#t=29.268,29.441"
        },
        ...
      ]
    }
  ]
}

The media-overlays_xxx.json files are generated statically by the r2-shared-js CLI, but they are parsed dynamically from SMIL (lazy loading) in Thorium. So for example instead of "href": "media-overlays_1.json", the URL is "href": "media-overlay.json?resource=OPS%2Fchapter_001.xhtml", where media-overlay.json is a route in the Express server of r2-streamer-js, and resource is a recognised URL query param with a value that maps to a XHTML publication resource (e.g. OPS/chapter_001.xhtml)

You can easily test this using the r2-streamer-js CLI:

Open web browser at https://127.0.0.1:3000 (or whatever the console shows) Click on the moby-dick-mo.epub link, and then click on the ./manifest.json/show/all link to visualise the RWPM.

You will notice with both r2-shared-js and r2-streamer-js CLI utilities that in addition to RWPM link alternate, there is also a properties field in the data structure. This is the legacy mechanism, only generated for backwards-compatiblity, not actually used anymore (because alternate takes precedence) Example:

    {
      "type": "application/xhtml+xml",
      "properties": {
        "media-overlay": "media-overlays_1.json"
      },
      "duration": 543,
      "alternate": [
        {
          "type": "application/vnd.syncnarr+json",
          "duration": 543,
          "href": "media-overlays_1.json"
        }
      ],
      "href": "OPS/chapter_002.xhtml"
    },

You will also notice the redundant duration property, on both the “root” RWPM link in the readingOrder (aka spine), and also in the alternate link. If I remember correctly, the order of precedence is also alternate first, and the “root” link duration is mostly here for backwards-compatibility (based on the old RWPM data model which used properties on the “root” link, before we migrated to a JSON syntax that aligns more with the W3C Sync Media / Narration draft, using link alternate).

Finally, note the RWPM metadata:

    "media-overlay": {
      "active-class": "-epub-media-overlay-active"
    },

This is still based on the original RWPM proposal, where a media-overlays object encapsulates active-class and playback-active-class properties (if they exist in the original EPUB3, otherwise this is empty, of course)

Clone this wiki locally