feat: add support for com.apple.fps.3_0 #2630

valotvince · 2020-04-07T08:38:57Z

This PR will...

Add support for Fairplay w/ MSE

Why is this Pull Request needed?

Based on a feature request @BedrockStreaming needed to capture EXT-X tags in the manifest so we could do Client-side Ad Insertion on Mac / Safari

Are there any points in the code the reviewer needs to double check?

EME Controller handles both Widevine & Fairplay correctly

Resolves issues:

Checklist

changes have been done against master branch, and PR does not conflict
new unit / functional tests have been added (whenever applicable)
API or design changes are documented in API.md

valotvince · 2020-04-07T08:42:43Z

src/controller/eme-controller.ts

@@ -344,6 +412,10 @@ class EMEController extends EventHandler {
    logger.log(`Generating key-session request for "${initDataType}" init data type`);
    keysListItem.mediaKeysSessionInitialized = true;

+    if (initDataType === 'sinf') {
+        keysListItem.mediaKeyId = this._findKeyInSinf(initData);


❓ Don't know if we should pass the whole tenc box here so any integrator could retrieve all the info (IV & encrypted info + ...) + could do their own transformation on the encrypted info from binary data

In my case that's the content keyId, so I guess I'll need to change the name because it depends on the integrator ?

is this a generic thing? getting the mediaKeyId from the tenc in order to send it in the XHR headers?

I think it depends on each DRM Provider because some would support sinf initDataType while others don't... In my use case, I need it but others might not need it at all.

Honestly, I'm getting to the point with a DRM solution for a video-engine layer like hls.js, if the best solution would be to get out of everyone's way, and simply offer a callback that a user can return a Promise from which we can use when it resolves to setMediaKeys on the video element. You can build on top of this incredibly simple design for each different DRM provider, but the core logic doesn't need to concern itself with this information.

We detect when the browser has signalled through the encrypted event, or the manifest has signalled through tags that it is encrypted, and provide the raw information to the user. They become responsible for providing the completed handshake which we can setMediaKeys with. We could still do the navigator key system check and provide this metadata, or not. I'm not convinced if we should or not, given that the priority of which key system gets used can change based on business arrangements.

I feel like we can offer built-in pure functions for the simpler DRM cases, but then there is nothing preventing a vendor from getting their custom need met.

I think this is inline with what Oren is suggesting about abstracting, but we could go even further and at the most "core" hls logic, completely abstract how a media key is resolved, and kick that out as a user-provided interface. This will then allow DRM support across the board, without blocking anyone, and we can continue to iterate on support for individual DRM vendors / systems to reduce duplication of effort, and complexity.

@robwalch @OrenMe @valotvince Thoughts on this as an approach?

valotvince · 2020-04-07T08:43:44Z

src/controller/eme-controller.ts

    const xhr = new XMLHttpRequest();
    const licenseXhrSetup = this._licenseXhrSetup;
+    const xhrSetupData : LicenseXHRAdditionalData = { keyId: keysListItem.mediaKeyId };


Should allow integrators to populate xhr headers / post info with correct data depending of their DRM providers

is keyId in header a generic FPS 3.0 requirement? If it is not then this is specific per a DRM service and should be abstracted.
Also the type LicenseXHRAdditionalData is for Fairplay but this is a method that handles all DRM systems so not sure this is fitting.
In general having a getter in every function that needs to know about the type of DRM system might get dirty as more implementation details will surface.
Might be better to rethink the whole stack - for instance:

have a generic interface to handle DRM and then write adapters for each DRM system, which should be quite easy as they all adhere to CDM interface and differ in internal non spec-ed per DRM system configs.

Use the network stack already built - xhr-loader - this will give access to same retry config and logic and performance metrics and is better suited to future work for adding fetch support.
3.maybe now it is a good time to expose a some sort of mechanism to add middleware like to request/response instead of todays xhrSetup or loaders config. Shaka are doing it and it is quite nice. Of course it will require to use same loader as mentioned in handle xhr timeout and i/o errors #2

See my answer above, i think it's more of a provider constraint than a requirement for FPS 3.0

valotvince · 2020-04-07T08:44:33Z

src/controller/eme-controller.ts

-    this._attemptKeySystemAccess(KeySystems.WIDEVINE, audioCodecs, videoCodecs);
+    console.log(data.levels);
+
+    this._attemptKeySystemAccess(KeySystems.FAIRPLAY, audioCodecs, videoCodecs);


🆘 From there, I don't really know how to select the good keySystem, based on what gives the manifest, any ideas on this ?

I think based on manifest metadata is best as you will have the key system type there.
You will have to add parser support for that.
Do you have a sample manifest to use?

I have one sample that works but I should rethink of #2606 so the encrypt information is still flagged but not loaded as done today

valotvince · 2020-04-07T08:51:26Z

src/utils/mediakeys-helper.ts

@@ -4,6 +4,7 @@
 export enum KeySystems {
  WIDEVINE = 'com.widevine.alpha',
  PLAYREADY = 'com.microsoft.playready',
+  FAIRPLAY = 'com.apple.fps.3_0'


❓ I don't really know the differences between 2_0 et 3_0 (only that from 2_0 we can use MSE)

do you have a sample that uses com.apple.fps.3_0 and that is actually supported on Safari for any given domain?

I have a working sample with our DRM provider (DRMToday), but it only works with assets having one encryption key. Assets with multiple encryption keys doesn't work yet but it seems to be a provider limitation rather than a Safari one.

valotvince · 2020-04-07T10:11:51Z

Hi @robwalch :)
I have a few questions before going on with the rest of the implem and the tests, do you mind taking a look at them whenever you have the time to ? Thanks !

src/controller/eme-controller.ts

OrenMe · 2020-04-12T04:49:41Z

src/controller/eme-controller.ts

    const xhr = new XMLHttpRequest();
    const licenseXhrSetup = this._licenseXhrSetup;
+    const xhrSetupData : LicenseXHRAdditionalData = { keyId: keysListItem.mediaKeyId };


is keyId in header a generic FPS 3.0 requirement? If it is not then this is specific per a DRM service and should be abstracted.
Also the type LicenseXHRAdditionalData is for Fairplay but this is a method that handles all DRM systems so not sure this is fitting.
In general having a getter in every function that needs to know about the type of DRM system might get dirty as more implementation details will surface.
Might be better to rethink the whole stack - for instance:

have a generic interface to handle DRM and then write adapters for each DRM system, which should be quite easy as they all adhere to CDM interface and differ in internal non spec-ed per DRM system configs.

Use the network stack already built - xhr-loader - this will give access to same retry config and logic and performance metrics and is better suited to future work for adding fetch support.
3.maybe now it is a good time to expose a some sort of mechanism to add middleware like to request/response instead of todays xhrSetup or loaders config. Shaka are doing it and it is quite nice. Of course it will require to use same loader as mentioned in handle xhr timeout and i/o errors #2

OrenMe · 2020-04-12T04:51:29Z

src/controller/eme-controller.ts

-    this._attemptKeySystemAccess(KeySystems.WIDEVINE, audioCodecs, videoCodecs);
+    console.log(data.levels);
+
+    this._attemptKeySystemAccess(KeySystems.FAIRPLAY, audioCodecs, videoCodecs);


I think based on manifest metadata is best as you will have the key system type there.
You will have to add parser support for that.
Do you have a sample manifest to use?

OrenMe · 2020-04-12T04:52:28Z

src/utils/mediakeys-helper.ts

@@ -4,6 +4,7 @@
 export enum KeySystems {
  WIDEVINE = 'com.widevine.alpha',
  PLAYREADY = 'com.microsoft.playready',
+  FAIRPLAY = 'com.apple.fps.3_0'


do you have a sample that uses com.apple.fps.3_0 and that is actually supported on Safari for any given domain?

singhcool · 2020-08-26T13:51:05Z

@valotvince Hla fair play support is completed? Can you give some demo source code?

valotvince · 2020-08-31T06:53:19Z

@singhcool Hi :)

This was complete, I was able to make it work with a single-key but not with a multi-key encrypted video... It should only be a limitation from my DRM provider.

    this.hls = new Hls({
      emeEnabled: true,
      licenseXhrSetup: async (xhr, url, { mediaKeySystemDomain }) => {
        const {
          drm: {
            token: { handler },
          },
        } = this.options;

        if (mediaKeySystemDomain === 'com.apple.fps.3_0') {
          const token = await handler();

          xhr.setRequestHeader('x-dt-auth-token', token.value);
          xhr.setRequestHeader('content-type', 'application/x-www-form-urlencoded');
        }

        return xhr;
      },
      emeGenerateLicenseChallengeFunc: ({ mediaKeySystemDomain, mediaKeyInitData }, message) => {
        if (mediaKeySystemDomain === 'com.apple.fps.3_0') {
          const spc = btoa(String.fromCharCode.apply(null, new Uint8Array(message)))
            .replace(/\+/g, '-')
            .replace(/\//g, '_');

          const keyId = binaryToHex(contentIdFromInitData(new Uint8Array(mediaKeyInitData)));

          return `spc=${spc}&keyId=${keyId}`;
        }

        return message;
      },
      fairplayCertificateUrl: '...',
      fairplayLicenseUrl: '...',
    });

Where contentIdFromInitData is a function retrieving the keyId from the schi > tenc mp4 box, to give it directly to the DRM provider (useful in my case, not necessarily in yours).

Hope that helps !

Best,

singhcool · 2020-08-31T09:36:17Z

@valotvince Thank you. If i need any support during integration, please help me. Advance Thanks ! :)

valotvince commented Apr 7, 2020

View reviewed changes

OrenMe reviewed Apr 12, 2020

View reviewed changes

valotvince force-pushed the feat-fps-3_0 branch from 8077c9e to 5f3a715 Compare April 29, 2020 15:33

itsjamie mentioned this pull request May 5, 2020

Proposal: DRM API for V1 #2700

Closed

valotvince added 6 commits August 31, 2020 09:02

feat: add support for com.apple.fps.3_0

a689ad1

add capability to pass certificate data into the options

ab11e7d

detect level key to know which keySystem to use

83a3416

let more control over license challening and xhr setup

1097b41

async xhr setup

cad60f2

give all the mediaKeyInfo

560bfb9

valotvince force-pushed the feat-fps-3_0 branch from 5f3a715 to 560bfb9 Compare August 31, 2020 07:07

robwalch added the DRM label Apr 6, 2021

valotvince closed this Dec 28, 2021

valotvince deleted the feat-fps-3_0 branch December 28, 2021 14:35

valotvince mentioned this pull request Dec 28, 2021

feat: add support for com.apple.fps keySystem #4474

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add support for com.apple.fps.3_0 #2630

feat: add support for com.apple.fps.3_0 #2630

valotvince commented Apr 7, 2020 •

edited

Loading

valotvince Apr 7, 2020

OrenMe Apr 12, 2020

valotvince Apr 15, 2020

itsjamie Apr 15, 2020 •

edited

Loading

valotvince Apr 7, 2020

OrenMe Apr 12, 2020

valotvince Apr 15, 2020

valotvince Apr 7, 2020

OrenMe Apr 12, 2020

valotvince Apr 15, 2020

valotvince Apr 7, 2020

OrenMe Apr 12, 2020

valotvince Apr 15, 2020

valotvince commented Apr 7, 2020

OrenMe Apr 12, 2020

OrenMe Apr 12, 2020

OrenMe Apr 12, 2020

singhcool commented Aug 26, 2020

valotvince commented Aug 31, 2020

singhcool commented Aug 31, 2020 •

edited

Loading

feat: add support for com.apple.fps.3_0 #2630

feat: add support for com.apple.fps.3_0 #2630

Conversation

valotvince commented Apr 7, 2020 • edited Loading

This PR will...

Why is this Pull Request needed?

Are there any points in the code the reviewer needs to double check?

Resolves issues:

Checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

itsjamie Apr 15, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

valotvince commented Apr 7, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

singhcool commented Aug 26, 2020

valotvince commented Aug 31, 2020

singhcool commented Aug 31, 2020 • edited Loading

valotvince commented Apr 7, 2020 •

edited

Loading

itsjamie Apr 15, 2020 •

edited

Loading

singhcool commented Aug 31, 2020 •

edited

Loading