feat: add support for com.apple.fps keySystem #4474

valotvince · 2021-12-28T15:02:44Z

This PR will...

Add support for com.apple.streamingkeydelivery KEYFORMAT with MSE (some copy/paste from #2630)
It brings several changes that would need be separated in other PRs (but to have a MVP, I chose to make it inside one PR)

licenseXhrSetup can return a Promise (backward-compatible with before)
Pass current keySystem to licenseXhrSetup & licenseResponseCallback to allow developers to do different treatment to xhr and data depending on the used keySystem
drmSystems configuration for any supported keySytem (backward-compatible with widevineLicenseUrl
fetch and set serverCertificate when the URL is specified inside drmSystems
Specific error when session.update rejects

To make it work with my own DRMProvider, I had to write that configuration on my app, hope this helps developers when integrating fps keySystem (maybe add it somewhere it the configuration ?)

{
  licenseResponseCallback: (xhr, url, keySystem) => {
    if (keySytem === 'com.apple.fps') {
      const utf8decoder = new TextDecoder();
      const base64 = utf8decoder.decode(xhr.response);
      const decoded = window.atob(base64);
  
      const result = new Uint8Array(decoded.length);
      for (let i = 0; i < decoded.length; i++) {
        result[i] = decoded.charCodeAt(i);
      }
  
      return result;
    }
  
    return xhr.response;
  },
  licenseXhrSetup: async (xhr, url) => {
    const { token: { handler: fetchToken } = {} } = this.options.drm;
  
    const token = await fetchToken();
  
    xhr.open('POST', url, true);
    xhr.setRequestHeader('x-dt-auth-token', token.value);
  },
}

⚠️ com.apple.fps keySystem correctly works with MSE when using fMP4 (or CMAF). It won't work properly when using mp2t with MSE, directly or when using mux.js (it might be different with the hls.js transmuxer, but I didn't took the opportunity to test it). See shaka-project/shaka-player#3776 for more information.

Still to be done and I might need tips to go faster 😄 :

Why is this Pull Request needed?

Because it allows to quit Safari's streaming black-box and use MSE/EME to play content

Are there any points in the code the reviewer needs to double check?

While this PR is in draft mode please don't take care of the files listed below, I had to add them to test it on my app, for unknown reasons yarn link doesn't work properly with hls.js
- dist/*
- .gitignore
- package.json
- demo/index.html
Widevine DRM decryption should still work as expected

Resolves issues:

Checklist

changes have been done against master branch, and PR does not conflict
new unit / functional tests have been added (whenever applicable)
API or design changes are documented in API.md

robwalch · 2022-07-16T00:44:54Z

Hi @valotvince,

Would you be able to resolve the conflicts in the PR? I know it's been a while. I'll be putting together a working branch for DRM work soon, and think these changes (minus the working files (dist/*)) provide a really nice and baseline for a future release with multi-DRM support.

valotvince · 2022-07-16T08:43:47Z

Hi @robwalch
Sure will do next week :)

valotvince · 2022-07-18T09:46:39Z

src/controller/eme-controller.ts

@@ -702,7 +832,8 @@ class EMEController implements ComponentAPI {
        (videoCodec: string | undefined): videoCodec is string => !!videoCodec
      );

-    this._attemptKeySystemAccess(KeySystems.WIDEVINE, audioCodecs, videoCodecs);
+    // TBD We should try a keySystem based on manifest information
+    this._attemptKeySystemAccess(KeySystems.FAIRPLAY, audioCodecs, videoCodecs);


For that part, what would be the best way you want it handled ?

Try all the keySystems and take the first one that works ?

We could add a config option called keySystemPreference to specify which key-system is preferred (see below).

If not set there, we could map each key in _drmSystems (+_widevineLicenseUrl) to a key-system. If there is only one, we can assume that's the one we need to access. If more than one, then we could check them all, at this point there should only be one that works and that would be the one we want.

I think it would make more sense to not set this.mediaKeysPromise until onMediaEncrypted. At that point the playlist and media are loaded, and from that we could reduce how many calls to requestMediaKeySystemAccess we need to make, especially if the playlist keys tell us what decrypt method and keysystems can be used.

Let me know your thoughts. I'm making some assumptions looking at the code, but have a limited set of sample streams and license servers to work with at the moment.

### `keySystemPreference` (default: `void 0`) The value used to select the right key system. The playlist may signal multiple key systems, selection of the key system is done based on the preference specified here. Possible values, - `fairplaystreaming`: com.apple.streamingkeydelivery - `playready`: com.microsoft.playready - `widevine`: urn:uuid:edef8ba9-79d6-4ace-a3c8-27dcd51d21ed - `clearkey`: org.w3.clearkey - `undefined`: Select the key system from the first compatible EXT-X-KEY tag

Note that for backwards compatibility I cannot merge this change as-is, as it breaks WIDEVINE support. If you can address this, perhaps checking access of both types but using the first that passes then I can take it from there.

Hi @valotvince,

The test branch I posted solves key-system access by selecting the key-system based on key formats found in the playlist or the systems present for the configured license URLs. Access is no longer attempted on ATTACH_MEDIA. It happens when the first key(s) for a segment are loaded. This is when available key formats are known which is mapped to key-systems to request access to. This happens later in the process of loading an HLS stream, so there could be a desire to request access earlier even if it means querying systems the stream does not have keys for. Session keys will be the first option if present in the manifest (#4927). The queries could be done on ATTACH_MEDIA based on licenses in the config, but I prefer there to avoid requesting access to key-systems for HLS streams that are not encrypted, even when emeEnabled is true and license urls are defined.

valotvince · 2022-07-18T09:48:16Z

src/loader/m3u8-parser.ts

@@ -367,8 +367,10 @@ export default class M3U8Parser {
            const decryptkeyformat =
              keyAttrs.enumeratedString('KEYFORMAT') ?? 'identity';

+            // TBD we might need to check if we try to MSE playback mp2t content


Do we need to throw an handled error if someone tries to read MP2T Fairplay-encrypted content ? Or do we only rely on documentation ?

If there's a need for an error to be thrown, I might need some guidance to see where I should do it :)

No need. We can rely on documented supported features in the README.

I want us to avoid any kind of file type detection as that has tended to be problematic.

valotvince · 2022-07-18T13:51:55Z

👋 @robwalch The branch is rebased, still two things left to do but I'll wait for your input before going further

robwalch · 2022-08-01T21:36:52Z

src/loader/m3u8-parser.ts

@@ -367,8 +367,10 @@ export default class M3U8Parser {
            const decryptkeyformat =
              keyAttrs.enumeratedString('KEYFORMAT') ?? 'identity';

+            // TBD we might need to check if we try to MSE playback mp2t content


No need. We can rely on documented supported features in the README.

I want us to avoid any kind of file type detection as that has tended to be problematic.

robwalch · 2022-08-01T23:41:58Z

src/controller/eme-controller.ts

+    if (keySystem === KeySystems.WIDEVINE && this._widevineLicenseUrl) {
+      return this._widevineLicenseUrl;


_widevineLicenseUrl can be replaced with _config.widevineLicenseUrl.
(remove private _widevineLicenseUrl?: string)

robwalch · 2022-08-02T01:12:18Z

src/controller/eme-controller.ts

@@ -702,7 +832,8 @@ class EMEController implements ComponentAPI {
        (videoCodec: string | undefined): videoCodec is string => !!videoCodec
      );

-    this._attemptKeySystemAccess(KeySystems.WIDEVINE, audioCodecs, videoCodecs);
+    // TBD We should try a keySystem based on manifest information
+    this._attemptKeySystemAccess(KeySystems.FAIRPLAY, audioCodecs, videoCodecs);


We could add a config option called keySystemPreference to specify which key-system is preferred (see below).

If not set there, we could map each key in _drmSystems (+_widevineLicenseUrl) to a key-system. If there is only one, we can assume that's the one we need to access. If more than one, then we could check them all, at this point there should only be one that works and that would be the one we want.

I think it would make more sense to not set this.mediaKeysPromise until onMediaEncrypted. At that point the playlist and media are loaded, and from that we could reduce how many calls to requestMediaKeySystemAccess we need to make, especially if the playlist keys tell us what decrypt method and keysystems can be used.

Let me know your thoughts. I'm making some assumptions looking at the code, but have a limited set of sample streams and license servers to work with at the moment.

### `keySystemPreference` (default: `void 0`) The value used to select the right key system. The playlist may signal multiple key systems, selection of the key system is done based on the preference specified here. Possible values, - `fairplaystreaming`: com.apple.streamingkeydelivery - `playready`: com.microsoft.playready - `widevine`: urn:uuid:edef8ba9-79d6-4ace-a3c8-27dcd51d21ed - `clearkey`: org.w3.clearkey - `undefined`: Select the key system from the first compatible EXT-X-KEY tag

robwalch · 2022-08-02T01:15:04Z

src/controller/eme-controller.ts

@@ -702,7 +832,8 @@ class EMEController implements ComponentAPI {
        (videoCodec: string | undefined): videoCodec is string => !!videoCodec
      );

-    this._attemptKeySystemAccess(KeySystems.WIDEVINE, audioCodecs, videoCodecs);
+    // TBD We should try a keySystem based on manifest information
+    this._attemptKeySystemAccess(KeySystems.FAIRPLAY, audioCodecs, videoCodecs);


Note that for backwards compatibility I cannot merge this change as-is, as it breaks WIDEVINE support. If you can address this, perhaps checking access of both types but using the first that passes then I can take it from there.

robwalch added the DRM label Jan 3, 2022

sturlath mentioned this pull request Jan 20, 2022

Video Player Megabit/Blazorise#3298

Merged

robwalch added this to DRM (needs prioritization) in Release Planning and Backlog Mar 12, 2022

robwalch added the Stale label Jul 16, 2022

stale bot removed the Stale label Jul 16, 2022

feat: add support for com.apple.fps keySystem

3595921

valotvince force-pushed the feat-apple-fps branch from 3b5667f to 3595921 Compare July 18, 2022 09:27

revert hls.js.api.md

acebb54

valotvince commented Jul 18, 2022

View reviewed changes

valotvince added 5 commits July 18, 2022 11:58

api extract report

92fd095

try/catch around setting serverCertificate

171479a

fine error handling of server certificate

fe1bfcb

add some unit tests

7c3f695

add another test

1df3e59

update api report

20b72f6

robwalch reviewed Aug 2, 2022

View reviewed changes

robwalch added this to the 1.3.0 milestone Aug 27, 2022

Merge branch 'master' into feat-apple-fps

eb79ea2

robwalch changed the base branch from master to feature/drm-key-systems-with-fps August 31, 2022 16:07

robwalch marked this pull request as ready for review September 2, 2022 21:46

robwalch merged commit 6b484a8 into video-dev:feature/drm-key-systems-with-fps Sep 2, 2022

valotvince deleted the feat-apple-fps branch September 5, 2022 08:16

robwalch mentioned this pull request Sep 28, 2022

Multi-DRM Support #4930

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add support for com.apple.fps keySystem #4474

feat: add support for com.apple.fps keySystem #4474

valotvince commented Dec 28, 2021 •

edited

robwalch commented Jul 16, 2022 •

edited

valotvince commented Jul 16, 2022

valotvince Jul 18, 2022

robwalch Aug 2, 2022 •

edited

robwalch Aug 2, 2022

robwalch Sep 27, 2022 •

edited

valotvince Jul 18, 2022

robwalch Aug 1, 2022

valotvince commented Jul 18, 2022

robwalch Aug 1, 2022

robwalch Aug 1, 2022

robwalch Aug 2, 2022 •

edited

robwalch Aug 2, 2022

		if (keySystem === KeySystems.WIDEVINE && this._widevineLicenseUrl) {
		return this._widevineLicenseUrl;

feat: add support for com.apple.fps keySystem #4474

feat: add support for com.apple.fps keySystem #4474

Conversation

valotvince commented Dec 28, 2021 • edited

This PR will...

Why is this Pull Request needed?

Are there any points in the code the reviewer needs to double check?

Resolves issues:

Checklist

robwalch commented Jul 16, 2022 • edited

valotvince commented Jul 16, 2022

valotvince Jul 18, 2022

Choose a reason for hiding this comment

robwalch Aug 2, 2022 • edited

Choose a reason for hiding this comment

robwalch Aug 2, 2022

Choose a reason for hiding this comment

robwalch Sep 27, 2022 • edited

Choose a reason for hiding this comment

valotvince Jul 18, 2022

Choose a reason for hiding this comment

robwalch Aug 1, 2022

Choose a reason for hiding this comment

valotvince commented Jul 18, 2022

robwalch Aug 1, 2022

Choose a reason for hiding this comment

robwalch Aug 1, 2022

Choose a reason for hiding this comment

robwalch Aug 2, 2022 • edited

Choose a reason for hiding this comment

robwalch Aug 2, 2022

Choose a reason for hiding this comment

valotvince commented Dec 28, 2021 •

edited

robwalch commented Jul 16, 2022 •

edited

robwalch Aug 2, 2022 •

edited

robwalch Sep 27, 2022 •

edited

robwalch Aug 2, 2022 •

edited