MSE-in-Workers: Clarify window/worker coherency/communication of track instance creation/removal and track enabled/selected changes #278
Labels
TPAC2024
Topic for discussion at TPAC 2024
Comments
wolenetz
added a commit
to wolenetz/media-source
that referenced
this issue
Jun 18, 2021
wolenetz
added a commit
to wolenetz/media-source
that referenced
this issue
Jun 30, 2021
wolenetz
added a commit
to wolenetz/media-source
that referenced
this issue
Jul 26, 2021
Specifies expansion of MSE API to usage from DedicatedWorker contexts. This is an MSEv2 feature tracked by issue w3c#175. See also earlier WICG explainer [1]. This is a squashed commit of 23 original commits, notably: * Adds `closing` issue and link to w3c#276. * Adds the `github:` respecConfig key and an issues-summary appendix. * Removes redundant 'Repository' otherLinks config (addition of 'github' to respecConfig has make the manual enumeration in the 'Repository' otherLinks portion of respecConfig redundant.) * Adds subsections for each of the extended HTMLMediaElement's .seekable and .buffered attributes. * Adds more normative detail to the .buffered extension. * Specifies what .seekable and .buffered do when worker has terminated, and references w3c#277 for further discussion. * Describes cross-context track object aliasing. AudioTrack, VideoTrack, TextTrack extensions' sourceBuffer attribute must be null in the Window context's alias of the track if the track is an alias to a worker-created track from MSE-in-Worker. This is to prevent any assumption that cross-context object references are somehow now allowed in MSE or HTMLMediaElement+MSE. * Describes a MessageChannel-based cross-context communication mechanism in a new section, including how MessagePorts on that channel are given to each side (media element, MediaSource) for communicating state needed by algorithms. Updates multiple algorithms and methods to use this mechanism where necessary. Resolves w3c#279. * Specifies how to detect this feature using new `canConstructInDedicatedWorker` attribute without requiring the detection happening in a DedicatedWorker context. * Modernizes to use respec's new variable syntax (`|...|` with types inlined)) for related lines. Wider modernization is an editorial item that is out of scope of this feature. * Modernizes to use internal slots for the new cross-context communication mechanism's channel and ports. Wider modernization is an editorial item out of scope of this feature.o Note, multiple simultaneous changes to track's `selected`, `enabled` or `hidden`/`showing` states could be in-flight from window to worker, and from worker to window. A non-normative note might be good to add here to warn API users of this possibility. See w3c#278. Note, {Audio,Video,Track}{,List} IDL need to be exposed in DedicatedWorker. This might need to be directly in the media element spec. See w3c#280. Note, a reference to an MSE-in-Worker example/demo would be good in the merged PR. Such a demo exists already at [2]. [1] https://github.com/wicg/media-source/blob/mse-in-workers-using-handle/mse-in-workers-using-handle-explainer.md [2] https://wolenetz.github.io/mse-in-workers-demo/mse-in-workers-demo.html
wolenetz
added a commit
that referenced
this issue
Aug 31, 2021
Specifies expansion of MSE API to usage from DedicatedWorker contexts. This is an MSEv2 feature tracked by issue #175. See also earlier WICG explainer [1]. This is a squashed commit of 23 original commits, notably: * Adds `closing` issue and link to #276. * Adds the `github:` respecConfig key and an issues-summary appendix. * Removes redundant 'Repository' otherLinks config (addition of 'github' to respecConfig has make the manual enumeration in the 'Repository' otherLinks portion of respecConfig redundant.) * Adds subsections for each of the extended HTMLMediaElement's .seekable and .buffered attributes. * Adds more normative detail to the .buffered extension. * Specifies what .seekable and .buffered do when worker has terminated, and references #277 for further discussion. * Describes cross-context track object aliasing. AudioTrack, VideoTrack, TextTrack extensions' sourceBuffer attribute must be null in the Window context's alias of the track if the track is an alias to a worker-created track from MSE-in-Worker. This is to prevent any assumption that cross-context object references are somehow now allowed in MSE or HTMLMediaElement+MSE. * Describes a MessageChannel-based cross-context communication mechanism in a new section, including how MessagePorts on that channel are given to each side (media element, MediaSource) for communicating state needed by algorithms. Updates multiple algorithms and methods to use this mechanism where necessary. Resolves #279. * Specifies how to detect this feature using new `canConstructInDedicatedWorker` attribute without requiring the detection happening in a DedicatedWorker context. * Modernizes to use respec's new variable syntax (`|...|` with types inlined)) for related lines. Wider modernization is an editorial item that is out of scope of this feature. * Modernizes to use internal slots for the new cross-context communication mechanism's channel and ports. Wider modernization is an editorial item out of scope of this feature.o Note, multiple simultaneous changes to track's `selected`, `enabled` or `hidden`/`showing` states could be in-flight from window to worker, and from worker to window. A non-normative note might be good to add here to warn API users of this possibility. See #278. Note, {Audio,Video,Track}{,List} IDL need to be exposed in DedicatedWorker. This might need to be directly in the media element spec. See #280. There were some still unresolved cosmetic comments on this PR during later portion of review that I'll follow-up on in a later PR. Note, a reference to an MSE-in-Worker example/demo would be good in the merged PR. Such a demo exists already at [2]. [1] https://github.com/wicg/media-source/blob/mse-in-workers-using-handle/mse-in-workers-using-handle-explainer.md [2] https://wolenetz.github.io/mse-in-workers-demo/mse-in-workers-demo.html
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
MSE-in-Workers feature issue is #175.
This issue tracks specifying how track changes are communicated across the
WindowandDedicatedWorkerGlobalScopeinstances of tracks originating from a {{SourceBuffer}} in the worker.When MSE-in-Workers is being used:
Track creation should occur in the worker MSE API, and any state necessary to populate copies of the created track metadata in the HTMLMediaElement in the window needs to be proxied/communicated to the element. Delay should be no greater than if that communication were done by an internal, nonexposed, MessageChannel between the worker and the window.
Track removal can also occur in the worker MSE API, and such removal should be proxied similarly to the window HTMLMediaElement (and similarly, can be initiated by the element removing all tracks).
It is unclear if the
selected,enabled, andshowing/hiddenstate needs to be writable by worker API users. The initial assumption is to not disallow that (since the MSE API internally needs this to correctly add/remove tracks and schedule appropriate events on the TrackLists). Expecting that the same internal MessageChannel proxy communication concept is used, it is entirely possible that simultaneous changes to tracks'selected,enabledandhidden/showingstates could be in-flight from window to worker, and from worker to window. At least a non-normative note describing this situation might be a good idea so that it is not a surprise to API users.The text was updated successfully, but these errors were encountered: