Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MSE-in-Workers: Clarify window/worker coherency/communication of track instance creation/removal and track enabled/selected changes #278

Open
wolenetz opened this issue Jun 18, 2021 · 0 comments
Labels
TPAC2024 Topic for discussion at TPAC 2024

Comments

@wolenetz
Copy link
Member

MSE-in-Workers feature issue is #175.

This issue tracks specifying how track changes are communicated across the Window and DedicatedWorkerGlobalScope instances of tracks originating from a {{SourceBuffer}} in the worker.

When MSE-in-Workers is being used:

Track creation should occur in the worker MSE API, and any state necessary to populate copies of the created track metadata in the HTMLMediaElement in the window needs to be proxied/communicated to the element. Delay should be no greater than if that communication were done by an internal, nonexposed, MessageChannel between the worker and the window.

Track removal can also occur in the worker MSE API, and such removal should be proxied similarly to the window HTMLMediaElement (and similarly, can be initiated by the element removing all tracks).

It is unclear if the selected, enabled, and showing/hidden state needs to be writable by worker API users. The initial assumption is to not disallow that (since the MSE API internally needs this to correctly add/remove tracks and schedule appropriate events on the TrackLists). Expecting that the same internal MessageChannel proxy communication concept is used, it is entirely possible that simultaneous changes to tracks' selected, enabled and hidden/showing states could be in-flight from window to worker, and from worker to window. At least a non-normative note describing this situation might be a good idea so that it is not a surprise to API users.

wolenetz added a commit to wolenetz/media-source that referenced this issue Jun 18, 2021
wolenetz added a commit to wolenetz/media-source that referenced this issue Jun 30, 2021
wolenetz added a commit to wolenetz/media-source that referenced this issue Jul 26, 2021
Specifies expansion of MSE API to usage from DedicatedWorker contexts.

This is an MSEv2 feature tracked by issue w3c#175.

See also earlier WICG explainer [1].

This is a squashed commit of 23 original commits, notably:
* Adds `closing` issue and link to w3c#276.
* Adds the `github:` respecConfig key and an issues-summary appendix.
* Removes redundant 'Repository' otherLinks config (addition of 'github'
  to respecConfig has make the manual enumeration in the 'Repository'
  otherLinks portion of respecConfig redundant.)
* Adds subsections for each of the extended HTMLMediaElement's .seekable
  and .buffered attributes.
* Adds more normative detail to the .buffered extension.
* Specifies what .seekable and .buffered do when worker has terminated,
  and references w3c#277 for further discussion.
* Describes cross-context track object aliasing. AudioTrack, VideoTrack,
  TextTrack extensions' sourceBuffer attribute must be null in the
  Window context's alias of the track if the track is an alias to a
  worker-created track from MSE-in-Worker. This is to prevent any
  assumption that cross-context object references are somehow now
  allowed in MSE or HTMLMediaElement+MSE.
* Describes a MessageChannel-based cross-context communication mechanism
  in a new section, including how MessagePorts on that channel are given
  to each side (media element, MediaSource) for communicating state
  needed by algorithms. Updates multiple algorithms and methods to use
  this mechanism where necessary. Resolves w3c#279.
* Specifies how to detect this feature using new
  `canConstructInDedicatedWorker` attribute without requiring the
  detection happening in a DedicatedWorker context.
* Modernizes to use respec's new variable syntax (`|...|` with types
  inlined)) for related lines. Wider modernization is an editorial item
  that is out of scope of this feature.
* Modernizes to use internal slots for the new cross-context
  communication mechanism's channel and ports. Wider modernization is an
  editorial item out of scope of this feature.o

Note, multiple simultaneous changes to track's `selected`, `enabled` or
`hidden`/`showing` states could be in-flight from window to worker, and
from worker to window. A non-normative note might be good to add here to
warn API users of this possibility. See w3c#278.

Note, {Audio,Video,Track}{,List} IDL need to be exposed in
DedicatedWorker. This might need to be directly in the media element
spec. See w3c#280.

Note, a reference to an MSE-in-Worker example/demo would be good in
the merged PR. Such a demo exists already at [2].
[1] https://github.com/wicg/media-source/blob/mse-in-workers-using-handle/mse-in-workers-using-handle-explainer.md
[2] https://wolenetz.github.io/mse-in-workers-demo/mse-in-workers-demo.html
wolenetz added a commit that referenced this issue Aug 31, 2021
Specifies expansion of MSE API to usage from DedicatedWorker contexts.

This is an MSEv2 feature tracked by issue #175.

See also earlier WICG explainer [1].

This is a squashed commit of 23 original commits, notably:
* Adds `closing` issue and link to #276.
* Adds the `github:` respecConfig key and an issues-summary appendix.
* Removes redundant 'Repository' otherLinks config (addition of 'github'
  to respecConfig has make the manual enumeration in the 'Repository'
  otherLinks portion of respecConfig redundant.)
* Adds subsections for each of the extended HTMLMediaElement's .seekable
  and .buffered attributes.
* Adds more normative detail to the .buffered extension.
* Specifies what .seekable and .buffered do when worker has terminated,
  and references #277 for further discussion.
* Describes cross-context track object aliasing. AudioTrack, VideoTrack,
  TextTrack extensions' sourceBuffer attribute must be null in the
  Window context's alias of the track if the track is an alias to a
  worker-created track from MSE-in-Worker. This is to prevent any
  assumption that cross-context object references are somehow now
  allowed in MSE or HTMLMediaElement+MSE.
* Describes a MessageChannel-based cross-context communication mechanism
  in a new section, including how MessagePorts on that channel are given
  to each side (media element, MediaSource) for communicating state
  needed by algorithms. Updates multiple algorithms and methods to use
  this mechanism where necessary. Resolves #279.
* Specifies how to detect this feature using new
  `canConstructInDedicatedWorker` attribute without requiring the
  detection happening in a DedicatedWorker context.
* Modernizes to use respec's new variable syntax (`|...|` with types
  inlined)) for related lines. Wider modernization is an editorial item
  that is out of scope of this feature.
* Modernizes to use internal slots for the new cross-context
  communication mechanism's channel and ports. Wider modernization is an
  editorial item out of scope of this feature.o

Note, multiple simultaneous changes to track's `selected`, `enabled` or
`hidden`/`showing` states could be in-flight from window to worker, and
from worker to window. A non-normative note might be good to add here to
warn API users of this possibility. See #278.

Note, {Audio,Video,Track}{,List} IDL need to be exposed in
DedicatedWorker. This might need to be directly in the media element
spec. See #280.

There were some still unresolved cosmetic comments on this PR during
later portion of review that I'll follow-up on in a later PR.

Note, a reference to an MSE-in-Worker example/demo would be good in
the merged PR. Such a demo exists already at [2].
[1] https://github.com/wicg/media-source/blob/mse-in-workers-using-handle/mse-in-workers-using-handle-explainer.md
[2] https://wolenetz.github.io/mse-in-workers-demo/mse-in-workers-demo.html
@chrisn chrisn added the TPAC2024 Topic for discussion at TPAC 2024 label Aug 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
TPAC2024 Topic for discussion at TPAC 2024
Projects
None yet
Development

No branches or pull requests

2 participants