-
Notifications
You must be signed in to change notification settings - Fork 4
/
index.html
366 lines (342 loc) · 15.5 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
<!DOCTYPE html>
<html lang="en-us">
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type" />
<title>Capture all screens</title>
<script src="https://www.w3.org/Tools/respec/respec-w3c" class="remove"></script>
<script class="remove">
const respecConfig = {
group: "sccg",
specStatus: "CG-DRAFT",
github: {
repoURL: "https://github.com/screen-share/capture-all-screens/",
branch: "master",
},
editors: [
{
name: "Simon Hangl",
email: "simonha@google.com",
company: "Google",
},
{
name: "Elad Alon",
email: "eladalon@google.com",
company: "Google",
},
],
xref: ["html", "infra", "permissions", "permissions-policy", "dom", "mediacapture-streams", "webidl", "screen-capture", "csp", "trusted-types"],
shortName: "capture-all-screens",
subjectPrefix: "[capture-all-screens]",
};
</script>
</head>
<body>
<section id="abstract">
<h2>Abstract</h2>
<p>
This document defines a new API {{MediaDevices/getAllScreensMedia}}
for capturing multiple
<a data-cite="screen-capture#dfn-display-surface">monitors</a>
following one user gesture.
It is an extension to the Screen Capture API [[screen-capture]].
</p>
</section>
<section id="sotd">
<p>
This document is a draft. It is subject to major changes and, while early
experimentations are encouraged, it is therefore not intended for implementation.
</p>
</section>
<section class="informative" id="introduction">
<h2>
Introduction
</h2>
<p>
The Screen Capture API [[screen-capture]] enables the capturing of a single
<a data-cite="screen-capture#dfn-display-surface">display surface</a> in the
form of a video track.
</p>
<p>
Users currently cannot capture all
<a data-cite="screen-capture#dfn-monitor">monitors</a> attached to a device
at once without having to execute a two step sequence multiple times,
i.e. an interaction with the page through a [=transient activation=] that calls
<a data-cite="screen-capture#dom-mediadevices-getdisplaymedia">getDisplayMedia</a>
and a subsequent interaction with the media picker.
This is a cumbersome process and does not guarantee that all
<a data-cite="screen-capture#dfn-monitor">monitors</a> are captured
(which may be required by regulations).
</p>
<p>
This document describes {{MediaDevices/getAllScreensMedia}}, an extension to
the Screen Capture API [[screen-capture]].
{{MediaDevices/getAllScreensMedia}} enables
the capturing of several of the user's <a data-cite="screen-capture#dfn-monitor">monitors</a>,
or parts thereof, without a [=transient activation=].
It returns a list of media streams
(each containing a track corresponding to the captured
<a data-cite="screen-capture#dfn-monitor">monitor</a>).
As no [=transient activation=] is required, the <a href="#dfn-user-agent">user agent</a>
must ensure that appropriate protection, e.g. by allowlisting permitted
<a data-cite="url#concept-url-origin">origin</a>.
</p>
</section>
<section id="conformance">
<p>
This specification defines conformance criteria that apply to a single product: the
<dfn data-lt="user-agent">user agent</dfn> that implements the interfaces that it
contains.
</p>
<p>
Implementations that use ECMAScript [[ECMA-262]] to implement the APIs defined in this
specification must implement them in a manner consistent with the ECMAScript Bindings
defined in the Web IDL specification [[!WEBIDL]], as this specification uses that
specification and terminology.
</p>
</section>
<section>
<h2>
Example
</h2>
<p>
The following example demonstrates a request for all
<a data-cite="screen-capture#dfn-monitor">monitors</a>
using the
<code>navigator.mediaDevices.getAllScreensMedia</code>
method defined in this document.
</p>
<pre class="example highlight">
try {
const mediaStreams = await navigator.mediaDevices.getAllScreensMedia();
mediaStreams.forEach((mediaStream, index) => {
files.push(saveToFile(mediaStream));
})
} catch (e) {
console.log('Unable to acquire screen captures: ' + e);
}
</pre>
</section>
<section id="terminology">
<h2>
Terminology
</h2>
<p>
This document uses the definition of {{MediaStream}}, {{MediaStreamTrack}}
from [[!GETUSERMEDIA]], <a data-cite="url#concept-url-origin">origin</a> from [[!url]],
{{ScreenDetailed}} from [[!window-management]],
and <a data-cite="screen-capture#dfn-display-surface">monitor</a>
from [[!screen-capture]].
</p>
</section>
<section id="capture_multiple_display_surfaces">
<h2>
Capturing multiple monitors
</h2>
<p>
Capture of all <a data-cite="screen-capture#dfn-monitor">monitors</a>
is enabled through the addition of a new
{{MediaDevices/getAllScreensMedia}} method on the {{MediaDevices}}
interface.
</p>
<section>
<h2>
<dfn>MediaDevices</dfn> Additions
</h2>
<pre class="idl">
partial interface MediaDevices {
[Exposed=(Window), IsolatedContext]
Promise<sequence<MediaStream>> getAllScreensMedia();
};
</pre>
<dl data-link-for="MediaDevices" data-dfn-for="MediaDevices" class="methods">
<dt>
<dfn>getAllScreensMedia</dfn>
</dt>
<dd>
<p>When the {{MediaDevices/getAllScreensMedia()}}
method is called, the <a href="#dfn-user-agent">user agent</a> MUST run the following
steps:</p>
<ol>
<li>
<p>Let <var>p</var> be a new promise.</p>
</li>
<li>
If [=this=]'s [=relevant global object=]'s [=associated Document=] is
not [=allowed to use=] the [=policy-controlled feature=] named "all-screens-capture", [=reject=]
<var>p</var> with a new {{DOMException}} object whose {{DOMException/name}} attribute has the
value {{NotAllowedError}} and return <var>p</var>.
</li>
<li>
<p>Run the following steps in parallel:</p>
<ol>
<li>
<p>The <a href="#dfn-user-agent">user agent</a> MUST obtain permission
by checking an allowlist of <a data-cite="url#concept-url-origin">origins</a> specified
by an administrator or device owner. If the <a data-cite="url#concept-url-origin">origin</a>
is not in the allowlist defined by the administrator, [=reject=]
<var>p</var> with a new {{DOMException}} object whose {{DOMException/name}} attribute has the
value {{NotAllowedError}} and return <var>p</var>.
</p>
</li>
<li>
<p>Optionally, due to platform limitations, [=reject=]
<var>p</var> with a new {{DOMException}} object whose {{DOMException/name}} attribute has the
value {{NotAllowedError}}.
</p>
</li>
<li>
<p>
Enumerate all <a data-cite="screen-capture#dfn-monitor">monitors</a>,
resulting in a set <var>monitorsMedia</var> of media to capture.
</p>
<p>
Each provided media in <var>monitorsMedia</var> MUST include precisely one video track.
Once selected, the source of each {{MediaDevices/ScreenCaptureMediaStreamTrack}} MUST NOT change.
</p>
<p>Each provided media in <var>monitorsMedia</var> MUST not include any audio track.</p>
<p>If a hardware error
such as an OS/program/webpage lock prevents access to at least one device,
<a>reject</a> <var>p</var> with a new
{{DOMException}} object whose
{{DOMException/name}} attribute has the value
{{NotReadableError}} and abort these steps.
</p>
<p>For each device that is sourcing the selected medias in <var>monitorsMedia</var>,
using a stable and private id for the device, <var>deviceId</var>,
set [[\devicesLiveMap]]<var>[deviceId]</var> to
<code>true</code>, if it isn’t already <code>true</code>,
and set the
[[\devicesAccessibleMap]]<var>[deviceId]</var> to
<code>true</code>, if it isn’t already
<code>true</code>.
</p>
<p>If device access fails for
any reason other than those listed above, <a>reject</a>
<var>p</var> with a new {{DOMException}}
object whose {{DOMException/name}} attribute has the
value {{AbortError}} and abort these steps.
</p>
</li>
<li>
<p>Let <var>streams</var> be the
list of {{MediaStream}} objects for which the permission was granted.</p>
</li>
<li>
<p><a>Resolve</a> <var>p</var> with <var>streams</var> and
abort these steps.</p>
</li>
</ol>
</li>
<li>
<p>Return <var>p</var>.</p>
</li>
</ol>
</dd>
</dl>
</section>
<section>
<h2>ScreenCaptureMediaStreamTrack</h2>
{{MediaDevices/ScreenCaptureMediaStreamTrack}} extends {{MediaStreamTrack}} and
corresponds to a <a data-cite="screen-capture#dfn-monitor">monitor</a>.
It additionally provides a connection to the window placement API [[!window-management]]
with the {{MediaDevices/ScreenCaptureMediaStreamTrack/screenDetailed}} function with which metadata about the
captured <a data-cite="screen-capture#dfn-monitor">monitor</a> can be retrieved.
<pre class="idl">
[Exposed=Window, IsolatedContext]
interface ScreenCaptureMediaStreamTrack : MediaStreamTrack {
ScreenDetailed screenDetailed();
};
</pre>
<dl data-link-for="ScreenCaptureMediaStreamTrack" data-dfn-for="ScreenCaptureMediaStreamTrack" class="methods">
<dt>
<dfn>screenDetailed</dfn>
</dt>
The {{screenDetailed()}} MUST return a {{ScreenDetailed}} object [[!window-management]]
that corresponds to the <a data-cite="screen-capture#dfn-monitor">monitor</a> that
is captured by the {{ScreenCaptureMediaStreamTrack}} object.
</dl>
</section>
</section>
<section>
<h1 id="feature-policy-integration">Permissions Policy Integration</h1>
<p>This specification uses a [=policy-controlled feature=] identified by
the string "all-screens-capture". Its
[=policy-controlled feature/default allowlist=] is <code>"self"</code>.</p>
<div class="note">
<p>A [=document=]'s [=Document/permissions policy=] determines whether
any content in that document is allowed to use
{{MediaDevices/getAllScreensMedia}}. If disabled in any document, no content
in the document will be [=allowed to use=]
{{MediaDevices/getAllScreensMedia}}.
</p>
</div>
</section>
<section id="privacyandsecurityconsiderations">
<h2>
Privacy & Security Considerations
</h2>
<section id="privacyindicatorsrequirements">
<h2>
Privacy Considerations & Usage Rndicator Requirements
</h2>
<p>
References in this specification to [[\devicesLiveMap]], and
[[\devicesAccessibleMap]] refer to the
definitions already created to support Privacy Indicator Requirements for
{{MediaDevices/getDisplayMedia()}}.
</p>
<p>
This specification extends the <a data-cite="screen-capture#privacy-indicator-requirements">
Privacy Indicator Requirements</a> of
{{MediaDevices/getDisplayMedia()}} to include {{MediaDevices/getAllScreensMedia()}}.
In addition to these requirements, <a href="#dfn-user-agent">user agents</a> MUST
ensure that privacy indicators are visible at all times and that dismissal of the indicators
must not be persisted. The privacy indicators must inform about the <a data-cite="url#concept-url-origin">origin</a>
of the application capturing the screens. The indicators MUST clearly inform the user that the
<a data-cite="screen-capture#dfn-monitor">monitors</a> are captured.
Only the user may dismiss the privacy indicator.
The indicators MUST remain active for at least five seconds, even if the capturing is ended earlier to prevent
applications from capturing the screens without the user noticing.
</p>
<p>
The <a href="#dfn-user-agent">user agents</a> MUST provide the user with
the means to look up whether any <a data-cite="url#concept-url-origin">origin</a>
is allowed to call {{MediaDevices/getAllScreensMedia()}}.
The <a href="#dfn-user-agent">user agents</a> MUST further provide the user
with information on the implications thereof.
</p>
<p>
The <a href="#dfn-user-agent">user agents</a> MUST notify the user that
capturing may happen in the future if {{MediaDevices/getAllScreensMedia()}}
is enabled. The notification must be shown before sensitive browser content
can be exposed, e.g. on user login. A <a href="#dfn-user-agent">user agent</a>
MUST ensure that the administrator changes the allow-list while
the user is viewing senstive browser content.
</p>
</section>
<section id="securityconsiderations">
This section is non-normative.
<h3>Security Considerations</h3>
This sections discusses the major threats and mitigations.
<h4>Threat: Cross-site scripting</h4>
Attackers might use cross-site scripting to get access to sensitive information by using elevated permissions
of the allowlisted apps.
<h5>Mitigation</h5>
The API is exposed in isolated contexts (i.e. in isolated web apps) only. Isolated web apps are intended to mitigate
client-side cross-site scripting attacks by enforcing strict <a data-xref-type="http-header">Content-Security-Policy</a>
and {{TrustedType}} and server-side cross-site scripting attacks by bundling and signing of the app.
<h4>Threat: Violation of organization policies</h4>
Use of the API may violate organization policies, that control which apps should have access to sensitive information.
<h5>Mitigation</h5>
User agents must restrict the use of the API based on allowlists defined by the organization's administrator.
<h4>Threat: Third-party iframes initiating screen capture</h4>
Third party iframes might initiate screen capture.
<h5>Mitigation</h5>
The "all-screens-capture" permissions policy will control access, preventing third-party use by default.
To further safeguard from potential third-party attacks, isolated web apps employ a strict
<a data-xref-type="http-header">Content-Security-Policy</a> that makes using external resources
(i.e. the ones not originating from the Web Bundle itself) difficult and enforce cross-origin-isolation.
</section>
</section>
</body>
</html>