-
Notifications
You must be signed in to change notification settings - Fork 22.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add new screen capture constraints and options #22733
Changes from 2 commits
7c129ee
79fb868
41f5a55
cb5fef8
1d9639c
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,63 @@ | ||
--- | ||
title: CaptureController() | ||
slug: Web/API/CaptureController/CaptureController | ||
page-type: web-api-constructor | ||
tags: | ||
- API | ||
- Constructor | ||
- Experimental | ||
- Reference | ||
browser-compat: api.CaptureController.setFocusBehavior | ||
--- | ||
|
||
{{APIRef("Screen Capture API")}}{{seecompattable}} | ||
|
||
The {{domxref("CaptureController")}} constructor creates a new `CaptureController` object instance. | ||
|
||
## Syntax | ||
|
||
```js-nolint | ||
CaptureController() | ||
``` | ||
|
||
### Parameters | ||
|
||
None. | ||
|
||
## Examples | ||
|
||
```js | ||
// Create a new CaptureController instance | ||
const controller = new CaptureController(); | ||
|
||
// Prompt the user to share a tab, window, or screen. | ||
const stream = | ||
await navigator.mediaDevices.getDisplayMedia({ controller }); | ||
|
||
// Query the displaySurface value of the captured video track | ||
const [track] = stream.getVideoTracks(); | ||
const displaySurface = track.getSettings().displaySurface; | ||
|
||
if (displaySurface == 'browser') { | ||
// Focus the captured tab. | ||
controller.setFocusBehavior('focus-captured-surface'); | ||
} else if (displaySurface == 'window') { | ||
// Do not move focus to the captured window. | ||
// Keep the capturing page focused. | ||
controller.setFocusBehavior('no-focus-change'); | ||
} | ||
``` | ||
|
||
## Specifications | ||
|
||
{{Specifications}} | ||
|
||
## Browser compatibility | ||
|
||
{{Compat}} | ||
|
||
## See also | ||
|
||
- [Screen Capture API](/en-US/docs/Web/API/Screen_Capture_API) | ||
- {{domxref("MediaDevices.getDisplayMedia()")}} | ||
- [Better screen sharing with Conditional Focus](https://developer.chrome.com/docs/web-platform/conditional-focus/) |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,66 @@ | ||
--- | ||
title: CaptureController | ||
slug: Web/API/CaptureController | ||
page-type: web-api-interface | ||
tags: | ||
- API | ||
- Interface | ||
- CaptureController | ||
- Reference | ||
- Experimental | ||
browser-compat: api.CaptureController | ||
--- | ||
|
||
{{APIRef("Screen Capture API")}}{{seecompattable}} | ||
|
||
The **`CaptureController`** interface provides methods that can be used to further manipulate a capture session separate from its initiation via {{domxref("MediaDevices.getDisplayMedia()")}}. | ||
|
||
A `CaptureController` object is associated with a capture session by passing it into a {{domxref("MediaDevices.getDisplayMedia", "getDisplayMedia()")}} call as the value of the options object's `controller` property. | ||
|
||
## Constructor | ||
|
||
- {{ domxref("CaptureController.CaptureController", "CaptureController()") }} | ||
- : Creates a new `CaptureController` object instance. | ||
|
||
## Instance methods | ||
|
||
- {{ domxref("CaptureController.setFocusBehavior", "setFocusBehavior()") }} | ||
- : Controls whether the captured tab or window will be focused or whether the focus will remain with the tab containing the capturing app. | ||
|
||
## Examples | ||
|
||
```js | ||
// Create a new CaptureController instance | ||
const controller = new CaptureController(); | ||
|
||
// Prompt the user to share a tab, window, or screen. | ||
const stream = | ||
await navigator.mediaDevices.getDisplayMedia({ controller }); | ||
|
||
// Query the displaySurface value of the captured video track | ||
const [track] = stream.getVideoTracks(); | ||
const displaySurface = track.getSettings().displaySurface; | ||
|
||
if (displaySurface == 'browser') { | ||
// Focus the captured tab. | ||
controller.setFocusBehavior('focus-captured-surface'); | ||
} else if (displaySurface == 'window') { | ||
// Do not move focus to the captured window. | ||
// Keep the capturing page focused. | ||
controller.setFocusBehavior('no-focus-change'); | ||
} | ||
``` | ||
|
||
## Specifications | ||
|
||
{{Specifications}} | ||
|
||
## Browser compatibility | ||
|
||
{{Compat}} | ||
|
||
## See also | ||
|
||
- [Screen Capture API](/en-US/docs/Web/API/Screen_Capture_API) | ||
- {{domxref("MediaDevices.getDisplayMedia()")}} | ||
- [Better screen sharing with Conditional Focus](https://developer.chrome.com/docs/web-platform/conditional-focus/) |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,79 @@ | ||
--- | ||
title: CaptureController.setFocusBehavior() | ||
slug: Web/API/CaptureController/setFocusBehavior | ||
page-type: web-api-instance-method | ||
tags: | ||
- API | ||
- Experimental | ||
- Method | ||
- Reference | ||
- setFocusBehavior | ||
browser-compat: api.CaptureController.setFocusBehavior | ||
--- | ||
|
||
{{APIRef("Screen Capture API")}}{{seecompattable}} | ||
|
||
The {{domxref("CaptureController")}} interface's **`setFocusBehavior()`** method controls whether the captured tab or window will be focused when an associated {{domxref("MediaDevices.getDisplayMedia()")}} {{jsxref("Promise")}} fulfills, or whether the focus will remain with the tab containing the capturing app. | ||
|
||
You can set this behavior multiple times before the {{domxref("MediaDevices.getDisplayMedia()")}} call, or once immediately after its `Promise` resolves. After that, the focus behavior is said to be finalized, and can't be changed. | ||
|
||
## Syntax | ||
|
||
```js-nolint | ||
setFocusBehavior(focusBehavior) | ||
``` | ||
|
||
### Parameters | ||
|
||
- `focusBehavior` | ||
- : An enumerated value that describes whether the user agent should transfer focus to the captured display surface, or keep the capturing app focused. Possible values are `focus-captured-surface` (transfer focus) and `no-focus-change` (keep focus on the capturing app). | ||
|
||
### Return value | ||
|
||
`Undefined`. | ||
|
||
### Exceptions | ||
|
||
- `InvalidStateError` {{domxref("DOMException")}} | ||
- : Thrown if: | ||
- The capture stream has been stopped. | ||
- The user chose to share a screen ({{domxref("MediaTrackSettings.displaySurface", "displaySurface")}} type `monitor`) rather than a `browser` tab or `window` — you can't focus a monitor. In this case the exception is thrown after the {{domxref("MediaDevices.getDisplayMedia()")}} `Promise` resolves. | ||
- Enough time has elapsed after the {{domxref("MediaDevices.getDisplayMedia()")}} `Promise` fulfills that the focus behavior has been finalized. | ||
|
||
## Examples | ||
|
||
```js | ||
// Create a new CaptureController instance | ||
const controller = new CaptureController(); | ||
|
||
// Prompt the user to share a tab, window, or screen. | ||
const stream = | ||
await navigator.mediaDevices.getDisplayMedia({ controller }); | ||
|
||
// Query the displaySurface value of the captured video track | ||
const [track] = stream.getVideoTracks(); | ||
const displaySurface = track.getSettings().displaySurface; | ||
|
||
if (displaySurface == 'browser') { | ||
// Focus the captured tab. | ||
controller.setFocusBehavior('focus-captured-surface'); | ||
} else if (displaySurface == 'window') { | ||
// Do not move focus to the captured window. | ||
// Keep the capturing page focused. | ||
controller.setFocusBehavior('no-focus-change'); | ||
} | ||
``` | ||
|
||
## Specifications | ||
|
||
{{Specifications}} | ||
|
||
## Browser compatibility | ||
|
||
{{Compat}} | ||
|
||
## See also | ||
|
||
- [Screen Capture API](/en-US/docs/Web/API/Screen_Capture_API) | ||
- {{domxref("MediaDevices.getDisplayMedia()")}} | ||
- [Better screen sharing with Conditional Focus](https://developer.chrome.com/docs/web-platform/conditional-focus/) |
Original file line number | Diff line number | Diff line change | ||||||
---|---|---|---|---|---|---|---|---|
|
@@ -22,10 +22,8 @@ browser-compat: api.MediaDevices.getDisplayMedia | |||||||
|
||||||||
{{DefaultAPISidebar("Screen Capture API")}} | ||||||||
|
||||||||
The {{domxref("MediaDevices")}} interface's | ||||||||
**`getDisplayMedia()`** method prompts the user to select and | ||||||||
grant permission to capture the contents of a display or portion thereof (such as a | ||||||||
window) as a {{domxref("MediaStream")}}. | ||||||||
The {{domxref("MediaDevices")}} interface's **`getDisplayMedia()`** method prompts the user to select and | ||||||||
grant permission to capture the contents of a display or portion thereof (such as a window) as a {{domxref("MediaStream")}}. | ||||||||
|
||||||||
The resulting stream can then be | ||||||||
recorded using the [MediaStream Recording API](/en-US/docs/Web/API/MediaStream_Recording_API) or transmitted as part of a [WebRTC](/en-US/docs/Web/API/WebRTC_API) session. | ||||||||
|
@@ -35,28 +33,37 @@ See [Using the Screen Capture API](/en-US/docs/Web/API/Screen_Capture_API/Using_ | |||||||
## Syntax | ||||||||
|
||||||||
```js-nolint | ||||||||
getDisplayMedia(constraints) | ||||||||
getDisplayMedia(options) | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
I guess, since this is optional? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Makes sense; added in next commit |
||||||||
``` | ||||||||
|
||||||||
### Parameters | ||||||||
|
||||||||
- `constraints` {{optional_inline}} | ||||||||
- : An optional object specifying requirements for | ||||||||
the returned {{domxref("MediaStream")}}. Since `getDisplayMedia()` requires | ||||||||
a video track, the returned stream will have one even if no video track is expressly | ||||||||
requested by the `constraints` object. For more details, see the [constraints](/en-US/docs/Web/API/MediaDevices/getUserMedia#parameters) | ||||||||
section under the {{domxref("MediaDevices.getUserMedia()")}} method, as well | ||||||||
as the article [Capabilities, constraints, and settings](/en-US/docs/Web/API/Media_Capture_and_Streams_API/Constraints). | ||||||||
- `options` {{optional_inline}} | ||||||||
- : An optional object specifying requirements for the returned {{domxref("MediaStream")}}. The options for `getDisplayMedia()` work in the same as the [constraints](/en-US/docs/Web/API/MediaDevices/getUserMedia#parameters) for the {{domxref("MediaDevices.getUserMedia()")}} method, although in that case only `audio` and `video` can be specified. The list of possible option properties for `getDisplayMedia()` is as follows: | ||||||||
Comment on lines
+42
to
+43
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I tried to fix this for ages when I was originally writing this, and gave up ;-) I just had another look, and realized that it was because the note following the |
||||||||
- `video` | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Presumably also optional? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. fixed, ta. |
||||||||
- : A boolean; a value of `true` indicates that the returned {{domxref("MediaStream")}} will contain a video track. Be aware that, since `getDisplayMedia()` requires a video track, the returned stream will have one even if no video track is expressly requested. If the `video` property is explicitly given a value of `false`, the returned {{jsxref("Promise")}} will reject with a `TypeError`. | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The IDL has this and This seems like a puzzling option. It's boolean, but if I set it to
Which makes me wonder why we would ever pass it. Am I confused here? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Changes made. as requested. Yup, it is quite confusing. I don't have a good answer for your question. |
||||||||
- `audio` {{optional_inline}} | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think for all optional options we should say what the default is. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done; I think I've worked these out correctly... |
||||||||
- : A boolean; a value of `true` indicates that the returned {{domxref("MediaStream")}} will contain an audio track, if audio is supported and available for the display surface chosen by the user. | ||||||||
- `controller` {{optional_inline}} | ||||||||
- : A {{domxref("CaptureController")}} object instance containing methods that can be used to further manipulate the capture session. | ||||||||
- `preferCurrentTab` {{non-standard_inline}} {{optional_inline}} | ||||||||
- : A boolean; a value of `true` instructs the browser to offer the current tab as the most prominent capture source, i.e. as a separate "This Tab" option in the "Choose what to share" options presented to the user. This is useful as many app types generally just want to share the current tab. For example, a slide deck app might want to let the user stream the current tab containing the presentation to a virtual conference. | ||||||||
- `selfBrowserSurface` {{optional_inline}} | ||||||||
- : An enumerated value specifying whether the browser should allow the user to select the current tab for capture. This helps to avoid the "infinite hall of mirrors" effect experienced when a video conferencing app inadvertently shares its own display. Possible values are `include`, which hints that the browser should include the current tab in the choices offered for capture, and `exclude`, which hints that it should be excluded. | ||||||||
- `surfaceSwitching` {{optional_inline}} | ||||||||
- : An enumerated value specifying whether the browser should display a control to allow the user to dynamically switch the shared tab during screen-sharing. This is much more convenient than having to go through the whole sharing process again each time a user wants to switch the shared tab. Possible values are `include`, which hints that the browser should include the control, and `exclude`, which hints that it should not be shown. | ||||||||
- `systemAudio` {{optional_inline}} | ||||||||
- : An enumerated value specifying whether the browser should include the system audio among the possible audio sources offered to the user. Possible values are `include`, which hints that the browser should include the system audio in the list of choices, and `exclude`, which hints that it should be excluded. | ||||||||
|
||||||||
> **Note:** See the article [Capabilities, constraints, and settings](/en-US/docs/Web/API/Media_Capture_and_Streams_API/Constraints) for a lot more detail on how these options work. | ||||||||
|
||||||||
### Return value | ||||||||
|
||||||||
A {{jsxref("Promise")}} that resolves to a {{domxref("MediaStream")}} containing a | ||||||||
video track whose contents come from a user-selected screen area, as well as an optional | ||||||||
audio track. | ||||||||
|
||||||||
> **Note:** Browser support for audio tracks varies, both in terms of | ||||||||
> whether or not they're supported at all by the media recorder and in terms of the | ||||||||
> audio sources supported. Check the [compatibility table](#browser_compatibility) for details for each browser. | ||||||||
> **Note:** Browser support for audio tracks varies, both in terms of whether or not they're supported at all by the media recorder and in terms of the audio sources supported. Check the [compatibility table](#browser_compatibility) for details for each browser. | ||||||||
|
||||||||
### Exceptions | ||||||||
|
||||||||
|
@@ -73,17 +80,15 @@ audio track. | |||||||
- `NotFoundError` {{domxref("DOMException")}} | ||||||||
- : Returned if no sources of screen video are available for capture. | ||||||||
- `NotReadableError` {{domxref("DOMException")}} | ||||||||
- : Returned if the user selected a screen, window, tab, or other source of screen data, but a | ||||||||
- : Returned if the user selected a screen, window, tab, or another source of screen data, but a | ||||||||
hardware or operating system level error or lockout occurred, preventing the sharing | ||||||||
of the selected source. | ||||||||
- `OverconstrainedError` {{domxref("DOMException")}} | ||||||||
- : Returned if, after creating the stream, applying the specified `constraints` fails | ||||||||
- : Returned if, after creating the stream, applying any specified constraints fails | ||||||||
because no compatible stream could be generated. | ||||||||
- {{jsxref("TypeError")}} | ||||||||
- : Returned if the specified `constraints` include constraints which are not permitted | ||||||||
when calling `getDisplayMedia()`. These unsupported constraints are | ||||||||
`advanced` and any constraints which in turn have a member named | ||||||||
`min` or `exact`. | ||||||||
- : Returned if the specified `options` include values that are not permitted | ||||||||
when calling `getDisplayMedia()`, for example a `video` property set to false, or if any specified constraints are not permitted. | ||||||||
|
||||||||
## Security | ||||||||
|
||||||||
|
@@ -92,9 +97,9 @@ source of significant privacy and security concerns. For that reason, the specif | |||||||
details measures browsers are required to take in order to fully support | ||||||||
`getDisplayMedia()`. | ||||||||
|
||||||||
- The specified `constraints` can't be used to limit the options available | ||||||||
- The specified options can't be used to limit the choices available | ||||||||
to the user. Instead, they must be applied after the user chooses a source, in order | ||||||||
to generate output that matches the constraints. | ||||||||
to generate output that matches the options. | ||||||||
- The go-ahead permission to use `getDisplayMedia()` cannot be persisted | ||||||||
for reuse. The user must be prompted for permission every time. | ||||||||
- [Transient user activation](/en-US/docs/Web/Security/User_activation) is required. The user has to interact with the page or a UI element in order for this feature to work. | ||||||||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -47,21 +47,21 @@ video {{domxref("MediaStreamTrack")}}, then checking the value of the returned | |
{{domxref("MediaTrackSettings.displaySurface", "displaySurface")}} object. | ||
|
||
For example, if your app needs to know that the surface being shared is a monitor or | ||
application—meaning that there's possibly a non-content backdrop—it can use code similar | ||
window—meaning that there's possibly a non-content backdrop—it can use code similar | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not sure which non-content backdrop appears in a single window? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Good point; I've changed it so that it only applies to a fullscreen/monitor capture. |
||
to this: | ||
|
||
```js | ||
let mayHaveBackdropFlag = false; | ||
let displaySurface = displayStream.getVideoTracks()[0].getSettings().displaySurface; | ||
|
||
if (displaySurface === "monitor" || displaySurface ==="application") { | ||
if (displaySurface === "monitor" || displaySurface ==="window") { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not sure we want to replace "application" with "window" there. If so, you may want to update the paragraph above that still uses "application" then. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ah, good spot; I missed that mention of "application". I've updated it to "window" to match the code snippet, as the "application" value was removed from the spec. I'm not sure if it has been removed from browsers yet, but I thought this would be good defensive writing for either eventuality. |
||
mayHaveBackdropFlag = true; | ||
} | ||
``` | ||
|
||
Following this code, `mayHaveBackdrop` is `true` if the display | ||
surface contained in the stream is of type `monitor` or | ||
`application`; either of these _may_ have non-content backdrop areas. | ||
`window`; either of these _may_ have non-content backdrop areas. | ||
Later code can use this flag to determine whether or not to perform special processing, | ||
such as to remove or replace the backdrop, or to "cut" the individual display areas out | ||
of the received frames of video. | ||
|
@@ -73,8 +73,6 @@ use of the `displaySurface` property. In addition, see | |
{{SectionOnPage("/en-US/docs/Web/API/Media_Capture_and_Streams_API/Constraints", "Example: | ||
Constraint exerciser")}} for a complete example showing how constraints are used. | ||
|
||
TBD | ||
|
||
## Specifications | ||
|
||
{{Specifications}} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Change made in next commit.