Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new screen capture constraints and options #22733

Merged
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
---
title: CaptureController()
slug: Web/API/CaptureController/CaptureController
page-type: web-api-constructor
tags:
- API
- Constructor
- Experimental
- Reference
browser-compat: api.CaptureController.setFocusBehavior
---

{{APIRef("Screen Capture API")}}{{seecompattable}}

The {{domxref("CaptureController")}} constructor creates a new `CaptureController` object instance.

## Syntax

```js-nolint
CaptureController()
```

### Parameters

None.

## Examples

```js
// Create a new CaptureController instance
const controller = new CaptureController();

// Prompt the user to share a tab, window, or screen.
const stream =
await navigator.mediaDevices.getDisplayMedia({ controller });

// Query the displaySurface value of the captured video track
const [track] = stream.getVideoTracks();
const displaySurface = track.getSettings().displaySurface;

if (displaySurface == 'browser') {
// Focus the captured tab.
controller.setFocusBehavior('focus-captured-surface');
} else if (displaySurface == 'window') {
// Do not move focus to the captured window.
// Keep the capturing page focused.
controller.setFocusBehavior('no-focus-change');
}
```

## Specifications

{{Specifications}}

## Browser compatibility

{{Compat}}

## See also

- [Screen Capture API](/en-US/docs/Web/API/Screen_Capture_API)
- {{domxref("MediaDevices.getDisplayMedia()")}}
- [Better screen sharing with Conditional Focus](https://developer.chrome.com/docs/web-platform/conditional-focus/)
66 changes: 66 additions & 0 deletions files/en-us/web/api/capturecontroller/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
---
title: CaptureController
slug: Web/API/CaptureController
page-type: web-api-interface
tags:
- API
- Interface
- CaptureController
- Reference
- Experimental
browser-compat: api.CaptureController
---

{{APIRef("Screen Capture API")}}{{seecompattable}}

The **`CaptureController`** interface provides methods that can be used to further manipulate a capture session separate from its initiation via {{domxref("MediaDevices.getDisplayMedia()")}}.

A `CaptureController` object is associated with a capture session by passing it into a {{domxref("MediaDevices.getDisplayMedia", "getDisplayMedia()")}} call as the value of the options object's `controller` property.

## Constructor

- {{ domxref("CaptureController.CaptureController", "CaptureController()") }}
- : Creates a new `CaptureController` object instance.

## Instance methods

- {{ domxref("CaptureController.setFocusBehavior", "setFocusBehavior()") }}
- : Controls whether the captured tab or window will be focused or whether the focus will remain with the tab containing the capturing app.

## Examples

```js
// Create a new CaptureController instance
const controller = new CaptureController();

// Prompt the user to share a tab, window, or screen.
const stream =
await navigator.mediaDevices.getDisplayMedia({ controller });

// Query the displaySurface value of the captured video track
const [track] = stream.getVideoTracks();
const displaySurface = track.getSettings().displaySurface;

if (displaySurface == 'browser') {
// Focus the captured tab.
controller.setFocusBehavior('focus-captured-surface');
} else if (displaySurface == 'window') {
// Do not move focus to the captured window.
// Keep the capturing page focused.
controller.setFocusBehavior('no-focus-change');
}
```

## Specifications

{{Specifications}}

## Browser compatibility

{{Compat}}

## See also

- [Screen Capture API](/en-US/docs/Web/API/Screen_Capture_API)
- {{domxref("MediaDevices.getDisplayMedia()")}}
- [Better screen sharing with Conditional Focus](https://developer.chrome.com/docs/web-platform/conditional-focus/)
76 changes: 76 additions & 0 deletions files/en-us/web/api/capturecontroller/setfocusbehavior/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
---
title: CaptureController.setFocusBehavior()
slug: Web/API/CaptureController/setFocusBehavior
page-type: web-api-instance-method
tags:
- API
- Experimental
- Method
- Reference
- setFocusBehavior
browser-compat: api.CaptureController.setFocusBehavior
---

{{APIRef("Screen Capture API")}}{{seecompattable}}

The {{domxref("CaptureController")}} interface's **`setFocusBehavior()`** method controls whether the captured tab or window will be focused when an associated {{domxref("MediaDevices.getDisplayMedia()")}} {{jsxref("Promise")}} fulfills, or whether the focus will remain with the tab containing the capturing app.

You can set this behavior multiple times before the {{domxref("MediaDevices.getDisplayMedia()")}} call, or once immediately after its `Promise` resolves. After that, the focus behavior is said to be finalized, and can't be changed.

## Syntax

```js-nolint
setFocusBehavior(focusBehavior)
```

### Parameters

- `focusBehavior`
- : An enumerated value that describes whether the user agent should transfer focus to the captured display surface, or keep the capturing app focused. Possible values are `focus-captured-surface` (transfer focus) and `no-focus-change` (keep focus on the capturing app).

### Return value

`Undefined`.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
`Undefined`.
None (`undefined`).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change made in next commit.


### Exceptions

- `InvalidStateError` {{domxref("DOMException")}}
- : Thrown if the capture stream has been stopped, or if enough time has elapsed after the {{domxref("MediaDevices.getDisplayMedia()")}} `Promise` fulfills that the focus behavior has been finalized.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Calling setFocusBehavior() also throws after the getDisplayMedia() returned promise resolves, if the user shared a screen (not a tab or a window).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, updating this in my next commit. Since the number of different circumstances for throwing an InvalidState error got to three, I've split them out into a bulleted list to make them easier to read.


## Examples

```js
// Create a new CaptureController instance
const controller = new CaptureController();

// Prompt the user to share a tab, window, or screen.
const stream =
await navigator.mediaDevices.getDisplayMedia({ controller });

// Query the displaySurface value of the captured video track
const [track] = stream.getVideoTracks();
const displaySurface = track.getSettings().displaySurface;

if (displaySurface == 'browser') {
// Focus the captured tab.
controller.setFocusBehavior('focus-captured-surface');
} else if (displaySurface == 'window') {
// Do not move focus to the captured window.
// Keep the capturing page focused.
controller.setFocusBehavior('no-focus-change');
}
```

## Specifications

{{Specifications}}

## Browser compatibility

{{Compat}}

## See also

- [Screen Capture API](/en-US/docs/Web/API/Screen_Capture_API)
- {{domxref("MediaDevices.getDisplayMedia()")}}
- [Better screen sharing with Conditional Focus](https://developer.chrome.com/docs/web-platform/conditional-focus/)
51 changes: 28 additions & 23 deletions files/en-us/web/api/mediadevices/getdisplaymedia/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,8 @@ browser-compat: api.MediaDevices.getDisplayMedia

{{DefaultAPISidebar("Screen Capture API")}}

The {{domxref("MediaDevices")}} interface's
**`getDisplayMedia()`** method prompts the user to select and
grant permission to capture the contents of a display or portion thereof (such as a
window) as a {{domxref("MediaStream")}}.
The {{domxref("MediaDevices")}} interface's **`getDisplayMedia()`** method prompts the user to select and
grant permission to capture the contents of a display or portion thereof (such as a window) as a {{domxref("MediaStream")}}.

The resulting stream can then be
recorded using the [MediaStream Recording API](/en-US/docs/Web/API/MediaStream_Recording_API) or transmitted as part of a [WebRTC](/en-US/docs/Web/API/WebRTC_API) session.
Expand All @@ -35,28 +33,37 @@ See [Using the Screen Capture API](/en-US/docs/Web/API/Screen_Capture_API/Using_
## Syntax

```js-nolint
getDisplayMedia(constraints)
getDisplayMedia(options)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
getDisplayMedia(options)
getDisplayMedia()
getDisplayMedia(options)

I guess, since this is optional?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense; added in next commit

```

### Parameters

- `constraints` {{optional_inline}}
- : An optional object specifying requirements for
the returned {{domxref("MediaStream")}}. Since `getDisplayMedia()` requires
a video track, the returned stream will have one even if no video track is expressly
requested by the `constraints` object. For more details, see the [constraints](/en-US/docs/Web/API/MediaDevices/getUserMedia#parameters)
section under the {{domxref("MediaDevices.getUserMedia()")}} method, as well
as the article [Capabilities, constraints, and settings](/en-US/docs/Web/API/Media_Capture_and_Streams_API/Constraints).
- `options` {{optional_inline}}
- : An optional object specifying requirements for the returned {{domxref("MediaStream")}}. The options for `getDisplayMedia()` work in the same as the [constraints](/en-US/docs/Web/API/MediaDevices/getUserMedia#parameters) for the {{domxref("MediaDevices.getUserMedia()")}} method, although in that case only `audio` and `video` can be specified. The list of possible option properties for `getDisplayMedia()` is as follows:
Comment on lines +42 to +43
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This <dl> is broken, probably because of the indentation. (it's a bit of a shame that our <dl> is quite fragile).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to fix this for ages when I was originally writing this, and gave up ;-)

I just had another look, and realized that it was because the note following the <dl> was indented. Fixed in next commit.

- `video`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Presumably also optional?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed, ta.

- : A boolean; a value of `true` indicates that the returned {{domxref("MediaStream")}} will contain a video track. Be aware that, since `getDisplayMedia()` requires a video track, the returned stream will have one even if no video track is expressly requested. If the `video` property is explicitly given a value of `false`, the returned {{jsxref("Promise")}} will reject with a `TypeError`.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The IDL has this and audio as (boolean or MediaTrackConstraints) (https://w3c.github.io/mediacapture-screen-share/#dom-displaymediastreamoptions).

This seems like a puzzling option. It's boolean, but if I set it to false the promise rejects?! And if I set it to true it will contain a video track. Does "even if no video track is expressly requested" here mean "even if this option is omitted? If so we could say something like:

  • if this is option is omitted or set to true, the stream will contain a video track
  • if this option is set to false the promise will reject with an error

Which makes me wonder why we would ever pass it. Am I confused here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes made. as requested. Yup, it is quite confusing. I don't have a good answer for your question.

- `audio` {{optional_inline}}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think for all optional options we should say what the default is.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done; I think I've worked these out correctly...

- : A boolean; a value of `true` indicates that the returned {{domxref("MediaStream")}} will contain an audio track, if audio is supported and available for the display surface chosen by the user.
- `controller` {{optional_inline}}
- : A {{domxref("CaptureController")}} object instance containing methods that can be used to further manipulate the capture session.
- `preferCurrentTab` {{non-standard_inline}} {{optional_inline}}
- : A boolean; a value of `true` instructs the browser to offer the current tab as the most prominent capture source, i.e. as a separate "This Tab" option in the "Choose what to share" options presented to the user. This is useful as many app types generally just want to share the current tab. For example, a slide deck app might want to let the user stream the current tab containing the presentation to a virtual conference.
- `selfBrowserSurface` {{optional_inline}}
- : An enumerated value specifying whether the browser should allow the user to select the current tab for capture. This helps to avoid the "infinite hall of mirrors" effect experienced when a video conferencing app inadvertently shares its own display. Possible values are `include`, which hints that the browser should include the current tab in the choices offered for capture, and `exclude`, which hints that it should be excluded.
- `surfaceSwitching` {{optional_inline}}
- : An enumerated value specifying whether the browser should display a control to allow the user to dynamically switch the shared tab during screen-sharing. This is much more convenient than having to go through the whole sharing process again each time a user wants to switch the shared tab. Possible values are `include`, which hints that the browser should include the control, and `exclude`, which hints that it should not be shown.
- `systemAudio` {{optional_inline}}
- : An enumerated value specifying whether the browser should include the system audio among the possible audio sources offered to the user. Possible values are `include`, which hints that the browser should include the system audio in the list of choices, and `exclude`, which hints that it should be excluded.

> **Note:** See the article [Capabilities, constraints, and settings](/en-US/docs/Web/API/Media_Capture_and_Streams_API/Constraints) for a lot more detail on how these options work.

### Return value

A {{jsxref("Promise")}} that resolves to a {{domxref("MediaStream")}} containing a
video track whose contents come from a user-selected screen area, as well as an optional
audio track.

> **Note:** Browser support for audio tracks varies, both in terms of
> whether or not they're supported at all by the media recorder and in terms of the
> audio sources supported. Check the [compatibility table](#browser_compatibility) for details for each browser.
> **Note:** Browser support for audio tracks varies, both in terms of whether or not they're supported at all by the media recorder and in terms of the audio sources supported. Check the [compatibility table](#browser_compatibility) for details for each browser.

### Exceptions

Expand All @@ -73,17 +80,15 @@ audio track.
- `NotFoundError` {{domxref("DOMException")}}
- : Returned if no sources of screen video are available for capture.
- `NotReadableError` {{domxref("DOMException")}}
- : Returned if the user selected a screen, window, tab, or other source of screen data, but a
- : Returned if the user selected a screen, window, tab, or another source of screen data, but a
hardware or operating system level error or lockout occurred, preventing the sharing
of the selected source.
- `OverconstrainedError` {{domxref("DOMException")}}
- : Returned if, after creating the stream, applying the specified `constraints` fails
- : Returned if, after creating the stream, applying any specified constraints fails
because no compatible stream could be generated.
- {{jsxref("TypeError")}}
- : Returned if the specified `constraints` include constraints which are not permitted
when calling `getDisplayMedia()`. These unsupported constraints are
`advanced` and any constraints which in turn have a member named
`min` or `exact`.
- : Returned if the specified `options` include values that are not permitted
when calling `getDisplayMedia()`, for example a `video` property set to false, or if any specified constraints are not permitted.

## Security

Expand All @@ -92,9 +97,9 @@ source of significant privacy and security concerns. For that reason, the specif
details measures browsers are required to take in order to fully support
`getDisplayMedia()`.

- The specified `constraints` can't be used to limit the options available
- The specified options can't be used to limit the choices available
to the user. Instead, they must be applied after the user chooses a source, in order
to generate output that matches the constraints.
to generate output that matches the options.
- The go-ahead permission to use `getDisplayMedia()` cannot be persisted
for reuse. The user must be prompted for permission every time.
- [Transient user activation](/en-US/docs/Web/Security/User_activation) is required. The user has to interact with the page or a UI element in order for this feature to work.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,14 +54,14 @@ to this:
let mayHaveBackdropFlag = false;
let displaySurface = displayStream.getVideoTracks()[0].getSettings().displaySurface;

if (displaySurface === "monitor" || displaySurface ==="application") {
if (displaySurface === "monitor" || displaySurface ==="window") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure we want to replace "application" with "window" there. If so, you may want to update the paragraph above that still uses "application" then.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, good spot; I missed that mention of "application". I've updated it to "window" to match the code snippet, as the "application" value was removed from the spec. I'm not sure if it has been removed from browsers yet, but I thought this would be good defensive writing for either eventuality.

mayHaveBackdropFlag = true;
}
```

Following this code, `mayHaveBackdrop` is `true` if the display
surface contained in the stream is of type `monitor` or
`application`; either of these _may_ have non-content backdrop areas.
`window`; either of these _may_ have non-content backdrop areas.
Later code can use this flag to determine whether or not to perform special processing,
such as to remove or replace the backdrop, or to "cut" the individual display areas out
of the received frames of video.
Expand All @@ -73,8 +73,6 @@ use of the `displaySurface` property. In addition, see
{{SectionOnPage("/en-US/docs/Web/API/Media_Capture_and_Streams_API/Constraints", "Example:
Constraint exerciser")}} for a complete example showing how constraints are used.

TBD

## Specifications

{{Specifications}}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
---
title: MediaTrackConstraints.suppressLocalAudioPlayback
slug: Web/API/MediaTrackConstraints/suppressLocalAudioPlayback
page-type: web-api-instance-property
tags:
- API
- Audio
- Experimental
- Media
- Media Capture and Streams API
- Media Streams API
- MediaTrackConstraints
- Property
- Web
- suppressLocalAudioPlayback
browser-compat: api.MediaTrackConstraints.suppressLocalAudioPlayback
---

{{APIRef("Media Capture and Streams")}}{{seecompattable}}

The {{domxref("MediaTrackConstraints")}} dictionary's **`suppressLocalAudioPlayback`** property is a [`ConstrainBoolean`](/en-US/docs/Web/API/MediaTrackConstraints#constrainboolean) describing the requested or mandatory constraints placed upon the value of the {{domxref("MediaTrackSettings.suppressLocalAudioPlayback","suppressLocalAudioPlayback")}} constrainable property. This property controls whether the audio playing in a tab will continue to be played out of a user's local speakers when the tab is captured.

If needed, you can determine whether or not this constraint is supported by checking
the value of {{domxref("MediaTrackSupportedConstraints.suppressLocalAudioPlayback")}} as returned
by a call to {{domxref("MediaDevices.getSupportedConstraints()")}}. However, typically
this is unnecessary since browsers will ignore any constraints they're unfamiliar with.

## Value

If this value is a simple `true` or `false`, the user agent will
attempt to obtain media with local audio playback enabled or disabled as specified, if
possible, but will not fail if this can't be done. If, instead, the value is given as an
object with an `exact` field, that field's Boolean value indicates a required
setting for the noise suppression feature; if it can't be met, then the request will
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact values are not allowed in getDisplayMedia calls. They are in applyConstraints calls.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've had a go at updating this section, but I do find these constraint object values rather confusing. You'll have to let me know if you think I'm any closer to getting it right. This also makes me think of two other issues:

  • We ought to look at the other screen capture-specific constraints, and see if they are correct in terms of not using exact.
  • I ought to put the note about exact not being permitted in getDisplayMedia calls somewhere in the main http://localhost:5042/en-US/docs/Web/API/MediaTrackConstraints page, so it is more visible. But probably not on every individual constraint page.
    What do you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call. I've checked, and it is kind of covered on the getDisplayMedia() page, in the TypeError entry in the Exceptions section, but I've added in a line explicitly to say that min and exact are not allowed.

I've also followed my plan as outlined above.

result in an error.

## Examples

See {{SectionOnPage("/en-US/docs/Web/API/Media_Capture_and_Streams_API/Constraints", "Example: Constraint exerciser")}} for an example.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are deprecating SectionOnPage: mdn/yari#6117 (comment)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But also this didn't seem to include an example of suppressLocalAudioPlayback?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

macro removed. I've also updated the example section to include a one-line example,


## Specifications

{{Specifications}}

## Browser compatibility

{{Compat}}

## See also

- [Media Capture and Streams API](/en-US/docs/Web/API/Media_Capture_and_Streams_API)
- [Capabilities, constraints, and settings](/en-US/docs/Web/API/Media_Capture_and_Streams_API/Constraints)
- {{domxref("MediaTrackConstraints")}}
- {{domxref("MediaDevices.getSupportedConstraints()")}}
- {{domxref("MediaTrackSupportedConstraints")}}
- {{domxref("MediaStreamTrack")}}
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,6 @@ surface being captured.

The value of `displaySurface` is a string that comes from the `DisplayCaptureSurfaceType` enumerated type, and is one of the following:

- `application`
- : The stream's video track contains all of the windows belonging to the application chosen by the user.
The windows are aggregated into a single video track, with any empty space filled with a backdrop; that backdrop is selected by the {{Glossary("user agent")}}.
- `browser`
- : The stream's video track presents the entire contents of a single browser tab which the user selected during the {{domxref("MediaDevices.getDisplayMedia","getDisplayMedia()")}} call.
- `monitor`
Expand Down