WebRTC “end-to-end-encryption”

[jgraham]

Description

The RTCRtpScriptTransform API allows websites to plug JavaScript workers into WebRTC’s realtime media pipeline, to transform sent and received encodings. Websites use this API to implement their own audiovisual encryption to protect users from middleboxes the website may employ (but not from the website itself).

Unfortunately, bifurcation exists in implementations, with two browsers implementing the spec, and one implementing a now non-standard predecessor, forcing websites today to work against two different APIs, causing a major interoperability headache.

Specification

https://w3c.github.io/webrtc-encoded-transform/

Open Issues

No response

Tests

• Tests starting with “script-” in https://wpt.fyi/results/webrtc-encoded-transform
• Other tests of non-standards-track APIs in this folder are not part of this nomination

Current Implementations

• Blink
• Gecko
• WebKit

Standards Positions

No response

Browser bug reports

No response

Developer discussions

No response

Polls & Surveys

No response

Existing Usage

No response

Workarounds

No response

Accessibility Impact

No response

Privacy Impact

This proposal covers a technology that's explicitly designed to be privacy-enhancing.

Other

Note that WebRTC in general does not show up as a priority on developer surveys as it's generally used by a small number of sites that specialize in video conferencing and related services. However those sites are often very popular and compatibility issues highly visible to users (see e.g. webcompat.com issue reports)

[jgraham]

CC @jan-ivar

[youennf]

This makes sense to me. There might be interactions between script transforms and things like simulcast that would be useful to drill into (aka write more tests) as well.

[foolip]

For survey data and web developer demand, in preliminary results from State of HTML 2023, WebRTC was a somewhat common response to the freeform question "Which existing HTML features or browser APIs are you unable to use because of browser differences or lack of support?"

[foolip]

There are some important use cases covered by the older version of the spec that Chrome implemented, which aren't yet handled by the new spec. The Chrome team is engaged on these issues to try to resolve this, so that we can converge on a single interoperable API:

• Generalize ScriptTransform constructor to allow main-thread processing w3c/webrtc-encoded-transform#89
• Is frame-level fanout in scope of WebRTC encoded transform? w3c/webrtc-encoded-transform#211
• Frame-level fanout requires ability to move frames between PCs w3c/webrtc-encoded-transform#200
• Expose flow control information to transform w3c/webrtc-encoded-transform#206
• Tell SDP negotiation to negotiate support for non-standard codecs w3c/webrtc-encoded-transform#172
• Add a metadata modifying method to RTCEncodedFrame w3c/webrtc-encoded-transform#162
• Allow incoming encoded frames to be forwarded to one or more senders on other PeerConnections w3c/webrtc-encoded-transform#160

My understanding is it's unlikely that work on the new API will start until we're confident it can fully replace what we've already shipped.

[jan-ivar]

Thanks @foolip for the context.

My understanding is it's unlikely that work on the new API will start until we're confident it can fully replace what we've already shipped.

What prevents Google from implementing both? The standards-track API already interoperates in Safari and Firefox, satisfying the requirement to exit CR. Chrome's API lacks consensus. What's the best way forward for interop?

Main-thread processing is possible in a worker-first API. The burden of transfer merely goes the other way.

The other issues are all newer and seem orthogonal to the API shape and main-thread processing. Can you explain what the concern is?

[alvestrand]

The lack of concern shown with meeting Chrome requirements (see timeline on #89, lack of progress on #200) means that we have no customers for the Firefox/Safari proposal. The proposal is not yet CR (no consensus call has been issued), so it's a bit early to talk about exiting CR.
(We have a successful (August 2021) consensus call for adoption).

[fippo]

What prevents Google from implementing both?

Web developer feedback from April 2020 here, here and here has been enthusiastic, the API is in production and doing well and getting used for additional use-cases such as dynamic audio redundancy?

See here for adoption of the new API (or lack thereof).

Framing Chromium as the interop problem after not shipping anything for three years... wow!

[jan-ivar]

The lack of concern shown with meeting Chrome requirements (see timeline on #89, lack of progress on #200)

Do you mean w3c/webrtc-encoded-transform#89 and w3c/webrtc-encoded-transform#200?

[alvestrand]

Yes, forgot about autoformatting linking these wrongly. Sorry.

[jgraham]

Thank you for proposing WebRTC “end-to-end-encryption” for inclusion in Interop 2024.

We wanted to let you know that this proposal was not selected to be part of Interop this year.

We did not have consensus to include this proposal. This should not be taken as a comment on the technology as a whole

For an overview of our process, see proposal selection. Thank you again for contributing to Interop 2024!

Posted on behalf of the Interop team.