AV1 Support

[davidzhao]

This issue tracks AV1 support within LiveKit. AV1 offers higher bandwidth efficiency, improved visual quality, as well as SVC. We aim to support AV1 in the SFU and all of our SDKs. Currently Chrome is the only browser that supports AV1 SVC under a field trial (as of M104): --force-fieldtrials=WebRTC-DependencyDescriptorAdvertised/Enabled/

One of the main challenges with new codecs in an SFU is backwards compatibility. What happens when some users in the room are publishing AV1, but others aren't able to decode it?

Our solution to this is multi-codec simulcast: allowing the publisher to dynamically publish a backup codec if anyone in the room requests for it. This technique will be covered in detail in a future blog post.

SFU

• Dependency descriptor parsing
• AV1 header & frame parsing
• Support SVC layers
• Multi-codec simulcast protocol
• PRs
  • add support for vp9 and av1 #574
  • add support for av1 svc #669
  • simulcast codecs support #720

SDKs

JS

• SVC publishing
• Receives AV1 streams
• Multi-codec simulcast
• Chrome (not enabled by default)
• Firefox
• Safari
• PRs
  • add av1 svc codec client-sdk-js#221
  • simulcast codecs support client-sdk-js#240
  • publish new codec when subscriber need client-sdk-js#267
  • mung sdp for av1 bitrate setting client-sdk-js#314
  • Add experimental fallback codec support client-sdk-js#334

Swift

• SVC publishing
• Receives AV1 streams
• Multi-codec simulcast

Kotlin (Android)

• SVC publishing
• Receives AV1 streams
• Multi-codec simulcast

Flutter

• SVC publishing
• Receives AV1 streams
• Multi-codec simulcast

React-Native

• SVC publishing
• Receives AV1 streams
• Multi-codec simulcast

Rust based SDKs (Python, Node, Unity)

work is planned

• [] SVC publishing
• [] Receives AV1 streams
• [] Multi-codec simulcast

Go

work is planned

• SVC publishing
• Receives AV1 streams
• Multi-codec simulcast

[rriemann]

https://caniuse.com/av1 reports of 90% reach for AV1 in browsers. Let's make 2024 the AV1 year! 😎