CMAF Low Latency Chunks

[fadomire]

Hello,

i wanted to know if it is planned to support CMAF Low latency chunks ?

I saw reference to that in the following ticket comparing dash.js and shaka, but no status for shaka
#1351

thanks for the work by the way

[theodab]

We will look into this feature. We will add it if we can, but we'll need to investigate how it would work with MediaSource.

[fadomire]

thanks for the reply

it may be interesting to check how dashjs made their implementation when the subject come in the roadmap

one thread discussing low latency specifics :
Dash-Industry-Forum/dash.js#1474

[PatrikCarlander]

Is it possible to priorities CMAF support? CMAF is what the OTT industry asking for now. At NAB-Show (April 6 - 11, 2019 | Las Vegas) we have to show CMAF with Dashif instead of Shaka and it's sad.

[PatrikCarlander]

The live stream simulator has available test vectors which expose low latency chunking. This stream has 8s segments with 1s chunks:

https://vm2.dashif.org/livesim-chunked/chunkdur_1/ato_7/testpic4_8s/Manifest300.mpd

It is possible to set the ato and chunkdur to floating point numbers like chunkdur_0.5/ato_7.5/ in the URL. The original content is 30Hz, so setting ato to 7.9 and chunkdur to 0.1 is also possible.

[joeyparrish]

Hi, everyone. Just to clarify: CMAF is already supported. Low-latency CMAF will be added later this year.

[PatrikCarlander]

It was Low-latency CMAF we asked for.
thanks for the reply

[joeyparrish]

@PatrikCarlander, yes, but I was clarifying based on your first comment "Is it possible to priorities CMAF support". But I think we are all on the same page now.

[Romain-Bernier]

Any update about Low Latency CMAF ?
Thanks

[joeyparrish]

We haven't begun work on this yet. We anticipate working on this in Q3 2019.

[ramoncaldeira]

+1 for low-latency DASH

[PatrikCarlander]

Q3 2019 is ending soon, is this feature postponed? LL is a highly requested feature from our customers. We need this in Shaka to get LL in chromecast too.

[ismena]

Hi @PatrikCarlander
So, there's good news and there's bad news. The bad news is, we got entangled in pre-work for this feature (cleaning up and optimizing our hls code), so we weren't able to start actual work when we'd hoped to.
The good news is, that the pre-work is almost done (I think I saw the last optimization chunks go into into code review last week :)). We've already allocated time for LL and the developer has started research.
I'll sync with them next week (after the research) to see if we can give you a better estimate.
I will also do my best to cajole our TL @joeyparrish into making LL one of our biggest priorities for the rest of the year.

[PatrikCarlander]

This can be a good input to define the scope.
ExoPlayer Low Latency design doc

[michellezhuogg]

Hello @PatrikCarlander ,

I saw that both Community Hls spec and Apple Hls spec are mentioned in the document. Which one are you looking forward to use?

[PatrikCarlander]

@michellezhuogg For a start ULL-DASH, after that we will see if Apple LLHLS will replace Community LHLS.

[quanghuy1288]

waiting for hls cmaf low-latency :)

[buhlaze]

I am confused about cmaf. Is Cmaf just a mpeg-dash manifest with Availabilitytimeoffset added to it? I want to use it instead of mpeg-dash. Also, how do you make a CMAF manifest with ffmpeg with or without shaka player? I have looked everywhere and can't find an example of how to do so.

[leandromoreira]

@buhlaze CMAF is a container (much like .TS or .MKV or yet .MP4)

[buhlaze]

low latency CMAF, in order to take advantage of it, does it just involve adding Availabilitytimeoffset to your dash manifest in the BASEURL with segmented video X number of seconds and a player that supports low latency, for example LowLatencyEnabled? Could I please get an explanation?

[kanongil]

The preload hints don't support the BYTERANGE-START attribute. It incorrectly uses a non-specced BYTERANGE attribute.

[joeyparrish]

@kanongil, thanks for pointing that out! It has now been fixed in the master branch.

[vlee-harmonicinc]

It is great that shaka player support ll-hls!
Apart from delta update (_HLS_skip), is there any plan for Blocking Playlist Reload (_HLS_MSN, _HLS_part) as well?
https://tools.ietf.org/html/draft-pantos-hls-rfc8216bis-08#section-6.2.5.2

[joeyparrish]

Our internal streaming model requires us to update all playlists, rather than just the media playlist for the currently-selected streams. So we decided that blocking playlist updates might not make sense for Shaka Player at this time.

@michellezhuogg, let's discuss this Tuesday and double-check our assumptions to make sure we made the right call on that.

[joeyparrish]

@michellezhuogg and I are not sure if we made the right decision about blocking playlist updates. We originally decided not to do them because we're updating all media playlists.

It might work to block on a media playlist update, even if we are going to update all playlists serially. If all are going to be blocked on the same segment number, for example, the first one would block, and subsequent playlist requests would not, since that segment would already be ready in all the others by the time the first was ready.

At least, that's what we're thinking now.

I'm also wondering if there are other configs that would need to be changed to accommodate this. Manifest request timeout comes to mind, but we could also override that per-request in the HLS parser without changing the configuration per se.

Any thoughts on this? Anyone interested to experiment with the HLS parser to see what breaks if we block?

[vlee-harmonicinc]

A client MUST NOT assume that segments with the same Media Sequence Number in different Variant Streams or Renditions have the same position in the presentation; Playlists MAY have independent Media Sequence Numbers.
https://tools.ietf.org/html/draft-pantos-hls-rfc8216bis-07#section-6.3.2

For example Apple's LL-HLS CMAF stream have different duration and MSN for audio and video. Blocking one media playlist still works if playback latency large enough to tolerate the difference, but probably not a prefect design.

I tested implement blocking playlist reload via saving next MSN and next part number in streamInfo and marking preload in segmentReference. It seems works, but there is some other issue related to lint/compile :P

By the way, how about #1936: Lazy-load HLS media playlists on adaptation ?

A client MUST load the Media Playlist file of every Rendition selected for playback in order to locate the media specific to that Rendition. But, to prevent unnecessary load on the server, it SHOULD NOT load the Playlist file of any other Rendition.

Lazy-load is not mandatory and not directly related to this issue, but LL-HLS better have lazy-load in practice. Since the partial segment of LL-HLS is short, update frequency of LL-HLS is much higher than standard HLS. For example, if playing a LL-HLS stream with 2 audio 5 video and 1s partial duration, updating all playlist requires downloading. >7 media playlist per second, while only 2 selected playlists are necessary.

[joeyparrish]

The problem with lazy-loading is Shaka Player's current architecture. Manifest parsers don't know what the currently-selected stream is or when it changes, so there is no good way to know which playlist needs to be updated. We designed manifest parsing to be completely decoupled from streaming, but it seems that in this case, they are too decoupled.

[avelad]

I think it would be nice to add a table in the readme of the features needed for low latency to work. For example in DASH it is necessary the ReadableStream API

[maximk-1]

Hi shaka team!
Do you plan to release LL-DASH support this year?

[joeyparrish]

@htleeab wrote:

For example Apple's LL-HLS CMAF stream have different duration and MSN for audio and video. Blocking one media playlist still works if playback latency large enough to tolerate the difference, but probably not a prefect design.

Based primarily on this, and on our current architecture (manifest parsers completely independent of streaming), @michellezhuogg and I have decided not to add blocking playlist updates for now. If a feature meant to reduce latency on updates ends up adding latency because of how it interacts with our existing architecture, it's best not to do it for now.

@Mkozlov1 wrote:

Do you plan to release LL-DASH support this year?

Well, we certainly planned it that way, but has anyone else noticed how 2020 has gone so far? We're doing what we can with very limited resources (team members temporarily allocated to other internal priorities). We intend to wrap up the features (basically done now) and close enough serious bugs to feel confident about the stability of the master branch before we release v3.1. We hope that can be accomplished before the end of the year, but if it takes until January 15th to make v3.1 usable, we'd rather have the release work as intended than have it technically come out before midnight on Dec 31.

With that, I think our initial LL feature work can be called complete. We're open to discussions of blocking playlist updates and other features with accompanying architecture changes at a later date.

Thanks!

[michellezhuogg]

Hello everyone,

The Low Latency HLS and Low Latency Dash features are ready for testing on our Nightly build demo:
https://nightly-dot-shaka-player-demo.appspot.com/demo/#audiolang=en-US;textlang=en-US;lowLatencyMode=true;uilang=en-US;panel=HOME;build=uncompiled
We would like to get a few sample manifests for testing, or any bug report before our official release of v3.1.
Thank you!

[kanongil]

While it seems to be somewhat workable with LL-HLS on a good connection, the ABR experience quickly breaks down. There are several minor/major issues, but I won't list them, as it seems that this has hardly been tested. Note that using the Chrome network throttling is not representative of real-world issues, since it always triggers a navigator.connection 'change' event, which resets the estimator.

I will mention one non-ABR problem: Playback does not respect the INDEPENDENT=YES flag, and starts from whatever fragment, causing playback issues.

Anyway, I probably won't be using this as long as it doesn't support blocking playlist reloads.

[michellezhuogg]

@kanongil ,

Thank you for your feedback! Would you please share some sample low-latency manifests with us, so that we can test and triage the minor/major issues that you mentioned? You can send them to shaka-player-issues@google.com if they're private.
We'll also look into the problem with INDEPENDENT=YES flag. Thank you!

[michellezhuogg]

Closing the issue due to inactivity. If anyone has further questions or comments, please open another new issue. Thank you!