Getting the precise in-video time while having frequent EXT-X-DISCONTINUITIes in the stream

[lnstadrum]

We have a video stream consisting of small individual videos concatenated together. Every video consists of a few continuous MPEG-TS fragments. #EXT-X-DISCONTINUITY tag is inserted between the individual videos (as far as there is an actual discontinuity of the timestamps). The playback works great.

We need to be able to map user actions during playback into the time domain of every individual video. In a simpler way, we would like to get the id of the fragment being played.

A naive solution of using currentTime and searching in the playlist does not work: the extrapolated in-video position diverges quickly from the actual one. Our assumption is that this is because of frequent occurrences of #EXT-X-DISCONTINUITY, which seem to add a small variable delay every time it is added to the playlist.

Inserting #EXT-X-PROGRAM-DATE-TIME before each individual video and using playingDate then does not work properly: it allows to identify the fragment being played, but sometimes jumps in the future with respect to the actual playback position. Listening for FRAG_CHANGED behaves very similarly, which is quite surprising. (Tested with HLS.js 1.5.4. We should be able to provide a stream sample.)

We have tried a bunch of other things, such as putting #EXT-X-DATERANGE tags to mark the individual parts of the video and listening for cuechange events on the text metadata track (this seems to do the same as if we were just extrapolating from currentTime) or finely adjusting the fragment duration declared with #EXTINF before each discontinuity.

Is there a way to achieve what we are trying to do?

Thanks.

[robwalch]

Inserting #EXT-X-PROGRAM-DATE-TIME before each individual video and using playingDate then does not work properly: it allows to identify the fragment being played, but sometimes jumps in the future with respect to the actual playback position. Listening for FRAG_CHANGED behaves very similarly, which is quite surprising. (Tested with HLS.js 1.5.4. We should be able to provide a stream sample.)

Program-Date-Time and FRAG_CHANGED would be my suggestions. Please provide examples of these issues (jumps in the future, similar frag change behavior).

There are some improvements to Program-Date-Time (PDT) mapping coming in 1.6 when PDT is not aligned with segment durations or media durations differ from the playlist description. If you provide examples with reproduction steps (please file an issue) we can look into whether playlist date anchoring resolves them, as well root-causing the issues with segment durations or PDT values at discontinuity boundaries.

[lnstadrum]

Hi @robwalch,

Thank you for your quick reply. Much appreciated!

After digging further, we discovered that jumping FRAG_CHANGED and playingDate were related to a corrupted audio stream start timestamp in one of TS fragments. With this fixed, playingDate works perfectly as expected. Sorry for the rush.

We submitted this issue however, which is related to the metadata track synchronization with the video track. Using the metadata (i.e., listening to cuechange events) to identify the video piece currently played appears a better approach, since it would only use common browser API and would (in theory) work in iOS.

Thanks!