HLS Live Playback: Segment Gaps when Seeking Forward after Pause

[softworkz]

I'm experiencing the following issue when playing live playlists:

Segment merging/stitching appears to be inaccurate when segments aren't played immediately (e.g. after pausing).
Two scenarios, the first one works fine, the second one produces issues:

1. Playing at the Live Edge, then seeking backward

=> Can seek to arbitrary (earlier) positions, playback always working flawlessly

2. Pausing Playback for a while, then seeking forward

After playing for a while, I pause and wait until no more segments are downloaded, then I wait an additional 2 or 3 minutes, then resume playback

=> When seeking to past positions, playback is working flawlessly like above

=> But when seeking to positions newer than the pause position, hlsjs starts to encounter "holes" in the stream frequently and hangs for a short moment each time:

Environment

• Chrome 89
• Windows 10
• HLSJS master and HLSJS v1.0.0
• With my PR 3755 (Fix Video Glitches with Fetch Loader #3755) and without
• With progressive set to true and false

I've tried many different combinations of the variants above. The strength of the symptoms varies from extremely annoying to hardly noticeable, but they are always there.

At this time, I don't have a useful conclusion..

[robwalch]

Not sure what's going on here but may be related to continuity in the transmuxer.

I'm not sure exactly how to reproduce this based on the description. Can you provide one primary use case, including a stream and hls.js config, with ordered steps to reproduce the issue?

[softworkz]

Not sure what's going on here but may be related to continuity in the transmuxer.

I'm not sure exactly how to reproduce this based on the description. Can you provide one primary use case, including a stream and hls.js config, with ordered steps to reproduce the issue?

Thanks, I got something now to test. Here are the steps:

• Open https://hls-js.netlify.app/demo/
• Enter video URL: https://softworkz.github.io/hlstestfiles/gaps/media.m3u8
• Add "progressive": true and press Apply
• Let it play for a minute
• Hit Pause
• Open F12 tools > Network
• Wait until no more segments are being loaded
• Switch to 'Console' tab
• Seek to positions in the second half
• Watch playback behavior and console log output
• In case of playback hangs on certain position, seek to replay those positions

I've seen different effects: Sometimes hangs/gaps and sometimes short phases of super-fast videos
It's not 100% exactly identical to what I've seen when the playlist is a growing live-playlist, but most likely the same issue.

Please let me know whether you can reproduce.
Thanks

[softworkz]

Are you able to reproduce?

There are certain specifics to MPEGTS streams from tv broadcasts:

• Timestamp values (PCR/DTS) don't start from zero, so segment 0 doesn't start with timestamp 0
• There's always a delay between video and audio streams, with the audio always being ahead
  in the provided sample stream, it's about 400 ms
• Key frames don't occur at an exact interval (but at least once every x seconds)
  • Target Duration in the example stream is 4s (as it is the maximum possible duration)
  • Segments are listed with 3s in the playlist
  • The segmentation is done in a way that a 3s average is constantly maintained

Here are the initial segment durations from the provided sample stream:

• C: Container
• V: Video Stream
• A: Audio Stream

[robwalch]

The 400ms gap between audio and video sounds like what we have in this sample:

https://hls-js-dev.netlify.app/demo/?src=https%3A%2F%2Fplayertest.longtailvideo.com%2Fadaptive%2Fboxee%2Fplaylist.m3u8&demoConfig=eyJlbmFibGVTdHJlYW1pbmciOnRydWUsImF1dG9SZWNvdmVyRXJyb3IiOnRydWUsInN0b3BPblN0YWxsIjpmYWxzZSwiZHVtcGZNUDQiOmZhbHNlLCJsZXZlbENhcHBpbmciOi0xLCJsaW1pdE1ldHJpY3MiOi0xfQ==

I took a look at the sample stream provided again on latest:
https://hls-js-dev.netlify.app/demo/?src=https%3A%2F%2Fplayertest.longtailvideo.com%2Fadaptive%2Fboxee%2Fplaylist.m3u8&demoConfig=eyJlbmFibGVTdHJlYW1pbmciOnRydWUsImF1dG9SZWNvdmVyRXJyb3IiOnRydWUsInN0b3BPblN0YWxsIjpmYWxzZSwiZHVtcGZNUDQiOmZhbHNlLCJsZXZlbENhcHBpbmciOi0xLCJsaW1pdE1ldHJpY3MiOi0xfQ==

I still see some audio gaps on occasion, but the recent backtracking improvements appear to be kicking in and loading SN-1 since now we recognize that keyframes are not always present at the start of a segment, so we load the previous one preemptively.

How does the behavior look to you now @softworkz ?

[softworkz]

How does the behavior look to you now @softworkz ?

From a first glance, it appears to be fixed.

Great work!

Thanks a lot for looking into this.