HLS.js dropping frames after discontinuities

[James-aai]

What version of Hls.js are you using?

Have observed bug in both HLS.js v0.14.0 and v0.13.2

What browser and OS are you using?

Tested in Chrome and Firefox, both showing same bug.

Test stream:

Example stream: https://aai-helper-apps.s3-us-west-2.amazonaws.com/OneSecondChunks/atest_discont.m3u8
Permalink to demo player: https://hls-js.netlify.app/demo/?src=https%3A%2F%2Faai-helper-apps.s3-us-west-2.amazonaws.com%2FOneSecondChunks%2Fatest_discont.m3u8&demoConfig=eyJlbmFibGVTdHJlYW1pbmciOnRydWUsImF1dG9SZWNvdmVyRXJyb3IiOnRydWUsImR1bXBmTVA0IjpmYWxzZSwibGV2ZWxDYXBwaW5nIjotMSwibGltaXRNZXRyaWNzIjotMX0=

Checklist

• [ X ] The issue observed is not already reported by searching on Github under https://github.com/video-dev/hls.js/issues
• [ X ] The issue occurs in the stable client on https://hls-js.netlify.com/demo and not just on my page
• [ ~ The example above does not, but our prod environment does ] The stream has correct Access-Control-Allow-Origin headers (CORS)
• [ X ] There are no network errors such as 404s in the browser console when trying to play the stream

Steps to reproduce

In HLS.js a frame is dropped following a discontinuity. It seems to happen about 70% of the time that the player drops either frame 0 or frame 1 (i.e. the 1st or 2nd frame in the TS segment).

I have experimented with different TS segment lengths, different videos, different frame rates and bit rates. The problem seems pretty consistent.

To reproduce what I'm seeing (on windows 10), create a test video at 25 FPS with FFMPEG:

ffmpeg -f lavfi -i testsrc=duration=60:size=1960x1080:rate=25 -vf "drawtext=fontsize=30:fontcolor=White:fontfile='c:\Windows\Fonts\arial.ttf':text='Frame: %{eif:mod(n,25):d}':x=2:y=2:box=1: boxcolor=0x00000000@1" -b:v 1m atestsrc.mpg

Then slice the video into 1 second long TS segments:

ffmpeg -i atestsrc.mpg -force_key_frames "expr:gte(t,n_forced*1)" -strict -2 -c:a aac -c:v libx264 -f segment -segment_list_type m3u8 -segment_list_size 0 -segment_time 1.0 -segment_time_delta 0 -segment_list atest.m3u8 seg%02d.ts

The resulting output plays back just fine, until I add the #EXT-X-DISCONTINUITY tags. See example stream above.

Expected behavior

I expect the player to play 60 x 1 second TS segments back in 60 seconds. It actually plays back in about 58 seconds due to dropped frames.
I expect each frame of the TS segment to rendered on screen. Attachment shows slow-motion playback where the 2nd frame (frame 1) is not rendered.

Actual behavior

you can see that it plays back a 60 second video in 58 seconds. The same symptom occurs in several different HLS.js based players, such as the demo player and video-react.

If you play it back at 0.1 times speed you can see that it is not rendering some frames, often frame 0 or 1 (see attachment).

Console output

HLS.js does not appear to produce any relevant console output regarding this issue.

[robwalch]

I see that in Safari, with video.src = playback, the duration and frame timing is accurate.

HLS.js does not appear to produce any relevant console output regarding this issue.

Actually it does, although poorly. This may not be the issue though if the same thing is logged without discontinuities. Disabling the code that produces these logs only introduced gaps.

[log] > AVC: 40 ms overlapping between fragments detected
[log] > Video: PTS/DTS adjusted: 1000/920, delta: -40 ms

Thanks for filing the issue!

[robwalch]

@James-aai can you upload another version of the stream without the discontinuities? It would help if I could compare how the stream is remuxed with and without them. Thank you.

[James-aai]

Thanks @robwalch - here's a version of the stream without the discontinuities:
https://aai-helper-apps.s3-us-west-2.amazonaws.com/OneSecondChunks/atest.m3u8

[robwalch]

Thanks @James-aai,

I think I got it #2876
https://deploy-preview-2876--hls-js-dev.netlify.app/demo/

Once it passes review, we'll get it out into a patch.

[James-aai]

Awesome work @robwalch! - I've checked a few of my problem files in the preview-2876 demo player, and it's looking great!
Many thanks for your speedy response - this is a huge help to us.

[James-aai]

Just to let you know that Version 0.14.1 looks great - thanks again @robwalch and team for sorting this out so quickly!

[robwalch]

Thanks @James-aai,

If you support IE11, please update to v0.14.2. Released earlier today, it patches a regression introduced in v0.14.1 that interfered with playback in IE11 only.