Fix initial audio video desync for fMP4

[jungdaniel]

This PR will...

Solves initial AV desync for fMP4.

Why is this Pull Request needed?

It seems that there are cases when initPTS handling is faulty and it results in misaligned offsets for A/V.
I suspect that audio demuxing can start here without knowing the video's initPTS: https://github.com/video-dev/hls.js/blob/master/src/controller/audio-stream-controller.js#L526

Resolves issues:

Might be the same issue: #2545

Checklist

• changes have been done against master branch, and PR does not conflict
• new unit / functional tests have been added (whenever applicable)
• API or design changes are documented in API.md

[jungdaniel]

Hi,
My original claim was that if video loads prior to the audio, then there is no desync. I checked that case by delaying audio downloads and indeed there was no desync.

@robwalch suggested that only the video track should establish initPTS, to avoid different code paths.
Here is a different attempt for solving the issue: jungdaniel@d42d2ea
Let me know if this is a better start and I'll update the PR.

[jungdaniel]

Hi @robwalch, @itsjamie,

I checked this AV desync issue on the v0.14.5 release, and it seems to be still there. I updated the PR. Here's a sample playlist I reproduced the issue with: https://jungdaniel.github.io/hls.js/sample/playlist.m3u8

With the current version what I see is that both the video and audio tracks are shifted to 0.

Here I pushed the demo page with the fix if someone would take a look. I think this is the correct timeline:

For sanity check here's a little test code for playing the same sample with MSE. No magic or timeline shifting, but the tracks are in sync: https://jungdaniel.github.io/hls.js/sample/index.html

As a side note, sometimes the current version also produces the correct timeline. I still believe that it happens when the audio arrives later then the video and the video's initPTS can be used during audio demuxing. I know that removing details.initSegment would defer audio demuxing, until the initPTS is known. Is it a problem? For ts chunks that is already undefined I think. Let me know if you have suggestions. Thanks.

[robwalch]

Hi @jungdaniel,

The v0.14.5 fixes were for .ts file AVC remuxing, so it makes sense it didn't help here.

I love how simple the fix is! Thank you for these additional details.

sometimes the current version also produces the correct timeline. I still believe that it happens when the audio arrives later then the video and the video's initPTS can be used during audio demuxing. I know that removing details.initSegment would defer audio demuxing, until the initPTS is known. Is it a problem?

I don't think it is a problem to defer handling of the initial audio payload. Audio loads and decodes faster so deferring the parsing>buffering is OK. It used to be the audio playlist loading that was deferred, which was bad, but prevented this issue. Always basing the timeline offset on the main stream will provide more stable results. :)

I'd like to do some testing and debugging before giving review feedback, and merging the change. Hold tight! I'll get this in soon.

[robwalch]

Doesn't solve #2545 but I don't see any issue with the change. If it prevents an edge case resulting in different buffered ranges it's a great improvement.