HLS in separate audio and video always out of sync

[cgjh1]

Have you read the FAQ and checked for duplicate open issues?
yes

If the problem is related to FairPlay, have you read the tutorial?

not fairplay problem

What version of Shaka Player are you using?
https://cdnjs.cloudflare.com/ajax/libs/shaka-player/4.7.9/shaka-player.compiled.js
also tested on demo page
https://shaka-player-demo.appspot.com

Can you reproduce the issue with our latest release version?
yes

Can you reproduce the issue with the latest code from main?
yes

Are you using the demo app or your own custom app?
tried both

If custom app, can you reproduce the issue using our demo app?
yes

What browser and OS are you using?
Tested on Chrome and Safari

For embedded devices (smart TVs, etc.), what model and firmware version are you using?
no embedded involved

What are the manifest and license server URIs?
It fails on all hls when audio is not in the same file as the video.
If you set :
Prefer native HLS playback when available = true
and play on safari it works,
but on chrome hls with audio and video seperat, always out of sync.
Not ONE single example found online showing use of hls live with audio/video separate
Can not use dash since no player can recover if a live feed stops and the restarts, must have discontinuity support.

What configuration are you using? What is the output of player.getConfiguration()?

What did you do?

What did you expect to happen?

What actually happened?

[joeyparrish]

It fails on all hls when audio is not in the same file as the video.
Not ONE single example found online showing use of hls live with audio/video separate

There are several built into the demo app:

https://shaka-player-demo.appspot.com/demo/#panel=ALL_CONTENT;panelData=HLS

For example:

• https://shaka-player-demo.appspot.com/demo/#asset=https://storage.googleapis.com/shaka-demo-assets/bbb-dark-truths-hls/hls.m3u8
• https://shaka-player-demo.appspot.com/demo/#asset=https://storage.googleapis.com/shaka-demo-assets/angel-one-hls/hls.m3u8
• https://shaka-player-demo.appspot.com/demo/#asset=https://storage.googleapis.com/shaka-demo-assets/sintel-fmp4-aes/master.m3u8
• https://shaka-player-demo.appspot.com/demo/#asset=https://bitdash-a.akamaihd.net/content/MI201109210084_1/m3u8s/f08e80da-bf1d-4e3d-8899-f0f6155f6efa.m3u8

All of these have separate audio and video streams, and all of these are in sync for me. I checked each one in Chrome. Do you get different results?

[cgjh1]

Hi Thank you so much for the response, I really appreciate it, however I checked all the links you sent me and the first 3 are clearly marked VOD, in their segmentplaylist: for example: if you do : `curl https://storage.googleapis.com/shaka-demo-assets/bbb-dark-truths-hls/hls.m3u8` you get ``` #EXTM3U ## Generated with https://github.com/google/shaka-packager version v2.3.0-5bf8ad5-release #EXT-X-MEDIA:TYPE=AUDIO,URI="playlist_a-eng-0128k-aac-2c.mp4.m3u8",GROUP-ID="default-audio-group",LANGUAGE="en",NAME="stream_0",AUTOSELECT=YES,CHANNELS="2" #EXT-X-STREAM-INF:BANDWIDTH=255636,AVERAGE-BANDWIDTH=212901,CODECS="avc1.42c01e,mp4a.40.2",RESOLUTION=250x144,AUDIO="default-audio-group" playlist_v-0144p-0100k-libx264.mp4.m3u8 ``` ..... etc now look at the playlist :playlist_v-0144p-0100k-libx264.mp4.m3u8 `curl https://storage.googleapis.com/shaka-demo-assets/bbb-dark-truths-hls/playlist_v-0144p-0100k-libx264.mp4.m3u8` you get: ``` #EXTM3U #EXT-X-VERSION:6 ## Generated with https://github.com/google/shaka-packager version v2.3.0-5bf8ad5-release #EXT-X-TARGETDURATION:5 #EXT-X-PLAYLIST-TYPE:VOD.  <--------- this is not live #EXT-X-MAP:URI="v-0144p-0100k-libx264-init.mp4" #EXTINF:4.000, v-0144p-0100k-libx264-s1.mp4 #EXTINF:4.000, v-0144p-0100k-libx264-s2.mp4 ``` ... etc you will get the same for 2 more an the last look also like a vod since its segment number starts on 0 and its an. older version 3 hls list So as far as I can tell none of the examples you gave are live, that is, none of them start reading from the end of the playlist. Vod work for me as well, To reproduce my problem is really easy, you can take any source, lets say a video with audio, let ffmpeg loop it, to produce udp in 1x speed using -re in the input, that goes forevever on that loop. Then start shaka packager to generate a live hls manifest with one video one audio streamplaylist  and the audio will never be in sync when playing the hls manifest on chrome. Here is how you can test it: ******************************** 1.Download a test source: `wget https://storage.googleapis.com/gtv-videos-bucket/sample/TearsOfSteel.mp4` 2.Loop it to an udp stream to simulate a running live feed (source is already h264 and aac so no conversion is needed): `ffmpeg -re -stream_loop -1 -i TearsOfSteel.mp4 -c:a copy -c:v copy -f mpegts udp://239.255.50.36:5020` 3.check that you have udp data being produced by doing: `ffprobe udp://239.255.50.36:5020` Now you should get: ``` Input #0, mpegts, from 'udp://239.255.50.36:5020':   Duration: N/A, start: 11.105944, bitrate: N/A   Program 1     Metadata:       service_name    : Service01       service_provider: FFmpeg   Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1280x534, 24 fps, 24 tbr, 90k tbn   Stream #0:1[0x101](und): Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 194 kb/s ``` All is good 4 Time for the shaka packager, download it here (I am running on linux, I hope you do the same): ``` wget https://github.com/shaka-project/shaka-packager/releases/download/v2.6.1/packager-linux-x64 chmod +x packager-linux-x64 ./packager-linux-x64 --version packager-linux-x64 version v2.6.1-634af65-release ``` Nice the packager is working. 5. Start packager to create a live HLS manifest that we then will play in shaka player: ``` MYHTMLPATH="/var/www/html/mytest" ./packager-linux-x64 \ 'in=udp://239.255.50.36:5020? reuse=1, stream=audio,playlist_name=audio.m3u8, segment_template=$MYHTMLPATH/audio_$Time$.ts' \ 'in=udp://239.255.50.36:5020? reuse=1, stream=video,playlist_name=video_sd.m3u8,segment_template=$MYHTMLPATH/video_sd_$Time$.ts' \ --mpd_output /cryptoguard/content/startrek/master.mpd \ --hls_master_playlist_output $MYHTMLPATH/master.m3u8 \ --hls_playlist_type LIVE --time_shift_buffer_depth 120 \ --preserved_segments_outside_live_window 5 --segment_duration 10 ``` Done If you play this in shaka player demo as a custom asset on chome using the master.m3u8 it will not be in sync Again thank you so much for looking into this, please let me know if anything is unclear and I will try to explain better. Regards Jörgen

[roberto-sartori-gl]

I think I have the same issue: I have a bunch of hls and a bunch of dash streams.

All the dash streams are fine, but all the hls streams show the out of sync problem mentioned here. The hls manifests have separate tracks for audio and video.

[bazeport-jhe]

I can report that I am experiencing what I think is the same issue as described above. AV out of sync with separate audio and video streams. Safari is OK, Chrome and Firefox not OK. Tested with v4.7.11 and 4.8.1

Update: Setup a two test applications one with Shaka player and one with HLS.js - HLS.js plays the content fine in all browsers, Shaka struggles as mentioned above. Another thing to note is that in Shaka the audio is behind almost exactly 3 seconds.