A question about audio and video transmission!!!

[MardenSSS]

I am a novice on dash.js. I noticed that the server can store audio and video decoupled encoding chunks separately. When playing in dash.js, it will fetch the corresponding video chunk and audio chunk from the server respectively.

I'm curious about how dash.js fetches audio chunk and video chunk in this process. In my practice, I found that audio chunk and video chunk with the same index are always downloaded next to each other. Is there any coupling relationship between them? Are they serial transmission or parallel transmission?

[dsilhavy]

Audio and video segments are downloaded in parallel by dash.js according to the buffer targets, see https://reference.dashif.org/dash.js/nightly/samples/buffer/buffer-target.html. Note that only one segment per type is downloaded at the same time, i.e. no download of two video segments at the same time.

[MardenSSS]

@dsilhavy Thanks for your reply, but I can not find anything related to audio and video buffers in the reference link you gave. But I roughly understand what you mean. Taking video chunks as an example, you can only download chunks corresponding to a certain index at the same time, instead of downloading video chunks corresponding to multiple indexes at the same time. Right?

Finally, I would like to ask, are the downloads of audio chunks and video chunks aligned on their index? For example, the audio chunk with index 2 has been downloaded, but the video chunk with index 2 is being transmitted. Will the audio chunk with index 3 start to be transmitted at this time? Or do I need to wait for the video chunk with index 2 to be transmitted before the audio chunk and video chunk with index 3 can be transmitted together?

Thank you!!!

[dsilhavy]

@dsilhavy Thanks for your reply, but I can not find anything related to audio and video buffers in the reference link you gave. But I roughly understand what you mean. Taking video chunks as an example, you can only download chunks corresponding to a certain index at the same time, instead of downloading video chunks corresponding to multiple indexes at the same time. Right?

Yes, there are two media buffers that you can fill at the same time, one for audio one for video.

Finally, I would like to ask, are the downloads of audio chunks and video chunks aligned on their index? For example, the audio chunk with index 2 has been downloaded, but the video chunk with index 2 is being transmitted. Will the audio chunk with index 3 start to be transmitted at this time? Or do I need to wait for the video chunk with index 2 to be transmitted before the audio chunk and video chunk with index 3 can be transmitted together?

Audio and video segments can be transmitted independently of each other. Audio segments 3,4,5 might already be successfully transmitted while video segment 2 is still being downloaded. dash.js monitors the buffer level for each media type individually and issues the requests accordingly.

[MardenSSS]

@dsilhavy I understand! Thank you very much for your reply!!