ts file with audio opus codec - the audio is disabled

[obecny]

What version of Hls.js are you using?

1.5.7

What browser (including version) are you using?

chrome Version 122.0.6261.129 (Official Build) (arm64)

What OS (including version) are you using?

mac os Monterey

Test stream

https://github.com/obecny/hls-opus-testing

Configuration

{}

Additional player setup steps

No response

Checklist

• The issue observed is not already reported by searching on Github under https://github.com/video-dev/hls.js/issues
• The issue occurs in the stable client (latest release) on https://hlsjs.video-dev.org/demo and not just on my page
• The issue occurs in the latest client (main branch) on https://hlsjs-dev.video-dev.org/demo and not just on my page
• The stream has correct Access-Control-Allow-Origin headers (CORS)
• There are no network errors such as 404s in the browser console when trying to play the stream

Steps to reproduce

1. Checkout page https://github.com/obecny/hls-opus-testing
2. run for example http-server
3. Open the page and the stream should play fine but there is not audio

here is the output from ffmpeg for file1.ts

Program 1
Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1280x720, 24 fps, 23.92 tbr, 90k tbn
Stream #0:1[0x101]: Audio: opus (Opus / 0x7375704F), 48000 Hz, stereo, fltp

Expected behaviour

the audio works fine

What actually happened?

there is no audio

Console output

15:37:42.577 hls.min.js:1 [log] > Debug logs enabled for "Hls instance" in hls.js version 1.5.7
15:37:42.578 hls.min.js:1 [log] > attachMedia
15:37:42.578 hls.min.js:1 [log] > [buffer-controller] created media source: MediaSource
15:37:42.594 hls.min.js:1 [log] > [buffer-controller] Media source opened
15:37:42.594 hls.min.js:1 [log] > stopLoad
15:37:42.595 hls.min.js:1 [log] > loadSource:http://127.0.0.1:8080/index.m3u8
15:37:42.595 hls.min.js:1 [log] > [stream-controller]: Trigger BUFFER_RESET
15:37:42.604 hls.min.js:1 [log] > [level-controller]: manifest loaded, 1 level(s) found, first bitrate: 2162956
15:37:42.604 hls.min.js:1 [log] > setting initial bwe to 2162956
15:37:42.604 hls.min.js:1 [log] > [buffer-controller] 1 bufferCodec event(s) expected
15:37:42.604 hls.min.js:1 [log] > startLoad(-1)
15:37:42.604 hls.min.js:1 [log] > [level-controller]: Switching to level 0 (720p SDR avc1,opus @2162956) from level -1
15:37:42.605 hls.min.js:1 [log] > [level-controller]: Loading level index 0 with http://127.0.0.1:8080/stream.m3u8
15:37:42.605 hls.min.js:1 [log] > [stream-controller]: STOPPED->IDLE
15:37:42.605 hls.min.js:1 [log] > [subtitle-stream-controller]: STOPPED->IDLE
15:37:42.605 (index):48 Uncaught (in promise) DOMException: play() failed because the user didn't interact with the document first. https://goo.gl/xX8pDD
(anonymous) @ (index):48
o.emit @ hls.min.js:1
e.emit @ hls.min.js:1
e.trigger @ hls.min.js:1
r.filterAndSortMediaOptions @ hls.min.js:1
r.onManifestLoaded @ hls.min.js:1
o.emit @ hls.min.js:1
e.emit @ hls.min.js:1
e.trigger @ hls.min.js:1
e.handleMasterPlaylist @ hls.min.js:1
onSuccess @ hls.min.js:1
e.readystatechange @ hls.min.js:1
XMLHttpRequest.send (async)
e.openAndSendXhr @ hls.min.js:1
e.loadInternal @ hls.min.js:1
e.load @ hls.min.js:1
e.load @ hls.min.js:1
e.onManifestLoading @ hls.min.js:1
o.emit @ hls.min.js:1
e.emit @ hls.min.js:1
e.trigger @ hls.min.js:1
e.loadSource @ hls.min.js:1
(anonymous) @ (index):44
o.emit @ hls.min.js:1
e.emit @ hls.min.js:1
e.trigger @ hls.min.js:1
_onMediaSourceOpen @ hls.min.js:1
15:37:42.620 hls.min.js:1 [log] > [stream-controller]: Level 0 loaded [0,2][part-2--1], cc [0, 0] duration:3
15:37:42.620 hls.min.js:1 [log] > [buffer-controller] Updating Media Source duration to 3.000
15:37:42.621 hls.min.js:1 [log] > [stream-controller]: Loading fragment 0 cc: 0 of [0-2] level: 0, target: 0
15:37:42.621 hls.min.js:1 [log] > [stream-controller]: IDLE->FRAG_LOADING
15:37:42.637 hls.min.js:1 [log] > injecting Web Worker for "main"
15:37:42.639 hls.min.js:1 [log] > [transmuxer-interface, main]: Starting new transmux session for sn: 0 p: -1 level: 0 id: 1
        discontinuity: true
        trackSwitch: true
        contiguous: false
        accurateTimeOffset: true
        timeOffset: 0
        initSegmentChange: true
15:37:42.639 hls.min.js:1 [log] > [stream-controller]: Loaded fragment 0 of level 0
15:37:42.657 964f40bd-725b-4d39-8c71-d2e9f6eba890:1 [log] > Debug logs enabled for "main" in hls.js version 1.5.7
15:37:42.658 hls.min.js:1 [log] > [mp4-remuxer]: ISGenerated flag reset
15:37:42.658 hls.min.js:1 [log] > [mp4-remuxer]: initPTS & initDTS reset
15:37:42.658 hls.min.js:1 [log] > [mp4-remuxer]: reset next timestamp
15:37:42.658 hls.min.js:1 [warn] > MPEG-TS PMT found at 18800 after unknown PID '256'. Backtracking to sync byte @0 to parse all TS packets.
r.onWorkerMessage @ hls.min.js:1
onwmsg @ hls.min.js:1
15:37:42.664 hls.min.js:1 [log] > [stream-controller]: FRAG_LOADING->PARSING
15:37:42.664 hls.min.js:1 [log] > [stream-controller]: Init video buffer, container:video/mp4, codecs[level/parsed]=[avc1.64001f/avc1.64001f]
15:37:42.664 hls.min.js:1 [log] > [buffer-controller] 0 bufferCodec event(s) expected video
15:37:42.664 hls.min.js:1 [log] > [buffer-controller] creating sourceBuffer(video/mp4;codecs=avc1.64001f)
15:37:42.664 hls.min.js:1 [log] > [audio-stream-controller]: InitPTS for cc: 0 found from main: 767892
15:37:42.665 hls.min.js:1 [log] > [transmuxer.ts]: Flushed fragment 0 of level 0
15:37:42.665 hls.min.js:1 [log] > [stream-controller]: PARSING->PARSED
15:37:42.666 hls.min.js:1 [log] > [stream-controller]: Buffered main sn: 0 of level 0 (frag:[0.000-9.138] > buffer:[0.000-9.138])
15:37:42.666 hls.min.js:1 [log] > [stream-controller]: PARSED->IDLE
15:37:42.667 hls.min.js:1 [log] > [stream-controller]: Loading fragment 1 cc: 0 of [0-2] level: 0, target: 9.138
15:37:42.667 hls.min.js:1 [log] > [stream-controller]: IDLE->FRAG_LOADING
15:37:42.712 hls.min.js:1 [log] > [stream-controller]: Loaded fragment 1 of level 0
15:37:42.719 hls.min.js:1 [log] > [stream-controller]: FRAG_LOADING->PARSING
15:37:42.720 hls.min.js:1 [log] > [transmuxer.ts]: Flushed fragment 1 of level 0
15:37:42.720 hls.min.js:1 [log] > [stream-controller]: PARSING->PARSED
15:37:42.723 hls.min.js:1 [log] > [stream-controller]: Buffered main sn: 1 of level 0 (frag:[9.138-17.928] > buffer:[0.000-17.928])
15:37:42.723 hls.min.js:1 [log] > [stream-controller]: PARSED->IDLE
15:37:42.723 hls.min.js:1 [log] > [stream-controller]: Loading fragment 2 cc: 0 of [0-2] level: 0, target: 17.928
15:37:42.723 hls.min.js:1 [log] > [stream-controller]: IDLE->FRAG_LOADING
15:37:42.734 hls.min.js:1 [log] > [stream-controller]: Loaded fragment 2 of level 0
15:37:42.735 hls.min.js:1 [log] > [stream-controller]: FRAG_LOADING->PARSING
15:37:42.735 hls.min.js:1 [log] > [transmuxer.ts]: Flushed fragment 2 of level 0
15:37:42.735 hls.min.js:1 [log] > [stream-controller]: PARSING->PARSED
15:37:42.735 hls.min.js:1 [log] > [stream-controller]: Buffered main sn: 2 of level 0 (frag:[17.928-19.136] > buffer:[0.000-19.136])
15:37:42.735 hls.min.js:1 [log] > [stream-controller]: PARSED->IDLE
15:37:42.736 hls.min.js:1 [log] > [buffer-controller] video sourceBuffer now EOS
15:37:42.736 hls.min.js:1 [log] > [buffer-controller] Queueing mediaSource.endOfStream()
15:37:42.736 hls.min.js:1 [log] > [stream-controller]: IDLE->ENDED
15:37:42.736 hls.min.js:1 [log] > [buffer-controller] Calling mediaSource.endOfStream()
15:37:42.736 hls.min.js:1 [log] > [buffer-controller] Media source ended

Chrome media internals output

No response

[robwalch]

Opus in TS is not a supported segment format of HLS.js.

[DavidDudson]

@obecny Although not officially supported. We have got opus working via.

    this.hlsJs = new Hls({
      defaultAudioCodec: 'opus',
    })

Which then gets passed to the browser. YMMV though and I doubt it would be considered an hls.js bug if you have issues

[avelad]

@robwalch In Shaka I have implemented this support, see shaka-project/shaka-player#6387 (if you are interested...)

[robwalch]

Containerless opus and opus in fmp4 are supported. You don't need any special configuration with the latest release.

To support Opus in TS we would need a feature request and contribution (PR) adding support. Some details on why you need this format would be helpful as well. Adding more TS decoders to HLS.js bloats the library payload for users who only need to play HLS with codecs and containers supported on a majority of platforms.