Chrome: PIPELINE_ERROR_DECODE not seen on other browsers

[gregdolby]

What version of Hls.js are you using?

0.13.1

What browser and OS are you using?

Chrome (This issue does not appear when using in the hls.js demo page on either Firefox or Safari)

Test stream:

https://static.realeyes.cloud/53_source_92720_361823_hls_1765618_558/53_source_92720_361823_hls_1765618_558.m3u8

https://hls-js.netlify.com/demo/?src=https%3A%2F%2Fstatic.realeyes.cloud%2F53_source_92720_361823_hls_1765618_558%2F53_source_92720_361823_hls_1765618_558.m3u8&demoConfig=eyJlbmFibGVTdHJlYW1pbmciOnRydWUsImF1dG9SZWNvdmVyRXJyb3IiOnRydWUsImR1bXBmTVA0IjpmYWxzZSwibGV2ZWxDYXBwaW5nIjotMSwibGltaXRNZXRyaWNzIjotMX0=

Checklist

• The issue observed is not already reported by searching on Github under https://github.com/video-dev/hls.js/issues
• The issue occurs in the stable client on https://hls-js.netlify.com/demo and not just on my page

• The issue occurs in the latest client on https://hls-js-latest.netlify.com/demo and not just on my page

• The stream has correct Access-Control-Allow-Origin headers (CORS)
• There are no network errors such as 404s in the browser console when trying to play the stream

Steps to reproduce

1. Please provide clear steps to reproduce your problem
   Click the link to the hls.js demo page provided and look in the console logs. This error only occurs on chrome browsers but plays fine in Firefox and Safari (although the warnings still exist)
2. If the bug is intermittent, give a rough frequency

Expected behavior

No PIPELINE_ERROR_DECODE: and the ad plays through the whole 30s

Actual behavior

On chrome the PIPELINE_ERROR_DECODE: occurs and the stream crashes

Console output

[log] > parsed codec:mp4a.40.5,rate:44100,nb channel:2
[log] > AAC: align PTS for overlapping frames by -23
[warn] > parsing error:AAC PES did not start with ADTS header,offset:6
Error event: {type: "mediaError", details: "fragParsingError", fatal: false, reason: "AAC PES did not start with ADTS header,offset:6", frag: Fragment, …}
[log] > audio sampling rate : 44100
[log] > InitPTS for cc: 0 found from video track: 129916
[log] > creating sourceBuffer(audio/mp4;codecs=mp4a.40.2)
[log] > creating sourceBuffer(video/mp4;codecs=avc1.640028)
[log] > main track:audio,container:audio/mp4,codecs[level/parsed]=[mp4a.40.2/mp4a.40.5]
[log] > main track:video,container:video/mp4,codecs[level/parsed]=[avc1.640028/avc1.640028]
[log] > Parsed audio,PTS:[0.000,5.805],DTS:[0.000/5.805],nb:250,dropped:0
[log] > Parsed video,PTS:[0.023,5.996],DTS:[0.000/5.963],nb:180,dropped:0
[log] > main stream-controller: PARSING->PARSED
[log] > main buffered : [0.023,5.805]
[log] > latency/loading/parsing/append/kbps:483/3987/55/16/5478
[log] > main stream-controller: PARSED->IDLE
[log] > Loading 1 of [0 ,4],level 6, currentTime:5.805,bufferEnd:5.805
[log] > main stream-controller: IDLE->FRAG_LOADING
[log] > target start position not buffered, seek to buffered.start(0) 3.118119 from current time 0
[log] > media seeking to 3.118
[log] > media seeked to 3.118
[log] > recoverMediaError
[log] > detachMedia
[log] > media source detaching
[log] > main stream-controller: FRAG_LOADING->STOPPED
[log] > audio stream:IDLE->STOPPED
[log] > attachMedia
main.js:740 The video playback was aborted due to a corruption problem or because the video used features your browser did not support - PIPELINE_ERROR_DECODE: Failed to send audio packet for decoding: timestamp=3390113 duration=23219 size=5774 side_data_size=0 is_key_frame=1 encrypted=0 discard_padding (us)=(0, 0)
Paste the contents of the browser console here.

[log] > parsed codec:mp4a.40.5,rate:44100,nb channel:2
[log] > AAC: align PTS for overlapping frames by -23
[warn] > parsing error:AAC PES did not start with ADTS header,offset:6
Error event: {type: "mediaError", details: "fragParsingError", fatal: false, reason: "AAC PES did not start with ADTS header,offset:6", frag: Fragment, …}
[log] > audio sampling rate : 44100
[log] > InitPTS for cc: 0 found from video track: 129916
[log] > creating sourceBuffer(audio/mp4;codecs=mp4a.40.2)
[log] > creating sourceBuffer(video/mp4;codecs=avc1.640028)
[log] > main track:audio,container:audio/mp4,codecs[level/parsed]=[mp4a.40.2/mp4a.40.5]
[log] > main track:video,container:video/mp4,codecs[level/parsed]=[avc1.640028/avc1.640028]
[log] > Parsed audio,PTS:[0.000,5.805],DTS:[0.000/5.805],nb:250,dropped:0
[log] > Parsed video,PTS:[0.023,5.996],DTS:[0.000/5.963],nb:180,dropped:0
[log] > main stream-controller: PARSING->PARSED
[log] > main buffered : [0.023,5.805]
[log] > latency/loading/parsing/append/kbps:483/3987/55/16/5478
[log] > main stream-controller: PARSED->IDLE
[log] > Loading 1 of [0 ,4],level 6, currentTime:5.805,bufferEnd:5.805
[log] > main stream-controller: IDLE->FRAG_LOADING
[log] > target start position not buffered, seek to buffered.start(0) 3.118119 from current time 0 
[log] > media seeking to 3.118
[log] > media seeked to 3.118
[log] > recoverMediaError
[log] > detachMedia
[log] > media source detaching
[log] > main stream-controller: FRAG_LOADING->STOPPED
[log] > audio stream:IDLE->STOPPED
[log] > attachMedia
main.js:740 The video playback was aborted due to a corruption problem or because the video used features your browser did not support - PIPELINE_ERROR_DECODE: Failed to send audio packet for decoding: timestamp=3390113 duration=23219 size=5774 side_data_size=0 is_key_frame=1 encrypted=0 discard_padding (us)=(0, 0)

For media errors reported on Chrome browser, please also paste the output of chrome://media-internals

[robwalch]

Matt Wolenetz said:

It could be that hls.js is misdetecting sbr-ness of the AAC stream. Chrome needs this to be more precise, currently. I see in the logs in that issue that mp4a.40.5 is parsed, but the sourcebuffer is added using mp4a.40.2. This could be a red herring, but maybe it is root cause of eventual ffmpeg decode error on attempted playback.

Hi @gregrealeyes,
All renditions in the stream appear to use LC AAC mp4a.40.2 as the audio codec. I'm not sure why we're getting an error parsing the AAC in your stream. This is not something we're seeing frequently resulting in media errors.

[robwalch]

Not sure exactly what the issue is, but it seems to begin at packet 3693 in this TS file (similar issue in all renditions) https://static.realeyes.cloud/53_source_92720_361823_hls_1765618_558/53_source_92720_361823_hls_1765618_558-0_00000.ts

TSDemuxer logs "AAC PES did not start with ADTS header,offset:6". It finds the ADTS header at byte 6 instead of 0, and then cannot find any AAC frames. It overflows this data starting at the header offset of 6 so we continue to not find any frames for the remainder of the segment.

[robwalch]

@daveh on videe-dev slack pointed out the problem:

I wonder if the code can handle the case where the ADTS frame header itself straddles a PES boundary?
I think this is the problem. While evaluating the condition here,
https://github.com/video-dev/hls.js/blob/master/src/demux/tsdemuxer.js#L1039
if (ADTS.isHeader(data, offset) && (offset + 5) < len) {
we see that just before the failure, ADTS.isHeader(data, offset) evaluates to true, while (offset + 5) < len evaluates to false

Turns out that check is causing frames to be dropped.

[raymondjacobson]

This doesn't seem to be fixed in 0.13.2 for me -- is this a known issue?

4.013 | pipeline_state | kPlaying
-- | -- | --
4.014 | pipeline_state | kSeeking
4.014 | pipeline_state | kPlaying
4.015 | audio_buffering_state | BUFFERING_HAVE_ENOUGH
4.015 | for_suspended_start | false
4.015 | pipeline_buffering_state | BUFFERING_HAVE_ENOUGH
13.328 | seek_target | 1269.90389
13.328 | pipeline_state | kSeeking
13.328 | audio_buffering_state | BUFFERING_HAVE_NOTHING
13.388 | pipeline_state | kPlaying
13.389 | audio_buffering_state | BUFFERING_HAVE_ENOUGH
13.392 | for_suspended_start | false
13.392 | pipeline_buffering_state | BUFFERING_HAVE_ENOUGH
16.838 | error | Failed to send audio packet for decoding: timestamp=1273578666 duration=21333 size=818 side_data_size=0 is_key_frame=1 encrypted=0 discard_padding (us)=(0, 0)
16.838 | error | audio decode error!
16.856 | error | audio error during playing, status: PIPELINE_ERROR_DECODE
16.856 | pipeline_error | 3
16.856 | pipeline_state | kStopping
16.856 | pipeline_state | kStopped
19.914 | seek_target | 1273.472519
21.910 | Event | WEBMEDIAPLAYER_DESTROYED

Seems to be the same issue?

[robwalch]

@raymondjacobson Here's the fix included in 0.13.2 https://github.com/video-dev/hls.js/pull/2530/files

If you're still experiencing issues, please file a new bug report issue and include steps to reproduce with a test stream or encoding instructions to produce media that reproduces the issue.

[raymondjacobson]

Thx @robwalch -- I did happen to see this fix, but it doesn't seem to work for us (running against 0.13.2). We still see the same error.

I was able to manually implement our own "nudging" to fix this by catching audio.onerror (HTML Audio Element) and setting audio.currentTime += offset and that worked.

Is there anything else we need to do to get that change to work other than upgrading?

(on 0.13.2)

[robwalch]

@raymondjacobson I don't think your issue is directly related to the one in this ticket. Same symptom, different issue. Please fill out the Bug Report Template as part of your issue, making sure to include:

• Test stream/page (if possible)
• Steps to reproduce
• Expected behavior
• Actual behavior

If the issue is related to your stream, and you cannot share the stream, please include all the information we would need to reproduce the issue. This includes how generate a stream that reproduces the issue.

[raymondjacobson]

@robwalch forwarded this to the chromium folks, but didn't really get anywhere (they blamed ffmpeg). There are some steps to reproduce there as well as the ffmpeg ticket.

On my end, i'm able to sort-of progress playback by following a similar method here. I'm able to "absorb" this kind of error (in audio.onerror) on the client by reloading the audio source, nudging the playhead, and reattaching media to the audio object. Trying to do this on Hls.onError doesn't seem to work as intended.

his.audio.onerror = e => {
      this.onError(AudioError.AUDIO, e)

      // Handle audio errors by trying to nudge the playhead and re attach media.
      // Simply nudging the media doesn't work.
      //
      // This kind of error only seems to manifest on chrome because, as they say
      // "We tend to be more strict about decoding errors than other browsers.
      // Ignoring them will lead to a/v sync issues."
      // https://bugs.chromium.org/p/chromium/issues/detail?id=1071899
      if (IS_CHROME_LIKE) {
        // Likely there isn't a case where an error is thrown while we're in a paused
        // state, but just in case, we record what state we were in.
        const wasPlaying = !this.audio.paused
        if (this.url) {
          const newTime = this.audio.currentTime + ON_ERROR_NUDGE_SECONDS
          this.hls.loadSource(this.url)
          this.hls.attachMedia(this.audio)
          // Set the new time to the current plus the nudge. If this nudge
          // wasn't enough, this error will be thrown again and we will just continue
          // to nudge the playhead forward until the errors stop or the song ends.
          this.audio.currentTime = newTime
          if (wasPlaying) {
            this.audio.play()
          }
        }
      }
    }
  }

Not sure if changes in hls.js really make sense unless there's a way to bundle what i've done here into the library.

https://bugs.chromium.org/p/chromium/issues/detail?id=1071899
https://trac.ffmpeg.org/ticket/8621

Thank you for your help & attention!!

[raymondjacobson]

Minimum repro

Here's the source file
​https://drive.google.com/file/d/1FziM3tAXvRzjA8XmxGUEan4Z0dtEPYez/view?usp=sharing

Here's the command
ffmpeg -i graves-master-2.mp3 -ar 48000 -b:a 320k -hls_time 6 -hls_list_size 0 -hls_base_url segments/ -hls_segment_filename segments/segment%03d.ts -vn out.m3u8

Then run this:
ffmpeg -i segments/segment134.ts -vn -f wav -y /dev/null

[akovacs123]

I do experience the same using v0.14.16. Or is it a different error?

PIPELINE_ERROR_DECODE: Failed to send audio packet for decoding: {timestamp=2273216020 duration=21333 size=175 is_key_frame=1 encrypted=0}

[robwalch]

Hi @raymondjacobson and @akovacs123,

This issue has been closed. If you would like help with a similar issue, please file a new bug report and include steps to reproduce with a test stream or encoding instructions to produce media that reproduces the issue.