Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix PTS calculation on rollover. #2930

Merged
merged 6 commits into from Aug 4, 2020

Conversation

itsjamie
Copy link
Collaborator

@itsjamie itsjamie commented Jul 29, 2020

This PR will...

When a rollover was occuring the timeoffset value was incorrectly calculated due to taking the minimum PTS from the new set of rollovers, this lead to a very large timeOffset delta on the audio and led to incorrect PTS values after running through PTSNormalize function.

This happened to be fixed by the silent gap code just below, which set the sample pts and dts to be the expected next value, but this now makes it so on well-formed media, the else branch is essentially a no-op because the delta is zero.

} else {
// Otherwise, just adjust pts
if (Math.abs(delta) > (0.1 * inputSampleDuration)) {
// logger.log(`Invalid frame delta ${Math.round(delta + inputSampleDuration)} at PTS ${Math.round(pts / 90)} (should be ${Math.round(inputSampleDuration)}).`);
}
sample.pts = sample.dts = nextPts;
nextPts += inputSampleDuration;
i++;
}

Why is this Pull Request needed?

Investigating #1796.

Are there any points in the code the reviewer needs to double check?

Did this against your branch @robwalch since your change fixed the time duration, this now resolves an internal issue with the PTS value because of the timeOffset on a rollover.

I think it just needs a thorough test suite run.

Behaviour prior:

First segment after PTS normalization

Screen Shot 2020-07-29 at 7 26 25 PM

Contiguous Segment Prior to PTS Normalization

image

Contiguous Segment Post PTS Normalization

Screen Shot 2020-07-29 at 7 27 45 PM

New Behaviour

Contiguous Segment Post PTS Normalization

image

Resolves issues:

Checklist

  • changes have been done against master branch, and PR does not conflict
  • new unit / functional tests have been added (whenever applicable)
  • API or design changes are documented in API.md

@itsjamie itsjamie requested a review from robwalch July 29, 2020 23:24
Base automatically changed from bugfix/handle-wrapped-pts-before-dts to master July 30, 2020 16:06
src/remux/mp4-remuxer.js Outdated Show resolved Hide resolved
robwalch pushed a commit that referenced this pull request Jul 30, 2020
@itsjamie itsjamie mentioned this pull request Jul 31, 2020
3 tasks
@robwalch robwalch self-requested a review July 31, 2020 17:04
Copy link
Collaborator

@robwalch robwalch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's nice to have this centralized in the remuxer :)

src/demux/tsdemuxer.js Show resolved Hide resolved
@robwalch robwalch self-requested a review July 31, 2020 17:27
@robwalch robwalch added this to the 0.15.0 milestone Jul 31, 2020
robwalch pushed a commit that referenced this pull request Jul 31, 2020
@robwalch robwalch force-pushed the bugfix/handle-wrapping-pts-timeoffset-calculation branch from 8f0b2e5 to aa15efa Compare July 31, 2020 20:30
robwalch
robwalch previously approved these changes Jul 31, 2020
@robwalch
Copy link
Collaborator

robwalch commented Aug 1, 2020

@itsjamie we might want to look into this test failure in more detail:
should "smooth switch" to highest level and still play(readyState === 4) after 12s for ARTE China,ABR:
It's timing out with only a couple of seconds of logs ending with

2020-07-31 22:13:31.574: "[log] > audio sourceBuffer now EOS"
2020-07-31 22:13:31.574: "[log] > video sourceBuffer now EOS"
2020-07-31 22:13:31.575: "[log] > all media data are available, signal endOfStream() to MediaSource and stop loading fragment"
2020-07-31 22:13:31.576: "[log] > main stream-controller: IDLE->ENDED"
2020-07-31 22:13:31.580: "[log] > media source ended"

@robwalch
Copy link
Collaborator

robwalch commented Aug 1, 2020

The video PTS is getting messed up with these changes:

[log] > Loaded 0 of [0 ,50],level 0
hls.js:21463 [log] > Parsing 0 of [0 ,50],level 0, cc 0
...
hls.js:21463 [log] > Parsed audio,PTS:[0.000,9.899],DTS:[0.000/9.899],nb:232,dropped:0
hls.js:21463 [log] > Parsed video,PTS:[95443.718,95453.718],DTS:[95443.584/95453.584],nb:150,dropped:0

Our initPTS is correct at 0. The initial dts samples need to wrap down. This used to be done in ts-demuxer.

0: {key: true, pts: 0, dts: 8589922592, units: Array(4), debug: "", …}
1: {key: false, pts: 24000, dts: 8589928592, units: Array(1), debug: "", …}
2: {key: false, pts: 12000, dts: 0, units: Array(1), debug: "", …}
3: {key: false, pts: 6000, dts: 6000, units: Array(1), debug: "", …}

nextAvcDts is way off because cts is not calculated correctly (-8589922592).

    if (!contiguous) {
      var pts = timeOffset * timeScale;
      var cts = inputSamples[0].pts - inputSamples[0].dts; // if not contiguous, let's use target timeOffset

      nextAvcDts = pts - cts;
    }

robwalch pushed a commit that referenced this pull request Aug 1, 2020
@robwalch robwalch force-pushed the bugfix/handle-wrapping-pts-timeoffset-calculation branch from 7c80447 to a104506 Compare August 1, 2020 21:32
@robwalch
Copy link
Collaborator

robwalch commented Aug 1, 2020

Here's the fix:

a104506#diff-14a34ee6c08b15d0e309d10c2c05f3d0R235

Normalizing samples[0].dts to samples[0].pts ensures we get a positive composition timestamp. This is taken care of in the TS Demuxer in v0.14.7, but with these changes, needs to happen here, since we haven't normalized the values yet.

@robwalch robwalch modified the milestone: 0.15.0 Aug 1, 2020
itsjamie and others added 6 commits August 4, 2020 00:04
When a rollover was occuring the timeoffset value was incorrectly
calculated due to taking the minimum PTS from the new set of rollovers,
this lead to a very large delta on the PTS for the audio and led to
incorrect PTS values.

This happened to be fixed by the silent gap code just below, which set
the sample pts and dts to be the expected next value, but this now makes
it so on well-formed media, the else branch is essentially a no-op
because the delta is zero.
@robwalch robwalch force-pushed the bugfix/handle-wrapping-pts-timeoffset-calculation branch from 34b6811 to cf9a969 Compare August 4, 2020 04:08
@robwalch robwalch merged commit e9c9bf3 into master Aug 4, 2020
@robwalch robwalch deleted the bugfix/handle-wrapping-pts-timeoffset-calculation branch August 4, 2020 15:37
robwalch pushed a commit that referenced this pull request Aug 4, 2020
* upstream/master:
  Disable flakey smooth switch test on stream with large start gap
  Require min cue duration of 0.25
  Package lock update.
  Minimal Logging on Functional Tests.
  Handle DTS wrapping in initial AVC CTS calculation
  Apply PTSNormalize to ID3 and Text samples to handle timestamp rollover
  Add method to find start pts #2930
  Fix PTS calculation on rollover.
  Bump netlify-cli from 2.58.0 to 2.59.0
  Use ES5 in JavaScript executed by webdriver (for IE11)
  Fix chart fragment rendering issue
  Do not flush audio on audio track switch when both tracks are main variant streams #2837
  Bump @types/mocha from 8.0.0 to 8.0.1
  Bump @babel/plugin-proposal-optional-chaining from 7.10.4 to 7.11.0
  Bump @babel/core from 7.10.5 to 7.11.0
  Improve demo video size menu
  Clear source buffers from chart when new ones are created
  Show multiple demo test tabs using modifier key
  Configure streams for audio and video buffer descrepency in buffer length test
robwalch pushed a commit to jwplayer/hls.js that referenced this pull request Aug 6, 2020
* upstream_hls.js/master: (30 commits)
  Handle detach and attach media when calling loadSource with media attached
  Stop loading and reset selected audio track state when reloading a stream
  Add comment describing additional live start gap allowance
  Remove cue duration warning, as this is standard for represending event vs time-ranged based metadata in TextTracks that can be signaled during playback with activeCues change events
  Disable flakey smooth switch test on stream with large start gap
  Require min cue duration of 0.25
  Package lock update.
  Minimal Logging on Functional Tests.
  Handle DTS wrapping in initial AVC CTS calculation
  Apply PTSNormalize to ID3 and Text samples to handle timestamp rollover
  Add method to find start pts video-dev#2930
  Fix PTS calculation on rollover.
  Bump netlify-cli from 2.58.0 to 2.59.0
  Use ES5 in JavaScript executed by webdriver (for IE11)
  Fix chart fragment rendering issue
  Do not flush audio on audio track switch when both tracks are main variant streams video-dev#2837
  Bump @types/mocha from 8.0.0 to 8.0.1
  Bump @babel/plugin-proposal-optional-chaining from 7.10.4 to 7.11.0
  Bump @babel/core from 7.10.5 to 7.11.0
  Improve demo video size menu
@robwalch robwalch modified the milestones: 0.15.0, 0.14.8 Aug 7, 2020
alangdm pushed a commit to alangdm/hls.js that referenced this pull request Sep 15, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants