Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Save-to-webm: use timestamp from samplebuilder #130

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

tmatth
Copy link
Contributor

@tmatth tmatth commented Sep 30, 2022

This avoids desyncing by accumulating drift.

Description

In my own application I've been able to force audio and video to get badly out of sync by interfering with the source (causing dips in FPS etc., I don't fully grok the desync, but I can reproduce it reliably), which would cause the mkv written to disk to be badly out of sync. This change uses the sambuilder packet timestamp directly - the starting offset instead of accumulating durations.

Some caveats:

  • the starting offset logic is a bit naive, it assumes that the tracks start in sync
  • it's possible I should be using some external time info (i.e. from RTCP) that I'm not aware of instead
  • i'm not handling timestamp wrap
  • a more sophisticated approach would be like what galene is doing here: jech/galene@db21575

@tmatth tmatth marked this pull request as draft September 30, 2022 11:48
@tmatth
Copy link
Contributor Author

tmatth commented Sep 30, 2022

If/when this gets merged I can do the same for the Twitch example.

@tmatth
Copy link
Contributor Author

tmatth commented Sep 30, 2022

Tweaked to use sample.PacketTimestamp directly instead of PopWithTimestamp after @atoppi pointed it out.

This avoids desyncing by accumulating drift.
Copy link
Member

@at-wat at-wat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basic idea looks good.
I think PacketTimestamp overflow should be handled.

In the event that the packet timestamp wraps past uint32max, cast to
int64 and add uint32_max so that the delta (and thus the webm timestamp)
stays positive.
@tmatth
Copy link
Contributor Author

tmatth commented Nov 12, 2022

Basic idea looks good. I think PacketTimestamp overflow should be handled.

Hopefully something like aca9a01 is what you had in mind, I would squash before merging I just wanted the change to accommodate overflow to be clear.

@tmatth tmatth marked this pull request as ready for review November 12, 2022 13:11
Instead we use the sample.Duration (as before) for the first packet
only, subsequent packets will be computed as a delta.
Copy link
Member

@at-wat at-wat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm busy for next two weeks later, so please ask anyone else for reviewing if need to hurry.

As it's an example, there is an option to abort the program when reaching the point it can't handle timestamp correctly, and keep the example code simple.

Comment on lines +77 to +82
timestampSinceStart := int64(sample.PacketTimestamp) - int64(s.startAudioOffset)
// handle range where PacketTimestamp has wrapped past uint32 by operating in int64 range until the timestamp has caught up to the offset
if timestampSinceStart < 0 {
timestampSinceStart = int64(sample.PacketTimestamp+math.MaxUint32) - int64(s.startAudioOffset)
}
ts = timestampSinceStart / 48 // convert from RTPTime to sample time
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it work after the second overflow of packet timestamp?

@tmatth
Copy link
Contributor Author

tmatth commented Mar 10, 2023

Sorry for the radio silence, I finally circled back to this and I'm tempted to drop it as in my own application, it is not fixing the problem it should be.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants