Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Using playback rate to help reduce rebuffering and maintain buffer health. #6131

Open
gkatsev opened this issue Jan 19, 2024 · 6 comments · May be fixed by #6193
Open

Using playback rate to help reduce rebuffering and maintain buffer health. #6131

gkatsev opened this issue Jan 19, 2024 · 6 comments · May be fixed by #6193
Assignees
@gkatsev
Labels
priority: P3 Useful but not urgent type: enhancement New feature or request
Milestone
v4.9

Comments

@gkatsev
Copy link
Contributor

gkatsev commented Jan 19, 2024

Have you read the FAQ and checked for duplicate open issues?
Yes, there aren't existing issues.

Is your feature request related to a problem? Please describe.

We would like to further reduce the likelihood of rebuffering and increase buffer health by tweaking the playback rate of the video.

Describe the solution you'd like

This involves 3 changes that are all related.

  • For VOD
    • streaming.vodDynamicPlaybackRate: when enabled, begin playback at a low playback rate (0.95x, configurable) and when the buffer is high, increase rate to 1x, if the buffer is low, decrease back to 0.95x.
  • For live
    • streaming.liveSyncPanicMode: if enabled, When the player rebuffers, enter "panic" mode, which sets the player to the liveSyncMinPlaybackRate for a specified duration, say 60s by default.
    • streaming.liveSyncDynamicTargetLatency: if enabled, when the playback is progressing smoothly, move the target latency closer to the min latency, and if playback is having issues move it back towards the max latency.

Describe alternatives you've considered

It's possible to introduce other ABR algorithms, but this gives us another tool to maintain buffer health for both live and vod content and help reduce rebuffering for users.

Additional context

The first two features, dynamic playback rate for vod and panic mode for live sync, can both be implemented as extensions on top of what's currently there. Though, if there are pointers on how to approach this that would fit the codebase better, that would be great!
For example, for the panic mode, we'd need to store when the last rebuffering event happened so that we can check against it in the onTimeUpate_ method in player.js, where should that var be stored? And for the vod dynamic playback rate, should the play rate controller own this functionality?

The third feature, dynamic target latency, is a bit trickier. Currently, live sync is set up by looking at the min and max latencies (either configured or received via DASH ServiceDescriptions) and then playback rate is set up based on that. However, for this feature, we'd need to have a third value, target latency -- which the DASH LL Spec does define, actually -- but switching from a min/max latency to a target latency would potentially be a breaking change or would potentially duplicate a lot of code (since we'd want to fork the onTimeUpdate_ method depending on whether dynamic target latency is enabled or not).
While, the DASH spec, technically, says that the target latency is the only required one, I'm not sure if it's necessarily worth migrating to, given the risk just outlined. However, if y'all have thoughts on how to do that or any other alternatives, it would be nice as this feature can help reduce rebufferings.
@gkatsev gkatsev added the type: enhancement New feature or request label Jan 19, 2024
@shaka-bot shaka-bot added this to the Backlog milestone Jan 19, 2024
@avelad
Copy link
Collaborator

avelad commented Jan 22, 2024

I've been thinking about this and I think we need to split it into 3 PRs to make it easier to review and merge.

Some comments:

streaming.vodDynamicPlaybackRate: when enabled, begin playback at a low playback rate (0.95x, configurable) and when the buffer is high, increase rate to 1x, if the buffer is low, decrease back to 0.95x.

I see this functionality as perfect, when we have a high buffer level I think we should calculate it as a percentage of the bufferingGoal value that we have configured.
Example: config.vodDynamicPlaybackRateBufferPercentage * config.bufferingGoal
We have to add the vodDynamicPlaybackRateBufferPercentage configuration

streaming.liveSyncPanicMode: if enabled, When the player rebuffers, enter "panic" mode, which sets the player to the liveSyncMinPlaybackRate for a specified duration, say 60s by default.

I see the functionality correct, but the time value must be configurable.

For example, for the panic mode, we'd need to store when the last rebuffering event happened so that we can check against it in the onTimeUpate_ method in player.js, where should that var be stored?

The last rebuffering time I think should be stored in shaka.media.BufferingObserver

And for the vod dynamic playback rate, should the play rate controller own this functionality?

It doesn't need to know about this functionality because already it doesn't know anything about liveSync.

streaming.liveSyncDynamicTargetLatency: if enabled, when the playback is progressing smoothly, move the target latency closer to the min latency, and if playback is having issues move it back towards the max latency.

Regarding this functionality I have many doubts, and before I can continue moving forward with the definition, how do you know if the playback has issues? (the core part that would enable this functionality).

@avelad avelad added the priority: P3 Useful but not urgent label Jan 22, 2024
@gkatsev
Copy link
Contributor Author

gkatsev commented Jan 22, 2024

I've been thinking about this and I think we need to split it into 3 PRs to make it easier to review and merge.

Yeah, makes sense to have them as 3 PRs.

streaming.vodDynamicPlaybackRate: when enabled, begin playback at a low playback rate (0.95x, configurable) and when the buffer is high, increase rate to 1x, if the buffer is low, decrease back to 0.95x.

I see this functionality as perfect, when we have a high buffer level I think we should calculate it as a percentage of the bufferingGoal value that we have configured. Example: config.vodDynamicPlaybackRateBufferPercentage * config.bufferingGoal We have to add the vodDynamicPlaybackRateBufferPercentage configuration

Is the idea to set the playback rate to a percentage as a percentage of the bufferingGoal? so the closer we are to the bufferingGoal the closer to a playback rate of 1?

streaming.liveSyncPanicMode: if enabled, When the player rebuffers, enter "panic" mode, which sets the player to the liveSyncMinPlaybackRate for a specified duration, say 60s by default.

I see the functionality correct, but the time value must be configurable.

Yeah, definitely needs to be configurable.

For example, for the panic mode, we'd need to store when the last rebuffering event happened so that we can check against it in the onTimeUpate_ method in player.js, where should that var be stored?

The last rebuffering time I think should be stored in shaka.media.BufferingObserver

And for the vod dynamic playback rate, should the play rate controller own this functionality?

It doesn't need to know about this functionality because already it doesn't know anything about liveSync.

Sounds good!

streaming.liveSyncDynamicTargetLatency: if enabled, when the playback is progressing smoothly, move the target latency closer to the min latency, and if playback is having issues move it back towards the max latency.

Regarding this functionality I have many doubts, and before I can continue moving forward with the definition, how do you know if the playback has issues? (the core part that would enable this functionality).

Yeah, what I wrote is a bit vague. I believe the idea is also to use rebuferring events, but let me gather more info and write a more detailed explanation.

@avelad
Copy link
Collaborator

avelad commented Jan 22, 2024

streaming.vodDynamicPlaybackRate: when enabled, begin playback at a low playback rate (0.95x, configurable) and when the buffer is high, increase rate to 1x, if the buffer is low, decrease back to 0.95x.

I see this functionality as perfect, when we have a high buffer level I think we should calculate it as a percentage of the bufferingGoal value that we have configured. Example: config.vodDynamicPlaybackRateBufferPercentage * config.bufferingGoal We have to add the vodDynamicPlaybackRateBufferPercentage configuration

Is the idea to set the playback rate to a percentage as a percentage of the bufferingGoal? so the closer we are to the bufferingGoal the closer to a playback rate of 1?

Example:
playbackRate 0.95(buffer is low), if the buffer we have is less than 25% of bufferingGoal
playbackRate 1 (buffer is high), if buffer is greater than 25% of bufferingGoal

@gkatsev
Copy link
Contributor Author

gkatsev commented Jan 23, 2024

streaming.liveSyncDynamicTargetLatency: if enabled, when the playback is progressing smoothly, move the target latency closer to the min latency, and if playback is having issues move it back towards the max latency.

Regarding this functionality I have many doubts, and before I can continue moving forward with the definition, how do you know if the playback has issues? (the core part that would enable this functionality).

Yeah, what I wrote is a bit vague. I believe the idea is also to use rebuferring events, but let me gather more info and write a more detailed explanation.

So, the approach here is like so:
Keep track of the last time we reached the target latency (if we're within some threshold from the target, I think the ll dash spec recommends 500ms, but can be configurable). Then, periodically, check to see if the last time we've updated the playback rate back to 1 was longer than some stability_thershold, we can adjust the current target time forward towards the minimum latency. The move towards the min latency is done in two passes, so, if min is 1, and target is 5, it'll first set target to 3, then the second time around it'll be set to 1.
Additionally, on rebuffer, move the current target latency back towards max target latency by an increment of latency_increase_on_rebuffer(500ms by default) per rebuffer count. So, on first rebuffer, add 500ms to the current target latency. On 5th, add 500*5, or 2500 (and clamp to max).
The current target latency will also be clamped to the min/max values (probably accounting for the threshold).

Hopefully, that makes sense.

@gkatsev gkatsev mentioned this issue Jan 23, 2024
Merged
avelad pushed a commit that referenced this issue Jan 25, 2024
@gkatsev
feat: add a live sync panic mode (#6149)
65981e2 
This PR introduces a live sync panic mode
(`streaming.liveSyncPanicMode`) which sets the player into the
`streaming.liveSyncMinPlaybackRate` while we're within the
`streaming.liveSyncPanicThreshold`. This should help reduce the change
of subsequent rebuffering events by moving further away from the live
edge.

Related to #6131.
@gkatsev
Copy link
Contributor Author

gkatsev commented Jan 25, 2024

@avelad hey, did you get a chance to think about the dynamic target latency? Thanks!

@avelad
Copy link
Collaborator

avelad commented Jan 25, 2024

@gkatsev I've reviewed it and it makes sense. I'm OK with implementing that.

@gkatsev gkatsev mentioned this issue Jan 26, 2024
Merged
@avelad avelad modified the milestones: Backlog, v5.0 Jan 29, 2024
@gkatsev gkatsev linked a pull request Jan 31, 2024 that will close this issue
Open
avelad pushed a commit that referenced this issue Feb 2, 2024
@gkatsev
feat: vod dynamic playback rate buffer control (#6172)
8fc292b 
Dynamically update playback rate to keep the buffer full.

Related to #6131
@avelad avelad modified the milestones: v4.8, v4.9 Apr 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@gkatsev gkatsev

Labels
priority: P3 Useful but not urgent type: enhancement New feature or request
Projects
None yet
Milestone
v4.9
3 participants
@gkatsev @avelad @shaka-bot