Setting the minimum ID3 cue length to 0.25 seconds breaks frame metadata use case

[bxthomas]

PR #2951, introduced a minimum ID3 cue length of 0.25 seconds. However, ID3 tags can be used to hold user metadata that may describe data specific to a particular video frame. By imposing this artificial limitation, we can no longer correlate ID3 cues to video frames, and this breaks useful functionality.

[itsjamie]

So the underlying reason for the original PR as explained to me was that the TextTrack events would sometimes not fire because the ID3 tags had such short durations they were never part of the activeCues properties.

@bxthomas it would be useful to know, what are you using to detect the active ID3 tag on the web platform for ones where your needing sample level accuracy so we can make sure to test that in the future (include a unit test) where the current time doesn't advance at sample-level accuracy.

As a follow-up based on that, I'm curious if having two properties, a "realDuration" and a fake "duration" intended for the web API to enable "cuechange" based usage would be nice, or if @bxthomas simply has a better answer 👍

[robwalch]

So the underlying reason for the original PR as explained to me was that the TextTrack events would sometimes not fire because the ID3 tags had such short durations they were never part of the activeCues properties.

In this case, it was always with cues found on the last sample, where cue duration was being set to 0.0001. The change to make the minimum for all cues 0.25 followed the thinking that browsers still may trigger "timeupdate" events as infrequently as 4 times a second, and that active cue changes only occur on time updates when currentTime intersects with a cue's time range. It was a defensive change that didn't consider use cases where the sample time based duration was critical. And, I'm to blame for steering @jnatalzia in that direction with peer feedback on the fix.

@bxthomas it would be useful to know, what are you using to detect the active ID3 tag on the web platform for ones where your needing sample level accuracy so we can make sure to test that in the future (include a unit test) where the current time doesn't advance at sample-level accuracy.

I would like to know more as well. For now though I am happy to provide a patch that should work for either use-case with #2964.

[tpaszun]

@bxthomas it would be useful to know, what are you using to detect the active ID3 tag on the web platform for ones where your needing sample level accuracy so we can make sure to test that in the future (include a unit test) where the current time doesn't advance at sample-level accuracy.

Hi all, I'll answer for Brian. Our use case is that we send frame metadata in ID3 tags (e.g. timecode). And then we do not use activeCues to access cues, we do a binary search for the right cue from cues instead (we use startTime and endTime properties to find the active cue).

[robwalch]

Thank you @bxthomas and @tpaszun for creating this issue and providing feedback.

Please let me know if this build https://deploy-preview-2964--hls-js-dev.netlify.app/demo/ of #2964 addresses this issue. I will cut a patch once confirmed. Thanks!

[robwalch]

@bxthomas ^

[robwalch]

https://github.com/video-dev/hls.js/releases/tag/v0.14.9