-
Notifications
You must be signed in to change notification settings - Fork 45
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
support frame side data #270
Comments
Can you give us examples of what kind of data you might use this for? We already have a system for associating arbitrary/predefined data, it's called Tags, and I'm hesitant to add yet another one. Of course the granularity of tags is worse at the moment as you can only associate them down to the track level but not the frame level. |
Tags are not suitable for info per frame, more adapted for info per track as a whole, am I wrong? Example of side metadata per frame: EBU Tech 3349, or Cooke protocol (summary in MediaInfo library). A transcoder would be requested to transfer such metadata as is when compressing the video part. |
Thanks for the example, Jérôme. Yeah, it makes sense having it interleaved together with the frame data. We would need something like this, I guess:
It's similar to the tagging system but simplified. Of course we could make it more complicated, the structure deeper (e.g. one master inside the A |
Thinking if perhaps rather than a new Master Element, if |
I add my praise to this feature request. Is work on this currently stalled? |
thank you @ceztko, yes it's apparently stalled but I'm still very interested in it and it's been on my mind to complete this part. Do you mind sharing what interests you about the request or how you would envision using such a feature? |
My use case would be re-encoding a WebRTC video stream and add an absolute NTP capture timestamp to each frame. NTP timestamps differ from a presentation timestamp in the sense it may (and will) have different time base and different starting value and can be used to synchronize with external/stream unrelated events.
The proposal from @mbunkus looks good to me but I would simplify it even more by having just a [1] https://github.com/FFmpeg/FFmpeg/blob/ab4795a085cd3deacb5e4bbaf527d66171361024/libavutil/frame.h#L581 |
I suggest a method to transit/store frame side metadata in Matroska (arbitrary or pre-defined). There is a similar mechanism defined in nut (see the 'sm_data / side_data / meta_data' section of http://ffmpeg.org/~michael/nut.txt). I looked at BlockAdditional but it doesn't seem to provide enough structure for this purpose.
The text was updated successfully, but these errors were encountered: