Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inband Timed Metadata for FMP4 (ID3 within Emsgv1) #2360

Closed
damienlevin opened this issue Aug 30, 2019 · 16 comments · Fixed by #4458
Closed

Inband Timed Metadata for FMP4 (ID3 within Emsgv1) #2360

damienlevin opened this issue Aug 30, 2019 · 16 comments · Fixed by #4458

Comments

@damienlevin
Copy link

HLS JS should surface inband Timed Metadata for FMP4 (ID3 within Emsgv1).

The full AOM spec can be found here:
https://aomediacodec.github.io/av1-id3/

Apple documentation:
https://developer.apple.com/documentation/http_live_streaming/about_the_common_media_application_format_with_http_live_streaming

You can include one or more Event Message boxes (‘emsg’) in each segment. You must use version 1 of the Event Message box standard found in ISO-23009-1. The following values define the semantics:

scheme_id_uri
Set to https://aomedia.org/emsg/ID3 to identify boxes that carry ID3v2 metadata.

value
The URI that defines the semantics of the ID field. Any relative URI is considered to be relative to scheme_id_uri.

message_data
Contains data compatible with ID3 version 2.x.x.

Happy to provide sample HLS playlists if needed.

@johnBartos johnBartos self-assigned this Aug 30, 2019
@johnBartos johnBartos added this to the 1.0.0 milestone Aug 30, 2019
@johnBartos
Copy link
Collaborator

@damienlevin Was made aware of this recently - thanks for filing an issue. Happy to try and squeeze this in for 1.0.0

@damienlevin
Copy link
Author

That would be amazing, thanks !

@johnBartos
Copy link
Collaborator

johnBartos commented Aug 30, 2019 via email

@damienlevin
Copy link
Author

FYI, Apple players don't have support for the scheme mentioned in the links I provided (At least ATM. They do however support https://developer.apple.com/streaming/emsg-id3 ).
I would suggest supporting both https://aomedia.org/emsg/ID3 and https://developer.apple.com/streaming/emsg-id3

See:
https://forums.developer.apple.com/thread/118860

Here is a stream with an AOM scheme:
https://storage.googleapis.com/dlevin/tmp/fmp4-emsg-id3/1567191409/master.m3u8

@damienlevin
Copy link
Author

Hey @johnBartos,
Any ETA for that feature ?
Thanks

Damien

@robwalch
Copy link
Collaborator

robwalch commented Mar 3, 2020

@damienlevin Thanks for submitting the feature request.

Can you ask for AOM scheme support in Safari? 😉

This would be a great feature for v1.0.0, but I can't promise it will make the cut.

@gogonowski
Copy link

gogonowski commented Jun 23, 2020

We originally submitted the request for ES (Elementary Stream ADTS) id3v2.4 metadata a few years ago. Now we move on to fMP4.

Here are some audio-only fMP4 URLs from the StreamS Live HLS Encoder, complete with compliant emsg id3v2.4 metadata. This has all been verified to be correct. They play and display in iTunes correctly, with the exception of the new xHE-AAC at the bottom of the list. Those require macOS 11 beta and Safari.

AAC-LC 320kbps fMP4 HLS Streams
https://www.maxradio.ca/UHD/CKLGfMP4/320k/CKLG.m3u8
https://www.maxradio.ca/UHD/LG73fMP4/320k/LG73.m3u8
https://www.maxradio.ca/UHD/MaxfMP4/320k/Max.m3u8

HE-AACv1 80kbps fMP4 HLS Streams
https://www.maxradio.ca/UHD/CKLGfMP4/80k/CKLG.m3u8
https://www.maxradio.ca/UHD/LG73fMP4/80k/LG73.m3u8
https://www.maxradio.ca/UHD/MaxfMP4/80k/Max.m3u8

These are new xHE-AAC streams, verified to play native in macOS 11 beta Safari.
emsg id2v2.4 metadata is the same.

xHE-AAC 40kbps fMP4 HLS Streams
https://www.maxradio.ca/UHD/CKLG/40k/CKLG.m3u8
https://www.maxradio.ca/UHD/LG73/40k/LG73.m3u8
https://www.maxradio.ca/UHD/MaxRadio/40k/Max.m3u8
https://www.maxradio.ca/UHD/NewWestRock/40k/NewWestRock.m3u8
https://www.maxradio.ca/UHD/UptownRadio/40k/Uptown.m3u8

We are planning on making a Metadata Timing Test available for fMP4 very soon.
ES (ADTS Elementary Stream) Version is here:
https://la2.indexcom.com/player/pad/hls/program.m3u8

Please let me know if you need any other information on this. Happy to help.
/greg.
StreamS

@johndeu
Copy link

johndeu commented Dec 11, 2020

Hey @johnBartos - did the emsg v1 support ever make it in? Just checking as we are in need of the 64 bit timestamp support now

@gogonowski
Copy link

gogonowski commented Dec 11, 2020 via email

@robwalch
Copy link
Collaborator

Hi @gogonowski and @johndeu, you can direct these questions to me. ID3 within Emsgv1 is not present in feature/v1.0.0 and will not make it before beta. It will most likely come in a minor following the release of v1.0.0, but that depends on whether we get contributions before then. I will update the issue milestone accordingly once we reach beta in the coming weeks. You can read more about the release here:

#2861
#3072

We do have a couple of PRs that add CMAF CC support. I believe that work is related and adds some of the tooling necessary to extract EMSG ID3, so it's not far off: #2623

@robwalch
Copy link
Collaborator

robwalch commented Dec 14, 2020

There are ways you can help! Please provide sample HLS streams containing Emsgv1 data; what the metadata contains and when each starts and ends. Confirming that the metadata is exposed in TextTracks by Safari when played in a video element with <video src='stream'/> would also help us get it right in hls.js. Thanks!

@victor-at-work
Copy link

victor-at-work commented Jan 6, 2021

I have a PR for this ready for review: victor-at-work#1, but it depends on some of the utilities added in #3016. I'll rebase when #3016 has been merged.

@robwalch robwalch modified the milestones: 1.0.0, 1.1.0 Feb 22, 2021
@robwalch robwalch added this to To do in Metadata via automation May 20, 2021
@monyone
Copy link

monyone commented Jul 2, 2021

Hi @robwalch, Hasn't there been any progress on this issue?

I created fork for CMAF ID3 support without CMAF CC support from current master branch (v1.0.7).
https://github.com/monyone/hls.js/tree/feature/cmaf-id3 (monyone@f361dd9)
It based on @victor-at-work's cmaf-id3 branch without CMAF CC support and rebase from current master branch.

May I create PR from this branch?
Shulod I wait untill @victor-at-work's CMAF CC (v0.0.15) and CMAF ID3 (v1.1.0) has been merged?

@victor-at-work
Copy link

Hey @robwalch, I was planning on addressing @johndeu's comment on my PR, but if we're going to merge in @monyone's PR from v1 then I'll hold off from doing that, so please let me know :)

@gogonowski
Copy link

gogonowski commented Oct 25, 2021

HLS fMP4 Audio-Only with emsg id3v2.4 Timed Metadata Reference Streams

There has been much question on how this is to be implemented.
This is how you do it.

These streams have been produced with StreamS HLSdirect(tm) Live Encoder.

AAC-LC
https://la2.indexcom.com/player/pad/fmp4/hls/128k/program.m3u8

AAC-HEv1
https://la2.indexcom.com/player/pad/fmp4/hls/64k/program.m3u8

xHE-AAC
https://la2.indexcom.com/player/pad/fmp4/hls/40k/program.m3u8

FLAC - Lossless
https://la2.indexcom.com/player/pad/fmp4/hls/FLAC/program.m3u8

This can be viewed in action here:
https://la2.indexcom.com/player/eval
https://la2.indexcom.com/player/pad/fmp4

/greg.
StreamS

@robwalch robwalch modified the milestones: 1.1.0, 1.2.0 Nov 10, 2021
@nicoweilelemental
Copy link

Here is another live stream sample with ID3 tags over emsg v1 (in the video segments, not the audio ones), produced on AWS Elemental MediaPackage:
https://d24rwxnt7vw9qb.cloudfront.net/out/v1/62a7ec8a0f3b4f19ad76eac54f2f2dce/cmaf-clear/index.m3u8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Development

Successfully merging a pull request may close this issue.

8 participants