Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for DASH patch manifests (<PatchLocation>) #2228

Open
joeyparrish opened this issue Nov 7, 2019 · 22 comments · May be fixed by #5247
Open

Add support for DASH patch manifests (<PatchLocation>) #2228

joeyparrish opened this issue Nov 7, 2019 · 22 comments · May be fixed by #5247
Labels
component: DASH The issue involves the MPEG DASH manifest format priority: P3 Useful but not urgent type: enhancement New feature or request
Milestone

Comments

@joeyparrish
Copy link
Member

Have you read the FAQ and checked for duplicate open issues?
Yes

Is your feature request related to a problem? Please describe.

Manifest updates can be expensive in some cases.

Describe the solution you'd like

DASH has now introduced <PatchLocation> and <Patch>, which give a way to update a small part of the existing manifest instead of re-parsing and merging every time.

Additional context

TODO: Get references to the spec on this (AMD.1, landing in 4th edition according to Hulu)

TOTO: Get sample content for this (should be coming soon to the DASH live sim, according to Hulu)

@joeyparrish joeyparrish added the type: enhancement New feature or request label Nov 7, 2019
@joeyparrish joeyparrish added this to the Backlog milestone Nov 7, 2019
@sr1990
Copy link
Contributor

sr1990 commented Dec 27, 2019

Hi @joeyparrish, Any idea where I can find spec and sample content for this one? Thanks.

@joeyparrish
Copy link
Member Author

I was told that this is part of Amendment 1 to the 3rd edition of the DASH spec (which has the official designation "ISO/IEC 23009-1:2014/AMD 1:2015", and is also rolled into the 4th edition of the DASH spec (which has the official designation "ISO/IEC 23009-1:2019".

Sadly, I don't have a publicly available PDF for the spec. It would be nice if there were a public doc, but ISO will sell you a PDF for 198 Swiss Francs (currently about 203 USD or 182 EUR). The "preview" that ISO has available only contains the table of contents, and not even the entire table. :-(

@joeyparrish joeyparrish modified the milestones: Backlog, Backlog 2 Jan 28, 2020
@technogeek00
Copy link

@joeyparrish Sorry for the long delay here, wanted to get you links on the final text and some other aspects kept iterating keeping it from vote, but you can find the final 4th and amd1 over on https://github.com/cta-wave/Resources#specification-development-2

@acheung-harmonicinc
Copy link

Hi, my buddy and I are checking if we can have DASH patch manifest support based on Shaka or have some preliminary implementation. May I know if you have any plan to start it?

@zuzzurro
Copy link

We are extremely interested in this functionality because of the traffic reduction it can bring with large DVR windows. Can you share what your plans are for adding support for it?

@TheModMaker
Copy link
Contributor

We have not started any work on this issue. It is currently on the Backlog, so we have no plans to implement it for the next major release. If this is a priority for someone, we'd be happy to review a PR for this. Since this is a larger feature, you should post a design first (e.g. using a Google doc).

@joeyparrish joeyparrish added the priority: P3 Useful but not urgent label Sep 29, 2021
@avelad avelad added the component: DASH The issue involves the MPEG DASH manifest format label May 2, 2022
@dave-nicholas
Copy link
Contributor

Hi guys @joeyparrish @TheModMaker.
We are currently looking into this as part of an upcoming requirement.
We want to approach this work in a way that would allow us to upstream these changes.
I have made a doc with our proposed implementation.
Your guidance and input would be most welcome.

https://docs.google.com/document/d/1Z3GkKUe9mM4nub1w_WlP3Bva5oNtlywlzRxTuui_Yac/edit?usp=sharing

@technogeek00
Copy link

technogeek00 commented Dec 16, 2022

Chiming in to drop a link to the finalized text in the 5th edition which is freely available on ISO: https://standards.iso.org/ittf/PubliclyAvailableStandards/c083314_ISO_IEC%2023009-1_2022(en).zip

I'd defer to Shaka team on the player semantics in the proposal, but left some comments on the path selection understanding as there are explicit path addressing restrictions that should simplify implementations and must be followed for a compliant implementation:

5.15.3.4 Element Addressing Restrictions
The following element addressing restrictions apply for DASH MPD elements:

  • Elements with an @id attribute with uniqueness requirement among its siblings present shall only be identified by the value of this attribute. In particular, Period, AdaptationSet, Representation, and SubRepresentation shall be only addressed via their @id.
  • DescriptorType elements with an @schemeIdUri attribute shall be identified by the value of this attribute, for example SupplementalProperty.
  • S elements shall be addressed in one of the following ways:
    • @t or @n attribute of S elements if the said attribute is present in the S element.
    • by position.
  • All other elements shall be addressed by position, e.g. SegmentTemplate[1].

Examples:

/MPD/Period[@id="1"]/AdaptationSet[@id="1"]/SegmentTemplate[1]/SegmentTimeline[1]/S[@t="12345"]

/MPD/Period[@id="1"]/AdaptationSet[@id="1"]/SegmentTemplate[1]/SegmentTimeline[1]/S[42]@r

I'm happy to review a PR for standards compliance semantics when it is ready.

@dave-nicholas
Copy link
Contributor

@technogeek00 Thank you greatly for your input, we are in the process of trying to get an AdapatationSet@id added to our streams. I am in the currently creating a POC that I will present for review once ready.

@zuzzurro
Copy link

Just out of curiosity, @dave-nicholas, were you able to make any progress on this?

@dave-nicholas
Copy link
Contributor

Hi @zuzzurro, We are currently waiting on some other work from our organisation to be merged before opening a PR with these changes, as the MPD Patch work is dependent on this.

@acheung-harmonicinc
Copy link

acheung-harmonicinc commented Apr 27, 2023

Hi, it may be a bit late.

We have a fork with patch manifest support
https://github.com/harmonicinc-com/shaka-player/commits/feature/dash-patch/v4.0.x

Please let me know if you are interested, then we can create a PR.

@zuzzurro
Copy link

zuzzurro commented May 4, 2023

As I said previously (in 2021, how time flies...) we are extremely interested in this feature. And the sooner we have it in Shaka, the better. How is your implementation different from the one @dave-nicholas is working on, @acheung-harmonicinc ?

@dave-nicholas
Copy link
Contributor

@zuzzurro I hope the open this PR by early next week.

Our approach differs from @acheung-harmonicinc 's implementation in that we do not re-parse the entire manifest each time, some of our full events can have a size up to 11 hours and re-parsing a manifest that size on a low - mid end device would not be a solution for us (hence the dash manifest parser improvements were brought into scope for us first). We instead parse and process the patch only.

@zuzzurro
Copy link

zuzzurro commented May 9, 2023

Well, I totally agree with your goal, and my question was mostly for @acheung-harmonicinc , in order to understand better the pros and cons of both implementations.. Clearly I expect that using patch manifests should bring down the requirements for low end devices. If not, why bother?

@dave-nicholas dave-nicholas linked a pull request May 25, 2023 that will close this issue
@zuzzurro
Copy link

zuzzurro commented Jul 6, 2023

@dave-nicholas , out of curiosity, do you have a list of vendors that generate DASH streams compatible with your patch?

@zuzzurro
Copy link

What is the status of this issue? It looks like the pull request from @dave-nicholas went nowhere? Inquiring minds want to know..

@dave-nicholas
Copy link
Contributor

Hi @zuzzurro , @joeyparrish is on leave at the moment but will review the PR when he returns.

@zuzzurro
Copy link

zuzzurro commented Dec 13, 2023

We can hardly wait for this to happen.. Also, @dave-nicholas , do you have an answer for my previous question about vendors supporting this stream format?

@dave-nicholas
Copy link
Contributor

@zuzzurro I don't unfortunately. We are testing Patch internally but don't currently have any public streams yet.

@technogeek00
Copy link

I found out there is an NGINX module that can generate patches: https://github.com/ab2022/mpddiffs Not quite sure on the setup requirements but maybe a possibility for you.

@tobbee
Copy link

tobbee commented Apr 23, 2024

There is now MPD Patch support in DASH-IF livesim2. See https://github.com/Dash-Industry-Forum/livesim2/wiki/MPD-Patch for more information. It's intended to work with all live MPDs that can be produced including multi-period live MPDs.

The official DASH-IF instance at https://livesim2.dashif.org has also been updated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: DASH The issue involves the MPEG DASH manifest format priority: P3 Useful but not urgent type: enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants