Byte Range Access to non-complete Segments

[haudiobe]

A problem was identified when developing the conformance software. A proposed solution is documented here: https://members.dashif.org/wg/DASH/document/4449?downloadRevision=4466

[acbegen]

If there is no resync, I don't think there is any problem as the clients will not be sending any partial GET requests (I don't see a point why they would).
If there is a sync, naturally, we are implying that the chunks will sequentially become available and be sent to the client over time. They are not necessarily produced at constant time intervals and even if they were, the byte ranges would normally vary since chunks are of different sizes. I think that @dl values will take care of that. We can make the clarifications on slide 11, but I don't think we can assume slide 10 will hold (That is, the promise does not necessarily hold).
See https://tools.ietf.org/html/rfc7233#section-4.4. Response code 416 is the correct one when the requested range is not available on the server, but the RFC says clients should not depend on this response code. Furthermore, if one asks 100-200 but only 0-150 is available, most servers return 100-150 with 200 response code.
So, I did not get the problem in the conformance software.

[RufaelDev]

another point to keep in mind for byte range on non complete segments,
the inconsistent behaviour resulting from this would be even more confusing in practical systems where one or more proxies are used in between the content server and client, that each may have different ways of handling and different numbers of bytes available, this would make debugging quite challenging. Also such range requests may be hard to cache

regarding slide 10, this should be the recommended behaviour, but @acbegen mentioned cannot be guaranteed in practice as delay happens during the delivery, and may also not be produced at constant rate. I am also wondering how checking the timing of intermediate chunks would happen and what recommended client and server behaviour should be.

perhaps a recommendation on chunk production, like in slide 10
in combination with @availabilityTimeOffset and Latency@target could be sufficient

[haudiobe]

Responding to @acbegen

If there is no resync, I don't think there is any problem as the clients will not be sending any partial GET requests (I don't see a point why they would).

They could, why not. Is it permitted? It is nevertheless the least important point.

If there is a sync, naturally, we are implying that the chunks will sequentially become available and be sent to the client over time. They are not necessarily produced at constant time intervals and even if they were, the byte ranges would normally vary since chunks are of different sizes. I think that @dl values will take care of that. We can make the clarifications on slide 11, but I don't think we can assume slide 10 will hold (That is, the promise does not necessarily hold).

I agree that we cannot expect anything, but if you produce constant duration chunks (and there is no reason to no do it), then you can tell the client that you doing so.

See https://tools.ietf.org/html/rfc7233#section-4.4. Response code 416 is the correct one when the requested range is not available on the server, but the RFC says clients should not depend on this response code. Furthermore, if one asks 100-200 but only 0-150 is available, most servers return 100-150 with 200 response code.

This is good and we should add so. The conformance software typically gets into a mode where it asks for 151 onwards, i.e. s.t. that is not available at all. See the reason in the slides why this is the case.

So, I did not get the problem in the conformance software.

The conformance software downloads only high level boxes, not mdat except for the first few bytes. This is done to make it real-time capable. No media streams. As you start downloading chunked content, you need to do moof/mdat. But you hit a point where the moof is not available as it is still to be produced.

[haudiobe]

Responding to @RufaelDev

another point to keep in mind for byte range on non complete segments,
the inconsistent behaviour resulting from this would be even more confusing in practical systems where one or more proxies are used in between the content server and client, that each may have different ways of handling and different numbers of bytes available, this would make debugging quite challenging. Also such range requests may be hard to cache

I agree, but we need to be consistent on what is permitted.

regarding slide 10, this should be the recommended behaviour, but @acbegen mentioned cannot be guaranteed in practice as delay happens during the delivery, and may also not be produced at constant rate. I am also wondering how checking the timing of intermediate chunks would happen and what recommended client and server behaviour should be.

The behaviour is checked by one of the two means

• if possible, send a byte range request to get the next moof. If the server does not support, we may need to do differently
• send a 200 and monitor the incoming chunks in the open request. If the access throughput is high enough, that provides a good idea of the promise. Bad thing that you likely can only do this for the lowest bitrate.

perhaps a recommendation on chunk production, like in slide 10
in combination with @availabilityTimeOffset and Latency@target could be sufficient

I believe that everything is very simple. I added the proposal based on the e-mail below. This is fully aligned with the slides.

[haudiobe]

In summary such that you do not have to read the slides:

• If the MPD says @availabilityTimeComplete=False and Resync@dT is present, then the Content Provider promises that chunks are sent progressively with @dT pacing
• If the MPD says @availabilityTimeComplete=False and Resync@dT is present, then the Content Provider promises in addition that byte ranges can be accessed progressively with the range growing by @dImin.
• If the MPD says @availabilityTimeComplete=False and Resync@dT is not present, then data can be sent progressively with @dT pacing, but the client cannot rely on this. We can now say that we change this requirement for the server, but generally that seems not appropriate. We could add another signal for this. Whatever you like.

A client getting an MPD with @availabilityTimeComplete=False and Resync@dT is not present

• Can access the Segment earlier and hope that the data gets delivered in time.
• Cannot expect that it can access byte ranges and get a valid response, but could well be.

A client getting an MPD with @availabilityTimeComplete=False and Resync@dT is present

• Can rely that data is provided progressively with chunk duration @dT
• Can issue byte range requests to the initial part of the segment

For the conformance software, it would now check the following:

• With increasing time and according to @dT, it checks that the data is indeed progressively made available.

If Resync is not a good signal for this, we can add yet another signal, or we can add a flag in Resync that permits this. I want to avoid more and more signaling.

I believe that FFMPEG already does this correctly, so all good.

I want that the conformance software is properly implemented and I am spending many hours to get this fixed. But conformance can only implement to check well written requirements, even if the behavior is obvious.

[haudiobe]

Also the issue need on the CDN needs to be checked for this. We know that existing CDNs have this problems. While the Segment is still loaded, it is typically not supported.
http://mediapm.edgesuite.net/will/dash/lowlatency/test-for-range-requests-of-CTE-segments.html?url=https://cmafref.akamaized.net/cmaf/live-ull/2009972/test9/out.mpd

[RufaelDev]

@haudiobe the signalling pacing in principle, this is ok, but it is testing this when not using range request is still not clear to me, as there is no behavior defined for the client and results will be different on different clients based on the network/server configuration. I would recommend, if this is important, the additional signalling, such as supplemental property, complementary to resync, could be used to indicate the reference behavior.

[haudiobe]

Consider to add a segment sequence
18.m4s
18_1.m4s (SAP=1,2)
(Thomas) propose such a solution proposal