sound and rendering issues in an EPUB sample

[llemeurfr]

We've been offered an EPUB FXL sample featuring some js effects. It is stored at https://drive.google.com/file/d/1EVHnCXPCGGdGL5AXjVMCVp5cWorMAgNi/view?usp=sharing

It shows some issues:

• pages are rendered in two steps, which is not perfect.
• no sound is heard on the animation in page 3

Note that this EPUB plays well on iBooks

[danielweck]

pages are rendered in two steps

I am not able to reproduce this bug, but I think this is because my computer is fast enough to rapidly apply the CSS zoom transformation on the page?

[danielweck]

no sound is heard on the animation in page 3

I am able to reproduce this bug, and I can see why this occurs (this is related to the recent workaround for Electron's bug in its registerStream/HttpProtocol() API. I am looking into this...

[danielweck]

Oh, I see, there is a dynamic scrip-driven DOM mutation from:

<div data-audio-type="inline" data-autoplay="false" data-controls="false" data-loop="false" data-src="images/staubsauger.mp3" data-widget="AUDIO" id="box5-Audio11">

to:

<audio src="images/staubsauger.mp3" ... />

...so obviously it is not possible to statically discover the <audio> elements in order to patch their @src (or their child <source @src>).

This is probably a common pattern with authoring tools like InDesign, so a possible workaround is to discover the special syntax data-src (etc.). However, a proper fix in the readium-desktop SDK will be available once the Electron bug is fixed, see:
electron/electron#21018 (comment)

[danielweck]

pages are rendered in two steps

I am not able to reproduce this bug, but I think this is because my computer is fast enough to rapidly apply the CSS zoom transformation on the page?

Note that I am able to see a short flash or show/hide content when opening the HTML files directly in Chrome ... so there is obviously some delay in the authored Javascript. The reason why this is not visible / perceivable in some reading systems is because of rendering out-of-view / pre-rendering.

[JayPanoz]

This is probably a common pattern with authoring tools like InDesign

Looks like it’s a QuarkXPress output.

[danielweck]

I fixed the audio playback issue (dynamic DOM mutation creates audio element that is undetected by the reading system's transform which patches the src URLs to point to the HTTP server instead of the sandboxed per-publication URLs with custom protocol / scheme handler)

The fix consists in injecting a different <base href="HTTP_URL" /> in HTML documents served with the special protocol / scheme. I tested with fixed layout publications that save their own data in window.localStorage, and even though the <base> is present to dictate how relative URLs are resolved to absolute URLs, it has no impact on the sandboxing of localStorage, indexedDb etc. I assume this is by design, not a coincidence. So, this seems like a viable solution, I am publishing an update to the relevant R2 package (r2-navigator-js), and to Thorium (readium-desktop).

[llemeurfr]

The small flash at rendition time seems specific to this sample, which includes some XPress scripts, therefore we'll not investigate much more on this front.
The audio playback issue was more interesting, many thanks to Daniel for having found a solution so quickly.

[danielweck]

r2-navigator-js changelog: https://github.com/readium/r2-navigator-js/blob/develop/CHANGELOG.md#1152
...I am preparing a readium-deskop update which will close this issue.