New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Stream protocols and fs.createReadStream sometimes never complete in Electron 7 #21018
Comments
Also worth noting, these incomplete network requests don't happen if you read files into memory and create a stream from them: https://gist.github.com/e442a020541a0e7470abf603812159bb
|
Can confirm that I'm seeing the same behavior in my work |
I also found that @itsananderson's fix does work, but using into-stream with |
Same problem here; it seems like the responses randomly get cut off. May be related/similar to: #16213 |
This PR may be of interest: #19869 |
The proposed fix does not fix this problem for me. After upgrading electron from version 6 to 7.1.2, I noticed that images randomly did not load when they have As the original project is quite big, I created an example project based on itsananderson's gist which shows that images sometimes do not complete loading. Also, you can see that the Feel free to clone the example project, build and run the app. Please note that the bug does not appear when starting the app with My configuration:
|
...yep, looks like this isn't fixed after all. (Your example also reproduces for me when running |
Are there any updates on this? Or pointers towards where this regression might have originated? |
Looking into the commits between 7.0.0-beta.3 and 4: It looks like this commit could be the culprit: Pinging @codebytere |
I spent the day digging through this and I think I've found the cause, but I don't have a solution yet. The issue seems to occur only when responses exceed a certain size. In my tests, the failure would coincide with a I can't say for certain that's the correct diagnosis, but it seems logical that it would be a backpressure issue given that it only manifests on larger responses. I believe this should be handled by watching the producer for available capacity and then trying the write. I'm going to spend a little more time this evening trying to implement that, but I'm fairly unfamiliar with the Electron/Chromium codebase so I may need to punt it to the pros. |
UPDATE - Unfortunately I'm less sure about this now that I've dug in more. |
Okay here's all I've found: Based on some logging, I've found the failure seems to consistently occur when mojo tells the producer to wait (
I can't venture any guesses toward a cause, but finding the |
Thanks for the info, @pfrazee! I'm picking this up again after the break. I've been trying to reproduce it consistently without using fs.createReadStream but unfortunately without much luck. Here's the fiddle I've been playing with: https://gist.github.com/cf281af81f415285acacafdb998b637e It uses |
@nornagon Have you tried reading from a data source via a stream, as opposed to responding with a buffer? The workaround that @itsananderson shared reads the file synchronously, so I'm wondering if the async streaming read affects it somehow. |
@pfrazee hm, i'm not sure exactly what you're suggesting. I'm attempting to replicate this without |
@nornagon Okay this Electron Fiddle is demonstrating the failure without using any other component: https://gist.github.com/pfrazee/9df2dacc935afdfe4ea6f6d2a28f7786 I basically introduced a bunch of async into the stream. There's a commented-out bit in main.js you can uncomment to prove the stream is not broken. When I run it, I just get no response from |
@pfrazee thanks for that repro case! I was able to track down what I believe was the issue. #21733 fixes the gist you made, as well as the repro that @electron7User created, I believe. |
@nornagon you rock! 🍻 |
Unfortunately I've updated to 8.0.0-beta.7, which includes the recent fix, and I'm still seeing the issue. |
The testcase fiddle I created is working correctly in 8.0.0-beta.7, but my application is still failing. I'll try to produce another test case. |
Okay I was able to produce an example case, this time I did need to use the fs and a https://gist.github.com/pfrazee/9200ecc8edb63c415748a9b825b67a2a |
Similar situation here. I now updated our main application to 7.1.10 and videos in the If necessary, I can also try to update the example project I posted earlier. I didn't do that yet, because we're very busy at the moment, but if you need that, I'll see what I can do. |
For us the problem does seem to be resolved in Electron 7.1.10 (using meteor-desktop), but we're not using videos (simply a relatively large minified JS bundle). |
FWIW I used |
I found something useful here. The I'm not sure what the right fix is. It might indicate we need to be able to hook into |
Anything new here? I just tried Electron 8.2.0, but the problem still exists. |
With Electron 8.2.0, the way I am using This seems to work reliably, whereas before when I was using I worked around this issue by bypassing So, hopefully Electron folks will figure out what the culprit is, I will be watching this space. Keep up the good work, much appreciated!! :) |
PS (to my comment above): I reported about broken |
I'm fairly sure based on my research that Chromium' media player just doesnt support custom streaming protocols at the moment, and the fix will be a patch to its |
Playing videos and audio files did work with Electron 6, but doesn't anymore since Electron 7. And as for your comment from 23 Jan, you said you've seen this issue for regular files as well.
|
@electron7User I forgot about that! What I found with the media player was that video/audio would start working when I made the modification I noted. So either we're dealing with 2 separate issues (a streaming issue and a media-player issue) or I was mistaken about other large files - I'd need to try to reproduce again to confirm. |
With large image files (5MB PNG), I had major issues when using Classic dev story: fixing the PNG image bug introduced the audio/video media "streaming" issue :) (which is why I was forced to bypass |
I'm working on a fix which adds a registry of "streaming protocols" which I'm stuck at the moment with a linker error related to my chromium patch:
I'm going to try to get into the electronhq slack to get more help but it'll take some time. If anybody knows what I'm missing, please point me in the right direction! |
I updated my example project to directly show the video streaming bug in electron 7, 8, and 9. After building the project, the video at the top does not load. When building the same project with electron 6, the video does load and can be viewed (update the |
I can't reproduce it with electron 10.1.5 |
Surprising, as this will be fixed in v11, apparently: |
Electron 11
...but not in Electron 10: |
I confirm that
Anybody experiencing seeking issues? |
@danielweck Unfortunately yes, I've experienced the same |
Same for me with video... |
Yes - I have faced with issue related to seeking when try play local stored video. When you open video the first time clicking on scrub bar(or another seek action) restarts video but when you open the same video in the second time everything works fine |
The Electron version reported on this issue is no longer supported. See our supported versions documentation. If you're still experiencing this issue on a supported version, please open a new issue with an updated repro - a Fiddle is very appreciated. Electron has a large issues backlog. To help our team prioritize, we're closing older issues and asking for new issues with updated repro steps if it affects a supported version. This helps sort what issues are still relevant and helps us fix them more quickly. Thanks for your patience and understanding! |
Preflight Checklist
Issue Details
Expected Behavior
According to the docs, one should be able to register a stream protocol handler which serves files from the file system.
The following code snippet is suggested by the docs:
The expected behavior is that Electron should read from the stream and (assuming no read errors etc.) serve a valid response for the custom protocol request.
Actual Behavior
Starting with Electron 7.0.0-beta.4 it looks like the request sometimes randomly never completes on the Renderer side. This can be seen in the Network tab as well as by inspecting AJAX calls at runtime. The read stream seems to be created successfully, and the response starts to be sent back to the renderer, but the final state change never happens.
To Reproduce
Run the following Fiddle to see the behavior change. In Electron < 7.0.0-beta4, the page can make an AJAX request to the custom scheme in a loop and print the contents. In Electron >= 7.0.0-beta4 the page can occasionally make a few AJAX requests, but it will eventually halt when one request never reaches a
XMLHttpRequest.DONE
state.https://gist.github.com/33a9b357c606c7cbe7c2cac99bbda973
Good console output (7.0.0-beta3 and bellow) looks like:
Bad console output (7.0.0-beta4+) looks like:
Screenshots
Network info for request that completes successfully:
Network info for request that never completes:
Additional Information
The text was updated successfully, but these errors were encountered: