Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[feature request] Output method to display video in browser #107

Open
PrashantSaikia opened this issue May 26, 2021 · 11 comments
Open

[feature request] Output method to display video in browser #107

PrashantSaikia opened this issue May 26, 2021 · 11 comments
Labels
bug Something isn't working

Comments

@PrashantSaikia
Copy link

PrashantSaikia commented May 26, 2021 •
edited

I am making a computer vision app where I need to display the webcam stream on the browser. Is there a way to do this currently in pywebio?

For example, in flask, we can do it this way. But, of course, since quite a lot of effort is needed to achieve this functionality in flask, I would very much love it if pywebio can handle this. Something like 'put_video', similar to 'put_image' would be great!
@PrashantSaikia PrashantSaikia added the bug Something isn't working label May 26, 2021
@PrashantSaikia PrashantSaikia changed the title [feature request] Is there any output method to display video in browser? [feature request] Output method to display video in browser May 26, 2021
@helincao
Copy link
Member

helincao commented Jun 10, 2021 •
edited

Video streaming is a very interesting use case, however, the tricky part is how do you handle your back-end content delivery network? It's very challenging to globally deliver high-quality live streaming services over the internet. I think a workaround can be using a 3rd party stream API or service, then embedding an HTML block using put_html() in your app.

Thoughts?

@PrashantSaikia
Copy link
Author

PrashantSaikia commented Jun 10, 2021 •
edited

Interesting idea. This is what I tried just now:

import cv2
from pywebio.output import put_html

cap = cv2.VideoCapture(0)

while True:
    if cap.grab():
        flag, frame = cap.retrieve()
        if not flag:
            continue
        else:
            cv2.imwrite("image.png", frame)
            html = "<html><img src='image.png'></html"
            put_html(html)

Its a simple webcam stream. And the idea is to save each frame in the video stream as an image, convert the image into an html object, and use put_html() to display the output in browser. But it produces blank images, and also seems to try to display the images next to each other instead of replacing the previous image with the current one (to give the appearance of a video). Any ideas on how to make it work?

Screenshot 2021-06-10 at 4 17 25 PM

I also tried saving each image as an html file, and then reading that html file, but I get the same output:

...
...
else:
    cv2.imwrite("image.png", frame)
    html_str = "<html><img src='image.png'></html"
    Html_file= open("filename.html","w")
    Html_file.write(html_str)
    Html_file.close()
    with open("filename.html", 'r') as f:
        html = f.read()
        put_html(html)

I also tried displaying each image directly using put_image(), and it displays each image one below the other, it doesn't replace the previous image with the current one:

import cv2
from pywebio.output import put_image

cap = cv2.VideoCapture('path/to/video.mp4') # or leave as 0 for webcam

while True:
    if cap.grab():
        flag, frame = cap.retrieve()
        if not flag:
            continue
        else:
            cv2.imwrite("image.png", frame)
            put_image(open('image.png', 'rb').read())

Screenshot 2021-06-10 at 4 32 54 PM

@helincao
Copy link
Member

I tried use_scope() (https://pywebio.readthedocs.io/en/latest/guide.html#use-scope) and time.sleep() to only keep one image at a time. The caveat of this workaround is the fade-in effect upon clearing a scope.

import cv2
from pywebio.output import *
import time

cap = cv2.VideoCapture(0)

while True:
	
    if cap.grab():
        flag, frame = cap.read()
        if not flag:
            continue
        else:
            cv2.imwrite("image.png", frame)
            img = open("image.png", 'rb').read() 
            #put_image(img)

            with use_scope('stream', clear=True): 
                put_image(img)
            time.sleep(0.5)

@wang0618
Copy link
Member

@helincao

The caveat of this workaround is the fade-in effect upon clearing a scope.

The fade-in effect can be turn off by pywebio.session.set_env(output_animation=False)

@PrashantSaikia
Copy link
Author

The fade-in effect can be turn off by pywebio.session.set_env(output_animation=False)

Thanks for this suggestion. Where exactly in the code should this line be? I put it at the end, and it doesn't do anything.

@wang0618
Copy link
Member

The fade-in effect can be turn off by pywebio.session.set_env(output_animation=False)

Thanks for this suggestion. Where exactly in the code should this line be? I put it at the end, and it doesn't do anything.

@PrashantSaikia You should put the set_env() at the beginning of your code.

@PrashantSaikia
Copy link
Author

Tried it just now. Still the same, no change.

@helincao
Copy link
Member

Indeed, there is file reading/writing latency that prevents your "streaming" from being smooth and yields this flicker effect. This is pretty much how far you can get using the currently available PyWebIO modules. A dedicated module for streaming output is not a trivial task, and we do not have a plan to support it by the end of 2021 yet. However, things can change if the demand for this module becomes high :)

@Shahin-rmz
Copy link

is there any road maps which features will be added on in the curse of time? @helincao

@helincao
Copy link
Member

helincao commented Jun 17, 2021 •
edited

It's not there at this moment. An open roadmap or to-do list might be an interesting idea. Let us do a little exploration and see how to support it. If you have ideas, please join our Discord server for brainstorming: https://discord.gg/MvaCcg76Z7

@Shahin-rmz
Copy link

Thanks @helincao for introducing me to the group.
just joined.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@wang0618 @helincao @Shahin-rmz @PrashantSaikia