Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add healthcheck #707

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

Add healthcheck #707

wants to merge 1 commit into from

Conversation

felipecrs
Copy link
Contributor

@felipecrs felipecrs commented Oct 20, 2023

This is work in progress, I will write more details before I mark it as ready for review.

Benefits:

  • Hass Supervisor watchdog (if enabled) will restart the container once it is unhealthy

For now, this is blocked by:

@felipecrs felipecrs force-pushed the healthcheck branch 2 times, most recently from f8401c7 to 5689589 Compare October 20, 2023 01:09
@AlexxIT
Copy link
Owner

AlexxIT commented Oct 20, 2023

API port may be closed or using another number

@felipecrs
Copy link
Contributor Author

Right, I have an idea to fix that.

@bonuzzz
Copy link

bonuzzz commented Nov 23, 2023

@felipecrs as go2rtc has static config path in docker, you could parse config file and use it in healthcheck sh script. It's possibility describes here https://stackoverflow.com/questions/55375371/using-a-sh-script-for-docker-healthchecks
Anyway I've never seen go2rtc hanging...
As variant, you could parse output existence of "go2rtc --version"

@felipecrs
Copy link
Contributor Author

felipecrs commented Nov 23, 2023

Anyway I've never seen go2rtc hanging...

People in Frigate's world have seen go2rtc hanging many times. That's why Frigate (myself) built its own healthcheck for go2rtc to restart it when it hangs.

I, for one, have an issue open that describes a hanging situation happening in latest version 1.8.4:

As variant, you could parse output existence of "go2rtc --version"

I don't see how invoking a new go2rtc process would help to assess the functionality of the existing one.

@felipecrs
Copy link
Contributor Author

@felipecrs as go2rtc has static config path in docker, you could parse config file and use it in healthcheck sh script. It's possibility describes here https://stackoverflow.com/questions/55375371/using-a-sh-script-for-docker-healthchecks

That's a very good idea, and it was what I was intending to do too. But then I forgot about this PR. lol

@bonuzzz
Copy link

bonuzzz commented Nov 23, 2023

I mean I had never seen hanging go2rtc myself and I don't know what exactly needs to check like rtsp port or web port. What is staying accessed when process hangs.

@bonuzzz
Copy link

bonuzzz commented Nov 23, 2023

Probably it could be related to this pr #440 to track for example sighup signal

@felipecrs
Copy link
Contributor Author

felipecrs commented Nov 23, 2023

@bonuzzz, can you elaborate?

@bonuzzz
Copy link

bonuzzz commented Nov 23, 2023

@felipecrs unfortunately I don't know how to reproduce go2rtc hanging in my environment. I think more complex solution is needed here than just to check open ports, because we don't know how the process behaves itself in hanging state.
I saw your homeassistant automation to check stream status and in my opinion this is the best solution at the moment.

@bonuzzz
Copy link

bonuzzz commented Nov 24, 2023

Very dirty solution for frigate is parsing /dev/shm/logs/go2rtc/current for phrase "sync.(*Mutex).Lock(...)", "goroutine" of "WRN [streams] json locked" like tail -f /dev/shm/logs/go2rtc/current | grep "json locked"

@felipecrs
Copy link
Contributor Author

I think the current solution Frigate uses is already very good. :)

@AlexxIT AlexxIT added the doubt label Apr 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants