Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The docker-autoheal container always becomes unhealthy after some time on my system (on EC2) #87

Open
AbrieJCronje opened this issue Jun 8, 2022 · 1 comment

Comments

@AbrieJCronje
Copy link

I have a very standard EC2 instance on which I run docker-autoheal to watch over about 8 other containers. It works absolutely perfectly and always restarts these containers when they become unhealthy, so super thanks for this awesome awesome tool! However, it always becomes unhealthy after running for some time (few weeks), and then it stops restarting the other containers. Any ideas why this might happen? This is the state from docker inspect

"State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 4600,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2022-06-02T05:18:01.345930625Z",
            "FinishedAt": "2022-06-02T05:17:59.341327446Z",
            "Health": {
                "Status": "unhealthy",
                "FailingStreak": 97892,
                "Log": [
                    {
                        "Start": "2022-06-08T14:07:59.829228335Z",
                        "End": "2022-06-08T14:07:59.832537865Z",
                        "ExitCode": -1,
                        "Output": "cannot exec in a stopped state: unknown"
                    },
                    {
                        "Start": "2022-06-08T14:08:04.835753734Z",
                        "End": "2022-06-08T14:08:04.838610024Z",
                        "ExitCode": -1,
                        "Output": "cannot exec in a stopped state: unknown"
                    },
                    {
                        "Start": "2022-06-08T14:08:09.841808883Z",
                        "End": "2022-06-08T14:08:09.844731715Z",
                        "ExitCode": -1,
                        "Output": "cannot exec in a stopped state: unknown"
                    },
                    {
                        "Start": "2022-06-08T14:08:14.847771049Z",
                        "End": "2022-06-08T14:08:14.850751022Z",
                        "ExitCode": -1,
                        "Output": "cannot exec in a stopped state: unknown"
                    },
                    {
                        "Start": "2022-06-08T14:08:19.854019851Z",
                        "End": "2022-06-08T14:08:19.86835772Z",
                        "ExitCode": -1,
                        "Output": "cannot exec in a stopped state: unknown"
                    }
                ]
            }
        },

@hasnat
Copy link
Contributor

hasnat commented Jun 8, 2022

@AbrieJCronje
my guess would be.
if a docker daemon is live-reloaded it might kill the connection on unix:///var/run/docker.sock only restart of autoheal container itself would fix it,
can you check uptime of your docker daemon and its config

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants