Support running as PID 1

[Marwes]

When running watchexec (really, cargo-watch) in a docker container I would like that the watchexec process shutsdown when I request that the docker container shutsdown.

Currently SIGTERM is handled and sent to the child process which shutsdown my program as expected, but after that nothing happens. What I would like is that watchexec, after seeing the child process shutdown also shutsdown itself.

[passcod]

That sounds reasonable

[passcod]

Actually I can't reproduce this. Sending SIGTERM to watchexec kills it (and the child). Can you provide more detail as to how and what you're running?

[Marwes]

Right, forgot that detail. It only seems to happen when running it as the main process in docker (PID 1) which has some special behaviour around signals.

To reproduce

Dockerfile

FROM rust:1.38

RUN cargo install watchexec

COPY test.sh ./

test.sh

trap 'echo "Received SIGTERM" && exit 1' TERM
while true; do sleep 1; done

Build and run docker

docker build . -t test
docker run -it --rm --name test test watchexec --verbose 'sh ./test.sh'

With the container running docker kill -s SIGTERM test can be ran after which watchexec says that it received the signal.

Remove the watchexec invocation and run the bare sh ./test.sh to sanity check that the script exits on SIGTERM.

[passcod]

(Sorry for taking so long, I am looking at this and haven't forgotten.)

[passcod]

Tried fixing this for a few days but ran into issues and it's delaying release, so I'm shelving this for now. Not happy about it, but I have to get other stuff out of the door before coming back to it.

[LucasPickering]

@passcod I'm still running into this issue, did you ever get a chance to do more debugging? Is there anything I can do to help with the debugging?

[passcod]

Haven't done anything more since. Workaround is obviously to not run as PID 1 (not hard, just wrap it in a shell), but I haven't looked deeper at how the way signals work for PID 1 affects its handling in watchexec.

It's been a year and I've forgotten things (also I'm busy, november is basically the worst time as I do nanowrimo); if you want to help better repro it from scratch and look at the handling code in signal.rs.

[LucasPickering]

ok thanks, I will use an init process in the mean time and I'll try to do some debugging next week

[passcod]

Dropping a note for future visitors that for the specific case of Docker, running with the --init flag solves this without needing your own init/shell/wrapper. It would still be nice to properly support being PID1 at some point though.

[tgrondier]

If you're using watchexec as a sidecar in a kubernetes pod, a workaround for that (but only if your scripts can handle ungraceful stops), a workaround is to add the following to your watchexec container Dockerfile:

STOPSIGNAL SIGKILL

[matu3ba]

@Marwes pid 1 ignores most signaling to not "accidentially shutting down the system", because child processes may use pid 1 as process group to signal the other children.

Thus SIGKILL should be expected. Can you confirm that SIGKILL works? If yes, then please close this issue.

[passcod]

This will be in 1.21.1