Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sockets kept open by child process #22

Open
jgehrcke opened this issue Aug 24, 2016 · 2 comments
Open

Sockets kept open by child process #22

jgehrcke opened this issue Aug 24, 2016 · 2 comments

Comments

@jgehrcke
Copy link
Owner

Originally reported by: eurleif (Bitbucket: eurleif, GitHub: Unknown)


Here's a toy example which accepts TCP connections on port 12345. For each connection, it spawns a child process, which sleeps for 5 seconds before exiting. It does not pass the socket into the child process. In the parent process, it sends a message to the client, and closes the socket.

My expected behavior when connecting to this server is for it to send me the message, then immediately close the connection. In reality, it sends me the message, then waits 5 seconds before closing the connection. It seems as though the socket is being kept open by the child process, even though the socket was not passed to it.

I've tested this with Python 2.7 and 3.5, with gevent 1.1.2 and gipc 0.6.0, on Ubuntu 16.04.

Is this intended behavior? I certainly find it unexpected, since the docs mention closing handles.

#!python

from gevent import sleep, spawn
from gevent.server import StreamServer
import gipc

def child():
    sleep(5)

def handle(socket, address):
    gipc.start_process(target=child)
    socket.sendall(b'Spawned a child process. Good bye!\r\n')
    socket.close()
    
if __name__ == '__main__':
    server = StreamServer(('127.0.0.1', 12345), handle)
    server.serve_forever()


@jgehrcke
Copy link
Owner Author

Original comment by Jan-Philip Gehrcke (Bitbucket: jgehrcke, GitHub: jgehrcke):


Thanks for reporting that. Only gipc-related handles that are not explicitly passed to children are automatically closed in children, i.e. those of type gipc._GIPCHandle. Regarding all other file descriptors inherited during a fork, there is no magic approach that tries to automatically infer which ones the application wants to retain and which ones it does not (that's an ill-posed problem). You have to take care of these yourself.

Does that make sense?

Thanks for using gipc!

On a related note, now that gevent 1.1.x has equilibrated (a lot changed under the hood recently), I will also take care of gipc again in the near future and see if anything needs to change (and take care of processing the open issues).

@jgehrcke
Copy link
Owner Author

Original comment by eurleif (Bitbucket: eurleif, GitHub: Unknown):


Thanks, that does make sense -- but I think the docs should mention it. I almost pushed broken code to my project since I thought gipc would close all handles.

Great work on gipc, btw. Very useful.

millerdev added a commit to dimagi/commcare-hq that referenced this issue Jan 28, 2020
When gipc forks processes it does not close non-gipc related sockets (jgehrcke/gipc#22). In practice this meant that the parent and child processes shared connections. This was observed with couchdbkit, which sometimes returned the wrong results from a view call.

Postgres connections are supposedly vulnerable to the same issue (https://stackoverflow.com/q/40482555/10840), but that has not been observed.
millerdev added a commit to dimagi/commcare-hq that referenced this issue Jan 28, 2020
When gipc forks processes it does not close non-gipc related sockets (jgehrcke/gipc#22). In practice this meant that the parent and child processes shared connections. This was observed with couchdbkit, which sometimes returned the wrong results from a view call.

Postgres connections are supposedly vulnerable to the same issue (https://stackoverflow.com/q/40482555/10840), but that has not been observed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant