Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: IPC socket drops messages, ignores LINGER #4660

Open
Azmisov opened this issue Feb 17, 2024 · 1 comment
Open

BUG: IPC socket drops messages, ignores LINGER #4660

Azmisov opened this issue Feb 17, 2024 · 1 comment

Comments

@Azmisov
Copy link

Azmisov commented Feb 17, 2024

What pyzmq version?

25.1.0

What libzmq version?

4.3.3

Python version (and how it was installed)

3.11.5

OS

Ubuntu 22.04

What happened?

Setting linger option for the sender doesn't block when the socket/context are closed. It exits immediately, but receiver doesn't end up reading all messages, so something is amiss.

It only seems to manifest if I set HWM, which I'm guessing is because without the limit libzmq is able to buffer everything in the client almost immediately.

Code to reproduce bug

import zmq, threading, time

def server():
	linger = 1000*60
	ctx = zmq.Context()
	sock = ctx.socket(zmq.PAIR)
	sock.setsockopt(zmq.LINGER, linger)
	sock.bind("ipc:///tmp/lingertest")

	print("server ready")
	for i in range(10):
		sock.send_json(i)
		print("server send", i)

	sock.close(linger)
	ctx.destroy(linger)
	print("server exiting")

def client():
	ctx = zmq.Context()
	sock = ctx.socket(zmq.PAIR)
	sock.setsockopt(zmq.RCVHWM, 1)
	sock.connect("ipc:///tmp/lingertest")

	print("client ready")
	while True:
		time.sleep(.5)
		i = sock.recv_json()
		print("client recv", i)
		if i == 9:
			break
	
	sock.close()
	ctx.destroy()
	print("client exiting")

threading.Thread(target=server).start()
threading.Thread(target=client).start()

Traceback, if applicable

n/a

More info

In this case I'm using a pair socket, so easy enough to have the server/client communicate when each are done instead of relying on linger.

@minrk
Copy link
Member

minrk commented Feb 18, 2024

This sounds like a bug that should be reported to libzmq, since it's not related to pyzmq. I guess ipc messages can drop when sockets close, apparently ignoring LINGER. You don't need to specify LINGER, either, since even with the default of waiting forever, messages are dropped. I don't know at what stage that's happening, but since adding sleep before sender.close results in delivered messages, it does sound like LINGER is just not implemented correctly for ipc senders. Setting SNDHWM to 1 also further accentuates the issue.

@minrk minrk changed the title BUG: Linger doesn't wait for receiver to finish reading libzmq BUG: Linger doesn't wait for receiver to finish reading Feb 19, 2024
@minrk minrk changed the title libzmq BUG: Linger doesn't wait for receiver to finish reading BUG: IPC socket drops messages, ignores LINGER Feb 23, 2024
@minrk minrk transferred this issue from zeromq/pyzmq Feb 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants