Inconsistent group_send behavior in RedisChannelLayer when called in consumer disconnect()

[agsimmons]

In my project, I want a message to be sent to a group when a client connects or disconnects. The "listener" of that group is a different consumer.

To detect connections, I group_send in the connect method.
To detect disconnections, I group_send in the disconnect method.

Events sent in connect work perfectly, and are always received. Events sent in disconnect however are very inconsistent and not received every time.

I've narrowed down the issue and have determined that it only occurs when using the channels_redis.core.RedisChannelLayer channel layer. channels.layers.InMemoryChannelLayer and channels_redis.pubsub.RedisPubSubChannelLayer work perfectly.

I reproduced the issue in this demo repo: https://github.com/agsimmons/redis-channel-layer-bug

This demo has a chat and a user log on a web page. The chat functionality uses the first consumer. When a user connects or disconnects, this information is printed to the chat log. When a user sends a message, it is printed to the chat log. The user log only shows connections and disconnections, but no chat messages. The user log connects to a second consumer. When a user connects or disconnects, their consumer additionally sends a group message to the user log consumer.

If you open multiple tabs of the demo page, and then close them one by one, you can observe in a different window that disconnections are always recorded in the chat log, but they are inconsistently recorded in the user log. The correct behavior would be for all disconnections to be logged in both places.

Steps to Reproduce

1. Clone the repo
2. Create a virtual environment
3. Install the dependencies in requirements.txt
4. Start the server with python manage.py runserver
5. Open a web browser to 127.0.0.1:8000
6. Duplicate the tab so that you have around 10 - 15 total instances of the web page
7. While viewing one of the tabs consistently, start closing tabs one by one. Observe that the disconnection message for each always appears in the chat log, but they inconsistently appear in the user log.

In redis_channel_layer_bug/settings.py, I have commented some other CHANNEL_LAYER configurations. The only one this issue is observed with is channels_redis.core.RedisChannelLayer.

Prior to creating this demo, I experimented with having a single ChatConsumer. ChatConsumers join the "chat" group on connect, and leave the "chat" group on disconnect. ChatConsumers group_send to the "chat" group on connect, and group_send to the "chat" group on disconnect. This appeared to always work correctly. I believe this means that the issue specifically relates to communication between different consumers.

OS: Arch Linux
ASGI Server: daphne

asgiref==3.7.2
attrs==23.1.0
autobahn==23.6.2
Automat==22.10.0
cffi==1.16.0
channels==4.0.0
channels-redis==4.1.0
constantly==15.1.0
cryptography==41.0.4
daphne==4.0.0
Django==4.2.5
hyperlink==21.0.0
idna==3.4
incremental==22.10.0
msgpack==1.0.7
pyasn1==0.5.0
pyasn1-modules==0.3.0
pycparser==2.21
pyOpenSSL==23.2.0
redis==5.0.1
service-identity==23.1.0
six==1.16.0
sqlparse==0.4.4
Twisted==23.8.0
txaio==23.1.1
typing_extensions==4.8.0
zope.interface==6.0

[carltongibson]

Thanks for the report. That looks good.

There have been various reports about group send with the original channel layer. (I wonder if it's the Redis commands not completing in time. 🤔)

It would be good to resolve all those (but I suspect it means a bit of a rethink.)