Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] AsyncRedisCluster keep cost fd at idle state #559

Open
lisay-yan opened this issue Apr 12, 2024 · 6 comments
Open

[BUG] AsyncRedisCluster keep cost fd at idle state #559

lisay-yan opened this issue Apr 12, 2024 · 6 comments

Comments

@lisay-yan
Copy link

Describe the bug
After new AsyncRedisCluster client, and keep it idle, not send any traffic, I observed the fd keep costing.

To Reproduce
Minimal code to reproduce the bug.
client = new AsyncRedisCluster(connection_options, pool_options);
Then, sleep, without any traffic.

Expected behavior
Discovery cluster node status, and set up one connection to each node. Keep ESTABLISHED status.

Environment:

  • OS: linux
  • Compiler: v 4.8.5
  • hiredis version: v1.0.0
  • redis-plus-plus version: master

Additional context
Please provide solution and info for why AsyncRedisCluster keep cost fd after just created, and no traffic drop in.
image

@lisay-yan
Copy link
Author

Update

Due to Redis cluster passwd required, I didn't set it to cluster client.
After set right passwd, the network behavior is controllable.

But, in the product env,
if Redis cluster is switch from no passwd to auth required, then existing client is keep costing fd till re-create the redis++ client with passwd.

can I know why so many fd since no traffic?
Any suggestion to prevent fd costing when client No or Wrong passwd???

@lisay-yan
Copy link
Author

One more question.
I found work around if need send a PING, but my response always is "connection is closing".
While, other commands, set/get, looks good. Why?

// Create an AsyncRedis object with hash-tag, so that we can send commands that has no key.// It connects to Redis instance that holds the given key, i.e. hash-tag.
auto r = async_cluster.redis("hash-tag");

Future ping_res = r.command("ping");

@sewenew
Copy link
Owner

sewenew commented Apr 15, 2024

can I know why so many fd since no traffic?

Once you create an AsyncRedisCluster object, it tries to connect to a random cluster node, and fetch the slot-node mapping. If it fails, it retries serval times (on other random nodes). Then it sleeps 1 second, and retry fetching the map.

In your case, you set a wrong password, redis-plus-plus always fails to get the slot-node mapping, and closes the connections. That's why you see so many connections.

Any suggestion to prevent fd costing when client No or Wrong passwd???

Once you send any command to Redis with AsyncRedisCluster, you'll get an exception that password is wrong. And you can handle the error.

While, other commands, set/get, looks good. Why?

Do you mean, you create an AsyncRedisCluster object with incorrect password, and the ping command throws "connection is closing" exception? I cannot reproduce your problem with the latest code on master. But if I remember correctly, I fixed a similar problem, and you can try if the latest code works well for you.

Regards

@lisay-yan
Copy link
Author

hi,

Yes, I try to use ping to understand the rainy day case, and expected "NOAUTH Authentication required" to take action and prevent fd costing.

But after I set correct password to AsyncRedisCluster, and send ping.
auto r = async_cluster.redis("hash-tag");
Future ping_res = r.command("ping");

Here is my output for set command and ping. It can prove async cluster is normal, while, ping can't work.
[lisay@sessiondbproxy-ci Andy]$ ./11
send set
send set
send set
set val = OK
set val = OK
set val = OK
send ping
e = connection is closing

Is there misusage in my ping?

@lisay-yan
Copy link
Author

If wrong passwd to AsyncRedisCluster, the test result as below

send set
set e = NOAUTH Authentication required.
send ping
e = connection is closing

@sewenew
Copy link
Owner

sewenew commented Apr 18, 2024

Is there misusage in my ping?

Have you waited the returned future before destroying the AsyncRedisCluster object? You need to ensure that all commands have been sent and the replies have been received before AsyncRedisCluster is destroyed. Otherwise, when AsyncRedisCluster is destructing, it closes the connection and mark all pending reply as error, i.e. connection is closing.

Regards

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants