How does RedisCluster pick from connections from connection pool?

[gskema]

We have a RedisCluster client with 4 nodes, 2 of them are replicas.

PHP config

Thread Safety | disabled
redis.clusters.cache_slots=1
redis.pconnect.pooling_enabled=1

Client initialized like:

 // single seeds, point to "dedicated configuration 
$client = new RedisCluster(null, ['....amazonaws.com:6379'], 0.1, 0.3, true);

When fetching random keys we consistently notice this pattern of decreasing time taken:

0.858 GET 1
0.635 GET 2
0.504 GET 3
0.441 GET 4
0.421 GET 5
0.406 GET 6

When doing random concurrent testing (with these 6 fetch ops), PHP has 8 processes (unique PIDs), and CLIENT LIST also has 8 unique connections that are kept alive, so that matches.

Question:
Does RedisCluster need to get a one of 4 direct node connections from the connection pool (based on key hash) from C level extension, and could that be the reason that of decreasing avg time for consecutive operations? E.g. for first operation it doesn't have the correct connection (not the right direct node connection for given slot hash), but with each op it gets more and keeps all possible node connections, so it is able to use them immediately later?

[michael-grunder]

The only way to be sure would be to do low level profiling that could capture granular samples of which part of the operation takes what amount of time.

If I'm just speculating my assumption is that it just comes down to network idiosyncrasies and that the connections become slightly faster after they've just been used.

That said, I'm basically ignorant to the details around ElastiCache as I have only ever used it in a debug environment to help people with issues now and again.

Pulling the persistent connection from the pool is likely a nanosecond/microsecond level operation. It's a tiny hash table mapped to a (in your case) single element linked list.

Another thing you could try is testing latency to the various redis nodes just via the command-line, to see if you see a similar warming up after repeated operations.