Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

keyPrefix not applied for xgroup #1659

Open
pgarrett-twc opened this issue Sep 26, 2022 · 1 comment
Open

keyPrefix not applied for xgroup #1659

pgarrett-twc opened this issue Sep 26, 2022 · 1 comment

Comments

@pgarrett-twc
Copy link

pgarrett-twc commented Sep 26, 2022

Hi, when I enable keyPrefix, the xgroup create command is not respecting the setting.

Minimal recreation:

const Redis = require('ioredis')
const conn = new Redis('127.0.0.1:6379', { keyPrefix: 'myprefix:' })
conn.on('ready', async () => {
  await conn.xgroup('CREATE', 'mystream', 'grp', '0', 'MKSTREAM')
  await conn.xreadgroup('GROUP', 'grp', 'cons', 'STREAMS', 'mystream', '>')
  await conn.quit()
})

This produces:

  ioredis:redis write command[127.0.0.1:6379]: 0 -> xgroup([ 'CREATE', 'mystream', 'grp', '0', 'MKSTREAM' ]) +1ms
  ioredis:redis write command[127.0.0.1:6379]: 0 -> xreadgroup([ 'GROUP', 'grp', 'cons', 'STREAMS', 'myprefix:mystream', '$' ]) +1ms

Note that the argument 'mystream' in the first command should be 'myprefix:mystream' like it is in the second command.

I think this might be related to #1591 and #1610. Although it doesn't look like @ioredis/commands 1.2.0 will fix the problem, because the commands.json config for xgroup looks wrong.

@pgarrett-twc
Copy link
Author

pgarrett-twc commented Feb 28, 2023

To work around this problem, I use this function to wrap my keys before calling stream commands like xgroup, xinfo.

import { getKeyIndexes } from '@ioredis/commands'
const needKeyPrefixHack = getKeyIndexes('xgroup', ['CREATE', 'x', 'g', 0, 'MKSTREAM']).length == 0

export function getXgroupKey(k: string, redis: Redis | Cluster) {
  if (needKeyPrefixHack) {
    const prefix = redis.options?.keyPrefix
    return prefix == undefined ? k : `${prefix}${k}`
  } else {
    return k
  }
}

Another problem caused by xgroup not being fully supported is that, in cluster mode, it frequently receives a MOVED reply, which causes ioredis to refresh the slot cache as if the topology had changed. This caused a problem for me because I'm using AWS MemoryDB which occasionally responds slowly to CLUSTER SLOTS, so I was seeing ClusterAllFailedError.

So instead of calling redisConn.xgroup I am calling redisConn.sendCommand, which allows me to influence the original slot by providing getKeys:

import { Command } from 'ioredis'

if (needKeyPrefixHack) {
  const xgk = getXgroupKey(key, redisConn)
  const command = new Command('XGROUP', ['CREATE', xgk, consumerGroup, '0', 'MKSTREAM'], {
    replyEncoding: 'utf8',
  })
  command.getKeys = () => [xgk]
  redisConn.sendCommand(command)
  await command.promise
}
else {
  await redisConn.xgroup('CREATE', key, consumerGroup, 0, 'MKSTREAM')
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant