Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pymemcache.exceptions.MemcacheUnknownCommandError: b'get' when hitting maximum number of connections #550

Open
cbowal opened this issue Nov 16, 2023 · 1 comment
Labels

Comments

@cbowal
Copy link

cbowal commented Nov 16, 2023

Hello!

I'm using:

  • django v4.2
  • memcache v1.6.22
  • pymemcache v4.0.0

I started seeing errors like this pop up:

  File "django/core/cache/backends/memcached.py", line 96, in get_many
          ret = self._cache.get_multi(key_map.keys())
  File "pymemcache/client/hash.py", line 398, in get_many
              result = self._safely_run_func(client, get_func, {}, *new_args, **kwargs)
  File "pymemcache/client/hash.py", line 211, in _safely_run_func
              result = func(*args, **kwargs)
  File "pymemcache/client/base.py", line 706, in get_many
          return self._fetch_cmd(b"get", keys, False, key_prefix=self.key_prefix)
  File "pymemcache/client/base.py", line 1149, in _fetch_cmd
                  self._raise_errors(line, name)
  File "pymemcache/client/base.py", line 1034, in _raise_errors
              raise MemcacheUnknownCommandError(name)
pymemcache.exceptions.MemcacheUnknownCommandError: b'get'

This was weird since 'get' is obviously a valid command.

Looking at the logs of the memcache instance I noticed that this is happening when I've hit the maximum number of connections. I'm seeing these memcache error logs:
ts=2023-11-16T19:27:25.256Z caller=exporter.go:771 level=error msg="Could not query stats settings" err="memcache: unexpected stats line format \"ERROR Too many open connections\\r\\n\""

I'm not too familiar with the pymemcache internals but it seems to be incorrectly raising MemcacheUnknownCommandError. I'm not sure what the expected error is when there's no available connection, perhaps MemcacheServerError or MemcacheUnexpectedCloseError.

@jogo
Copy link
Contributor

jogo commented Nov 22, 2023

The current code scan the response for lines that start with 'ERROR,' and sounds like that logic doesn't work in this case.

Do you have the full steps to reproduce the issue? If so I can take a look and work on a fix.

https://github.com/pinterest/pymemcache/blob/v4.0.0/pymemcache/client/base.py#L1149

def _raise_errors(self, line: bytes, name: bytes) -> None:

@jogo jogo added the bug label Nov 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants