Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sensible Defaults for Python Redis Client #71

Open
erikwright opened this issue Apr 20, 2018 · 1 comment
Open

Sensible Defaults for Python Redis Client #71

erikwright opened this issue Apr 20, 2018 · 1 comment

Comments

@erikwright
Copy link
Contributor

By default no timeouts are set for the Redis client. It's poorly documented, but as far as I can tell, the Redis client will wait hang infinitely by default. This is not a sensible default for ci-queue clients, forcing every client to specify this value. socket_connect_timeout defaults to whatever is specified for socket_timeout, but since that is handled deep down in the Redis client and certainly not documented by ci-queue a responsible client might not rely on that.

I suggest explicitly setting sensible defaults for these two values to reduce the burden on clients.

def build_queue(queue_url, tests_index=None):
spec = uritools.urisplit(queue_url)
if spec.scheme == 'list':
return ciqueue.Static(spec.path.split(':'))
elif spec.scheme == 'file':
return ciqueue.File(spec.path)
elif spec.scheme == 'redis':
redis_args = parse_redis_args(spec)
redis_client = redis.StrictRedis(**redis_args)
worker_args = parse_worker_args(spec.query, tests_index)
retry = bool(worker_args['retry'])
del worker_args['retry']
klass = ciqueue.distributed.Worker
if tests_index is None:
klass = ciqueue.distributed.Supervisor
queue = klass(tests=tests_index, redis=redis_client, **worker_args)
if retry and tests_index:
queue = queue.retry_queue()
return queue
else:
raise "Unknown queue scheme: " + repr(spec.scheme)

@casperisfine
Copy link
Contributor

It's poorly documented, but as far as I can tell, the Redis client will wait hang infinitely by default.

In general when you don't set a timeout you get some default set at the kernel level which tend to be 2 minutes (which is HUGE).

So yeah 👍 to setting a default timeout. There is no reason we should have to wait long for Redis. If it takes more than a second to respond, something's went south.

I checked, we also forgot to do it in the Ruby version. I propose something like 1 or 2 seconds.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants