Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Server "streaming" option #2

Open
iarna opened this issue Jul 27, 2014 · 2 comments
Open

Server "streaming" option #2

iarna opened this issue Jul 27, 2014 · 2 comments

Comments

@iarna
Copy link
Owner

iarna commented Jul 27, 2014

Currently, the server has two modes of operation. At start, it works the same as the C++ gearmand. When a connection is made to it and the "streaming" option is selected, it changes semantics to those better supporting streaming clients. Currently this means:

  1. Clients receive a FAIL event if the worker disconnects, instead of being silently retried by the server.
  2. Uniqueid jobs have their results queued and sent to newly connected clients, so that the series of DATA/WARNING/COMPLETE events are identical across all workers.

On reflection, I want to change the second one to be:

  1. Uniqueid foreground jobs result in an error.

This would remove substantial complexity, while maintaining the same promises. I don't believe that streaming uniqueid foreground jobs are sufficiently useful to justify the code needed to support them.

@amv
Copy link
Collaborator

amv commented Jul 28, 2014

I think this mode could be very beneficial also for scenarios other than streaming, but the naming does not really promote it's use. If I understand this correctly, the feature could also be named "fail fast"-mode? This would probably promote it's use in scenarios where fail fast operation is the wanted behaviour.

I personally think that having a "fail fast" mode more prominently documented would also be a good way to convey users the "default" retry logic of the gearmand server. Currently this information is not very easy to come by in the official gearmand documentation(s).

Do you envision the streaming option having some other needs than those which could be described as "fail fast" operation? And if so, could the streaming mode be an extra mode on top of the "fail fast" mode?

@iarna
Copy link
Owner Author

iarna commented Jul 28, 2014

This isn't prominently documented because it hasn't been released yet. Beyond that, it requires that you be using a server that supports the feature, that is, the Abraxas server.

But, you're right, one is "execute no more then once", the other is either "make uniqueid foreground connections all receive the same packets" or "disallow uniqueid foreground connections". I'd be fine with having separate options for these.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants
@amv @iarna and others