Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Consumer Pause #142

Open
Jarema opened this issue Feb 27, 2024 · 0 comments
Open

Implement Consumer Pause #142

Jarema opened this issue Feb 27, 2024 · 0 comments
Labels
accepted The defect or proposal as been accepted good first issue Good for newcomers help wanted

Comments

@Jarema
Copy link
Member

Jarema commented Feb 27, 2024

Proposed change

Implement the new API for pausing consumers.

The Server PR with related change has been merged here: nats-io/nats-server#5066
Schemas PR, which is probably the best source of information how to implement the feature:
https://github.com/nats-io/jsm.go/pull/522/files
ADR issue: nats-io/nats-architecture-and-design#266

Details

Request and response schemas:

// io.nats.jetstream.api.v1.consumer_pause_request
type JSApiConsumerPauseRequest struct {
	PauseUntil time.Time `json:"pause_until,omitempty"`
}

// io.nats.jetstream.api.v1.consumer_pause_response
type JSApiConsumerPauseResponse struct {
	JSApiResponse
	Paused         bool          `json:"paused"`
	PauseUntil     time.Time     `json:"pause_until"`
	PauseRemaining time.Duration `json:"pause_remaining,omitempty"`
}

Consumer info includes 2 new fields:

Paused bool json:"paused,omitempty"
PauseRemaining time.Duration json:"pause_remaining,omitempty"

The paused state and time time would need to be persisted to the raft layer such that server restarts would not unpause paused consumers. This is done using the consumer configuration that has a new value:

PauseUntil time.Time json:"pause_until,omitempty"

If in doubt, refer to the Server/Schema PR

Feature be available in the server version 2.11

Hints

Probably the easiest way to implement it, is to look how calls to methods for creating / deleting streams and consumers are done.

Use case

It is difficult to schedule maintenance on central resources on a large distributed system where 100s or 1000s of clients are accessing data in a stream.

This change allows to pause a Consumer such that it appears healthy but just doesnt deliver any messages.

During the pause maintenance can happen and resources accessed by clients will not be under constant pressure, later the stream can be unpaused and work will continue.

This happen without impacting running clients - other than they would see pending messages in stream info but not get any deliveries.

Applies to push and pull consumers.

Use case

Contribution

This should be pretty easy to contribute to anyone interested!

@Jarema Jarema added good first issue Good for newcomers accepted The defect or proposal as been accepted help wanted labels Feb 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
accepted The defect or proposal as been accepted good first issue Good for newcomers help wanted
Projects
None yet
Development

No branches or pull requests

1 participant