Add Consumers with Multiple Filters ADR-34

[Jarema]

Title JetStream Consumers Multiple Filters

Metadata	Value
Date	2023-01-18
Author	@Jarema
Status	Approved
Tags	jetstream, client, server

Context and Problem Statement

Initially, JetStream Consumers could have only one Filter Subject.
As the number of feature requests to specify multiple subjects increased, this feature was added.
That could also reduce the number of Consumers in general.

Context

Server PR: nats-io/nats-server#3500

Design

Client side considerations

To implement the feature without any breaking changes, a new field should be added to Consumer config, both on the server and the clients.

1. The new field is:
   FilterSubjects []string json:"filter_subjects"

2. Subjects can't overlap each other and have to fit the interest of the Stream.
   In case of overlapping subjects, error (10136) will be returned.

3. Only one of FilterSubject or FilterSubjects can be passed. Passing both results in an error (10134)

4. Until future improvements, only old JS API for consumer creation can be used (the one without FilterSubject) in consumer create request. Using new API will yield an error (10135).

5. Each client, to support this feature, needs to add a new field that Marshals into a json array of strings.

Example

{
  "durable_name": "consumer",
  "filter_subjects": ["events", "data"]
}

6. Client does not have to check if both, single and multiple filters were passed, as server will validate it.
   Client should add new errors to return them in client language idiomatic fashion.

Server side

To make this change possible and reasonable peformant, server will have a buffer of first message for each subject filtered and will deliver them in order. After delivering message for a subject, buffer for that subject will be filled again, resulting in close to no overhead after the initial buffer fill.

This can be optimized but will not affect API.

Signed-off-by: Tomasz Pietrek tomasz@nats.io

[derekcollison]

LGTM

[scottf]

@Jarema @aricart For something like this, we usually, but not always, add client side checks against the server version to deliver different behavior. What is the expectation for this feature?

[scottf]

@Jarema @aricart Also, currently we do some subscribe subject validation against the consumer filter subjects. This may not be an issue in simplification, but will be an issue for existing API.

[aricart]

@scottf not quite sure - supporting it on consumer creation from the JSM-type APIs if the server is older it won't see the fields, and the user would not get an error, so you would need to verify that field against a server version.

Previously filter_subject couldn't be modified - not sure if we have relaxed that option. If it is editable, now there will be quite a few things that need doing.

I wouldn't implement any changes to subscribe() for this. If folks want to use it, they should create the consumer and then bind to it, this will be the behavior of simplification.

[ripienaar]

Checking the version of the server you are connected to is not an answer. It does not tell you what the version of the JetStream server otherside of the world is.

[Jarema]

@scottf That's a good point.

The problem I see though is if client is new, but server is old.
then you would pass empty filter subject and filled multiple filter subjects.
Old server would treat it as a consumer without a filter - seeing empty filter, not seeing multiple filtered field.

Checking against server version would not help much, as @ripienaar pointed out.

Ugly, but functional workaround for backwards compatibility would be to put some reserver word that is not a proper subject filter into filter_subject if filter_subjects is passed.
Old servers would error on that and clients could translate that error into a meaningful error.
New servers would recognize that keyword and ignore it.

EDIT it would be really nice to have JetStream protocol version ;).

[ripienaar]

io.nats.jetstream.api.v1.stream_create_response includes the config of the newly made stream, clients can detect if setting it didnt work on create and degrade gracefully. Ditto io.nats.jetstream.api.v1.stream_update_response

[Jarema]

@ripienaar was thinking about that as an alternative.
Now as I think about it - that would maybe require additional operation (delete the consumer if it was created without additional subjects. That's why I disregarded that idea initially), but at least it would not require ugly server code.
+1 from me.

[Jarema]

@scottf Subscribing to Consumer should be fine.

example config:

   "name": "multi-consumer",
   "filter_subject": "",
   "filter_subjects": ["events", "data"]

so you can subscribe with

(pseudocode)

js.Subscribe("", Bind("stream_name", "multi-consumer"))

So the inconvinience for non-simplified API is:

1. Users have to bind to consumer by name
2. Subscribe subject field should be empty

I would avoid adding a method to subscribe to multiple Subjects, as we're close to releasing simplification betas.
It would look ugly:

js.SubscribeMulti([]string{"one", "two"})

So let's don't do that :).

[aricart]

I have the implementation for JavaScript - a couple of observations:

• filter_subject and filter_subjects are both updatable (previously filter_subject was not) but the change flew under the radar
• The logic on the js.subscribe functionality needs to insure that doesn't attempt on consumer creation/update try to set filter_subject to the subject (the one that locates the stream) unless both filter_subject and filter_subjects are not set.
• client validation of the consumer after calling is suboptimal, because it will create a consumer on older servers (so now the issue is that since it could be an upsert, likely don't want the consumer to be deleted) unless it was created.

[aricart]

$JS..CONSUMER.CREATE.>

[aricart]

More importantly, if it is an update, the consumer could have been created using the new API, so it could have a name, depending on how the client is handling this it may reject the name. Effectively, if Array.isArray(config.filter_subjects), it should use the old API. Server will properly catch if filter_subject and filter_subjects` are both set.

[ripienaar]

Do we prevent a consumer update from switching between FilterSubjet and FilterSubjects?

[Jarema]

No.
User can switch the consumer from being single-filtered to multi-filtered.
Considering above, we might want to deny it until we land solution for multiple filters and new API.

[aricart]

LGTM - we should put a hint on what the new API is.

[aricart]

More importantly, if it is an update, the consumer could have been created using the new API, so it could have a name, depending on how the client is handling this it may reject the name. Effectively, if Array.isArray(config.filter_subjects), it should use the old API. Server will properly catch if filter_subject and filter_subjects` are both set.

[bruth]

Does there need to be an ADR for the clients to add this field?

[ripienaar]

Does there need to be an ADR for the clients to add this field?

Cant see why that would be needed, we tend to open an issue referencing this one. If there's a ADR that covers the full client design that might need an update.

[bruth]

Sorry, I meant an issue in this repo that creates a checklist of clients to implement.

[ripienaar]

Sorry, I meant an issue in this repo that creates a checklist of clients to implement.

would need that yes

[bruth]

Ok created #196