Stream Info Pagination Support

[aricart]

Overview

Stream info requests can now result in paginated responses. The pagination is will trigger if the request includes a subjects_filter, which would include more than 100,000 subject entries in the response. As with other paginated responses, the server will set total, offset, and limit in the response where:

• total is the length of the dataset
• offset is the index of the current row where the server started sending data for the request
• limit is the server set limit on the length of the data set that it will return in a single response.

Note that, unlike other responses, this is still a StreamInfo response, the paged data is the StreamInfo.state.subjects.

To handle the paging, the client can compare the total and limit fields and keep track of all the totals seen. It could also count the number of entries in the subjects map for each request until it accounts for at least the total number of entries. Note that total could change (increase or decrease) if subjects are added or deleted, so the client should update the value returned by the server with each page request.

Also, note that each request is a complete StreamInfo request, only the subject information is paged. The client should return the body of the StreamInfo read from the last paged response after aggregating all the subjects entries in this last response to include all values read by the previous pagination requests.

Clients and Tools

• Schemas @ripienaar
• CLI @ripienaar
• Go @piotrpio [CHANGED] StreamInfo() will now return all subjects when requested nats.go#1072
• Java @scottf streams / names with subject filtering, info pagination nats.java#759
• JavaScript nats-io/nats.deno@1d1cc6e [FIX] paged responses for stream info didn't take into account changing total reported by the server nats.deno#368
• .Net @scottf streams / names with subject filtering, info pagination nats.net#696
• C done by kozlovic [CHANGED] JetStream: js_GetStreamInfo() now returns all subjects nats.c#588
• Python @wallyqs
• Ruby @wallyqs
• Rust @Jarema @caspervonb

Other Tasks

• docs.nats.io updated @jnmoyne
• Update ADR to Implemented
• Update client features spreadsheet

Client authors please update with your progress. If you open issues in your own repositories as a result of this request, please link them to this one by pasting the issue URL in a comment or main issue description.