Access to lists of data items must support pagination to protect the service against overload as well as to support client side iteration and batch processing experience. This holds true for all lists that are (potentially) larger than just a few hundred entries.
There are two well known page iteration techniques:
-
Offset-based pagination: numeric offset identifies the first page-entry
-
Cursor-based pagination — aka key-based pagination: a unique key identifies the first page-entry (see also Twitter API or Facebook API)
The technical conception of pagination should also consider user experience (see Pagination Usability Findings In eCommerce), for instance, jumping to a specific page is far less used than navigation via {next}/{prev} page links (see {SHOULD} use pagination links). This favors an API design using cursor-based instead of offset-based pagination — see {SHOULD} prefer cursor-based pagination, avoid offset-based pagination.
Note: To provide a consistent look and feel of pagination patterns, you must stick to the common query parameter names defined in [137].
Cursor-based pagination is usually better and more efficient when compared to offset-based pagination. Especially when it comes to high-data volumes and/or storage in NoSQL databases.
Before choosing cursor-based pagination, consider the following trade-offs:
-
Usability/framework support:
-
Offset-based pagination is more widely known than cursor-based pagination, so it has more framework support and is easier to use for API clients.
-
-
Use case - jump to a certain page:
-
If jumping to a particular page in a range (e.g., 51 of 100) is really a required use case, cursor-based navigation may not be feasible.
-
-
Data changes may lead to anomalies in result pages:
-
Offset-based pagination may create duplicates or lead to missing entries if rows are inserted or deleted between two subsequent paging requests.
-
If implemented incorrectly, cursor-based pagination may fail when the cursor entry has been deleted before fetching the pages.
-
-
Performance considerations - efficient server-side processing using offset-based pagination is hardly feasible for:
-
Very big data sets, especially if they cannot reside in the main memory of the database.
-
Sharded or NoSQL databases.
-
The {cursor} used for pagination is an opaque pointer to a page, that must never be inspected or constructed by clients. It usually encodes (encrypts) the page position, i.e. the unique identifier of the first or last page element, the pagination direction, and the applied query filters (or a hash over these) to safely recreate the collection (see also best practice [cursor-based-pagination] below).
For iterating over collections (result sets) we propose to either use cursors (see {SHOULD} prefer cursor-based pagination, avoid offset-based pagination) or simple hypertext control links (see {SHOULD} use pagination links). To implement these in a consistent way, we have defined a response page object pattern with the following field semantics:
Pagination responses should contain the following additional array field to transport the page content:
To simplify user experience, the applied query filters may be returned using the following field (see also {GET-with-body}):
As Result, the standard response page using cursors or pagination links may be defined as follows:
ResponsePage:
type: object
required:
- items
properties:
self:
description: Pagination link|cursor pointing to the current page.
type: string
format: uri|cursor
first:
description: Pagination link|cursor pointing to the first page.
type: string
format: uri|cursor
prev:
description: Pagination link|cursor pointing to the previous page.
type: string
format: uri|cursor
next:
description: Pagination link|cursor pointing to the next page.
type: string
format: uri|cursor
last:
description: Pagination link|cursor pointing to the last page.
type: string
format: uri|cursor
query:
description: >
Object containing the query filters applied to the collection resource.
type: object
properties: ...
items:
description: Array of collection items.
type: array
required: false
items:
type: ...Note: While you may support cursors for {next}, {prev}, {first}, {last}, and {self}, it is best practice to replace these with pagination links — see {SHOULD} use pagination links.
To simplify client design, APIs should support simplified hypertext controls as standard pagination links where applicable:
{
"self": "http://my-service.zalandoapis.com/resources?cursor=<self-position>",
"first": "http://my-service.zalandoapis.com/resources?cursor=<first-position>",
"prev": "http://my-service.zalandoapis.com/resources?cursor=<previous-position>",
"next": "http://my-service.zalandoapis.com/resources?cursor=<next-position>",
"last": "http://my-service.zalandoapis.com/resources?cursor=<last-position>",
"query": {
"query-param-<1>": ...,
"query-param-<n>": ...
},
"items": [...]
}See also {SHOULD} use pagination response page object for details on the pagination fields and page result object.
In pagination responses you should generally avoid providing a total result count, since calculating it is a costly operation that is usually not required by clients. Counting the total number of results for complex queries usually requires a full scan of all involved indexes, as it is difficult to calculate and cache it in advance. While this is only an implementation detail, it is important to consider that providing these total counts over the life-span of a service might become expensive as the data set grows over time.
As clients may integrate against these counts over time alongside data set growth, removing them will be more difficult than not providing them in the first place.
If your consumer really requires a total result count in the response, you may
support this requirement via the {Prefer} header adding the directive
return=total-count (see also [181]).