Broken pagination using page_token in Ory Hydra APIs

[Urbansson]

Preflight checklist

• I could not find a solution in the existing issues, docs, nor discussions.
• I agree to follow this project's Code of Conduct.
• I have read and am following this repository's Contribution Guidelines.
• This issue affects my Ory Network project.
• I have joined the Ory Community Slack.
• I am signed up to the Ory Security Patch Newsletter.

Describe the bug

When paginating over the oauth2 clients using the /admin/clients endpoint the the result returned jumps over multiple clients.

Reproducing the bug

Make a request to /admin/clients?page_size=3
The expected result is to have 3 clients returned and a link header containing
</admin/clients?page_size=3&page_token=eyJwYWdlIjoiMyIsInYiOjF9>; rel="next"

Make a request using the next Link returned int the header /admin/clients?page_size=3&page_token=eyJwYWdlIjoiMyIsInYiOjF9
Here we expect to get the following 3 clients. But instead it jumps multiple clients.

You can spot this by comparing the result from a request to /admin/clients?page_size=300

Relevant log output

No response

Relevant configuration

serve:
  cookies:
    same_site_mode: Lax

urls:
  self:
    issuer: http://127.0.0.1:4444
  consent: http://127.0.0.1:3000/consent
  login: http://127.0.0.1:3000/login
  logout: http://127.0.0.1:3000/logout
  error: http://127.0.0.1:3000/error

secrets:
  system:
    - youReallyNeedToChangeThis

webfinger:
  oidc_discovery:
    supported_claims:
      - email
      - email_verified
      - name
      - family_name
      - given_name
      - picture
      - updated_at
    supported_scope:
      - offline
      - openid
      - profile
      - userinfo.profile
      - email
      - userinfo.email
      
    userinfo_url: http://127.0.0.1:3000/api/userinfo

oidc:
  subject_identifiers:
    supported_types:
      - pairwise
      - public
    pairwise:
      salt: youReallyNeedToChangeThis

log:
  level: debug
  format: text
  leak_sensitive_values: true

Version

2.0.2

On which operating system are you observing this issue?

macOS

In which environment are you deploying?

Docker

Additional Context

If we take a deeper look into the token returned from the /admin/clients?page_size=3 request we see that it contains: {"page":"3","v":1}.

If it was the page return we should expect to get the value 1 and not 3, it instead looks like we are getting a offset to the next page.

But If we look when parsing the token in the request using the token we see the following:

hydra/client/handler.go

Lines 507 to 514 in c65342e

	func (h *Handler) listOAuth2Clients(w http.ResponseWriter, r *http.Request, ps httprouter.Params) {
	page, itemsPerPage := x.ParsePagination(r)
	filters := Filter{
	Limit: itemsPerPage,
	Offset: page * itemsPerPage,
	Name: r.URL.Query().Get("client_name"),
	Owner: r.URL.Query().Get("owner"),
	}

Here we still handle it as a page and not a offset getting the values page=3 and itemsPerPage=3 ParsePagination and then calculating a offset Offset: page * itemsPerPag ending up as 9. This means we are jumping over the clients with index 3-9 when paging.

So we are mixing offset and page based pagination. Which is the way we want to do stuff?

I would probably prefer changing the page_token content from {"page":"3","v":1} to {"offset":"3","v":1} to keep the token independent from the page_size query param.

[aeneasr]

Thank you for the report! Adding this to our backlog.

[aeneasr]

ps: contributions also welcomed, we will probably not be able to work on this in the next weeks

[Urbansson]

I'm happy to help with this issue, would you prefer storing the offset or a page index in the page_token?

[aeneasr]

The page index would be nicer to store in the page_token! but offset is also fine - whatever is easier :)

[bulolo]

how can i know my current page?

example:
i want to have currentPage

[Urbansson]

I would say you can't do it.
Usually when paginating with cursors/tokens you probably want to go for a infinite scroll solution so you just load more as it is requested.