docs: ETU-74448: Updated guidelines on pagination.

[rikard-swahn]

No description provided.

[hw-knowit]

A good start!

I think the guide needs a lot of massaging both in content and structure to read clearly as an introduction to keyset pagination for readers who may not have heard of it before, and we will need some discussion on the exact format expected of all applications.

Many of my comments aren't necessarily to be taken as "this must be changed exactly so", but to open up a discussion with a concrete suggestion to start off.

I'm excited to see this implemented.

[hw-knowit]

My suggested interface in summary. Names debatable of course. While writing these out side by side, it seemed natural to provide a paging object for both interfaces so they appear more similar. If we are to put 4 (maybe) required fields in root the response gets pretty messy, and doing it like this makes it easy to provide a Paging type implementation for clients in the future.

---

If we allow the application to decide whether to provide totalItems, we should probably decide on just one of 2.a or 2.b to limit scope. I think hasMore is a little awkward and requires some implementation to support (fetching n+1 and returning just n + hasMore) so maybe 2.b is preferable. I do think there is real value in allowing the application to omit totalItems, as detailed in another PR comment.

For 1, maybe either just number of pages, or just page size is sufficient without always needing to provide both.

Offset Pagination:

{
  items: [...],

  # EITHER (1) - Client sees exact no. pages/items:
  paging: {
    size: 100,
    pages: 13,
    items: 1250
  }

  # OR (2.a) - Client sees just whether result set is exhausted:
  paging: {
    size: 100,
    hasMore: true
  }

  # OR (2.b) - `items.size == paging.size` signals more data may exist. Client must query to know for sure.
  paging: {
    size: 100
  }
}

---

Keyset Pagination:

{
  items: [...],

  # Application decides:
  #   Either - Cursor existence means at least one more page exists
  #   Or - Cursor existence means there _may_ be more pages, client must fetch to check.
  paging: {
    cursor?: "<opaque data>"
  }
}

[hw-knowit]

And an additional consideration:

Spring has HATEOAS which returns the following structure. With the number of Spring apps in Entur it may be wise to allow (or enforce..?) pagination responses to look like this.

Note the self and next links which act as a sort of cursor already. I also like that they return the page number in the response.

https://docs.spring.io/spring-data/rest/reference/paging-and-sorting.html#paging-and-sorting.paging.prev-and-next-links

{
  "_links" : {
    "self" : {
      "href" : "http://localhost:8080/persons{&sort,page,size}",
      "templated" : true
    },
    "next" : {
      "href" : "http://localhost:8080/persons?page=1&size=5{&sort}",
      "templated" : true
    }
  },
  "_embedded" : {
  	… data …
  },
  "page" : {
    "size" : 5,
    "totalElements" : 50,
    "totalPages" : 10,
    "number" : 0
  }
}

[arnabkd]

In GraphQL, cursor pagination is the norm and there the spec looks somewhat like this which enables the following questions to be answered for the client:

• Is there a "next" page?
• What is the total number of items
  The client can also ask for first where first describes the number of items it wants after a certain cursor. Alternatively, last which is the number of items before a certain cursor. first and last can never be used together.

Additionally it would be useful to implement hasPreviousPage as well to allow for pagination both ways in situations where a client traverses a list of items backwards.

In the context of REST APIs, I would probably do it similar to what is described above by @hw-knowit but I might make some of the fields optional. It might cost a lot to calculate totalElements for instance.

[bnordli]

Just some comments from experience.